From noreply@sourceforge.net  Sat Feb  1 01:40:56 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Fri, 31 Jan 2003 17:40:56 -0800
Subject: [Python-bugs-list] [ python-Bugs-678352 ] list.append memory leak ?
Message-ID: <E18emeO-0000fG-00@sc8-sf-web3.sourceforge.net>

Bugs item #678352, was opened at 2003-01-31 15:18
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=678352&group_id=5470

Category: Python Interpreter Core
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: thanos  vassilakis (thanos)
Assigned to: Nobody/Anonymous (nobody)
Summary: list.append memory leak ?

Initial Comment:
# this SEEMS to leak
l = xrange(10000)
while 1:
        a=[]
        map(a.append, l)
        del a

# this does not 
while 1:
       a=range(10000)
       del a



----------------------------------------------------------------------

>Comment By: Tim Peters (tim_one)
Date: 2003-01-31 20:40

Message:
Logged In: YES 
user_id=31435

Again, please identify the Python version and 
OS+compiler+C library in use.  I'm not arguing with you, I'm 
trying to help <wink>.  Since I don't see any evidence of 
anything like I leak when I try it (memory usage didn't 
increase *at all* when I tried it) , the answer to why you see 
it must lie in something you're using that I'm not -- only you 
can tell us what that might be.  I already told you the 
Python version, OS and C compiler/library version I used.

----------------------------------------------------------------------

Comment By: thanos  vassilakis (thanos)
Date: 2003-01-31 17:47

Message:
Logged In: YES 
user_id=91277

HP UX B.11.00
BSD 4.7
python 2.2

Its possible it is not a bug that is why I used the word SEEMS
but why does the second code fragment behave? 


----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2003-01-31 16:00

Message:
Logged In: YES 
user_id=31435

Please identify the Python version and OS+compiler+C 
library in use.  The process size remains steady for me under 
current CVS Python on Win2K w/ MSVC6.

A true leak is possible but unlikely.  More likely is that your 
platform C malloc()/free() decide not to return memory to the 
OS.  If so, that's outside of Python's control

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=678352&group_id=5470


From noreply@sourceforge.net  Sat Feb  1 02:23:16 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Fri, 31 Jan 2003 18:23:16 -0800
Subject: [Python-bugs-list] [ python-Bugs-678518 ] Another parsermodule validation error
Message-ID: <E18enJM-0003UH-00@sc8-sf-web1.sourceforge.net>

Bugs item #678518, was opened at 2003-01-31 20:23
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=678518&group_id=5470

Category: Parser/Compiler
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: logistix (logistix)
Assigned to: Nobody/Anonymous (nobody)
Summary: Another parsermodule validation error

Initial Comment:
The following code generates an validation error.  It's an 
invalid ast tree where the global_stmt doesn't have the 
keyword 'global' as a child.

I tracked down the fix to validate_global_stmt() this time.

Python 2.3a1 (#38, Dec 31 2002, 17:53:59) [MSC 
v.1200 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more 
information.
>>> import symbol,token,parser
>>> ast = [symbol.file_input,
...  [symbol.stmt, [symbol.simple_stmt, 
[symbol.small_stmt, [symbol.global_stmt,
 [1, 'foo']]], [token.NEWLINE, '']]],
...  [token.NEWLINE, ''],
...  [token.ENDMARKER, '']]
>>> a = parser.sequence2ast(ast)
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
parser.ParserError: VALIDATION FAILURE: report this 
to the maintainer!
>>>

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=678518&group_id=5470


From noreply@sourceforge.net  Sat Feb  1 02:26:09 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Fri, 31 Jan 2003 18:26:09 -0800
Subject: [Python-bugs-list] [ python-Bugs-678519 ] StringIO self-iterator
Message-ID: <E18enM9-0001vO-00@sc8-sf-web3.sourceforge.net>

Bugs item #678519, was opened at 2003-01-31 18:26
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=678519&group_id=5470

Category: Python Library
Group: Feature Request
Status: Open
Resolution: None
Priority: 5
Submitted By: Brett Cannon (bcannon)
Assigned to: Nobody/Anonymous (nobody)
Summary: StringIO self-iterator

Initial Comment:
As requested by Guido at http://mail.python.org/pipermail/python-dev/2003-January/032629.html , this is a feature request to make both cStringIO and StringIO self-iterators (``__iter__()`` returns self and the class defines ``next()``).  This is so they can be more like files.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=678519&group_id=5470


From noreply@sourceforge.net  Sat Feb  1 02:42:34 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Fri, 31 Jan 2003 18:42:34 -0800
Subject: [Python-bugs-list] [ python-Bugs-678519 ] StringIO self-iterator
Message-ID: <E18enc2-0003ww-00@sc8-sf-web1.sourceforge.net>

Bugs item #678519, was opened at 2003-01-31 21:26
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=678519&group_id=5470

Category: Python Library
Group: Feature Request
Status: Open
Resolution: None
Priority: 5
Submitted By: Brett Cannon (bcannon)
>Assigned to: Tim Peters (tim_one)
Summary: StringIO self-iterator

Initial Comment:
As requested by Guido at http://mail.python.org/pipermail/python-dev/2003-January/032629.html , this is a feature request to make both cStringIO and StringIO self-iterators (``__iter__()`` returns self and the class defines ``next()``).  This is so they can be more like files.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=678519&group_id=5470


From noreply@sourceforge.net  Sat Feb  1 02:42:59 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Fri, 31 Jan 2003 18:42:59 -0800
Subject: [Python-bugs-list] [ python-Bugs-678519 ] StringIO self-iterator
Message-ID: <E18encR-0003xO-00@sc8-sf-web1.sourceforge.net>

Bugs item #678519, was opened at 2003-01-31 21:26
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=678519&group_id=5470

Category: Python Library
Group: Feature Request
Status: Open
Resolution: None
Priority: 5
Submitted By: Brett Cannon (bcannon)
>Assigned to: Raymond Hettinger (rhettinger)
Summary: StringIO self-iterator

Initial Comment:
As requested by Guido at http://mail.python.org/pipermail/python-dev/2003-January/032629.html , this is a feature request to make both cStringIO and StringIO self-iterators (``__iter__()`` returns self and the class defines ``next()``).  This is so they can be more like files.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=678519&group_id=5470


From noreply@sourceforge.net  Sat Feb  1 03:07:44 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Fri, 31 Jan 2003 19:07:44 -0800
Subject: [Python-bugs-list] [ python-Bugs-678352 ] list.append memory leak ?
Message-ID: <E18eo0O-0003QX-00@sc8-sf-web4.sourceforge.net>

Bugs item #678352, was opened at 2003-01-31 15:18
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=678352&group_id=5470

Category: Python Interpreter Core
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: thanos  vassilakis (thanos)
Assigned to: Nobody/Anonymous (nobody)
Summary: list.append memory leak ?

Initial Comment:
# this SEEMS to leak
l = xrange(10000)
while 1:
        a=[]
        map(a.append, l)
        del a

# this does not 
while 1:
       a=range(10000)
       del a



----------------------------------------------------------------------

>Comment By: thanos  vassilakis (thanos)
Date: 2003-01-31 22:07

Message:
Logged In: YES 
user_id=91277

 We use HP UX 11 and BSD 4.7 but what were the origial build compilers and libraries  will have to wait till Monday - when I will be back at work and have access to the NYSE trading  floor systems.

I am chasing some memory leak that occurred when we moved from python 1.5.2 to 2.2


I don't think this is it, but...

Thanks for your quick response.




----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2003-01-31 20:40

Message:
Logged In: YES 
user_id=31435

Again, please identify the Python version and 
OS+compiler+C library in use.  I'm not arguing with you, I'm 
trying to help <wink>.  Since I don't see any evidence of 
anything like I leak when I try it (memory usage didn't 
increase *at all* when I tried it) , the answer to why you see 
it must lie in something you're using that I'm not -- only you 
can tell us what that might be.  I already told you the 
Python version, OS and C compiler/library version I used.

----------------------------------------------------------------------

Comment By: thanos  vassilakis (thanos)
Date: 2003-01-31 17:47

Message:
Logged In: YES 
user_id=91277

HP UX B.11.00
BSD 4.7
python 2.2

Its possible it is not a bug that is why I used the word SEEMS
but why does the second code fragment behave? 


----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2003-01-31 16:00

Message:
Logged In: YES 
user_id=31435

Please identify the Python version and OS+compiler+C 
library in use.  The process size remains steady for me under 
current CVS Python on Win2K w/ MSVC6.

A true leak is possible but unlikely.  More likely is that your 
platform C malloc()/free() decide not to return memory to the 
OS.  If so, that's outside of Python's control

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=678352&group_id=5470


From noreply@sourceforge.net  Sat Feb  1 03:33:08 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Fri, 31 Jan 2003 19:33:08 -0800
Subject: [Python-bugs-list] [ python-Bugs-640230 ] Discovered typo in zlib test.
Message-ID: <E18eoOy-0003JR-00@sc8-sf-web3.sourceforge.net>

Bugs item #640230, was opened at 2002-11-18 18:36
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=640230&group_id=5470

Category: None
Group: None
Status: Open
Resolution: None
Priority: 6
Submitted By: Scott David Daniels (scott_daniels)
Assigned to: Nobody/Anonymous (nobody)
Summary: Discovered typo in zlib test.

Initial Comment:
In test_zlib.py (while chasing what appears to be a
documentation problem), I found a flaw in the code
that tests max_length limitted output from a 
decompression object test.

Starting at line 100, the existing code is:
...
bufs.append(deco.flush())
decomp2 = ''.join(buf)
if decomp2 != buf:
    print "max_length decompressobj failed"
else:
    print "max_length decompressobj succeeded"
...

This test will always succeed (''.join(str) == str).

What seems obviously meant is:
...
bufs.append(deco.flush())
decomp2 = ''.join(bufs)
if decomp2 != buf:
    print "max_length decompressobj failed"
else:
    print "max_length decompressobj succeeded"
...



----------------------------------------------------------------------

>Comment By: Scott David Daniels (scott_daniels)
Date: 2003-02-01 03:33

Message:
Logged In: YES 
user_id=493818

I've attached a PyUnit-style replacement for test_zlib.py that 
tests this condition to Patch #678531,  which provides a
context 
diff updating zlibmodule.c to fix this problem.  Hope this
is enough 
to get the fix reviewed.


----------------------------------------------------------------------

Comment By: Scott David Daniels (scott_daniels)
Date: 2002-12-12 18:32

Message:
Logged In: YES 
user_id=493818

The following replacement for PyZlib_unflush addresses some 
of the problem, but leaves unaddressed the issues of what 
unused_data and unconsumed_tail should be at the end of the 
routine.

static PyObject *
PyZlib_unflush(compobject *self, PyObject *args)
{
    int err, length = DEFAULTALLOC;
    PyObject * retval = NULL;
    unsigned long start_total_out;

    if (!PyArg_ParseTuple(args, "|i:flush", &length))
	return NULL;
    if (!(retval = PyString_FromStringAndSize(NULL, length)))
	return NULL;


    ENTER_ZLIB

    start_total_out = self->zst.total_out;
    self->zst.avail_out = length;
    self->zst.next_out = (Byte *)PyString_AS_STRING(retval);

    Py_BEGIN_ALLOW_THREADS
    err = inflate(&(self->zst), Z_FINISH);
    Py_END_ALLOW_THREADS

    /* while Z_OK and the output buffer is full, there might
be more output,
       so extend the output buffer and try again */
    while ((err == Z_OK || err == Z_BUF_ERROR) &&
self->zst.avail_out == 0) {
	if (_PyString_Resize(&retval, length << 1) < 0)
	    goto error;
	self->zst.next_out = (Byte *)PyString_AS_STRING(retval) +
length;
	self->zst.avail_out = length;
	length = length << 1;

	Py_BEGIN_ALLOW_THREADS
	err = inflate(&(self->zst), Z_FINISH);
	Py_END_ALLOW_THREADS
    }

    /* Maybe _always_ call inflateEnd if clearing
is_initialized */
    if (err == Z_STREAM_END) {
	err = inflateEnd(&(self->zst));
	if (err != Z_OK) {
	    zlib_error(self->zst, err, "from inflateEnd()");
	    Py_DECREF(retval);
	    retval = NULL;
	    goto error;
	}
    }
    self->is_initialised = 0;
    _PyString_Resize(&retval, self->zst.total_out -
start_total_out);
    /* ??? Here fix unused_data and unconsumed_tail */

error:

    LEAVE_ZLIB

    return retval;
}



----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2002-11-19 22:08

Message:
Logged In: YES 
user_id=31435

I'd say it's a bug in flush(), or in the docs, then.  They say "all 
pending input is processed, and a string containing the 
remaining uncompressed output is returned", and that's not 
happening here.  That's pretty clearly what the author of this 
test *expected* to happen, too (the original bug you 
discovered looked much more like a typo than a failure to 
understand the module interface).

----------------------------------------------------------------------

Comment By: Scott David Daniels (scott_daniels)
Date: 2002-11-19 21:40

Message:
Logged In: YES 
user_id=493818

OK, The reason the code fails after the fix is as follows:
deco.flush() does not return any output.
The loop control says, "while we have more unexamined 
source."

However, the decompressor can consume all of the input 
before it has provided all of its output.  So, there are two 
possible fixes:

1) Minimal change:  Change the loop control to say, in 
effect, "While we have more input or are producing output.":
Around line 92:
Change:
    bufs = []
    while cb:
        max_length = 1 + len(cb)/10
To:
    bufs = []
    while cb or chunk:
        max_length = 1 + len(cb)/10


2) More reasonable(?) change:  After the insertion loop, (just 
before the flush) extract all remaining output:
Around line 99:
        cb = deco.unconsumed_tail
    bufs.append(deco.flush())
    decomp2 = ''.join(bufs)
Becomes:
        cb = deco.unconsumed_tail
    while bufs[-1]:
        bufs.append(deco.decompress('', max_length))
    bufs.append(deco.flush())
    decomp2 = ''.join(bufs)


---
Note, in any case, the
    bufs.append(deco.flush())
originally on line 100 _always_ appends a zero-length string.
I would suggest changing it to simply:
    deco.flush()

    

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2002-11-19 09:50

Message:
Logged In: YES 
user_id=6656

Assuming Tim meant what he said, not what he did: raising
priority.

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2002-11-18 20:56

Message:
Logged In: YES 
user_id=31435

Good eye!  Unfortunately, when I repair that, the test fails 
here -- decomp2 is a proper prefix of buf then.  It's "too 
short".  Raised priority but left unassigned (someone who 
understands zlib better will have to jump in).

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=640230&group_id=5470


From noreply@sourceforge.net  Sat Feb  1 06:37:38 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Fri, 31 Jan 2003 22:37:38 -0800
Subject: [Python-bugs-list] [ python-Bugs-676342 ] after using pdb readline does not work correctly
Message-ID: <E18erHW-0001Wk-00@sc8-sf-web1.sourceforge.net>

Bugs item #676342, was opened at 2003-01-28 20:53
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=676342&group_id=5470

Category: Python Library
Group: Python 2.2.2
Status: Open
Resolution: None
Priority: 5
Submitted By: Michael Stone (mbrierst)
Assigned to: Nobody/Anonymous (nobody)
Summary: after using pdb readline does not work correctly

Initial Comment:

After I use pdb in the interpreter my readline settings are messed up.  It only knows about the pdb readline stuff forever afterward.  In fact, this happens as soon as a Pdb object is instantiated, regardless of whether or not you use it.

This is a result of some aspects of the Cmd object in Lib/cmd.py.

Currently Cmd registers a new readline completer as soon as a Cmd object is instantiated.  This is probably incorrect.  I believe the correct place to register the new handler is in the preloop hook.  That way the new readline completer is only used while the Cmd object is actually responsible for processing user input.

Next, the old readline completer should probably be re-registered in the postloop hook.  In order to do this, a new call must be added to the readline module to return the current completer so it can be saved and later restored.

The files included patch readline.c (version 2.41.64) and cmd.py (version 1.26.16.2) to make the changes described above.  After this patch readline works as expected in the interpreter.

Let me know what you think of it.

----------------------------------------------------------------------

>Comment By: Michael Stone (mbrierst)
Date: 2003-02-01 06:37

Message:
Logged In: YES 
user_id=670441

I started using the latest cvs version today.
I have attached a diff to the latest cvs as patchcmd2 if that's
more convenient for you to apply.

----------------------------------------------------------------------

Comment By: Michael Stone (mbrierst)
Date: 2003-01-31 21:25

Message:
Logged In: YES 
user_id=670441

I started using the latest cvs version today.
I have attached a diff to the latest cvs as patchcmd2 if that's
more convenient for you to apply.

----------------------------------------------------------------------

Comment By: Michael Stone (mbrierst)
Date: 2003-01-30 16:59

Message:
Logged In: YES 
user_id=670441

That sounds good to me.  Go ahead and check in patchcmd.

Thanks for your suggestions, I will follow them in future patches.  In this one, I was trying to follow the conventions I saw being used in the module, but they were outdated or incorrect (and I didn't always follow them correctly anyway!).  Okay that's definitely enough commentary for a few line patch.

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2003-01-30 10:22

Message:
Logged In: YES 
user_id=6656

Thanks for the new patch, it looks OK.  There are a couple
of things that spring to mind for future patches: 

- the function should be METH_NOARGS (or you should check
the args tuple)

- there should be a blank line between the argspec line in
the docstring and the description

- you should use the PyDoc_STRVAR macro (but this was only
used in the CVS trunk's version of readline).

I made these changes and checked in revision 2.59 of
Modules/readline.c.

Wrt the cmd.py changes, I think I'll probably let that
sleeping dog lie.  I agree with you in general: cmd.py
probably shouldn't be rebinding any keys, but

1) the interface is "specify the completion key to get
completion",   so backwards compatibility raises its head

2) as you say, most people do probably rebind tab anyway

so I propose to check in your patchcmd, close the bug and
wait for someone to submit a bug report about pdb messing up
their bindings.

----------------------------------------------------------------------

Comment By: Michael Stone (mbrierst)
Date: 2003-01-29 18:19

Message:
Logged In: YES 
user_id=670441

Actually, the readline library does have the functionality to get the current binding of the tab key, so it could be saved and restored.  However, to do this in python some stuff would have to be added to the readline.c module, to create some python layer that deals with the way readline stores macros, functions, and keymaps.  It's probably not worth the effort unless someone wants to do major changes to the readline module to add the full functionality of the underlying library to the python interface. 

Probably cmd.py shouldn't be rebinding the tab key to complete at all.  If the user wants that, won't they have already done it somewhere else, like in their pythonrc or inputrc files?  And if they don't it might be impolite to change their bindings.

----------------------------------------------------------------------

Comment By: Michael Stone (mbrierst)
Date: 2003-01-29 15:27

Message:
Logged In: YES 
user_id=670441

I attached a changed readline.c patch to hopefully get rid of any "sickness".
(There was a not-checking-for-null problem too... ouch)

I think it's good now.  But note, there may not be a way to do this sort of thing perfectly.  If you mess around enough with starting debuggers within debuggers (or in general anything that uses a Cmd object) it might be possible to end up with an unexpected completer function in the end.

And it looks like under the current system there's no way to get back the completer key that was initially used.  But this isn't SUCH a big deal since I think everyone uses tab, right?

At any rate, the change is definitely an improvement for me.

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2003-01-29 10:36

Message:
Logged In: YES 
user_id=6656

I approve of this patch in principle.  I hadn't done enough
investigation to see that it was so easy to get the current
completer function!

OTOH:

return Py_BuildValue("O", completer);

is sick.  Change that to

Py_INCREF(completer);
return completer;

and if noone else howls, I'll check it in.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=676342&group_id=5470


From noreply@sourceforge.net  Sat Feb  1 06:41:54 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Fri, 31 Jan 2003 22:41:54 -0800
Subject: [Python-bugs-list] [ python-Bugs-676342 ] after using pdb readline does not work correctly
Message-ID: <E18erLe-0008S2-00@sc8-sf-web2.sourceforge.net>

Bugs item #676342, was opened at 2003-01-28 20:53
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=676342&group_id=5470

Category: Python Library
Group: Python 2.2.2
Status: Open
Resolution: None
Priority: 5
Submitted By: Michael Stone (mbrierst)
Assigned to: Nobody/Anonymous (nobody)
Summary: after using pdb readline does not work correctly

Initial Comment:

After I use pdb in the interpreter my readline settings are messed up.  It only knows about the pdb readline stuff forever afterward.  In fact, this happens as soon as a Pdb object is instantiated, regardless of whether or not you use it.

This is a result of some aspects of the Cmd object in Lib/cmd.py.

Currently Cmd registers a new readline completer as soon as a Cmd object is instantiated.  This is probably incorrect.  I believe the correct place to register the new handler is in the preloop hook.  That way the new readline completer is only used while the Cmd object is actually responsible for processing user input.

Next, the old readline completer should probably be re-registered in the postloop hook.  In order to do this, a new call must be added to the readline module to return the current completer so it can be saved and later restored.

The files included patch readline.c (version 2.41.64) and cmd.py (version 1.26.16.2) to make the changes described above.  After this patch readline works as expected in the interpreter.

Let me know what you think of it.

----------------------------------------------------------------------

>Comment By: Michael Stone (mbrierst)
Date: 2003-02-01 06:41

Message:
Logged In: YES 
user_id=670441

I started using the latest cvs version today.
I have attached a diff to the latest cvs as patchcmd2 if that's
more convenient for you to apply.

----------------------------------------------------------------------

Comment By: Michael Stone (mbrierst)
Date: 2003-02-01 06:37

Message:
Logged In: YES 
user_id=670441

I started using the latest cvs version today.
I have attached a diff to the latest cvs as patchcmd2 if that's
more convenient for you to apply.

----------------------------------------------------------------------

Comment By: Michael Stone (mbrierst)
Date: 2003-01-31 21:25

Message:
Logged In: YES 
user_id=670441

I started using the latest cvs version today.
I have attached a diff to the latest cvs as patchcmd2 if that's
more convenient for you to apply.

----------------------------------------------------------------------

Comment By: Michael Stone (mbrierst)
Date: 2003-01-30 16:59

Message:
Logged In: YES 
user_id=670441

That sounds good to me.  Go ahead and check in patchcmd.

Thanks for your suggestions, I will follow them in future patches.  In this one, I was trying to follow the conventions I saw being used in the module, but they were outdated or incorrect (and I didn't always follow them correctly anyway!).  Okay that's definitely enough commentary for a few line patch.

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2003-01-30 10:22

Message:
Logged In: YES 
user_id=6656

Thanks for the new patch, it looks OK.  There are a couple
of things that spring to mind for future patches: 

- the function should be METH_NOARGS (or you should check
the args tuple)

- there should be a blank line between the argspec line in
the docstring and the description

- you should use the PyDoc_STRVAR macro (but this was only
used in the CVS trunk's version of readline).

I made these changes and checked in revision 2.59 of
Modules/readline.c.

Wrt the cmd.py changes, I think I'll probably let that
sleeping dog lie.  I agree with you in general: cmd.py
probably shouldn't be rebinding any keys, but

1) the interface is "specify the completion key to get
completion",   so backwards compatibility raises its head

2) as you say, most people do probably rebind tab anyway

so I propose to check in your patchcmd, close the bug and
wait for someone to submit a bug report about pdb messing up
their bindings.

----------------------------------------------------------------------

Comment By: Michael Stone (mbrierst)
Date: 2003-01-29 18:19

Message:
Logged In: YES 
user_id=670441

Actually, the readline library does have the functionality to get the current binding of the tab key, so it could be saved and restored.  However, to do this in python some stuff would have to be added to the readline.c module, to create some python layer that deals with the way readline stores macros, functions, and keymaps.  It's probably not worth the effort unless someone wants to do major changes to the readline module to add the full functionality of the underlying library to the python interface. 

Probably cmd.py shouldn't be rebinding the tab key to complete at all.  If the user wants that, won't they have already done it somewhere else, like in their pythonrc or inputrc files?  And if they don't it might be impolite to change their bindings.

----------------------------------------------------------------------

Comment By: Michael Stone (mbrierst)
Date: 2003-01-29 15:27

Message:
Logged In: YES 
user_id=670441

I attached a changed readline.c patch to hopefully get rid of any "sickness".
(There was a not-checking-for-null problem too... ouch)

I think it's good now.  But note, there may not be a way to do this sort of thing perfectly.  If you mess around enough with starting debuggers within debuggers (or in general anything that uses a Cmd object) it might be possible to end up with an unexpected completer function in the end.

And it looks like under the current system there's no way to get back the completer key that was initially used.  But this isn't SUCH a big deal since I think everyone uses tab, right?

At any rate, the change is definitely an improvement for me.

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2003-01-29 10:36

Message:
Logged In: YES 
user_id=6656

I approve of this patch in principle.  I hadn't done enough
investigation to see that it was so easy to get the current
completer function!

OTOH:

return Py_BuildValue("O", completer);

is sick.  Change that to

Py_INCREF(completer);
return completer;

and if noone else howls, I'll check it in.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=676342&group_id=5470


From noreply@sourceforge.net  Sat Feb  1 17:10:46 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sat, 01 Feb 2003 09:10:46 -0800
Subject: [Python-bugs-list] [ python-Bugs-678723 ] ./configure fails w/error: cannot compute sizeof (wchar_t),
Message-ID: <E18f1AE-00075z-00@sc8-sf-web4.sourceforge.net>

Bugs item #678723, was opened at 2003-02-01 12:10
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=678723&group_id=5470

Category: Installation
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Jeff Stephens (jeffste)
Assigned to: Nobody/Anonymous (nobody)
Summary: ./configure fails w/error: cannot compute sizeof (wchar_t), 

Initial Comment:
 I am running RH 7.3, kernel  2.4.18-19.7.x, and gcc-2.96-113.    
I attempted to configure python-2.3a1 as follows:   
   
./configure  --enable-shared --with-threads --with-pth  
  
Got the following error:  
  
checking size of wchar_t... configure: error: cannot compute 
sizeof (wchar_t), 77 
 
  

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=678723&group_id=5470


From noreply@sourceforge.net  Sat Feb  1 17:54:30 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sat, 01 Feb 2003 09:54:30 -0800
Subject: [Python-bugs-list] [ python-Bugs-678739 ] Leak
Message-ID: <E18f1qY-0002HF-00@sc8-sf-web2.sourceforge.net>

Bugs item #678739, was opened at 2003-02-01 12:54
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=678739&group_id=5470

Category: None
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Raymond Hettinger (rhettinger)
Assigned to: Nobody/Anonymous (nobody)
Summary: Leak

Initial Comment:
I haven't had time to trace down the cause yet, but 
the attached code fragment reveals a leak for 
recursive calls in an inner function definition, inside a 
method definition.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=678739&group_id=5470


From noreply@sourceforge.net  Sat Feb  1 18:33:10 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sat, 01 Feb 2003 10:33:10 -0800
Subject: [Python-bugs-list] [ python-Bugs-678723 ] ./configure fails w/error: cannot compute sizeof (wchar_t),
Message-ID: <E18f2Ry-0003Bx-00@sc8-sf-web2.sourceforge.net>

Bugs item #678723, was opened at 2003-02-01 12:10
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=678723&group_id=5470

Category: Installation
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Jeff Stephens (jeffste)
Assigned to: Nobody/Anonymous (nobody)
Summary: ./configure fails w/error: cannot compute sizeof (wchar_t), 

Initial Comment:
 I am running RH 7.3, kernel  2.4.18-19.7.x, and gcc-2.96-113.    
I attempted to configure python-2.3a1 as follows:   
   
./configure  --enable-shared --with-threads --with-pth  
  
Got the following error:  
  
checking size of wchar_t... configure: error: cannot compute 
sizeof (wchar_t), 77 
 
  

----------------------------------------------------------------------

>Comment By: Jeff Stephens (jeffste)
Date: 2003-02-01 13:33

Message:
Logged In: YES 
user_id=702289

I reran ./configure several times, varying the options, and found  
that this error only occurs when the option "--with-pth" is  
included.  For example: 
 
./configure --enable-shared --with-pth   #error occurs 
./configure --enable-shared --with-threads  #error does NOT occur 
 
I don't know whether this is a bug or not, so I am leaving it to 
the experts to say yea or nay.  

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=678723&group_id=5470


From noreply@sourceforge.net  Sat Feb  1 21:01:39 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sat, 01 Feb 2003 13:01:39 -0800
Subject: [Python-bugs-list] [ python-Bugs-678739 ] Leak
Message-ID: <E18f4lf-0001Gs-00@sc8-sf-web3.sourceforge.net>

Bugs item #678739, was opened at 2003-02-01 17:54
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=678739&group_id=5470

Category: None
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Raymond Hettinger (rhettinger)
Assigned to: Nobody/Anonymous (nobody)
Summary: Leak

Initial Comment:
I haven't had time to trace down the cause yet, but 
the attached code fragment reveals a leak for 
recursive calls in an inner function definition, inside a 
method definition.

----------------------------------------------------------------------

Comment By: Michael Stone (mbrierst)
Date: 2003-02-01 21:01

Message:
Logged In: YES 
user_id=670441

hmmm.... Is this really a leak?
try the following:

def reftest():
  prrefs = sys.gettotalrefcount()
  for i in xrange(2000):
    a = x.f(4)
    nref = sys.gettotalrefcount()
    if nref < prrefs:
      print prrefs, nref, i
    prrefs = nref

shows that every few hundred iterations or so, the refcount drops drastically.  Somehow it's getting cleaned up every once in a while.  Over many many runs, I see no leak in the long term.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=678739&group_id=5470


From noreply@sourceforge.net  Sat Feb  1 21:42:01 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sat, 01 Feb 2003 13:42:01 -0800
Subject: [Python-bugs-list] [ python-Bugs-678518 ] Another parsermodule validation error
Message-ID: <E18f5Oj-0002aN-00@sc8-sf-web3.sourceforge.net>

Bugs item #678518, was opened at 2003-01-31 20:23
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=678518&group_id=5470

Category: Parser/Compiler
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: logistix (logistix)
Assigned to: Nobody/Anonymous (nobody)
Summary: Another parsermodule validation error

Initial Comment:
The following code generates an validation error.  It's an 
invalid ast tree where the global_stmt doesn't have the 
keyword 'global' as a child.

I tracked down the fix to validate_global_stmt() this time.

Python 2.3a1 (#38, Dec 31 2002, 17:53:59) [MSC 
v.1200 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more 
information.
>>> import symbol,token,parser
>>> ast = [symbol.file_input,
...  [symbol.stmt, [symbol.simple_stmt, 
[symbol.small_stmt, [symbol.global_stmt,
 [1, 'foo']]], [token.NEWLINE, '']]],
...  [token.NEWLINE, ''],
...  [token.ENDMARKER, '']]
>>> a = parser.sequence2ast(ast)
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
parser.ParserError: VALIDATION FAILURE: report this 
to the maintainer!
>>>

----------------------------------------------------------------------

>Comment By: logistix (logistix)
Date: 2003-02-01 15:42

Message:
Logged In: YES 
user_id=699438

And I found a few more by doing a roundtrip 
ast2Tuple/tuple2ast on all files in pythonpath.  These actually 
affect valid AST's, so they're probably a bigger deal.

Valid assert statements were failing.  The patch was trivial, 
so its attached.

Also, files that took advantage of encoding_decl (from PEP 
263) were failing.  I couldn't tell if PEP 263 was offically 
closed, and didn't know what the offical grammar was, so I 
didn't do anything there.

I'll also attach the test script that turned up these errors.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=678518&group_id=5470


From noreply@sourceforge.net  Sat Feb  1 22:40:58 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sat, 01 Feb 2003 14:40:58 -0800
Subject: [Python-bugs-list] [ python-Bugs-676990 ] memory leak in pyexpat.c::my_ElementDeclHandler()
Message-ID: <E18f6Jm-0002JX-00@sc8-sf-web4.sourceforge.net>

Bugs item #676990, was opened at 2003-01-29 14:52
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=676990&group_id=5470

Category: XML
Group: Python 2.3
Status: Open
>Resolution: Accepted
Priority: 6
Submitted By: Neal Norwitz (nnorwitz)
>Assigned to: Fred L. Drake, Jr. (fdrake)
Summary: memory leak in pyexpat.c::my_ElementDeclHandler()

Initial Comment:
Fred, I'm assigning to you since you've seen this once
already. :-)  Not sure who would know best how to fix
this problem.  Perhaps Martin, but he's on vacation.

As of now, 2003/1/29, there is a memory leak in
xmlparse.c::doProlog() line 3979.

The XML_Content is dynamically allocated, but never
freed in the block.  The lifetime of the variable
content is not clear.  It is passed to
elementDeclHandler().  Does content need to be dynamic,
or can it be a local (stack) variable?  Does
elementDeclHandler() take ownership of content?

I can make the change in Python (which is where this
problem was found from test_minidom).

20 bytes in 1 blocks are definitely lost in loss record
2 of 15
  at 0x4015D480: malloc (vg_clientfuncs.c:100)
doProlog (Modules/expat/xmlparse.c:3979)
prologProcessor (Modules/expat/xmlparse.c:3185)
prologInitProcessor (Modules/expat/xmlparse.c:3016)
XML_ParseBuffer (Modules/expat/xmlparse.c:1439)
XML_Parse (Modules/expat/xmlparse.c:1427)
xmlparse_Parse (Modules/pyexpat.c:853)

This problem was originally from this bug report in expat:

http://sf.net/tracker/?func=detail&atid=110127&aid=676926&group_id=10127


----------------------------------------------------------------------

>Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-01 17:40

Message:
Logged In: YES 
user_id=33168

I used valgrind on test_minidom.  This patch does fix the leak.

----------------------------------------------------------------------

Comment By: Fred L. Drake, Jr. (fdrake)
Date: 2003-01-31 12:22

Message:
Logged In: YES 
user_id=3066

Oops, patch really attached now.

----------------------------------------------------------------------

Comment By: Fred L. Drake, Jr. (fdrake)
Date: 2003-01-30 10:38

Message:
Logged In: YES 
user_id=3066

The attached patch frees the content model passed to the
handler.  This should fix the leak; please test using
whatever tool you used to discover this.

Thanks!

----------------------------------------------------------------------

Comment By: Fred L. Drake, Jr. (fdrake)
Date: 2003-01-30 09:44

Message:
Logged In: YES 
user_id=3066

Fixed summary.

----------------------------------------------------------------------

Comment By: Fred L. Drake, Jr. (fdrake)
Date: 2003-01-29 15:03

Message:
Logged In: YES 
user_id=3066

As noted in the Expat tracker, this is not a memory leak in 
Expat, but in pyexpat.  I've taken a quick look at the relevant 
code in pyexpat; it won't be hard to fix, but it will take longer 
than writing this response.  I'll try to knock it out tonight.

(For the record: this isn't due to any changes in Expat, it's 
just that the bug in pyexpat has only now been caught.)

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=676990&group_id=5470


From noreply@sourceforge.net  Sun Feb  2 01:45:52 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sat, 01 Feb 2003 17:45:52 -0800
Subject: [Python-bugs-list] [ python-Bugs-662840 ] configure script fails with wchat_t size.
Message-ID: <E18f9Ci-00080l-00@sc8-sf-web4.sourceforge.net>

Bugs item #662840, was opened at 2003-01-05 17:03
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=662840&group_id=5470

Category: Build
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Nadav Horesh (nadavhoresh)
Assigned to: Nobody/Anonymous (nobody)
Summary: configure script fails with wchat_t size. 

Initial Comment:
Python version 2.3a1
System RH 7.3
Compiler: gcc 3.2.1

The configure script bails out when trying to determain
the size of wchar_t. The configure runs OK with gcc 2.96.
.
.
.

checking wchar.h usability... yes
checking wchar.h presence... yes
checking for wchar.h... yes
checking for wchar_t... yes
checking size of wchar_t... configure: error: cannot
compute sizeof (wchar_t), 77


----------------------------------------------------------------------

>Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-01 20:45

Message:
Logged In: YES 
user_id=33168

Seems to be a duplicate of 678723 (or at least related, both
are redhat 7.3).


----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2003-01-06 17:01

Message:
Logged In: YES 
user_id=21627

I see. The problem lies in these lines

configure:15806: ./conftest
./configure: line 1:  6876 Segmentation fault     
./conftest$ac_exeext

Now, it is not clear why the test program (printed below
these lines) could possible cause a segmentation fault -
unless there is a bug in the compiler. So I would conclude
this to be a gcc bug, on this configuration.

It would be possible to track this further down, but that
would require access to the machine, using a debugger, etc.

----------------------------------------------------------------------

Comment By: Nadav Horesh (nadavhoresh)
Date: 2003-01-06 15:02

Message:
Logged In: YES 
user_id=75473

Further info:
On the same system (RH7.3+gcc3.2.1+PIII) I compliled python
2.2.2 without any problems. On another system: RH8.0 +
gcc3.2.1 + dual PIII --- the configure script concluded
without errors. I didn't try to compile though.

----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2003-01-05 18:12

Message:
Logged In: YES 
user_id=21627

Can you please attach the resulting config.log?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=662840&group_id=5470


From noreply@sourceforge.net  Sun Feb  2 01:46:07 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sat, 01 Feb 2003 17:46:07 -0800
Subject: [Python-bugs-list] [ python-Bugs-678723 ] ./configure fails w/error: cannot compute sizeof (wchar_t),
Message-ID: <E18f9Cx-0002vZ-00@sc8-sf-web1.sourceforge.net>

Bugs item #678723, was opened at 2003-02-01 12:10
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=678723&group_id=5470

Category: Installation
Group: Python 2.3
Status: Open
>Resolution: Duplicate
Priority: 5
Submitted By: Jeff Stephens (jeffste)
Assigned to: Nobody/Anonymous (nobody)
Summary: ./configure fails w/error: cannot compute sizeof (wchar_t), 

Initial Comment:
 I am running RH 7.3, kernel  2.4.18-19.7.x, and gcc-2.96-113.    
I attempted to configure python-2.3a1 as follows:   
   
./configure  --enable-shared --with-threads --with-pth  
  
Got the following error:  
  
checking size of wchar_t... configure: error: cannot compute 
sizeof (wchar_t), 77 
 
  

----------------------------------------------------------------------

>Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-01 20:46

Message:
Logged In: YES 
user_id=33168

Jeff, I think we should try to fix this.  It looks like a
duplicate of 662840.  It would be great if you can help
debug this further.

The problem seems to be related to redhat 7.3.  The other
bug which fails exactly the same way is using gcc 3.2 and
doesn't mention config options (although they may be used).

----------------------------------------------------------------------

Comment By: Jeff Stephens (jeffste)
Date: 2003-02-01 13:33

Message:
Logged In: YES 
user_id=702289

I reran ./configure several times, varying the options, and found  
that this error only occurs when the option "--with-pth" is  
included.  For example: 
 
./configure --enable-shared --with-pth   #error occurs 
./configure --enable-shared --with-threads  #error does NOT occur 
 
I don't know whether this is a bug or not, so I am leaving it to 
the experts to say yea or nay.  

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=678723&group_id=5470


From noreply@sourceforge.net  Sun Feb  2 01:49:30 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sat, 01 Feb 2003 17:49:30 -0800
Subject: [Python-bugs-list] [ python-Bugs-678352 ] list.append memory leak ?
Message-ID: <E18f9GE-0001S2-00@sc8-sf-web3.sourceforge.net>

Bugs item #678352, was opened at 2003-01-31 15:18
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=678352&group_id=5470

Category: Python Interpreter Core
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: thanos  vassilakis (thanos)
Assigned to: Nobody/Anonymous (nobody)
Summary: list.append memory leak ?

Initial Comment:
# this SEEMS to leak
l = xrange(10000)
while 1:
        a=[]
        map(a.append, l)
        del a

# this does not 
while 1:
       a=range(10000)
       del a



----------------------------------------------------------------------

>Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-01 20:49

Message:
Logged In: YES 
user_id=33168

Was the python version 2.2 or 2.2.2?  If you are using 2.2,
you should probably try to upgrade to 2.2.2 since there have
been many bug fixes.

----------------------------------------------------------------------

Comment By: thanos  vassilakis (thanos)
Date: 2003-01-31 22:07

Message:
Logged In: YES 
user_id=91277

 We use HP UX 11 and BSD 4.7 but what were the origial build compilers and libraries  will have to wait till Monday - when I will be back at work and have access to the NYSE trading  floor systems.

I am chasing some memory leak that occurred when we moved from python 1.5.2 to 2.2


I don't think this is it, but...

Thanks for your quick response.




----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2003-01-31 20:40

Message:
Logged In: YES 
user_id=31435

Again, please identify the Python version and 
OS+compiler+C library in use.  I'm not arguing with you, I'm 
trying to help <wink>.  Since I don't see any evidence of 
anything like I leak when I try it (memory usage didn't 
increase *at all* when I tried it) , the answer to why you see 
it must lie in something you're using that I'm not -- only you 
can tell us what that might be.  I already told you the 
Python version, OS and C compiler/library version I used.

----------------------------------------------------------------------

Comment By: thanos  vassilakis (thanos)
Date: 2003-01-31 17:47

Message:
Logged In: YES 
user_id=91277

HP UX B.11.00
BSD 4.7
python 2.2

Its possible it is not a bug that is why I used the word SEEMS
but why does the second code fragment behave? 


----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2003-01-31 16:00

Message:
Logged In: YES 
user_id=31435

Please identify the Python version and OS+compiler+C 
library in use.  The process size remains steady for me under 
current CVS Python on Win2K w/ MSVC6.

A true leak is possible but unlikely.  More likely is that your 
platform C malloc()/free() decide not to return memory to the 
OS.  If so, that's outside of Python's control

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=678352&group_id=5470


From noreply@sourceforge.net  Sun Feb  2 03:07:57 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sat, 01 Feb 2003 19:07:57 -0800
Subject: [Python-bugs-list] [ python-Bugs-678739 ] Leak
Message-ID: <E18fAU9-0003Ex-00@sc8-sf-web3.sourceforge.net>

Bugs item #678739, was opened at 2003-02-01 12:54
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=678739&group_id=5470

Category: None
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Raymond Hettinger (rhettinger)
>Assigned to: Raymond Hettinger (rhettinger)
Summary: Leak

Initial Comment:
I haven't had time to trace down the cause yet, but 
the attached code fragment reveals a leak for 
recursive calls in an inner function definition, inside a 
method definition.

----------------------------------------------------------------------

>Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-01 22:07

Message:
Logged In: YES 
user_id=33168

Raymond, it looks like the issue is cyclic garbage.  If you
add gc.collect() after the call to x.f(4), the refcount
stays constant.  So it looks ok to me.

----------------------------------------------------------------------

Comment By: Michael Stone (mbrierst)
Date: 2003-02-01 16:01

Message:
Logged In: YES 
user_id=670441

hmmm.... Is this really a leak?
try the following:

def reftest():
  prrefs = sys.gettotalrefcount()
  for i in xrange(2000):
    a = x.f(4)
    nref = sys.gettotalrefcount()
    if nref < prrefs:
      print prrefs, nref, i
    prrefs = nref

shows that every few hundred iterations or so, the refcount drops drastically.  Somehow it's getting cleaned up every once in a while.  Over many many runs, I see no leak in the long term.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=678739&group_id=5470


From noreply@sourceforge.net  Sun Feb  2 03:18:49 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sat, 01 Feb 2003 19:18:49 -0800
Subject: [Python-bugs-list] [ python-Bugs-678739 ] Leak
Message-ID: <E18fAef-00020m-00@sc8-sf-web4.sourceforge.net>

Bugs item #678739, was opened at 2003-02-01 12:54
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=678739&group_id=5470

Category: None
Group: Python 2.3
>Status: Closed
>Resolution: Invalid
Priority: 5
Submitted By: Raymond Hettinger (rhettinger)
Assigned to: Raymond Hettinger (rhettinger)
Summary: Leak

Initial Comment:
I haven't had time to trace down the cause yet, but 
the attached code fragment reveals a leak for 
recursive calls in an inner function definition, inside a 
method definition.

----------------------------------------------------------------------

>Comment By: Raymond Hettinger (rhettinger)
Date: 2003-02-01 22:18

Message:
Logged In: YES 
user_id=80475

That's cool.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-01 22:07

Message:
Logged In: YES 
user_id=33168

Raymond, it looks like the issue is cyclic garbage.  If you
add gc.collect() after the call to x.f(4), the refcount
stays constant.  So it looks ok to me.

----------------------------------------------------------------------

Comment By: Michael Stone (mbrierst)
Date: 2003-02-01 16:01

Message:
Logged In: YES 
user_id=670441

hmmm.... Is this really a leak?
try the following:

def reftest():
  prrefs = sys.gettotalrefcount()
  for i in xrange(2000):
    a = x.f(4)
    nref = sys.gettotalrefcount()
    if nref < prrefs:
      print prrefs, nref, i
    prrefs = nref

shows that every few hundred iterations or so, the refcount drops drastically.  Somehow it's getting cleaned up every once in a while.  Over many many runs, I see no leak in the long term.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=678739&group_id=5470


From noreply@sourceforge.net  Sun Feb  2 03:27:37 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sat, 01 Feb 2003 19:27:37 -0800
Subject: [Python-bugs-list] [ python-Bugs-678352 ] list.append memory leak ?
Message-ID: <E18fAnB-0003ai-00@sc8-sf-web3.sourceforge.net>

Bugs item #678352, was opened at 2003-01-31 15:18
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=678352&group_id=5470

Category: Python Interpreter Core
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: thanos  vassilakis (thanos)
Assigned to: Nobody/Anonymous (nobody)
Summary: list.append memory leak ?

Initial Comment:
# this SEEMS to leak
l = xrange(10000)
while 1:
        a=[]
        map(a.append, l)
        del a

# this does not 
while 1:
       a=range(10000)
       del a



----------------------------------------------------------------------

>Comment By: thanos  vassilakis (thanos)
Date: 2003-02-01 22:27

Message:
Logged In: YES 
user_id=91277

Good idea

I will try it out on my OSX mac tomorrow (after upgrading !)



----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-01 20:49

Message:
Logged In: YES 
user_id=33168

Was the python version 2.2 or 2.2.2?  If you are using 2.2,
you should probably try to upgrade to 2.2.2 since there have
been many bug fixes.

----------------------------------------------------------------------

Comment By: thanos  vassilakis (thanos)
Date: 2003-01-31 22:07

Message:
Logged In: YES 
user_id=91277

 We use HP UX 11 and BSD 4.7 but what were the origial build compilers and libraries  will have to wait till Monday - when I will be back at work and have access to the NYSE trading  floor systems.

I am chasing some memory leak that occurred when we moved from python 1.5.2 to 2.2


I don't think this is it, but...

Thanks for your quick response.




----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2003-01-31 20:40

Message:
Logged In: YES 
user_id=31435

Again, please identify the Python version and 
OS+compiler+C library in use.  I'm not arguing with you, I'm 
trying to help <wink>.  Since I don't see any evidence of 
anything like I leak when I try it (memory usage didn't 
increase *at all* when I tried it) , the answer to why you see 
it must lie in something you're using that I'm not -- only you 
can tell us what that might be.  I already told you the 
Python version, OS and C compiler/library version I used.

----------------------------------------------------------------------

Comment By: thanos  vassilakis (thanos)
Date: 2003-01-31 17:47

Message:
Logged In: YES 
user_id=91277

HP UX B.11.00
BSD 4.7
python 2.2

Its possible it is not a bug that is why I used the word SEEMS
but why does the second code fragment behave? 


----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2003-01-31 16:00

Message:
Logged In: YES 
user_id=31435

Please identify the Python version and OS+compiler+C 
library in use.  The process size remains steady for me under 
current CVS Python on Win2K w/ MSVC6.

A true leak is possible but unlikely.  More likely is that your 
platform C malloc()/free() decide not to return memory to the 
OS.  If so, that's outside of Python's control

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=678352&group_id=5470


From noreply@sourceforge.net  Sun Feb  2 04:02:34 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sat, 01 Feb 2003 20:02:34 -0800
Subject: [Python-bugs-list] [ python-Bugs-676990 ] memory leak in pyexpat.c::my_ElementDeclHandler()
Message-ID: <E18fBL0-0004MB-00@sc8-sf-web3.sourceforge.net>

Bugs item #676990, was opened at 2003-01-29 14:52
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=676990&group_id=5470

Category: XML
Group: Python 2.3
>Status: Closed
Resolution: Accepted
Priority: 6
Submitted By: Neal Norwitz (nnorwitz)
Assigned to: Fred L. Drake, Jr. (fdrake)
Summary: memory leak in pyexpat.c::my_ElementDeclHandler()

Initial Comment:
Fred, I'm assigning to you since you've seen this once
already. :-)  Not sure who would know best how to fix
this problem.  Perhaps Martin, but he's on vacation.

As of now, 2003/1/29, there is a memory leak in
xmlparse.c::doProlog() line 3979.

The XML_Content is dynamically allocated, but never
freed in the block.  The lifetime of the variable
content is not clear.  It is passed to
elementDeclHandler().  Does content need to be dynamic,
or can it be a local (stack) variable?  Does
elementDeclHandler() take ownership of content?

I can make the change in Python (which is where this
problem was found from test_minidom).

20 bytes in 1 blocks are definitely lost in loss record
2 of 15
  at 0x4015D480: malloc (vg_clientfuncs.c:100)
doProlog (Modules/expat/xmlparse.c:3979)
prologProcessor (Modules/expat/xmlparse.c:3185)
prologInitProcessor (Modules/expat/xmlparse.c:3016)
XML_ParseBuffer (Modules/expat/xmlparse.c:1439)
XML_Parse (Modules/expat/xmlparse.c:1427)
xmlparse_Parse (Modules/pyexpat.c:853)

This problem was originally from this bug report in expat:

http://sf.net/tracker/?func=detail&atid=110127&aid=676926&group_id=10127


----------------------------------------------------------------------

>Comment By: Fred L. Drake, Jr. (fdrake)
Date: 2003-02-01 23:02

Message:
Logged In: YES 
user_id=3066

Committed as Modules/pyexpat.c revision 2.80.
Backport candidate.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-01 17:40

Message:
Logged In: YES 
user_id=33168

I used valgrind on test_minidom.  This patch does fix the leak.

----------------------------------------------------------------------

Comment By: Fred L. Drake, Jr. (fdrake)
Date: 2003-01-31 12:22

Message:
Logged In: YES 
user_id=3066

Oops, patch really attached now.

----------------------------------------------------------------------

Comment By: Fred L. Drake, Jr. (fdrake)
Date: 2003-01-30 10:38

Message:
Logged In: YES 
user_id=3066

The attached patch frees the content model passed to the
handler.  This should fix the leak; please test using
whatever tool you used to discover this.

Thanks!

----------------------------------------------------------------------

Comment By: Fred L. Drake, Jr. (fdrake)
Date: 2003-01-30 09:44

Message:
Logged In: YES 
user_id=3066

Fixed summary.

----------------------------------------------------------------------

Comment By: Fred L. Drake, Jr. (fdrake)
Date: 2003-01-29 15:03

Message:
Logged In: YES 
user_id=3066

As noted in the Expat tracker, this is not a memory leak in 
Expat, but in pyexpat.  I've taken a quick look at the relevant 
code in pyexpat; it won't be hard to fix, but it will take longer 
than writing this response.  I'll try to knock it out tonight.

(For the record: this isn't due to any changes in Expat, it's 
just that the bug in pyexpat has only now been caught.)

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=676990&group_id=5470


From noreply@sourceforge.net  Sun Feb  2 23:41:44 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sun, 02 Feb 2003 15:41:44 -0800
Subject: [Python-bugs-list] [ python-Bugs-679251 ] PEP 277 in what's new for 2.3
Message-ID: <E18fTk8-0003U0-00@sc8-sf-web1.sourceforge.net>

Bugs item #679251, was opened at 2003-02-02 16:41
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=679251&group_id=5470

Category: Documentation
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Andrew Dalke (dalke)
Assigned to: Nobody/Anonymous (nobody)
Summary: PEP 277 in what's new for 2.3

Initial Comment:
> Applications can test whether arbitrary Unicode strings
> are supported as file names by checking
>  os.path.unicode_file_names

That should be os.path.supports_unicode_filenames

>>> import os
>>> [s for s in dir(os.path) if "unicode" in s]
['supports_unicode_filenames']
>>>


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=679251&group_id=5470


From noreply@sourceforge.net  Mon Feb  3 03:22:47 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sun, 02 Feb 2003 19:22:47 -0800
Subject: [Python-bugs-list] [ python-Bugs-640230 ] Discovered typo in zlib test.
Message-ID: <E18fXC3-0000lp-00@sc8-sf-web1.sourceforge.net>

Bugs item #640230, was opened at 2002-11-18 13:36
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=640230&group_id=5470

Category: None
Group: None
Status: Open
Resolution: None
>Priority: 7
Submitted By: Scott David Daniels (scott_daniels)
Assigned to: Nobody/Anonymous (nobody)
Summary: Discovered typo in zlib test.

Initial Comment:
In test_zlib.py (while chasing what appears to be a
documentation problem), I found a flaw in the code
that tests max_length limitted output from a 
decompression object test.

Starting at line 100, the existing code is:
...
bufs.append(deco.flush())
decomp2 = ''.join(buf)
if decomp2 != buf:
    print "max_length decompressobj failed"
else:
    print "max_length decompressobj succeeded"
...

This test will always succeed (''.join(str) == str).

What seems obviously meant is:
...
bufs.append(deco.flush())
decomp2 = ''.join(bufs)
if decomp2 != buf:
    print "max_length decompressobj failed"
else:
    print "max_length decompressobj succeeded"
...



----------------------------------------------------------------------

>Comment By: Tim Peters (tim_one)
Date: 2003-02-02 22:22

Message:
Logged In: YES 
user_id=31435

It looks like the problem with getting this reviewed is that 
no current developer understands the zlib code or its test.  
I bumped the priority in a sneaky attempt to attract more 
interest <wink>.  *If* nobody else bites soon, and I can 
make time, I'll assign it to myself.

Your efforts are appreciated!  Even if it doesn't seem like it.

----------------------------------------------------------------------

Comment By: Scott David Daniels (scott_daniels)
Date: 2003-01-31 22:33

Message:
Logged In: YES 
user_id=493818

I've attached a PyUnit-style replacement for test_zlib.py that 
tests this condition to Patch #678531,  which provides a
context 
diff updating zlibmodule.c to fix this problem.  Hope this
is enough 
to get the fix reviewed.


----------------------------------------------------------------------

Comment By: Scott David Daniels (scott_daniels)
Date: 2002-12-12 13:32

Message:
Logged In: YES 
user_id=493818

The following replacement for PyZlib_unflush addresses some 
of the problem, but leaves unaddressed the issues of what 
unused_data and unconsumed_tail should be at the end of the 
routine.

static PyObject *
PyZlib_unflush(compobject *self, PyObject *args)
{
    int err, length = DEFAULTALLOC;
    PyObject * retval = NULL;
    unsigned long start_total_out;

    if (!PyArg_ParseTuple(args, "|i:flush", &length))
	return NULL;
    if (!(retval = PyString_FromStringAndSize(NULL, length)))
	return NULL;


    ENTER_ZLIB

    start_total_out = self->zst.total_out;
    self->zst.avail_out = length;
    self->zst.next_out = (Byte *)PyString_AS_STRING(retval);

    Py_BEGIN_ALLOW_THREADS
    err = inflate(&(self->zst), Z_FINISH);
    Py_END_ALLOW_THREADS

    /* while Z_OK and the output buffer is full, there might
be more output,
       so extend the output buffer and try again */
    while ((err == Z_OK || err == Z_BUF_ERROR) &&
self->zst.avail_out == 0) {
	if (_PyString_Resize(&retval, length << 1) < 0)
	    goto error;
	self->zst.next_out = (Byte *)PyString_AS_STRING(retval) +
length;
	self->zst.avail_out = length;
	length = length << 1;

	Py_BEGIN_ALLOW_THREADS
	err = inflate(&(self->zst), Z_FINISH);
	Py_END_ALLOW_THREADS
    }

    /* Maybe _always_ call inflateEnd if clearing
is_initialized */
    if (err == Z_STREAM_END) {
	err = inflateEnd(&(self->zst));
	if (err != Z_OK) {
	    zlib_error(self->zst, err, "from inflateEnd()");
	    Py_DECREF(retval);
	    retval = NULL;
	    goto error;
	}
    }
    self->is_initialised = 0;
    _PyString_Resize(&retval, self->zst.total_out -
start_total_out);
    /* ??? Here fix unused_data and unconsumed_tail */

error:

    LEAVE_ZLIB

    return retval;
}



----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2002-11-19 17:08

Message:
Logged In: YES 
user_id=31435

I'd say it's a bug in flush(), or in the docs, then.  They say "all 
pending input is processed, and a string containing the 
remaining uncompressed output is returned", and that's not 
happening here.  That's pretty clearly what the author of this 
test *expected* to happen, too (the original bug you 
discovered looked much more like a typo than a failure to 
understand the module interface).

----------------------------------------------------------------------

Comment By: Scott David Daniels (scott_daniels)
Date: 2002-11-19 16:40

Message:
Logged In: YES 
user_id=493818

OK, The reason the code fails after the fix is as follows:
deco.flush() does not return any output.
The loop control says, "while we have more unexamined 
source."

However, the decompressor can consume all of the input 
before it has provided all of its output.  So, there are two 
possible fixes:

1) Minimal change:  Change the loop control to say, in 
effect, "While we have more input or are producing output.":
Around line 92:
Change:
    bufs = []
    while cb:
        max_length = 1 + len(cb)/10
To:
    bufs = []
    while cb or chunk:
        max_length = 1 + len(cb)/10


2) More reasonable(?) change:  After the insertion loop, (just 
before the flush) extract all remaining output:
Around line 99:
        cb = deco.unconsumed_tail
    bufs.append(deco.flush())
    decomp2 = ''.join(bufs)
Becomes:
        cb = deco.unconsumed_tail
    while bufs[-1]:
        bufs.append(deco.decompress('', max_length))
    bufs.append(deco.flush())
    decomp2 = ''.join(bufs)


---
Note, in any case, the
    bufs.append(deco.flush())
originally on line 100 _always_ appends a zero-length string.
I would suggest changing it to simply:
    deco.flush()

    

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2002-11-19 04:50

Message:
Logged In: YES 
user_id=6656

Assuming Tim meant what he said, not what he did: raising
priority.

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2002-11-18 15:56

Message:
Logged In: YES 
user_id=31435

Good eye!  Unfortunately, when I repair that, the test fails 
here -- decomp2 is a proper prefix of buf then.  It's "too 
short".  Raised priority but left unassigned (someone who 
understands zlib better will have to jump in).

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=640230&group_id=5470


From noreply@sourceforge.net  Mon Feb  3 11:00:44 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Mon, 03 Feb 2003 03:00:44 -0800
Subject: [Python-bugs-list] [ python-Bugs-678518 ] Another parsermodule validation error
Message-ID: <E18feLE-0000TQ-00@sc8-sf-web1.sourceforge.net>

Bugs item #678518, was opened at 2003-02-01 02:23
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=678518&group_id=5470

Category: Parser/Compiler
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: logistix (logistix)
>Assigned to: Michael Hudson (mwh)
Summary: Another parsermodule validation error

Initial Comment:
The following code generates an validation error.  It's an 
invalid ast tree where the global_stmt doesn't have the 
keyword 'global' as a child.

I tracked down the fix to validate_global_stmt() this time.

Python 2.3a1 (#38, Dec 31 2002, 17:53:59) [MSC 
v.1200 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more 
information.
>>> import symbol,token,parser
>>> ast = [symbol.file_input,
...  [symbol.stmt, [symbol.simple_stmt, 
[symbol.small_stmt, [symbol.global_stmt,
 [1, 'foo']]], [token.NEWLINE, '']]],
...  [token.NEWLINE, ''],
...  [token.ENDMARKER, '']]
>>> a = parser.sequence2ast(ast)
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
parser.ParserError: VALIDATION FAILURE: report this 
to the maintainer!
>>>

----------------------------------------------------------------------

>Comment By: Michael Hudson (mwh)
Date: 2003-02-03 11:00

Message:
Logged In: YES 
user_id=6656

Thanks for doing this.  I'll probably check these in after
I've finished crunching my way through monday morning email...

----------------------------------------------------------------------

Comment By: logistix (logistix)
Date: 2003-02-01 21:42

Message:
Logged In: YES 
user_id=699438

And I found a few more by doing a roundtrip 
ast2Tuple/tuple2ast on all files in pythonpath.  These actually 
affect valid AST's, so they're probably a bigger deal.

Valid assert statements were failing.  The patch was trivial, 
so its attached.

Also, files that took advantage of encoding_decl (from PEP 
263) were failing.  I couldn't tell if PEP 263 was offically 
closed, and didn't know what the offical grammar was, so I 
didn't do anything there.

I'll also attach the test script that turned up these errors.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=678518&group_id=5470


From noreply@sourceforge.net  Mon Feb  3 11:11:13 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Mon, 03 Feb 2003 03:11:13 -0800
Subject: [Python-bugs-list] [ python-Bugs-676342 ] after using pdb readline does not work correctly
Message-ID: <E18feVN-0003hu-00@sc8-sf-web3.sourceforge.net>

Bugs item #676342, was opened at 2003-01-28 20:53
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=676342&group_id=5470

Category: Python Library
Group: Python 2.2.2
>Status: Closed
>Resolution: Accepted
Priority: 5
Submitted By: Michael Stone (mbrierst)
>Assigned to: Michael Hudson (mwh)
Summary: after using pdb readline does not work correctly

Initial Comment:

After I use pdb in the interpreter my readline settings are messed up.  It only knows about the pdb readline stuff forever afterward.  In fact, this happens as soon as a Pdb object is instantiated, regardless of whether or not you use it.

This is a result of some aspects of the Cmd object in Lib/cmd.py.

Currently Cmd registers a new readline completer as soon as a Cmd object is instantiated.  This is probably incorrect.  I believe the correct place to register the new handler is in the preloop hook.  That way the new readline completer is only used while the Cmd object is actually responsible for processing user input.

Next, the old readline completer should probably be re-registered in the postloop hook.  In order to do this, a new call must be added to the readline module to return the current completer so it can be saved and later restored.

The files included patch readline.c (version 2.41.64) and cmd.py (version 1.26.16.2) to make the changes described above.  After this patch readline works as expected in the interpreter.

Let me know what you think of it.

----------------------------------------------------------------------

>Comment By: Michael Hudson (mwh)
Date: 2003-02-03 11:11

Message:
Logged In: YES 
user_id=6656

Checked in as 

Lib/cmd.py revision 1.34

Thanks!  You probably didn't need to attach the patch three
times, though :-)

----------------------------------------------------------------------

Comment By: Michael Stone (mbrierst)
Date: 2003-02-01 06:41

Message:
Logged In: YES 
user_id=670441

I started using the latest cvs version today.
I have attached a diff to the latest cvs as patchcmd2 if that's
more convenient for you to apply.

----------------------------------------------------------------------

Comment By: Michael Stone (mbrierst)
Date: 2003-02-01 06:37

Message:
Logged In: YES 
user_id=670441

I started using the latest cvs version today.
I have attached a diff to the latest cvs as patchcmd2 if that's
more convenient for you to apply.

----------------------------------------------------------------------

Comment By: Michael Stone (mbrierst)
Date: 2003-01-31 21:25

Message:
Logged In: YES 
user_id=670441

I started using the latest cvs version today.
I have attached a diff to the latest cvs as patchcmd2 if that's
more convenient for you to apply.

----------------------------------------------------------------------

Comment By: Michael Stone (mbrierst)
Date: 2003-01-30 16:59

Message:
Logged In: YES 
user_id=670441

That sounds good to me.  Go ahead and check in patchcmd.

Thanks for your suggestions, I will follow them in future patches.  In this one, I was trying to follow the conventions I saw being used in the module, but they were outdated or incorrect (and I didn't always follow them correctly anyway!).  Okay that's definitely enough commentary for a few line patch.

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2003-01-30 10:22

Message:
Logged In: YES 
user_id=6656

Thanks for the new patch, it looks OK.  There are a couple
of things that spring to mind for future patches: 

- the function should be METH_NOARGS (or you should check
the args tuple)

- there should be a blank line between the argspec line in
the docstring and the description

- you should use the PyDoc_STRVAR macro (but this was only
used in the CVS trunk's version of readline).

I made these changes and checked in revision 2.59 of
Modules/readline.c.

Wrt the cmd.py changes, I think I'll probably let that
sleeping dog lie.  I agree with you in general: cmd.py
probably shouldn't be rebinding any keys, but

1) the interface is "specify the completion key to get
completion",   so backwards compatibility raises its head

2) as you say, most people do probably rebind tab anyway

so I propose to check in your patchcmd, close the bug and
wait for someone to submit a bug report about pdb messing up
their bindings.

----------------------------------------------------------------------

Comment By: Michael Stone (mbrierst)
Date: 2003-01-29 18:19

Message:
Logged In: YES 
user_id=670441

Actually, the readline library does have the functionality to get the current binding of the tab key, so it could be saved and restored.  However, to do this in python some stuff would have to be added to the readline.c module, to create some python layer that deals with the way readline stores macros, functions, and keymaps.  It's probably not worth the effort unless someone wants to do major changes to the readline module to add the full functionality of the underlying library to the python interface. 

Probably cmd.py shouldn't be rebinding the tab key to complete at all.  If the user wants that, won't they have already done it somewhere else, like in their pythonrc or inputrc files?  And if they don't it might be impolite to change their bindings.

----------------------------------------------------------------------

Comment By: Michael Stone (mbrierst)
Date: 2003-01-29 15:27

Message:
Logged In: YES 
user_id=670441

I attached a changed readline.c patch to hopefully get rid of any "sickness".
(There was a not-checking-for-null problem too... ouch)

I think it's good now.  But note, there may not be a way to do this sort of thing perfectly.  If you mess around enough with starting debuggers within debuggers (or in general anything that uses a Cmd object) it might be possible to end up with an unexpected completer function in the end.

And it looks like under the current system there's no way to get back the completer key that was initially used.  But this isn't SUCH a big deal since I think everyone uses tab, right?

At any rate, the change is definitely an improvement for me.

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2003-01-29 10:36

Message:
Logged In: YES 
user_id=6656

I approve of this patch in principle.  I hadn't done enough
investigation to see that it was so easy to get the current
completer function!

OTOH:

return Py_BuildValue("O", completer);

is sick.  Change that to

Py_INCREF(completer);
return completer;

and if noone else howls, I'll check it in.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=676342&group_id=5470


From noreply@sourceforge.net  Mon Feb  3 14:22:52 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Mon, 03 Feb 2003 06:22:52 -0800
Subject: [Python-bugs-list] [ python-Bugs-679520 ] long(3.1415) gives zero on Solaris 8
Message-ID: <E18fhUq-0003VG-00@sc8-sf-web3.sourceforge.net>

Bugs item #679520, was opened at 2003-02-03 14:22
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=679520&group_id=5470

Category: Python Interpreter Core
Group: Platform-specific
Status: Open
Resolution: None
Priority: 5
Submitted By: Mark Dickinson (marketdickinson)
Assigned to: Nobody/Anonymous (nobody)
Summary: long(3.1415) gives zero on Solaris 8

Initial Comment:
On Solaris 8, running Python 2.2.2, the built-in 
function long() gave me the following peculiar results
when applied to a floating-point argument.

(Unfortunately I don't have administrative access to
this machine, so couldn't test to see whether Python
2.3a has the same problems.)

s0100% uname -a
SunOS s0100.math.lsa.umich.edu 5.8 Generic_108528-14
sun4u sparc SUNW,Sun-Blade-1000
s0100% python
Python 2.2.2 (#22, Dec 23 2002, 12:02:55) 
[GCC 3.0.3] on sunos5
Type "help", "copyright", "credits" or "license" for
more information.
>>> long(3.1415)
0L
>>> _ == 0L
0
>>> long(6e20)
00000000000000000L
>>> _ - 1
37778931862957161709567L
>>> int(3.1415)
3


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=679520&group_id=5470


From noreply@sourceforge.net  Mon Feb  3 15:21:55 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Mon, 03 Feb 2003 07:21:55 -0800
Subject: [Python-bugs-list] [ python-Bugs-679251 ] PEP 277 in what's new for 2.3
Message-ID: <E18fiPz-0003SU-00@sc8-sf-web2.sourceforge.net>

Bugs item #679251, was opened at 2003-02-02 18:41
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=679251&group_id=5470

Category: Documentation
Group: Python 2.3
>Status: Closed
>Resolution: Fixed
Priority: 5
Submitted By: Andrew Dalke (dalke)
>Assigned to: A.M. Kuchling (akuchling)
Summary: PEP 277 in what's new for 2.3

Initial Comment:
> Applications can test whether arbitrary Unicode strings
> are supported as file names by checking
>  os.path.unicode_file_names

That should be os.path.supports_unicode_filenames

>>> import os
>>> [s for s in dir(os.path) if "unicode" in s]
['supports_unicode_filenames']
>>>


----------------------------------------------------------------------

>Comment By: A.M. Kuchling (akuchling)
Date: 2003-02-03 10:21

Message:
Logged In: YES 
user_id=11375

Corrected in CVS; thanks!

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=679251&group_id=5470


From noreply@sourceforge.net  Mon Feb  3 15:27:05 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Mon, 03 Feb 2003 07:27:05 -0800
Subject: [Python-bugs-list] [ python-Bugs-676273 ] BaseHTTPServer; HTTP/1.1 spec not "vague"
Message-ID: <E18fiUz-0003eH-00@sc8-sf-web2.sourceforge.net>

Bugs item #676273, was opened at 2003-01-28 14:09
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=676273&group_id=5470

Category: Python Library
Group: Python 2.3
>Status: Closed
>Resolution: Fixed
Priority: 5
Submitted By: Andrew Dalke (dalke)
>Assigned to: A.M. Kuchling (akuchling)
>Summary: BaseHTTPServer; HTTP/1.1 spec not "vague"

Initial Comment:
The module docstring for BaseHTTPServer says

    The protocol is vague about whether lines are
separated by LF
    characters or by CRLF pairs -- for compatibility
with the widest
    range of clients, both should be accepted. 
Similarly, whitespace
    in the request line should be treated sensibly
(allowing multiple
    spaces between components and allowing trailing
whitespace).

HTTP/1.1 is not vague.  It says in section 2.2 of the spec


    HTTP/1.1 defines the sequence CR LF as the
end-of-line marker for
    all protocol elements except the entity-body

but with the suggestion in 19.3 that

    Clients SHOULD be tolerant in parsing the
Status-Line and servers
    tolerant when parsing the Request-Line. In
particular, they SHOULD
    accept any amount of SP or HT characters between
fields, even
    though only a single SP is required.

    The line terminator for message-header fields is
the sequence
    CRLF. However, we recommend that applications, when
parsing such
    headers, recognize a single LF as a line terminator
and ignore the
    leading CR.

This means that 1) the spec is not vague and 2) the Python
code works according to the spec.  So my suggestion is
to remove the comment about being "vague" from the spec.

A diff is attached.


----------------------------------------------------------------------

>Comment By: A.M. Kuchling (akuchling)
Date: 2003-02-03 10:27

Message:
Logged In: YES 
user_id=11375

Checked in to CVS as rev. 1.23; thanks!


----------------------------------------------------------------------

Comment By: Andrew Dalke (dalke)
Date: 2003-01-29 19:24

Message:
Logged In: YES 
user_id=190903

Silly interface

----------------------------------------------------------------------

Comment By: Andrew Dalke (dalke)
Date: 2003-01-29 19:24

Message:
Logged In: YES 
user_id=190903

Silly interface

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-01-29 16:21

Message:
Logged In: YES 
user_id=33168

There's no uploaded file!  You have to check the
checkbox labeled "Check to Upload & Attach File"
when you upload a file.

Please try again.

(This is a SourceForge annoyance that we can do
nothing about. :-( )

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=676273&group_id=5470


From noreply@sourceforge.net  Mon Feb  3 15:42:22 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Mon, 03 Feb 2003 07:42:22 -0800
Subject: [Python-bugs-list] [ python-Bugs-678077 ] add warning to os.getlogin
Message-ID: <E18fijm-0006jC-00@sc8-sf-web1.sourceforge.net>

Bugs item #678077, was opened at 2003-01-31 06:23
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=678077&group_id=5470

Category: Documentation
Group: Python 2.2.2
>Status: Closed
>Resolution: Fixed
Priority: 5
Submitted By: Gernot Hillier (yoda_gh)
>Assigned to: A.M. Kuchling (akuchling)
Summary: add warning to os.getlogin

Initial Comment:
I experienced problems when using os.getlogin(). It doesn't work for 
exmaple after doing a "su". It still reports the old login. 
 
Also "man 3 getlogin" tells me on GNU/Linux (SuSE 8.1): 
 
"Unfortunately,  it  is  often  rather easy to fool getlogin().  
Sometimes it does not work at all, because some program messed up 
the utmp file. Often, it gives only the first 8 characters of the login 
name. The user currently logged in on the controlling tty of our 
program need not be the user  who  started it.  Avoid getlogin() for 
security-related purposes. 
 
Nobody  knows  precisely  what cuserid() does - avoid it in portable 
programs - avoid it altogether - use getpwuid(geteuid()) instead, if 
that is what you meant.  DO NOT USE cuserid()" 
 
It would be nice if you could add some warning to the documentation 
of Python's os.getlogin() telling the user it's better to use 
"pwd.getpwuid(os.getuid())[0]". That's at least what I do now and it 
works as expected. 
 
TIA! 

----------------------------------------------------------------------

>Comment By: A.M. Kuchling (akuchling)
Date: 2003-02-03 10:42

Message:
Logged In: YES 
user_id=11375

Checked in to rev. 1.110 of libos.tex; thanks!

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=678077&group_id=5470


From noreply@sourceforge.net  Mon Feb  3 16:41:09 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Mon, 03 Feb 2003 08:41:09 -0800
Subject: [Python-bugs-list] [ python-Bugs-678375 ] test_long fails
Message-ID: <E18fjef-0001JE-00@sc8-sf-web4.sourceforge.net>

Bugs item #678375, was opened at 2003-01-31 15:44
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=678375&group_id=5470

Category: Python Library
Group: Python 2.3
>Status: Closed
>Resolution: Fixed
Priority: 5
Submitted By: Michael Stone (mbrierst)
>Assigned to: Tim Peters (tim_one)
Summary: test_long fails

Initial Comment:

on linux kernel 2.2.17, libc2.2

this code demonstrates the problem:

s='12345'*1000
float(s)

Traceback (most recent call last):
  File "<stdin>", line 1, in ?
ValueError: invalid literal for float(): 1234512...

caused by PyFloat_FromString in floatobject.c:
Testing shows strtod only reads the first 721 digits of the string it gets (after any leading zeros).  Python then thinks the remaining digits aren't valid because strtod did not process them, even though strtod returned inf as is proper.  I guess this is a strtod problem, maybe it depends on the version of libc?

PyFloat_FromString could deal with this case, but really only correctly if it reimpliments strtod, as there could be an exponent way on the end of the string.  I don't know what the correct solution to this is, but surely something should at least be done about the failed test.

----------------------------------------------------------------------

>Comment By: Tim Peters (tim_one)
Date: 2003-02-03 11:41

Message:
Logged In: YES 
user_id=31435

IMO the platform strtod() doesn't conform to the C standard 
here, and is certainly unreasonable in this case.  You should 
file a bug report against it.  (Wheher or not it returns Inf isn't 
the point here, it's how it sets strtod's **endptr argument -- 
the standard has clear rules about that).

We're not going to write our own strtod(), so Guido changed 
the test to use a 600-character string instead.  That should 
hide the symptom for you.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=678375&group_id=5470


From noreply@sourceforge.net  Mon Feb  3 18:22:57 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Mon, 03 Feb 2003 10:22:57 -0800
Subject: [Python-bugs-list] [ python-Bugs-676292 ] BaseHTTPServer incorrectly parses protocol
Message-ID: <E18flFB-0005TT-00@sc8-sf-web2.sourceforge.net>

Bugs item #676292, was opened at 2003-01-28 14:38
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=676292&group_id=5470

Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Andrew Dalke (dalke)
>Assigned to: A.M. Kuchling (akuchling)
Summary: BaseHTTPServer incorrectly parses protocol

Initial Comment:
The code in
BaseHTTPServer.BaseHTTPRequestHandler.parse_request says

            try:
                version_number =
float(version.split('/', 1)[1])
            except ValueError:
                self.send_error(400, "Bad request
version (%s)" % `version`)
                return False
            if version_number >= 1.1 and
self.protocol_version >= "HTTP/1.1":
                self.close_connection = 0
            if version_number >= 2.0:
                self.send_error(505,
                                "Invalid HTTP Version
(%f)" % version_number)

This does not agree with the HTTP/1.1 spec which states

Note that the major and minor numbers MUST be treated
as separate
integers and that each MAY be incremented higher than a
single
digit. Thus, HTTP/2.4 is a lower version than
HTTP/2.13, which in turn
is lower than HTTP/12.3. Leading zeros MUST be ignored
by recipients
and MUST NOT be sent.

I also noticed there were errors if the version string
was "HTTP/1.2.3" or even simply "BLAH".  I've fixed
them so they don't give tracebacks.

Finally, if there is an error in the parsing, it calls
the 'send_error'
method, which does a check if 'self.command' is a HEAD.
 However,
if there's an error parsing the first line of the HTTP
request the
self.command wasn't yet set, so I forced self.command
to initialized to None before doing anything which can
yield an error.

Patch is attached.



----------------------------------------------------------------------

Comment By: Andrew Dalke (dalke)
Date: 2003-01-29 19:12

Message:
Logged In: YES 
user_id=190903

Silly interface.  Okay, should be attached now.


----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-01-29 16:22

Message:
Logged In: YES 
user_id=33168

There's no uploaded file!  You have to check the
checkbox labeled "Check to Upload & Attach File"
when you upload a file.

Please try again.

(This is a SourceForge annoyance that we can do
nothing about. :-( )

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=676292&group_id=5470


From noreply@sourceforge.net  Mon Feb  3 19:17:33 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Mon, 03 Feb 2003 11:17:33 -0800
Subject: [Python-bugs-list] [ python-Bugs-676292 ] BaseHTTPServer incorrectly parses protocol
Message-ID: <E18fm61-0007tX-00@sc8-sf-web2.sourceforge.net>

Bugs item #676292, was opened at 2003-01-28 14:38
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=676292&group_id=5470

Category: None
Group: None
>Status: Closed
>Resolution: Fixed
Priority: 5
Submitted By: Andrew Dalke (dalke)
Assigned to: A.M. Kuchling (akuchling)
Summary: BaseHTTPServer incorrectly parses protocol

Initial Comment:
The code in
BaseHTTPServer.BaseHTTPRequestHandler.parse_request says

            try:
                version_number =
float(version.split('/', 1)[1])
            except ValueError:
                self.send_error(400, "Bad request
version (%s)" % `version`)
                return False
            if version_number >= 1.1 and
self.protocol_version >= "HTTP/1.1":
                self.close_connection = 0
            if version_number >= 2.0:
                self.send_error(505,
                                "Invalid HTTP Version
(%f)" % version_number)

This does not agree with the HTTP/1.1 spec which states

Note that the major and minor numbers MUST be treated
as separate
integers and that each MAY be incremented higher than a
single
digit. Thus, HTTP/2.4 is a lower version than
HTTP/2.13, which in turn
is lower than HTTP/12.3. Leading zeros MUST be ignored
by recipients
and MUST NOT be sent.

I also noticed there were errors if the version string
was "HTTP/1.2.3" or even simply "BLAH".  I've fixed
them so they don't give tracebacks.

Finally, if there is an error in the parsing, it calls
the 'send_error'
method, which does a check if 'self.command' is a HEAD.
 However,
if there's an error parsing the first line of the HTTP
request the
self.command wasn't yet set, so I forced self.command
to initialized to None before doing anything which can
yield an error.

Patch is attached.



----------------------------------------------------------------------

>Comment By: A.M. Kuchling (akuchling)
Date: 2003-02-03 14:17

Message:
Logged In: YES 
user_id=11375

Checked in as revision 1.24 of BaseHTTPServer.py; thanks!  
(You should probably double-check the CVS version, just to be sure I didn't mess something up.)


----------------------------------------------------------------------

Comment By: Andrew Dalke (dalke)
Date: 2003-01-29 19:12

Message:
Logged In: YES 
user_id=190903

Silly interface.  Okay, should be attached now.


----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-01-29 16:22

Message:
Logged In: YES 
user_id=33168

There's no uploaded file!  You have to check the
checkbox labeled "Check to Upload & Attach File"
when you upload a file.

Please try again.

(This is a SourceForge annoyance that we can do
nothing about. :-( )

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=676292&group_id=5470


From noreply@sourceforge.net  Mon Feb  3 19:23:44 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Mon, 03 Feb 2003 11:23:44 -0800
Subject: [Python-bugs-list] [ python-Bugs-665460 ] Crash in binascii_a2b_uu
Message-ID: <E18fmC0-0008Ej-00@sc8-sf-web2.sourceforge.net>

Bugs item #665460, was opened at 2003-01-10 02:51
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=665460&group_id=5470

Category: Python Library
Group: None
>Status: Closed
Resolution: None
Priority: 5
Submitted By: Michael Scharf (scharf)
Assigned to: Nobody/Anonymous (nobody)
Summary: Crash in binascii_a2b_uu

Initial Comment:
I submitted a patch to patches 
(http://sourceforge.net/tracker/index.php?
func=detail&aid=665458&group_id=5470&atid=305470)

its a similar type of bug as 
http://sourceforge.net/tracker/index.php?
func=detail&aid=665014&group_id=5470&atid=105470

Michael

----------------------------------------------------------------------

>Comment By: Jeremy Hylton (jhylton)
Date: 2003-02-03 19:23

Message:
Logged In: YES 
user_id=31392

There's no need for a bug report if it doesn't have any
information.  The patch should be enough.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=665460&group_id=5470


From noreply@sourceforge.net  Mon Feb  3 19:25:37 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Mon, 03 Feb 2003 11:25:37 -0800
Subject: [Python-bugs-list] [ python-Bugs-661042 ] urllib2 and proxy
Message-ID: <E18fmDp-0008Ip-00@sc8-sf-web2.sourceforge.net>

Bugs item #661042, was opened at 2003-01-02 11:32
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=661042&group_id=5470

>Category: Python Library
Group: Python 2.2.2
Status: Open
Resolution: None
Priority: 5
Submitted By: Jean-Marc Desprez (jmdesprez)
>Assigned to: Jeremy Hylton (jhylton)
Summary: urllib2 and proxy

Initial Comment:
the following code fails to open the url, through a
proxy (it tries to connect directly, as if no proxy was
supplied) 

#! /usr/bin/env python
import urllib2
if __name__ == "__main__":
        h_prox = urllib2.ProxyHandler({'http' :
'http://10.4.4.123:3128'})
        #opner = urllib2.build_opener(h_prox,
urllib2.HTTPHandler)
        opner = urllib2.build_opener(h_prox)
        urllib2.install_opener(opner)
        urllib2.urlopen('http://www.python.org')

but it works if I add ' urllib2.HTTPHandler' in
build_opener call (commented line)

It seems that 'urllib2:build_opener' calls first
'urllib2:OpenerDirector.add_handler' with all default
handlers (including HTTPhandler when not supplied in
'build_opener'), and after the user supplied
handlers,so the proxy handler is linked in second
position in 'OpenerDirector.handle_open['http'],
whereas it should be first in the open chain.

I checked my theory with a quick and dirty hack in
'add_handler', to force instance of ProxyHandler to be
inserted first in the handler_open['http'] list (it
works), but I think a more apropriated fix shoud be
done in 'build_opener'

regards

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=661042&group_id=5470


From noreply@sourceforge.net  Mon Feb  3 19:41:59 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Mon, 03 Feb 2003 11:41:59 -0800
Subject: [Python-bugs-list] [ python-Bugs-679520 ] long(3.1415) gives zero on Solaris 8
Message-ID: <E18fmTf-0002A6-00@sc8-sf-web3.sourceforge.net>

Bugs item #679520, was opened at 2003-02-03 14:22
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=679520&group_id=5470

Category: Python Interpreter Core
Group: Platform-specific
Status: Open
Resolution: None
Priority: 5
Submitted By: Mark Dickinson (marketdickinson)
Assigned to: Nobody/Anonymous (nobody)
Summary: long(3.1415) gives zero on Solaris 8

Initial Comment:
On Solaris 8, running Python 2.2.2, the built-in 
function long() gave me the following peculiar results
when applied to a floating-point argument.

(Unfortunately I don't have administrative access to
this machine, so couldn't test to see whether Python
2.3a has the same problems.)

s0100% uname -a
SunOS s0100.math.lsa.umich.edu 5.8 Generic_108528-14
sun4u sparc SUNW,Sun-Blade-1000
s0100% python
Python 2.2.2 (#22, Dec 23 2002, 12:02:55) 
[GCC 3.0.3] on sunos5
Type "help", "copyright", "credits" or "license" for
more information.
>>> long(3.1415)
0L
>>> _ == 0L
0
>>> long(6e20)
00000000000000000L
>>> _ - 1
37778931862957161709567L
>>> int(3.1415)
3


----------------------------------------------------------------------

Comment By: Inyeol Lee (inyeol)
Date: 2003-02-03 19:41

Message:
Logged In: YES 
user_id=595280

My machine has similar configuration, but works fine;

$ uname -a
SunOS abbey 5.8 Generic_108528-18 sun4u sparc
SUNW,Sun-Blade-1000

$ python22
Python 2.2.2 (#1, Oct 15 2002, 14:20:16)
[GCC 2.95.3 20010315 (release)] on sunos5
>>> long(3.1415)
3L
>>> long(6e20)
600000000000000000000L
>>> _ - 1
599999999999999999999L

I think it's gcc/glibc optimization issue.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=679520&group_id=5470


From noreply@sourceforge.net  Mon Feb  3 19:50:44 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Mon, 03 Feb 2003 11:50:44 -0800
Subject: [Python-bugs-list] [ python-Bugs-662099 ] urllib2 exceptions need improvement
Message-ID: <E18fmc8-0002gs-00@sc8-sf-web4.sourceforge.net>

Bugs item #662099, was opened at 2003-01-04 12:00
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=662099&group_id=5470

Category: Python Library
Group: Python 2.2.2
>Status: Closed
Resolution: None
Priority: 5
Submitted By: Thomas Viehmann (t-v)
Assigned to: Nobody/Anonymous (nobody)
Summary: urllib2 exceptions need improvement

Initial Comment:
Hi.

Thanks for urllib2.
Urllib2 would be even better if Exceptions were more
unified, i.e. if there was way to test any URLException
for common failure causes, regardless of the protocol.
Example: 'File not found' can happen with
http/ftp/gopher/...

Cheers

Thomas

----------------------------------------------------------------------

>Comment By: Jeremy Hylton (jhylton)
Date: 2003-02-03 19:50

Message:
Logged In: YES 
user_id=31392

Thomas, This sounds like a nice feature but it doesn't
really feel like a bug.  I'm also unsure about how many
error cases in other protocols map naturally to HTTP error
numbers.  Feel free to submit a patch if you find that this
really works for any signficant number of cases.


----------------------------------------------------------------------

Comment By: Thomas Viehmann (t-v)
Date: 2003-01-05 11:18

Message:
Logged In: YES 
user_id=680463

It should be the latter, not the letter, in the previous
followup.

----------------------------------------------------------------------

Comment By: Thomas Viehmann (t-v)
Date: 2003-01-05 11:16

Message:
Logged In: YES 
user_id=680463

Maybe I was unclear (and maybe it's possible already to do
what I want but I'm just to blind to see it). So here is
what I want:
Attempt one:
I'd something like a HTTP status code returned for all
protocols, with the library making up the appropriate ones
as necessary, if there is none. (So that I can test a
gopher-URLError for "e.code==404" 
Attempt two:
When trying to retrieve an URL, I'd like to have a method to
detect "Not found", so I would like the following code
fragment, which (modulo mistakes introduced in copying this
from my code) works with http, to work with gopher (someday
ftp or rsync?) as well.

try:
  f = urllib2.open(anurl)
except urllib2.URLError,e:
  if e.code == 404:
     print "Not found"
     ... # e.g. try alternative location
  else:
     print "Something more serious...."
     raise e

Does this make it more clear?
Similar to urllib/urllib2 providing common method to
retrieve data for multiple protocols, there should be way to
be able to identify errors that are not protocol-specific.
(And the letter I did not find.)

Thanks

Thomas


----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-01-05 00:32

Message:
Logged In: YES 
user_id=33168

Perhaps I'm missing something, but I don't understand what
you want.  All the exceptions in urllib2 derive from
URLError, which itself derives from IOError.  There are a
few cases where IOError is raised.  All other cases raise
URLError, HTTPError, or GopherError.  Isn't IOError (or
possibly URLError) what you are looking for?

So you would do:

  try:
    # your code here
  except IOError:
    # handle any urllib2 exception

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=662099&group_id=5470


From noreply@sourceforge.net  Mon Feb  3 20:27:29 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Mon, 03 Feb 2003 12:27:29 -0800
Subject: [Python-bugs-list] [ python-Bugs-666958 ] repr.repr not always safe
Message-ID: <E18fnBh-0002uw-00@sc8-sf-web2.sourceforge.net>

Bugs item #666958, was opened at 2003-01-12 23:17
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=666958&group_id=5470

Category: Python Library
Group: Python 2.2
Status: Open
Resolution: None
Priority: 5
Submitted By: Bob Ippolito (etrepum)
Assigned to: Nobody/Anonymous (nobody)
Summary: repr.repr not always safe

Initial Comment:
>>> class A:
...   def __repr__(self):
...     raise Exception
...   def someMethod(self):
...     pass
... 
>>> A()
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "<stdin>", line 3, in __repr__
Exception
>>> repr.repr(A())
'<A instance at 237360>'
>>> repr.repr(A().someMethod)
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "/usr/lib/python2.2/repr.py", line 15, in repr
    return self.repr1(x, self.maxlevel)
  File "/usr/lib/python2.2/repr.py", line 24, in repr1
    s = `x`
  File "<stdin>", line

----------------------------------------------------------------------

>Comment By: Bob Ippolito (etrepum)
Date: 2003-02-03 15:27

Message:
Logged In: YES 
user_id=139309

Yes, of course repr.repr is for debugging use.. and it
*really* sucks if your debugger crashes!  I'm sure that's
why it's inconsistent with __builtins__.repr, the bug I'm
reporting is that it's inconsistent with itself.

Going back to the example of a list again.  If you're
debugging a list that has one or two items that raise an
exception, it's infinitely more useful to know *which* items
have a broken __repr__ (along with the context of the valid
objects), rather than knowing *one or more* items in the
list has a broken __repr__.  repr.repr is often called
recursively; if something bad happens far down in the call
stack it can really be a pain to track down where it
happened (all tracebacks to repr are going to look mostly
the same) unless repr.repr is fixed to handle this situation.

----------------------------------------------------------------------

Comment By: Christos Georgiou (tzot)
Date: 2003-01-21 15:58

Message:
Logged In: YES 
user_id=539787

Sorry, incomplete last post, I clicked in the wrong place (ie 
SUBMIT) to come back to IE.

The case is that repr.py is documented as for debugging 
use... you suggest that all exceptions are caught, but that 
would not be consistent with __builtins__.repr .

If the module was intended for generic use, I would suggest 
the opposite; no exceptions should be caught (for 
consistency).

----------------------------------------------------------------------

Comment By: Christos Georgiou (tzot)
Date: 2003-01-21 15:47

Message:
Logged In: YES 
user_id=539787

Do you suggest to remove the exception catching in 
Repr.repr_instance or adding a try clause in the s = `x` line of 
Repr.repr1 ?

----------------------------------------------------------------------

Comment By: Bob Ippolito (etrepum)
Date: 2003-01-20 09:11

Message:
Logged In: YES 
user_id=139309

It's not my module, it comes with Python (at least in my 2.2, and 2.3 CVS HEAD).  Look for yourself!

The issue has nothing to do with the fact that it has the same name as the builtin.

----------------------------------------------------------------------

Comment By: Christos Georgiou (tzot)
Date: 2003-01-20 00:28

Message:
Logged In: YES 
user_id=539787

Did you try to change the module name to, say, repr2 and 
then run it again?  Any specific reason you named the 
module as an internal function?
BTW, what is repr.repr supposed to do?  Please supply the 
repr.py script, along with the full traceback.

----------------------------------------------------------------------

Comment By: Bob Ippolito (etrepum)
Date: 2003-01-13 09:54

Message:
Logged In: YES 
user_id=139309

I say it's a bug because it's inconsistent behavior.  You
can repr the class instance that throws during a __repr__
just fine ('<A instance at 237360>' ), but you can't
__repr__ a bound method of the class instance without
throwing an exception.  In fact, if the __repr__ of the
class instance returns anything but a string, say it returns
None for example, then the exception will bubble all the way
up and you get no useful output from repr.repr.

It's very annoying if you're doing something equivalent to,
but more useful than:
repr.repr([1, 2, 3, 4, A().someMethod, 6, 7, 8, A()])

Note that repr.repr([1, 2, 3, 4, A(), 6, 7, 8]) does not
throw any exceptions.

There should be some sort of graceful way to do it, no?  I
think it ought to handle the bound method broken instance
case just like it handles the broken instance case directly.

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2003-01-12 23:34

Message:
Logged In: YES 
user_id=31435

Why do say this is a bug?  If you raise an exception, yes, 
you get an exception.  Most __repr__ *implementations* 
try hard not to raise any exceptions, but it's never 
guaranteed that they won't.  For example, just about any 
__repr__ implementation may run out of memory while 
trying to build its result string, so MemoryError is almost 
always a possible exception.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=666958&group_id=5470


From noreply@sourceforge.net  Mon Feb  3 20:52:37 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Mon, 03 Feb 2003 12:52:37 -0800
Subject: [Python-bugs-list] [ python-Bugs-640230 ] Discovered typo in zlib test.
Message-ID: <E18fna1-0005PK-00@sc8-sf-web4.sourceforge.net>

Bugs item #640230, was opened at 2002-11-18 13:36
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=640230&group_id=5470

Category: None
Group: None
>Status: Closed
>Resolution: Accepted
Priority: 7
Submitted By: Scott David Daniels (scott_daniels)
Assigned to: Nobody/Anonymous (nobody)
Summary: Discovered typo in zlib test.

Initial Comment:
In test_zlib.py (while chasing what appears to be a
documentation problem), I found a flaw in the code
that tests max_length limitted output from a 
decompression object test.

Starting at line 100, the existing code is:
...
bufs.append(deco.flush())
decomp2 = ''.join(buf)
if decomp2 != buf:
    print "max_length decompressobj failed"
else:
    print "max_length decompressobj succeeded"
...

This test will always succeed (''.join(str) == str).

What seems obviously meant is:
...
bufs.append(deco.flush())
decomp2 = ''.join(bufs)
if decomp2 != buf:
    print "max_length decompressobj failed"
else:
    print "max_length decompressobj succeeded"
...



----------------------------------------------------------------------

>Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-03 15:52

Message:
Logged In: YES 
user_id=6380

Fixed in CVS.

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2003-02-02 22:22

Message:
Logged In: YES 
user_id=31435

It looks like the problem with getting this reviewed is that 
no current developer understands the zlib code or its test.  
I bumped the priority in a sneaky attempt to attract more 
interest <wink>.  *If* nobody else bites soon, and I can 
make time, I'll assign it to myself.

Your efforts are appreciated!  Even if it doesn't seem like it.

----------------------------------------------------------------------

Comment By: Scott David Daniels (scott_daniels)
Date: 2003-01-31 22:33

Message:
Logged In: YES 
user_id=493818

I've attached a PyUnit-style replacement for test_zlib.py that 
tests this condition to Patch #678531,  which provides a
context 
diff updating zlibmodule.c to fix this problem.  Hope this
is enough 
to get the fix reviewed.


----------------------------------------------------------------------

Comment By: Scott David Daniels (scott_daniels)
Date: 2002-12-12 13:32

Message:
Logged In: YES 
user_id=493818

The following replacement for PyZlib_unflush addresses some 
of the problem, but leaves unaddressed the issues of what 
unused_data and unconsumed_tail should be at the end of the 
routine.

static PyObject *
PyZlib_unflush(compobject *self, PyObject *args)
{
    int err, length = DEFAULTALLOC;
    PyObject * retval = NULL;
    unsigned long start_total_out;

    if (!PyArg_ParseTuple(args, "|i:flush", &length))
	return NULL;
    if (!(retval = PyString_FromStringAndSize(NULL, length)))
	return NULL;


    ENTER_ZLIB

    start_total_out = self->zst.total_out;
    self->zst.avail_out = length;
    self->zst.next_out = (Byte *)PyString_AS_STRING(retval);

    Py_BEGIN_ALLOW_THREADS
    err = inflate(&(self->zst), Z_FINISH);
    Py_END_ALLOW_THREADS

    /* while Z_OK and the output buffer is full, there might
be more output,
       so extend the output buffer and try again */
    while ((err == Z_OK || err == Z_BUF_ERROR) &&
self->zst.avail_out == 0) {
	if (_PyString_Resize(&retval, length << 1) < 0)
	    goto error;
	self->zst.next_out = (Byte *)PyString_AS_STRING(retval) +
length;
	self->zst.avail_out = length;
	length = length << 1;

	Py_BEGIN_ALLOW_THREADS
	err = inflate(&(self->zst), Z_FINISH);
	Py_END_ALLOW_THREADS
    }

    /* Maybe _always_ call inflateEnd if clearing
is_initialized */
    if (err == Z_STREAM_END) {
	err = inflateEnd(&(self->zst));
	if (err != Z_OK) {
	    zlib_error(self->zst, err, "from inflateEnd()");
	    Py_DECREF(retval);
	    retval = NULL;
	    goto error;
	}
    }
    self->is_initialised = 0;
    _PyString_Resize(&retval, self->zst.total_out -
start_total_out);
    /* ??? Here fix unused_data and unconsumed_tail */

error:

    LEAVE_ZLIB

    return retval;
}



----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2002-11-19 17:08

Message:
Logged In: YES 
user_id=31435

I'd say it's a bug in flush(), or in the docs, then.  They say "all 
pending input is processed, and a string containing the 
remaining uncompressed output is returned", and that's not 
happening here.  That's pretty clearly what the author of this 
test *expected* to happen, too (the original bug you 
discovered looked much more like a typo than a failure to 
understand the module interface).

----------------------------------------------------------------------

Comment By: Scott David Daniels (scott_daniels)
Date: 2002-11-19 16:40

Message:
Logged In: YES 
user_id=493818

OK, The reason the code fails after the fix is as follows:
deco.flush() does not return any output.
The loop control says, "while we have more unexamined 
source."

However, the decompressor can consume all of the input 
before it has provided all of its output.  So, there are two 
possible fixes:

1) Minimal change:  Change the loop control to say, in 
effect, "While we have more input or are producing output.":
Around line 92:
Change:
    bufs = []
    while cb:
        max_length = 1 + len(cb)/10
To:
    bufs = []
    while cb or chunk:
        max_length = 1 + len(cb)/10


2) More reasonable(?) change:  After the insertion loop, (just 
before the flush) extract all remaining output:
Around line 99:
        cb = deco.unconsumed_tail
    bufs.append(deco.flush())
    decomp2 = ''.join(bufs)
Becomes:
        cb = deco.unconsumed_tail
    while bufs[-1]:
        bufs.append(deco.decompress('', max_length))
    bufs.append(deco.flush())
    decomp2 = ''.join(bufs)


---
Note, in any case, the
    bufs.append(deco.flush())
originally on line 100 _always_ appends a zero-length string.
I would suggest changing it to simply:
    deco.flush()

    

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2002-11-19 04:50

Message:
Logged In: YES 
user_id=6656

Assuming Tim meant what he said, not what he did: raising
priority.

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2002-11-18 15:56

Message:
Logged In: YES 
user_id=31435

Good eye!  Unfortunately, when I repair that, the test fails 
here -- decomp2 is a proper prefix of buf then.  It's "too 
short".  Raised priority but left unassigned (someone who 
understands zlib better will have to jump in).

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=640230&group_id=5470


From noreply@sourceforge.net  Mon Feb  3 20:56:49 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Mon, 03 Feb 2003 12:56:49 -0800
Subject: [Python-bugs-list] [ python-Bugs-662099 ] urllib2 exceptions need improvement
Message-ID: <E18fne5-0006Fs-00@sc8-sf-web1.sourceforge.net>

Bugs item #662099, was opened at 2003-01-04 13:00
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=662099&group_id=5470

Category: Python Library
Group: Python 2.2.2
Status: Closed
Resolution: None
Priority: 5
Submitted By: Thomas Viehmann (t-v)
Assigned to: Nobody/Anonymous (nobody)
Summary: urllib2 exceptions need improvement

Initial Comment:
Hi.

Thanks for urllib2.
Urllib2 would be even better if Exceptions were more
unified, i.e. if there was way to test any URLException
for common failure causes, regardless of the protocol.
Example: 'File not found' can happen with
http/ftp/gopher/...

Cheers

Thomas

----------------------------------------------------------------------

>Comment By: Thomas Viehmann (t-v)
Date: 2003-02-03 21:56

Message:
Logged In: YES 
user_id=680463

Sorry, I thought of "wishlist bug". (And believe there
should be such a thing.)
What would be a chance of a technically sound patch, let's
say one that maps ftp-Errors correctly to start with, to be
accepted?
I should think that rather a lot of protocols should map
rather good, though the error reporting might not be as
granular as with http. Anything from not found to access
denied to timeout is covered. (The mapping doesn't have to
be surjective.)
In particular, since this would be just an addition, it
would certainly not hurt.


----------------------------------------------------------------------

Comment By: Jeremy Hylton (jhylton)
Date: 2003-02-03 20:50

Message:
Logged In: YES 
user_id=31392

Thomas, This sounds like a nice feature but it doesn't
really feel like a bug.  I'm also unsure about how many
error cases in other protocols map naturally to HTTP error
numbers.  Feel free to submit a patch if you find that this
really works for any signficant number of cases.


----------------------------------------------------------------------

Comment By: Thomas Viehmann (t-v)
Date: 2003-01-05 12:18

Message:
Logged In: YES 
user_id=680463

It should be the latter, not the letter, in the previous
followup.

----------------------------------------------------------------------

Comment By: Thomas Viehmann (t-v)
Date: 2003-01-05 12:16

Message:
Logged In: YES 
user_id=680463

Maybe I was unclear (and maybe it's possible already to do
what I want but I'm just to blind to see it). So here is
what I want:
Attempt one:
I'd something like a HTTP status code returned for all
protocols, with the library making up the appropriate ones
as necessary, if there is none. (So that I can test a
gopher-URLError for "e.code==404" 
Attempt two:
When trying to retrieve an URL, I'd like to have a method to
detect "Not found", so I would like the following code
fragment, which (modulo mistakes introduced in copying this
from my code) works with http, to work with gopher (someday
ftp or rsync?) as well.

try:
  f = urllib2.open(anurl)
except urllib2.URLError,e:
  if e.code == 404:
     print "Not found"
     ... # e.g. try alternative location
  else:
     print "Something more serious...."
     raise e

Does this make it more clear?
Similar to urllib/urllib2 providing common method to
retrieve data for multiple protocols, there should be way to
be able to identify errors that are not protocol-specific.
(And the letter I did not find.)

Thanks

Thomas


----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-01-05 01:32

Message:
Logged In: YES 
user_id=33168

Perhaps I'm missing something, but I don't understand what
you want.  All the exceptions in urllib2 derive from
URLError, which itself derives from IOError.  There are a
few cases where IOError is raised.  All other cases raise
URLError, HTTPError, or GopherError.  Isn't IOError (or
possibly URLError) what you are looking for?

So you would do:

  try:
    # your code here
  except IOError:
    # handle any urllib2 exception

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=662099&group_id=5470


From noreply@sourceforge.net  Mon Feb  3 21:02:35 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Mon, 03 Feb 2003 13:02:35 -0800
Subject: [Python-bugs-list] [ python-Bugs-450225 ] urljoin fails RFC tests
Message-ID: <E18fnjf-0004dr-00@sc8-sf-web2.sourceforge.net>

Bugs item #450225, was opened at 2001-08-12 05:10
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=450225&group_id=5470

Category: Python Library
Group: Python 2.1.1
Status: Open
Resolution: None
Priority: 5
Submitted By: Aaron Swartz (aaronsw)
Assigned to: Fred L. Drake, Jr. (fdrake)
Summary: urljoin fails RFC tests

Initial Comment:
I've put together a test suite for Python's URLparse 
module, based on the tests in Appendix C of 
RFC2396 (the URI RFC). They're available at:

http://lists.w3.org/Archives/Public/uri/2001Aug/
0013.html

The major problem seems to be that it treats 
queries and parameters as special components 
(not just normal parts of the path), making this 
related to:

http://sourceforge.net/tracker/?group_id=5470&
atid=105470&func=detail&aid=210834

----------------------------------------------------------------------

Comment By: Michael Stone (mbrierst)
Date: 2003-02-03 21:02

Message:
Logged In: YES 
user_id=670441

The two failing tests could not pass because RFC 1808 and RFC 2396 seem to conflict when a relative URI is given as just ;y or just ?y.

RFC 2396 claims to update RFC 1808, so presumably it describes the correct behavior.  The patch in this message (I can't upload it on sourceforge here for some reason) brings urljoin's behavior in line with RFC 2396, and changes the appropriate test cases.  I think if you apply this patch this bug can be closed.  Let me know what you think


Index: python/dist/src/Lib/urlparse.py
===================================================================
RCS file: /cvsroot/python/python/dist/src/Lib/urlparse.py,v
retrieving revision 1.39
diff -c -r1.39 urlparse.py
*** python/dist/src/Lib/urlparse.py	7 Jan 2003 02:09:16 -0000	1.39
--- python/dist/src/Lib/urlparse.py	3 Feb 2003 20:51:08 -0000
***************
*** 157,169 ****
      if path[:1] == '/':
          return urlunparse((scheme, netloc, path,
                             params, query, fragment))
!     if not path:
!         if not params:
!             params = bparams
!             if not query:
!                 query = bquery
          return urlunparse((scheme, netloc, bpath,
!                            params, query, fragment))
      segments = bpath.split('/')[:-1] + path.split('/')
      # XXX The stuff below is bogus in various ways...
      if segments[-1] == '.':
--- 157,165 ----
      if path[:1] == '/':
          return urlunparse((scheme, netloc, path,
                             params, query, fragment))
!     if not (path or params or query):
          return urlunparse((scheme, netloc, bpath,
!                            bparams, bquery, fragment))
      segments = bpath.split('/')[:-1] + path.split('/')
      # XXX The stuff below is bogus in various ways...
      if segments[-1] == '.':
Index: python/dist/src/Lib/test/test_urlparse.py
===================================================================
RCS file: /cvsroot/python/python/dist/src/Lib/test/test_urlparse.py,v
retrieving revision 1.11
diff -c -r1.11 test_urlparse.py
*** python/dist/src/Lib/test/test_urlparse.py	6 Jan 2003 20:27:03 -0000	1.11
--- python/dist/src/Lib/test/test_urlparse.py	3 Feb 2003 20:51:12 -0000
***************
*** 54,59 ****
--- 54,63 ----
              self.assertEqual(urlparse.urlunparse(urlparse.urlparse(u)), u)
  
      def test_RFC1808(self):
+         # updated by RFC 2396
+ #        self.checkJoin(RFC1808_BASE, '?y', 'http://a/b/c/d;p?y')
+ #        self.checkJoin(RFC1808_BASE, ';x', 'http://a/b/c/d;x')
+ 
          # "normal" cases from RFC 1808:
          self.checkJoin(RFC1808_BASE, 'g:h', 'g:h')
          self.checkJoin(RFC1808_BASE, 'g', 'http://a/b/c/g')
***************
*** 61,74 ****
          self.checkJoin(RFC1808_BASE, 'g/', 'http://a/b/c/g/')
          self.checkJoin(RFC1808_BASE, '/g', 'http://a/g')
          self.checkJoin(RFC1808_BASE, '//g', 'http://g')
-         self.checkJoin(RFC1808_BASE, '?y', 'http://a/b/c/d;p?y')
          self.checkJoin(RFC1808_BASE, 'g?y', 'http://a/b/c/g?y')
          self.checkJoin(RFC1808_BASE, 'g?y/./x', 'http://a/b/c/g?y/./x')
          self.checkJoin(RFC1808_BASE, '#s', 'http://a/b/c/d;p?q#s')
          self.checkJoin(RFC1808_BASE, 'g#s', 'http://a/b/c/g#s')
          self.checkJoin(RFC1808_BASE, 'g#s/./x', 'http://a/b/c/g#s/./x')
          self.checkJoin(RFC1808_BASE, 'g?y#s', 'http://a/b/c/g?y#s')
-         self.checkJoin(RFC1808_BASE, ';x', 'http://a/b/c/d;x')
          self.checkJoin(RFC1808_BASE, 'g;x', 'http://a/b/c/g;x')
          self.checkJoin(RFC1808_BASE, 'g;x?y#s', 'http://a/b/c/g;x?y#s')
          self.checkJoin(RFC1808_BASE, '.', 'http://a/b/c/')
--- 65,76 ----
***************
*** 103,111 ****
      def test_RFC2396(self):
          # cases from RFC 2396
  
!         ### urlparse.py as of v 1.32 fails on these two
!         #self.checkJoin(RFC2396_BASE, '?y', 'http://a/b/c/?y')
!         #self.checkJoin(RFC2396_BASE, ';x', 'http://a/b/c/;x')
  
          self.checkJoin(RFC2396_BASE, 'g:h', 'g:h')
          self.checkJoin(RFC2396_BASE, 'g', 'http://a/b/c/g')
--- 105,113 ----
      def test_RFC2396(self):
          # cases from RFC 2396
  
!         # conflict with RFC 1808, tests commented out there
!         self.checkJoin(RFC2396_BASE, '?y', 'http://a/b/c/?y')
!         self.checkJoin(RFC2396_BASE, ';x', 'http://a/b/c/;x')
  
          self.checkJoin(RFC2396_BASE, 'g:h', 'g:h')
          self.checkJoin(RFC2396_BASE, 'g', 'http://a/b/c/g')


----------------------------------------------------------------------

Comment By: Skip Montanaro (montanaro)
Date: 2002-03-23 05:34

Message:
Logged In: YES 
user_id=44345

added Aaron's RFC 2396 tests to test_urlparse.py
version 1.4 - the two failing tests are commented out



----------------------------------------------------------------------

Comment By: Jon Ribbens (jribbens)
Date: 2002-03-18 14:22

Message:
Logged In: YES 
user_id=76089

I think it would be better btw if '..' components taking 
you 'off the top' were stripped. RFC 2396 says this is 
valid behaviour, and it's what 'real' browsers do.

i.e.
  http://a/b/ + ../../../d == http://a/d


----------------------------------------------------------------------

Comment By: Aaron Swartz (aaronsw)
Date: 2001-11-05 18:34

Message:
Logged In: YES 
user_id=122141

Oops, meant to attach it...

----------------------------------------------------------------------

Comment By: Aaron Swartz (aaronsw)
Date: 2001-11-05 18:30

Message:
Logged In: YES 
user_id=122141

Sure, here they are:



import urlparse

base = 'http://a/b/c/d;p?q'

assert urlparse.urljoin(base, 'g:h') == 'g:h'
assert urlparse.urljoin(base, 'g') ==   'http://a/b/c/g'
assert urlparse.urljoin(base, './g') == 'http://a/b/c/g'
assert urlparse.urljoin(base, 'g/') ==  'http://a/b/c/g/'
assert urlparse.urljoin(base, '/g') ==  'http://a/g'
assert urlparse.urljoin(base, '//g') == 'http://g'
assert urlparse.urljoin(base, '?y') ==  'http://a/b/c/?y'
assert urlparse.urljoin(base, 'g?y') == 'http://a/b/c/g?y'
assert urlparse.urljoin(base, '#s') ==  'http://a/b/c/
d;p?q#s'
assert urlparse.urljoin(base, 'g#s') == 'http://a/b/c/g#s'
assert urlparse.urljoin(base, 'g?y#s') == 'http://a/b/c/
g?y#s'
assert urlparse.urljoin(base, ';x') == 'http://a/b/c/;x'
assert urlparse.urljoin(base, 'g;x') ==  'http://a/b/c/g;x'
assert urlparse.urljoin(base, 'g;x?y#s') == 'http://a/b/c/
g;x?y#s'
assert urlparse.urljoin(base, '.') ==  'http://a/b/c/'
assert urlparse.urljoin(base, './') ==  'http://a/b/c/'
assert urlparse.urljoin(base, '..') ==  'http://a/b/'
assert urlparse.urljoin(base, '../') ==  'http://a/b/'
assert urlparse.urljoin(base, '../g') ==  'http://a/b/g'
assert urlparse.urljoin(base, '../..') ==  'http://a/'
assert urlparse.urljoin(base, '../../') ==  'http://a/'
assert urlparse.urljoin(base, '../../g') ==  'http://a/g'

assert urlparse.urljoin(base, '') == base

assert urlparse.urljoin(base, '../../../g')    ==  'http://a/../g'
assert urlparse.urljoin(base, '../../../../g') ==  'http://a/../../g'

assert urlparse.urljoin(base, '/./g') ==  'http://a/./g'
assert urlparse.urljoin(base, '/../g')         ==  'http://a/../g'
assert urlparse.urljoin(base, 'g.')            ==  'http://a/b/c/
g.'
assert urlparse.urljoin(base, '.g')            ==  'http://a/b/c/
.g'
assert urlparse.urljoin(base, 'g..')           == 'http://a/b/c/
g..'
assert urlparse.urljoin(base, '..g')           == 'http://a/b/c/
..g'

assert urlparse.urljoin(base, './../g')        ==  'http://a/b/g'
assert urlparse.urljoin(base, './g/.')         ==  'http://a/b/c/
g/'
assert urlparse.urljoin(base, 'g/./h')         ==  'http://a/b/c/
g/h'
assert urlparse.urljoin(base, 'g/../h')        ==  'http://a/b/c/
h'
assert urlparse.urljoin(base, 'g;x=1/./y')     ==  
'http://a/b/c/g;x=1/y'
assert urlparse.urljoin(base, 'g;x=1/../y')    ==  'http://a/b/
c/y'

assert urlparse.urljoin(base, 'g?y/./x')       ==  
'http://a/b/c/g?y/./x'
assert urlparse.urljoin(base, 'g?y/../x')      == 
'http://a/b/c/g?y/../x'
assert urlparse.urljoin(base, 'g#s/./x')       ==  'http://a/b/
c/g#s/./x'
assert urlparse.urljoin(base, 'g#s/../x')      ==  'http://a/b/
c/g#s/../x'



----------------------------------------------------------------------

Comment By: Fred L. Drake, Jr. (fdrake)
Date: 2001-11-05 18:05

Message:
Logged In: YES 
user_id=3066

This looks like its probably related to #478038; I'll try to
tackle them together.  Can you attach your tests to the bug
report on SF?  Thanks!

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=450225&group_id=5470


From noreply@sourceforge.net  Mon Feb  3 21:33:29 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Mon, 03 Feb 2003 13:33:29 -0800
Subject: [Python-bugs-list] [ python-Bugs-426866 ] urllib and socket fail with MS proxy
Message-ID: <E18foDZ-0006Ju-00@sc8-sf-web2.sourceforge.net>

Bugs item #426866, was opened at 2001-05-24 02:27
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=426866&group_id=5470

Category: Python Library
Group: Platform-specific
>Status: Closed
>Resolution: Wont Fix
Priority: 5
Submitted By: Nobody/Anonymous (nobody)
Assigned to: Nobody/Anonymous (nobody)
Summary: urllib and socket fail with MS proxy 

Initial Comment:
Both urllib and socket fail to work with MS proxy with 
winsock client install.

urllib's proxy detection will correctly detect the 
existing of the proxy server but fail to work with it. 
It will return an empty page when an read() is issued.

socket will report 1006 winsock error.

----------------------------------------------------------------------

>Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-03 16:33

Message:
Logged In: YES 
user_id=6380

That's a good hint, but nobody here has access to MS
Exchange server to test this, nor do we have time to
reverse-engineer distributed.net code.

So I'm closing this bug as WON'T FIX -- the OP has had half
a year to respond.

If you have a patch, please upload to a new patch tracker issue.

----------------------------------------------------------------------

Comment By: Mike Looijmans (milosoftware)
Date: 2003-01-29 05:23

Message:
Logged In: YES 
user_id=698929

Maybe this hint helps:

The MS proxy apparently uses NT's built-in authorization,
not really the 'basic' authentication as in RFC 2068. The
distributed.net RC5 client suffered from the same problem,
but they managed to work around it. You may find the
solution there.

The MS Proxy is included with MS Exchange server.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2002-03-31 18:44

Message:
Logged In: YES 
user_id=6380

Can you see if the patch from SF bug 503031
(http://www.python.org/sf/503031) solves your problem?

----------------------------------------------------------------------

Comment By: Skip Montanaro (montanaro)
Date: 2002-03-09 20:18

Message:
Logged In: YES 
user_id=44345

Jeremy,

I assigned this to you simply because you commented on 
it.
It was submitted anonymously and the OP never responded 
to
your comment.  I think it should be marked "invalid" and
closed.  Whaddya think?

Skip


----------------------------------------------------------------------

Comment By: Jeremy Hylton (jhylton)
Date: 2001-07-06 12:16

Message:
Logged In: YES 
user_id=31392

Can you provide more details?  What is MS proxy?  What URLs 
have you tried and what exactly is the response -- headers 
would help?  Under what conditions does socket raise a 
winsock error?

Do socket and urllib work on your platform without the 
proxy?


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=426866&group_id=5470


From noreply@sourceforge.net  Mon Feb  3 21:36:10 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Mon, 03 Feb 2003 13:36:10 -0800
Subject: [Python-bugs-list] [ python-Bugs-679827 ] Header loses lines, formats strangely
Message-ID: <E18foGA-0006O9-00@sc8-sf-web2.sourceforge.net>

Bugs item #679827, was opened at 2003-02-03 21:36
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=679827&group_id=5470

Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Michael Stone (mbrierst)
Assigned to: Nobody/Anonymous (nobody)
Summary: Header loses lines, formats strangely

Initial Comment:

Biggest problem:
print email.message_from_string("""header: problems
\tthis line is lost
\tthis is not""")

This was caused by revision 1.11 of Header.py which was in response to bug #601392.  The bugs notes show that when this change was checked in, side effects were worried about, and this is it.

Besides this problem, Header does strange and inconsistent formatting of header strings when breaking lines.  Tabs are not consistently taken to be 8 spaces when splitting lines (sometimes they are, sometimes not), and a different module (quopriMIME) handles UN-breaking lines (in the _max_append routine) causing strange formatting.

quopriMIME doesn't deal with the max line length quite correctly and in header_encode ignores leading spaces when calculating line length (maybe this is on purpose?)

All of these issues probably relate somewhat to the discussion in bug #594893 about how to deal with header lines.  Some of this may be related to bug #675420 as well.

I have worked a bit on a patch, and could finish it if someone wants to see it, but I would have to ask some questions about how lines should be broken and unbroken (how to deal with whitespace and such) first.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=679827&group_id=5470


From noreply@sourceforge.net  Mon Feb  3 22:06:15 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Mon, 03 Feb 2003 14:06:15 -0800
Subject: [Python-bugs-list] [ python-Bugs-679827 ] Header loses lines, formats strangely
Message-ID: <E18fojH-00088U-00@sc8-sf-web2.sourceforge.net>

Bugs item #679827, was opened at 2003-02-03 16:36
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=679827&group_id=5470

Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Michael Stone (mbrierst)
>Assigned to: Barry A. Warsaw (bwarsaw)
Summary: Header loses lines, formats strangely

Initial Comment:

Biggest problem:
print email.message_from_string("""header: problems
\tthis line is lost
\tthis is not""")

This was caused by revision 1.11 of Header.py which was in response to bug #601392.  The bugs notes show that when this change was checked in, side effects were worried about, and this is it.

Besides this problem, Header does strange and inconsistent formatting of header strings when breaking lines.  Tabs are not consistently taken to be 8 spaces when splitting lines (sometimes they are, sometimes not), and a different module (quopriMIME) handles UN-breaking lines (in the _max_append routine) causing strange formatting.

quopriMIME doesn't deal with the max line length quite correctly and in header_encode ignores leading spaces when calculating line length (maybe this is on purpose?)

All of these issues probably relate somewhat to the discussion in bug #594893 about how to deal with header lines.  Some of this may be related to bug #675420 as well.

I have worked a bit on a patch, and could finish it if someone wants to see it, but I would have to ask some questions about how lines should be broken and unbroken (how to deal with whitespace and such) first.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=679827&group_id=5470


From noreply@sourceforge.net  Mon Feb  3 22:15:47 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Mon, 03 Feb 2003 14:15:47 -0800
Subject: [Python-bugs-list] [ python-Bugs-660144 ] typeobject provides incorrect __mul__
Message-ID: <E18fosV-0000LY-00@sc8-sf-web2.sourceforge.net>

Bugs item #660144, was opened at 2002-12-30 15:29
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=660144&group_id=5470

Category: Python Interpreter Core
Group: None
Status: Open
Resolution: Accepted
Priority: 5
Submitted By: Neil Schemenauer (nascheme)
>Assigned to: Neil Schemenauer (nascheme)
Summary: typeobject provides incorrect __mul__

Initial Comment:
type __mul__ is wierd:

>> 'a'.__mul__(3.4)
'aaa'
>>> [1].__mul__(3.4)
[1, 1, 1]

Floating point numbers with fractions should not be
accepted.
I think the problem is that __mul__ should not be trying to
implement sq_repeat behavior (although I haven't dug
deeply into this problem yet).  Also, I think the code is
vulnerable to integer overflow.

Should also check __imul__ __add__ __iadd__.

----------------------------------------------------------------------

>Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-03 17:15

Message:
Logged In: YES 
user_id=6380

Loosk good -- please check it in. Probably requires
something in NEWS too.


----------------------------------------------------------------------

Comment By: Neil Schemenauer (nascheme)
Date: 2003-01-27 00:57

Message:
Logged In: YES 
user_id=35752

Attached is a patch that signals a DepreciationWarning
instead of rasing a TypeError.  I used a static function to
avoid duplication of code.  My version of GCC inlines it.

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-01-26 16:58

Message:
Logged In: YES 
user_id=45365

Neil,
your mod to PyArg_Parse is far too aggressive. Apparently the integer formats have accepted floating point arguments since day one, and all weekend I'm running across places where floats are passed to integers. For example, the MacPython IDE does this all the time, and I wouldn't be surprised if there's lots of windowing code that does this (compute pixel positions in floating point and then passing them to something expecting an integer).

If this is fixed in this way (disallowing floats where integers are expected) it should at least be done with a DeprecationWarning for one release cycle. I think the fix probably is a good idea in the long run, but it will break large bodies of code...

----------------------------------------------------------------------

Comment By: Neil Schemenauer (nascheme)
Date: 2003-01-24 17:20

Message:
Logged In: YES 
user_id=35752

I added the check for the other integer codes.  All the
tests pass
so I checked it in.  Hope that's okay.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-01-21 14:55

Message:
Logged In: YES 
user_id=6380

Looks like a good patch. And yes, the other types should be
doing this as well.

----------------------------------------------------------------------

Comment By: Neil Schemenauer (nascheme)
Date: 2003-01-19 14:35

Message:
Logged In: YES 
user_id=35752

The small patch is attached.  I only added the checking for
'i' and 'l'.  I wonder if it should be done for the other
integer codes as well (e.g. 'b', 'h').

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-01-16 19:19

Message:
Logged In: YES 
user_id=6380

Yes, this should go into 2.3a2.

Your attachment upload failed somehow.

----------------------------------------------------------------------

Comment By: Neil Schemenauer (nascheme)
Date: 2003-01-16 18:10

Message:
Logged In: YES 
user_id=35752

Would this be a change for the 2.3 release?  I tried adding
a PyFloat_Check
test to 'i' and 'l' in getargs.c.  It looks like all the
unit tests pass.  I agree that
checking for float catches the important cases.



----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-01-16 16:57

Message:
Logged In: YES 
user_id=6380

This is an age-old problem that crops up whenever "i" and
similar format codes are used.

The problem is that on the one hand you want "i" to accept
other int-ish types that have an __int__ method. But on the
other hand you don't want it to accept float, which also has
an __int__ method. Or other float-ish types.

I think the "i" format code has to be fixe, but I'm not sure
how -- maybe as a start it would be sufficient to test
explicitly for float and its subclasses and reject those.
That would still allow 3rd party float-ish classes that
implement __int__, but that's not so important.

----------------------------------------------------------------------

Comment By: Neil Schemenauer (nascheme)
Date: 2003-01-16 16:33

Message:
Logged In: YES 
user_id=35752

I think the problem is that wrap_intargfunc and
wrap_intintargfunc use PyArg_ParseTuple(args, "i", &i). 
This bug also is present in methods like __getitem__:

>>> "Python"[1.2]
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
TypeError: sequence index must be integer
>>> "Python".__getitem__(1.2)
'y'

I think the right fix is to use explictly only allow only
ints and longs to wrap_intargfunc and friends.  If Guido
agrees I will cook up a patch.  It seems like we should have
a code for PyParse_Tuple that only allows ints and longs.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=660144&group_id=5470


From noreply@sourceforge.net  Mon Feb  3 22:36:27 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Mon, 03 Feb 2003 14:36:27 -0800
Subject: [Python-bugs-list] [ python-Bugs-665835 ] filter() treatment of str and tuple inconsistent
Message-ID: <E18fpCV-000384-00@sc8-sf-web1.sourceforge.net>

Bugs item #665835, was opened at 2003-01-10 11:36
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=665835&group_id=5470

Category: Python Interpreter Core
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Walter Dörwald (doerwalter)
Assigned to: Guido van Rossum (gvanrossum)
Summary: filter() treatment of str and tuple inconsistent

Initial Comment:
class tuple2(tuple):
·  def __getitem__(self, index):
·  ·  return 2*tuple.__getitem__(self, index)

class str2(str):
·  def __getitem__(self, index):
·  ·  return chr(ord(str.__getitem__(self, index))+1)

print filter(lambda x: x>1, tuple2((1, 2)))
print filter(lambda x: x>"a", str2("ab"))

this prints:
(2,)
bc

i.e. the overwritten __getitem__ is ignored in the
first case, but honored in the second.


----------------------------------------------------------------------

>Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-03 17:36

Message:
Logged In: YES 
user_id=6380

Walter: if you can fix the bug in your latest message here,
go ahead and check it in. Seems like a case of a missing test.

Raymond: it turns out that the iterator in Python 2.2 has
the same problem with lists -- it special-cases lists. But
for tuples, the iterator uses PySequence_GetItem; the fast
tuple iterator in Python 2.3 introduces the problem for
tuples though.

I actually don't think there would be much disagreement that
this behavior (ignoring __getitem__) is a bug. There may be
disagreement over how important it is to fix it. Personally,
I've generally been on the side of "it needn't be fixed if
it slows down the common case", as long as a workaround
(like overriding __iter__ alongside the __getitem__
override) exists.

But I draw the line at being backwards incompatible with
Python 2.2. There fore I think the tuple iterator (and
probably also the string iterator) needs to be fixed, and I
still think that it would be best if the list iterator were
also fixed. One way to do this would be for the tp_iter
implementation to check whether
self->ob_type->tp_as_sequence->sq_item is not equal to the
list_item function (this is a good check to detect a
__getitem__ override) and then return an instance of the
generic sequence iterator instead of the list-specific iterator.

----------------------------------------------------------------------

Comment By: Walter Dörwald (doerwalter)
Date: 2003-01-27 07:24

Message:
Logged In: YES 
user_id=89016

Another problem with filter() is that filterstring() (and
the new filterunicode()) blindly assume that
tp_as_sequence->sq_item returns a str or unicode object with
len==1. This might fail with str or unicode subclasses:
----
class badstr(str):
   def __getitem__(self, index):
      return 42

s = filter(lambda x: x>=42, badstr("1234"))
print len(s), repr(s)
----
This prints
4 '\x00\x00\x00\x00'

----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2003-01-26 20:13

Message:
Logged In: YES 
user_id=80475

One other thought:  A major reason for implementing 
__iter__ in the first place is that objects were overriding 
__getitem__ and disregarding the index -- the __getitem__ 
interface just didn't make sense for iteration in some 
situations.  __iter__ was supposed to provide enormous 
flexibility in various ways to loop over a collection (inorder, 
preorder, postorder, priorityorder, sortedorder, hashorder, 
randomorder, etc).  Making iter() default to using 
__getitem__ was only supposed to be an expedient for 
backwards compatability.  Always using __getitem__ 
diminishes the flexibility and speed advantages.

Maybe the discussion belongs on python-dev. I'm sure a 
number of people feel strongly one way or the other.  The 
question might as well be addressed head-on before 2.3 
goes out the door. 

----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2003-01-26 19:54

Message:
Logged In: YES 
user_id=80475

I understand.  Ideally, *all* methods would respond to a 
single overridden method, but I think this is just a fact of 
life in object oriented programming.

I can't remember where you gave an example of a 
d.__getitem__() subclass override, but you were careful to 
point out that other methods, like d.get() also needed to 
be overridden so that the modified access applied 
everywhere.  Likewise, __iter__() or any other object 
access method must be assumed to access the underlying 
data structure directly and must be overridden.   For 
instance, creating a dictionary with case insensitive 
lookups entails overriding __getitem__(k), get(k,default), 
and pop(k) -- no one of them can be presumed to inform 
the others.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-01-26 19:17

Message:
Logged In: YES 
user_id=6380

Hm... that means that iter() of *amy* built-in type subclass
overriding __getitem__ bypasses the override, unless the
subclass also overrides __iter__. This sounds like a step in
the wrong direction. I think the built-in iterators should
be aware of subclasses overriding __getitem__ one way or
another. I hadn't realized this when we started the trend of
creating faster iterators for built-in types. :-(

----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2003-01-25 11:45

Message:
Logged In: YES 
user_id=80475

None of the existing iterators (incl dicts, lists, tuples, and 
files) use __getitem__.  Most likely, user defined iterators 
also access the data structure directly (for flexiblity and 
speed). Also, anything that uses PyTuple_GET_ITEM 
bypasses __getitem__.

If string/unicode iterators are added, they should also go
directly to the underlying data; otherwise, there is no point 
to it.

Also, the proposal to change filtertuple(), doesn't solve
inconsistencies within filterstring() which uses __getitem__ 
when there is a function call, but bypasses it when the 
function parameter is Py_None.

I think the right answer is to change filterstring() to use an 
iterator and to implement string/unicode iterators that 
access the data directly (not using __getitem__).

FYI for Tim:  MvL noticed and fixed the unicode vs string 
difference.  His patch, SF #636005, has not been applied 
yet.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-01-25 08:51

Message:
Logged In: YES 
user_id=6380

(But in addition th that, I don't mind having a custom
string iterator -- as long as it calls __getitem__ properly.
Hm, shouldn't the tuple iterator call __getitem__ properly too?)


----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2003-01-25 08:45

Message:
Logged In: YES 
user_id=31435

Just noting that filter() is unique in special-casing the type 
of the input.  It's always been surprising that way, and, 
e.g., filtering a string produces a string, but filtering a 
Unicode string produces a list.

map() and reduce() don't play games like that, and always 
use the iteration protocol to march over their inputs.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-01-25 08:36

Message:
Logged In: YES 
user_id=6380

I don't know which Python sources Raymond has been reading,
but in the sources I've got in front of me, there are
special cases for strings and tuples, and these *don't* use
iter(). It so happens that the tuple special-case calls
PyTuple_GetItem(), which doesn't call your __getitem__,
while the string special-case calls the sq_item slot
function, which (in your case) will be a wrapper that calls
your __getitem__.

A minimal fix would be to only call filtertuple for strict
tuples -- although this changes the output type, but I don't
think one should count on filter() of a tuple subclass
returning a tuple (and it can't be made to return an
instance of the subclass either -- we don't know the
constructor signature).

Similar fixes probably need to be made to map() and maybe
reduce().

----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2003-01-24 22:47

Message:
Logged In: YES 
user_id=80475

The problem isn't with filter() which correctly calls iter() in 
both cases.

Tuple object have their own iterator which loops over 
elements directly and has no intervening calls to 
__getitem__().

String objects do not define a custom iterator, so iter() 
wraps itself around consecutive calls to __getitem__().

The resolution is to provide string objects with their own 
iterator. As a side benefit, iteration will run just a tiny bit 
faster.  The same applies to unicode objects.

Guido, do you care about this and want me to fix it or 
would you like to close it as "won't fix".


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=665835&group_id=5470


From noreply@sourceforge.net  Mon Feb  3 22:36:50 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Mon, 03 Feb 2003 14:36:50 -0800
Subject: [Python-bugs-list] [ python-Bugs-665835 ] filter() treatment of str and tuple inconsistent
Message-ID: <E18fpCs-00038M-00@sc8-sf-web1.sourceforge.net>

Bugs item #665835, was opened at 2003-01-10 11:36
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=665835&group_id=5470

Category: Python Interpreter Core
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Walter Dörwald (doerwalter)
>Assigned to: Raymond Hettinger (rhettinger)
Summary: filter() treatment of str and tuple inconsistent

Initial Comment:
class tuple2(tuple):
·  def __getitem__(self, index):
·  ·  return 2*tuple.__getitem__(self, index)

class str2(str):
·  def __getitem__(self, index):
·  ·  return chr(ord(str.__getitem__(self, index))+1)

print filter(lambda x: x>1, tuple2((1, 2)))
print filter(lambda x: x>"a", str2("ab"))

this prints:
(2,)
bc

i.e. the overwritten __getitem__ is ignored in the
first case, but honored in the second.


----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-03 17:36

Message:
Logged In: YES 
user_id=6380

Walter: if you can fix the bug in your latest message here,
go ahead and check it in. Seems like a case of a missing test.

Raymond: it turns out that the iterator in Python 2.2 has
the same problem with lists -- it special-cases lists. But
for tuples, the iterator uses PySequence_GetItem; the fast
tuple iterator in Python 2.3 introduces the problem for
tuples though.

I actually don't think there would be much disagreement that
this behavior (ignoring __getitem__) is a bug. There may be
disagreement over how important it is to fix it. Personally,
I've generally been on the side of "it needn't be fixed if
it slows down the common case", as long as a workaround
(like overriding __iter__ alongside the __getitem__
override) exists.

But I draw the line at being backwards incompatible with
Python 2.2. There fore I think the tuple iterator (and
probably also the string iterator) needs to be fixed, and I
still think that it would be best if the list iterator were
also fixed. One way to do this would be for the tp_iter
implementation to check whether
self->ob_type->tp_as_sequence->sq_item is not equal to the
list_item function (this is a good check to detect a
__getitem__ override) and then return an instance of the
generic sequence iterator instead of the list-specific iterator.

----------------------------------------------------------------------

Comment By: Walter Dörwald (doerwalter)
Date: 2003-01-27 07:24

Message:
Logged In: YES 
user_id=89016

Another problem with filter() is that filterstring() (and
the new filterunicode()) blindly assume that
tp_as_sequence->sq_item returns a str or unicode object with
len==1. This might fail with str or unicode subclasses:
----
class badstr(str):
   def __getitem__(self, index):
      return 42

s = filter(lambda x: x>=42, badstr("1234"))
print len(s), repr(s)
----
This prints
4 '\x00\x00\x00\x00'

----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2003-01-26 20:13

Message:
Logged In: YES 
user_id=80475

One other thought:  A major reason for implementing 
__iter__ in the first place is that objects were overriding 
__getitem__ and disregarding the index -- the __getitem__ 
interface just didn't make sense for iteration in some 
situations.  __iter__ was supposed to provide enormous 
flexibility in various ways to loop over a collection (inorder, 
preorder, postorder, priorityorder, sortedorder, hashorder, 
randomorder, etc).  Making iter() default to using 
__getitem__ was only supposed to be an expedient for 
backwards compatability.  Always using __getitem__ 
diminishes the flexibility and speed advantages.

Maybe the discussion belongs on python-dev. I'm sure a 
number of people feel strongly one way or the other.  The 
question might as well be addressed head-on before 2.3 
goes out the door. 

----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2003-01-26 19:54

Message:
Logged In: YES 
user_id=80475

I understand.  Ideally, *all* methods would respond to a 
single overridden method, but I think this is just a fact of 
life in object oriented programming.

I can't remember where you gave an example of a 
d.__getitem__() subclass override, but you were careful to 
point out that other methods, like d.get() also needed to 
be overridden so that the modified access applied 
everywhere.  Likewise, __iter__() or any other object 
access method must be assumed to access the underlying 
data structure directly and must be overridden.   For 
instance, creating a dictionary with case insensitive 
lookups entails overriding __getitem__(k), get(k,default), 
and pop(k) -- no one of them can be presumed to inform 
the others.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-01-26 19:17

Message:
Logged In: YES 
user_id=6380

Hm... that means that iter() of *amy* built-in type subclass
overriding __getitem__ bypasses the override, unless the
subclass also overrides __iter__. This sounds like a step in
the wrong direction. I think the built-in iterators should
be aware of subclasses overriding __getitem__ one way or
another. I hadn't realized this when we started the trend of
creating faster iterators for built-in types. :-(

----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2003-01-25 11:45

Message:
Logged In: YES 
user_id=80475

None of the existing iterators (incl dicts, lists, tuples, and 
files) use __getitem__.  Most likely, user defined iterators 
also access the data structure directly (for flexiblity and 
speed). Also, anything that uses PyTuple_GET_ITEM 
bypasses __getitem__.

If string/unicode iterators are added, they should also go
directly to the underlying data; otherwise, there is no point 
to it.

Also, the proposal to change filtertuple(), doesn't solve
inconsistencies within filterstring() which uses __getitem__ 
when there is a function call, but bypasses it when the 
function parameter is Py_None.

I think the right answer is to change filterstring() to use an 
iterator and to implement string/unicode iterators that 
access the data directly (not using __getitem__).

FYI for Tim:  MvL noticed and fixed the unicode vs string 
difference.  His patch, SF #636005, has not been applied 
yet.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-01-25 08:51

Message:
Logged In: YES 
user_id=6380

(But in addition th that, I don't mind having a custom
string iterator -- as long as it calls __getitem__ properly.
Hm, shouldn't the tuple iterator call __getitem__ properly too?)


----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2003-01-25 08:45

Message:
Logged In: YES 
user_id=31435

Just noting that filter() is unique in special-casing the type 
of the input.  It's always been surprising that way, and, 
e.g., filtering a string produces a string, but filtering a 
Unicode string produces a list.

map() and reduce() don't play games like that, and always 
use the iteration protocol to march over their inputs.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-01-25 08:36

Message:
Logged In: YES 
user_id=6380

I don't know which Python sources Raymond has been reading,
but in the sources I've got in front of me, there are
special cases for strings and tuples, and these *don't* use
iter(). It so happens that the tuple special-case calls
PyTuple_GetItem(), which doesn't call your __getitem__,
while the string special-case calls the sq_item slot
function, which (in your case) will be a wrapper that calls
your __getitem__.

A minimal fix would be to only call filtertuple for strict
tuples -- although this changes the output type, but I don't
think one should count on filter() of a tuple subclass
returning a tuple (and it can't be made to return an
instance of the subclass either -- we don't know the
constructor signature).

Similar fixes probably need to be made to map() and maybe
reduce().

----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2003-01-24 22:47

Message:
Logged In: YES 
user_id=80475

The problem isn't with filter() which correctly calls iter() in 
both cases.

Tuple object have their own iterator which loops over 
elements directly and has no intervening calls to 
__getitem__().

String objects do not define a custom iterator, so iter() 
wraps itself around consecutive calls to __getitem__().

The resolution is to provide string objects with their own 
iterator. As a side benefit, iteration will run just a tiny bit 
faster.  The same applies to unicode objects.

Guido, do you care about this and want me to fix it or 
would you like to close it as "won't fix".


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=665835&group_id=5470


From noreply@sourceforge.net  Mon Feb  3 22:46:57 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Mon, 03 Feb 2003 14:46:57 -0800
Subject: [Python-bugs-list] [ python-Bugs-679872 ] Documentation error -- time module
Message-ID: <E18fpMf-0003Dj-00@sc8-sf-web3.sourceforge.net>

Bugs item #679872, was opened at 2003-02-03 14:46
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=679872&group_id=5470

Category: Documentation
Group: Python 2.2.2
Status: Open
Resolution: None
Priority: 5
Submitted By: Alfred Morgan (zectbumo)
Assigned to: Nobody/Anonymous (nobody)
Summary: Documentation error -- time module

Initial Comment:
Documentation error
http://www.python.org/doc/current/lib/module-time.html

"earlier versions, up to Python 1.5.1 and 1.5.2a1,
would add 1900 to year values below 1900."


...you mean year values below 100.


-alfred


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=679872&group_id=5470


From noreply@sourceforge.net  Mon Feb  3 23:05:37 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Mon, 03 Feb 2003 15:05:37 -0800
Subject: [Python-bugs-list] [ python-Bugs-679880 ] 'compile' refuses BOM.
Message-ID: <E18fpej-0003mL-00@sc8-sf-web4.sourceforge.net>

Bugs item #679880, was opened at 2003-02-03 23:05
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=679880&group_id=5470

Category: Unicode
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Kirill Simonov (kirill_simonov)
Assigned to: M.-A. Lemburg (lemburg)
Summary: 'compile' refuses BOM.

Initial Comment:
The builtin 'compile' function refuses source files
with UTF-8 BOM marks.

For example,

$ python2.3 testutf8.py

works correctly, but

$ python2.3
>>> compile(open('testutf8.py').read(), 'testutf8.py',
'exec')

fails.



----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=679880&group_id=5470


From noreply@sourceforge.net  Mon Feb  3 23:20:31 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Mon, 03 Feb 2003 15:20:31 -0800
Subject: [Python-bugs-list] [ python-Bugs-514627 ] pydoc fails to generate html doc
Message-ID: <E18fpt9-0003NC-00@sc8-sf-web2.sourceforge.net>

Bugs item #514627, was opened at 2002-02-08 02:04
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=514627&group_id=5470

Category: Demos and Tools
Group: Python 2.2
Status: Open
Resolution: None
Priority: 5
Submitted By: Raj Kunjithapadam (mmaster25)
Assigned to: Nobody/Anonymous (nobody)
Summary: pydoc fails to generate html doc

Initial Comment:
pydoc on the python 2.2 distribution fails to generate
html doc(when option -w is given)
Traceback follows

Traceback (most recent call last):
  File "/opt/dump/Python-2.2/Lib/pydoc.py", line 2101, in ?
    if __name__ == '__main__': cli()
  File "/opt/dump/Python-2.2/Lib/pydoc.py", line 2070,
in cli
    writedoc(arg)
  File "/opt/dump/Python-2.2/Lib/pydoc.py", line 1341,
in writedoc
    object = locate(key, forceload)
  File "/opt/dump/Python-2.2/Lib/pydoc.py", line 1293,
in locate
    parts = split(path, '.')
  File "/opt/dump/Python-2.2/Lib/string.py", line 117,
in split
    return s.split(sep, maxsplit)
AttributeError: 'module' object has no attribute 'split'


On further investigation I was able to fix it.
Attached is the fix.

----------------------------------------------------------------------

Comment By: Michael Stone (mbrierst)
Date: 2003-02-03 23:20

Message:
Logged In: YES 
user_id=670441

Version 1.68 of pydoc.py fixed this in response to bug #586931.
That bug has been closed, someone should close this one too.

----------------------------------------------------------------------

Comment By: A.M. Kuchling (akuchling)
Date: 2002-03-19 16:20

Message:
Logged In: YES 
user_id=11375

Attaching a diff for Raj's modified version of pydoc.py, 
so the changes are clearly visible.




----------------------------------------------------------------------

Comment By: Just van Rossum (jvr)
Date: 2002-03-10 18:04

Message:
Logged In: YES 
user_id=92689

I can reproduce the traceback by going into the Python 
src dir and typing
% pydoc -w Lib/os.py
The problem begins in cli(): writedoc() expects a string 
but arg as just been assigned a module by 
importfile(arg). I don't understand what's supposed to 
happen, but I doubt Raj's patch is the proper solution.
(Not sure how helpful this was, but Skip's cron job is 
pretty cool... ;-)

----------------------------------------------------------------------

Comment By: Raj Kunjithapadam (mmaster25)
Date: 2002-02-10 03:03

Message:
Logged In: YES 
user_id=452533

the commandline was
$pydoc -w <name of my python sourcefile>
I cannot give you the exact commandline and traceback as
I am unable to get thru my VPN now.
But I am pretty sure that was the commandline.

Refer to [ #514628 ] bug in pydoc on python 2.2 release
in Python - Patches.
--Raj


----------------------------------------------------------------------

Comment By: Raj Kunjithapadam (mmaster25)
Date: 2002-02-10 02:54

Message:
Logged In: YES 
user_id=452533

It happened to me on Redhat Linux 7.1
when I ran pydoc using the -w option to generate html output.

-w invokes the writedoc method and it expects an arg(filename)
instead of the module object returned by imp_load.

I have also submitted a fix for this.
Thanks for following up on this quickly.
--Raj


----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2002-02-08 21:48

Message:
Logged In: YES 
user_id=6380

I want to believe you, but I cannot reproduce the traceback.
Can you tell me which command line you used to cause the
traceback, and on which operating system?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=514627&group_id=5470


From noreply@sourceforge.net  Tue Feb  4 01:35:37 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Mon, 03 Feb 2003 17:35:37 -0800
Subject: [Python-bugs-list] [ python-Bugs-514627 ] pydoc fails to generate html doc
Message-ID: <E18frzt-00009r-00@sc8-sf-web2.sourceforge.net>

Bugs item #514627, was opened at 2002-02-07 21:04
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=514627&group_id=5470

Category: Demos and Tools
Group: Python 2.2
>Status: Closed
>Resolution: Duplicate
Priority: 5
Submitted By: Raj Kunjithapadam (mmaster25)
Assigned to: Nobody/Anonymous (nobody)
Summary: pydoc fails to generate html doc

Initial Comment:
pydoc on the python 2.2 distribution fails to generate
html doc(when option -w is given)
Traceback follows

Traceback (most recent call last):
  File "/opt/dump/Python-2.2/Lib/pydoc.py", line 2101, in ?
    if __name__ == '__main__': cli()
  File "/opt/dump/Python-2.2/Lib/pydoc.py", line 2070,
in cli
    writedoc(arg)
  File "/opt/dump/Python-2.2/Lib/pydoc.py", line 1341,
in writedoc
    object = locate(key, forceload)
  File "/opt/dump/Python-2.2/Lib/pydoc.py", line 1293,
in locate
    parts = split(path, '.')
  File "/opt/dump/Python-2.2/Lib/string.py", line 117,
in split
    return s.split(sep, maxsplit)
AttributeError: 'module' object has no attribute 'split'


On further investigation I was able to fix it.
Attached is the fix.

----------------------------------------------------------------------

>Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-03 20:35

Message:
Logged In: YES 
user_id=6380

OK, closing.

----------------------------------------------------------------------

Comment By: Michael Stone (mbrierst)
Date: 2003-02-03 18:20

Message:
Logged In: YES 
user_id=670441

Version 1.68 of pydoc.py fixed this in response to bug #586931.
That bug has been closed, someone should close this one too.

----------------------------------------------------------------------

Comment By: A.M. Kuchling (akuchling)
Date: 2002-03-19 11:20

Message:
Logged In: YES 
user_id=11375

Attaching a diff for Raj's modified version of pydoc.py, 
so the changes are clearly visible.




----------------------------------------------------------------------

Comment By: Just van Rossum (jvr)
Date: 2002-03-10 13:04

Message:
Logged In: YES 
user_id=92689

I can reproduce the traceback by going into the Python 
src dir and typing
% pydoc -w Lib/os.py
The problem begins in cli(): writedoc() expects a string 
but arg as just been assigned a module by 
importfile(arg). I don't understand what's supposed to 
happen, but I doubt Raj's patch is the proper solution.
(Not sure how helpful this was, but Skip's cron job is 
pretty cool... ;-)

----------------------------------------------------------------------

Comment By: Raj Kunjithapadam (mmaster25)
Date: 2002-02-09 22:03

Message:
Logged In: YES 
user_id=452533

the commandline was
$pydoc -w <name of my python sourcefile>
I cannot give you the exact commandline and traceback as
I am unable to get thru my VPN now.
But I am pretty sure that was the commandline.

Refer to [ #514628 ] bug in pydoc on python 2.2 release
in Python - Patches.
--Raj


----------------------------------------------------------------------

Comment By: Raj Kunjithapadam (mmaster25)
Date: 2002-02-09 21:54

Message:
Logged In: YES 
user_id=452533

It happened to me on Redhat Linux 7.1
when I ran pydoc using the -w option to generate html output.

-w invokes the writedoc method and it expects an arg(filename)
instead of the module object returned by imp_load.

I have also submitted a fix for this.
Thanks for following up on this quickly.
--Raj


----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2002-02-08 16:48

Message:
Logged In: YES 
user_id=6380

I want to believe you, but I cannot reproduce the traceback.
Can you tell me which command line you used to cause the
traceback, and on which operating system?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=514627&group_id=5470


From noreply@sourceforge.net  Tue Feb  4 01:54:30 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Mon, 03 Feb 2003 17:54:30 -0800
Subject: [Python-bugs-list] [ python-Bugs-679953 ] zipfile.py - pack filesize as unsigned allows files > 2 gig
Message-ID: <E18fsIA-0003Hs-00@sc8-sf-web1.sourceforge.net>

Bugs item #679953, was opened at 2003-02-03 20:54
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=679953&group_id=5470

Category: Python Library
Group: Python 2.2.2
Status: Open
Resolution: None
Priority: 5
Submitted By: Jimmy Burgett (laurelboa)
Assigned to: Nobody/Anonymous (nobody)
Summary: zipfile.py - pack filesize as unsigned allows files > 2 gig

Initial Comment:
Python 2.2.2 
Windows XP (all serice packs installed)
Windows 2000 (all service packs installed)

The filesize and compressed file size numbers in the zip 
header need to "struct.packed" as unsigned ints, not 
signed ints. This allows zipfile.py to compress files 
greater than 2 gigabytes in size. Currently, an attempt 
to compress such a large file gives you this error:

Traceback (most recent call last):
  File "<interactive input>", line 1, in ?
  File "C:\Python22\lib\zipfile.py", line 426, in write
    zinfo.file_size))
OverflowError: long int too large to convert to int

where the line in question is:
self.fp.write(struct.pack("<lll", zinfo.CRC, 
zinfo.compress_size,
     zinfo.file_size))

I believe that the four changes below are all that is 
needed. This is from version 2.2.2, but zipfile.py in 2.3a1 
still had the file size packed/unpacked as a signed 
integer.

I have not tested whether the ziplib routines can seek 
past the 2 gig boundary in order to extract a file whose 
beginning is past the 2 gig boundary. My application 
requires compressing very large files one at a time and 
zipfile.py lets me use either WinZip or the built-in 
Windows "unzip" function for extraction.
These changes allow that use.

-------------- Change Line #28

# Here are some struct module formats for reading 
headers
structEndArchive = "<4s4H2lH"     # 9 items, end of 
archive, 22 bytes
stringEndArchive = "PK\005\006"   # magic number for 
end of archive record
structCentralDir = "<4s4B4H3l5H2l"# 19 items, central 
directory, 46 bytes

to

structCentralDir = "<4s4B4HlLL5H2L"# 19 items, central 
directory, 46 bytes

--------------- change line #306

    def printdir(self):
        """Print a table of contents for the zip file."""
        print "%-46s %19s %12s" % ("File 
Name", "Modified    ", "Size")
        for zinfo in self.filelist:
            date = "%d-%02d-%02d %02d:%02d:%02d" % 
zinfo.date_time
            print "%-46s %s %12d" % (zinfo.filename, date, 
zinfo.file_size)
to 
            print "%-46s %s %12u" % (zinfo.filename, date, 
zinfo.file_size)


---------------- change line #425

        # Seek backwards and write CRC and file sizes
        position = self.fp.tell()       # Preserve current 
position in file
        self.fp.seek(zinfo.header_offset + 14, 0)
        self.fp.write(struct.pack("<lll", zinfo.CRC, 
zinfo.compress_size,
              zinfo.file_size))
to
        self.fp.write(struct.pack("<lLL", zinfo.CRC, 
zinfo.compress_size,
              zinfo.file_size))

---------------- change line #450
        if zinfo.flag_bits & 0x08:
            # Write CRC and file sizes after the file data
            self.fp.write(struct.pack("<lll", zinfo.CRC, 
zinfo.compress_size,
                  zinfo.file_size))

to
            self.fp.write(struct.pack("<lLL", zinfo.CRC, 
zinfo.compress_size,
                  zinfo.file_size))


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=679953&group_id=5470


From noreply@sourceforge.net  Mon Feb  3 21:43:41 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Mon, 03 Feb 2003 13:43:41 -0800
Subject: [Python-bugs-list] [ python-Bugs-673797 ] setting socket timeout crashes SSL?
Message-ID: <E18foNR-0000M7-00@sc8-sf-web1.sourceforge.net>

Bugs item #673797, was opened at 2003-01-23 22:58
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=673797&group_id=5470

Category: Python Library
Group: Python 2.3
>Status: Closed
>Resolution: Fixed
Priority: 5
Submitted By: Craig Allen (cba037)
Assigned to: Nobody/Anonymous (nobody)
Summary: setting socket timeout crashes SSL?

Initial Comment:
First: bravo for compiling SSL support in the 2.3 binaries!

I can connect fine to a secure web server using HTTPS.
 However, when I set a socket default timeout, I get
errors:

import socket
socket.setdefaulttimeout(30.0)
import urllib
f =
urllib.urlopen('https://members.tufts-health.com/memindex.html')
print f.read()
===================
Traceback (most recent call last):
  File "testssl.py", line 9, in ?
    f =
urllib.urlopen('https://members.tufts-health.com/memindex.html')
  File "/usr/lib/python2.3/urllib.py", line 76, in urlopen
    return opener.open(url)
  File "/usr/lib/python2.3/urllib.py", line 181, in open
    return getattr(self, name)(url)
  File "/usr/lib/python2.3/urllib.py", line 375, in
open_https
    h.endheaders()
  File "/usr/lib/python2.3/httplib.py", line 695, in
endheaders
    self._send_output()
  File "/usr/lib/python2.3/httplib.py", line 581, in
_send_output
    self.send(msg)
  File "/usr/lib/python2.3/httplib.py", line 548, in send
    self.connect()
  File "/usr/lib/python2.3/httplib.py", line 945, in
connect
    ssl = socket.ssl(realsock, self.key_file,
self.cert_file)
socket.sslerror: (2, 'The operation did not complete
(read)')
===================
This is on Linux; similar behaviour on Win2K.


----------------------------------------------------------------------

>Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-03 16:43

Message:
Logged In: YES 
user_id=33168

The patch was checked in, so I'm closing this bug.  Please
re-open if there are any problems.

----------------------------------------------------------------------

Comment By: Geoff Talvola (gtalvola)
Date: 2003-01-31 12:55

Message:
Logged In: YES 
user_id=88162

This is fixed by patch 676472 so when that patch is checked
in, this bug can be closed.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=673797&group_id=5470


From noreply@sourceforge.net  Tue Feb  4 02:33:46 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Mon, 03 Feb 2003 18:33:46 -0800
Subject: [Python-bugs-list] [ python-Bugs-679872 ] Documentation error -- time module
Message-ID: <E18fsuA-0002qi-00@sc8-sf-web3.sourceforge.net>

Bugs item #679872, was opened at 2003-02-03 17:46
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=679872&group_id=5470

Category: Documentation
Group: Python 2.2.2
>Status: Closed
>Resolution: Invalid
Priority: 5
Submitted By: Alfred Morgan (zectbumo)
>Assigned to: Neal Norwitz (nnorwitz)
Summary: Documentation error -- time module

Initial Comment:
Documentation error
http://www.python.org/doc/current/lib/module-time.html

"earlier versions, up to Python 1.5.1 and 1.5.2a1,
would add 1900 to year values below 1900."


...you mean year values below 100.


-alfred


----------------------------------------------------------------------

>Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-03 21:33

Message:
Logged In: YES 
user_id=33168

After looking at the code, I believe the comment is correct!
 It appears that the value 1800 would be treated as the year
3700.  I guess it's good the behaviour changed. :-)

Before the code was (see Modules/timemodule.c revision r151
in the gettmarg() function):

        if (p->tm_year >= 1900)
                p->tm_year -= 1900;

So if the year passed in was 1800, tm_year would remain
1800.  Since tm_year is "The number of years since 1900" the
value would mean 1900 + 1800 == 3700.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=679872&group_id=5470


From noreply@sourceforge.net  Tue Feb  4 09:18:38 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 04 Feb 2003 01:18:38 -0800
Subject: [Python-bugs-list] [ python-Bugs-679880 ] 'compile' refuses BOM.
Message-ID: <E18fzDy-0007cG-00@sc8-sf-web3.sourceforge.net>

Bugs item #679880, was opened at 2003-02-04 00:05
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=679880&group_id=5470

Category: Unicode
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Kirill Simonov (kirill_simonov)
>Assigned to: Martin v. Löwis (loewis)
Summary: 'compile' refuses BOM.

Initial Comment:
The builtin 'compile' function refuses source files
with UTF-8 BOM marks.

For example,

$ python2.3 testutf8.py

works correctly, but

$ python2.3
>>> compile(open('testutf8.py').read(), 'testutf8.py',
'exec')

fails.



----------------------------------------------------------------------

>Comment By: M.-A. Lemburg (lemburg)
Date: 2003-02-04 10:18

Message:
Logged In: YES 
user_id=38388

No time for this. Assigning to Martin. Feel free to pass it on.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=679880&group_id=5470


From noreply@sourceforge.net  Tue Feb  4 14:07:25 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 04 Feb 2003 06:07:25 -0800
Subject: [Python-bugs-list] [ python-Bugs-413144 ] sdist can't create empty directories
Message-ID: <E18g3jR-0005KV-00@sc8-sf-web1.sourceforge.net>

Bugs item #413144, was opened at 2001-04-02 11:05
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=413144&group_id=5470

Category: Distutils
Group: Feature Request
>Status: Closed
>Resolution: Wont Fix
Priority: 5
Submitted By: A.M. Kuchling (akuchling)
Assigned to: Nobody/Anonymous (nobody)
Summary: sdist can't create empty directories

Initial Comment:
If a file fails an os.path.isfile()
check, sdist.py skips it and prints a warning.  That's
probably worth
loosening so an empty directory can be created in a
source distribution by listing it in the MANIFEST.



----------------------------------------------------------------------

>Comment By: A.M. Kuchling (akuchling)
Date: 2003-02-04 09:07

Message:
Logged In: YES 
user_id=11375

Probably not worth fixing.  Closing.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-01-28 21:41

Message:
Logged In: YES 
user_id=33168

amk, should this be fixed?  Is the problem at line 420 in
distutils/command/sdist.py?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=413144&group_id=5470


From noreply@sourceforge.net  Tue Feb  4 15:19:51 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 04 Feb 2003 07:19:51 -0800
Subject: [Python-bugs-list] [ python-Bugs-671731 ] time module: time tuple not returned by certain functions
Message-ID: <E18g4rX-00069Z-00@sc8-sf-web2.sourceforge.net>

Bugs item #671731, was opened at 2003-01-21 06:14
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=671731&group_id=5470

Category: Documentation
Group: Python 2.2.2
>Status: Closed
>Resolution: Fixed
Priority: 3
Submitted By: Martin Miller (mrmiller)
Assigned to: Fred L. Drake, Jr. (fdrake)
Summary: time module: time tuple not returned by certain functions

Initial Comment:
On win32 the gmtime() and localtime() functions in the
time module return values of type 'time.time_struct'. I
believe the proper return value should be a tuple. This
is implied by the documentation (see
<http://www.python.org/doc/current/lib/module-time.html>
where it discusses the *time tuple* returned by the
various module functions and says it 'is a tuple of 9
integers'.

The time_struct value returned does behave like a tuple
instance in most respects, except that its type is not
type 'tuple'.

I think this is something that was changed in the not
too distant past, as I noticed the issue after
downloading code recently written by others which
depends on the type of the return value from these
functions being a tuple. If nothing else, if this was
changed, it is not backward compatible and the
documention ought to be updated.

Example:
> Python 2.2.2 (#37, Oct 14 2002, 17:02:34) [MSC 32 bit
(Intel)] on win32
> Type "help", "copyright", "credits" or "license" for
more information.
> >>> import time
> >>> print time.localtime()
> (2003, 1, 21, 3, 3, 2, 1, 21, 0)
> >>> print type(time.localtime()) is type(())
> 0
> >>> print type(time.localtime())
> <type 'time.struct_time'>
> >>>



----------------------------------------------------------------------

>Comment By: Fred L. Drake, Jr. (fdrake)
Date: 2003-02-04 10:19

Message:
Logged In: YES 
user_id=3066

The previous "fix" was pretty dodgy, I think, since I didn't
have a lot of time for it.  it really only added information
about the attributes, but did not change references to time
tuples throughout.  Perhaps it should have remained open,
pending a better fix.

I've checked in additional changes that provide more
thorough updates.

Doc/lib/libtime.tex 1.55, 1.48.6.4.

----------------------------------------------------------------------

Comment By: Martin Miller (mrmiller)
Date: 2003-01-24 17:23

Message:
Logged In: YES 
user_id=257085

Looks like a duplicate of closed bug 604128
"time.struct_time undocumented" which was assigned and fixed
by fdrake on 2002-11-13. However I've looked at the
Doc/lib/libtime.tex file that included with 2.2.2 and see
nothing written about the type.

I'll try assigning this bug to him since I can't find the fix.


----------------------------------------------------------------------

Comment By: Martin Miller (mrmiller)
Date: 2003-01-24 15:55

Message:
Logged In: YES 
user_id=257085

The strcut_time return value change broke a module I had
downloaded called NormalDate by Jeff Bauer
<jbauer@rubic.com>. Fixing it myself turned out to be
relatively easy, once I determined what was the problem
(which was a little time-consuming, considering the fact
that the docs hadn't been updated). 

Later I located an updated version of module (v1.3) that had
been fixed by the author. The technique he used was to
create a private module attribute set to the type of value
returned from localtime(), so the updated module would still
work with versions of Python < 2.2.

The struct_time pseudo-sequence now returned by the
functions now sounds like a useful improvement, provided
that this fact is documented along with a description of
what the type is, which would allow others to also take
advantage of its attribute fetching behavior if they wished.
This could be done by copying the description in
<http://www.python.org/doc/current/whatsnew/node10.html>
into the developer documentation for the the time module
(and backporting that into the 2.2 documentation.

I'm lowering the priorty to 3 and marking it as a
documentation bug.




----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2003-01-22 05:08

Message:
Logged In: YES 
user_id=6656

Oh yeah.  That should *definitely* be backported to the 2.2
docs. Could be more thorough, though.

----------------------------------------------------------------------

Comment By: Brett Cannon (bcannon)
Date: 2003-01-21 19:55

Message:
Logged In: YES 
user_id=357491

There is a mention of it in the developer docs at http://www.python.org/dev/doc/devel/lib/module-time.html .  The note for struct_time says what functions return a struct_time object.

A patch for the docs that changed everything else to say it returns a struct_time object instead of a time tuple wouldn't hurt, though.

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2003-01-21 06:40

Message:
Logged In: YES 
user_id=6656

This was very deliberate.  It *is* a bit shocking that the
docs didn't get updated, though.

There's a start of a description in:

http://www.python.org/doc/current/whatsnew/node10.html

Do you have code that actually broke because of this change?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=671731&group_id=5470


From noreply@sourceforge.net  Tue Feb  4 16:19:26 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 04 Feb 2003 08:19:26 -0800
Subject: [Python-bugs-list] [ python-Bugs-531145 ] socket.sslerror is not a socket.error
Message-ID: <E18g5nC-0004f3-00@sc8-sf-web1.sourceforge.net>

Bugs item #531145, was opened at 2002-03-17 23:14
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=531145&group_id=5470

Category: Python Library
Group: None
Status: Open
Resolution: None
Priority: 3
Submitted By: Bastian Kleineidam (calvin)
Assigned to: Nobody/Anonymous (nobody)
Summary: socket.sslerror is not a socket.error

Initial Comment:
Python 2.1.2 (#1, Mar 16 2002, 00:56:55) 
[GCC 2.95.4 20011002 (Debian prerelease)] on linux2
Type "copyright", "credits" or "license" for more
information.
>>> import socket
>>> socket.sslerror
<class socket.sslerror at 0x809c39c>
>>> try: raise socket.sslerror
... except socket.error: pass
... 
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
socket.sslerror
>>> 


----------------------------------------------------------------------

Comment By: Michael Stone (mbrierst)
Date: 2003-02-04 16:19

Message:
Logged In: YES 
user_id=670441

If this behavior is desired, the patch below implements it.  If it's not desired, this bug sould be closed.

It isn't much work at all.  test_socket and test_socket_ssl still pass fine.  The following case will work after this patch, all other combinations will work as before:

>>> try: raise socket.sslerror
... except socket.error: print 'caught'
caught
>>>

patch (I cannot attach the file in sourceforge for some reason):
Index: dist/src/Modules/socketmodule.h
===================================================================
RCS file: /cvsroot/python/python/dist/src/Modules/socketmodule.h,v
retrieving revision 1.8
diff -c -r1.8 socketmodule.h
*** dist/src/Modules/socketmodule.h	13 Jun 2002 15:07:44 -0000	1.8
--- dist/src/Modules/socketmodule.h	4 Feb 2003 16:05:27 -0000
***************
*** 140,145 ****
--- 140,146 ----
  /* C API for usage by other Python modules */
  typedef struct {
  	PyTypeObject *Sock_Type;
+         PyObject *Sock_Error;
  } PySocketModule_APIObject;
  
  /* XXX The net effect of the following appears to be to define a function
Index: dist/src/Modules/socketmodule.c
===================================================================
RCS file: /cvsroot/python/python/dist/src/Modules/socketmodule.c,v
retrieving revision 1.251
diff -c -r1.251 socketmodule.c
*** dist/src/Modules/socketmodule.c	6 Jan 2003 12:41:26 -0000	1.251
--- dist/src/Modules/socketmodule.c	4 Feb 2003 16:05:43 -0000
***************
*** 3117,3122 ****
--- 3117,3123 ----
  PySocketModule_APIObject PySocketModuleAPI =
  {
  	&sock_type,
+         NULL,
  };
  
  
***************
*** 3178,3183 ****
--- 3179,3185 ----
  		return;
  
  	/* Export C API */
+         PySocketModuleAPI.Sock_Error = socket_error;
  	if (PyModule_AddObject(m, PySocket_CAPI_NAME,
  	       PyCObject_FromVoidPtr((void *)&PySocketModuleAPI, NULL)
  				 ) != 0)
Index: dist/src/Modules/_ssl.c
===================================================================
RCS file: /cvsroot/python/python/dist/src/Modules/_ssl.c,v
retrieving revision 1.9
diff -c -r1.9 _ssl.c
*** dist/src/Modules/_ssl.c	27 Jan 2003 22:19:21 -0000	1.9
--- dist/src/Modules/_ssl.c	4 Feb 2003 16:05:45 -0000
***************
*** 543,549 ****
  	SSLeay_add_ssl_algorithms();
  
  	/* Add symbols to module dict */
! 	PySSLErrorObject = PyErr_NewException("socket.sslerror", NULL, NULL);
  	if (PySSLErrorObject == NULL)
  		return;
  	PyDict_SetItemString(d, "sslerror", PySSLErrorObject);
--- 543,549 ----
  	SSLeay_add_ssl_algorithms();
  
  	/* Add symbols to module dict */
! 	PySSLErrorObject = PyErr_NewException("socket.sslerror", PySocketModule.Sock_Error, NULL);
  	if (PySSLErrorObject == NULL)
  		return;
  	PyDict_SetItemString(d, "sslerror", PySSLErrorObject);


----------------------------------------------------------------------

Comment By: Bastian Kleineidam (calvin)
Date: 2002-11-11 17:12

Message:
Logged In: YES 
user_id=9205

Yes, I want socket.sslerror to be a subclass of
socket.error. It simplifies code layout (see my previous
answer) and it follows the documentation.
And yes, its work, but only a little :)

----------------------------------------------------------------------

Comment By: Gerhard Häring (ghaering)
Date: 2002-10-05 00:05

Message:
Logged In: YES 
user_id=163326

So do you propose to make socket.sslerror a subclass of
socket.error. Is this desirable? I'm not sure. Is it work? Yes.

----------------------------------------------------------------------

Comment By: Bastian Kleineidam (calvin)
Date: 2002-03-18 20:23

Message:
Logged In: YES 
user_id=9205

The documentation says for socket.error: "This exception is
raised for socket- or address-related errors." I think
socket.sslerror is such an error, because then you can write

try: sock.write("") # could be ssl-socket
except socket.error: pass

The other way would be
_exceptions = [socket.error]
if hasattr(socket, "sslerror"):
    _exceptions.append(socket.sslerror)
try: sock.write("")
except _exceptions: pass

Anyway, I assume this is a minor "bug".

----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2002-03-18 10:44

Message:
Logged In: YES 
user_id=21627

Why is this a bug?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=531145&group_id=5470


From noreply@sourceforge.net  Tue Feb  4 17:15:17 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 04 Feb 2003 09:15:17 -0800
Subject: [Python-bugs-list] [ python-Bugs-665835 ] filter() treatment of str and tuple inconsistent
Message-ID: <E18g6fF-0005pK-00@sc8-sf-web4.sourceforge.net>

Bugs item #665835, was opened at 2003-01-10 17:36
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=665835&group_id=5470

Category: Python Interpreter Core
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Walter Dörwald (doerwalter)
Assigned to: Raymond Hettinger (rhettinger)
Summary: filter() treatment of str and tuple inconsistent

Initial Comment:
class tuple2(tuple):
·  def __getitem__(self, index):
·  ·  return 2*tuple.__getitem__(self, index)

class str2(str):
·  def __getitem__(self, index):
·  ·  return chr(ord(str.__getitem__(self, index))+1)

print filter(lambda x: x>1, tuple2((1, 2)))
print filter(lambda x: x>"a", str2("ab"))

this prints:
(2,)
bc

i.e. the overwritten __getitem__ is ignored in the
first case, but honored in the second.


----------------------------------------------------------------------

>Comment By: Walter Dörwald (doerwalter)
Date: 2003-02-04 18:15

Message:
Logged In: YES 
user_id=89016

OK, the problem of __getitem__ not returning str or unicode
is fixed. Unfortunately the result is rather ugly. With the
following class:

class u(unicode):
   def __getitem__(self, index):
      return u(2*unicode.__getitem__(self, index))

filter neither returns a list nor an u object, but a unicode
object, defeating the whole purpose of the special treatment
of str/unicode. If we remove the special treatment this
problem would go away, furthermore __getitem__ returning
objects that are not str/unicode instances wouldn't be
problem any longer.


----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-03 23:36

Message:
Logged In: YES 
user_id=6380

Walter: if you can fix the bug in your latest message here,
go ahead and check it in. Seems like a case of a missing test.

Raymond: it turns out that the iterator in Python 2.2 has
the same problem with lists -- it special-cases lists. But
for tuples, the iterator uses PySequence_GetItem; the fast
tuple iterator in Python 2.3 introduces the problem for
tuples though.

I actually don't think there would be much disagreement that
this behavior (ignoring __getitem__) is a bug. There may be
disagreement over how important it is to fix it. Personally,
I've generally been on the side of "it needn't be fixed if
it slows down the common case", as long as a workaround
(like overriding __iter__ alongside the __getitem__
override) exists.

But I draw the line at being backwards incompatible with
Python 2.2. There fore I think the tuple iterator (and
probably also the string iterator) needs to be fixed, and I
still think that it would be best if the list iterator were
also fixed. One way to do this would be for the tp_iter
implementation to check whether
self->ob_type->tp_as_sequence->sq_item is not equal to the
list_item function (this is a good check to detect a
__getitem__ override) and then return an instance of the
generic sequence iterator instead of the list-specific iterator.

----------------------------------------------------------------------

Comment By: Walter Dörwald (doerwalter)
Date: 2003-01-27 13:24

Message:
Logged In: YES 
user_id=89016

Another problem with filter() is that filterstring() (and
the new filterunicode()) blindly assume that
tp_as_sequence->sq_item returns a str or unicode object with
len==1. This might fail with str or unicode subclasses:
----
class badstr(str):
   def __getitem__(self, index):
      return 42

s = filter(lambda x: x>=42, badstr("1234"))
print len(s), repr(s)
----
This prints
4 '\x00\x00\x00\x00'

----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2003-01-27 02:13

Message:
Logged In: YES 
user_id=80475

One other thought:  A major reason for implementing 
__iter__ in the first place is that objects were overriding 
__getitem__ and disregarding the index -- the __getitem__ 
interface just didn't make sense for iteration in some 
situations.  __iter__ was supposed to provide enormous 
flexibility in various ways to loop over a collection (inorder, 
preorder, postorder, priorityorder, sortedorder, hashorder, 
randomorder, etc).  Making iter() default to using 
__getitem__ was only supposed to be an expedient for 
backwards compatability.  Always using __getitem__ 
diminishes the flexibility and speed advantages.

Maybe the discussion belongs on python-dev. I'm sure a 
number of people feel strongly one way or the other.  The 
question might as well be addressed head-on before 2.3 
goes out the door. 

----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2003-01-27 01:54

Message:
Logged In: YES 
user_id=80475

I understand.  Ideally, *all* methods would respond to a 
single overridden method, but I think this is just a fact of 
life in object oriented programming.

I can't remember where you gave an example of a 
d.__getitem__() subclass override, but you were careful to 
point out that other methods, like d.get() also needed to 
be overridden so that the modified access applied 
everywhere.  Likewise, __iter__() or any other object 
access method must be assumed to access the underlying 
data structure directly and must be overridden.   For 
instance, creating a dictionary with case insensitive 
lookups entails overriding __getitem__(k), get(k,default), 
and pop(k) -- no one of them can be presumed to inform 
the others.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-01-27 01:17

Message:
Logged In: YES 
user_id=6380

Hm... that means that iter() of *amy* built-in type subclass
overriding __getitem__ bypasses the override, unless the
subclass also overrides __iter__. This sounds like a step in
the wrong direction. I think the built-in iterators should
be aware of subclasses overriding __getitem__ one way or
another. I hadn't realized this when we started the trend of
creating faster iterators for built-in types. :-(

----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2003-01-25 17:45

Message:
Logged In: YES 
user_id=80475

None of the existing iterators (incl dicts, lists, tuples, and 
files) use __getitem__.  Most likely, user defined iterators 
also access the data structure directly (for flexiblity and 
speed). Also, anything that uses PyTuple_GET_ITEM 
bypasses __getitem__.

If string/unicode iterators are added, they should also go
directly to the underlying data; otherwise, there is no point 
to it.

Also, the proposal to change filtertuple(), doesn't solve
inconsistencies within filterstring() which uses __getitem__ 
when there is a function call, but bypasses it when the 
function parameter is Py_None.

I think the right answer is to change filterstring() to use an 
iterator and to implement string/unicode iterators that 
access the data directly (not using __getitem__).

FYI for Tim:  MvL noticed and fixed the unicode vs string 
difference.  His patch, SF #636005, has not been applied 
yet.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-01-25 14:51

Message:
Logged In: YES 
user_id=6380

(But in addition th that, I don't mind having a custom
string iterator -- as long as it calls __getitem__ properly.
Hm, shouldn't the tuple iterator call __getitem__ properly too?)


----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2003-01-25 14:45

Message:
Logged In: YES 
user_id=31435

Just noting that filter() is unique in special-casing the type 
of the input.  It's always been surprising that way, and, 
e.g., filtering a string produces a string, but filtering a 
Unicode string produces a list.

map() and reduce() don't play games like that, and always 
use the iteration protocol to march over their inputs.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-01-25 14:36

Message:
Logged In: YES 
user_id=6380

I don't know which Python sources Raymond has been reading,
but in the sources I've got in front of me, there are
special cases for strings and tuples, and these *don't* use
iter(). It so happens that the tuple special-case calls
PyTuple_GetItem(), which doesn't call your __getitem__,
while the string special-case calls the sq_item slot
function, which (in your case) will be a wrapper that calls
your __getitem__.

A minimal fix would be to only call filtertuple for strict
tuples -- although this changes the output type, but I don't
think one should count on filter() of a tuple subclass
returning a tuple (and it can't be made to return an
instance of the subclass either -- we don't know the
constructor signature).

Similar fixes probably need to be made to map() and maybe
reduce().

----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2003-01-25 04:47

Message:
Logged In: YES 
user_id=80475

The problem isn't with filter() which correctly calls iter() in 
both cases.

Tuple object have their own iterator which loops over 
elements directly and has no intervening calls to 
__getitem__().

String objects do not define a custom iterator, so iter() 
wraps itself around consecutive calls to __getitem__().

The resolution is to provide string objects with their own 
iterator. As a side benefit, iteration will run just a tiny bit 
faster.  The same applies to unicode objects.

Guido, do you care about this and want me to fix it or 
would you like to close it as "won't fix".


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=665835&group_id=5470


From noreply@sourceforge.net  Tue Feb  4 17:33:09 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 04 Feb 2003 09:33:09 -0800
Subject: [Python-bugs-list] [ python-Bugs-665835 ] filter() treatment of str and tuple inconsistent
Message-ID: <E18g6wX-0005Ei-00@sc8-sf-web3.sourceforge.net>

Bugs item #665835, was opened at 2003-01-10 11:36
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=665835&group_id=5470

Category: Python Interpreter Core
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Walter Dörwald (doerwalter)
Assigned to: Raymond Hettinger (rhettinger)
Summary: filter() treatment of str and tuple inconsistent

Initial Comment:
class tuple2(tuple):
·  def __getitem__(self, index):
·  ·  return 2*tuple.__getitem__(self, index)

class str2(str):
·  def __getitem__(self, index):
·  ·  return chr(ord(str.__getitem__(self, index))+1)

print filter(lambda x: x>1, tuple2((1, 2)))
print filter(lambda x: x>"a", str2("ab"))

this prints:
(2,)
bc

i.e. the overwritten __getitem__ is ignored in the
first case, but honored in the second.


----------------------------------------------------------------------

>Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-04 12:33

Message:
Logged In: YES 
user_id=6380

Yes, the special treatment of tuple, str and unicode is
problematic. :-(
I wish filter() had always returned a list for all input types.
But it's too late to change that.

However, I don't think that filter() should ever return a
*subclass* of tuple, str or unicode. Note that slicing a
subclass of these also doesn't return a subclass instance,
unless the subclass specifically overrides __getslice__.

I note that filter() of a tuple *almost* implements what I
think it should, except that if it receives an empty tuple
subclass, it returns it unchanged. The slicing and other
methods (e.g. lower()) have all been modified to make a copy
whose type is the base class; I think filter() should follow
suit.

Similar for filter() of strings and unicode.

----------------------------------------------------------------------

Comment By: Walter Dörwald (doerwalter)
Date: 2003-02-04 12:15

Message:
Logged In: YES 
user_id=89016

OK, the problem of __getitem__ not returning str or unicode
is fixed. Unfortunately the result is rather ugly. With the
following class:

class u(unicode):
   def __getitem__(self, index):
      return u(2*unicode.__getitem__(self, index))

filter neither returns a list nor an u object, but a unicode
object, defeating the whole purpose of the special treatment
of str/unicode. If we remove the special treatment this
problem would go away, furthermore __getitem__ returning
objects that are not str/unicode instances wouldn't be
problem any longer.


----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-03 17:36

Message:
Logged In: YES 
user_id=6380

Walter: if you can fix the bug in your latest message here,
go ahead and check it in. Seems like a case of a missing test.

Raymond: it turns out that the iterator in Python 2.2 has
the same problem with lists -- it special-cases lists. But
for tuples, the iterator uses PySequence_GetItem; the fast
tuple iterator in Python 2.3 introduces the problem for
tuples though.

I actually don't think there would be much disagreement that
this behavior (ignoring __getitem__) is a bug. There may be
disagreement over how important it is to fix it. Personally,
I've generally been on the side of "it needn't be fixed if
it slows down the common case", as long as a workaround
(like overriding __iter__ alongside the __getitem__
override) exists.

But I draw the line at being backwards incompatible with
Python 2.2. There fore I think the tuple iterator (and
probably also the string iterator) needs to be fixed, and I
still think that it would be best if the list iterator were
also fixed. One way to do this would be for the tp_iter
implementation to check whether
self->ob_type->tp_as_sequence->sq_item is not equal to the
list_item function (this is a good check to detect a
__getitem__ override) and then return an instance of the
generic sequence iterator instead of the list-specific iterator.

----------------------------------------------------------------------

Comment By: Walter Dörwald (doerwalter)
Date: 2003-01-27 07:24

Message:
Logged In: YES 
user_id=89016

Another problem with filter() is that filterstring() (and
the new filterunicode()) blindly assume that
tp_as_sequence->sq_item returns a str or unicode object with
len==1. This might fail with str or unicode subclasses:
----
class badstr(str):
   def __getitem__(self, index):
      return 42

s = filter(lambda x: x>=42, badstr("1234"))
print len(s), repr(s)
----
This prints
4 '\x00\x00\x00\x00'

----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2003-01-26 20:13

Message:
Logged In: YES 
user_id=80475

One other thought:  A major reason for implementing 
__iter__ in the first place is that objects were overriding 
__getitem__ and disregarding the index -- the __getitem__ 
interface just didn't make sense for iteration in some 
situations.  __iter__ was supposed to provide enormous 
flexibility in various ways to loop over a collection (inorder, 
preorder, postorder, priorityorder, sortedorder, hashorder, 
randomorder, etc).  Making iter() default to using 
__getitem__ was only supposed to be an expedient for 
backwards compatability.  Always using __getitem__ 
diminishes the flexibility and speed advantages.

Maybe the discussion belongs on python-dev. I'm sure a 
number of people feel strongly one way or the other.  The 
question might as well be addressed head-on before 2.3 
goes out the door. 

----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2003-01-26 19:54

Message:
Logged In: YES 
user_id=80475

I understand.  Ideally, *all* methods would respond to a 
single overridden method, but I think this is just a fact of 
life in object oriented programming.

I can't remember where you gave an example of a 
d.__getitem__() subclass override, but you were careful to 
point out that other methods, like d.get() also needed to 
be overridden so that the modified access applied 
everywhere.  Likewise, __iter__() or any other object 
access method must be assumed to access the underlying 
data structure directly and must be overridden.   For 
instance, creating a dictionary with case insensitive 
lookups entails overriding __getitem__(k), get(k,default), 
and pop(k) -- no one of them can be presumed to inform 
the others.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-01-26 19:17

Message:
Logged In: YES 
user_id=6380

Hm... that means that iter() of *amy* built-in type subclass
overriding __getitem__ bypasses the override, unless the
subclass also overrides __iter__. This sounds like a step in
the wrong direction. I think the built-in iterators should
be aware of subclasses overriding __getitem__ one way or
another. I hadn't realized this when we started the trend of
creating faster iterators for built-in types. :-(

----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2003-01-25 11:45

Message:
Logged In: YES 
user_id=80475

None of the existing iterators (incl dicts, lists, tuples, and 
files) use __getitem__.  Most likely, user defined iterators 
also access the data structure directly (for flexiblity and 
speed). Also, anything that uses PyTuple_GET_ITEM 
bypasses __getitem__.

If string/unicode iterators are added, they should also go
directly to the underlying data; otherwise, there is no point 
to it.

Also, the proposal to change filtertuple(), doesn't solve
inconsistencies within filterstring() which uses __getitem__ 
when there is a function call, but bypasses it when the 
function parameter is Py_None.

I think the right answer is to change filterstring() to use an 
iterator and to implement string/unicode iterators that 
access the data directly (not using __getitem__).

FYI for Tim:  MvL noticed and fixed the unicode vs string 
difference.  His patch, SF #636005, has not been applied 
yet.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-01-25 08:51

Message:
Logged In: YES 
user_id=6380

(But in addition th that, I don't mind having a custom
string iterator -- as long as it calls __getitem__ properly.
Hm, shouldn't the tuple iterator call __getitem__ properly too?)


----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2003-01-25 08:45

Message:
Logged In: YES 
user_id=31435

Just noting that filter() is unique in special-casing the type 
of the input.  It's always been surprising that way, and, 
e.g., filtering a string produces a string, but filtering a 
Unicode string produces a list.

map() and reduce() don't play games like that, and always 
use the iteration protocol to march over their inputs.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-01-25 08:36

Message:
Logged In: YES 
user_id=6380

I don't know which Python sources Raymond has been reading,
but in the sources I've got in front of me, there are
special cases for strings and tuples, and these *don't* use
iter(). It so happens that the tuple special-case calls
PyTuple_GetItem(), which doesn't call your __getitem__,
while the string special-case calls the sq_item slot
function, which (in your case) will be a wrapper that calls
your __getitem__.

A minimal fix would be to only call filtertuple for strict
tuples -- although this changes the output type, but I don't
think one should count on filter() of a tuple subclass
returning a tuple (and it can't be made to return an
instance of the subclass either -- we don't know the
constructor signature).

Similar fixes probably need to be made to map() and maybe
reduce().

----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2003-01-24 22:47

Message:
Logged In: YES 
user_id=80475

The problem isn't with filter() which correctly calls iter() in 
both cases.

Tuple object have their own iterator which loops over 
elements directly and has no intervening calls to 
__getitem__().

String objects do not define a custom iterator, so iter() 
wraps itself around consecutive calls to __getitem__().

The resolution is to provide string objects with their own 
iterator. As a side benefit, iteration will run just a tiny bit 
faster.  The same applies to unicode objects.

Guido, do you care about this and want me to fix it or 
would you like to close it as "won't fix".


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=665835&group_id=5470


From noreply@sourceforge.net  Tue Feb  4 18:24:52 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 04 Feb 2003 10:24:52 -0800
Subject: [Python-bugs-list] [ python-Bugs-680379 ] Incorrect permissions set in lib-dynload.
Message-ID: <E18g7ka-0000GG-00@sc8-sf-web2.sourceforge.net>

Bugs item #680379, was opened at 2003-02-04 13:24
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=680379&group_id=5470

Category: Distutils
Group: Python 2.2.2
Status: Open
Resolution: None
Priority: 5
Submitted By: Arcady Genkin (antipode)
Assigned to: Nobody/Anonymous (nobody)
Summary: Incorrect permissions set in lib-dynload.

Initial Comment:
I have seen a number of bugs on this issue, but all of
them seem dot be marked as "closed" for some reason. 
This has been the case in every release since Python
2.1 and is still the case in 2.2.2.

When istalling Python by typing "make install" (with
umask set to 022) everything is installed with correct
permissions, *except* for the stuff in the lib-dynload
directory, where the files are installed with mode 700
(should be of 755).

Thanks.
-- 
Arcady Genkin

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=680379&group_id=5470


From noreply@sourceforge.net  Tue Feb  4 19:03:14 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 04 Feb 2003 11:03:14 -0800
Subject: [Python-bugs-list] [ python-Bugs-543344 ] Interpreter crashes when recoding
Message-ID: <E18g8Li-0004nB-00@sc8-sf-web1.sourceforge.net>

Bugs item #543344, was opened at 2002-04-13 08:12
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=543344&group_id=5470

Category: Unicode
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Martin v. Löwis (loewis)
Assigned to: M.-A. Lemburg (lemburg)
Summary: Interpreter crashes when recoding

Initial Comment:
Python crashes when executing

import codecs
f = open("syllables", "w+")
f2 = codecs.EncodedFile(f, "unicode_internal", "utf-8")
f2.write(u"a")
f2.close()


----------------------------------------------------------------------

Comment By: Michael Stone (mbrierst)
Date: 2003-02-04 19:03

Message:
Logged In: YES 
user_id=670441

This was a problem in Modules/_codecsmodule.c
Throughout most of the module, objects retrieved through PyArg_ParseTuple are converted to new types, but in this one line the object is used directly.  In that case we must INCREF it as PyArg_ParseTuple will not.

Patch below:

Index: dist/src/Modules/_codecsmodule.c
===================================================================
RCS file: /cvsroot/python/python/dist/src/Modules/_codecsmodule.c,v
retrieving revision 2.16
diff -c -r2.16 _codecsmodule.c
*** dist/src/Modules/_codecsmodule.c	31 Oct 2002 13:36:29 -0000	2.16
--- dist/src/Modules/_codecsmodule.c	4 Feb 2003 18:53:50 -0000
***************
*** 167,174 ****
  			  &obj, &errors))
  	return NULL;
  
!     if (PyUnicode_Check(obj))
  	return codec_tuple(obj, PyUnicode_GET_SIZE(obj));
      else {
  	if (PyObject_AsReadBuffer(obj, (const void **)&data, &size))
  	    return NULL;
--- 167,176 ----
  			  &obj, &errors))
  	return NULL;
  
!     if (PyUnicode_Check(obj)) {
! 	Py_INCREF(obj);
  	return codec_tuple(obj, PyUnicode_GET_SIZE(obj));
+     }
      else {
  	if (PyObject_AsReadBuffer(obj, (const void **)&data, &size))
  	    return NULL;


----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2002-04-13 08:36

Message:
Logged In: YES 
user_id=31435

Here's a clue from a debug-mode build:

>>> import codecs
[9477 refs]
>>> f = open("syllables", "w+")
[9483 refs]
>>> f2 = codecs.EncodedFile(f, "unicode_internal", "utf-8")
[9830 refs]
>>> f2.write(u"a")
C:\Code\python\Objects\tupleobject.c:147 negative ref 
count -1


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=543344&group_id=5470


From noreply@sourceforge.net  Tue Feb  4 19:23:25 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 04 Feb 2003 11:23:25 -0800
Subject: [Python-bugs-list] [ python-Bugs-543344 ] Interpreter crashes when recoding
Message-ID: <E18g8fF-0005xy-00@sc8-sf-web1.sourceforge.net>

Bugs item #543344, was opened at 2002-04-13 10:12
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=543344&group_id=5470

Category: Unicode
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Martin v. Löwis (loewis)
Assigned to: M.-A. Lemburg (lemburg)
Summary: Interpreter crashes when recoding

Initial Comment:
Python crashes when executing

import codecs
f = open("syllables", "w+")
f2 = codecs.EncodedFile(f, "unicode_internal", "utf-8")
f2.write(u"a")
f2.close()


----------------------------------------------------------------------

>Comment By: M.-A. Lemburg (lemburg)
Date: 2003-02-04 20:23

Message:
Logged In: YES 
user_id=38388

Good catch. I'll check in the patch.
Thanks.


----------------------------------------------------------------------

Comment By: Michael Stone (mbrierst)
Date: 2003-02-04 20:03

Message:
Logged In: YES 
user_id=670441

This was a problem in Modules/_codecsmodule.c
Throughout most of the module, objects retrieved through PyArg_ParseTuple are converted to new types, but in this one line the object is used directly.  In that case we must INCREF it as PyArg_ParseTuple will not.

Patch below:

Index: dist/src/Modules/_codecsmodule.c
===================================================================
RCS file: /cvsroot/python/python/dist/src/Modules/_codecsmodule.c,v
retrieving revision 2.16
diff -c -r2.16 _codecsmodule.c
*** dist/src/Modules/_codecsmodule.c	31 Oct 2002 13:36:29 -0000	2.16
--- dist/src/Modules/_codecsmodule.c	4 Feb 2003 18:53:50 -0000
***************
*** 167,174 ****
  			  &obj, &errors))
  	return NULL;
  
!     if (PyUnicode_Check(obj))
  	return codec_tuple(obj, PyUnicode_GET_SIZE(obj));
      else {
  	if (PyObject_AsReadBuffer(obj, (const void **)&data, &size))
  	    return NULL;
--- 167,176 ----
  			  &obj, &errors))
  	return NULL;
  
!     if (PyUnicode_Check(obj)) {
! 	Py_INCREF(obj);
  	return codec_tuple(obj, PyUnicode_GET_SIZE(obj));
+     }
      else {
  	if (PyObject_AsReadBuffer(obj, (const void **)&data, &size))
  	    return NULL;


----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2002-04-13 10:36

Message:
Logged In: YES 
user_id=31435

Here's a clue from a debug-mode build:

>>> import codecs
[9477 refs]
>>> f = open("syllables", "w+")
[9483 refs]
>>> f2 = codecs.EncodedFile(f, "unicode_internal", "utf-8")
[9830 refs]
>>> f2.write(u"a")
C:\Code\python\Objects\tupleobject.c:147 negative ref 
count -1


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=543344&group_id=5470


From noreply@sourceforge.net  Tue Feb  4 19:55:43 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 04 Feb 2003 11:55:43 -0800
Subject: [Python-bugs-list] [ python-Bugs-680429 ] __module__ broken for extension classes
Message-ID: <E18g9AV-0004wG-00@sc8-sf-web2.sourceforge.net>

Bugs item #680429, was opened at 2003-02-04 11:55
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=680429&group_id=5470

Category: Python Interpreter Core
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Ralf W. Grosse-Kunstleve (rwgk)
Assigned to: Nobody/Anonymous (nobody)
Summary: __module__ broken for extension classes

Initial Comment:
We are having problems using Boost.Python with Python 
2.3a1 because
under some circumstances the result of __module__ is 
different compared
to earlier Python versions:

Python 2.2.1 (#2, Jun 17 2002, 12:06:51) 
[GCC 2.96 20000731 (Red Hat Linux 7.3 2.96-110)] on 
linux2
Type "help", "copyright", "credits" or "license" for more 
information.
>>> import simple
>>> simple.empty.__module__
'simple'
>>> 

Python 2.3a1 (#1, Jan  6 2003, 14:17:56) 
[GCC 2.96 20000731 (Red Hat Linux 7.3 2.96-110)] on 
linux2
Type "help", "copyright", "credits" or "license" for more 
information.
>>> import simple
>>> simple.empty.__module__
'__main__'
>>> 

Because of this we can no longer pickle our extension 
classes.
For your reference the code for the simple module is 
attached.
This is using Boost release 1.29.0 (www.boost.org).

We have done some debugging. Boost.Python's internal 
idea of the module
associated with an extension class is still correct even when 
using
Python 2.3a1. David Abrahams (main Boost.Python author) 
is telling me
that he "changed Boost.Python to work the way Guido 
suggested before
2.2.2." Therefore we suspect that the __module__ problem 
is due to
a change/bug in Python 2.3a1.

Ralf


#include <boost/python/module.hpp>
#include <boost/python/class.hpp>

namespace sandbx { namespace {

  struct empty {};

  void init_module()
  {
    using namespace boost::python;
    class_<empty>("empty");
  }

}} // namespace sandbx::<anonymous>

BOOST_PYTHON_MODULE(simple)
{
  sandbx::init_module();
}


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=680429&group_id=5470


From noreply@sourceforge.net  Tue Feb  4 20:02:23 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 04 Feb 2003 12:02:23 -0800
Subject: [Python-bugs-list] [ python-Bugs-545855 ] Wrong exception from re.compile()
Message-ID: <E18g9Gx-0006G3-00@sc8-sf-web4.sourceforge.net>

Bugs item #545855, was opened at 2002-04-18 22:22
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=545855&group_id=5470

Category: Regular Expressions
Group: Python 2.2
Status: Open
Resolution: None
Priority: 5
Submitted By: Barry A. Warsaw (bwarsaw)
Assigned to: Fredrik Lundh (effbot)
Summary: Wrong exception from re.compile()

Initial Comment:
re.compile('foo[a-') raises a TypeError exception
instead of re.error.

Python 2.2.1 (#1, Apr  4 2002, 17:22:15) 
[GCC 2.96 20000731 (Mandrake Linux 8.2 2.96-0.76mdk)]
on linux2
Type "help", "copyright", "credits" or "license" for
more information.
>>> import re
>>> re.compile('foo[a-')
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "/usr/local/lib/python2.2/sre.py", line 178, in
compile
    return _compile(pattern, flags)
  File "/usr/local/lib/python2.2/sre.py", line 226, in
_compile
    p = sre_compile.compile(pattern, flags)
  File "/usr/local/lib/python2.2/sre_compile.py", line
430, in compile
    p = sre_parse.parse(p, flags)
  File "/usr/local/lib/python2.2/sre_parse.py", line
623, in parse
    p = _parse_sub(source, pattern, 0)
  File "/usr/local/lib/python2.2/sre_parse.py", line
318, in _parse_sub
    items.append(_parse(source, state))
  File "/usr/local/lib/python2.2/sre_parse.py", line
424, in _parse
    if this[0] == "\":
TypeError: unsubscriptable object


----------------------------------------------------------------------

Comment By: Michael Stone (mbrierst)
Date: 2003-02-04 20:02

Message:
Logged In: YES 
user_id=670441

Patch below fixes the problem.  Didn't check for end of expression in one case.

Index: dist/src/Lib/sre_parse.py
===================================================================
RCS file: /cvsroot/python/python/dist/src/Lib/sre_parse.py,v
retrieving revision 1.55
diff -c -r1.55 sre_parse.py
*** dist/src/Lib/sre_parse.py	2 Jun 2002 00:40:05 -0000	1.55
--- dist/src/Lib/sre_parse.py	4 Feb 2003 19:53:14 -0000
***************
*** 420,429 ****
                          set.append((LITERAL, ord("-")))
                          break
                      else:
!                         if this[0] == "\":
                              code2 = _class_escape(source, this)
!                         else:
                              code2 = LITERAL, ord(this)
                          if code1[0] != LITERAL or code2[0] != LITERAL:
                              raise error, "bad character range"
                          lo = code1[1]
--- 420,431 ----
                          set.append((LITERAL, ord("-")))
                          break
                      else:
!                         if this and this[0] == "\":
                              code2 = _class_escape(source, this)
!                         elif this:
                              code2 = LITERAL, ord(this)
+                         else:
+                             raise error, "unexpected end of regular expression"
                          if code1[0] != LITERAL or code2[0] != LITERAL:
                              raise error, "bad character range"
                          lo = code1[1]


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=545855&group_id=5470


From noreply@sourceforge.net  Tue Feb  4 19:41:53 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 04 Feb 2003 11:41:53 -0800
Subject: [Python-bugs-list] [ python-Bugs-543344 ] Interpreter crashes when recoding
Message-ID: <E18g8x7-0003Uz-00@sc8-sf-web3.sourceforge.net>

Bugs item #543344, was opened at 2002-04-13 10:12
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=543344&group_id=5470

Category: Unicode
Group: None
>Status: Closed
>Resolution: Fixed
Priority: 5
Submitted By: Martin v. Löwis (loewis)
Assigned to: M.-A. Lemburg (lemburg)
Summary: Interpreter crashes when recoding

Initial Comment:
Python crashes when executing

import codecs
f = open("syllables", "w+")
f2 = codecs.EncodedFile(f, "unicode_internal", "utf-8")
f2.write(u"a")
f2.close()


----------------------------------------------------------------------

>Comment By: M.-A. Lemburg (lemburg)
Date: 2003-02-04 20:41

Message:
Logged In: YES 
user_id=38388

Checking in Lib/test/test_codecs.py;
/cvsroot/python/python/dist/src/Lib/test/test_codecs.py,v 
<--  test_codecs.py
new revision: 1.5; previous revision: 1.4
done
Checking in Modules/_codecsmodule.c;
/cvsroot/python/python/dist/src/Modules/_codecsmodule.c,v 
<--  _codecsmodule.c
new revision: 2.17; previous revision: 2.16
done


----------------------------------------------------------------------

Comment By: M.-A. Lemburg (lemburg)
Date: 2003-02-04 20:23

Message:
Logged In: YES 
user_id=38388

Good catch. I'll check in the patch.
Thanks.


----------------------------------------------------------------------

Comment By: Michael Stone (mbrierst)
Date: 2003-02-04 20:03

Message:
Logged In: YES 
user_id=670441

This was a problem in Modules/_codecsmodule.c
Throughout most of the module, objects retrieved through PyArg_ParseTuple are converted to new types, but in this one line the object is used directly.  In that case we must INCREF it as PyArg_ParseTuple will not.

Patch below:

Index: dist/src/Modules/_codecsmodule.c
===================================================================
RCS file: /cvsroot/python/python/dist/src/Modules/_codecsmodule.c,v
retrieving revision 2.16
diff -c -r2.16 _codecsmodule.c
*** dist/src/Modules/_codecsmodule.c	31 Oct 2002 13:36:29 -0000	2.16
--- dist/src/Modules/_codecsmodule.c	4 Feb 2003 18:53:50 -0000
***************
*** 167,174 ****
  			  &obj, &errors))
  	return NULL;
  
!     if (PyUnicode_Check(obj))
  	return codec_tuple(obj, PyUnicode_GET_SIZE(obj));
      else {
  	if (PyObject_AsReadBuffer(obj, (const void **)&data, &size))
  	    return NULL;
--- 167,176 ----
  			  &obj, &errors))
  	return NULL;
  
!     if (PyUnicode_Check(obj)) {
! 	Py_INCREF(obj);
  	return codec_tuple(obj, PyUnicode_GET_SIZE(obj));
+     }
      else {
  	if (PyObject_AsReadBuffer(obj, (const void **)&data, &size))
  	    return NULL;


----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2002-04-13 10:36

Message:
Logged In: YES 
user_id=31435

Here's a clue from a debug-mode build:

>>> import codecs
[9477 refs]
>>> f = open("syllables", "w+")
[9483 refs]
>>> f2 = codecs.EncodedFile(f, "unicode_internal", "utf-8")
[9830 refs]
>>> f2.write(u"a")
C:\Code\python\Objects\tupleobject.c:147 negative ref 
count -1


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=543344&group_id=5470


From noreply@sourceforge.net  Tue Feb  4 20:26:49 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 04 Feb 2003 12:26:49 -0800
Subject: [Python-bugs-list] [ python-Bugs-547176 ] rlcompleter does not expand on [ ]
Message-ID: <E18g9eb-00010X-00@sc8-sf-web1.sourceforge.net>

Bugs item #547176, was opened at 2002-04-22 17:39
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=547176&group_id=5470

Category: Extension Modules
Group: Feature Request
Status: Open
Resolution: None
Priority: 5
Submitted By: Jaco Schieke (schiekjd)
Assigned to: Nobody/Anonymous (nobody)
Summary: rlcompleter does not expand on [ ]

Initial Comment:
A thread on comp.lang.python details the problem:

http://groups.google.com/groups?
hl=en&threadm=b5ddd682.0204180902.2866a9c0%
40posting.google.com&rnum=1&prev=/groups%3Fhl%3Den%
26selm%3Db5ddd682.0204180902.2866a9c0%
2540posting.google.com

rlcompleter.py does not expand correctly on something 
like:
>>> sim[0].<tab>
since
  m = re.match(r"(\w+(\.\w+)*)\.(\w*)", text)
does not allow for brackets in the match.

I propose:
  m = re.match(r"([\w\[\]]+(\.[\w\[\]]+)*)\.(\w*)", 
text)

to fix this on line 127 in cvs version 1.11




----------------------------------------------------------------------

Comment By: Michael Stone (mbrierst)
Date: 2003-02-04 20:26

Message:
Logged In: YES 
user_id=670441

This isn't really a bug, as (at least currently) this behavior is mentioned in the documentation for rlcompleter (at least it's in the docstring).  

The reason a[0].<tab> isn't completed is that we want to avoid executing arbitrary code.  The current completion mechanism is already a bit dangerous as an object's getattr could be called to execute code with unknown side effects while completing.  It just seems to dangerous to allow indexing, as lots of objects use this for their own purposes.

If this behavior is determined desirable despite the danger, someone with commit privileges should say so, and then someone can submit a patch (I could do it).
Probably this bug should just be closed.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=547176&group_id=5470


From noreply@sourceforge.net  Tue Feb  4 20:34:18 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 04 Feb 2003 12:34:18 -0800
Subject: [Python-bugs-list] [ python-Bugs-665835 ] filter() treatment of str and tuple inconsistent
Message-ID: <E18g9lq-000818-00@sc8-sf-web4.sourceforge.net>

Bugs item #665835, was opened at 2003-01-10 17:36
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=665835&group_id=5470

Category: Python Interpreter Core
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Walter Dörwald (doerwalter)
Assigned to: Raymond Hettinger (rhettinger)
Summary: filter() treatment of str and tuple inconsistent

Initial Comment:
class tuple2(tuple):
·  def __getitem__(self, index):
·  ·  return 2*tuple.__getitem__(self, index)

class str2(str):
·  def __getitem__(self, index):
·  ·  return chr(ord(str.__getitem__(self, index))+1)

print filter(lambda x: x>1, tuple2((1, 2)))
print filter(lambda x: x>"a", str2("ab"))

this prints:
(2,)
bc

i.e. the overwritten __getitem__ is ignored in the
first case, but honored in the second.


----------------------------------------------------------------------

>Comment By: Walter Dörwald (doerwalter)
Date: 2003-02-04 21:34

Message:
Logged In: YES 
user_id=89016

The subclass problem has been fixed in:
Python/bltinmodule.c 2.275
Lib/test/test_builtin.py 1.9

But now something strange happens:
---
class badstr(str):
···def __getitem__(self, index):
······return str.__getitem__(self, index).upper()

print filter(None, badstr("abc"))
print filter(lambda x: x, badstr("abc"))
---
This prints
---
abc
ABC
---
although according to the filter docstring they should be
the same.


----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-04 18:33

Message:
Logged In: YES 
user_id=6380

Yes, the special treatment of tuple, str and unicode is
problematic. :-(
I wish filter() had always returned a list for all input types.
But it's too late to change that.

However, I don't think that filter() should ever return a
*subclass* of tuple, str or unicode. Note that slicing a
subclass of these also doesn't return a subclass instance,
unless the subclass specifically overrides __getslice__.

I note that filter() of a tuple *almost* implements what I
think it should, except that if it receives an empty tuple
subclass, it returns it unchanged. The slicing and other
methods (e.g. lower()) have all been modified to make a copy
whose type is the base class; I think filter() should follow
suit.

Similar for filter() of strings and unicode.

----------------------------------------------------------------------

Comment By: Walter Dörwald (doerwalter)
Date: 2003-02-04 18:15

Message:
Logged In: YES 
user_id=89016

OK, the problem of __getitem__ not returning str or unicode
is fixed. Unfortunately the result is rather ugly. With the
following class:

class u(unicode):
   def __getitem__(self, index):
      return u(2*unicode.__getitem__(self, index))

filter neither returns a list nor an u object, but a unicode
object, defeating the whole purpose of the special treatment
of str/unicode. If we remove the special treatment this
problem would go away, furthermore __getitem__ returning
objects that are not str/unicode instances wouldn't be
problem any longer.


----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-03 23:36

Message:
Logged In: YES 
user_id=6380

Walter: if you can fix the bug in your latest message here,
go ahead and check it in. Seems like a case of a missing test.

Raymond: it turns out that the iterator in Python 2.2 has
the same problem with lists -- it special-cases lists. But
for tuples, the iterator uses PySequence_GetItem; the fast
tuple iterator in Python 2.3 introduces the problem for
tuples though.

I actually don't think there would be much disagreement that
this behavior (ignoring __getitem__) is a bug. There may be
disagreement over how important it is to fix it. Personally,
I've generally been on the side of "it needn't be fixed if
it slows down the common case", as long as a workaround
(like overriding __iter__ alongside the __getitem__
override) exists.

But I draw the line at being backwards incompatible with
Python 2.2. There fore I think the tuple iterator (and
probably also the string iterator) needs to be fixed, and I
still think that it would be best if the list iterator were
also fixed. One way to do this would be for the tp_iter
implementation to check whether
self->ob_type->tp_as_sequence->sq_item is not equal to the
list_item function (this is a good check to detect a
__getitem__ override) and then return an instance of the
generic sequence iterator instead of the list-specific iterator.

----------------------------------------------------------------------

Comment By: Walter Dörwald (doerwalter)
Date: 2003-01-27 13:24

Message:
Logged In: YES 
user_id=89016

Another problem with filter() is that filterstring() (and
the new filterunicode()) blindly assume that
tp_as_sequence->sq_item returns a str or unicode object with
len==1. This might fail with str or unicode subclasses:
----
class badstr(str):
   def __getitem__(self, index):
      return 42

s = filter(lambda x: x>=42, badstr("1234"))
print len(s), repr(s)
----
This prints
4 '\x00\x00\x00\x00'

----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2003-01-27 02:13

Message:
Logged In: YES 
user_id=80475

One other thought:  A major reason for implementing 
__iter__ in the first place is that objects were overriding 
__getitem__ and disregarding the index -- the __getitem__ 
interface just didn't make sense for iteration in some 
situations.  __iter__ was supposed to provide enormous 
flexibility in various ways to loop over a collection (inorder, 
preorder, postorder, priorityorder, sortedorder, hashorder, 
randomorder, etc).  Making iter() default to using 
__getitem__ was only supposed to be an expedient for 
backwards compatability.  Always using __getitem__ 
diminishes the flexibility and speed advantages.

Maybe the discussion belongs on python-dev. I'm sure a 
number of people feel strongly one way or the other.  The 
question might as well be addressed head-on before 2.3 
goes out the door. 

----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2003-01-27 01:54

Message:
Logged In: YES 
user_id=80475

I understand.  Ideally, *all* methods would respond to a 
single overridden method, but I think this is just a fact of 
life in object oriented programming.

I can't remember where you gave an example of a 
d.__getitem__() subclass override, but you were careful to 
point out that other methods, like d.get() also needed to 
be overridden so that the modified access applied 
everywhere.  Likewise, __iter__() or any other object 
access method must be assumed to access the underlying 
data structure directly and must be overridden.   For 
instance, creating a dictionary with case insensitive 
lookups entails overriding __getitem__(k), get(k,default), 
and pop(k) -- no one of them can be presumed to inform 
the others.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-01-27 01:17

Message:
Logged In: YES 
user_id=6380

Hm... that means that iter() of *amy* built-in type subclass
overriding __getitem__ bypasses the override, unless the
subclass also overrides __iter__. This sounds like a step in
the wrong direction. I think the built-in iterators should
be aware of subclasses overriding __getitem__ one way or
another. I hadn't realized this when we started the trend of
creating faster iterators for built-in types. :-(

----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2003-01-25 17:45

Message:
Logged In: YES 
user_id=80475

None of the existing iterators (incl dicts, lists, tuples, and 
files) use __getitem__.  Most likely, user defined iterators 
also access the data structure directly (for flexiblity and 
speed). Also, anything that uses PyTuple_GET_ITEM 
bypasses __getitem__.

If string/unicode iterators are added, they should also go
directly to the underlying data; otherwise, there is no point 
to it.

Also, the proposal to change filtertuple(), doesn't solve
inconsistencies within filterstring() which uses __getitem__ 
when there is a function call, but bypasses it when the 
function parameter is Py_None.

I think the right answer is to change filterstring() to use an 
iterator and to implement string/unicode iterators that 
access the data directly (not using __getitem__).

FYI for Tim:  MvL noticed and fixed the unicode vs string 
difference.  His patch, SF #636005, has not been applied 
yet.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-01-25 14:51

Message:
Logged In: YES 
user_id=6380

(But in addition th that, I don't mind having a custom
string iterator -- as long as it calls __getitem__ properly.
Hm, shouldn't the tuple iterator call __getitem__ properly too?)


----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2003-01-25 14:45

Message:
Logged In: YES 
user_id=31435

Just noting that filter() is unique in special-casing the type 
of the input.  It's always been surprising that way, and, 
e.g., filtering a string produces a string, but filtering a 
Unicode string produces a list.

map() and reduce() don't play games like that, and always 
use the iteration protocol to march over their inputs.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-01-25 14:36

Message:
Logged In: YES 
user_id=6380

I don't know which Python sources Raymond has been reading,
but in the sources I've got in front of me, there are
special cases for strings and tuples, and these *don't* use
iter(). It so happens that the tuple special-case calls
PyTuple_GetItem(), which doesn't call your __getitem__,
while the string special-case calls the sq_item slot
function, which (in your case) will be a wrapper that calls
your __getitem__.

A minimal fix would be to only call filtertuple for strict
tuples -- although this changes the output type, but I don't
think one should count on filter() of a tuple subclass
returning a tuple (and it can't be made to return an
instance of the subclass either -- we don't know the
constructor signature).

Similar fixes probably need to be made to map() and maybe
reduce().

----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2003-01-25 04:47

Message:
Logged In: YES 
user_id=80475

The problem isn't with filter() which correctly calls iter() in 
both cases.

Tuple object have their own iterator which loops over 
elements directly and has no intervening calls to 
__getitem__().

String objects do not define a custom iterator, so iter() 
wraps itself around consecutive calls to __getitem__().

The resolution is to provide string objects with their own 
iterator. As a side benefit, iteration will run just a tiny bit 
faster.  The same applies to unicode objects.

Guido, do you care about this and want me to fix it or 
would you like to close it as "won't fix".


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=665835&group_id=5470


From noreply@sourceforge.net  Tue Feb  4 21:06:20 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 04 Feb 2003 13:06:20 -0800
Subject: [Python-bugs-list] [ python-Bugs-665835 ] filter() treatment of str and tuple inconsistent
Message-ID: <E18gAGq-0008GL-00@sc8-sf-web2.sourceforge.net>

Bugs item #665835, was opened at 2003-01-10 11:36
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=665835&group_id=5470

Category: Python Interpreter Core
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Walter Dörwald (doerwalter)
Assigned to: Raymond Hettinger (rhettinger)
Summary: filter() treatment of str and tuple inconsistent

Initial Comment:
class tuple2(tuple):
·  def __getitem__(self, index):
·  ·  return 2*tuple.__getitem__(self, index)

class str2(str):
·  def __getitem__(self, index):
·  ·  return chr(ord(str.__getitem__(self, index))+1)

print filter(lambda x: x>1, tuple2((1, 2)))
print filter(lambda x: x>"a", str2("ab"))

this prints:
(2,)
bc

i.e. the overwritten __getitem__ is ignored in the
first case, but honored in the second.


----------------------------------------------------------------------

>Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-04 16:06

Message:
Logged In: YES 
user_id=6380

So it does. I guess the special shortcut for None should
only be taken
when it's a proper str instance.

----------------------------------------------------------------------

Comment By: Walter Dörwald (doerwalter)
Date: 2003-02-04 15:34

Message:
Logged In: YES 
user_id=89016

The subclass problem has been fixed in:
Python/bltinmodule.c 2.275
Lib/test/test_builtin.py 1.9

But now something strange happens:
---
class badstr(str):
···def __getitem__(self, index):
······return str.__getitem__(self, index).upper()

print filter(None, badstr("abc"))
print filter(lambda x: x, badstr("abc"))
---
This prints
---
abc
ABC
---
although according to the filter docstring they should be
the same.


----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-04 12:33

Message:
Logged In: YES 
user_id=6380

Yes, the special treatment of tuple, str and unicode is
problematic. :-(
I wish filter() had always returned a list for all input types.
But it's too late to change that.

However, I don't think that filter() should ever return a
*subclass* of tuple, str or unicode. Note that slicing a
subclass of these also doesn't return a subclass instance,
unless the subclass specifically overrides __getslice__.

I note that filter() of a tuple *almost* implements what I
think it should, except that if it receives an empty tuple
subclass, it returns it unchanged. The slicing and other
methods (e.g. lower()) have all been modified to make a copy
whose type is the base class; I think filter() should follow
suit.

Similar for filter() of strings and unicode.

----------------------------------------------------------------------

Comment By: Walter Dörwald (doerwalter)
Date: 2003-02-04 12:15

Message:
Logged In: YES 
user_id=89016

OK, the problem of __getitem__ not returning str or unicode
is fixed. Unfortunately the result is rather ugly. With the
following class:

class u(unicode):
   def __getitem__(self, index):
      return u(2*unicode.__getitem__(self, index))

filter neither returns a list nor an u object, but a unicode
object, defeating the whole purpose of the special treatment
of str/unicode. If we remove the special treatment this
problem would go away, furthermore __getitem__ returning
objects that are not str/unicode instances wouldn't be
problem any longer.


----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-03 17:36

Message:
Logged In: YES 
user_id=6380

Walter: if you can fix the bug in your latest message here,
go ahead and check it in. Seems like a case of a missing test.

Raymond: it turns out that the iterator in Python 2.2 has
the same problem with lists -- it special-cases lists. But
for tuples, the iterator uses PySequence_GetItem; the fast
tuple iterator in Python 2.3 introduces the problem for
tuples though.

I actually don't think there would be much disagreement that
this behavior (ignoring __getitem__) is a bug. There may be
disagreement over how important it is to fix it. Personally,
I've generally been on the side of "it needn't be fixed if
it slows down the common case", as long as a workaround
(like overriding __iter__ alongside the __getitem__
override) exists.

But I draw the line at being backwards incompatible with
Python 2.2. There fore I think the tuple iterator (and
probably also the string iterator) needs to be fixed, and I
still think that it would be best if the list iterator were
also fixed. One way to do this would be for the tp_iter
implementation to check whether
self->ob_type->tp_as_sequence->sq_item is not equal to the
list_item function (this is a good check to detect a
__getitem__ override) and then return an instance of the
generic sequence iterator instead of the list-specific iterator.

----------------------------------------------------------------------

Comment By: Walter Dörwald (doerwalter)
Date: 2003-01-27 07:24

Message:
Logged In: YES 
user_id=89016

Another problem with filter() is that filterstring() (and
the new filterunicode()) blindly assume that
tp_as_sequence->sq_item returns a str or unicode object with
len==1. This might fail with str or unicode subclasses:
----
class badstr(str):
   def __getitem__(self, index):
      return 42

s = filter(lambda x: x>=42, badstr("1234"))
print len(s), repr(s)
----
This prints
4 '\x00\x00\x00\x00'

----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2003-01-26 20:13

Message:
Logged In: YES 
user_id=80475

One other thought:  A major reason for implementing 
__iter__ in the first place is that objects were overriding 
__getitem__ and disregarding the index -- the __getitem__ 
interface just didn't make sense for iteration in some 
situations.  __iter__ was supposed to provide enormous 
flexibility in various ways to loop over a collection (inorder, 
preorder, postorder, priorityorder, sortedorder, hashorder, 
randomorder, etc).  Making iter() default to using 
__getitem__ was only supposed to be an expedient for 
backwards compatability.  Always using __getitem__ 
diminishes the flexibility and speed advantages.

Maybe the discussion belongs on python-dev. I'm sure a 
number of people feel strongly one way or the other.  The 
question might as well be addressed head-on before 2.3 
goes out the door. 

----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2003-01-26 19:54

Message:
Logged In: YES 
user_id=80475

I understand.  Ideally, *all* methods would respond to a 
single overridden method, but I think this is just a fact of 
life in object oriented programming.

I can't remember where you gave an example of a 
d.__getitem__() subclass override, but you were careful to 
point out that other methods, like d.get() also needed to 
be overridden so that the modified access applied 
everywhere.  Likewise, __iter__() or any other object 
access method must be assumed to access the underlying 
data structure directly and must be overridden.   For 
instance, creating a dictionary with case insensitive 
lookups entails overriding __getitem__(k), get(k,default), 
and pop(k) -- no one of them can be presumed to inform 
the others.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-01-26 19:17

Message:
Logged In: YES 
user_id=6380

Hm... that means that iter() of *amy* built-in type subclass
overriding __getitem__ bypasses the override, unless the
subclass also overrides __iter__. This sounds like a step in
the wrong direction. I think the built-in iterators should
be aware of subclasses overriding __getitem__ one way or
another. I hadn't realized this when we started the trend of
creating faster iterators for built-in types. :-(

----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2003-01-25 11:45

Message:
Logged In: YES 
user_id=80475

None of the existing iterators (incl dicts, lists, tuples, and 
files) use __getitem__.  Most likely, user defined iterators 
also access the data structure directly (for flexiblity and 
speed). Also, anything that uses PyTuple_GET_ITEM 
bypasses __getitem__.

If string/unicode iterators are added, they should also go
directly to the underlying data; otherwise, there is no point 
to it.

Also, the proposal to change filtertuple(), doesn't solve
inconsistencies within filterstring() which uses __getitem__ 
when there is a function call, but bypasses it when the 
function parameter is Py_None.

I think the right answer is to change filterstring() to use an 
iterator and to implement string/unicode iterators that 
access the data directly (not using __getitem__).

FYI for Tim:  MvL noticed and fixed the unicode vs string 
difference.  His patch, SF #636005, has not been applied 
yet.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-01-25 08:51

Message:
Logged In: YES 
user_id=6380

(But in addition th that, I don't mind having a custom
string iterator -- as long as it calls __getitem__ properly.
Hm, shouldn't the tuple iterator call __getitem__ properly too?)


----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2003-01-25 08:45

Message:
Logged In: YES 
user_id=31435

Just noting that filter() is unique in special-casing the type 
of the input.  It's always been surprising that way, and, 
e.g., filtering a string produces a string, but filtering a 
Unicode string produces a list.

map() and reduce() don't play games like that, and always 
use the iteration protocol to march over their inputs.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-01-25 08:36

Message:
Logged In: YES 
user_id=6380

I don't know which Python sources Raymond has been reading,
but in the sources I've got in front of me, there are
special cases for strings and tuples, and these *don't* use
iter(). It so happens that the tuple special-case calls
PyTuple_GetItem(), which doesn't call your __getitem__,
while the string special-case calls the sq_item slot
function, which (in your case) will be a wrapper that calls
your __getitem__.

A minimal fix would be to only call filtertuple for strict
tuples -- although this changes the output type, but I don't
think one should count on filter() of a tuple subclass
returning a tuple (and it can't be made to return an
instance of the subclass either -- we don't know the
constructor signature).

Similar fixes probably need to be made to map() and maybe
reduce().

----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2003-01-24 22:47

Message:
Logged In: YES 
user_id=80475

The problem isn't with filter() which correctly calls iter() in 
both cases.

Tuple object have their own iterator which loops over 
elements directly and has no intervening calls to 
__getitem__().

String objects do not define a custom iterator, so iter() 
wraps itself around consecutive calls to __getitem__().

The resolution is to provide string objects with their own 
iterator. As a side benefit, iteration will run just a tiny bit 
faster.  The same applies to unicode objects.

Guido, do you care about this and want me to fix it or 
would you like to close it as "won't fix".


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=665835&group_id=5470


From noreply@sourceforge.net  Tue Feb  4 21:02:41 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 04 Feb 2003 13:02:41 -0800
Subject: [Python-bugs-list] [ python-Bugs-660144 ] typeobject provides incorrect __mul__
Message-ID: <E18gADJ-00081B-00@sc8-sf-web2.sourceforge.net>

Bugs item #660144, was opened at 2002-12-30 20:29
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=660144&group_id=5470

Category: Python Interpreter Core
Group: None
>Status: Closed
Resolution: Accepted
Priority: 5
Submitted By: Neil Schemenauer (nascheme)
Assigned to: Neil Schemenauer (nascheme)
Summary: typeobject provides incorrect __mul__

Initial Comment:
type __mul__ is wierd:

>> 'a'.__mul__(3.4)
'aaa'
>>> [1].__mul__(3.4)
[1, 1, 1]

Floating point numbers with fractions should not be
accepted.
I think the problem is that __mul__ should not be trying to
implement sq_repeat behavior (although I haven't dug
deeply into this problem yet).  Also, I think the code is
vulnerable to integer overflow.

Should also check __imul__ __add__ __iadd__.

----------------------------------------------------------------------

>Comment By: Neil Schemenauer (nascheme)
Date: 2003-02-04 21:02

Message:
Logged In: YES 
user_id=35752

Checked in as getargs.c 2.96 with NEWS.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-03 22:15

Message:
Logged In: YES 
user_id=6380

Loosk good -- please check it in. Probably requires
something in NEWS too.


----------------------------------------------------------------------

Comment By: Neil Schemenauer (nascheme)
Date: 2003-01-27 05:57

Message:
Logged In: YES 
user_id=35752

Attached is a patch that signals a DepreciationWarning
instead of rasing a TypeError.  I used a static function to
avoid duplication of code.  My version of GCC inlines it.

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-01-26 21:58

Message:
Logged In: YES 
user_id=45365

Neil,
your mod to PyArg_Parse is far too aggressive. Apparently the integer formats have accepted floating point arguments since day one, and all weekend I'm running across places where floats are passed to integers. For example, the MacPython IDE does this all the time, and I wouldn't be surprised if there's lots of windowing code that does this (compute pixel positions in floating point and then passing them to something expecting an integer).

If this is fixed in this way (disallowing floats where integers are expected) it should at least be done with a DeprecationWarning for one release cycle. I think the fix probably is a good idea in the long run, but it will break large bodies of code...

----------------------------------------------------------------------

Comment By: Neil Schemenauer (nascheme)
Date: 2003-01-24 22:20

Message:
Logged In: YES 
user_id=35752

I added the check for the other integer codes.  All the
tests pass
so I checked it in.  Hope that's okay.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-01-21 19:55

Message:
Logged In: YES 
user_id=6380

Looks like a good patch. And yes, the other types should be
doing this as well.

----------------------------------------------------------------------

Comment By: Neil Schemenauer (nascheme)
Date: 2003-01-19 19:35

Message:
Logged In: YES 
user_id=35752

The small patch is attached.  I only added the checking for
'i' and 'l'.  I wonder if it should be done for the other
integer codes as well (e.g. 'b', 'h').

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-01-17 00:19

Message:
Logged In: YES 
user_id=6380

Yes, this should go into 2.3a2.

Your attachment upload failed somehow.

----------------------------------------------------------------------

Comment By: Neil Schemenauer (nascheme)
Date: 2003-01-16 23:10

Message:
Logged In: YES 
user_id=35752

Would this be a change for the 2.3 release?  I tried adding
a PyFloat_Check
test to 'i' and 'l' in getargs.c.  It looks like all the
unit tests pass.  I agree that
checking for float catches the important cases.



----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-01-16 21:57

Message:
Logged In: YES 
user_id=6380

This is an age-old problem that crops up whenever "i" and
similar format codes are used.

The problem is that on the one hand you want "i" to accept
other int-ish types that have an __int__ method. But on the
other hand you don't want it to accept float, which also has
an __int__ method. Or other float-ish types.

I think the "i" format code has to be fixe, but I'm not sure
how -- maybe as a start it would be sufficient to test
explicitly for float and its subclasses and reject those.
That would still allow 3rd party float-ish classes that
implement __int__, but that's not so important.

----------------------------------------------------------------------

Comment By: Neil Schemenauer (nascheme)
Date: 2003-01-16 21:33

Message:
Logged In: YES 
user_id=35752

I think the problem is that wrap_intargfunc and
wrap_intintargfunc use PyArg_ParseTuple(args, "i", &i). 
This bug also is present in methods like __getitem__:

>>> "Python"[1.2]
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
TypeError: sequence index must be integer
>>> "Python".__getitem__(1.2)
'y'

I think the right fix is to use explictly only allow only
ints and longs to wrap_intargfunc and friends.  If Guido
agrees I will cook up a patch.  It seems like we should have
a code for PyParse_Tuple that only allows ints and longs.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=660144&group_id=5470


From noreply@sourceforge.net  Tue Feb  4 21:24:23 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 04 Feb 2003 13:24:23 -0800
Subject: [Python-bugs-list] [ python-Bugs-679880 ] 'compile' refuses BOM.
Message-ID: <E18gAYJ-0000eQ-00@sc8-sf-web2.sourceforge.net>

Bugs item #679880, was opened at 2003-02-03 23:05
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=679880&group_id=5470

Category: Unicode
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Kirill Simonov (kirill_simonov)
Assigned to: Martin v. Löwis (loewis)
Summary: 'compile' refuses BOM.

Initial Comment:
The builtin 'compile' function refuses source files
with UTF-8 BOM marks.

For example,

$ python2.3 testutf8.py

works correctly, but

$ python2.3
>>> compile(open('testutf8.py').read(), 'testutf8.py',
'exec')

fails.



----------------------------------------------------------------------

>Comment By: Kirill Simonov (kirill_simonov)
Date: 2003-02-04 21:24

Message:
Logged In: YES 
user_id=36553

I have uploaded a patch to fix this bug: #680474.
Hope somebody find time to apply it.

----------------------------------------------------------------------

Comment By: M.-A. Lemburg (lemburg)
Date: 2003-02-04 09:18

Message:
Logged In: YES 
user_id=38388

No time for this. Assigning to Martin. Feel free to pass it on.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=679880&group_id=5470


From noreply@sourceforge.net  Tue Feb  4 21:55:33 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 04 Feb 2003 13:55:33 -0800
Subject: [Python-bugs-list] [ python-Bugs-680494 ] filecmp.py uses obsolete statcache module.
Message-ID: <E18gB2T-0003Nn-00@sc8-sf-web4.sourceforge.net>

Bugs item #680494, was opened at 2003-02-04 21:55
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=680494&group_id=5470

Category: Python Library
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Inyeol Lee (inyeol)
Assigned to: Nobody/Anonymous (nobody)
Summary: filecmp.py uses obsolete statcache module.

Initial Comment:
2.3a1 filecmp module, which is not obsolete, uses
obsolete
statcache module and generates DeprecationWarning.

Inyeol Lee

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=680494&group_id=5470


From noreply@sourceforge.net  Wed Feb  5 00:22:00 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 04 Feb 2003 16:22:00 -0800
Subject: [Python-bugs-list] [ python-Bugs-680577 ] urllib2 authentication problem
Message-ID: <E18gDKC-0004F9-00@sc8-sf-web1.sourceforge.net>

Bugs item #680577, was opened at 2003-02-05 00:22
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=680577&group_id=5470

Category: Python Library
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Gary Donovan (gazzadee)
Assigned to: Nobody/Anonymous (nobody)
Summary: urllib2 authentication problem

Initial Comment:
I've found a problem using the authentication in urllib2.

When matching up host-names in order to find a
password, then putting the protocol in the address
makes it seem like a different address. eg...

I create a HTTPBasicAuthHandler with a
HTTPPasswordMgrWithDefaultRealm, and add the tuple
(None, "http://proxy.blah.com:17828", "foo", "bar") to it.

I then setup the proxy to use
http://proxy.blah.com:17828 (which requires
authentication).

When I connect, the password lookup fails, because it
is trying to find a match for "proxy.blah.com:17828"
rather than "http://proxy.blah.com:17828"

This problem doesn't exist if I pass
"proxy.blah.com:17828" to the password manager.

There seems to be some stuff in HTTPPasswordMgr to deal
with variations on site names, but I guess it's not
working in this case (unless this is intentional).

Version Info:
Python 2.2 (#1, Feb 24 2002, 16:21:58)
[GCC 2.96 20000731 (Mandrake Linux 8.2 2.96-0.76mdk)]
on linux-i386


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=680577&group_id=5470


From noreply@sourceforge.net  Wed Feb  5 01:56:03 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 04 Feb 2003 17:56:03 -0800
Subject: [Python-bugs-list] [ python-Bugs-672491 ] 2.3a1 computes lastindex incorrectly
Message-ID: <E18gEnD-0003Gn-00@sc8-sf-web3.sourceforge.net>

Bugs item #672491, was opened at 2003-01-22 06:28
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=672491&group_id=5470

Category: Regular Expressions
Group: Python 2.3
Status: Open
Resolution: None
Priority: 6
Submitted By: Martin v. Löwis (loewis)
Assigned to: Fredrik Lundh (effbot)
Summary: 2.3a1 computes lastindex incorrectly

Initial Comment:
In Python 2.[012], the code

import re
exp = re.compile("(?P<NCName>[a-zA-Z_](\w|[_.-])*)")
print exp.match("namespace").lastgroup

prints "NCName". In Python 2.3a1, it prints "None". The
problem is that last index is 2, instead of 1, as it
should be.

----------------------------------------------------------------------

Comment By: Greg Chapman (glchapman)
Date: 2003-02-04 16:56

Message:
Logged In: YES 
user_id=86307

I believe the discrepancy was deliberately introduced in revision 2.84 of _sre.c.  I agree with you that lastindex should return the the index of the matching group with the rightmost closing parenthesis (perhaps some elaboration in the docs is also in order).  If this is the correct interpretation, two places need to be patched: 1) the handling of SRE_OP_MARK needs to be reverted to the 2.22 code and 2) the code in the lastmark_restore function needs to be tweaked so that lastindex is not accidentally set to the last matched group entered.

Thinking further though, given a (contrived) pattern like this: 

  re.match('((x))y(?:(a)b|ac)', 'xyac')

what should lastindex be?  I assume 1, given the definition above (lastindex = matching group with rightmost close parens).  In 2.22 it is 3, since group 3 matched before the branch failed at the 'b'.  In 2.3a1 it is 2, since lastindex is restored (after the branch fails) using the saved lastmark.  Anyway, if it should be 1, then I think _sre.c will have to save lastindex as well as lastmark when processing the three opcodes which may end up calling lastmark_restore.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=672491&group_id=5470


From noreply@sourceforge.net  Wed Feb  5 02:35:24 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 04 Feb 2003 18:35:24 -0800
Subject: [Python-bugs-list] [ python-Bugs-674264 ] re.compile fails for verbose re with more than one group
Message-ID: <E18gFPI-0008KO-00@sc8-sf-web1.sourceforge.net>

Bugs item #674264, was opened at 2003-01-24 11:36
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=674264&group_id=5470

Category: Regular Expressions
Group: Python 2.2.1
Status: Open
Resolution: None
Priority: 5
Submitted By: Garth Corral (gcorral)
Assigned to: Fredrik Lundh (effbot)
Summary: re.compile fails for verbose re with more than one group

Initial Comment:
The third of the following three regular expressions
fails to compile with "unbalanced parenthesis" error:


[foo.py]

import re

_good_re =
re.compile(r'\s*(elif|else|except|finally).*:\s*(#.*)$')

_also_good_re = re.compile(r'''
    \s*
    ( elif
    | else
    | except
    | finally
    )
    .*:
''', re.VERBOSE)

_bad_re = re.compile(r'''
    \s*
    ( elif
    | else
    | except
    | finally
    )
    .*:\s*(#.*)?$
''', re.VERBOSE)


>>> import foo
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "zzz.py", line 15, in ?
    _bad_re = re.compile(r'''
  File "/usr/local/lib/python2.2/sre.py", line 178, in
compile
    return _compile(pattern, flags)
  File "/usr/local/lib/python2.2/sre.py", line 228, in
_compile
    raise error, v # invalid expression
sre_constants.error: unbalanced parenthesis


----------------------------------------------------------------------

Comment By: Greg Chapman (glchapman)
Date: 2003-02-04 17:35

Message:
Logged In: YES 
user_id=86307

The problem here is the '#' character in the third pattern.  When using re.VERBOSE, it is taken as beginning a comment (unless preceeded by a backslash).  Change that line to:

.*:\s*(\#.*)?$

and the pattern will compile.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=674264&group_id=5470


From noreply@sourceforge.net  Wed Feb  5 02:39:14 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 04 Feb 2003 18:39:14 -0800
Subject: [Python-bugs-list] [ python-Bugs-672115 ] Assignment to __bases__ of direct object subclasses
Message-ID: <E18gFT0-0008Rp-00@sc8-sf-web1.sourceforge.net>

Bugs item #672115, was opened at 2003-01-21 13:45
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=672115&group_id=5470

Category: Python Interpreter Core
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Greg Chapman (glchapman)
Assigned to: Michael Hudson (mwh)
Summary: Assignment to __bases__ of direct object subclasses

Initial Comment:
I'm not entirely sure this is a bug, but I think it is 
surprising:

Python 2.3a1 (#38, Dec 31 2002, 17:53:59) [MSC 
v.1200 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more 
information.
>>> class A(object):
...     pass
...
>>> class B(object):
...     pass
...
>>> B.__bases__ = (A,)
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
TypeError: __bases__ assignment: 'A' deallocator differs 
from 'object'

It seems like you should be able to change the 
__bases__ of a new-style class (implemented in 
Python) which inherits directly from object to another 
new-style class.  (Will the deallocator issue ever come 
into play if the only types involved are HEAPTYPES and 
object as the ultimate base?)


----------------------------------------------------------------------

>Comment By: Greg Chapman (glchapman)
Date: 2003-02-04 17:39

Message:
Logged In: YES 
user_id=86307

Well, I wrote a small patch which I'm attaching.  However, I 
can't say that I'm partcularly confident in it.  It works for the 
simple cases I've  been able to think of (and for 
test_descr.py), but looking at typeobject.c, I get the nagging 
feeling that a lot more tests are required to be sure it's OK.  
The problem is, I'm just not sure what they (the tests) are.


----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2003-01-31 09:09

Message:
Logged In: YES 
user_id=6656

Are you interested in working up a patch for this?  Hacking
this kind of stuff requires motivation I'm not sure I can
drum up in time for 2.3...

----------------------------------------------------------------------

Comment By: Greg Chapman (glchapman)
Date: 2003-01-23 08:03

Message:
Logged In: YES 
user_id=86307

Sorry about the parenthetical comment; I think what I was trying 
to say is basically what you have in your last paragraph.

As for use cases, I don't have any myself (I ran into this with 
some test code for a metaclass which "overrides" __bases__).  
However, grepping through the standard library, I note that one 
place where assignment to __bases__ is used is in 
xmlrpclib.SlowParser.  It appears to me that if SlowParser and 
xmllib.XMLParser (neither of which has a base class) were 
converted to new-style classes, the assignment to __bases__ 
would generate this exception.  Of course, that shouldn't be too 
hard to work around if that turns out to be necessary.



----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2003-01-22 02:50

Message:
Logged In: YES 
user_id=6656

I agree this is a bit surprising.  When I was writing this
code I went for the conservative-as-possible approach as I
didn't want to introduce instabilities to Python.

It certainly may be that I've overdone it.  In this case I
probably have; if the tp_dealloc of the class being adjusted
is subtype_dealloc and the tp_dealloc that ultimately gets
invoked is the same we're probably OK.  But I'm not sure and
it's been a while since I thought about this.

It also happens that my motivating use-case for this isn't
troubled by this restriction.

I don't understand your last, parenthetical, comment. 
HEAPTYPES as such doesn't come into it, does it?  

You might be right that we don't need to worry about
tp_dealloc if the ultimate solid_base doesn't change and all
the tp_deallocs on the way there are subtype_dealloc...

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=672115&group_id=5470


From noreply@sourceforge.net  Wed Feb  5 01:07:33 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 04 Feb 2003 17:07:33 -0800
Subject: [Python-bugs-list] [ python-Bugs-576079 ] Inconsistent behaviour in re grouping
Message-ID: <E18gE2H-0001fJ-00@sc8-sf-web3.sourceforge.net>

Bugs item #576079, was opened at 2002-07-01 18:22
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=576079&group_id=5470

Category: Regular Expressions
Group: None
Status: Open
Resolution: Accepted
Priority: 5
Submitted By: Pedro Rodriguez (pedro_rodriguez)
Assigned to: Fredrik Lundh (effbot)
Summary: Inconsistent behaviour in re grouping

Initial Comment:
The following expression (?P<name>.*) and
(?P<name>(.*)) don't behave in the same way. 

When the matching fails, the first group will be None,
but the last one will contain an empty string.

The problem occurs with python 2.1.1 and 2.2. (and
latest CVS for 2.3)

Python 1.5.2 OTH works fine.


(example file attached)

----------------------------------------------------------------------

Comment By: Michael Stone (mbrierst)
Date: 2003-02-05 01:07

Message:
Logged In: YES 
user_id=670441

This was fixed when patch 527371 was checked in.  Someone close this bug.

----------------------------------------------------------------------

Comment By: Fredrik Lundh (effbot)
Date: 2002-07-12 11:09

Message:
Logged In: YES 
user_id=38376

Makes sense to me.  I'll check it in soonish.

Thanks /F

----------------------------------------------------------------------

Comment By: Pedro Rodriguez (pedro_rodriguez)
Date: 2002-07-09 07:20

Message:
Logged In: YES 
user_id=426450

Greg,

I tried your patch and it fixes the problems as Tim and
myself reported them.

If each time the 'lastmark' field is set to a value which is
less than the current one, the 'memset' operation should be
performed, your patch makes sense IMO, and is consistent
with other places in the code.

----------------------------------------------------------------------

Comment By: Greg Chapman (glchapman)
Date: 2002-07-07 16:09

Message:
Logged In: YES 
user_id=86307

I believe this is another example of the bug at which Patch 
527371 was aimed.  With that patch applied to the 2.2.1 
_sre.c, I get this:

>>> pat2 = re.compile(r"(((.*))x)?(y)")
>>> print pat2.match('y').groups()
(None, None, None, 'y')

I see that the patch is marked as accepted, but it does not 
yet appear to have made it into _sre.c even in CVS (at least 
it's not in version 2.80).  Perhaps this is an oversight?


----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2002-07-01 18:36

Message:
Logged In: YES 
user_id=31435

Here's a simpler example:

import re
pat1 = re.compile(r"((.*)x)?(y)")
pat2 = re.compile(r"(((.*))x)?(y)")
print pat1.match('y').groups()
print pat2.match('y').groups()

That prints

(None, None, 'y')
(None, '', None, 'y')

If (y) in the regexps is changed to plain y:

pat1 = re.compile(r"((.*)x)?y")
pat2 = re.compile(r"(((.*))x)?y")
print pat1.match('y').groups()
print pat2.match('y').groups()

the output changes to (the expected):

(None, None)
(None, None, None)

So it's not *just* the extra level of parens -- whether there's 
a capturing group "to the right" also affects the outcome.

FWIW, I agree it's a buglet.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=576079&group_id=5470


From noreply@sourceforge.net  Wed Feb  5 08:06:34 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Wed, 05 Feb 2003 00:06:34 -0800
Subject: [Python-bugs-list] [ python-Bugs-576079 ] Inconsistent behaviour in re grouping
Message-ID: <E18gKZm-0005st-00@sc8-sf-web4.sourceforge.net>

Bugs item #576079, was opened at 2002-07-01 20:22
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=576079&group_id=5470

Category: Regular Expressions
Group: None
>Status: Closed
Resolution: Accepted
Priority: 5
Submitted By: Pedro Rodriguez (pedro_rodriguez)
Assigned to: Fredrik Lundh (effbot)
Summary: Inconsistent behaviour in re grouping

Initial Comment:
The following expression (?P<name>.*) and
(?P<name>(.*)) don't behave in the same way. 

When the matching fails, the first group will be None,
but the last one will contain an empty string.

The problem occurs with python 2.1.1 and 2.2. (and
latest CVS for 2.3)

Python 1.5.2 OTH works fine.


(example file attached)

----------------------------------------------------------------------

>Comment By: Fredrik Lundh (effbot)
Date: 2003-02-05 09:06

Message:
Logged In: YES 
user_id=38376

Done.  Thanks /F

----------------------------------------------------------------------

Comment By: Michael Stone (mbrierst)
Date: 2003-02-05 02:07

Message:
Logged In: YES 
user_id=670441

This was fixed when patch 527371 was checked in.  Someone close this bug.

----------------------------------------------------------------------

Comment By: Fredrik Lundh (effbot)
Date: 2002-07-12 13:09

Message:
Logged In: YES 
user_id=38376

Makes sense to me.  I'll check it in soonish.

Thanks /F

----------------------------------------------------------------------

Comment By: Pedro Rodriguez (pedro_rodriguez)
Date: 2002-07-09 09:20

Message:
Logged In: YES 
user_id=426450

Greg,

I tried your patch and it fixes the problems as Tim and
myself reported them.

If each time the 'lastmark' field is set to a value which is
less than the current one, the 'memset' operation should be
performed, your patch makes sense IMO, and is consistent
with other places in the code.

----------------------------------------------------------------------

Comment By: Greg Chapman (glchapman)
Date: 2002-07-07 18:09

Message:
Logged In: YES 
user_id=86307

I believe this is another example of the bug at which Patch 
527371 was aimed.  With that patch applied to the 2.2.1 
_sre.c, I get this:

>>> pat2 = re.compile(r"(((.*))x)?(y)")
>>> print pat2.match('y').groups()
(None, None, None, 'y')

I see that the patch is marked as accepted, but it does not 
yet appear to have made it into _sre.c even in CVS (at least 
it's not in version 2.80).  Perhaps this is an oversight?


----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2002-07-01 20:36

Message:
Logged In: YES 
user_id=31435

Here's a simpler example:

import re
pat1 = re.compile(r"((.*)x)?(y)")
pat2 = re.compile(r"(((.*))x)?(y)")
print pat1.match('y').groups()
print pat2.match('y').groups()

That prints

(None, None, 'y')
(None, '', None, 'y')

If (y) in the regexps is changed to plain y:

pat1 = re.compile(r"((.*)x)?y")
pat2 = re.compile(r"(((.*))x)?y")
print pat1.match('y').groups()
print pat2.match('y').groups()

the output changes to (the expected):

(None, None)
(None, None, None)

So it's not *just* the extra level of parens -- whether there's 
a capturing group "to the right" also affects the outcome.

FWIW, I agree it's a buglet.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=576079&group_id=5470


From noreply@sourceforge.net  Wed Feb  5 11:00:07 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Wed, 05 Feb 2003 03:00:07 -0800
Subject: [Python-bugs-list] [ python-Bugs-680789 ] Debug with long array takes forever
Message-ID: <E18gNHj-0004fX-00@sc8-sf-web2.sourceforge.net>

Bugs item #680789, was opened at 2003-02-05 11:00
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=680789&group_id=5470

Category: Python Library
Group: Python 2.2
Status: Open
Resolution: None
Priority: 5
Submitted By: Jurjen N.E. Bos (jneb)
Assigned to: Nobody/Anonymous (nobody)
Summary: Debug with long array takes forever

Initial Comment:
This is a bug and a partial patch.

If I debug a program that contains a ridiculously large array (8M entries in my case), the debugger takes forever.
It happens in Mac OS X, Python 2.2, but I found the bug in is the repr module, so it is probably universal.
The thing is, that after the fix below, it still doesn't work!
Did I miss something trivial (like repr is builtin, or something like that?). Would someone with Mac OS X experience help out here, please (Jack?).

Here's the diff to make repr.repr work with large arrays:
13a14
>       self.maxarray = 5
50a52,62
>     def repr_array(self, x, level):
>         n = len(x)
>       header = "array('"+x.typecode+"', ["
>         if n == 0: return header+"])"
>         if level <= 0: return header+"...])"
>         s = ''
>         for i in range(min(n, self.maxarray)):
>             if s: s = s + ', '
>             s = s + self.repr1(x[i], level-1)
>         if n > self.maxarray: s = s + ', ...'
>         return header + s + "])"

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=680789&group_id=5470


From noreply@sourceforge.net  Wed Feb  5 11:12:03 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Wed, 05 Feb 2003 03:12:03 -0800
Subject: [Python-bugs-list] [ python-Bugs-680797 ] expat doesn't build on MacOS9
Message-ID: <E18gNTH-0005UG-00@sc8-sf-web4.sourceforge.net>

Bugs item #680797, was opened at 2003-02-05 12:11
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=680797&group_id=5470

Category: Extension Modules
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Jack Jansen (jackjansen)
Assigned to: Martin v. Löwis (loewis)
Summary: expat doesn't build on MacOS9

Initial Comment:
Expat doesn't build anymore on Mac OS 9. The problem is that the new expat.h has a couple of functions returning an "enum XML_Status", but that enum is declared only further down in expat.h and the Metrowerks compiler chokes on this.

It's easy enough to fix (either moving the routines down or moving the enum declaration up), but as I seem to recall that this file is imported from the expat distribution I'm not sure whether that's a good idea, or something else needs to be done.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=680797&group_id=5470


From noreply@sourceforge.net  Wed Feb  5 13:15:49 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Wed, 05 Feb 2003 05:15:49 -0800
Subject: [Python-bugs-list] [ python-Bugs-680864 ] test_datetime fails for non-unix epoch
Message-ID: <E18gPP3-0005G5-00@sc8-sf-web1.sourceforge.net>

Bugs item #680864, was opened at 2003-02-05 14:15
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=680864&group_id=5470

Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Jack Jansen (jackjansen)
Assigned to: Tim Peters (tim_one)
Summary: test_datetime fails for non-unix epoch

Initial Comment:
test_datetime fails on Mac OS 9 in test_tzinfo_fromtimestamp, it seems to be depending on the Unix epoch of 1970:

timestamp = 1000000000  #  2001-09-09 01:46:40 UTC, give or take

The mac epoch is 1-jan-1904, localtime (sic!).

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=680864&group_id=5470


From noreply@sourceforge.net  Wed Feb  5 14:10:48 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Wed, 05 Feb 2003 06:10:48 -0800
Subject: [Python-bugs-list] [ python-Bugs-680429 ] __module__ broken for extension classes
Message-ID: <E18gQGG-0008VV-00@sc8-sf-web1.sourceforge.net>

Bugs item #680429, was opened at 2003-02-04 19:55
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=680429&group_id=5470

Category: Python Interpreter Core
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Ralf W. Grosse-Kunstleve (rwgk)
Assigned to: Nobody/Anonymous (nobody)
Summary: __module__ broken for extension classes

Initial Comment:
We are having problems using Boost.Python with Python 
2.3a1 because
under some circumstances the result of __module__ is 
different compared
to earlier Python versions:

Python 2.2.1 (#2, Jun 17 2002, 12:06:51) 
[GCC 2.96 20000731 (Red Hat Linux 7.3 2.96-110)] on 
linux2
Type "help", "copyright", "credits" or "license" for more 
information.
>>> import simple
>>> simple.empty.__module__
'simple'
>>> 

Python 2.3a1 (#1, Jan  6 2003, 14:17:56) 
[GCC 2.96 20000731 (Red Hat Linux 7.3 2.96-110)] on 
linux2
Type "help", "copyright", "credits" or "license" for more 
information.
>>> import simple
>>> simple.empty.__module__
'__main__'
>>> 

Because of this we can no longer pickle our extension 
classes.
For your reference the code for the simple module is 
attached.
This is using Boost release 1.29.0 (www.boost.org).

We have done some debugging. Boost.Python's internal 
idea of the module
associated with an extension class is still correct even when 
using
Python 2.3a1. David Abrahams (main Boost.Python author) 
is telling me
that he "changed Boost.Python to work the way Guido 
suggested before
2.2.2." Therefore we suspect that the __module__ problem 
is due to
a change/bug in Python 2.3a1.

Ralf


#include <boost/python/module.hpp>
#include <boost/python/class.hpp>

namespace sandbx { namespace {

  struct empty {};

  void init_module()
  {
    using namespace boost::python;
    class_<empty>("empty");
  }

}} // namespace sandbx::<anonymous>

BOOST_PYTHON_MODULE(simple)
{
  sandbx::init_module();
}


----------------------------------------------------------------------

>Comment By: Michael Hudson (mwh)
Date: 2003-02-05 14:10

Message:
Logged In: YES 
user_id=6656

There has certainly been a change here.

Look at typeobject.c:type_module() (about 100 lines in).

If you can tell me what you want that function to do, I can
have a go at it.

Is simple.empty a HEAPTYPE?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=680429&group_id=5470


From noreply@sourceforge.net  Wed Feb  5 14:12:57 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Wed, 05 Feb 2003 06:12:57 -0800
Subject: [Python-bugs-list] [ python-Bugs-674264 ] re.compile fails for verbose re with more than one group
Message-ID: <E18gQIL-0000Ar-00@sc8-sf-web1.sourceforge.net>

Bugs item #674264, was opened at 2003-01-24 20:36
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=674264&group_id=5470

Category: Regular Expressions
Group: Python 2.2.1
>Status: Closed
>Resolution: Invalid
Priority: 5
Submitted By: Garth Corral (gcorral)
Assigned to: Fredrik Lundh (effbot)
Summary: re.compile fails for verbose re with more than one group

Initial Comment:
The third of the following three regular expressions
fails to compile with "unbalanced parenthesis" error:


[foo.py]

import re

_good_re =
re.compile(r'\s*(elif|else|except|finally).*:\s*(#.*)$')

_also_good_re = re.compile(r'''
    \s*
    ( elif
    | else
    | except
    | finally
    )
    .*:
''', re.VERBOSE)

_bad_re = re.compile(r'''
    \s*
    ( elif
    | else
    | except
    | finally
    )
    .*:\s*(#.*)?$
''', re.VERBOSE)


>>> import foo
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "zzz.py", line 15, in ?
    _bad_re = re.compile(r'''
  File "/usr/local/lib/python2.2/sre.py", line 178, in
compile
    return _compile(pattern, flags)
  File "/usr/local/lib/python2.2/sre.py", line 228, in
_compile
    raise error, v # invalid expression
sre_constants.error: unbalanced parenthesis


----------------------------------------------------------------------

>Comment By: Michael Hudson (mwh)
Date: 2003-02-05 14:12

Message:
Logged In: YES 
user_id=6656

Ta.

----------------------------------------------------------------------

Comment By: Greg Chapman (glchapman)
Date: 2003-02-05 02:35

Message:
Logged In: YES 
user_id=86307

The problem here is the '#' character in the third pattern.  When using re.VERBOSE, it is taken as beginning a comment (unless preceeded by a backslash).  Change that line to:

.*:\s*(\#.*)?$

and the pattern will compile.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=674264&group_id=5470


From noreply@sourceforge.net  Wed Feb  5 14:22:35 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Wed, 05 Feb 2003 06:22:35 -0800
Subject: [Python-bugs-list] [ python-Bugs-672115 ] Assignment to __bases__ of direct object subclasses
Message-ID: <E18gQRf-0005iv-00@sc8-sf-web2.sourceforge.net>

Bugs item #672115, was opened at 2003-01-21 22:45
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=672115&group_id=5470

Category: Python Interpreter Core
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Greg Chapman (glchapman)
Assigned to: Michael Hudson (mwh)
Summary: Assignment to __bases__ of direct object subclasses

Initial Comment:
I'm not entirely sure this is a bug, but I think it is 
surprising:

Python 2.3a1 (#38, Dec 31 2002, 17:53:59) [MSC 
v.1200 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more 
information.
>>> class A(object):
...     pass
...
>>> class B(object):
...     pass
...
>>> B.__bases__ = (A,)
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
TypeError: __bases__ assignment: 'A' deallocator differs 
from 'object'

It seems like you should be able to change the 
__bases__ of a new-style class (implemented in 
Python) which inherits directly from object to another 
new-style class.  (Will the deallocator issue ever come 
into play if the only types involved are HEAPTYPES and 
object as the ultimate base?)


----------------------------------------------------------------------

>Comment By: Michael Hudson (mwh)
Date: 2003-02-05 14:22

Message:
Logged In: YES 
user_id=6656

Mhmm.  That looks OK to me (after realizing that solid_base
worries about __slots__).

But I don't know how one can be sure :-(

----------------------------------------------------------------------

Comment By: Greg Chapman (glchapman)
Date: 2003-02-05 02:39

Message:
Logged In: YES 
user_id=86307

Well, I wrote a small patch which I'm attaching.  However, I 
can't say that I'm partcularly confident in it.  It works for the 
simple cases I've  been able to think of (and for 
test_descr.py), but looking at typeobject.c, I get the nagging 
feeling that a lot more tests are required to be sure it's OK.  
The problem is, I'm just not sure what they (the tests) are.


----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2003-01-31 18:09

Message:
Logged In: YES 
user_id=6656

Are you interested in working up a patch for this?  Hacking
this kind of stuff requires motivation I'm not sure I can
drum up in time for 2.3...

----------------------------------------------------------------------

Comment By: Greg Chapman (glchapman)
Date: 2003-01-23 17:03

Message:
Logged In: YES 
user_id=86307

Sorry about the parenthetical comment; I think what I was trying 
to say is basically what you have in your last paragraph.

As for use cases, I don't have any myself (I ran into this with 
some test code for a metaclass which "overrides" __bases__).  
However, grepping through the standard library, I note that one 
place where assignment to __bases__ is used is in 
xmlrpclib.SlowParser.  It appears to me that if SlowParser and 
xmllib.XMLParser (neither of which has a base class) were 
converted to new-style classes, the assignment to __bases__ 
would generate this exception.  Of course, that shouldn't be too 
hard to work around if that turns out to be necessary.



----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2003-01-22 11:50

Message:
Logged In: YES 
user_id=6656

I agree this is a bit surprising.  When I was writing this
code I went for the conservative-as-possible approach as I
didn't want to introduce instabilities to Python.

It certainly may be that I've overdone it.  In this case I
probably have; if the tp_dealloc of the class being adjusted
is subtype_dealloc and the tp_dealloc that ultimately gets
invoked is the same we're probably OK.  But I'm not sure and
it's been a while since I thought about this.

It also happens that my motivating use-case for this isn't
troubled by this restriction.

I don't understand your last, parenthetical, comment. 
HEAPTYPES as such doesn't come into it, does it?  

You might be right that we don't need to worry about
tp_dealloc if the ultimate solid_base doesn't change and all
the tp_deallocs on the way there are subtype_dealloc...

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=672115&group_id=5470


From noreply@sourceforge.net  Wed Feb  5 14:34:21 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Wed, 05 Feb 2003 06:34:21 -0800
Subject: [Python-bugs-list] [ python-Bugs-680429 ] __module__ broken for extension classes
Message-ID: <E18gQd3-0005au-00@sc8-sf-web4.sourceforge.net>

Bugs item #680429, was opened at 2003-02-04 19:55
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=680429&group_id=5470

Category: Python Interpreter Core
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Ralf W. Grosse-Kunstleve (rwgk)
>Assigned to: Michael Hudson (mwh)
Summary: __module__ broken for extension classes

Initial Comment:
We are having problems using Boost.Python with Python 
2.3a1 because
under some circumstances the result of __module__ is 
different compared
to earlier Python versions:

Python 2.2.1 (#2, Jun 17 2002, 12:06:51) 
[GCC 2.96 20000731 (Red Hat Linux 7.3 2.96-110)] on 
linux2
Type "help", "copyright", "credits" or "license" for more 
information.
>>> import simple
>>> simple.empty.__module__
'simple'
>>> 

Python 2.3a1 (#1, Jan  6 2003, 14:17:56) 
[GCC 2.96 20000731 (Red Hat Linux 7.3 2.96-110)] on 
linux2
Type "help", "copyright", "credits" or "license" for more 
information.
>>> import simple
>>> simple.empty.__module__
'__main__'
>>> 

Because of this we can no longer pickle our extension 
classes.
For your reference the code for the simple module is 
attached.
This is using Boost release 1.29.0 (www.boost.org).

We have done some debugging. Boost.Python's internal 
idea of the module
associated with an extension class is still correct even when 
using
Python 2.3a1. David Abrahams (main Boost.Python author) 
is telling me
that he "changed Boost.Python to work the way Guido 
suggested before
2.2.2." Therefore we suspect that the __module__ problem 
is due to
a change/bug in Python 2.3a1.

Ralf


#include <boost/python/module.hpp>
#include <boost/python/class.hpp>

namespace sandbx { namespace {

  struct empty {};

  void init_module()
  {
    using namespace boost::python;
    class_<empty>("empty");
  }

}} // namespace sandbx::<anonymous>

BOOST_PYTHON_MODULE(simple)
{
  sandbx::init_module();
}


----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2003-02-05 14:10

Message:
Logged In: YES 
user_id=6656

There has certainly been a change here.

Look at typeobject.c:type_module() (about 100 lines in).

If you can tell me what you want that function to do, I can
have a go at it.

Is simple.empty a HEAPTYPE?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=680429&group_id=5470


From noreply@sourceforge.net  Wed Feb  5 15:52:28 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Wed, 05 Feb 2003 07:52:28 -0800
Subject: [Python-bugs-list] [ python-Bugs-681003 ] itertools issues
Message-ID: <E18gRqe-0002fm-00@sc8-sf-web2.sourceforge.net>

Bugs item #681003, was opened at 2003-02-05 06:52
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=681003&group_id=5470

Category: Python Library
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Greg Chapman (glchapman)
Assigned to: Nobody/Anonymous (nobody)
Summary: itertools issues

Initial Comment:
I've been checking out the new itertools module.  It's a great 
addition; I look forward to using it.  However, I did notice a 
few things:

1) I believe islice should worry about overflow when it adds 
step to next.  Given a call like islice(seq, 1, 10, sys.maxint), 
it looks like islice will not stop after retrieving item 1 
because this:

  lz->next += lz->step;

will result in lz->next becoming negative.

2) I note that starmap_next passes the args obtained from 
PyIter_Next directly to PyObject_Call.  PyObject_Call 
assumes that its arg param is a tuple (and so do the various 
tp_call slots).  However, there is nothing to ensure that the 
args object obtained from the iterator in starmap_next is in 
fact a tuple.  I believe either a PyTuple_Check or a 
PySequence_Tuple call is in order before passing the args 
to PyObject_Call.

3) I don't think it's safe for imap_next to reuse the same 
argtuple for all the calls.  This tuple is passed directly to C 
functions, and there's nothing to stop them from saving a 
reference to it someplace (unless it's documented 
somewhere that you shouldn't save references to the args 
tuple).

4) I think count_next should worry about overflow also 
(consider count(sys.maxint)).


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=681003&group_id=5470


From noreply@sourceforge.net  Wed Feb  5 15:56:36 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Wed, 05 Feb 2003 07:56:36 -0800
Subject: [Python-bugs-list] [ python-Bugs-681003 ] itertools issues
Message-ID: <E18gRue-0005jT-00@sc8-sf-web1.sourceforge.net>

Bugs item #681003, was opened at 2003-02-05 15:52
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=681003&group_id=5470

Category: Python Library
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Greg Chapman (glchapman)
>Assigned to: Raymond Hettinger (rhettinger)
Summary: itertools issues

Initial Comment:
I've been checking out the new itertools module.  It's a great 
addition; I look forward to using it.  However, I did notice a 
few things:

1) I believe islice should worry about overflow when it adds 
step to next.  Given a call like islice(seq, 1, 10, sys.maxint), 
it looks like islice will not stop after retrieving item 1 
because this:

  lz->next += lz->step;

will result in lz->next becoming negative.

2) I note that starmap_next passes the args obtained from 
PyIter_Next directly to PyObject_Call.  PyObject_Call 
assumes that its arg param is a tuple (and so do the various 
tp_call slots).  However, there is nothing to ensure that the 
args object obtained from the iterator in starmap_next is in 
fact a tuple.  I believe either a PyTuple_Check or a 
PySequence_Tuple call is in order before passing the args 
to PyObject_Call.

3) I don't think it's safe for imap_next to reuse the same 
argtuple for all the calls.  This tuple is passed directly to C 
functions, and there's nothing to stop them from saving a 
reference to it someplace (unless it's documented 
somewhere that you shouldn't save references to the args 
tuple).

4) I think count_next should worry about overflow also 
(consider count(sys.maxint)).


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=681003&group_id=5470


From noreply@sourceforge.net  Wed Feb  5 17:31:53 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Wed, 05 Feb 2003 09:31:53 -0800
Subject: [Python-bugs-list] [ python-Bugs-580107 ] Subclassing WeakValueDictionary impossib
Message-ID: <E18gTOr-0003Sl-00@sc8-sf-web3.sourceforge.net>

Bugs item #580107, was opened at 2002-07-11 16:04
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=580107&group_id=5470

Category: Python Interpreter Core
Group: None
>Status: Closed
Resolution: None
Priority: 5
Submitted By: Thomas Heller (theller)
Assigned to: Nobody/Anonymous (nobody)
Summary: Subclassing WeakValueDictionary impossib

Initial Comment:
I'm trying to subclass weakref.WeakValueDictionary, so 
that it can send notifications when keys are removed.

I subclassed it and implemented a new __delitem__ 
method, but unfortunately this is not called because 
WeakValueDictionary manipulates UserDict's data 
member directly (in __makeremove), without calling the 
__delitem__ method.

Inspection of the sources shows that this is also the 
case in other methods as well...

I finally overrided __makeremovemethod _and_ 
__delitem__ ;-(

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=580107&group_id=5470


From noreply@sourceforge.net  Wed Feb  5 17:32:54 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Wed, 05 Feb 2003 09:32:54 -0800
Subject: [Python-bugs-list] [ python-Bugs-638595 ] Move bdist_wininst source code into PC
Message-ID: <E18gTPq-0003VO-00@sc8-sf-web3.sourceforge.net>

Bugs item #638595, was opened at 2002-11-14 20:30
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=638595&group_id=5470

Category: Distutils
Group: Python 2.3
>Status: Closed
>Resolution: Fixed
Priority: 5
Submitted By: Guido van Rossum (gvanrossum)
Assigned to: Thomas Heller (theller)
Summary: Move bdist_wininst source code into PC

Initial Comment:
Please move the source code for the binary that's used
by bdist_wininst into the distributed source subtree,
preferably into the PC directory.

(Per python-dev.)

----------------------------------------------------------------------

>Comment By: Thomas Heller (theller)
Date: 2003-02-05 18:32

Message:
Logged In: YES 
user_id=11105

Done, although I've not 'cvs remove'd the files from the 
distutils CVS rep.

----------------------------------------------------------------------

Comment By: Thomas Heller (theller)
Date: 2002-11-22 22:11

Message:
Logged In: YES 
user_id=11105

The move is done, I will close this report when I have verified 
that everything works and I have 'cvs remove'd the files in the 
distutils CVS repository.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2002-11-22 20:30

Message:
Logged In: YES 
user_id=6380

Good idea...

----------------------------------------------------------------------

Comment By: Thomas Heller (theller)
Date: 2002-11-22 20:26

Message:
Logged In: YES 
user_id=11105

The source files are PythonPowered.bmp, archive.h, 
extract.c, install.c, install.rc, resource.h, wininst.dsw, 
wininst.dsp.

I assume a bdist_wininst subdirctory of PC is in order?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=638595&group_id=5470


From noreply@sourceforge.net  Wed Feb  5 18:23:25 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Wed, 05 Feb 2003 10:23:25 -0800
Subject: [Python-bugs-list] [ python-Bugs-680494 ] filecmp.py uses obsolete statcache module.
Message-ID: <E18gUCj-0005bM-00@sc8-sf-web3.sourceforge.net>

Bugs item #680494, was opened at 2003-02-04 16:55
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=680494&group_id=5470

Category: Python Library
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Inyeol Lee (inyeol)
Assigned to: Nobody/Anonymous (nobody)
Summary: filecmp.py uses obsolete statcache module.

Initial Comment:
2.3a1 filecmp module, which is not obsolete, uses
obsolete
statcache module and generates DeprecationWarning.

Inyeol Lee

----------------------------------------------------------------------

>Comment By: A.M. Kuchling (akuchling)
Date: 2003-02-05 13:23

Message:
Logged In: YES 
user_id=11375

The simplest thing would be to simply use os.stat instead of statcache,
and deprecate use_statcache by logging a warning if use_statcache is true.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=680494&group_id=5470


From noreply@sourceforge.net  Wed Feb  5 18:24:47 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Wed, 05 Feb 2003 10:24:47 -0800
Subject: [Python-bugs-list] [ python-Bugs-660455 ] -hex/oct const generates wrong code
Message-ID: <E18gUE3-0003Af-00@sc8-sf-web2.sourceforge.net>

Bugs item #660455, was opened at 2002-12-31 13:11
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=660455&group_id=5470

Category: Parser/Compiler
Group: Python 2.2
Status: Open
Resolution: None
Priority: 6
Submitted By: Guido van Rossum (gvanrossum)
Assigned to: Nobody/Anonymous (nobody)
Summary: -hex/oct const generates wrong code

Initial Comment:
Since Python 2.2, there's code in com_factor() in
compile.c that expands negative constants inline. For
example, the expression -1 generates a single
LOAD_CONST opcode for -1, rather than (as in Python 2.1
and before) generating a LOAD_CONST for +1 followed by
a UNARY_NEGATIVE opcode.

Unfortunately there's an end case where this causes
surprising behavior: when the constant is an octal or
hexadecimal constant that already has a negative value
(despite having no sign), the generated code yields a
different result than expected.  For example:

>>> print -(0xffffffff)
1
>>> x = 0xffffffff 
>>> print -x
1
>>> print -0xffffffff 
-4294967295
>>> 

Despite the fact that in Python 2.4 (!) all of these
will print the third outcome, in Python 2.2 and 2.3, I
think the third outcome is wrong and unexpected.

No time to fix this for 2.3a1, but should fix it before
2.3a2.

----------------------------------------------------------------------

>Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-05 13:24

Message:
Logged In: YES 
user_id=6380

Hmm...  There's a philosophical issue here. I'll post to
python-dev (if anyone still reads that).

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2003-01-17 05:47

Message:
Logged In: YES 
user_id=6656

Oh, so maybe the fact that Lib/compiler has a problem in the
same place *isn't* a problem with the compiler package... 

What ever gets done here should probably coincide with
whatever happens in patch 661536.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-01-16 14:53

Message:
Logged In: YES 
user_id=6380

I believe that test relies on the bug. :-(

So the test should be fixed.

The value ought to be equal to -(020000000000L).

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-01-16 14:45

Message:
Logged In: YES 
user_id=33168

The test fails because of line test_grammar.py:40:

  if -2147483647-1 != -020000000000:

What should -020000000000 be?
Without the -, it is:  -(2**31)-1, so it can't be
represented in an int (on 32-bit arches).
With the patch, it is: 2147483648L.

Later, I'll take a look at doing what Guido suggested.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-01-16 13:38

Message:
Logged In: YES 
user_id=33168

mwh pointed out that my patch breaks test_grammar.  I'll
take another look later, unless Raymond beats me to it.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-01-16 12:32

Message:
Logged In: YES 
user_id=6380

I'd like to keep the optimization too.

I think you have to do something different. The code
currently tries to save some time by prepending a minus sign
to the literal. I think it should just evaluate the literal
and then apply PyNumber_Negative() to the result, all the
while catching exceptions and falling back to unoptimized
code if there are exceptions.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-01-16 10:18

Message:
Logged In: YES 
user_id=33168

Ideally, I'd like to keep the code, rather than rip it out.
 Is it possible to keep the optimization, but if it is -
followed by a 0, then fallback to the original technique
(ie, UNARY_NEGATIVE, followed by the value)?

I have attached a patch which seems to implement this.

----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2003-01-12 06:35

Message:
Logged In: YES 
user_id=80475

See attached patch.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=660455&group_id=5470


From noreply@sourceforge.net  Wed Feb  5 18:17:22 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Wed, 05 Feb 2003 10:17:22 -0800
Subject: [Python-bugs-list] [ python-Bugs-681122 ] Built-in function dir() causes refcount leak in baseclasses
Message-ID: <E18gU6s-00050r-00@sc8-sf-web1.sourceforge.net>

Bugs item #681122, was opened at 2003-02-05 19:17
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=681122&group_id=5470

Category: Python Library
Group: Python 2.2.2
Status: Open
Resolution: None
Priority: 5
Submitted By: Troels Walsted Hansen (troels)
Assigned to: Nobody/Anonymous (nobody)
Summary: Built-in function dir() causes refcount leak in baseclasses

Initial Comment:
The bug appears to be present in Python 2.2.2 and 2.3a, 
but not in 2.2 or 2.2.1.

import sys

class Top:
    pass

class Bottom(Top):
    pass

print 'Refcount before:', sys.getrefcount(Top)
b=Bottom()
for i in range(1000):
    dir(b)
print 'Refcount after:', sys.getrefcount(Top)

$ ~/py221/bin/python refcount.py
Refcount before: 3
Refcount after: 3

$ ~/py222/bin/python refcount.py
Refcount before: 3
Refcount after: 1003

$ ~/py23a1/bin/python refcount.py
Refcount before: 3
Refcount after: 1003


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=681122&group_id=5470


From noreply@sourceforge.net  Wed Feb  5 18:25:04 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Wed, 05 Feb 2003 10:25:04 -0800
Subject: [Python-bugs-list] [ python-Bugs-660455 ] -hex/oct const generates wrong code
Message-ID: <E18gUEK-0003CS-00@sc8-sf-web2.sourceforge.net>

Bugs item #660455, was opened at 2002-12-31 13:11
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=660455&group_id=5470

Category: Parser/Compiler
>Group: Python 2.3
Status: Open
Resolution: None
>Priority: 7
Submitted By: Guido van Rossum (gvanrossum)
Assigned to: Nobody/Anonymous (nobody)
Summary: -hex/oct const generates wrong code

Initial Comment:
Since Python 2.2, there's code in com_factor() in
compile.c that expands negative constants inline. For
example, the expression -1 generates a single
LOAD_CONST opcode for -1, rather than (as in Python 2.1
and before) generating a LOAD_CONST for +1 followed by
a UNARY_NEGATIVE opcode.

Unfortunately there's an end case where this causes
surprising behavior: when the constant is an octal or
hexadecimal constant that already has a negative value
(despite having no sign), the generated code yields a
different result than expected.  For example:

>>> print -(0xffffffff)
1
>>> x = 0xffffffff 
>>> print -x
1
>>> print -0xffffffff 
-4294967295
>>> 

Despite the fact that in Python 2.4 (!) all of these
will print the third outcome, in Python 2.2 and 2.3, I
think the third outcome is wrong and unexpected.

No time to fix this for 2.3a1, but should fix it before
2.3a2.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-05 13:24

Message:
Logged In: YES 
user_id=6380

Hmm...  There's a philosophical issue here. I'll post to
python-dev (if anyone still reads that).

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2003-01-17 05:47

Message:
Logged In: YES 
user_id=6656

Oh, so maybe the fact that Lib/compiler has a problem in the
same place *isn't* a problem with the compiler package... 

What ever gets done here should probably coincide with
whatever happens in patch 661536.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-01-16 14:53

Message:
Logged In: YES 
user_id=6380

I believe that test relies on the bug. :-(

So the test should be fixed.

The value ought to be equal to -(020000000000L).

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-01-16 14:45

Message:
Logged In: YES 
user_id=33168

The test fails because of line test_grammar.py:40:

  if -2147483647-1 != -020000000000:

What should -020000000000 be?
Without the -, it is:  -(2**31)-1, so it can't be
represented in an int (on 32-bit arches).
With the patch, it is: 2147483648L.

Later, I'll take a look at doing what Guido suggested.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-01-16 13:38

Message:
Logged In: YES 
user_id=33168

mwh pointed out that my patch breaks test_grammar.  I'll
take another look later, unless Raymond beats me to it.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-01-16 12:32

Message:
Logged In: YES 
user_id=6380

I'd like to keep the optimization too.

I think you have to do something different. The code
currently tries to save some time by prepending a minus sign
to the literal. I think it should just evaluate the literal
and then apply PyNumber_Negative() to the result, all the
while catching exceptions and falling back to unoptimized
code if there are exceptions.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-01-16 10:18

Message:
Logged In: YES 
user_id=33168

Ideally, I'd like to keep the code, rather than rip it out.
 Is it possible to keep the optimization, but if it is -
followed by a 0, then fallback to the original technique
(ie, UNARY_NEGATIVE, followed by the value)?

I have attached a patch which seems to implement this.

----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2003-01-12 06:35

Message:
Logged In: YES 
user_id=80475

See attached patch.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=660455&group_id=5470


From noreply@sourceforge.net  Wed Feb  5 18:25:53 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Wed, 05 Feb 2003 10:25:53 -0800
Subject: [Python-bugs-list] [ python-Bugs-680429 ] __module__ broken for extension classes
Message-ID: <E18gUF7-0001oP-00@sc8-sf-web4.sourceforge.net>

Bugs item #680429, was opened at 2003-02-04 11:55
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=680429&group_id=5470

Category: Python Interpreter Core
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Ralf W. Grosse-Kunstleve (rwgk)
Assigned to: Michael Hudson (mwh)
Summary: __module__ broken for extension classes

Initial Comment:
We are having problems using Boost.Python with Python 
2.3a1 because
under some circumstances the result of __module__ is 
different compared
to earlier Python versions:

Python 2.2.1 (#2, Jun 17 2002, 12:06:51) 
[GCC 2.96 20000731 (Red Hat Linux 7.3 2.96-110)] on 
linux2
Type "help", "copyright", "credits" or "license" for more 
information.
>>> import simple
>>> simple.empty.__module__
'simple'
>>> 

Python 2.3a1 (#1, Jan  6 2003, 14:17:56) 
[GCC 2.96 20000731 (Red Hat Linux 7.3 2.96-110)] on 
linux2
Type "help", "copyright", "credits" or "license" for more 
information.
>>> import simple
>>> simple.empty.__module__
'__main__'
>>> 

Because of this we can no longer pickle our extension 
classes.
For your reference the code for the simple module is 
attached.
This is using Boost release 1.29.0 (www.boost.org).

We have done some debugging. Boost.Python's internal 
idea of the module
associated with an extension class is still correct even when 
using
Python 2.3a1. David Abrahams (main Boost.Python author) 
is telling me
that he "changed Boost.Python to work the way Guido 
suggested before
2.2.2." Therefore we suspect that the __module__ problem 
is due to
a change/bug in Python 2.3a1.

Ralf


#include <boost/python/module.hpp>
#include <boost/python/class.hpp>

namespace sandbx { namespace {

  struct empty {};

  void init_module()
  {
    using namespace boost::python;
    class_<empty>("empty");
  }

}} // namespace sandbx::<anonymous>

BOOST_PYTHON_MODULE(simple)
{
  sandbx::init_module();
}


----------------------------------------------------------------------

>Comment By: Ralf W. Grosse-Kunstleve (rwgk)
Date: 2003-02-05 10:25

Message:
Logged In: YES 
user_id=71407

> Look at typeobject.c:type_module() (about 100 lines in).
>
> Is simple.empty a HEAPTYPE?

Yes, it is a HEAPTYPE. I've established this by adding print 
statements
in typeobject.c:type_module():

static PyObject *
type_module(PyTypeObject *type, void *context)
{
        PyObject *mod;
        char *s;

        if (type->tp_flags & Py_TPFLAGS_HEAPTYPE) {
printf("IS HEAPTYPE\n");
                mod = PyDict_GetItemString(type-
>tp_dict, "__module__");
                Py_XINCREF(mod);
                s = PyString_AsString(mod);
printf("type->tp_dict, __module__ = %s\n", s);
                return mod;
        }

Result:

Python 2.3a1 (#2, Feb  5 2003, 09:39:30) 
[GCC 2.96 20000731 (Red Hat Linux 7.3 2.96-110)] on linux2
Type "help", "copyright", "credits" or "license" for more 
information.
>>> import simple
>>> simple.empty.__module__
IS HEAPTYPE
type->tp_dict, __module__ = __main__
'__main__'
>>> 

> If you can tell me what you want that function to do, I can
> have a go at it.

We expect "simple" as the result of simple.empty.__module__.

Current result if simple.so is moved to a package:

Python 2.3a1 (#2, Feb  5 2003, 09:39:30) 
[GCC 2.96 20000731 (Red Hat Linux 7.3 2.96-110)] on linux2
Type "help", "copyright", "credits" or "license" for more 
information.
>>> from sandbx import simple
>>> simple.empty.__module__
IS HEAPTYPE
type->tp_dict, __module__ = __main__
'__main__'
>>> 

Here we expect "sandbx.simple" .
Python 2.2.x produces the expected results using the exact same
Boost.Python source code.

David Abrahams provides this additional information:

  Remember that simple.empty is created by calling the metatype, 
so
  it's created on the heap, but it's Python's internal type creation
  mechanisms which do it -- we're not setting the flags manually.

Thanks!
        Ralf


----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2003-02-05 06:10

Message:
Logged In: YES 
user_id=6656

There has certainly been a change here.

Look at typeobject.c:type_module() (about 100 lines in).

If you can tell me what you want that function to do, I can
have a go at it.

Is simple.empty a HEAPTYPE?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=680429&group_id=5470


From noreply@sourceforge.net  Wed Feb  5 18:36:30 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Wed, 05 Feb 2003 10:36:30 -0800
Subject: [Python-bugs-list] [ python-Bugs-680789 ] Debug with long array takes forever
Message-ID: <E18gUPO-0002LX-00@sc8-sf-web4.sourceforge.net>

Bugs item #680789, was opened at 2003-02-05 06:00
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=680789&group_id=5470

Category: Python Library
Group: Python 2.2
Status: Open
Resolution: None
Priority: 5
Submitted By: Jurjen N.E. Bos (jneb)
>Assigned to: Jack Jansen (jackjansen)
Summary: Debug with long array takes forever

Initial Comment:
This is a bug and a partial patch.

If I debug a program that contains a ridiculously large array (8M entries in my case), the debugger takes forever.
It happens in Mac OS X, Python 2.2, but I found the bug in is the repr module, so it is probably universal.
The thing is, that after the fix below, it still doesn't work!
Did I miss something trivial (like repr is builtin, or something like that?). Would someone with Mac OS X experience help out here, please (Jack?).

Here's the diff to make repr.repr work with large arrays:
13a14
>       self.maxarray = 5
50a52,62
>     def repr_array(self, x, level):
>         n = len(x)
>       header = "array('"+x.typecode+"', ["
>         if n == 0: return header+"])"
>         if level <= 0: return header+"...])"
>         s = ''
>         for i in range(min(n, self.maxarray)):
>             if s: s = s + ', '
>             s = s + self.repr1(x[i], level-1)
>         if n > self.maxarray: s = s + ', ...'
>         return header + s + "])"

----------------------------------------------------------------------

>Comment By: Tim Peters (tim_one)
Date: 2003-02-05 13:36

Message:
Logged In: YES 
user_id=31435

Nice to see you, Jurgen!  I checked this into current CVS, 
and it works fine for me in isolation:

>>> len(a)
11055060
>>> repr.repr(a)
"array('i', [0, 1, 2, 3, 4, ...])"
>>>

That goes in an eyeblink.  So more detail is needed about 
what "it still doesn't work!" means.  Assigned to Jack, and he 
can use current CVS to try it.

Lib/repr.py; new revision: 1.15
Lib/test/test_repr.py; new revision: 1.16
Misc/NEWS; new revision: 1.642

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=680789&group_id=5470


From noreply@sourceforge.net  Wed Feb  5 20:40:29 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Wed, 05 Feb 2003 12:40:29 -0800
Subject: [Python-bugs-list] [ python-Bugs-471893 ] Security review of pickle/marshal docs
Message-ID: <E18gWLN-0000CD-00@sc8-sf-web2.sourceforge.net>

Bugs item #471893, was opened at 2001-10-16 18:42
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=471893&group_id=5470

Category: Extension Modules
Group: None
>Status: Open
Resolution: Fixed
Priority: 7
Submitted By: Tim Peters (tim_one)
Assigned to: Barry A. Warsaw (bwarsaw)
Summary: Security review of pickle/marshal docs

Initial Comment:
Paul Rubin points out that the security implications 
of using marshal and/or pickle aren't clear from the 
docs.  Assigning to Jeremy as he's more sensitive to 
such issues than I am; maybe Barry would like to get 
paranoid too <wink>.

A specific example:  the pickle docs say that pickle 
doesn't support code objects, and "at least this 
avoids the possibility of smuggling Trojan horses into 
a program".  However,

1) The marshal docs don't mention this vulnerability 
at all.

while

2) The pickle docs don't spell out possible dangers 
due to things pickle does that marshal doesn't (like 
importing modules, and running class constructors).

----------------------------------------------------------------------

>Comment By: A.M. Kuchling (akuchling)
Date: 2003-02-05 15:40

Message:
Logged In: YES 
user_id=11375

Re-opening.  With the changes to pickling in Python 2.3, 
the security material should be replaced with a warning to 
not unpickle untrusted data.  Patch attached.



----------------------------------------------------------------------

Comment By: Barry A. Warsaw (bwarsaw)
Date: 2001-11-18 11:08

Message:
Logged In: YES 
user_id=12800

You're right of course.  I meant to fix that and forgot. 
Will do so asap.  Glad you like the rest of it, though! :)

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2001-11-18 01:56

Message:
Logged In: NO 

Barry - the new docs just went up and they're a big
improvement over the old.  However the sentence 
"The issue here is usually not that a class's constructor
will get called -- it won't by the unpickler -- but that the
class's destructor (i.e. its __del__() method) might get
called when the object is garbage collected." isn't
correct.  

There's a flag in the pickle stream that tells
the unpickler that the pickler found a __getinitargs__
method at pickling time.  If the flag is set in the pickle,
then the unpickler calls the class constructor, whether
there's a __getinitargs__ method in the class or not.

The sample exploit that I posted earlier on,
<http://www.nightsong.com/phr/python/pickletest.py>,
is an example of an artificially concocted pickle that
makes a class constructor get called.


----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2001-11-17 17:07

Message:
Logged In: NO 

Are the new docs downloadable from somewhere?  thanx  --Paul


----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2001-11-15 19:43

Message:
Logged In: NO 

Barry, can you also do something about the Cookie module,
or at least about its documentation?  If Cookie is aliased
to SmartCookie, I think that needs mention.
 --Paul

----------------------------------------------------------------------

Comment By: Barry A. Warsaw (bwarsaw)
Date: 2001-11-15 19:12

Message:
Logged In: YES 
user_id=12800

I have rewritten the pickle documentation, and updated the
marshal, cPickle, and copy_reg documentation.  I believe all
the documentation issues raised here have been fixed.  

Implementation issues will be pushed to Python 2.3, and the
plan is to write a PEP to plan future work.

----------------------------------------------------------------------

Comment By: Barry A. Warsaw (bwarsaw)
Date: 2001-11-13 16:39

Message:
Logged In: YES 
user_id=12800

Assigning to Jeremy so he'll remember to forward me Jim's
email.  Jeremy can ping-pong it to Fred if he wants, and
then reassign to me after forwarding the email.

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2001-11-13 01:18

Message:
Logged In: NO 

The find_global variable sounds encouraging and if it
fixes this problem, that's great.  I'd still feel better
if the eval call goes away.  Is removing it allowed
under the 2.2 feature freeze?  It doesn't affect anything
visible, so no tests would have to change, etc.

Paul


----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2001-11-12 19:38

Message:
Logged In: NO 

It's possible that the eval is safe, but validating that
takes a line by line code review of all the paths that
evaling a string can go through.  It's also brittle in
that maybe someone will change the evaluator (say to
support Perl-like interpolation) in a future Python version
and not remember to change the unpickler.  Something like
that has already happened with the Cookie module.  My
guess is that it happened with the pickle module--the
whole point of that quoted string check is that the 
original pickle implementer didn't trust the input. 
The stuff about unpickling class instances was added
later (maybe by another person) without remembering the
security issue. 

Using eval this way is like storing a vat of cyanide in
your child's bedroom.  Sure, maybe if you check the seals
and locks on the vat carefully enough, you can convince
yourself that your child won't be able to get to the
cyanide.  But wouldn't you feel safer just not having the
vat there at all?  That's basic safety-consciousness.
Security consciousness works the same way.  Try to keep
dangerous ingredients and attackers as far away from each
other as possible.

Paul



----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2001-11-12 13:52

Message:
Logged In: YES 
user_id=31435

Why are people (Paul, Jeremy) concerned about eval'ing 
strings?  cPickle and pickle both check that they're 
properly quoted, and this isn't sh or Perl:  Python has 
no "dynamic" gimmicks buried in string literals.  All 
eval'ing a string literal can do is produce a binary blob 
via interpeting simple escape sequences.  They're like C 
strings this way -- maybe we'll run out of memory, but 
that's it.

I would agree that Python should be refactored internally 
to supply a clean function for changing string literals 
into binary blobs, but that would be for clarity and 
efficiency, not security.

----------------------------------------------------------------------

Comment By: Jim Fulton (dcjim)
Date: 2001-11-12 13:14

Message:
Logged In: YES 
user_id=73023

It sounds like there are some documentation bugs:

- The security ramifications are not discussed, nor are the
remedies.

- The cPickle module isn't documented very well. I submitted
some
   documentation a long time ago that never got incorporated
AFAIK.
   I wish I still had it. :)

- cPickle has a feature for turning off instance support and
for 
   restricting which classes can be unpickled. You can set
the find_global
   attribute on a cPickle.Unpickler. The find_global
attribute can be 
   a function or None.  If it is None, then no instances can
be 
   unpickled. If it is a function, then it should accept a
module and name
   and return the corresponding global. It is responsible
for looking
   up the global and can raise an error to prevent a
global's use.

   See the ZEO storage server implementation for an example
   of using this hook.


  



----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2001-11-12 01:33

Message:
Logged In: NO 

I like marshal and think it's much cleaner than pickle.
I'd like to see its format cleaned up and documented
so it can be a portable transport format.  Even the
recursion issue isn't so compelling--there's only a danger
on marshalling, not unmarshalling, and the application
has control over what it marshals.  So I think the idea
of documenting marshal (bug #467384) should be kept alive.
However, it shouldn't be changed in 2.2.  

If the Cookie class is still really aliased by default to
SmartCookie, that's a bad bug and should definitely be
fixed ASAP.  The Cookie docs clearly say not to use
SmartCookie or SerialCookie.  In fact I think SmartCookie
and SerialCookie should be disabled altogether.  Any
applications using them on the open internet have a security
but and should be fixed immediately.

Here's a possible temporary approach to fixing SmartCookie
and SerialCookie: append a cryptographic message
authentication code (MAC) to each pickled cookie.  That
means the application has some secret string K as a
configuration parameter (it should be 10 or more random
8-bit characters, or 20 or more random hex digits, etc). 
Then a smart cookie would be a pickle p, followed by
SHA(p+K).  The Cookie module would validate the MAC before
calling unpickle, and raise an exception if the MAC wasn't
valid.

The security of this scheme rests on K being kept secret
from attackers, which is generally not an easy thing to
manage, but it's something to think about.

Paul

----------------------------------------------------------------------

Comment By: Jeremy Hylton (jhylton)
Date: 2001-11-10 17:07

Message:
Logged In: YES 
user_id=31392

I don't think we should be doing anything about marshal.  
Maybe we should name in pyclib or something <0.9 wink>.  It 
works fine for .pyc files, but I don't see a reason for it 
to do anymore than is necessary for that purpose.

I think the notion of an unpickler that only handles 
builtin datatypes is the most attractive option you offer, 
but Paul has a good point about eval for strings.  (It 
currently has some hacks that address specific exploits, 
but I doubt they are sufficient.)  I also wonder how hard 
it is to handle builtin types and avoid subclasses of 
builtin types.

If there are any changes to pickle, I think we need to be 
careful about how it is described.  If we claim that an 
unpickler is safe for untrusted pickles, we've made a 
fairly strong claim.  I still think such a design change 
requires a PEP that includes some requirements and use 
cases and a thorough analysis of potential exploits. 


----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2001-11-10 14:42

Message:
Logged In: NO 

See bug #467384 for discussion about marshal.  Besides the
recursion issue, marshal's format is explicitly undocumented
and subject to change--you can't rely on it to interoperate
between two different Python versions, so it's no good as
an RPC serializer.  The format has kludges (e.g. the
representation of long ints) that make it undesirable to
freeze and document it and force future versions to be
backward compatible.

Adding a pickle.loads flag to prevent instance unpickling
isn't perfect but is probably the best alternative on 
your list.  Perhaps the flag can have a value that allows
unpickling the instances by restoring the instance 
attributes rather than calling the initializer.  That's
not always the right way to unpickle an instance (that's
why the unpickler no longer works that way) but it's good
enough a lot of the time.  

There's another issue with pickle/cPickle which is that they
unpickle quoted strings by evaling them.  This is scary.
While I don't see an immediate exploit, I also haven't
examined the 1000's of lines of code I'd need to examine
to convince myself that there's NOT an exploit.  I think
the unpickler should be changed to never call eval but just
parse the string as it needs to.  

Guido seemed to think pickle might have other possible
exploits.  I don't know what he had in mind but before
declaring it safe for untrusted data I think it needs to
be gone over with a fine toothed comb.

Paul


----------------------------------------------------------------------

Comment By: Barry A. Warsaw (bwarsaw)
Date: 2001-11-10 12:37

Message:
Logged In: YES 
user_id=12800

I'm going to agree with Paul that this is a problem needing
fixing, however there are really several issues.

1. Cookie module makes it too easy to code exploits.  Cookie
exports a class, also called Cookie, which is aliased to
SmartCookie, so that a naive program will simply pass cookie
data to Cookie.Cookie() and you're screwed.  So, Cookie
module's defaults should be for more security rather than
less, and Cookie.Cookie should be aliased to SimpleCookie
instead.

2. There is no built-in safe mechanism for de-serializing
data from untrusted sources.  You can't use pickle without
overloading a "magic" method.  You can't use cPickle because
you can't do the overloading trick.  You can't use marshal
because it isn't bulletproof against recursive
datastructures.  So how /do/ you do it?

I think it just may be serious enough to deal with in Python
2.2, and I volunteer to address it (so I'll steal this bug
report).  Without looking at the code, or the level of
effort necessary, I would make the following suggestions:

1. Add to the public interface of pickle and cPickle, a flag
that either disables the unpickling of instances altogether,
or disables calling any code with unpickled data, e.g.
constructors.

2. Fix marshal to be bulletproof against recursive
datastructures.
3. Update the docs for both pickle/cPickle and marshal to
explain how to safely write de-serializers of untrusted strings.

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2001-11-09 04:21

Message:
Logged In: NO 

Well, Guido and Tim agree with you that it's not a pickle
bug.  I still feel it is one, because its docs currently
make you think you can securely load untrusted pickles, and
because it's a natural, non-obscure thing to want to do
(see pyro and the cookie module), but whatever.  If it's
not a code bug then I feel it's a significant functionality
shortcoming in the python library.

Pyro uses pickle to serialize data for RPC calls over the
internet.  A malicious client could make a hostile pickle
take over the server.  The cookie module lets web
applications store user session state in browser cookies.
Its SerialCookie and SmartCookie classes let you put
arbitrary Python objects into the user session, and
serializes them when pickle.  Again, a malicious client
can make a hostile pickle, send it in a cookie header to
the http server, and take over the server when the 
application unpickles the cookie.

The current documentation for the pickle module makes it
very clear to me that the doc writer thought it was safe
to unpickle untrusted cookies.  If pickle wasn't designed
for that, then there was a communication failure between
the designer and the doc writer.

Yes, I'm willing to help with a PEP for fixing this
situation.

Paul


----------------------------------------------------------------------

Comment By: Jeremy Hylton (jhylton)
Date: 2001-11-08 12:37

Message:
Logged In: YES 
user_id=31392

I don't think of the issue you describe as a bug in the
code.  You're suggesting a new feature for pickle.  As far
as I can tell, the original design requirements for pickle
did not include the ability to securely load a pickle from
an untrusted source.

It may be a legitimate feature request, but it's too late to
make it into Python 2.2.  I suggest we look at the design
issues for Python 2.3 and decide if it's a feature we want
to support.  I imagine a PEP may be necessary to lay out the
issues and the solution.  Do you want to have a hand in that
PEP?

I still don't understand what it means that Pyro and cookie
were bit by a bug.  It sounds like they were using pickle in
ways that pickle was not intended to support.  A careful
analysis of how those two applications use pickle would be
helpful for generating requirements.


----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2001-11-07 18:54

Message:
Logged In: NO 

IMO it's a code bug that you can't unpickle strings from
untrusted sources.  Pyro and the cookie module are examples
of programs that got bitten by this bug.  Whether it's
really a bug is a matter of opinion--I had a big email
exchange with Guido and Tim about it, and they felt it
was enough to fix the pickle documentation.

Pickle has the same problem as cPickle, but with pickle
you can subclass the pickler and override the method that
unpickles class objects, and work around the (IMO) bug.
The workaround doesn't help cPickle since cPickle can't
be subclassed.  See bug #467384 for some related discussion.

Paul


----------------------------------------------------------------------

Comment By: Jeremy Hylton (jhylton)
Date: 2001-11-07 17:02

Message:
Logged In: YES 
user_id=31392

What's the code bug?  Your last message has a lot of gloom
and doom <wink>, but I'm not sure what specific problem
you'd like to see fixed.  Are you saying that something
needs to be fixed in cPickle and not in pickle?

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2001-11-07 15:08

Message:
Logged In: NO 

Irmen de Jong points out that the standard cookie module
uses pickling for serial and smart cookies.  The 2.1.1
cookie module docs explicitly say not to use those
classes because of the security hole--that they're provided
for backward compatibility only (but with what?!  Any
server that uses those classes on open ports needs to be
changed right away).

Irmen's library, http://pyro.sourceforge.net, also uses
unpickle insecurely (he's aware of this now and is figuring
out a fix).

IMO this is really a code bug rather than a documentation
bug, and should be fixed in the code rather than the docs.
Documenting the bug rather than fixing it leaves a
deficiency in the Python library: obvious uses of pickling,
like Pyro and the cookie module, can't be implemented
using cPickle and have to resort to a slower Python
deserializer, or use marshal and have possible compatibility
problems between versions (and not be able to serialize
class instances).

Paul



----------------------------------------------------------------------

Comment By: paul rubin (phr)
Date: 2001-10-16 19:06

Message:
Logged In: YES 
user_id=72053

Certainly anyone unserializing potentially malicious data
with pickle, marshal, or anything else, should check the
results before doing anything dangerous with them (like
executing code).  However, unpickle can potentially do
damage before it even returns, by creating loading modules
and creating initialized class instances.  So pickle.loads
should never be used on untrusted strings, except possibly
in a rexec wrapper (as proposed by Tim).  Another
possibility (also by Tim) is to override the load_inst
method of the Pickler class, though I don't think you can
do that for cPickle.

A sample exploit for unpickle can be found at
<http://www.nightsong.com/phr/python/pickletest.py>.
Unpickling the test string runs penguin.__init__ contrary
to the doc's saying no initialization unless there's a
__getinitargs__ method in the class def.

The "exploding penguin" class is artificial, but
applications are vulnerable if there's an unsafe
constructor anywhere in any class of the application or in
the python library (example: the NNTP constructor opens an
IP connection to an arbitrary address, so a malicious
imported string can send a message through your firewall
when you import it).


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=471893&group_id=5470


From noreply@sourceforge.net  Wed Feb  5 20:30:28 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Wed, 05 Feb 2003 12:30:28 -0800
Subject: [Python-bugs-list] [ python-Bugs-453515 ] filecmp.dircmp case sensitivity bug
Message-ID: <E18gWBg-0001SB-00@sc8-sf-web3.sourceforge.net>

Bugs item #453515, was opened at 2001-08-20 17:39
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=453515&group_id=5470

Category: Python Library
Group: Python 2.1.1
Status: Open
Resolution: None
Priority: 5
Submitted By: Rik Kabel (ving)
>Assigned to: A.M. Kuchling (akuchling)
Summary: filecmp.dircmp case sensitivity bug

Initial Comment:
(warning: python newbie submission)

Platforms: W2K w/Activestate 2.1.1 (same library source found in 2.0 and 2.11 on NetBSD).

filecmp.dircmp performs incorrect filename comparisons when building lists of common and 
directory-unique files. In particular, it sets a dictionary key to the filename (and value to 1) for 
each file in the right-hand tree, and looks for matching names (has_key). This fails on 
case-insensitive platforms when the names are equivalent except for case.

A simple workaround would be to use os.path.normcase() around the filenames before storing and 
comparing, but this is not case-preserving. Case preservation is to be preferred.

A case-preserving workaround might use os.path.normcase() for the dictionary entry keys, but 
store the unchanged filename as the value, and use that value when constructing the list from the 
dictionary.

-- 
Rik      creating tomorrow's legacy systems, today


----------------------------------------------------------------------

>Comment By: A.M. Kuchling (akuchling)
Date: 2003-02-05 15:30

Message:
Logged In: YES 
user_id=11375

Snatching this bug.  See bug #680494 for a test suite for filecmp.py.
Once that's checked in, I'll look into this one.


----------------------------------------------------------------------

Comment By: Fred L. Drake, Jr. (fdrake)
Date: 2001-12-06 16:14

Message:
Logged In: YES 
user_id=3066

I've attached a preliminary patch for this, completely
untested.  Problems:

- We don't have any tests for the filecmp module.
- Im not running Windows, so I can't test this in an environment
  similar to that for which the bug was reported.

If someone can create a test case for this, and test the
patch, that would really help.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2001-09-05 14:12

Message:
Logged In: YES 
user_id=6380

While this was reported as an ActivePython bug, the problem
is the same with filecmp.py in CVS, so I'm unassigning this
from Paul (it's not like Paul's track record of responding
to bug reports is so great :-).


----------------------------------------------------------------------

Comment By: Jeremy Hylton (jhylton)
Date: 2001-08-28 11:34

Message:
Logged In: YES 
user_id=31392

Can you look at this, Paul?


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=453515&group_id=5470


From noreply@sourceforge.net  Wed Feb  5 20:27:04 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Wed, 05 Feb 2003 12:27:04 -0800
Subject: [Python-bugs-list] [ python-Bugs-405554 ] pydoc should be integrated with HTML doc
Message-ID: <E18gW8O-0001JB-00@sc8-sf-web3.sourceforge.net>

Bugs item #405554, was opened at 2001-03-02 18:09
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=405554&group_id=5470

Category: Python Library
Group: Feature Request
>Status: Closed
>Resolution: Later
Priority: 5
Submitted By: Guido van Rossum (gvanrossum)
Assigned to: Ka-Ping Yee (ping)
Summary: pydoc should be integrated with HTML doc

Initial Comment:
I really like pydoc, but it misses one feature compared
to perldoc: often the *good* documentation for a module
or class is in the library reference manual.  There
should be a way to get that integrated with the stuff
that pydoc extracts from the doc strings. 

(Wasn't this Paul Prescod's original idea?)


----------------------------------------------------------------------

>Comment By: A.M. Kuchling (akuchling)
Date: 2003-02-05 15:27

Message:
Logged In: YES 
user_id=11375

Added to PEP 42 and closed.

----------------------------------------------------------------------

Comment By: Garth T Kidd (gtk)
Date: 2001-07-30 02:28

Message:
Logged In: YES 
user_id=59803

I haven't seen anything in the library reference that 
couldn't be sucked directly out of the source by some 
appropriate combination of formally parseable docstrings 
(PEP-256, PEP-256) and some suitable markup format 
(structuredtext.sf.net). :) 

As a less mind-bending alternative, the docstring 
processing framework importantly leaves dereferencing links 
to the caller. If you decide on a standard way of 
referencing pages in the manual, module authors will need 
only insert simple links in their docstrings to refer to 
the [externally maintained] documentation:: 

  Please refer to `ftp-objects`_ for more detail. 

... or::

  Please refer to `FTP Objects`_ for more detail. 

  .. _FTP Objects: %(ftp-objects)

... or whatever. 

Someone, tell me I'm nuts. :) 

----------------------------------------------------------------------

Comment By: Ka-Ping Yee (ping)
Date: 2001-04-13 08:59

Message:
Logged In: YES 
user_id=45338

Although there is now some integration with the
reference manual, this issue is still largely
unaddressed, since we haven't decided how to
present the two kinds of documentation
together or whether to merge them somehow.

Leaving this open for now.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=405554&group_id=5470


From noreply@sourceforge.net  Wed Feb  5 20:21:24 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Wed, 05 Feb 2003 12:21:24 -0800
Subject: [Python-bugs-list] [ python-Bugs-221327 ] threads within an embedded python interpreter
Message-ID: <E18gW2u-00013A-00@sc8-sf-web3.sourceforge.net>

Bugs item #221327, was opened at 2000-11-03 15:31
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=221327&group_id=5470

Category: Documentation
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Nobody/Anonymous (nobody)
Assigned to: Fred L. Drake, Jr. (fdrake)
Summary: threads within an embedded python interpreter

Initial Comment:
I have an application which I am required to embedd a python interpreter inside of a C++ application. The python script creates a separate thread of control to handle a TCP/IP socket connection. The thread can send data just find but hangs on the socket 'recv' fnction.

After a dya of debugging my code I went into the python socketmodule.c and at line 1082 I commented out the thread protection code:

/*Py_BEGIN_ALLOW_THREADS*/
n = recv(s->sock_fd, PyString_AsString(buf), len, flags);
/*Py_END_ALLOW_THREADS*/

I then recompiled the python library and then recompiled my C++ app and everything worked just fine.

The problem does not exist when I run the script alone using the python interpreter. Only when I embedd the interpreter, so this is some sort of configuration issue.

David Schere
Email: david_schere@yahoo.com





----------------------------------------------------------------------

>Comment By: A.M. Kuchling (akuchling)
Date: 2003-02-05 15:21

Message:
Logged In: YES 
user_id=11375

Typo in init.diff: "interpeter" -> "interpReter" in first line.

Any reason not to check in the example and close the bug?


----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2002-08-04 05:47

Message:
Logged In: YES 
user_id=21627

Doc/api/init.tex already has a paragraph about calling
Python from a C thread; this patch adds a cookbook example.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2002-03-01 17:58

Message:
Logged In: YES 
user_id=6380

but is it a doco bug or not?

----------------------------------------------------------------------

Comment By: Jeremy Hylton (jhylton)
Date: 2002-03-01 17:34

Message:
Logged In: YES 
user_id=31392

Probably ought to fix this for 2.3


----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2001-08-31 15:57

Message:
Logged In: NO 

Now u pay,.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2000-11-03 15:44

Message:
Most likely, the problem is not the socket module but how you call Python code from a thread that was started from C/C++. Look at t_bootstrap() in threadmodule.c for an example on what to do. (And yes, it's tricky.) There should also be something in the Python/C API documentation about this.

I'm not closing the bug report since this is really a documentation issue; it needs to be documented better. I'll assign it to Fred.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=221327&group_id=5470


From noreply@sourceforge.net  Wed Feb  5 19:50:41 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Wed, 05 Feb 2003 11:50:41 -0800
Subject: [Python-bugs-list] [ python-Bugs-680494 ] filecmp.py uses obsolete statcache module.
Message-ID: <E18gVZB-00053y-00@sc8-sf-web4.sourceforge.net>

Bugs item #680494, was opened at 2003-02-04 16:55
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=680494&group_id=5470

Category: Python Library
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Inyeol Lee (inyeol)
Assigned to: Raymond Hettinger (rhettinger)
Summary: filecmp.py uses obsolete statcache module.

Initial Comment:
2.3a1 filecmp module, which is not obsolete, uses
obsolete
statcache module and generates DeprecationWarning.

Inyeol Lee

----------------------------------------------------------------------

>Comment By: A.M. Kuchling (akuchling)
Date: 2003-02-05 14:50

Message:
Logged In: YES 
user_id=11375

Attaching test_filecmp.py.  Raymond, you've written test suites with unittest.py, right?  Am I using it in a reasonable way?


----------------------------------------------------------------------

Comment By: A.M. Kuchling (akuchling)
Date: 2003-02-05 14:37

Message:
Logged In: YES 
user_id=11375

Attaching test_filecmp.py.  Raymond, you've written test suites with unittest.py, right?  Am I using it in a reasonable way?


----------------------------------------------------------------------

Comment By: A.M. Kuchling (akuchling)
Date: 2003-02-05 14:07

Message:
Logged In: YES 
user_id=11375

There's no test suite for filecmp, so before making any changes, I'm 
going to write a test suite.


----------------------------------------------------------------------

Comment By: A.M. Kuchling (akuchling)
Date: 2003-02-05 13:23

Message:
Logged In: YES 
user_id=11375

The simplest thing would be to simply use os.stat instead of statcache,
and deprecate use_statcache by logging a warning if use_statcache is true.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=680494&group_id=5470


From noreply@sourceforge.net  Wed Feb  5 19:41:58 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Wed, 05 Feb 2003 11:41:58 -0800
Subject: [Python-bugs-list] [ python-Bugs-681122 ] Built-in function dir() causes refcount leak in baseclasses
Message-ID: <E18gVQk-0004bF-00@sc8-sf-web4.sourceforge.net>

Bugs item #681122, was opened at 2003-02-05 13:17
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=681122&group_id=5470

Category: Python Library
Group: Python 2.2.2
>Status: Closed
>Resolution: Fixed
Priority: 5
Submitted By: Troels Walsted Hansen (troels)
>Assigned to: Tim Peters (tim_one)
Summary: Built-in function dir() causes refcount leak in baseclasses

Initial Comment:
The bug appears to be present in Python 2.2.2 and 2.3a, 
but not in 2.2 or 2.2.1.

import sys

class Top:
    pass

class Bottom(Top):
    pass

print 'Refcount before:', sys.getrefcount(Top)
b=Bottom()
for i in range(1000):
    dir(b)
print 'Refcount after:', sys.getrefcount(Top)

$ ~/py221/bin/python refcount.py
Refcount before: 3
Refcount after: 3

$ ~/py222/bin/python refcount.py
Refcount before: 3
Refcount after: 1003

$ ~/py23a1/bin/python refcount.py
Refcount before: 3
Refcount after: 1003


----------------------------------------------------------------------

>Comment By: Tim Peters (tim_one)
Date: 2003-02-05 14:41

Message:
Logged In: YES 
user_id=31435

Good detective work!  It helps.  Fixed, in

Objects/object.c; new revision: 2.197

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=681122&group_id=5470


From noreply@sourceforge.net  Wed Feb  5 19:37:00 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Wed, 05 Feb 2003 11:37:00 -0800
Subject: [Python-bugs-list] [ python-Bugs-680494 ] filecmp.py uses obsolete statcache module.
Message-ID: <E18gVLw-0000FN-00@sc8-sf-web1.sourceforge.net>

Bugs item #680494, was opened at 2003-02-04 16:55
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=680494&group_id=5470

Category: Python Library
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Inyeol Lee (inyeol)
>Assigned to: Raymond Hettinger (rhettinger)
Summary: filecmp.py uses obsolete statcache module.

Initial Comment:
2.3a1 filecmp module, which is not obsolete, uses
obsolete
statcache module and generates DeprecationWarning.

Inyeol Lee

----------------------------------------------------------------------

>Comment By: A.M. Kuchling (akuchling)
Date: 2003-02-05 14:37

Message:
Logged In: YES 
user_id=11375

Attaching test_filecmp.py.  Raymond, you've written test suites with unittest.py, right?  Am I using it in a reasonable way?


----------------------------------------------------------------------

Comment By: A.M. Kuchling (akuchling)
Date: 2003-02-05 14:07

Message:
Logged In: YES 
user_id=11375

There's no test suite for filecmp, so before making any changes, I'm 
going to write a test suite.


----------------------------------------------------------------------

Comment By: A.M. Kuchling (akuchling)
Date: 2003-02-05 13:23

Message:
Logged In: YES 
user_id=11375

The simplest thing would be to simply use os.stat instead of statcache,
and deprecate use_statcache by logging a warning if use_statcache is true.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=680494&group_id=5470


From noreply@sourceforge.net  Wed Feb  5 19:28:32 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Wed, 05 Feb 2003 11:28:32 -0800
Subject: [Python-bugs-list] [ python-Bugs-612074 ] Cannot compile escaped unicode character
Message-ID: <E18gVDk-0005bG-00@sc8-sf-web2.sourceforge.net>

Bugs item #612074, was opened at 2002-09-20 03:23
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=612074&group_id=5470

Category: Regular Expressions
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Nathan L Miles (nathan22)
Assigned to: Fredrik Lundh (effbot)
Summary: Cannot compile escaped unicode character

Initial Comment:
The following RE fails to compile on Python 2.2/
Windows XP.

I am trying to work around this by writing my own 
version of .escape which does not escape code points > 
127.


import re

a = unichr(0x2039)
b = u"["+re.escape(a)+u"]"
re.compile(b)


----------------------------------------------------------------------

Comment By: Greg Chapman (glchapman)
Date: 2003-02-05 10:28

Message:
Logged In: YES 
user_id=86307

patch posted: http://www.python.org/sf/681152


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=612074&group_id=5470


From noreply@sourceforge.net  Wed Feb  5 19:07:39 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Wed, 05 Feb 2003 11:07:39 -0800
Subject: [Python-bugs-list] [ python-Bugs-680494 ] filecmp.py uses obsolete statcache module.
Message-ID: <E18gUtX-0004bH-00@sc8-sf-web2.sourceforge.net>

Bugs item #680494, was opened at 2003-02-04 16:55
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=680494&group_id=5470

Category: Python Library
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Inyeol Lee (inyeol)
>Assigned to: A.M. Kuchling (akuchling)
Summary: filecmp.py uses obsolete statcache module.

Initial Comment:
2.3a1 filecmp module, which is not obsolete, uses
obsolete
statcache module and generates DeprecationWarning.

Inyeol Lee

----------------------------------------------------------------------

>Comment By: A.M. Kuchling (akuchling)
Date: 2003-02-05 14:07

Message:
Logged In: YES 
user_id=11375

There's no test suite for filecmp, so before making any changes, I'm 
going to write a test suite.


----------------------------------------------------------------------

Comment By: A.M. Kuchling (akuchling)
Date: 2003-02-05 13:23

Message:
Logged In: YES 
user_id=11375

The simplest thing would be to simply use os.stat instead of statcache,
and deprecate use_statcache by logging a warning if use_statcache is true.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=680494&group_id=5470


From noreply@sourceforge.net  Wed Feb  5 21:29:37 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Wed, 05 Feb 2003 13:29:37 -0800
Subject: [Python-bugs-list] [ python-Bugs-471893 ] Security review of pickle/marshal docs
Message-ID: <E18gX6v-0004jW-00@sc8-sf-web3.sourceforge.net>

Bugs item #471893, was opened at 2001-10-16 18:42
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=471893&group_id=5470

Category: Extension Modules
Group: None
Status: Open
Resolution: Fixed
Priority: 7
Submitted By: Tim Peters (tim_one)
>Assigned to: Tim Peters (tim_one)
Summary: Security review of pickle/marshal docs

Initial Comment:
Paul Rubin points out that the security implications 
of using marshal and/or pickle aren't clear from the 
docs.  Assigning to Jeremy as he's more sensitive to 
such issues than I am; maybe Barry would like to get 
paranoid too <wink>.

A specific example:  the pickle docs say that pickle 
doesn't support code objects, and "at least this 
avoids the possibility of smuggling Trojan horses into 
a program".  However,

1) The marshal docs don't mention this vulnerability 
at all.

while

2) The pickle docs don't spell out possible dangers 
due to things pickle does that marshal doesn't (like 
importing modules, and running class constructors).

----------------------------------------------------------------------

>Comment By: Barry A. Warsaw (bwarsaw)
Date: 2003-02-05 16:29

Message:
Logged In: YES 
user_id=12800

Karmically (no, not comically) reassigning to Tim who
started this whole thing, and who happens to be dumping his
pickles these days.

----------------------------------------------------------------------

Comment By: A.M. Kuchling (akuchling)
Date: 2003-02-05 15:40

Message:
Logged In: YES 
user_id=11375

Re-opening.  With the changes to pickling in Python 2.3, 
the security material should be replaced with a warning to 
not unpickle untrusted data.  Patch attached.



----------------------------------------------------------------------

Comment By: Barry A. Warsaw (bwarsaw)
Date: 2001-11-18 11:08

Message:
Logged In: YES 
user_id=12800

You're right of course.  I meant to fix that and forgot. 
Will do so asap.  Glad you like the rest of it, though! :)

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2001-11-18 01:56

Message:
Logged In: NO 

Barry - the new docs just went up and they're a big
improvement over the old.  However the sentence 
"The issue here is usually not that a class's constructor
will get called -- it won't by the unpickler -- but that the
class's destructor (i.e. its __del__() method) might get
called when the object is garbage collected." isn't
correct.  

There's a flag in the pickle stream that tells
the unpickler that the pickler found a __getinitargs__
method at pickling time.  If the flag is set in the pickle,
then the unpickler calls the class constructor, whether
there's a __getinitargs__ method in the class or not.

The sample exploit that I posted earlier on,
<http://www.nightsong.com/phr/python/pickletest.py>,
is an example of an artificially concocted pickle that
makes a class constructor get called.


----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2001-11-17 17:07

Message:
Logged In: NO 

Are the new docs downloadable from somewhere?  thanx  --Paul


----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2001-11-15 19:43

Message:
Logged In: NO 

Barry, can you also do something about the Cookie module,
or at least about its documentation?  If Cookie is aliased
to SmartCookie, I think that needs mention.
 --Paul

----------------------------------------------------------------------

Comment By: Barry A. Warsaw (bwarsaw)
Date: 2001-11-15 19:12

Message:
Logged In: YES 
user_id=12800

I have rewritten the pickle documentation, and updated the
marshal, cPickle, and copy_reg documentation.  I believe all
the documentation issues raised here have been fixed.  

Implementation issues will be pushed to Python 2.3, and the
plan is to write a PEP to plan future work.

----------------------------------------------------------------------

Comment By: Barry A. Warsaw (bwarsaw)
Date: 2001-11-13 16:39

Message:
Logged In: YES 
user_id=12800

Assigning to Jeremy so he'll remember to forward me Jim's
email.  Jeremy can ping-pong it to Fred if he wants, and
then reassign to me after forwarding the email.

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2001-11-13 01:18

Message:
Logged In: NO 

The find_global variable sounds encouraging and if it
fixes this problem, that's great.  I'd still feel better
if the eval call goes away.  Is removing it allowed
under the 2.2 feature freeze?  It doesn't affect anything
visible, so no tests would have to change, etc.

Paul


----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2001-11-12 19:38

Message:
Logged In: NO 

It's possible that the eval is safe, but validating that
takes a line by line code review of all the paths that
evaling a string can go through.  It's also brittle in
that maybe someone will change the evaluator (say to
support Perl-like interpolation) in a future Python version
and not remember to change the unpickler.  Something like
that has already happened with the Cookie module.  My
guess is that it happened with the pickle module--the
whole point of that quoted string check is that the 
original pickle implementer didn't trust the input. 
The stuff about unpickling class instances was added
later (maybe by another person) without remembering the
security issue. 

Using eval this way is like storing a vat of cyanide in
your child's bedroom.  Sure, maybe if you check the seals
and locks on the vat carefully enough, you can convince
yourself that your child won't be able to get to the
cyanide.  But wouldn't you feel safer just not having the
vat there at all?  That's basic safety-consciousness.
Security consciousness works the same way.  Try to keep
dangerous ingredients and attackers as far away from each
other as possible.

Paul



----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2001-11-12 13:52

Message:
Logged In: YES 
user_id=31435

Why are people (Paul, Jeremy) concerned about eval'ing 
strings?  cPickle and pickle both check that they're 
properly quoted, and this isn't sh or Perl:  Python has 
no "dynamic" gimmicks buried in string literals.  All 
eval'ing a string literal can do is produce a binary blob 
via interpeting simple escape sequences.  They're like C 
strings this way -- maybe we'll run out of memory, but 
that's it.

I would agree that Python should be refactored internally 
to supply a clean function for changing string literals 
into binary blobs, but that would be for clarity and 
efficiency, not security.

----------------------------------------------------------------------

Comment By: Jim Fulton (dcjim)
Date: 2001-11-12 13:14

Message:
Logged In: YES 
user_id=73023

It sounds like there are some documentation bugs:

- The security ramifications are not discussed, nor are the
remedies.

- The cPickle module isn't documented very well. I submitted
some
   documentation a long time ago that never got incorporated
AFAIK.
   I wish I still had it. :)

- cPickle has a feature for turning off instance support and
for 
   restricting which classes can be unpickled. You can set
the find_global
   attribute on a cPickle.Unpickler. The find_global
attribute can be 
   a function or None.  If it is None, then no instances can
be 
   unpickled. If it is a function, then it should accept a
module and name
   and return the corresponding global. It is responsible
for looking
   up the global and can raise an error to prevent a
global's use.

   See the ZEO storage server implementation for an example
   of using this hook.


  



----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2001-11-12 01:33

Message:
Logged In: NO 

I like marshal and think it's much cleaner than pickle.
I'd like to see its format cleaned up and documented
so it can be a portable transport format.  Even the
recursion issue isn't so compelling--there's only a danger
on marshalling, not unmarshalling, and the application
has control over what it marshals.  So I think the idea
of documenting marshal (bug #467384) should be kept alive.
However, it shouldn't be changed in 2.2.  

If the Cookie class is still really aliased by default to
SmartCookie, that's a bad bug and should definitely be
fixed ASAP.  The Cookie docs clearly say not to use
SmartCookie or SerialCookie.  In fact I think SmartCookie
and SerialCookie should be disabled altogether.  Any
applications using them on the open internet have a security
but and should be fixed immediately.

Here's a possible temporary approach to fixing SmartCookie
and SerialCookie: append a cryptographic message
authentication code (MAC) to each pickled cookie.  That
means the application has some secret string K as a
configuration parameter (it should be 10 or more random
8-bit characters, or 20 or more random hex digits, etc). 
Then a smart cookie would be a pickle p, followed by
SHA(p+K).  The Cookie module would validate the MAC before
calling unpickle, and raise an exception if the MAC wasn't
valid.

The security of this scheme rests on K being kept secret
from attackers, which is generally not an easy thing to
manage, but it's something to think about.

Paul

----------------------------------------------------------------------

Comment By: Jeremy Hylton (jhylton)
Date: 2001-11-10 17:07

Message:
Logged In: YES 
user_id=31392

I don't think we should be doing anything about marshal.  
Maybe we should name in pyclib or something <0.9 wink>.  It 
works fine for .pyc files, but I don't see a reason for it 
to do anymore than is necessary for that purpose.

I think the notion of an unpickler that only handles 
builtin datatypes is the most attractive option you offer, 
but Paul has a good point about eval for strings.  (It 
currently has some hacks that address specific exploits, 
but I doubt they are sufficient.)  I also wonder how hard 
it is to handle builtin types and avoid subclasses of 
builtin types.

If there are any changes to pickle, I think we need to be 
careful about how it is described.  If we claim that an 
unpickler is safe for untrusted pickles, we've made a 
fairly strong claim.  I still think such a design change 
requires a PEP that includes some requirements and use 
cases and a thorough analysis of potential exploits. 


----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2001-11-10 14:42

Message:
Logged In: NO 

See bug #467384 for discussion about marshal.  Besides the
recursion issue, marshal's format is explicitly undocumented
and subject to change--you can't rely on it to interoperate
between two different Python versions, so it's no good as
an RPC serializer.  The format has kludges (e.g. the
representation of long ints) that make it undesirable to
freeze and document it and force future versions to be
backward compatible.

Adding a pickle.loads flag to prevent instance unpickling
isn't perfect but is probably the best alternative on 
your list.  Perhaps the flag can have a value that allows
unpickling the instances by restoring the instance 
attributes rather than calling the initializer.  That's
not always the right way to unpickle an instance (that's
why the unpickler no longer works that way) but it's good
enough a lot of the time.  

There's another issue with pickle/cPickle which is that they
unpickle quoted strings by evaling them.  This is scary.
While I don't see an immediate exploit, I also haven't
examined the 1000's of lines of code I'd need to examine
to convince myself that there's NOT an exploit.  I think
the unpickler should be changed to never call eval but just
parse the string as it needs to.  

Guido seemed to think pickle might have other possible
exploits.  I don't know what he had in mind but before
declaring it safe for untrusted data I think it needs to
be gone over with a fine toothed comb.

Paul


----------------------------------------------------------------------

Comment By: Barry A. Warsaw (bwarsaw)
Date: 2001-11-10 12:37

Message:
Logged In: YES 
user_id=12800

I'm going to agree with Paul that this is a problem needing
fixing, however there are really several issues.

1. Cookie module makes it too easy to code exploits.  Cookie
exports a class, also called Cookie, which is aliased to
SmartCookie, so that a naive program will simply pass cookie
data to Cookie.Cookie() and you're screwed.  So, Cookie
module's defaults should be for more security rather than
less, and Cookie.Cookie should be aliased to SimpleCookie
instead.

2. There is no built-in safe mechanism for de-serializing
data from untrusted sources.  You can't use pickle without
overloading a "magic" method.  You can't use cPickle because
you can't do the overloading trick.  You can't use marshal
because it isn't bulletproof against recursive
datastructures.  So how /do/ you do it?

I think it just may be serious enough to deal with in Python
2.2, and I volunteer to address it (so I'll steal this bug
report).  Without looking at the code, or the level of
effort necessary, I would make the following suggestions:

1. Add to the public interface of pickle and cPickle, a flag
that either disables the unpickling of instances altogether,
or disables calling any code with unpickled data, e.g.
constructors.

2. Fix marshal to be bulletproof against recursive
datastructures.
3. Update the docs for both pickle/cPickle and marshal to
explain how to safely write de-serializers of untrusted strings.

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2001-11-09 04:21

Message:
Logged In: NO 

Well, Guido and Tim agree with you that it's not a pickle
bug.  I still feel it is one, because its docs currently
make you think you can securely load untrusted pickles, and
because it's a natural, non-obscure thing to want to do
(see pyro and the cookie module), but whatever.  If it's
not a code bug then I feel it's a significant functionality
shortcoming in the python library.

Pyro uses pickle to serialize data for RPC calls over the
internet.  A malicious client could make a hostile pickle
take over the server.  The cookie module lets web
applications store user session state in browser cookies.
Its SerialCookie and SmartCookie classes let you put
arbitrary Python objects into the user session, and
serializes them when pickle.  Again, a malicious client
can make a hostile pickle, send it in a cookie header to
the http server, and take over the server when the 
application unpickles the cookie.

The current documentation for the pickle module makes it
very clear to me that the doc writer thought it was safe
to unpickle untrusted cookies.  If pickle wasn't designed
for that, then there was a communication failure between
the designer and the doc writer.

Yes, I'm willing to help with a PEP for fixing this
situation.

Paul


----------------------------------------------------------------------

Comment By: Jeremy Hylton (jhylton)
Date: 2001-11-08 12:37

Message:
Logged In: YES 
user_id=31392

I don't think of the issue you describe as a bug in the
code.  You're suggesting a new feature for pickle.  As far
as I can tell, the original design requirements for pickle
did not include the ability to securely load a pickle from
an untrusted source.

It may be a legitimate feature request, but it's too late to
make it into Python 2.2.  I suggest we look at the design
issues for Python 2.3 and decide if it's a feature we want
to support.  I imagine a PEP may be necessary to lay out the
issues and the solution.  Do you want to have a hand in that
PEP?

I still don't understand what it means that Pyro and cookie
were bit by a bug.  It sounds like they were using pickle in
ways that pickle was not intended to support.  A careful
analysis of how those two applications use pickle would be
helpful for generating requirements.


----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2001-11-07 18:54

Message:
Logged In: NO 

IMO it's a code bug that you can't unpickle strings from
untrusted sources.  Pyro and the cookie module are examples
of programs that got bitten by this bug.  Whether it's
really a bug is a matter of opinion--I had a big email
exchange with Guido and Tim about it, and they felt it
was enough to fix the pickle documentation.

Pickle has the same problem as cPickle, but with pickle
you can subclass the pickler and override the method that
unpickles class objects, and work around the (IMO) bug.
The workaround doesn't help cPickle since cPickle can't
be subclassed.  See bug #467384 for some related discussion.

Paul


----------------------------------------------------------------------

Comment By: Jeremy Hylton (jhylton)
Date: 2001-11-07 17:02

Message:
Logged In: YES 
user_id=31392

What's the code bug?  Your last message has a lot of gloom
and doom <wink>, but I'm not sure what specific problem
you'd like to see fixed.  Are you saying that something
needs to be fixed in cPickle and not in pickle?

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2001-11-07 15:08

Message:
Logged In: NO 

Irmen de Jong points out that the standard cookie module
uses pickling for serial and smart cookies.  The 2.1.1
cookie module docs explicitly say not to use those
classes because of the security hole--that they're provided
for backward compatibility only (but with what?!  Any
server that uses those classes on open ports needs to be
changed right away).

Irmen's library, http://pyro.sourceforge.net, also uses
unpickle insecurely (he's aware of this now and is figuring
out a fix).

IMO this is really a code bug rather than a documentation
bug, and should be fixed in the code rather than the docs.
Documenting the bug rather than fixing it leaves a
deficiency in the Python library: obvious uses of pickling,
like Pyro and the cookie module, can't be implemented
using cPickle and have to resort to a slower Python
deserializer, or use marshal and have possible compatibility
problems between versions (and not be able to serialize
class instances).

Paul



----------------------------------------------------------------------

Comment By: paul rubin (phr)
Date: 2001-10-16 19:06

Message:
Logged In: YES 
user_id=72053

Certainly anyone unserializing potentially malicious data
with pickle, marshal, or anything else, should check the
results before doing anything dangerous with them (like
executing code).  However, unpickle can potentially do
damage before it even returns, by creating loading modules
and creating initialized class instances.  So pickle.loads
should never be used on untrusted strings, except possibly
in a rexec wrapper (as proposed by Tim).  Another
possibility (also by Tim) is to override the load_inst
method of the Pickler class, though I don't think you can
do that for cPickle.

A sample exploit for unpickle can be found at
<http://www.nightsong.com/phr/python/pickletest.py>.
Unpickling the test string runs penguin.__init__ contrary
to the doc's saying no initialization unless there's a
__getinitargs__ method in the class def.

The "exploding penguin" class is artificial, but
applications are vulnerable if there's an unsafe
constructor anywhere in any class of the application or in
the python library (example: the NNTP constructor opens an
IP connection to an arbitrary address, so a malicious
imported string can send a message through your firewall
when you import it).


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=471893&group_id=5470


From noreply@sourceforge.net  Wed Feb  5 22:08:38 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Wed, 05 Feb 2003 14:08:38 -0800
Subject: [Python-bugs-list] [ python-Bugs-668433 ] Abort with "negative ref count"
Message-ID: <E18gXig-00045s-00@sc8-sf-web4.sourceforge.net>

Bugs item #668433, was opened at 2003-01-15 07:23
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=668433&group_id=5470

Category: Python Interpreter Core
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Walter Dörwald (doerwalter)
Assigned to: Guido van Rossum (gvanrossum)
>Summary: Abort with "negative ref count"

Initial Comment:
Python 2.3a1 and the current CVS version abort with the
following error message when executing the attached
test script:
Fatal Python error:
D:\Python-current\dist\src\Objects\descrobject.c:10
object at 007C3208 has negative ref count -1

This happens only in the debug build, both on Linux
(with configure --enable-unicode=ucs4 --with-pydebug
and 2.3a1) and on Windows (with the debug build and
current CVS).

----------------------------------------------------------------------

>Comment By: Tim Peters (tim_one)
Date: 2003-02-05 17:08

Message:
Logged In: YES 
user_id=31435

Attached a very short test case.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-01-15 16:52

Message:
Logged In: YES 
user_id=6380

Bah. Trashcan code. :-(

OK, I'll look at it.

When I have time.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-01-15 16:37

Message:
Logged In: YES 
user_id=33168

8/6/2002 works without complaints, 8/8/2002 breaks, 8/7/2002
breaks with a different error message: 
Objects/typeobject.c:466: subtype_dealloc: Assertion
`g->gc.gc_refs != (-2)' failed.

All dates are American, ie, August, 6-8.  The change
corresponds to Object/typeobject.c:

8/6 - 2.165 works
8/7 - 2.166 assertion listed above
8/8 - 2.167 assertion in bug report (current)

Guido made all the changes, assigning to him. :-)

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-01-15 15:37

Message:
Logged In: YES 
user_id=33168

I'm not running w/--enable-unicode=ucs4, but I still get the
problem.  I've simplified the test, but only a little so far. 
The problem did not affect 2.2.2, only 2.3.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-01-15 10:04

Message:
Logged In: YES 
user_id=33168

Walter, there's no file attached.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=668433&group_id=5470


From noreply@sourceforge.net  Wed Feb  5 22:40:44 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Wed, 05 Feb 2003 14:40:44 -0800
Subject: [Python-bugs-list] [ python-Bugs-680789 ] Debug with long array takes forever
Message-ID: <E18gYDk-0006sT-00@sc8-sf-web2.sourceforge.net>

Bugs item #680789, was opened at 2003-02-05 12:00
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=680789&group_id=5470

Category: Python Library
Group: Python 2.2
Status: Open
Resolution: None
Priority: 5
Submitted By: Jurjen N.E. Bos (jneb)
Assigned to: Jack Jansen (jackjansen)
Summary: Debug with long array takes forever

Initial Comment:
This is a bug and a partial patch.

If I debug a program that contains a ridiculously large array (8M entries in my case), the debugger takes forever.
It happens in Mac OS X, Python 2.2, but I found the bug in is the repr module, so it is probably universal.
The thing is, that after the fix below, it still doesn't work!
Did I miss something trivial (like repr is builtin, or something like that?). Would someone with Mac OS X experience help out here, please (Jack?).

Here's the diff to make repr.repr work with large arrays:
13a14
>       self.maxarray = 5
50a52,62
>     def repr_array(self, x, level):
>         n = len(x)
>       header = "array('"+x.typecode+"', ["
>         if n == 0: return header+"])"
>         if level <= 0: return header+"...])"
>         s = ''
>         for i in range(min(n, self.maxarray)):
>             if s: s = s + ', '
>             s = s + self.repr1(x[i], level-1)
>         if n > self.maxarray: s = s + ', ...'
>         return header + s + "])"

----------------------------------------------------------------------

>Comment By: Jack Jansen (jackjansen)
Date: 2003-02-05 23:40

Message:
Logged In: YES 
user_id=45365

The fix is fine (it works for me the same way as for Tim), but I think we're shooting past the problem here.

First, pdb doesn't use repr.repr(), it uses the normal builtin repr().

Second, I don't see any sluggishness in pdb with large arrays. I tried
debugging
def foo():
    a = range(8000000)
and there was no problem. Allocating the object took a bit of time, yes, and if you actually try to print it you'll stare at about 800K lines filled with digits scrolling over your screen, but that is to be expected.

Could it be your sluggishness is coming from something else? For example, MacOSX starts behaving *very* badly if your root disk is full, because then it can't allocate swap space, and due to its optimistic behaviour it comes to a grinding halt.

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2003-02-05 19:36

Message:
Logged In: YES 
user_id=31435

Nice to see you, Jurgen!  I checked this into current CVS, 
and it works fine for me in isolation:

>>> len(a)
11055060
>>> repr.repr(a)
"array('i', [0, 1, 2, 3, 4, ...])"
>>>

That goes in an eyeblink.  So more detail is needed about 
what "it still doesn't work!" means.  Assigned to Jack, and he 
can use current CVS to try it.

Lib/repr.py; new revision: 1.15
Lib/test/test_repr.py; new revision: 1.16
Misc/NEWS; new revision: 1.642

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=680789&group_id=5470


From noreply@sourceforge.net  Wed Feb  5 22:46:37 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Wed, 05 Feb 2003 14:46:37 -0800
Subject: [Python-bugs-list] [ python-Bugs-668433 ] Abort with "negative ref count"
Message-ID: <E18gYJR-00075o-00@sc8-sf-web2.sourceforge.net>

Bugs item #668433, was opened at 2003-01-15 07:23
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=668433&group_id=5470

Category: Python Interpreter Core
Group: Python 2.3
>Status: Closed
>Resolution: Fixed
Priority: 5
Submitted By: Walter Dörwald (doerwalter)
Assigned to: Guido van Rossum (gvanrossum)
>Summary: Abort with "negative ref count"

Initial Comment:
Python 2.3a1 and the current CVS version abort with the
following error message when executing the attached
test script:
Fatal Python error:
D:\Python-current\dist\src\Objects\descrobject.c:10
object at 007C3208 has negative ref count -1

This happens only in the debug build, both on Linux
(with configure --enable-unicode=ucs4 --with-pydebug
and 2.3a1) and on Windows (with the debug build and
current CVS).

----------------------------------------------------------------------

>Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-05 17:46

Message:
Logged In: YES 
user_id=6380

Fixed in CVS, typeobject.c 2.203. Much thanks to Tim, who
understands the trashgcan code much better than I do.
Explanation is in the code. I checked that none of the three
examples breaks with the new code.

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2003-02-05 17:08

Message:
Logged In: YES 
user_id=31435

Attached a very short test case.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-01-15 16:52

Message:
Logged In: YES 
user_id=6380

Bah. Trashcan code. :-(

OK, I'll look at it.

When I have time.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-01-15 16:37

Message:
Logged In: YES 
user_id=33168

8/6/2002 works without complaints, 8/8/2002 breaks, 8/7/2002
breaks with a different error message: 
Objects/typeobject.c:466: subtype_dealloc: Assertion
`g->gc.gc_refs != (-2)' failed.

All dates are American, ie, August, 6-8.  The change
corresponds to Object/typeobject.c:

8/6 - 2.165 works
8/7 - 2.166 assertion listed above
8/8 - 2.167 assertion in bug report (current)

Guido made all the changes, assigning to him. :-)

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-01-15 15:37

Message:
Logged In: YES 
user_id=33168

I'm not running w/--enable-unicode=ucs4, but I still get the
problem.  I've simplified the test, but only a little so far. 
The problem did not affect 2.2.2, only 2.3.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-01-15 10:04

Message:
Logged In: YES 
user_id=33168

Walter, there's no file attached.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=668433&group_id=5470


From noreply@sourceforge.net  Wed Feb  5 22:57:39 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Wed, 05 Feb 2003 14:57:39 -0800
Subject: [Python-bugs-list] [ python-Bugs-471893 ] Security review of pickle/marshal docs
Message-ID: <E18gYU7-0000bU-00@sc8-sf-web3.sourceforge.net>

Bugs item #471893, was opened at 2001-10-16 18:42
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=471893&group_id=5470

Category: Extension Modules
Group: None
Status: Open
Resolution: Fixed
Priority: 7
Submitted By: Tim Peters (tim_one)
>Assigned to: A.M. Kuchling (akuchling)
Summary: Security review of pickle/marshal docs

Initial Comment:
Paul Rubin points out that the security implications 
of using marshal and/or pickle aren't clear from the 
docs.  Assigning to Jeremy as he's more sensitive to 
such issues than I am; maybe Barry would like to get 
paranoid too <wink>.

A specific example:  the pickle docs say that pickle 
doesn't support code objects, and "at least this 
avoids the possibility of smuggling Trojan horses into 
a program".  However,

1) The marshal docs don't mention this vulnerability 
at all.

while

2) The pickle docs don't spell out possible dangers 
due to things pickle does that marshal doesn't (like 
importing modules, and running class constructors).

----------------------------------------------------------------------

>Comment By: Tim Peters (tim_one)
Date: 2003-02-05 17:57

Message:
Logged In: YES 
user_id=31435

Andrew, didn't you go overboard in deleting text here?  The 
__safe_for_unpickling__ check is gone in 2.3, but most of the 
rest of the text still applies.  In particular, Cookie is still a 
problem for people inclined to worry about this, and overriding 
methods like find_global and load_global is still valuable stuff 
(the unpickler still never imports anything that doesn't come 
from an opcode triggering one of these methods).  

----------------------------------------------------------------------

Comment By: Barry A. Warsaw (bwarsaw)
Date: 2003-02-05 16:29

Message:
Logged In: YES 
user_id=12800

Karmically (no, not comically) reassigning to Tim who
started this whole thing, and who happens to be dumping his
pickles these days.

----------------------------------------------------------------------

Comment By: A.M. Kuchling (akuchling)
Date: 2003-02-05 15:40

Message:
Logged In: YES 
user_id=11375

Re-opening.  With the changes to pickling in Python 2.3, 
the security material should be replaced with a warning to 
not unpickle untrusted data.  Patch attached.



----------------------------------------------------------------------

Comment By: Barry A. Warsaw (bwarsaw)
Date: 2001-11-18 11:08

Message:
Logged In: YES 
user_id=12800

You're right of course.  I meant to fix that and forgot. 
Will do so asap.  Glad you like the rest of it, though! :)

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2001-11-18 01:56

Message:
Logged In: NO 

Barry - the new docs just went up and they're a big
improvement over the old.  However the sentence 
"The issue here is usually not that a class's constructor
will get called -- it won't by the unpickler -- but that the
class's destructor (i.e. its __del__() method) might get
called when the object is garbage collected." isn't
correct.  

There's a flag in the pickle stream that tells
the unpickler that the pickler found a __getinitargs__
method at pickling time.  If the flag is set in the pickle,
then the unpickler calls the class constructor, whether
there's a __getinitargs__ method in the class or not.

The sample exploit that I posted earlier on,
<http://www.nightsong.com/phr/python/pickletest.py>,
is an example of an artificially concocted pickle that
makes a class constructor get called.


----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2001-11-17 17:07

Message:
Logged In: NO 

Are the new docs downloadable from somewhere?  thanx  --Paul


----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2001-11-15 19:43

Message:
Logged In: NO 

Barry, can you also do something about the Cookie module,
or at least about its documentation?  If Cookie is aliased
to SmartCookie, I think that needs mention.
 --Paul

----------------------------------------------------------------------

Comment By: Barry A. Warsaw (bwarsaw)
Date: 2001-11-15 19:12

Message:
Logged In: YES 
user_id=12800

I have rewritten the pickle documentation, and updated the
marshal, cPickle, and copy_reg documentation.  I believe all
the documentation issues raised here have been fixed.  

Implementation issues will be pushed to Python 2.3, and the
plan is to write a PEP to plan future work.

----------------------------------------------------------------------

Comment By: Barry A. Warsaw (bwarsaw)
Date: 2001-11-13 16:39

Message:
Logged In: YES 
user_id=12800

Assigning to Jeremy so he'll remember to forward me Jim's
email.  Jeremy can ping-pong it to Fred if he wants, and
then reassign to me after forwarding the email.

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2001-11-13 01:18

Message:
Logged In: NO 

The find_global variable sounds encouraging and if it
fixes this problem, that's great.  I'd still feel better
if the eval call goes away.  Is removing it allowed
under the 2.2 feature freeze?  It doesn't affect anything
visible, so no tests would have to change, etc.

Paul


----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2001-11-12 19:38

Message:
Logged In: NO 

It's possible that the eval is safe, but validating that
takes a line by line code review of all the paths that
evaling a string can go through.  It's also brittle in
that maybe someone will change the evaluator (say to
support Perl-like interpolation) in a future Python version
and not remember to change the unpickler.  Something like
that has already happened with the Cookie module.  My
guess is that it happened with the pickle module--the
whole point of that quoted string check is that the 
original pickle implementer didn't trust the input. 
The stuff about unpickling class instances was added
later (maybe by another person) without remembering the
security issue. 

Using eval this way is like storing a vat of cyanide in
your child's bedroom.  Sure, maybe if you check the seals
and locks on the vat carefully enough, you can convince
yourself that your child won't be able to get to the
cyanide.  But wouldn't you feel safer just not having the
vat there at all?  That's basic safety-consciousness.
Security consciousness works the same way.  Try to keep
dangerous ingredients and attackers as far away from each
other as possible.

Paul



----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2001-11-12 13:52

Message:
Logged In: YES 
user_id=31435

Why are people (Paul, Jeremy) concerned about eval'ing 
strings?  cPickle and pickle both check that they're 
properly quoted, and this isn't sh or Perl:  Python has 
no "dynamic" gimmicks buried in string literals.  All 
eval'ing a string literal can do is produce a binary blob 
via interpeting simple escape sequences.  They're like C 
strings this way -- maybe we'll run out of memory, but 
that's it.

I would agree that Python should be refactored internally 
to supply a clean function for changing string literals 
into binary blobs, but that would be for clarity and 
efficiency, not security.

----------------------------------------------------------------------

Comment By: Jim Fulton (dcjim)
Date: 2001-11-12 13:14

Message:
Logged In: YES 
user_id=73023

It sounds like there are some documentation bugs:

- The security ramifications are not discussed, nor are the
remedies.

- The cPickle module isn't documented very well. I submitted
some
   documentation a long time ago that never got incorporated
AFAIK.
   I wish I still had it. :)

- cPickle has a feature for turning off instance support and
for 
   restricting which classes can be unpickled. You can set
the find_global
   attribute on a cPickle.Unpickler. The find_global
attribute can be 
   a function or None.  If it is None, then no instances can
be 
   unpickled. If it is a function, then it should accept a
module and name
   and return the corresponding global. It is responsible
for looking
   up the global and can raise an error to prevent a
global's use.

   See the ZEO storage server implementation for an example
   of using this hook.


  



----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2001-11-12 01:33

Message:
Logged In: NO 

I like marshal and think it's much cleaner than pickle.
I'd like to see its format cleaned up and documented
so it can be a portable transport format.  Even the
recursion issue isn't so compelling--there's only a danger
on marshalling, not unmarshalling, and the application
has control over what it marshals.  So I think the idea
of documenting marshal (bug #467384) should be kept alive.
However, it shouldn't be changed in 2.2.  

If the Cookie class is still really aliased by default to
SmartCookie, that's a bad bug and should definitely be
fixed ASAP.  The Cookie docs clearly say not to use
SmartCookie or SerialCookie.  In fact I think SmartCookie
and SerialCookie should be disabled altogether.  Any
applications using them on the open internet have a security
but and should be fixed immediately.

Here's a possible temporary approach to fixing SmartCookie
and SerialCookie: append a cryptographic message
authentication code (MAC) to each pickled cookie.  That
means the application has some secret string K as a
configuration parameter (it should be 10 or more random
8-bit characters, or 20 or more random hex digits, etc). 
Then a smart cookie would be a pickle p, followed by
SHA(p+K).  The Cookie module would validate the MAC before
calling unpickle, and raise an exception if the MAC wasn't
valid.

The security of this scheme rests on K being kept secret
from attackers, which is generally not an easy thing to
manage, but it's something to think about.

Paul

----------------------------------------------------------------------

Comment By: Jeremy Hylton (jhylton)
Date: 2001-11-10 17:07

Message:
Logged In: YES 
user_id=31392

I don't think we should be doing anything about marshal.  
Maybe we should name in pyclib or something <0.9 wink>.  It 
works fine for .pyc files, but I don't see a reason for it 
to do anymore than is necessary for that purpose.

I think the notion of an unpickler that only handles 
builtin datatypes is the most attractive option you offer, 
but Paul has a good point about eval for strings.  (It 
currently has some hacks that address specific exploits, 
but I doubt they are sufficient.)  I also wonder how hard 
it is to handle builtin types and avoid subclasses of 
builtin types.

If there are any changes to pickle, I think we need to be 
careful about how it is described.  If we claim that an 
unpickler is safe for untrusted pickles, we've made a 
fairly strong claim.  I still think such a design change 
requires a PEP that includes some requirements and use 
cases and a thorough analysis of potential exploits. 


----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2001-11-10 14:42

Message:
Logged In: NO 

See bug #467384 for discussion about marshal.  Besides the
recursion issue, marshal's format is explicitly undocumented
and subject to change--you can't rely on it to interoperate
between two different Python versions, so it's no good as
an RPC serializer.  The format has kludges (e.g. the
representation of long ints) that make it undesirable to
freeze and document it and force future versions to be
backward compatible.

Adding a pickle.loads flag to prevent instance unpickling
isn't perfect but is probably the best alternative on 
your list.  Perhaps the flag can have a value that allows
unpickling the instances by restoring the instance 
attributes rather than calling the initializer.  That's
not always the right way to unpickle an instance (that's
why the unpickler no longer works that way) but it's good
enough a lot of the time.  

There's another issue with pickle/cPickle which is that they
unpickle quoted strings by evaling them.  This is scary.
While I don't see an immediate exploit, I also haven't
examined the 1000's of lines of code I'd need to examine
to convince myself that there's NOT an exploit.  I think
the unpickler should be changed to never call eval but just
parse the string as it needs to.  

Guido seemed to think pickle might have other possible
exploits.  I don't know what he had in mind but before
declaring it safe for untrusted data I think it needs to
be gone over with a fine toothed comb.

Paul


----------------------------------------------------------------------

Comment By: Barry A. Warsaw (bwarsaw)
Date: 2001-11-10 12:37

Message:
Logged In: YES 
user_id=12800

I'm going to agree with Paul that this is a problem needing
fixing, however there are really several issues.

1. Cookie module makes it too easy to code exploits.  Cookie
exports a class, also called Cookie, which is aliased to
SmartCookie, so that a naive program will simply pass cookie
data to Cookie.Cookie() and you're screwed.  So, Cookie
module's defaults should be for more security rather than
less, and Cookie.Cookie should be aliased to SimpleCookie
instead.

2. There is no built-in safe mechanism for de-serializing
data from untrusted sources.  You can't use pickle without
overloading a "magic" method.  You can't use cPickle because
you can't do the overloading trick.  You can't use marshal
because it isn't bulletproof against recursive
datastructures.  So how /do/ you do it?

I think it just may be serious enough to deal with in Python
2.2, and I volunteer to address it (so I'll steal this bug
report).  Without looking at the code, or the level of
effort necessary, I would make the following suggestions:

1. Add to the public interface of pickle and cPickle, a flag
that either disables the unpickling of instances altogether,
or disables calling any code with unpickled data, e.g.
constructors.

2. Fix marshal to be bulletproof against recursive
datastructures.
3. Update the docs for both pickle/cPickle and marshal to
explain how to safely write de-serializers of untrusted strings.

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2001-11-09 04:21

Message:
Logged In: NO 

Well, Guido and Tim agree with you that it's not a pickle
bug.  I still feel it is one, because its docs currently
make you think you can securely load untrusted pickles, and
because it's a natural, non-obscure thing to want to do
(see pyro and the cookie module), but whatever.  If it's
not a code bug then I feel it's a significant functionality
shortcoming in the python library.

Pyro uses pickle to serialize data for RPC calls over the
internet.  A malicious client could make a hostile pickle
take over the server.  The cookie module lets web
applications store user session state in browser cookies.
Its SerialCookie and SmartCookie classes let you put
arbitrary Python objects into the user session, and
serializes them when pickle.  Again, a malicious client
can make a hostile pickle, send it in a cookie header to
the http server, and take over the server when the 
application unpickles the cookie.

The current documentation for the pickle module makes it
very clear to me that the doc writer thought it was safe
to unpickle untrusted cookies.  If pickle wasn't designed
for that, then there was a communication failure between
the designer and the doc writer.

Yes, I'm willing to help with a PEP for fixing this
situation.

Paul


----------------------------------------------------------------------

Comment By: Jeremy Hylton (jhylton)
Date: 2001-11-08 12:37

Message:
Logged In: YES 
user_id=31392

I don't think of the issue you describe as a bug in the
code.  You're suggesting a new feature for pickle.  As far
as I can tell, the original design requirements for pickle
did not include the ability to securely load a pickle from
an untrusted source.

It may be a legitimate feature request, but it's too late to
make it into Python 2.2.  I suggest we look at the design
issues for Python 2.3 and decide if it's a feature we want
to support.  I imagine a PEP may be necessary to lay out the
issues and the solution.  Do you want to have a hand in that
PEP?

I still don't understand what it means that Pyro and cookie
were bit by a bug.  It sounds like they were using pickle in
ways that pickle was not intended to support.  A careful
analysis of how those two applications use pickle would be
helpful for generating requirements.


----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2001-11-07 18:54

Message:
Logged In: NO 

IMO it's a code bug that you can't unpickle strings from
untrusted sources.  Pyro and the cookie module are examples
of programs that got bitten by this bug.  Whether it's
really a bug is a matter of opinion--I had a big email
exchange with Guido and Tim about it, and they felt it
was enough to fix the pickle documentation.

Pickle has the same problem as cPickle, but with pickle
you can subclass the pickler and override the method that
unpickles class objects, and work around the (IMO) bug.
The workaround doesn't help cPickle since cPickle can't
be subclassed.  See bug #467384 for some related discussion.

Paul


----------------------------------------------------------------------

Comment By: Jeremy Hylton (jhylton)
Date: 2001-11-07 17:02

Message:
Logged In: YES 
user_id=31392

What's the code bug?  Your last message has a lot of gloom
and doom <wink>, but I'm not sure what specific problem
you'd like to see fixed.  Are you saying that something
needs to be fixed in cPickle and not in pickle?

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2001-11-07 15:08

Message:
Logged In: NO 

Irmen de Jong points out that the standard cookie module
uses pickling for serial and smart cookies.  The 2.1.1
cookie module docs explicitly say not to use those
classes because of the security hole--that they're provided
for backward compatibility only (but with what?!  Any
server that uses those classes on open ports needs to be
changed right away).

Irmen's library, http://pyro.sourceforge.net, also uses
unpickle insecurely (he's aware of this now and is figuring
out a fix).

IMO this is really a code bug rather than a documentation
bug, and should be fixed in the code rather than the docs.
Documenting the bug rather than fixing it leaves a
deficiency in the Python library: obvious uses of pickling,
like Pyro and the cookie module, can't be implemented
using cPickle and have to resort to a slower Python
deserializer, or use marshal and have possible compatibility
problems between versions (and not be able to serialize
class instances).

Paul



----------------------------------------------------------------------

Comment By: paul rubin (phr)
Date: 2001-10-16 19:06

Message:
Logged In: YES 
user_id=72053

Certainly anyone unserializing potentially malicious data
with pickle, marshal, or anything else, should check the
results before doing anything dangerous with them (like
executing code).  However, unpickle can potentially do
damage before it even returns, by creating loading modules
and creating initialized class instances.  So pickle.loads
should never be used on untrusted strings, except possibly
in a rexec wrapper (as proposed by Tim).  Another
possibility (also by Tim) is to override the load_inst
method of the Pickler class, though I don't think you can
do that for cPickle.

A sample exploit for unpickle can be found at
<http://www.nightsong.com/phr/python/pickletest.py>.
Unpickling the test string runs penguin.__init__ contrary
to the doc's saying no initialization unless there's a
__getinitargs__ method in the class def.

The "exploding penguin" class is artificial, but
applications are vulnerable if there's an unsafe
constructor anywhere in any class of the application or in
the python library (example: the NNTP constructor opens an
IP connection to an arbitrary address, so a malicious
imported string can send a message through your firewall
when you import it).


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=471893&group_id=5470


From noreply@sourceforge.net  Wed Feb  5 23:02:00 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Wed, 05 Feb 2003 15:02:00 -0800
Subject: [Python-bugs-list] [ python-Bugs-681325 ] Should Mac/scripts/makeclean.py go?
Message-ID: <E18gYYK-0000dO-00@sc8-sf-web1.sourceforge.net>

Bugs item #681325, was opened at 2003-02-06 00:02
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=681325&group_id=5470

Category: Macintosh
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Jack Jansen (jackjansen)
Assigned to: Just van Rossum (jvr)
Summary: Should Mac/scripts/makeclean.py go?

Initial Comment:
I think Mac/scripts/makeclean.py doesn't serve a useful purpose anymore. If you agree please move it to Unsupported.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=681325&group_id=5470


From noreply@sourceforge.net  Wed Feb  5 23:03:43 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Wed, 05 Feb 2003 15:03:43 -0800
Subject: [Python-bugs-list] [ python-Bugs-471893 ] Security review of pickle/marshal docs
Message-ID: <E18gYZz-0000sF-00@sc8-sf-web3.sourceforge.net>

Bugs item #471893, was opened at 2001-10-16 18:42
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=471893&group_id=5470

Category: Extension Modules
Group: None
Status: Open
Resolution: Fixed
Priority: 7
Submitted By: Tim Peters (tim_one)
Assigned to: A.M. Kuchling (akuchling)
Summary: Security review of pickle/marshal docs

Initial Comment:
Paul Rubin points out that the security implications 
of using marshal and/or pickle aren't clear from the 
docs.  Assigning to Jeremy as he's more sensitive to 
such issues than I am; maybe Barry would like to get 
paranoid too <wink>.

A specific example:  the pickle docs say that pickle 
doesn't support code objects, and "at least this 
avoids the possibility of smuggling Trojan horses into 
a program".  However,

1) The marshal docs don't mention this vulnerability 
at all.

while

2) The pickle docs don't spell out possible dangers 
due to things pickle does that marshal doesn't (like 
importing modules, and running class constructors).

----------------------------------------------------------------------

>Comment By: Barry A. Warsaw (bwarsaw)
Date: 2003-02-05 18:03

Message:
Logged In: YES 
user_id=12800

I'll just mention that anybody using anything other than
Cookie.SimpleCookie is insane, and even using the Cookie
module at all to parse real-world cookies is mildly nuts
because it's way too strict.  I prefer m&m's in my cookies.

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2003-02-05 17:57

Message:
Logged In: YES 
user_id=31435

Andrew, didn't you go overboard in deleting text here?  The 
__safe_for_unpickling__ check is gone in 2.3, but most of the 
rest of the text still applies.  In particular, Cookie is still a 
problem for people inclined to worry about this, and overriding 
methods like find_global and load_global is still valuable stuff 
(the unpickler still never imports anything that doesn't come 
from an opcode triggering one of these methods).  

----------------------------------------------------------------------

Comment By: Barry A. Warsaw (bwarsaw)
Date: 2003-02-05 16:29

Message:
Logged In: YES 
user_id=12800

Karmically (no, not comically) reassigning to Tim who
started this whole thing, and who happens to be dumping his
pickles these days.

----------------------------------------------------------------------

Comment By: A.M. Kuchling (akuchling)
Date: 2003-02-05 15:40

Message:
Logged In: YES 
user_id=11375

Re-opening.  With the changes to pickling in Python 2.3, 
the security material should be replaced with a warning to 
not unpickle untrusted data.  Patch attached.



----------------------------------------------------------------------

Comment By: Barry A. Warsaw (bwarsaw)
Date: 2001-11-18 11:08

Message:
Logged In: YES 
user_id=12800

You're right of course.  I meant to fix that and forgot. 
Will do so asap.  Glad you like the rest of it, though! :)

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2001-11-18 01:56

Message:
Logged In: NO 

Barry - the new docs just went up and they're a big
improvement over the old.  However the sentence 
"The issue here is usually not that a class's constructor
will get called -- it won't by the unpickler -- but that the
class's destructor (i.e. its __del__() method) might get
called when the object is garbage collected." isn't
correct.  

There's a flag in the pickle stream that tells
the unpickler that the pickler found a __getinitargs__
method at pickling time.  If the flag is set in the pickle,
then the unpickler calls the class constructor, whether
there's a __getinitargs__ method in the class or not.

The sample exploit that I posted earlier on,
<http://www.nightsong.com/phr/python/pickletest.py>,
is an example of an artificially concocted pickle that
makes a class constructor get called.


----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2001-11-17 17:07

Message:
Logged In: NO 

Are the new docs downloadable from somewhere?  thanx  --Paul


----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2001-11-15 19:43

Message:
Logged In: NO 

Barry, can you also do something about the Cookie module,
or at least about its documentation?  If Cookie is aliased
to SmartCookie, I think that needs mention.
 --Paul

----------------------------------------------------------------------

Comment By: Barry A. Warsaw (bwarsaw)
Date: 2001-11-15 19:12

Message:
Logged In: YES 
user_id=12800

I have rewritten the pickle documentation, and updated the
marshal, cPickle, and copy_reg documentation.  I believe all
the documentation issues raised here have been fixed.  

Implementation issues will be pushed to Python 2.3, and the
plan is to write a PEP to plan future work.

----------------------------------------------------------------------

Comment By: Barry A. Warsaw (bwarsaw)
Date: 2001-11-13 16:39

Message:
Logged In: YES 
user_id=12800

Assigning to Jeremy so he'll remember to forward me Jim's
email.  Jeremy can ping-pong it to Fred if he wants, and
then reassign to me after forwarding the email.

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2001-11-13 01:18

Message:
Logged In: NO 

The find_global variable sounds encouraging and if it
fixes this problem, that's great.  I'd still feel better
if the eval call goes away.  Is removing it allowed
under the 2.2 feature freeze?  It doesn't affect anything
visible, so no tests would have to change, etc.

Paul


----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2001-11-12 19:38

Message:
Logged In: NO 

It's possible that the eval is safe, but validating that
takes a line by line code review of all the paths that
evaling a string can go through.  It's also brittle in
that maybe someone will change the evaluator (say to
support Perl-like interpolation) in a future Python version
and not remember to change the unpickler.  Something like
that has already happened with the Cookie module.  My
guess is that it happened with the pickle module--the
whole point of that quoted string check is that the 
original pickle implementer didn't trust the input. 
The stuff about unpickling class instances was added
later (maybe by another person) without remembering the
security issue. 

Using eval this way is like storing a vat of cyanide in
your child's bedroom.  Sure, maybe if you check the seals
and locks on the vat carefully enough, you can convince
yourself that your child won't be able to get to the
cyanide.  But wouldn't you feel safer just not having the
vat there at all?  That's basic safety-consciousness.
Security consciousness works the same way.  Try to keep
dangerous ingredients and attackers as far away from each
other as possible.

Paul



----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2001-11-12 13:52

Message:
Logged In: YES 
user_id=31435

Why are people (Paul, Jeremy) concerned about eval'ing 
strings?  cPickle and pickle both check that they're 
properly quoted, and this isn't sh or Perl:  Python has 
no "dynamic" gimmicks buried in string literals.  All 
eval'ing a string literal can do is produce a binary blob 
via interpeting simple escape sequences.  They're like C 
strings this way -- maybe we'll run out of memory, but 
that's it.

I would agree that Python should be refactored internally 
to supply a clean function for changing string literals 
into binary blobs, but that would be for clarity and 
efficiency, not security.

----------------------------------------------------------------------

Comment By: Jim Fulton (dcjim)
Date: 2001-11-12 13:14

Message:
Logged In: YES 
user_id=73023

It sounds like there are some documentation bugs:

- The security ramifications are not discussed, nor are the
remedies.

- The cPickle module isn't documented very well. I submitted
some
   documentation a long time ago that never got incorporated
AFAIK.
   I wish I still had it. :)

- cPickle has a feature for turning off instance support and
for 
   restricting which classes can be unpickled. You can set
the find_global
   attribute on a cPickle.Unpickler. The find_global
attribute can be 
   a function or None.  If it is None, then no instances can
be 
   unpickled. If it is a function, then it should accept a
module and name
   and return the corresponding global. It is responsible
for looking
   up the global and can raise an error to prevent a
global's use.

   See the ZEO storage server implementation for an example
   of using this hook.


  



----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2001-11-12 01:33

Message:
Logged In: NO 

I like marshal and think it's much cleaner than pickle.
I'd like to see its format cleaned up and documented
so it can be a portable transport format.  Even the
recursion issue isn't so compelling--there's only a danger
on marshalling, not unmarshalling, and the application
has control over what it marshals.  So I think the idea
of documenting marshal (bug #467384) should be kept alive.
However, it shouldn't be changed in 2.2.  

If the Cookie class is still really aliased by default to
SmartCookie, that's a bad bug and should definitely be
fixed ASAP.  The Cookie docs clearly say not to use
SmartCookie or SerialCookie.  In fact I think SmartCookie
and SerialCookie should be disabled altogether.  Any
applications using them on the open internet have a security
but and should be fixed immediately.

Here's a possible temporary approach to fixing SmartCookie
and SerialCookie: append a cryptographic message
authentication code (MAC) to each pickled cookie.  That
means the application has some secret string K as a
configuration parameter (it should be 10 or more random
8-bit characters, or 20 or more random hex digits, etc). 
Then a smart cookie would be a pickle p, followed by
SHA(p+K).  The Cookie module would validate the MAC before
calling unpickle, and raise an exception if the MAC wasn't
valid.

The security of this scheme rests on K being kept secret
from attackers, which is generally not an easy thing to
manage, but it's something to think about.

Paul

----------------------------------------------------------------------

Comment By: Jeremy Hylton (jhylton)
Date: 2001-11-10 17:07

Message:
Logged In: YES 
user_id=31392

I don't think we should be doing anything about marshal.  
Maybe we should name in pyclib or something <0.9 wink>.  It 
works fine for .pyc files, but I don't see a reason for it 
to do anymore than is necessary for that purpose.

I think the notion of an unpickler that only handles 
builtin datatypes is the most attractive option you offer, 
but Paul has a good point about eval for strings.  (It 
currently has some hacks that address specific exploits, 
but I doubt they are sufficient.)  I also wonder how hard 
it is to handle builtin types and avoid subclasses of 
builtin types.

If there are any changes to pickle, I think we need to be 
careful about how it is described.  If we claim that an 
unpickler is safe for untrusted pickles, we've made a 
fairly strong claim.  I still think such a design change 
requires a PEP that includes some requirements and use 
cases and a thorough analysis of potential exploits. 


----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2001-11-10 14:42

Message:
Logged In: NO 

See bug #467384 for discussion about marshal.  Besides the
recursion issue, marshal's format is explicitly undocumented
and subject to change--you can't rely on it to interoperate
between two different Python versions, so it's no good as
an RPC serializer.  The format has kludges (e.g. the
representation of long ints) that make it undesirable to
freeze and document it and force future versions to be
backward compatible.

Adding a pickle.loads flag to prevent instance unpickling
isn't perfect but is probably the best alternative on 
your list.  Perhaps the flag can have a value that allows
unpickling the instances by restoring the instance 
attributes rather than calling the initializer.  That's
not always the right way to unpickle an instance (that's
why the unpickler no longer works that way) but it's good
enough a lot of the time.  

There's another issue with pickle/cPickle which is that they
unpickle quoted strings by evaling them.  This is scary.
While I don't see an immediate exploit, I also haven't
examined the 1000's of lines of code I'd need to examine
to convince myself that there's NOT an exploit.  I think
the unpickler should be changed to never call eval but just
parse the string as it needs to.  

Guido seemed to think pickle might have other possible
exploits.  I don't know what he had in mind but before
declaring it safe for untrusted data I think it needs to
be gone over with a fine toothed comb.

Paul


----------------------------------------------------------------------

Comment By: Barry A. Warsaw (bwarsaw)
Date: 2001-11-10 12:37

Message:
Logged In: YES 
user_id=12800

I'm going to agree with Paul that this is a problem needing
fixing, however there are really several issues.

1. Cookie module makes it too easy to code exploits.  Cookie
exports a class, also called Cookie, which is aliased to
SmartCookie, so that a naive program will simply pass cookie
data to Cookie.Cookie() and you're screwed.  So, Cookie
module's defaults should be for more security rather than
less, and Cookie.Cookie should be aliased to SimpleCookie
instead.

2. There is no built-in safe mechanism for de-serializing
data from untrusted sources.  You can't use pickle without
overloading a "magic" method.  You can't use cPickle because
you can't do the overloading trick.  You can't use marshal
because it isn't bulletproof against recursive
datastructures.  So how /do/ you do it?

I think it just may be serious enough to deal with in Python
2.2, and I volunteer to address it (so I'll steal this bug
report).  Without looking at the code, or the level of
effort necessary, I would make the following suggestions:

1. Add to the public interface of pickle and cPickle, a flag
that either disables the unpickling of instances altogether,
or disables calling any code with unpickled data, e.g.
constructors.

2. Fix marshal to be bulletproof against recursive
datastructures.
3. Update the docs for both pickle/cPickle and marshal to
explain how to safely write de-serializers of untrusted strings.

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2001-11-09 04:21

Message:
Logged In: NO 

Well, Guido and Tim agree with you that it's not a pickle
bug.  I still feel it is one, because its docs currently
make you think you can securely load untrusted pickles, and
because it's a natural, non-obscure thing to want to do
(see pyro and the cookie module), but whatever.  If it's
not a code bug then I feel it's a significant functionality
shortcoming in the python library.

Pyro uses pickle to serialize data for RPC calls over the
internet.  A malicious client could make a hostile pickle
take over the server.  The cookie module lets web
applications store user session state in browser cookies.
Its SerialCookie and SmartCookie classes let you put
arbitrary Python objects into the user session, and
serializes them when pickle.  Again, a malicious client
can make a hostile pickle, send it in a cookie header to
the http server, and take over the server when the 
application unpickles the cookie.

The current documentation for the pickle module makes it
very clear to me that the doc writer thought it was safe
to unpickle untrusted cookies.  If pickle wasn't designed
for that, then there was a communication failure between
the designer and the doc writer.

Yes, I'm willing to help with a PEP for fixing this
situation.

Paul


----------------------------------------------------------------------

Comment By: Jeremy Hylton (jhylton)
Date: 2001-11-08 12:37

Message:
Logged In: YES 
user_id=31392

I don't think of the issue you describe as a bug in the
code.  You're suggesting a new feature for pickle.  As far
as I can tell, the original design requirements for pickle
did not include the ability to securely load a pickle from
an untrusted source.

It may be a legitimate feature request, but it's too late to
make it into Python 2.2.  I suggest we look at the design
issues for Python 2.3 and decide if it's a feature we want
to support.  I imagine a PEP may be necessary to lay out the
issues and the solution.  Do you want to have a hand in that
PEP?

I still don't understand what it means that Pyro and cookie
were bit by a bug.  It sounds like they were using pickle in
ways that pickle was not intended to support.  A careful
analysis of how those two applications use pickle would be
helpful for generating requirements.


----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2001-11-07 18:54

Message:
Logged In: NO 

IMO it's a code bug that you can't unpickle strings from
untrusted sources.  Pyro and the cookie module are examples
of programs that got bitten by this bug.  Whether it's
really a bug is a matter of opinion--I had a big email
exchange with Guido and Tim about it, and they felt it
was enough to fix the pickle documentation.

Pickle has the same problem as cPickle, but with pickle
you can subclass the pickler and override the method that
unpickles class objects, and work around the (IMO) bug.
The workaround doesn't help cPickle since cPickle can't
be subclassed.  See bug #467384 for some related discussion.

Paul


----------------------------------------------------------------------

Comment By: Jeremy Hylton (jhylton)
Date: 2001-11-07 17:02

Message:
Logged In: YES 
user_id=31392

What's the code bug?  Your last message has a lot of gloom
and doom <wink>, but I'm not sure what specific problem
you'd like to see fixed.  Are you saying that something
needs to be fixed in cPickle and not in pickle?

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2001-11-07 15:08

Message:
Logged In: NO 

Irmen de Jong points out that the standard cookie module
uses pickling for serial and smart cookies.  The 2.1.1
cookie module docs explicitly say not to use those
classes because of the security hole--that they're provided
for backward compatibility only (but with what?!  Any
server that uses those classes on open ports needs to be
changed right away).

Irmen's library, http://pyro.sourceforge.net, also uses
unpickle insecurely (he's aware of this now and is figuring
out a fix).

IMO this is really a code bug rather than a documentation
bug, and should be fixed in the code rather than the docs.
Documenting the bug rather than fixing it leaves a
deficiency in the Python library: obvious uses of pickling,
like Pyro and the cookie module, can't be implemented
using cPickle and have to resort to a slower Python
deserializer, or use marshal and have possible compatibility
problems between versions (and not be able to serialize
class instances).

Paul



----------------------------------------------------------------------

Comment By: paul rubin (phr)
Date: 2001-10-16 19:06

Message:
Logged In: YES 
user_id=72053

Certainly anyone unserializing potentially malicious data
with pickle, marshal, or anything else, should check the
results before doing anything dangerous with them (like
executing code).  However, unpickle can potentially do
damage before it even returns, by creating loading modules
and creating initialized class instances.  So pickle.loads
should never be used on untrusted strings, except possibly
in a rexec wrapper (as proposed by Tim).  Another
possibility (also by Tim) is to override the load_inst
method of the Pickler class, though I don't think you can
do that for cPickle.

A sample exploit for unpickle can be found at
<http://www.nightsong.com/phr/python/pickletest.py>.
Unpickling the test string runs penguin.__init__ contrary
to the doc's saying no initialization unless there's a
__getinitargs__ method in the class def.

The "exploding penguin" class is artificial, but
applications are vulnerable if there's an unsafe
constructor anywhere in any class of the application or in
the python library (example: the NNTP constructor opens an
IP connection to an arbitrary address, so a malicious
imported string can send a message through your firewall
when you import it).


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=471893&group_id=5470


From noreply@sourceforge.net  Wed Feb  5 23:06:12 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Wed, 05 Feb 2003 15:06:12 -0800
Subject: [Python-bugs-list] [ python-Bugs-681325 ] Should Mac/scripts/makeclean.py go?
Message-ID: <E18gYcO-0006sv-00@sc8-sf-web4.sourceforge.net>

Bugs item #681325, was opened at 2003-02-06 00:02
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=681325&group_id=5470

Category: Macintosh
Group: None
>Status: Closed
>Resolution: Fixed
Priority: 5
Submitted By: Jack Jansen (jackjansen)
Assigned to: Just van Rossum (jvr)
Summary: Should Mac/scripts/makeclean.py go?

Initial Comment:
I think Mac/scripts/makeclean.py doesn't serve a useful purpose anymore. If you agree please move it to Unsupported.

----------------------------------------------------------------------

>Comment By: Just van Rossum (jvr)
Date: 2003-02-06 00:06

Message:
Logged In: YES 
user_id=92689

I don't even remeber _writing_ this script. Darn CVS logs ;-)

It's gone. I don't see the point of even moving it to Unsupported.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=681325&group_id=5470


From noreply@sourceforge.net  Wed Feb  5 23:08:39 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Wed, 05 Feb 2003 15:08:39 -0800
Subject: [Python-bugs-list] [ python-Bugs-680789 ] Debug with long array takes forever
Message-ID: <E18gYel-00015l-00@sc8-sf-web3.sourceforge.net>

Bugs item #680789, was opened at 2003-02-05 06:00
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=680789&group_id=5470

Category: Python Library
Group: Python 2.2
Status: Open
Resolution: None
Priority: 5
Submitted By: Jurjen N.E. Bos (jneb)
Assigned to: Jack Jansen (jackjansen)
Summary: Debug with long array takes forever

Initial Comment:
This is a bug and a partial patch.

If I debug a program that contains a ridiculously large array (8M entries in my case), the debugger takes forever.
It happens in Mac OS X, Python 2.2, but I found the bug in is the repr module, so it is probably universal.
The thing is, that after the fix below, it still doesn't work!
Did I miss something trivial (like repr is builtin, or something like that?). Would someone with Mac OS X experience help out here, please (Jack?).

Here's the diff to make repr.repr work with large arrays:
13a14
>       self.maxarray = 5
50a52,62
>     def repr_array(self, x, level):
>         n = len(x)
>       header = "array('"+x.typecode+"', ["
>         if n == 0: return header+"])"
>         if level <= 0: return header+"...])"
>         s = ''
>         for i in range(min(n, self.maxarray)):
>             if s: s = s + ', '
>             s = s + self.repr1(x[i], level-1)
>         if n > self.maxarray: s = s + ', ...'
>         return header + s + "])"

----------------------------------------------------------------------

>Comment By: Tim Peters (tim_one)
Date: 2003-02-05 18:08

Message:
Logged In: YES 
user_id=31435

pdb does import repr.py, but probably doesn't use it in 
whatever way Jurjen is using to display his big array.

WRT that, note that Jurjen is using array.array objects, not 
lists.  The internal array.array tp_repr slot is quadratic-time in 
the size of the array, while list's tp_repr is linear time.

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-02-05 17:40

Message:
Logged In: YES 
user_id=45365

The fix is fine (it works for me the same way as for Tim), but I think we're shooting past the problem here.

First, pdb doesn't use repr.repr(), it uses the normal builtin repr().

Second, I don't see any sluggishness in pdb with large arrays. I tried
debugging
def foo():
    a = range(8000000)
and there was no problem. Allocating the object took a bit of time, yes, and if you actually try to print it you'll stare at about 800K lines filled with digits scrolling over your screen, but that is to be expected.

Could it be your sluggishness is coming from something else? For example, MacOSX starts behaving *very* badly if your root disk is full, because then it can't allocate swap space, and due to its optimistic behaviour it comes to a grinding halt.

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2003-02-05 13:36

Message:
Logged In: YES 
user_id=31435

Nice to see you, Jurgen!  I checked this into current CVS, 
and it works fine for me in isolation:

>>> len(a)
11055060
>>> repr.repr(a)
"array('i', [0, 1, 2, 3, 4, ...])"
>>>

That goes in an eyeblink.  So more detail is needed about 
what "it still doesn't work!" means.  Assigned to Jack, and he 
can use current CVS to try it.

Lib/repr.py; new revision: 1.15
Lib/test/test_repr.py; new revision: 1.16
Misc/NEWS; new revision: 1.642

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=680789&group_id=5470


From noreply@sourceforge.net  Thu Feb  6 00:23:40 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Wed, 05 Feb 2003 16:23:40 -0800
Subject: [Python-bugs-list] [ python-Bugs-681367 ] C subtypes do not inherit tp_as_buffer
Message-ID: <E18gZpM-0002PY-00@sc8-sf-web2.sourceforge.net>

Bugs item #681367, was opened at 2003-02-05 15:23
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=681367&group_id=5470

Category: Python Interpreter Core
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Greg Chapman (glchapman)
Assigned to: Nobody/Anonymous (nobody)
Summary: C subtypes do not inherit tp_as_buffer

Initial Comment:
Subtypes of str (for example) implemented in C do not 
inherit str's tp_as_buffer funcs unless the subtypes provide 
a pointer to a tp_as_buffer structure in their type struct.  
This is different from how the other method suites 
(tp_as_sequence, etc.) are handled; near the end of 
PyType_Ready the other method suites are copied from 
the base into the subtype if the subtype's slot is NULL.  

It looks like the tp_as_buffer omission is simply an 
oversight, since the flag manipulations in inherit_special 
appear to be preparing the way for copying the base 
pointer into the subtype.  So I've attached a small patch 
(against the 2.3a1 version of typeobject.c) to fix this.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=681367&group_id=5470


From noreply@sourceforge.net  Thu Feb  6 00:25:39 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Wed, 05 Feb 2003 16:25:39 -0800
Subject: [Python-bugs-list] [ python-Bugs-681367 ] C subtypes do not inherit tp_as_buffer
Message-ID: <E18gZrH-0002U5-00@sc8-sf-web2.sourceforge.net>

Bugs item #681367, was opened at 2003-02-05 15:23
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=681367&group_id=5470

Category: Python Interpreter Core
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Greg Chapman (glchapman)
Assigned to: Nobody/Anonymous (nobody)
Summary: C subtypes do not inherit tp_as_buffer

Initial Comment:
Subtypes of str (for example) implemented in C do not 
inherit str's tp_as_buffer funcs unless the subtypes provide 
a pointer to a tp_as_buffer structure in their type struct.  
This is different from how the other method suites 
(tp_as_sequence, etc.) are handled; near the end of 
PyType_Ready the other method suites are copied from 
the base into the subtype if the subtype's slot is NULL.  

It looks like the tp_as_buffer omission is simply an 
oversight, since the flag manipulations in inherit_special 
appear to be preparing the way for copying the base 
pointer into the subtype.  So I've attached a small patch 
(against the 2.3a1 version of typeobject.c) to fix this.


----------------------------------------------------------------------

>Comment By: Greg Chapman (glchapman)
Date: 2003-02-05 15:25

Message:
Logged In: YES 
user_id=86307

aargh -- the patch didn't get attached; trying again.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=681367&group_id=5470


From noreply@sourceforge.net  Thu Feb  6 01:04:42 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Wed, 05 Feb 2003 17:04:42 -0800
Subject: [Python-bugs-list] [ python-Bugs-604716 ] faster [None]*n or []*n
Message-ID: <E18gaT4-0002tN-00@sc8-sf-web4.sourceforge.net>

Bugs item #604716, was opened at 2002-09-04 20:20
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=604716&group_id=5470

Category: Python Interpreter Core
Group: Not a Bug
Status: Open
Resolution: None
Priority: 5
Submitted By: Gregory Smith (gregsmith)
Assigned to: Nobody/Anonymous (nobody)
Summary: faster [None]*n or []*n

Initial Comment:
This is a suggestion to speed up multiplying
a list by an integer, in the case where 
the length of the list is 0 or 1. currently there
is a lot of overhead in the loop for these cases.
The current loop ( in list_repeat, in listobject.c) :

	p = np->ob_item;
	for (i = 0; i < n; i++) {
		for (j = 0; j < a->ob_size; j++) {
			*p = a->ob_item[j];
			Py_INCREF(*p);
			p++;
		}
	}
	return (PyObject *) np;

Suggested  ( where Py_INCREF_N(o,n) bumps
the refcount of  'o' by n): 

	p = np->ob_item;
        if( a->ob_size <= 1 ){
		if( a->ob_size ){  // is 1
			Pyobject *a0 = a->ob_item[0];
			for (i = 0; i < n; i++) {
				*p++ = a0;
			}
		        Py_INCREF_N(a0,n);
		}
        }else{
		for (i = 0; i < n; i++) {
			for (j = 0; j < a->ob_size; j++) {
				*p = a->ob_item[j];
				Py_INCREF(*p);
				p++;
			}
		}
        }
	return (PyObject *) np;


You could also do special cases for len=2, etc
(with *p++ = a0; *p++ =  a1; inside) but len=1
is a common case, with things like [None]*1000
used to create lists.

The best approach in general is to check
if the list being replicated is sufficiently smaller than
the replication count; and if so, use a different set
of loops nested in the other order, so that the
outer loop runs less often. 

 With the inverted loop case, you have 
'a->ob_size' calls to Py_INCREF_N instead of
'a->ob_size*n' calls to Py_INCREF. 


Exact same comments apply to tuplerepeat().

If any interest, I will code this and test it. 



----------------------------------------------------------------------

Comment By: Michael Stone (mbrierst)
Date: 2003-02-06 01:04

Message:
Logged In: YES 
user_id=670441

I went ahead and coded up a patch based on the discussion below.  It implements Py_INCREF_N and Py_DECREF_N (just for symmetry, not sure if anyone needs it) and puts the new INCREF to use in list_repeat.  Also, list_repeat makes special cases of 0's.  I tested with python like that described below:

from time import clock
niter = 10
iterations = [None] * niter
start, end = 0, 7
for length in range(start, end):
  l = [None]*length
  if length == 0:  multiplier = 10000000
  else: multiplier = 1000000/length
  start = clock()
  for i in iterations:
    lmult = l*multiplier
  total = clock() - start
  print "for list length %d, time %e" % (length, total)

I tested the patch on different list sizes, 0-7, 200-207, 1000-1007, and found about an 8% speed increase in all cases (except the 0 case of course, 100% speedup).  I was surprised to find the debug build didn't differ in improvment from the patch.

I also tried implementing the inner loop as memcpy, but found the results only started to improve noticably at list sizes of 1000 or so, and there was a sizable performance hit at small sizes of more than 20%.  This might be worth including for very large lists as a separate case, but I think it's pretty rare to repeat a list that size.

Let me know what you think of it or what changes you want.
(I can't attach it in sourceforge so I'm pasting it in):

Index: Objects/listobject.c
===================================================================
RCS file: /cvsroot/python/python/dist/src/Objects/listobject.c,v
retrieving revision 2.145
diff -c -r2.145 listobject.c
*** Objects/listobject.c	2 Jan 2003 20:51:08 -0000	2.145
--- Objects/listobject.c	6 Feb 2003 00:48:46 -0000
***************
*** 415,436 ****
  list_repeat(PyListObject *a, int n)
  {
  	int i, j;
! 	int size;
  	PyListObject *np;
! 	PyObject **p;
  	if (n < 0)
  		n = 0;
! 	size = a->ob_size * n;
! 	if (n && size/n != a->ob_size)
  		return PyErr_NoMemory();
  	np = (PyListObject *) PyList_New(size);
  	if (np == NULL)
  		return NULL;
  	p = np->ob_item;
  	for (i = 0; i < n; i++) {
! 		for (j = 0; j < a->ob_size; j++) {
! 			*p = a->ob_item[j];
! 			Py_INCREF(*p);
  			p++;
  		}
  	}
--- 415,441 ----
  list_repeat(PyListObject *a, int n)
  {
  	int i, j;
! 	int size, ob_size;
  	PyListObject *np;
! 	PyObject **p, **ob_item;
  	if (n < 0)
  		n = 0;
!         ob_size = a->ob_size;
! 	if (n == 0 || ob_size == 0)
! 		return PyList_New(0);
! 	size = ob_size * n;
! 	if (size/n != ob_size)
  		return PyErr_NoMemory();
  	np = (PyListObject *) PyList_New(size);
  	if (np == NULL)
  		return NULL;
  	p = np->ob_item;
+ 	ob_item = a->ob_item;
+ 	for (i = 0; i < ob_size; i++)
+ 		Py_INCREF_N(ob_item[i], n);
  	for (i = 0; i < n; i++) {
! 		for (j = 0; j < ob_size; j++) {
! 			*p = ob_item[j];
  			p++;
  		}
  	}
Index: Include/object.h
===================================================================
RCS file: /cvsroot/python/python/dist/src/Include/object.h,v
retrieving revision 2.112
diff -c -r2.112 object.h
*** Include/object.h	17 Nov 2002 17:52:44 -0000	2.112
--- Include/object.h	6 Feb 2003 00:48:51 -0000
***************
*** 526,532 ****
--- 526,534 ----
  PyAPI_FUNC(void) _Py_NegativeRefcount(const char *fname,
  					    int lineno, PyObject *op);
  #define _Py_INC_REFTOTAL	_Py_RefTotal++
+ #define _Py_INC_REFTOTAL_N(N)	_Py_RefTotal += (N)
  #define _Py_DEC_REFTOTAL	_Py_RefTotal--
+ #define _Py_DEC_REFTOTAL_N(N)	_Py_RefTotal -= (N)
  #define _Py_REF_DEBUG_COMMA	,
  #define _Py_CHECK_REFCNT(OP)					\
  {	if ((OP)->ob_refcnt < 0)				\
***************
*** 535,541 ****
--- 537,545 ----
  }
  #else
  #define _Py_INC_REFTOTAL
+ #define _Py_INC_REFTOTAL_N(N)
  #define _Py_DEC_REFTOTAL
+ #define _Py_DEC_REFTOTAL_N(N)
  #define _Py_REF_DEBUG_COMMA
  #define _Py_CHECK_REFCNT(OP)	/* a semicolon */;
  #endif /* Py_REF_DEBUG */
***************
*** 580,591 ****
--- 584,607 ----
  	_Py_INC_REFTOTAL  _Py_REF_DEBUG_COMMA	\
  	(op)->ob_refcnt++)
  
+ #define Py_INCREF_N(op, N) (				\
+ 	_Py_INC_REFTOTAL_N(N)  _Py_REF_DEBUG_COMMA	\
+ 	(op)->ob_refcnt += (N))
+ 
  #define Py_DECREF(op)					\
  	if (_Py_DEC_REFTOTAL  _Py_REF_DEBUG_COMMA	\
  	    --(op)->ob_refcnt != 0)			\
  		_Py_CHECK_REFCNT(op)			\
  	else						\
  		_Py_Dealloc((PyObject *)(op))
+ 
+ #define Py_DECREF_N(op, N) (				\
+ 	if (_Py_DEC_REFTOTAL_N(N)  _Py_REF_DEBUG_COMMA	\
+ 	    ((op)->ob_refcnt -= (N)) != 0)		\
+ 		_Py_CHECK_REFCNT(op)			\
+ 	else						\
+ 		_Py_Dealloc((PyObject *)(op))
+ 
  
  /* Macros to use in case the object pointer may be NULL: */
  #define Py_XINCREF(op) if ((op) == NULL) ; else Py_INCREF(op)


----------------------------------------------------------------------

Comment By: Gregory Smith (gregsmith)
Date: 2002-10-12 15:12

Message:
Logged In: YES 
user_id=292741

Sure, the loop inversion is not a big deal. There are
two slightly larger issues, though:

(1) O(n) time for []*n seems worth fixing; I noticed Guido was just in there
adding a size check, and bumping into the [..]*0 case (descr. in #622091 as []*5 ),
so another little edit won't hurt right?

I suggest changing

        size = a->ob_size * n;
        if (n && size/n != a->ob_size)
                return PyErr_NoMemory();
to

        size = a->ob_size * n;
        if( size == 0 )
              return PyList_New(0);
        if (size/n != a->ob_size)
                return PyErr_NoMemory();

which is hopefully correct by inductive inspection :-)

(2) If there is any merit seen in introducing Py_INCREF_N to
object.h for future use,  this seems as good a place as any to break ground.



----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2002-10-04 20:16

Message:
Logged In: YES 
user_id=80475

It's starting to look as if any sort of optimization effort will 
take more human time to create, test, and review than it 
could ever save in computer time.

Can this be closed?



----------------------------------------------------------------------

Comment By: Gregory Smith (gregsmith)
Date: 2002-09-12 21:13

Message:
Logged In: YES 
user_id=292741

OK, i'll try that when I have time. The problem is,
you can't do [None]*1000000 over and over again for 10
seconds unless you also destroy the lists, there isn't
enough RAM - and I was trying not to count the dispose
time, which of course is also dependent on the list length.
 I was calling calling clock() immediately before and
after the operation, and accumulating the differences,
thus excluding the rest of the loop with the list delete.
any uncertainty in clock() should in principle average
out over a lot of reps.  For what it's worth, the results
were reasonably repeatable from run to run (maybe
5%) and very linear with respect to changing the 
'*1000000' factor.

I know i'm effectively timing one call to clock() as part
of the timed code; but that won't change with the list
size and can thus be factored out.


----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2002-09-11 16:43

Message:
Logged In: YES 
user_id=21627

Please don't invoke clock() twice per iteration. The
standard timing procedure is

iterations = [None] * niter
start = clock()
for i in iterations:
  actions_to_time
total = clock() - start

That way, you don't measure the time it takes to create the
range object.
Also, don't trust any measurements that have a total time of
less then 10s (running it for a minute is even better). With
this algorithm, please re-report the two totals, and the
number of iterations. Run it several times in a row, to see
how results vary.

----------------------------------------------------------------------

Comment By: Gregory Smith (gregsmith)
Date: 2002-09-06 23:10

Message:
Logged In: YES 
user_id=292741

using this test code, and changing the 1000000, I have found
that it adds an additional 40 nsec for each additional
copy of the None. This doesn't change noticeably
when I add the 'improvement'.

If I change it to []*whatever, then the old implementation
needs 5 ns extra for each non-copy (and the new one
isn't sensitive). This is telling me that most of the time
in the [None]*lots is in the PyList_New() to get the list.

So I went and made a variant of PyList_New which doesn't
clear the list, and found that [None]*1000000 takes 22msec
instead of the 40 it used to take.
This doesn't make too much sense, looking at the code;
maybe cache effects. 

Time to go home now...
-------------------------
def timeit():
    from time import clock
    t0 = clock()
    tin = 0
    niter = 50
    for i in range(niter):
        ta,p,tb =  clock(),[None]*1000000,clock()
        tin += tb-ta
        p = 0   # deallocate list
    t0 = clock()-t0
    print "avg. time = ", tin/niter
    print "avg. overhead = ", t0/niter

timeit()
---------------------


----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2002-09-06 19:16

Message:
Logged In: YES 
user_id=80475

Yes, I looked at the all-at-once memcpy approach but 
found that MVC++'s memcpy() was too smart and would 
not use the buffer overlap as intended.  This is a dead-end.

Using memcpy() in a loop as I suggested will work fine.  
Neal implemented a number of micro-optimizations using 
this and/or memchr().

You've already hit on the common case and best speed-up 
by special casing length one and by py_incref_n.  Go for it.

----------------------------------------------------------------------

Comment By: Gregory Smith (gregsmith)
Date: 2002-09-06 18:41

Message:
Logged In: YES 
user_id=292741

[ sound effects: can opener, worms... ] ;-)

Use of memcpy/memset is a whole other issue- see below

What you've got there doesn't help for the case I mention.
If you do []*100000, you get 100000 calls to memcpy with
size = 0, and if you do [None]*100000, 100000 calls to memcpy
with size = 4; this could easily wind up being even slower. The
point is to reduce the number of times the inner loop quits
and the outer loop runs; the actual code in the inner loop
will run the same number of times regardless (although 
*p = regvar is faster than *p = ptr->a[j] ); it's the loop 
overhead (instruction queue flushes, etc) that make a
difference here. Bear in mind too that 'j < a->ob_size' will
load
a->ob_size on each loop, if the compiler can't be sure
you didn't write it via *p [yes i''ve spent too much time
looking
at C compiler output. real-time DSP applications mostly].

Many compilers do intrinsic memcpy so that there's no
call, but since the length of the copy is not fixed, code
still needs to be generated to test the length, and you
wind up being no better than the original for short source
lists.

Regarding the use of memcpy/memset in general; these
could be applied in a lot of cases - making a list slice,
filling a new list with null's etc - and on some machines they
make a big difference to the speed of the operation. Compared
to the time spent later working on that list, it may not show
up much in the end. Having said that, I've done some python
apps which process, say, 20Mbytes of binary floating-point data
in about 12 seconds, by avoiding any python looping ( using
Numeric and image file read/writes). In this kind of thing,
a faster slice copy can make difference.

I'm assuming the programmer would rather not have the
marginal speedup of these routines, if it poses even a small
possibility of creating portability issues on some weird
machine,
and this is a pretty wise policy on a project of this nature.

I suspect any machine where memcpy doesn't work in this
context would also fail to work with python's polymorphism
and general memory management, but whatever.

A really good way to make a lot of copies of a small list is
to make the first copy, then do a single
  memcpy( p, p+size, (n-1)*size * sizeof(*p)   ) 
which 'rolls' the same memory over and over. No loops
(except within memcpy and the assumption is that that
one is as good as it gets).
 But this may be even less portable than the other uses of
memcpy, because it makes assumptions about
how memcpy handles overlapped transfers. E.g., machines
with a lot of registers could do a bunch of reads and
a bunch of writes to help out the cache, thus breaking
this for small input lists.

And yes, I'll report some speedup results when I have some.







----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2002-09-06 18:38

Message:
Logged In: YES 
user_id=80475

There was a typo in my code fragment.  It should have 
read:  
 memcpy(p + i*n, a->ob_item, a->ob_size * sizeof
(PyObject *));

The memcpy speeds up cases where there are many 
items in the list.  It is actually slower for a list of length 
one which should still be special cased as the OP 
suggested.  

Also, his idea for adding Py_INCREF_N is great.  It can be 
used throughout python.  Since Py_INCREF has lots of 
macro magic for updating debug statistics, the OP's idea 
may have to be implemented as a function.

I think there is interest in the OP's ideas and that he 
should go-ahead and develop a tested patch along with 
timing statistics.  


----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2002-09-06 09:53

Message:
Logged In: YES 
user_id=21627

Can you report what speed-up you get, for what example?

----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2002-09-05 01:45

Message:
Logged In: YES 
user_id=80475

Here's  a faster approach that doesn't involve a special 
case:

	for (i = 0; i < n; i++)
		memcpy(p + i*n, p, size);
	for (j = 0; j < a->ob_size; j++){
		Py_INCREF_N(*p, n);
		p++;
	}

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=604716&group_id=5470


From noreply@sourceforge.net  Thu Feb  6 03:22:15 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Wed, 05 Feb 2003 19:22:15 -0800
Subject: [Python-bugs-list] [ python-Bugs-668433 ] Abort with "negative ref count"
Message-ID: <E18gccB-0000ta-00@sc8-sf-web1.sourceforge.net>

Bugs item #668433, was opened at 2003-01-15 06:23
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=668433&group_id=5470

Category: Python Interpreter Core
Group: Python 2.3
Status: Closed
Resolution: Fixed
Priority: 5
Submitted By: Walter Dörwald (doerwalter)
Assigned to: Guido van Rossum (gvanrossum)
>Summary: Abort with "negative ref count"

Initial Comment:
Python 2.3a1 and the current CVS version abort with the
following error message when executing the attached
test script:
Fatal Python error:
D:\Python-current\dist\src\Objects\descrobject.c:10
object at 007C3208 has negative ref count -1

This happens only in the debug build, both on Linux
(with configure --enable-unicode=ucs4 --with-pydebug
and 2.3a1) and on Windows (with the debug build and
current CVS).

----------------------------------------------------------------------

>Comment By: Skip Montanaro (montanaro)
Date: 2003-02-05 21:22

Message:
Logged In: YES 
user_id=44345

I'm still seeing a negative ref count error with a fresh debug build after
cvs up...

(gdb) p *old_value
$1 = {
  _ob_next = 0xdbdbdbdb, 
  _ob_prev = 0xdbdbdbdb, 
  ob_refcnt = -606348326, 
  ob_type = 0xdbdbdbdb
}

#0  0x9001b52c in kill ()
#1  0x9005ceec in abort ()
#2  0x00012ad8 in Py_AtExit (func=0x4bfe0 <_Py_NegativeRefcount+88>) at Python/pythonrun.c:1334
#3  0x0004bfe0 in _Py_NegativeRefcount (fname=0x1266ac "Objects/dictobject.c", lineno=373, op=0x473258) at Objects/object.c:104
#4  0x0006cd90 in insertdict (mp=0x4da428, key=0x54ba28, hash=1490138787, value=0x134aac) at Objects/dictobject.c:373
#5  0x0006fddc in PyDict_Merge (a=0x4da428, b=0x4752d8, override=1) at Objects/dictobject.c:1142
#6  0x0006fb3c in PyDict_Update (a=0x4da428, b=0x4752d8) at Objects/dictobject.c:1102
#7  0x00051188 in merge_class_dict (dict=0x4da428, aclass=0x465ce0) at Objects/object.c:1641
#8  0x0005131c in merge_class_dict (dict=0x4da428, aclass=0x465d28) at Objects/object.c:1665
#9  0x00051ae4 in PyObject_Dir (arg=0x472278) at Objects/object.c:1796
#10 0x000cfca0 in builtin_dir (self=0x0, args=0x4c89a0) at Python/bltinmodule.c:396
#11 0x000c4470 in PyCFunction_Call (func=0x404df8, arg=0x4c89a0, kw=0x0) at Objects/methodobject.c:73
#12 0x000b8630 in call_function (pp_stack=0xbfffbb50, oparg=1) at Python/ceval.c:3285
#13 0x000b2e04 in eval_frame (f=0x4a1a38) at Python/ceval.c:2041
...

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-05 16:46

Message:
Logged In: YES 
user_id=6380

Fixed in CVS, typeobject.c 2.203. Much thanks to Tim, who
understands the trashgcan code much better than I do.
Explanation is in the code. I checked that none of the three
examples breaks with the new code.

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2003-02-05 16:08

Message:
Logged In: YES 
user_id=31435

Attached a very short test case.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-01-15 15:52

Message:
Logged In: YES 
user_id=6380

Bah. Trashcan code. :-(

OK, I'll look at it.

When I have time.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-01-15 15:37

Message:
Logged In: YES 
user_id=33168

8/6/2002 works without complaints, 8/8/2002 breaks, 8/7/2002
breaks with a different error message: 
Objects/typeobject.c:466: subtype_dealloc: Assertion
`g->gc.gc_refs != (-2)' failed.

All dates are American, ie, August, 6-8.  The change
corresponds to Object/typeobject.c:

8/6 - 2.165 works
8/7 - 2.166 assertion listed above
8/8 - 2.167 assertion in bug report (current)

Guido made all the changes, assigning to him. :-)

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-01-15 14:37

Message:
Logged In: YES 
user_id=33168

I'm not running w/--enable-unicode=ucs4, but I still get the
problem.  I've simplified the test, but only a little so far. 
The problem did not affect 2.2.2, only 2.3.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-01-15 09:04

Message:
Logged In: YES 
user_id=33168

Walter, there's no file attached.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=668433&group_id=5470


From noreply@sourceforge.net  Thu Feb  6 03:33:26 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Wed, 05 Feb 2003 19:33:26 -0800
Subject: [Python-bugs-list] [ python-Bugs-668433 ] Abort with "negative ref count"
Message-ID: <E18gcn0-0001C2-00@sc8-sf-web1.sourceforge.net>

Bugs item #668433, was opened at 2003-01-15 07:23
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=668433&group_id=5470

Category: Python Interpreter Core
Group: Python 2.3
Status: Closed
Resolution: Fixed
Priority: 5
Submitted By: Walter Dörwald (doerwalter)
Assigned to: Guido van Rossum (gvanrossum)
>Summary: Abort with "negative ref count"

Initial Comment:
Python 2.3a1 and the current CVS version abort with the
following error message when executing the attached
test script:
Fatal Python error:
D:\Python-current\dist\src\Objects\descrobject.c:10
object at 007C3208 has negative ref count -1

This happens only in the debug build, both on Linux
(with configure --enable-unicode=ucs4 --with-pydebug
and 2.3a1) and on Windows (with the debug build and
current CVS).

----------------------------------------------------------------------

>Comment By: Tim Peters (tim_one)
Date: 2003-02-05 22:33

Message:
Logged In: YES 
user_id=31435

Skip, which program were you running?  I don't see any 
calls to dir() in any of the programs attached to this bug 
report, so I doubt that this bug report has anything to do 
with what you're seeing.

----------------------------------------------------------------------

Comment By: Skip Montanaro (montanaro)
Date: 2003-02-05 22:22

Message:
Logged In: YES 
user_id=44345

I'm still seeing a negative ref count error with a fresh debug build after
cvs up...

(gdb) p *old_value
$1 = {
  _ob_next = 0xdbdbdbdb, 
  _ob_prev = 0xdbdbdbdb, 
  ob_refcnt = -606348326, 
  ob_type = 0xdbdbdbdb
}

#0  0x9001b52c in kill ()
#1  0x9005ceec in abort ()
#2  0x00012ad8 in Py_AtExit (func=0x4bfe0 <_Py_NegativeRefcount+88>) at Python/pythonrun.c:1334
#3  0x0004bfe0 in _Py_NegativeRefcount (fname=0x1266ac "Objects/dictobject.c", lineno=373, op=0x473258) at Objects/object.c:104
#4  0x0006cd90 in insertdict (mp=0x4da428, key=0x54ba28, hash=1490138787, value=0x134aac) at Objects/dictobject.c:373
#5  0x0006fddc in PyDict_Merge (a=0x4da428, b=0x4752d8, override=1) at Objects/dictobject.c:1142
#6  0x0006fb3c in PyDict_Update (a=0x4da428, b=0x4752d8) at Objects/dictobject.c:1102
#7  0x00051188 in merge_class_dict (dict=0x4da428, aclass=0x465ce0) at Objects/object.c:1641
#8  0x0005131c in merge_class_dict (dict=0x4da428, aclass=0x465d28) at Objects/object.c:1665
#9  0x00051ae4 in PyObject_Dir (arg=0x472278) at Objects/object.c:1796
#10 0x000cfca0 in builtin_dir (self=0x0, args=0x4c89a0) at Python/bltinmodule.c:396
#11 0x000c4470 in PyCFunction_Call (func=0x404df8, arg=0x4c89a0, kw=0x0) at Objects/methodobject.c:73
#12 0x000b8630 in call_function (pp_stack=0xbfffbb50, oparg=1) at Python/ceval.c:3285
#13 0x000b2e04 in eval_frame (f=0x4a1a38) at Python/ceval.c:2041
...

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-05 17:46

Message:
Logged In: YES 
user_id=6380

Fixed in CVS, typeobject.c 2.203. Much thanks to Tim, who
understands the trashgcan code much better than I do.
Explanation is in the code. I checked that none of the three
examples breaks with the new code.

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2003-02-05 17:08

Message:
Logged In: YES 
user_id=31435

Attached a very short test case.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-01-15 16:52

Message:
Logged In: YES 
user_id=6380

Bah. Trashcan code. :-(

OK, I'll look at it.

When I have time.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-01-15 16:37

Message:
Logged In: YES 
user_id=33168

8/6/2002 works without complaints, 8/8/2002 breaks, 8/7/2002
breaks with a different error message: 
Objects/typeobject.c:466: subtype_dealloc: Assertion
`g->gc.gc_refs != (-2)' failed.

All dates are American, ie, August, 6-8.  The change
corresponds to Object/typeobject.c:

8/6 - 2.165 works
8/7 - 2.166 assertion listed above
8/8 - 2.167 assertion in bug report (current)

Guido made all the changes, assigning to him. :-)

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-01-15 15:37

Message:
Logged In: YES 
user_id=33168

I'm not running w/--enable-unicode=ucs4, but I still get the
problem.  I've simplified the test, but only a little so far. 
The problem did not affect 2.2.2, only 2.3.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-01-15 10:04

Message:
Logged In: YES 
user_id=33168

Walter, there's no file attached.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=668433&group_id=5470


From noreply@sourceforge.net  Thu Feb  6 03:44:10 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Wed, 05 Feb 2003 19:44:10 -0800
Subject: [Python-bugs-list] [ python-Bugs-668433 ] Abort with "negative ref count"
Message-ID: <E18gcxO-0001WB-00@sc8-sf-web1.sourceforge.net>

Bugs item #668433, was opened at 2003-01-15 06:23
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=668433&group_id=5470

Category: Python Interpreter Core
Group: Python 2.3
Status: Closed
Resolution: Fixed
Priority: 5
Submitted By: Walter Dörwald (doerwalter)
Assigned to: Guido van Rossum (gvanrossum)
>Summary: Abort with "negative ref count"

Initial Comment:
Python 2.3a1 and the current CVS version abort with the
following error message when executing the attached
test script:
Fatal Python error:
D:\Python-current\dist\src\Objects\descrobject.c:10
object at 007C3208 has negative ref count -1

This happens only in the debug build, both on Linux
(with configure --enable-unicode=ucs4 --with-pydebug
and 2.3a1) and on Windows (with the debug build and
current CVS).

----------------------------------------------------------------------

>Comment By: Skip Montanaro (montanaro)
Date: 2003-02-05 21:44

Message:
Logged In: YES 
user_id=44345

I was running a unittest script for the nascent csv module.  After posting,
I noticed that there was a bug in the test code (at the Python level), but 
the C code underneath seems fairly solid. (Based on Object Craft's csv
module which is pretty stable.)

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2003-02-05 21:33

Message:
Logged In: YES 
user_id=31435

Skip, which program were you running?  I don't see any 
calls to dir() in any of the programs attached to this bug 
report, so I doubt that this bug report has anything to do 
with what you're seeing.

----------------------------------------------------------------------

Comment By: Skip Montanaro (montanaro)
Date: 2003-02-05 21:22

Message:
Logged In: YES 
user_id=44345

I'm still seeing a negative ref count error with a fresh debug build after
cvs up...

(gdb) p *old_value
$1 = {
  _ob_next = 0xdbdbdbdb, 
  _ob_prev = 0xdbdbdbdb, 
  ob_refcnt = -606348326, 
  ob_type = 0xdbdbdbdb
}

#0  0x9001b52c in kill ()
#1  0x9005ceec in abort ()
#2  0x00012ad8 in Py_AtExit (func=0x4bfe0 <_Py_NegativeRefcount+88>) at Python/pythonrun.c:1334
#3  0x0004bfe0 in _Py_NegativeRefcount (fname=0x1266ac "Objects/dictobject.c", lineno=373, op=0x473258) at Objects/object.c:104
#4  0x0006cd90 in insertdict (mp=0x4da428, key=0x54ba28, hash=1490138787, value=0x134aac) at Objects/dictobject.c:373
#5  0x0006fddc in PyDict_Merge (a=0x4da428, b=0x4752d8, override=1) at Objects/dictobject.c:1142
#6  0x0006fb3c in PyDict_Update (a=0x4da428, b=0x4752d8) at Objects/dictobject.c:1102
#7  0x00051188 in merge_class_dict (dict=0x4da428, aclass=0x465ce0) at Objects/object.c:1641
#8  0x0005131c in merge_class_dict (dict=0x4da428, aclass=0x465d28) at Objects/object.c:1665
#9  0x00051ae4 in PyObject_Dir (arg=0x472278) at Objects/object.c:1796
#10 0x000cfca0 in builtin_dir (self=0x0, args=0x4c89a0) at Python/bltinmodule.c:396
#11 0x000c4470 in PyCFunction_Call (func=0x404df8, arg=0x4c89a0, kw=0x0) at Objects/methodobject.c:73
#12 0x000b8630 in call_function (pp_stack=0xbfffbb50, oparg=1) at Python/ceval.c:3285
#13 0x000b2e04 in eval_frame (f=0x4a1a38) at Python/ceval.c:2041
...

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-05 16:46

Message:
Logged In: YES 
user_id=6380

Fixed in CVS, typeobject.c 2.203. Much thanks to Tim, who
understands the trashgcan code much better than I do.
Explanation is in the code. I checked that none of the three
examples breaks with the new code.

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2003-02-05 16:08

Message:
Logged In: YES 
user_id=31435

Attached a very short test case.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-01-15 15:52

Message:
Logged In: YES 
user_id=6380

Bah. Trashcan code. :-(

OK, I'll look at it.

When I have time.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-01-15 15:37

Message:
Logged In: YES 
user_id=33168

8/6/2002 works without complaints, 8/8/2002 breaks, 8/7/2002
breaks with a different error message: 
Objects/typeobject.c:466: subtype_dealloc: Assertion
`g->gc.gc_refs != (-2)' failed.

All dates are American, ie, August, 6-8.  The change
corresponds to Object/typeobject.c:

8/6 - 2.165 works
8/7 - 2.166 assertion listed above
8/8 - 2.167 assertion in bug report (current)

Guido made all the changes, assigning to him. :-)

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-01-15 14:37

Message:
Logged In: YES 
user_id=33168

I'm not running w/--enable-unicode=ucs4, but I still get the
problem.  I've simplified the test, but only a little so far. 
The problem did not affect 2.2.2, only 2.3.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-01-15 09:04

Message:
Logged In: YES 
user_id=33168

Walter, there's no file attached.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=668433&group_id=5470


From noreply@sourceforge.net  Thu Feb  6 03:56:44 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Wed, 05 Feb 2003 19:56:44 -0800
Subject: [Python-bugs-list] [ python-Bugs-668433 ] Abort with "negative ref count"
Message-ID: <E18gd9Y-0008Ay-00@sc8-sf-web4.sourceforge.net>

Bugs item #668433, was opened at 2003-01-15 07:23
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=668433&group_id=5470

Category: Python Interpreter Core
Group: Python 2.3
Status: Closed
Resolution: Fixed
Priority: 5
Submitted By: Walter Dörwald (doerwalter)
Assigned to: Guido van Rossum (gvanrossum)
>Summary: Abort with "negative ref count"

Initial Comment:
Python 2.3a1 and the current CVS version abort with the
following error message when executing the attached
test script:
Fatal Python error:
D:\Python-current\dist\src\Objects\descrobject.c:10
object at 007C3208 has negative ref count -1

This happens only in the debug build, both on Linux
(with configure --enable-unicode=ucs4 --with-pydebug
and 2.3a1) and on Windows (with the debug build and
current CVS).

----------------------------------------------------------------------

>Comment By: Tim Peters (tim_one)
Date: 2003-02-05 22:56

Message:
Logged In: YES 
user_id=31435

Please open a new bug report then.  The cause of what 
you're seeing almost certainly has nothing to do with this 
bug report (for starters, you're nowhere near 
subtype_dealloc()), so mixing it in here is just going to 
cause ongoing confusion (as it already has).

----------------------------------------------------------------------

Comment By: Skip Montanaro (montanaro)
Date: 2003-02-05 22:44

Message:
Logged In: YES 
user_id=44345

I was running a unittest script for the nascent csv module.  After posting,
I noticed that there was a bug in the test code (at the Python level), but 
the C code underneath seems fairly solid. (Based on Object Craft's csv
module which is pretty stable.)

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2003-02-05 22:33

Message:
Logged In: YES 
user_id=31435

Skip, which program were you running?  I don't see any 
calls to dir() in any of the programs attached to this bug 
report, so I doubt that this bug report has anything to do 
with what you're seeing.

----------------------------------------------------------------------

Comment By: Skip Montanaro (montanaro)
Date: 2003-02-05 22:22

Message:
Logged In: YES 
user_id=44345

I'm still seeing a negative ref count error with a fresh debug build after
cvs up...

(gdb) p *old_value
$1 = {
  _ob_next = 0xdbdbdbdb, 
  _ob_prev = 0xdbdbdbdb, 
  ob_refcnt = -606348326, 
  ob_type = 0xdbdbdbdb
}

#0  0x9001b52c in kill ()
#1  0x9005ceec in abort ()
#2  0x00012ad8 in Py_AtExit (func=0x4bfe0 <_Py_NegativeRefcount+88>) at Python/pythonrun.c:1334
#3  0x0004bfe0 in _Py_NegativeRefcount (fname=0x1266ac "Objects/dictobject.c", lineno=373, op=0x473258) at Objects/object.c:104
#4  0x0006cd90 in insertdict (mp=0x4da428, key=0x54ba28, hash=1490138787, value=0x134aac) at Objects/dictobject.c:373
#5  0x0006fddc in PyDict_Merge (a=0x4da428, b=0x4752d8, override=1) at Objects/dictobject.c:1142
#6  0x0006fb3c in PyDict_Update (a=0x4da428, b=0x4752d8) at Objects/dictobject.c:1102
#7  0x00051188 in merge_class_dict (dict=0x4da428, aclass=0x465ce0) at Objects/object.c:1641
#8  0x0005131c in merge_class_dict (dict=0x4da428, aclass=0x465d28) at Objects/object.c:1665
#9  0x00051ae4 in PyObject_Dir (arg=0x472278) at Objects/object.c:1796
#10 0x000cfca0 in builtin_dir (self=0x0, args=0x4c89a0) at Python/bltinmodule.c:396
#11 0x000c4470 in PyCFunction_Call (func=0x404df8, arg=0x4c89a0, kw=0x0) at Objects/methodobject.c:73
#12 0x000b8630 in call_function (pp_stack=0xbfffbb50, oparg=1) at Python/ceval.c:3285
#13 0x000b2e04 in eval_frame (f=0x4a1a38) at Python/ceval.c:2041
...

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-05 17:46

Message:
Logged In: YES 
user_id=6380

Fixed in CVS, typeobject.c 2.203. Much thanks to Tim, who
understands the trashgcan code much better than I do.
Explanation is in the code. I checked that none of the three
examples breaks with the new code.

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2003-02-05 17:08

Message:
Logged In: YES 
user_id=31435

Attached a very short test case.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-01-15 16:52

Message:
Logged In: YES 
user_id=6380

Bah. Trashcan code. :-(

OK, I'll look at it.

When I have time.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-01-15 16:37

Message:
Logged In: YES 
user_id=33168

8/6/2002 works without complaints, 8/8/2002 breaks, 8/7/2002
breaks with a different error message: 
Objects/typeobject.c:466: subtype_dealloc: Assertion
`g->gc.gc_refs != (-2)' failed.

All dates are American, ie, August, 6-8.  The change
corresponds to Object/typeobject.c:

8/6 - 2.165 works
8/7 - 2.166 assertion listed above
8/8 - 2.167 assertion in bug report (current)

Guido made all the changes, assigning to him. :-)

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-01-15 15:37

Message:
Logged In: YES 
user_id=33168

I'm not running w/--enable-unicode=ucs4, but I still get the
problem.  I've simplified the test, but only a little so far. 
The problem did not affect 2.2.2, only 2.3.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-01-15 10:04

Message:
Logged In: YES 
user_id=33168

Walter, there's no file attached.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=668433&group_id=5470


From noreply@sourceforge.net  Thu Feb  6 04:43:35 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Wed, 05 Feb 2003 20:43:35 -0800
Subject: [Python-bugs-list] [ python-Bugs-668433 ] Abort with "negative ref count"
Message-ID: <E18gdst-00027P-00@sc8-sf-web2.sourceforge.net>

Bugs item #668433, was opened at 2003-01-15 06:23
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=668433&group_id=5470

Category: Python Interpreter Core
Group: Python 2.3
Status: Closed
Resolution: Fixed
Priority: 5
Submitted By: Walter Dörwald (doerwalter)
Assigned to: Guido van Rossum (gvanrossum)
>Summary: Abort with "negative ref count"

Initial Comment:
Python 2.3a1 and the current CVS version abort with the
following error message when executing the attached
test script:
Fatal Python error:
D:\Python-current\dist\src\Objects\descrobject.c:10
object at 007C3208 has negative ref count -1

This happens only in the debug build, both on Linux
(with configure --enable-unicode=ucs4 --with-pydebug
and 2.3a1) and on Windows (with the debug build and
current CVS).

----------------------------------------------------------------------

>Comment By: Skip Montanaro (montanaro)
Date: 2003-02-05 22:43

Message:
Logged In: YES 
user_id=44345

my apologies.  too quick with the assumptions.

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2003-02-05 21:56

Message:
Logged In: YES 
user_id=31435

Please open a new bug report then.  The cause of what 
you're seeing almost certainly has nothing to do with this 
bug report (for starters, you're nowhere near 
subtype_dealloc()), so mixing it in here is just going to 
cause ongoing confusion (as it already has).

----------------------------------------------------------------------

Comment By: Skip Montanaro (montanaro)
Date: 2003-02-05 21:44

Message:
Logged In: YES 
user_id=44345

I was running a unittest script for the nascent csv module.  After posting,
I noticed that there was a bug in the test code (at the Python level), but 
the C code underneath seems fairly solid. (Based on Object Craft's csv
module which is pretty stable.)

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2003-02-05 21:33

Message:
Logged In: YES 
user_id=31435

Skip, which program were you running?  I don't see any 
calls to dir() in any of the programs attached to this bug 
report, so I doubt that this bug report has anything to do 
with what you're seeing.

----------------------------------------------------------------------

Comment By: Skip Montanaro (montanaro)
Date: 2003-02-05 21:22

Message:
Logged In: YES 
user_id=44345

I'm still seeing a negative ref count error with a fresh debug build after
cvs up...

(gdb) p *old_value
$1 = {
  _ob_next = 0xdbdbdbdb, 
  _ob_prev = 0xdbdbdbdb, 
  ob_refcnt = -606348326, 
  ob_type = 0xdbdbdbdb
}

#0  0x9001b52c in kill ()
#1  0x9005ceec in abort ()
#2  0x00012ad8 in Py_AtExit (func=0x4bfe0 <_Py_NegativeRefcount+88>) at Python/pythonrun.c:1334
#3  0x0004bfe0 in _Py_NegativeRefcount (fname=0x1266ac "Objects/dictobject.c", lineno=373, op=0x473258) at Objects/object.c:104
#4  0x0006cd90 in insertdict (mp=0x4da428, key=0x54ba28, hash=1490138787, value=0x134aac) at Objects/dictobject.c:373
#5  0x0006fddc in PyDict_Merge (a=0x4da428, b=0x4752d8, override=1) at Objects/dictobject.c:1142
#6  0x0006fb3c in PyDict_Update (a=0x4da428, b=0x4752d8) at Objects/dictobject.c:1102
#7  0x00051188 in merge_class_dict (dict=0x4da428, aclass=0x465ce0) at Objects/object.c:1641
#8  0x0005131c in merge_class_dict (dict=0x4da428, aclass=0x465d28) at Objects/object.c:1665
#9  0x00051ae4 in PyObject_Dir (arg=0x472278) at Objects/object.c:1796
#10 0x000cfca0 in builtin_dir (self=0x0, args=0x4c89a0) at Python/bltinmodule.c:396
#11 0x000c4470 in PyCFunction_Call (func=0x404df8, arg=0x4c89a0, kw=0x0) at Objects/methodobject.c:73
#12 0x000b8630 in call_function (pp_stack=0xbfffbb50, oparg=1) at Python/ceval.c:3285
#13 0x000b2e04 in eval_frame (f=0x4a1a38) at Python/ceval.c:2041
...

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-05 16:46

Message:
Logged In: YES 
user_id=6380

Fixed in CVS, typeobject.c 2.203. Much thanks to Tim, who
understands the trashgcan code much better than I do.
Explanation is in the code. I checked that none of the three
examples breaks with the new code.

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2003-02-05 16:08

Message:
Logged In: YES 
user_id=31435

Attached a very short test case.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-01-15 15:52

Message:
Logged In: YES 
user_id=6380

Bah. Trashcan code. :-(

OK, I'll look at it.

When I have time.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-01-15 15:37

Message:
Logged In: YES 
user_id=33168

8/6/2002 works without complaints, 8/8/2002 breaks, 8/7/2002
breaks with a different error message: 
Objects/typeobject.c:466: subtype_dealloc: Assertion
`g->gc.gc_refs != (-2)' failed.

All dates are American, ie, August, 6-8.  The change
corresponds to Object/typeobject.c:

8/6 - 2.165 works
8/7 - 2.166 assertion listed above
8/8 - 2.167 assertion in bug report (current)

Guido made all the changes, assigning to him. :-)

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-01-15 14:37

Message:
Logged In: YES 
user_id=33168

I'm not running w/--enable-unicode=ucs4, but I still get the
problem.  I've simplified the test, but only a little so far. 
The problem did not affect 2.2.2, only 2.3.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-01-15 09:04

Message:
Logged In: YES 
user_id=33168

Walter, there's no file attached.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=668433&group_id=5470


From noreply@sourceforge.net  Thu Feb  6 04:53:46 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Wed, 05 Feb 2003 20:53:46 -0800
Subject: [Python-bugs-list] [ python-Bugs-681461 ] warning filter doc misleading
Message-ID: <E18ge2k-0003c3-00@sc8-sf-web1.sourceforge.net>

Bugs item #681461, was opened at 2003-02-06 15:53
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=681461&group_id=5470

Category: Documentation
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Richard Jones (richard)
Assigned to: Nobody/Anonymous (nobody)
Summary: warning filter doc misleading

Initial Comment:
The doc for warnings filtering states that "message is a compiled 
regular expression" when it actually has to be a string. It is compiled 
by filterwarnings. Same goes for "module". 
 

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=681461&group_id=5470


From noreply@sourceforge.net  Thu Feb  6 08:14:20 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 06 Feb 2003 00:14:20 -0800
Subject: [Python-bugs-list] [ python-Feature Requests-681533 ] Additional string stuff.
Message-ID: <E18ghAq-0007nH-00@sc8-sf-web2.sourceforge.net>

Feature Requests item #681533, was opened at 2003-02-06 03:14
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=681533&group_id=5470

Category: Python Library
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Jeremy Fincher (jemfinch)
Assigned to: Nobody/Anonymous (nobody)
Summary: Additional string stuff.

Initial Comment:
In a lot of my programs that mess with strings, I end up 
somewhere making a variable "ascii" via 
(string.maketrans('', '')).  It's just a 256-character string 
comprising the ascii character set. 
 
I use it, oftentimes, simply to be able to turn the 'translate' 
method on strings into a 'delete' method -- if I want to, 
say, remove all the spaces in aString, I'd do 
(aString.translate(ascii, string.whitespace)). 
 
Certainly an ascii variable in the string module couldn't 
hurt, would fit in with ascii_letters, etc. and would at least 
standarize the name of the full ascii set sure to be 
present in many Python programs. 
 
A little further out there, but I think just as useful, would 
be a delete method on strings.  So "foo bar 
baz".delete(string.whitespace) would return "foobarbaz".  
It would be equivalent to "foo bar baz".translate(ascii, 
string.whitespace), or the wildly inefficient: 
 
def delete(s, deleteChars): 
    l = [] 
    for c in s: 
        if c not in deleteChars: 
            l.append(c) 
    return ''.join(l) 
 
Anyway, that's all I can think of.  Do with it what you will. 
 
Jeremy 

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=681533&group_id=5470


From noreply@sourceforge.net  Thu Feb  6 09:19:25 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 06 Feb 2003 01:19:25 -0800
Subject: [Python-bugs-list] [ python-Feature Requests-681533 ] Additional string stuff.
Message-ID: <E18giBp-0003cR-00@sc8-sf-web1.sourceforge.net>

Feature Requests item #681533, was opened at 2003-02-06 03:14
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=681533&group_id=5470

Category: Python Library
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Jeremy Fincher (jemfinch)
Assigned to: Nobody/Anonymous (nobody)
Summary: Additional string stuff.

Initial Comment:
In a lot of my programs that mess with strings, I end up 
somewhere making a variable "ascii" via 
(string.maketrans('', '')).  It's just a 256-character string 
comprising the ascii character set. 
 
I use it, oftentimes, simply to be able to turn the 'translate' 
method on strings into a 'delete' method -- if I want to, 
say, remove all the spaces in aString, I'd do 
(aString.translate(ascii, string.whitespace)). 
 
Certainly an ascii variable in the string module couldn't 
hurt, would fit in with ascii_letters, etc. and would at least 
standarize the name of the full ascii set sure to be 
present in many Python programs. 
 
A little further out there, but I think just as useful, would 
be a delete method on strings.  So "foo bar 
baz".delete(string.whitespace) would return "foobarbaz".  
It would be equivalent to "foo bar baz".translate(ascii, 
string.whitespace), or the wildly inefficient: 
 
def delete(s, deleteChars): 
    l = [] 
    for c in s: 
        if c not in deleteChars: 
            l.append(c) 
    return ''.join(l) 
 
Anyway, that's all I can think of.  Do with it what you will. 
 
Jeremy 

----------------------------------------------------------------------

>Comment By: Jeremy Fincher (jemfinch)
Date: 2003-02-06 04:19

Message:
Logged In: YES 
user_id=99508

Let me also make one other suggestion:  the split method on a 
string object has, by default, behavior that can't be replicated 
by passing an argument as a separator.  That is, the default 
separated acts like re.split(r'\s+'), but it's impossible to pass 
any value into the method to achieve that same result. 
 
The problem arises when a user wants to use the maxsplit() 
parameter to the method.  Because maxsplit is a positional 
parameter instead of a keyword parameter, the user *must* 
declare a separate to split on, and thus loses his ability to split 
on whitespace-in-general.  If maxsplit was changed from being 
a positional parameter to being a keyword parameter, then a 
programmer wouldn't have to give up the default behavior of 
the split method in order to pass it a maxsplit. 
 
At present, negative maxsplit values don't differ in any way 
from split's default behavior (with no maxsplit parameter given).  
Thus, the keyword maxsplit could default to -1 with no 
breakage of code.  I can't see any place where changing 
maxsplit to a keyword parameter would break any existing 
code 
 
Jeremy 

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=681533&group_id=5470


From noreply@sourceforge.net  Thu Feb  6 11:22:13 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 06 Feb 2003 03:22:13 -0800
Subject: [Python-bugs-list] [ python-Bugs-680429 ] __module__ broken for extension classes
Message-ID: <E18gk6f-00066L-00@sc8-sf-web4.sourceforge.net>

Bugs item #680429, was opened at 2003-02-04 19:55
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=680429&group_id=5470

Category: Python Interpreter Core
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Ralf W. Grosse-Kunstleve (rwgk)
Assigned to: Michael Hudson (mwh)
Summary: __module__ broken for extension classes

Initial Comment:
We are having problems using Boost.Python with Python 
2.3a1 because
under some circumstances the result of __module__ is 
different compared
to earlier Python versions:

Python 2.2.1 (#2, Jun 17 2002, 12:06:51) 
[GCC 2.96 20000731 (Red Hat Linux 7.3 2.96-110)] on 
linux2
Type "help", "copyright", "credits" or "license" for more 
information.
>>> import simple
>>> simple.empty.__module__
'simple'
>>> 

Python 2.3a1 (#1, Jan  6 2003, 14:17:56) 
[GCC 2.96 20000731 (Red Hat Linux 7.3 2.96-110)] on 
linux2
Type "help", "copyright", "credits" or "license" for more 
information.
>>> import simple
>>> simple.empty.__module__
'__main__'
>>> 

Because of this we can no longer pickle our extension 
classes.
For your reference the code for the simple module is 
attached.
This is using Boost release 1.29.0 (www.boost.org).

We have done some debugging. Boost.Python's internal 
idea of the module
associated with an extension class is still correct even when 
using
Python 2.3a1. David Abrahams (main Boost.Python author) 
is telling me
that he "changed Boost.Python to work the way Guido 
suggested before
2.2.2." Therefore we suspect that the __module__ problem 
is due to
a change/bug in Python 2.3a1.

Ralf


#include <boost/python/module.hpp>
#include <boost/python/class.hpp>

namespace sandbx { namespace {

  struct empty {};

  void init_module()
  {
    using namespace boost::python;
    class_<empty>("empty");
  }

}} // namespace sandbx::<anonymous>

BOOST_PYTHON_MODULE(simple)
{
  sandbx::init_module();
}


----------------------------------------------------------------------

>Comment By: Michael Hudson (mwh)
Date: 2003-02-06 11:22

Message:
Logged In: YES 
user_id=6656

OK, thanks for the info.  More questions, I'm afraid:

In 2.2.x, how are you telling Python that __module__ should
be "simple"?  By setting tp_name to "simple.empty"?  What's
simple.__name__?

I think I know how __module__ is getting there, it's the
chunk of code currently at typeobject.c:1750.

Can you point me to the bit of the boost source that creates
the typeobject?  'cept sf's just fallen off the net.

----------------------------------------------------------------------

Comment By: Ralf W. Grosse-Kunstleve (rwgk)
Date: 2003-02-05 18:25

Message:
Logged In: YES 
user_id=71407

> Look at typeobject.c:type_module() (about 100 lines in).
>
> Is simple.empty a HEAPTYPE?

Yes, it is a HEAPTYPE. I've established this by adding print 
statements
in typeobject.c:type_module():

static PyObject *
type_module(PyTypeObject *type, void *context)
{
        PyObject *mod;
        char *s;

        if (type->tp_flags & Py_TPFLAGS_HEAPTYPE) {
printf("IS HEAPTYPE\n");
                mod = PyDict_GetItemString(type-
>tp_dict, "__module__");
                Py_XINCREF(mod);
                s = PyString_AsString(mod);
printf("type->tp_dict, __module__ = %s\n", s);
                return mod;
        }

Result:

Python 2.3a1 (#2, Feb  5 2003, 09:39:30) 
[GCC 2.96 20000731 (Red Hat Linux 7.3 2.96-110)] on linux2
Type "help", "copyright", "credits" or "license" for more 
information.
>>> import simple
>>> simple.empty.__module__
IS HEAPTYPE
type->tp_dict, __module__ = __main__
'__main__'
>>> 

> If you can tell me what you want that function to do, I can
> have a go at it.

We expect "simple" as the result of simple.empty.__module__.

Current result if simple.so is moved to a package:

Python 2.3a1 (#2, Feb  5 2003, 09:39:30) 
[GCC 2.96 20000731 (Red Hat Linux 7.3 2.96-110)] on linux2
Type "help", "copyright", "credits" or "license" for more 
information.
>>> from sandbx import simple
>>> simple.empty.__module__
IS HEAPTYPE
type->tp_dict, __module__ = __main__
'__main__'
>>> 

Here we expect "sandbx.simple" .
Python 2.2.x produces the expected results using the exact same
Boost.Python source code.

David Abrahams provides this additional information:

  Remember that simple.empty is created by calling the metatype, 
so
  it's created on the heap, but it's Python's internal type creation
  mechanisms which do it -- we're not setting the flags manually.

Thanks!
        Ralf


----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2003-02-05 14:10

Message:
Logged In: YES 
user_id=6656

There has certainly been a change here.

Look at typeobject.c:type_module() (about 100 lines in).

If you can tell me what you want that function to do, I can
have a go at it.

Is simple.empty a HEAPTYPE?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=680429&group_id=5470


From noreply@sourceforge.net  Thu Feb  6 11:48:55 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 06 Feb 2003 03:48:55 -0800
Subject: [Python-bugs-list] [ python-Bugs-681367 ] C subtypes do not inherit tp_as_buffer
Message-ID: <E18gkWV-0007Eo-00@sc8-sf-web2.sourceforge.net>

Bugs item #681367, was opened at 2003-02-06 00:23
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=681367&group_id=5470

Category: Python Interpreter Core
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Greg Chapman (glchapman)
>Assigned to: Guido van Rossum (gvanrossum)
Summary: C subtypes do not inherit tp_as_buffer

Initial Comment:
Subtypes of str (for example) implemented in C do not 
inherit str's tp_as_buffer funcs unless the subtypes provide 
a pointer to a tp_as_buffer structure in their type struct.  
This is different from how the other method suites 
(tp_as_sequence, etc.) are handled; near the end of 
PyType_Ready the other method suites are copied from 
the base into the subtype if the subtype's slot is NULL.  

It looks like the tp_as_buffer omission is simply an 
oversight, since the flag manipulations in inherit_special 
appear to be preparing the way for copying the base 
pointer into the subtype.  So I've attached a small patch 
(against the 2.3a1 version of typeobject.c) to fix this.


----------------------------------------------------------------------

Comment By: Greg Chapman (glchapman)
Date: 2003-02-06 00:25

Message:
Logged In: YES 
user_id=86307

aargh -- the patch didn't get attached; trying again.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=681367&group_id=5470


From noreply@sourceforge.net  Thu Feb  6 14:29:06 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 06 Feb 2003 06:29:06 -0800
Subject: [Python-bugs-list] [ python-Bugs-471893 ] Security review of pickle/marshal docs
Message-ID: <E18gn1W-0006Tv-00@sc8-sf-web2.sourceforge.net>

Bugs item #471893, was opened at 2001-10-16 18:42
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=471893&group_id=5470

Category: Extension Modules
Group: None
Status: Open
Resolution: Fixed
Priority: 7
Submitted By: Tim Peters (tim_one)
Assigned to: A.M. Kuchling (akuchling)
Summary: Security review of pickle/marshal docs

Initial Comment:
Paul Rubin points out that the security implications 
of using marshal and/or pickle aren't clear from the 
docs.  Assigning to Jeremy as he's more sensitive to 
such issues than I am; maybe Barry would like to get 
paranoid too <wink>.

A specific example:  the pickle docs say that pickle 
doesn't support code objects, and "at least this 
avoids the possibility of smuggling Trojan horses into 
a program".  However,

1) The marshal docs don't mention this vulnerability 
at all.

while

2) The pickle docs don't spell out possible dangers 
due to things pickle does that marshal doesn't (like 
importing modules, and running class constructors).

----------------------------------------------------------------------

>Comment By: A.M. Kuchling (akuchling)
Date: 2003-02-06 09:29

Message:
Logged In: YES 
user_id=11375

The Cookie classes that use pickle have DeprecationWarnings in 
2.3, and should disappear in 2.4.  If {find,load}_global are 
still going to be supported, though, then they should still be documented (though, given that you shouldn't be unpickling untrusted data, why would you ever bother overriding find_global?)


----------------------------------------------------------------------

Comment By: Barry A. Warsaw (bwarsaw)
Date: 2003-02-05 18:03

Message:
Logged In: YES 
user_id=12800

I'll just mention that anybody using anything other than
Cookie.SimpleCookie is insane, and even using the Cookie
module at all to parse real-world cookies is mildly nuts
because it's way too strict.  I prefer m&m's in my cookies.

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2003-02-05 17:57

Message:
Logged In: YES 
user_id=31435

Andrew, didn't you go overboard in deleting text here?  The 
__safe_for_unpickling__ check is gone in 2.3, but most of the 
rest of the text still applies.  In particular, Cookie is still a 
problem for people inclined to worry about this, and overriding 
methods like find_global and load_global is still valuable stuff 
(the unpickler still never imports anything that doesn't come 
from an opcode triggering one of these methods).  

----------------------------------------------------------------------

Comment By: Barry A. Warsaw (bwarsaw)
Date: 2003-02-05 16:29

Message:
Logged In: YES 
user_id=12800

Karmically (no, not comically) reassigning to Tim who
started this whole thing, and who happens to be dumping his
pickles these days.

----------------------------------------------------------------------

Comment By: A.M. Kuchling (akuchling)
Date: 2003-02-05 15:40

Message:
Logged In: YES 
user_id=11375

Re-opening.  With the changes to pickling in Python 2.3, 
the security material should be replaced with a warning to 
not unpickle untrusted data.  Patch attached.



----------------------------------------------------------------------

Comment By: Barry A. Warsaw (bwarsaw)
Date: 2001-11-18 11:08

Message:
Logged In: YES 
user_id=12800

You're right of course.  I meant to fix that and forgot. 
Will do so asap.  Glad you like the rest of it, though! :)

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2001-11-18 01:56

Message:
Logged In: NO 

Barry - the new docs just went up and they're a big
improvement over the old.  However the sentence 
"The issue here is usually not that a class's constructor
will get called -- it won't by the unpickler -- but that the
class's destructor (i.e. its __del__() method) might get
called when the object is garbage collected." isn't
correct.  

There's a flag in the pickle stream that tells
the unpickler that the pickler found a __getinitargs__
method at pickling time.  If the flag is set in the pickle,
then the unpickler calls the class constructor, whether
there's a __getinitargs__ method in the class or not.

The sample exploit that I posted earlier on,
<http://www.nightsong.com/phr/python/pickletest.py>,
is an example of an artificially concocted pickle that
makes a class constructor get called.


----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2001-11-17 17:07

Message:
Logged In: NO 

Are the new docs downloadable from somewhere?  thanx  --Paul


----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2001-11-15 19:43

Message:
Logged In: NO 

Barry, can you also do something about the Cookie module,
or at least about its documentation?  If Cookie is aliased
to SmartCookie, I think that needs mention.
 --Paul

----------------------------------------------------------------------

Comment By: Barry A. Warsaw (bwarsaw)
Date: 2001-11-15 19:12

Message:
Logged In: YES 
user_id=12800

I have rewritten the pickle documentation, and updated the
marshal, cPickle, and copy_reg documentation.  I believe all
the documentation issues raised here have been fixed.  

Implementation issues will be pushed to Python 2.3, and the
plan is to write a PEP to plan future work.

----------------------------------------------------------------------

Comment By: Barry A. Warsaw (bwarsaw)
Date: 2001-11-13 16:39

Message:
Logged In: YES 
user_id=12800

Assigning to Jeremy so he'll remember to forward me Jim's
email.  Jeremy can ping-pong it to Fred if he wants, and
then reassign to me after forwarding the email.

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2001-11-13 01:18

Message:
Logged In: NO 

The find_global variable sounds encouraging and if it
fixes this problem, that's great.  I'd still feel better
if the eval call goes away.  Is removing it allowed
under the 2.2 feature freeze?  It doesn't affect anything
visible, so no tests would have to change, etc.

Paul


----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2001-11-12 19:38

Message:
Logged In: NO 

It's possible that the eval is safe, but validating that
takes a line by line code review of all the paths that
evaling a string can go through.  It's also brittle in
that maybe someone will change the evaluator (say to
support Perl-like interpolation) in a future Python version
and not remember to change the unpickler.  Something like
that has already happened with the Cookie module.  My
guess is that it happened with the pickle module--the
whole point of that quoted string check is that the 
original pickle implementer didn't trust the input. 
The stuff about unpickling class instances was added
later (maybe by another person) without remembering the
security issue. 

Using eval this way is like storing a vat of cyanide in
your child's bedroom.  Sure, maybe if you check the seals
and locks on the vat carefully enough, you can convince
yourself that your child won't be able to get to the
cyanide.  But wouldn't you feel safer just not having the
vat there at all?  That's basic safety-consciousness.
Security consciousness works the same way.  Try to keep
dangerous ingredients and attackers as far away from each
other as possible.

Paul



----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2001-11-12 13:52

Message:
Logged In: YES 
user_id=31435

Why are people (Paul, Jeremy) concerned about eval'ing 
strings?  cPickle and pickle both check that they're 
properly quoted, and this isn't sh or Perl:  Python has 
no "dynamic" gimmicks buried in string literals.  All 
eval'ing a string literal can do is produce a binary blob 
via interpeting simple escape sequences.  They're like C 
strings this way -- maybe we'll run out of memory, but 
that's it.

I would agree that Python should be refactored internally 
to supply a clean function for changing string literals 
into binary blobs, but that would be for clarity and 
efficiency, not security.

----------------------------------------------------------------------

Comment By: Jim Fulton (dcjim)
Date: 2001-11-12 13:14

Message:
Logged In: YES 
user_id=73023

It sounds like there are some documentation bugs:

- The security ramifications are not discussed, nor are the
remedies.

- The cPickle module isn't documented very well. I submitted
some
   documentation a long time ago that never got incorporated
AFAIK.
   I wish I still had it. :)

- cPickle has a feature for turning off instance support and
for 
   restricting which classes can be unpickled. You can set
the find_global
   attribute on a cPickle.Unpickler. The find_global
attribute can be 
   a function or None.  If it is None, then no instances can
be 
   unpickled. If it is a function, then it should accept a
module and name
   and return the corresponding global. It is responsible
for looking
   up the global and can raise an error to prevent a
global's use.

   See the ZEO storage server implementation for an example
   of using this hook.


  



----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2001-11-12 01:33

Message:
Logged In: NO 

I like marshal and think it's much cleaner than pickle.
I'd like to see its format cleaned up and documented
so it can be a portable transport format.  Even the
recursion issue isn't so compelling--there's only a danger
on marshalling, not unmarshalling, and the application
has control over what it marshals.  So I think the idea
of documenting marshal (bug #467384) should be kept alive.
However, it shouldn't be changed in 2.2.  

If the Cookie class is still really aliased by default to
SmartCookie, that's a bad bug and should definitely be
fixed ASAP.  The Cookie docs clearly say not to use
SmartCookie or SerialCookie.  In fact I think SmartCookie
and SerialCookie should be disabled altogether.  Any
applications using them on the open internet have a security
but and should be fixed immediately.

Here's a possible temporary approach to fixing SmartCookie
and SerialCookie: append a cryptographic message
authentication code (MAC) to each pickled cookie.  That
means the application has some secret string K as a
configuration parameter (it should be 10 or more random
8-bit characters, or 20 or more random hex digits, etc). 
Then a smart cookie would be a pickle p, followed by
SHA(p+K).  The Cookie module would validate the MAC before
calling unpickle, and raise an exception if the MAC wasn't
valid.

The security of this scheme rests on K being kept secret
from attackers, which is generally not an easy thing to
manage, but it's something to think about.

Paul

----------------------------------------------------------------------

Comment By: Jeremy Hylton (jhylton)
Date: 2001-11-10 17:07

Message:
Logged In: YES 
user_id=31392

I don't think we should be doing anything about marshal.  
Maybe we should name in pyclib or something <0.9 wink>.  It 
works fine for .pyc files, but I don't see a reason for it 
to do anymore than is necessary for that purpose.

I think the notion of an unpickler that only handles 
builtin datatypes is the most attractive option you offer, 
but Paul has a good point about eval for strings.  (It 
currently has some hacks that address specific exploits, 
but I doubt they are sufficient.)  I also wonder how hard 
it is to handle builtin types and avoid subclasses of 
builtin types.

If there are any changes to pickle, I think we need to be 
careful about how it is described.  If we claim that an 
unpickler is safe for untrusted pickles, we've made a 
fairly strong claim.  I still think such a design change 
requires a PEP that includes some requirements and use 
cases and a thorough analysis of potential exploits. 


----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2001-11-10 14:42

Message:
Logged In: NO 

See bug #467384 for discussion about marshal.  Besides the
recursion issue, marshal's format is explicitly undocumented
and subject to change--you can't rely on it to interoperate
between two different Python versions, so it's no good as
an RPC serializer.  The format has kludges (e.g. the
representation of long ints) that make it undesirable to
freeze and document it and force future versions to be
backward compatible.

Adding a pickle.loads flag to prevent instance unpickling
isn't perfect but is probably the best alternative on 
your list.  Perhaps the flag can have a value that allows
unpickling the instances by restoring the instance 
attributes rather than calling the initializer.  That's
not always the right way to unpickle an instance (that's
why the unpickler no longer works that way) but it's good
enough a lot of the time.  

There's another issue with pickle/cPickle which is that they
unpickle quoted strings by evaling them.  This is scary.
While I don't see an immediate exploit, I also haven't
examined the 1000's of lines of code I'd need to examine
to convince myself that there's NOT an exploit.  I think
the unpickler should be changed to never call eval but just
parse the string as it needs to.  

Guido seemed to think pickle might have other possible
exploits.  I don't know what he had in mind but before
declaring it safe for untrusted data I think it needs to
be gone over with a fine toothed comb.

Paul


----------------------------------------------------------------------

Comment By: Barry A. Warsaw (bwarsaw)
Date: 2001-11-10 12:37

Message:
Logged In: YES 
user_id=12800

I'm going to agree with Paul that this is a problem needing
fixing, however there are really several issues.

1. Cookie module makes it too easy to code exploits.  Cookie
exports a class, also called Cookie, which is aliased to
SmartCookie, so that a naive program will simply pass cookie
data to Cookie.Cookie() and you're screwed.  So, Cookie
module's defaults should be for more security rather than
less, and Cookie.Cookie should be aliased to SimpleCookie
instead.

2. There is no built-in safe mechanism for de-serializing
data from untrusted sources.  You can't use pickle without
overloading a "magic" method.  You can't use cPickle because
you can't do the overloading trick.  You can't use marshal
because it isn't bulletproof against recursive
datastructures.  So how /do/ you do it?

I think it just may be serious enough to deal with in Python
2.2, and I volunteer to address it (so I'll steal this bug
report).  Without looking at the code, or the level of
effort necessary, I would make the following suggestions:

1. Add to the public interface of pickle and cPickle, a flag
that either disables the unpickling of instances altogether,
or disables calling any code with unpickled data, e.g.
constructors.

2. Fix marshal to be bulletproof against recursive
datastructures.
3. Update the docs for both pickle/cPickle and marshal to
explain how to safely write de-serializers of untrusted strings.

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2001-11-09 04:21

Message:
Logged In: NO 

Well, Guido and Tim agree with you that it's not a pickle
bug.  I still feel it is one, because its docs currently
make you think you can securely load untrusted pickles, and
because it's a natural, non-obscure thing to want to do
(see pyro and the cookie module), but whatever.  If it's
not a code bug then I feel it's a significant functionality
shortcoming in the python library.

Pyro uses pickle to serialize data for RPC calls over the
internet.  A malicious client could make a hostile pickle
take over the server.  The cookie module lets web
applications store user session state in browser cookies.
Its SerialCookie and SmartCookie classes let you put
arbitrary Python objects into the user session, and
serializes them when pickle.  Again, a malicious client
can make a hostile pickle, send it in a cookie header to
the http server, and take over the server when the 
application unpickles the cookie.

The current documentation for the pickle module makes it
very clear to me that the doc writer thought it was safe
to unpickle untrusted cookies.  If pickle wasn't designed
for that, then there was a communication failure between
the designer and the doc writer.

Yes, I'm willing to help with a PEP for fixing this
situation.

Paul


----------------------------------------------------------------------

Comment By: Jeremy Hylton (jhylton)
Date: 2001-11-08 12:37

Message:
Logged In: YES 
user_id=31392

I don't think of the issue you describe as a bug in the
code.  You're suggesting a new feature for pickle.  As far
as I can tell, the original design requirements for pickle
did not include the ability to securely load a pickle from
an untrusted source.

It may be a legitimate feature request, but it's too late to
make it into Python 2.2.  I suggest we look at the design
issues for Python 2.3 and decide if it's a feature we want
to support.  I imagine a PEP may be necessary to lay out the
issues and the solution.  Do you want to have a hand in that
PEP?

I still don't understand what it means that Pyro and cookie
were bit by a bug.  It sounds like they were using pickle in
ways that pickle was not intended to support.  A careful
analysis of how those two applications use pickle would be
helpful for generating requirements.


----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2001-11-07 18:54

Message:
Logged In: NO 

IMO it's a code bug that you can't unpickle strings from
untrusted sources.  Pyro and the cookie module are examples
of programs that got bitten by this bug.  Whether it's
really a bug is a matter of opinion--I had a big email
exchange with Guido and Tim about it, and they felt it
was enough to fix the pickle documentation.

Pickle has the same problem as cPickle, but with pickle
you can subclass the pickler and override the method that
unpickles class objects, and work around the (IMO) bug.
The workaround doesn't help cPickle since cPickle can't
be subclassed.  See bug #467384 for some related discussion.

Paul


----------------------------------------------------------------------

Comment By: Jeremy Hylton (jhylton)
Date: 2001-11-07 17:02

Message:
Logged In: YES 
user_id=31392

What's the code bug?  Your last message has a lot of gloom
and doom <wink>, but I'm not sure what specific problem
you'd like to see fixed.  Are you saying that something
needs to be fixed in cPickle and not in pickle?

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2001-11-07 15:08

Message:
Logged In: NO 

Irmen de Jong points out that the standard cookie module
uses pickling for serial and smart cookies.  The 2.1.1
cookie module docs explicitly say not to use those
classes because of the security hole--that they're provided
for backward compatibility only (but with what?!  Any
server that uses those classes on open ports needs to be
changed right away).

Irmen's library, http://pyro.sourceforge.net, also uses
unpickle insecurely (he's aware of this now and is figuring
out a fix).

IMO this is really a code bug rather than a documentation
bug, and should be fixed in the code rather than the docs.
Documenting the bug rather than fixing it leaves a
deficiency in the Python library: obvious uses of pickling,
like Pyro and the cookie module, can't be implemented
using cPickle and have to resort to a slower Python
deserializer, or use marshal and have possible compatibility
problems between versions (and not be able to serialize
class instances).

Paul



----------------------------------------------------------------------

Comment By: paul rubin (phr)
Date: 2001-10-16 19:06

Message:
Logged In: YES 
user_id=72053

Certainly anyone unserializing potentially malicious data
with pickle, marshal, or anything else, should check the
results before doing anything dangerous with them (like
executing code).  However, unpickle can potentially do
damage before it even returns, by creating loading modules
and creating initialized class instances.  So pickle.loads
should never be used on untrusted strings, except possibly
in a rexec wrapper (as proposed by Tim).  Another
possibility (also by Tim) is to override the load_inst
method of the Pickler class, though I don't think you can
do that for cPickle.

A sample exploit for unpickle can be found at
<http://www.nightsong.com/phr/python/pickletest.py>.
Unpickling the test string runs penguin.__init__ contrary
to the doc's saying no initialization unless there's a
__getinitargs__ method in the class def.

The "exploding penguin" class is artificial, but
applications are vulnerable if there's an unsafe
constructor anywhere in any class of the application or in
the python library (example: the NNTP constructor opens an
IP connection to an arbitrary address, so a malicious
imported string can send a message through your firewall
when you import it).


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=471893&group_id=5470


From noreply@sourceforge.net  Thu Feb  6 15:54:38 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 06 Feb 2003 07:54:38 -0800
Subject: [Python-bugs-list] [ python-Bugs-681003 ] itertools issues
Message-ID: <E18goMI-00040m-00@sc8-sf-web3.sourceforge.net>

Bugs item #681003, was opened at 2003-02-05 10:52
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=681003&group_id=5470

Category: Python Library
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Greg Chapman (glchapman)
Assigned to: Raymond Hettinger (rhettinger)
Summary: itertools issues

Initial Comment:
I've been checking out the new itertools module.  It's a great 
addition; I look forward to using it.  However, I did notice a 
few things:

1) I believe islice should worry about overflow when it adds 
step to next.  Given a call like islice(seq, 1, 10, sys.maxint), 
it looks like islice will not stop after retrieving item 1 
because this:

  lz->next += lz->step;

will result in lz->next becoming negative.

2) I note that starmap_next passes the args obtained from 
PyIter_Next directly to PyObject_Call.  PyObject_Call 
assumes that its arg param is a tuple (and so do the various 
tp_call slots).  However, there is nothing to ensure that the 
args object obtained from the iterator in starmap_next is in 
fact a tuple.  I believe either a PyTuple_Check or a 
PySequence_Tuple call is in order before passing the args 
to PyObject_Call.

3) I don't think it's safe for imap_next to reuse the same 
argtuple for all the calls.  This tuple is passed directly to C 
functions, and there's nothing to stop them from saving a 
reference to it someplace (unless it's documented 
somewhere that you shouldn't save references to the args 
tuple).

4) I think count_next should worry about overflow also 
(consider count(sys.maxint)).


----------------------------------------------------------------------

>Comment By: Raymond Hettinger (rhettinger)
Date: 2003-02-06 10:54

Message:
Logged In: YES 
user_id=80475

Glad you like the module.

Will implement the limit overflow checks and the tuple 
check.

I'm still thinking about the re-use of the arg tuple in 
imap_next().  The code was lifted directly from 
__builtins__.map() which has worked flawlessly since the 
early versions of Python.   But, if it does need fixing, then 
I'm looking at an approach that verifies the refcnt is one 
before re-using the tuple.  If that works out, then both the 
re-use and the guard can be applied to imap(), map(), izip
(), and enumerate().

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=681003&group_id=5470


From noreply@sourceforge.net  Thu Feb  6 16:02:55 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 06 Feb 2003 08:02:55 -0800
Subject: [Python-bugs-list] [ python-Bugs-680494 ] filecmp.py uses obsolete statcache module.
Message-ID: <E18goUJ-0002k9-00@sc8-sf-web4.sourceforge.net>

Bugs item #680494, was opened at 2003-02-04 16:55
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=680494&group_id=5470

Category: Python Library
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Inyeol Lee (inyeol)
Assigned to: Raymond Hettinger (rhettinger)
Summary: filecmp.py uses obsolete statcache module.

Initial Comment:
2.3a1 filecmp module, which is not obsolete, uses
obsolete
statcache module and generates DeprecationWarning.

Inyeol Lee

----------------------------------------------------------------------

>Comment By: Raymond Hettinger (rhettinger)
Date: 2003-02-06 11:02

Message:
Logged In: YES 
user_id=80475

Yes, the test suite looks fine.

The lines for suite.addTest are usually put in a loop.

Instead of the setup/teardown steps, consider adding 
some small static files to the test directory: 
filecmp_basecase, filecmp_samecase, filecmp_diffcase.
That would simplify your code and bypass issues of write 
permissions, running tests from a CD-rom, etc.

----------------------------------------------------------------------

Comment By: A.M. Kuchling (akuchling)
Date: 2003-02-05 14:50

Message:
Logged In: YES 
user_id=11375

Attaching test_filecmp.py.  Raymond, you've written test suites with unittest.py, right?  Am I using it in a reasonable way?


----------------------------------------------------------------------

Comment By: A.M. Kuchling (akuchling)
Date: 2003-02-05 14:37

Message:
Logged In: YES 
user_id=11375

Attaching test_filecmp.py.  Raymond, you've written test suites with unittest.py, right?  Am I using it in a reasonable way?


----------------------------------------------------------------------

Comment By: A.M. Kuchling (akuchling)
Date: 2003-02-05 14:07

Message:
Logged In: YES 
user_id=11375

There's no test suite for filecmp, so before making any changes, I'm 
going to write a test suite.


----------------------------------------------------------------------

Comment By: A.M. Kuchling (akuchling)
Date: 2003-02-05 13:23

Message:
Logged In: YES 
user_id=11375

The simplest thing would be to simply use os.stat instead of statcache,
and deprecate use_statcache by logging a warning if use_statcache is true.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=680494&group_id=5470


From noreply@sourceforge.net  Thu Feb  6 16:03:26 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 06 Feb 2003 08:03:26 -0800
Subject: [Python-bugs-list] [ python-Bugs-680494 ] filecmp.py uses obsolete statcache module.
Message-ID: <E18goUo-0002kq-00@sc8-sf-web4.sourceforge.net>

Bugs item #680494, was opened at 2003-02-04 16:55
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=680494&group_id=5470

Category: Python Library
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Inyeol Lee (inyeol)
>Assigned to: A.M. Kuchling (akuchling)
Summary: filecmp.py uses obsolete statcache module.

Initial Comment:
2.3a1 filecmp module, which is not obsolete, uses
obsolete
statcache module and generates DeprecationWarning.

Inyeol Lee

----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2003-02-06 11:02

Message:
Logged In: YES 
user_id=80475

Yes, the test suite looks fine.

The lines for suite.addTest are usually put in a loop.

Instead of the setup/teardown steps, consider adding 
some small static files to the test directory: 
filecmp_basecase, filecmp_samecase, filecmp_diffcase.
That would simplify your code and bypass issues of write 
permissions, running tests from a CD-rom, etc.

----------------------------------------------------------------------

Comment By: A.M. Kuchling (akuchling)
Date: 2003-02-05 14:50

Message:
Logged In: YES 
user_id=11375

Attaching test_filecmp.py.  Raymond, you've written test suites with unittest.py, right?  Am I using it in a reasonable way?


----------------------------------------------------------------------

Comment By: A.M. Kuchling (akuchling)
Date: 2003-02-05 14:37

Message:
Logged In: YES 
user_id=11375

Attaching test_filecmp.py.  Raymond, you've written test suites with unittest.py, right?  Am I using it in a reasonable way?


----------------------------------------------------------------------

Comment By: A.M. Kuchling (akuchling)
Date: 2003-02-05 14:07

Message:
Logged In: YES 
user_id=11375

There's no test suite for filecmp, so before making any changes, I'm 
going to write a test suite.


----------------------------------------------------------------------

Comment By: A.M. Kuchling (akuchling)
Date: 2003-02-05 13:23

Message:
Logged In: YES 
user_id=11375

The simplest thing would be to simply use os.stat instead of statcache,
and deprecate use_statcache by logging a warning if use_statcache is true.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=680494&group_id=5470


From noreply@sourceforge.net  Thu Feb  6 16:26:34 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 06 Feb 2003 08:26:34 -0800
Subject: [Python-bugs-list] [ python-Bugs-681461 ] warning filter doc misleading
Message-ID: <E18gorC-00072r-00@sc8-sf-web1.sourceforge.net>

Bugs item #681461, was opened at 2003-02-05 23:53
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=681461&group_id=5470

Category: Documentation
Group: Python 2.3
>Status: Closed
>Resolution: Fixed
Priority: 5
Submitted By: Richard Jones (richard)
Assigned to: Nobody/Anonymous (nobody)
Summary: warning filter doc misleading

Initial Comment:
The doc for warnings filtering states that "message is a compiled 
regular expression" when it actually has to be a string. It is compiled 
by filterwarnings. Same goes for "module". 
 

----------------------------------------------------------------------

>Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-06 11:26

Message:
Logged In: YES 
user_id=33168

amk checked in libwarnings.tex,1.11

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=681461&group_id=5470


From noreply@sourceforge.net  Thu Feb  6 16:47:10 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 06 Feb 2003 08:47:10 -0800
Subject: [Python-bugs-list] [ python-Bugs-681003 ] itertools issues
Message-ID: <E18gpB8-0006mF-00@sc8-sf-web3.sourceforge.net>

Bugs item #681003, was opened at 2003-02-05 06:52
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=681003&group_id=5470

Category: Python Library
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Greg Chapman (glchapman)
Assigned to: Raymond Hettinger (rhettinger)
Summary: itertools issues

Initial Comment:
I've been checking out the new itertools module.  It's a great 
addition; I look forward to using it.  However, I did notice a 
few things:

1) I believe islice should worry about overflow when it adds 
step to next.  Given a call like islice(seq, 1, 10, sys.maxint), 
it looks like islice will not stop after retrieving item 1 
because this:

  lz->next += lz->step;

will result in lz->next becoming negative.

2) I note that starmap_next passes the args obtained from 
PyIter_Next directly to PyObject_Call.  PyObject_Call 
assumes that its arg param is a tuple (and so do the various 
tp_call slots).  However, there is nothing to ensure that the 
args object obtained from the iterator in starmap_next is in 
fact a tuple.  I believe either a PyTuple_Check or a 
PySequence_Tuple call is in order before passing the args 
to PyObject_Call.

3) I don't think it's safe for imap_next to reuse the same 
argtuple for all the calls.  This tuple is passed directly to C 
functions, and there's nothing to stop them from saving a 
reference to it someplace (unless it's documented 
somewhere that you shouldn't save references to the args 
tuple).

4) I think count_next should worry about overflow also 
(consider count(sys.maxint)).


----------------------------------------------------------------------

>Comment By: Greg Chapman (glchapman)
Date: 2003-02-06 07:47

Message:
Logged In: YES 
user_id=86307

Just a quick note on imap.  I checked the 2.3a1 map function 
before posting (and just rechecked).  It creates the alist tuple at 
the beginning of the loop, and then eventually passes it to 
PyEval_CallObject before DECREFing it.  So it looks to me like 
the current version of map guards against someone saving a 
reference to alist.


----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2003-02-06 06:54

Message:
Logged In: YES 
user_id=80475

Glad you like the module.

Will implement the limit overflow checks and the tuple 
check.

I'm still thinking about the re-use of the arg tuple in 
imap_next().  The code was lifted directly from 
__builtins__.map() which has worked flawlessly since the 
early versions of Python.   But, if it does need fixing, then 
I'm looking at an approach that verifies the refcnt is one 
before re-using the tuple.  If that works out, then both the 
re-use and the guard can be applied to imap(), map(), izip
(), and enumerate().

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=681003&group_id=5470


From noreply@sourceforge.net  Thu Feb  6 16:49:27 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 06 Feb 2003 08:49:27 -0800
Subject: [Python-bugs-list] [ python-Bugs-680864 ] test_datetime fails for non-unix epoch
Message-ID: <E18gpDL-0007zn-00@sc8-sf-web1.sourceforge.net>

Bugs item #680864, was opened at 2003-02-05 08:15
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=680864&group_id=5470

Category: None
Group: None
>Status: Closed
>Resolution: Fixed
Priority: 5
Submitted By: Jack Jansen (jackjansen)
Assigned to: Tim Peters (tim_one)
Summary: test_datetime fails for non-unix epoch

Initial Comment:
test_datetime fails on Mac OS 9 in test_tzinfo_fromtimestamp, it seems to be depending on the Unix epoch of 1970:

timestamp = 1000000000  #  2001-09-09 01:46:40 UTC, give or take

The mac epoch is 1-jan-1904, localtime (sic!).

----------------------------------------------------------------------

>Comment By: Tim Peters (tim_one)
Date: 2003-02-06 11:49

Message:
Logged In: YES 
user_id=31435

datetime does assume POSIX timestamps, but weakly, and 
the test in question didn't really need to rely on POSIX.  So, 
it's fixed now, in Lib/test/test_datetime.py rev 1.36.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=680864&group_id=5470


From noreply@sourceforge.net  Thu Feb  6 17:19:09 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 06 Feb 2003 09:19:09 -0800
Subject: [Python-bugs-list] [ python-Bugs-681003 ] itertools issues
Message-ID: <E18gpg5-0005dc-00@sc8-sf-web4.sourceforge.net>

Bugs item #681003, was opened at 2003-02-05 17:52
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=681003&group_id=5470

Category: Python Library
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Greg Chapman (glchapman)
Assigned to: Raymond Hettinger (rhettinger)
Summary: itertools issues

Initial Comment:
I've been checking out the new itertools module.  It's a great 
addition; I look forward to using it.  However, I did notice a 
few things:

1) I believe islice should worry about overflow when it adds 
step to next.  Given a call like islice(seq, 1, 10, sys.maxint), 
it looks like islice will not stop after retrieving item 1 
because this:

  lz->next += lz->step;

will result in lz->next becoming negative.

2) I note that starmap_next passes the args obtained from 
PyIter_Next directly to PyObject_Call.  PyObject_Call 
assumes that its arg param is a tuple (and so do the various 
tp_call slots).  However, there is nothing to ensure that the 
args object obtained from the iterator in starmap_next is in 
fact a tuple.  I believe either a PyTuple_Check or a 
PySequence_Tuple call is in order before passing the args 
to PyObject_Call.

3) I don't think it's safe for imap_next to reuse the same 
argtuple for all the calls.  This tuple is passed directly to C 
functions, and there's nothing to stop them from saving a 
reference to it someplace (unless it's documented 
somewhere that you shouldn't save references to the args 
tuple).

4) I think count_next should worry about overflow also 
(consider count(sys.maxint)).


----------------------------------------------------------------------

Comment By: Christos Georgiou (tzot)
Date: 2003-02-06 19:19

Message:
Logged In: YES 
user_id=539787

Yes, itertools is great, and I wonder how Guido agreed to it 
(since he usually dislikes using FP stuff in Python).  I also 
think that the tuple way checking refcount will work.

Another point comes to mind: in many places, the code 
(Python in general, not only itertools) builds a tuple and calls 
PyEval_CallObject with it and without any keyword 
arguments; I believe in these cases the call should be 
PyObject_Call, since there's no use in checking every time if 
first arg is a tuple and then calling PyObject_Call...

----------------------------------------------------------------------

Comment By: Greg Chapman (glchapman)
Date: 2003-02-06 18:47

Message:
Logged In: YES 
user_id=86307

Just a quick note on imap.  I checked the 2.3a1 map function 
before posting (and just rechecked).  It creates the alist tuple at 
the beginning of the loop, and then eventually passes it to 
PyEval_CallObject before DECREFing it.  So it looks to me like 
the current version of map guards against someone saving a 
reference to alist.


----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2003-02-06 17:54

Message:
Logged In: YES 
user_id=80475

Glad you like the module.

Will implement the limit overflow checks and the tuple 
check.

I'm still thinking about the re-use of the arg tuple in 
imap_next().  The code was lifted directly from 
__builtins__.map() which has worked flawlessly since the 
early versions of Python.   But, if it does need fixing, then 
I'm looking at an approach that verifies the refcnt is one 
before re-using the tuple.  If that works out, then both the 
re-use and the guard can be applied to imap(), map(), izip
(), and enumerate().

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=681003&group_id=5470


From noreply@sourceforge.net  Thu Feb  6 17:59:42 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 06 Feb 2003 09:59:42 -0800
Subject: [Python-bugs-list] [ python-Bugs-680494 ] filecmp.py uses obsolete statcache module.
Message-ID: <E18gqJK-0007oM-00@sc8-sf-web4.sourceforge.net>

Bugs item #680494, was opened at 2003-02-04 16:55
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=680494&group_id=5470

Category: Python Library
Group: Python 2.3
>Status: Closed
>Resolution: Fixed
Priority: 5
Submitted By: Inyeol Lee (inyeol)
Assigned to: A.M. Kuchling (akuchling)
Summary: filecmp.py uses obsolete statcache module.

Initial Comment:
2.3a1 filecmp module, which is not obsolete, uses
obsolete
statcache module and generates DeprecationWarning.

Inyeol Lee

----------------------------------------------------------------------

>Comment By: A.M. Kuchling (akuchling)
Date: 2003-02-06 12:59

Message:
Logged In: YES 
user_id=11375

OK, I'll check it in.  I've added a suite.addTest loop, but left the setup/teardown steps alone for now.

I've also updated filecmp.py to no longer use the statcache module, so I'm marking this bug as closed.


----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2003-02-06 11:02

Message:
Logged In: YES 
user_id=80475

Yes, the test suite looks fine.

The lines for suite.addTest are usually put in a loop.

Instead of the setup/teardown steps, consider adding 
some small static files to the test directory: 
filecmp_basecase, filecmp_samecase, filecmp_diffcase.
That would simplify your code and bypass issues of write 
permissions, running tests from a CD-rom, etc.

----------------------------------------------------------------------

Comment By: A.M. Kuchling (akuchling)
Date: 2003-02-05 14:50

Message:
Logged In: YES 
user_id=11375

Attaching test_filecmp.py.  Raymond, you've written test suites with unittest.py, right?  Am I using it in a reasonable way?


----------------------------------------------------------------------

Comment By: A.M. Kuchling (akuchling)
Date: 2003-02-05 14:37

Message:
Logged In: YES 
user_id=11375

Attaching test_filecmp.py.  Raymond, you've written test suites with unittest.py, right?  Am I using it in a reasonable way?


----------------------------------------------------------------------

Comment By: A.M. Kuchling (akuchling)
Date: 2003-02-05 14:07

Message:
Logged In: YES 
user_id=11375

There's no test suite for filecmp, so before making any changes, I'm 
going to write a test suite.


----------------------------------------------------------------------

Comment By: A.M. Kuchling (akuchling)
Date: 2003-02-05 13:23

Message:
Logged In: YES 
user_id=11375

The simplest thing would be to simply use os.stat instead of statcache,
and deprecate use_statcache by logging a warning if use_statcache is true.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=680494&group_id=5470


From noreply@sourceforge.net  Thu Feb  6 18:03:55 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 06 Feb 2003 10:03:55 -0800
Subject: [Python-bugs-list] [ python-Bugs-472881 ] distutils does not deduce dependencies
Message-ID: <E18gqNP-00081b-00@sc8-sf-web4.sourceforge.net>

Bugs item #472881, was opened at 2001-10-19 14:49
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=472881&group_id=5470

Category: Distutils
>Group: Feature Request
>Status: Closed
>Resolution: Later
Priority: 5
Submitted By: Skip Montanaro (montanaro)
Assigned to: Nobody/Anonymous (nobody)
Summary: distutils does not deduce dependencies

Initial Comment:
I just "cvs up"d my Python tree and executed "make".
Nothing much changed, except patchlevel.h.  Make got
this right and rebuilt everything (patchlevel.h is
included from Python.h, so changing patchlevel.h
should cause make to rebuild just about everything).

Distutils failed to notice this, however, so it didn't rebuild any modules.  In this case, I think the
failure to rebuild is probably innocuous, but I'm
not at all confident it will do the right thing if I
modify some other file.  For example, I've noticed
that it's not sufficient to delete a module's .o file
when you want to rebuild it.  Distutils still thinks
the .so file is okay.

If distutils is going to take the place of make I
think it's going to need to do a much better job
deducing file dependencies or provide a robust
way for a person to tell it what those dependencies
are.


----------------------------------------------------------------------

>Comment By: A.M. Kuchling (akuchling)
Date: 2003-02-06 13:03

Message:
Logged In: YES 
user_id=11375

Added to PEP 42 and marking as closed.

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2002-12-05 13:20

Message:
Logged In: YES 
user_id=6656

This has been sitting on my plate forever.

AIUI, distutils can now be told about dependencies, but does
nothing to deduce them itself.  As this puts it in good
company (eg, make), I reckon we can probably close this?

At any rate, I don't see me getting to it.

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2002-01-23 05:28

Message:
Logged In: YES 
user_id=6656

Umm, I didn't mean to close this.  Oh well.

TBH, I'm tempted to close it again, with a resolution of
"wont fix", but am open to persuasion.

I'm not sure I can summon the motivation to add dependency
tracking to distutils.  And I don't like the hack I've
attached to the report.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2001-12-10 10:16

Message:
Logged In: YES 
user_id=6380

I agree.  Let's put this off until 2.3 (or maybe 2.2.1).

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2001-12-10 07:39

Message:
Logged In: YES 
user_id=6656

Here's an attempt.  TBH, I'd be a bit nervous about fiddling
with the build at this stage, but assigning to Guido to decide.

I'd recommend assigning it back to me Pending/Later, and
applying something like this early in 2.3.

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2001-12-07 10:49

Message:
Logged In: YES 
user_id=6656

OK, I'll try to get something concrete done in the next few
days.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2001-12-07 09:54

Message:
Logged In: YES 
user_id=6380

It depends on what kind of change you are making to a header
file. 99% of the time I find that a change to a header is
something innocuous like add a new function, and it still
causes a rebuild of the world. That's particularly annoying
when I'm experimenting with a new object type and adding
things to the header file one at a time -- each time the
header gets touched everything gets rebuilt.

But it's easy enough to only rebuild the core ("make
python") so I'm not sure if I really oppose your simple
solution.

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2001-12-07 08:53

Message:
Logged In: YES 
user_id=6656

Eh, I was going to add a os.system("touch last_build")
thingy to setup.py.  Not aiming for elegance here.

Is it really 99% of the time that changes in headers are
irrelevant?  Then what make does is a bit silly, surely.

It would presumably be easy enough to ask the user, but I
know I'd find that really annoying (I generally fire a biuld
off, do something else for a few minutes and come back. 
OTOH, I almost always go through a "rm -rfv build && mkdir
build && cd build && ../configure --prefix=$HOME && make"
ritual anyway so this issue doesn't affect me in the slightest).

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2001-12-07 08:40

Message:
Logged In: YES 
user_id=6380

Hm, how do you know when the last build was?

Also, since in 99% of the cases this is unnecessary and
rebuilding all extensions takes a long time, I'd like this
to be optional -- maybe it can ask the user before zapping
build?

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2001-12-07 07:36

Message:
Logged In: YES 
user_id=6656

Would a "good enough" solution be to check (in setup.py) if
pyconfig.h or anything in Include/ has changed since the
last build and if so blow away the & build directory?

Doing a "proper" job seems frankly unlikely, and would have
much the same effect as the above anyway for building Python...

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=472881&group_id=5470


From noreply@sourceforge.net  Thu Feb  6 18:15:58 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 06 Feb 2003 10:15:58 -0800
Subject: [Python-bugs-list] [ python-Bugs-481896 ] popen3 file objects hanging
Message-ID: <E18gqZ4-0003xB-00@sc8-sf-web1.sourceforge.net>

Bugs item #481896, was opened at 2001-11-14 18:13
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=481896&group_id=5470

Category: Python Library
>Group: Not a Bug
>Status: Closed
>Resolution: Wont Fix
Priority: 5
Submitted By: Nobody/Anonymous (nobody)
Assigned to: Nobody/Anonymous (nobody)
Summary: popen3 file objects hanging

Initial Comment:
RE:#210676
System: Python 2.0 on solaris 8

The above bug is reported closed but I cannot see why.

>>> sub = popen2.Popen3(cmd,1)
>>> sub.fromchild.readlines()  # blocked on stderr


As reported in the above sited bug report, we too are
suffering from this behaviour on file objects returned
by popen3.  It does appear that the stderr buffer is
filling (the app writes a lot of stuff to stderr) and
blocking stdout, but i cannot seem to flush the
buffer.  I need to acquire both stderr and stdout
data.  I have tried setting the bufsize to 0
(unbuffered) but that doesn't work.  I have made the
buffer big, but nothing there.  Might there be
some magic bufsize?

popen4 (which stuffs both stderr and stdout into one
file object) works perfectly as there is no blocking.  

Reading through the bugs about popen3, the bug report
above is exactly what we are experiencing.  The
"solution" offered is not a solution at all. We are
running 3rd party code, i.e. we can't close stderr
inside the application, nor would we want to. stderr
and stdout file descriptors must remain open for
the duration of the application.

Has there been some viable solution to this which has
not been documented?

----------------------------------------------------------------------

>Comment By: A.M. Kuchling (akuchling)
Date: 2003-02-06 13:15

Message:
Logged In: YES 
user_id=11375

Marking as closed.  (Unless someone can suggest text to be added to the documentation to clarify this?)


----------------------------------------------------------------------

Comment By: Gustavo Niemeyer (niemeyer)
Date: 2002-11-11 20:31

Message:
Logged In: YES 
user_id=7887

You can workaround the problem in the following ways:  
  
1) as martin explained in the previous mentioned bug;  
  
2)  using non-blocking I/O:   
     
fd = pop.childerr.fileno()     
flags = fcntl.fcntl (fd, fcntl.F_GETFL, 0)     
flags = flags | os.O_NONBLOCK     
fcntl.fcntl (fd, fcntl.F_SETFL, flags)     
     
3) piping the child error into /dev/null, if you don't want it.     
  
Either way, IMO that's something which belongs to how the operating  
system deals with that, since there's no way for Python to guess what  
you'd expect it to do in all possible cases, and it probably shouldn't try to.     
  
I suggest closing that bug. 
  

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=481896&group_id=5470


From noreply@sourceforge.net  Thu Feb  6 18:33:43 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 06 Feb 2003 10:33:43 -0800
Subject: [Python-bugs-list] [ python-Bugs-504282 ] python 2.2 shelve
Message-ID: <E18gqqF-00041W-00@sc8-sf-web2.sourceforge.net>

Bugs item #504282, was opened at 2002-01-16 04:48
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=504282&group_id=5470

Category: Extension Modules
Group: Python 2.2
>Status: Closed
Resolution: None
Priority: 5
Submitted By: tarski  (tarski9)
Assigned to: Nobody/Anonymous (nobody)
Summary: python 2.2 shelve

Initial Comment:
I've created a shelve 'db-test' under Red Hat Linux 7.1
and Python 2.1.1. When I want to open this shelve under
Win2k with Python 2.1.1 or Python 2.2, following occurs.


>>> db = shelve.open("C:\db-test")
Traceback (most recent call last):
  File "<pyshell#4>", line 1, in ?
    db = shelve.open("C:\db-test")
  File "C:\PROGRAMS\PYTHON22\lib\shelve.py", line 158,
in open
    return DbfilenameShelf(filename, flag)
  File "C:\PROGRAMS\PYTHON22\lib\shelve.py", line 148,
in __init__
    Shelf.__init__(self, anydbm.open(filename, flag))
  File "C:\PROGRAMS\PYTHON22\lib\anydbm.py", line 86,
in open
    return mod.open(file, flag, mode)
  File "C:\PROGRAMS\PYTHON22\lib\dbhash.py", line 16,
in open
    return bsddb.hashopen(file, flag, mode)
error: (22, 'Invalid argument')


Is there a bug in the bsddb.lib und Windows? Is there a
solution to create shelves under Linux and to use them
under Windows?

Thanks

tarski

----------------------------------------------------------------------

>Comment By: A.M. Kuchling (akuchling)
Date: 2003-02-06 13:33

Message:
Logged In: YES 
user_id=11375

Marking as closed; it seems unlikely the original reporter will respond after a year.


----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2002-01-16 07:07

Message:
Logged In: YES 
user_id=21627

Unfortunately, there are different versions of BSDDB in
circulation, which use different file formats.

On Linux, please do

ldd <path-to-python-lib>/lib-dynload/bsddbmodule.so

to find out what version you have linked with. Also, please
invoke file(1) to find out the file version. I can't tell
off-hand what version the standard Windows installation is
build with, but knowing what you use on Linux will get us
closer.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=504282&group_id=5470


From noreply@sourceforge.net  Thu Feb  6 18:36:22 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 06 Feb 2003 10:36:22 -0800
Subject: [Python-bugs-list] [ python-Bugs-680429 ] __module__ broken for extension classes
Message-ID: <E18gqso-0004xS-00@sc8-sf-web1.sourceforge.net>

Bugs item #680429, was opened at 2003-02-04 11:55
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=680429&group_id=5470

Category: Python Interpreter Core
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Ralf W. Grosse-Kunstleve (rwgk)
Assigned to: Michael Hudson (mwh)
Summary: __module__ broken for extension classes

Initial Comment:
We are having problems using Boost.Python with Python 
2.3a1 because
under some circumstances the result of __module__ is 
different compared
to earlier Python versions:

Python 2.2.1 (#2, Jun 17 2002, 12:06:51) 
[GCC 2.96 20000731 (Red Hat Linux 7.3 2.96-110)] on 
linux2
Type "help", "copyright", "credits" or "license" for more 
information.
>>> import simple
>>> simple.empty.__module__
'simple'
>>> 

Python 2.3a1 (#1, Jan  6 2003, 14:17:56) 
[GCC 2.96 20000731 (Red Hat Linux 7.3 2.96-110)] on 
linux2
Type "help", "copyright", "credits" or "license" for more 
information.
>>> import simple
>>> simple.empty.__module__
'__main__'
>>> 

Because of this we can no longer pickle our extension 
classes.
For your reference the code for the simple module is 
attached.
This is using Boost release 1.29.0 (www.boost.org).

We have done some debugging. Boost.Python's internal 
idea of the module
associated with an extension class is still correct even when 
using
Python 2.3a1. David Abrahams (main Boost.Python author) 
is telling me
that he "changed Boost.Python to work the way Guido 
suggested before
2.2.2." Therefore we suspect that the __module__ problem 
is due to
a change/bug in Python 2.3a1.

Ralf


#include <boost/python/module.hpp>
#include <boost/python/class.hpp>

namespace sandbx { namespace {

  struct empty {};

  void init_module()
  {
    using namespace boost::python;
    class_<empty>("empty");
  }

}} // namespace sandbx::<anonymous>

BOOST_PYTHON_MODULE(simple)
{
  sandbx::init_module();
}


----------------------------------------------------------------------

>Comment By: Ralf W. Grosse-Kunstleve (rwgk)
Date: 2003-02-06 10:36

Message:
Logged In: YES 
user_id=71407

> In 2.2.x, how are you telling Python that __module__ should
> be "simple"?  By setting tp_name to "simple.empty"?  What's
> simple.__name__?

Python 2.3a1 (#2, Feb  5 2003, 09:39:30) 
[GCC 2.96 20000731 (Red Hat Linux 7.3 2.96-110)] on linux2
Type "help", "copyright", "credits" or "license" for more 
information.
>>> import sandbx.simple
>>> sandbx.simple.__name__
'sandbx.simple'
>>> sandbx.simple.empty.__name__
'sandbx.simple.empty'
>>> sandbx.simple.empty.__module__
IS HEAPTYPE
type->tp_dict, __module__ = __main__
'__main__'
>>> 

> I think I know how __module__ is getting there, it's the
> chunk of code currently at typeobject.c:1750.

Yes, that's right. I've verified this by adding more print statements.

> Can you point me to the bit of the boost source that creates
> the typeobject?

Here is where it happens (file 
boost/libs/python/src/object/class.cpp):

object result = object(class_metatype())(module_prefix() + name, 
bases, d);

With print statements I've verified that module_prefix() + name is
correctly passed in as "sandbx.simple.empty" .

I am attaching the file class.cpp. Could it be that we have to
update our class_metatype_object?

Thanks!
        Ralf


----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2003-02-06 03:22

Message:
Logged In: YES 
user_id=6656

OK, thanks for the info.  More questions, I'm afraid:

In 2.2.x, how are you telling Python that __module__ should
be "simple"?  By setting tp_name to "simple.empty"?  What's
simple.__name__?

I think I know how __module__ is getting there, it's the
chunk of code currently at typeobject.c:1750.

Can you point me to the bit of the boost source that creates
the typeobject?  'cept sf's just fallen off the net.

----------------------------------------------------------------------

Comment By: Ralf W. Grosse-Kunstleve (rwgk)
Date: 2003-02-05 10:25

Message:
Logged In: YES 
user_id=71407

> Look at typeobject.c:type_module() (about 100 lines in).
>
> Is simple.empty a HEAPTYPE?

Yes, it is a HEAPTYPE. I've established this by adding print 
statements
in typeobject.c:type_module():

static PyObject *
type_module(PyTypeObject *type, void *context)
{
        PyObject *mod;
        char *s;

        if (type->tp_flags & Py_TPFLAGS_HEAPTYPE) {
printf("IS HEAPTYPE\n");
                mod = PyDict_GetItemString(type-
>tp_dict, "__module__");
                Py_XINCREF(mod);
                s = PyString_AsString(mod);
printf("type->tp_dict, __module__ = %s\n", s);
                return mod;
        }

Result:

Python 2.3a1 (#2, Feb  5 2003, 09:39:30) 
[GCC 2.96 20000731 (Red Hat Linux 7.3 2.96-110)] on linux2
Type "help", "copyright", "credits" or "license" for more 
information.
>>> import simple
>>> simple.empty.__module__
IS HEAPTYPE
type->tp_dict, __module__ = __main__
'__main__'
>>> 

> If you can tell me what you want that function to do, I can
> have a go at it.

We expect "simple" as the result of simple.empty.__module__.

Current result if simple.so is moved to a package:

Python 2.3a1 (#2, Feb  5 2003, 09:39:30) 
[GCC 2.96 20000731 (Red Hat Linux 7.3 2.96-110)] on linux2
Type "help", "copyright", "credits" or "license" for more 
information.
>>> from sandbx import simple
>>> simple.empty.__module__
IS HEAPTYPE
type->tp_dict, __module__ = __main__
'__main__'
>>> 

Here we expect "sandbx.simple" .
Python 2.2.x produces the expected results using the exact same
Boost.Python source code.

David Abrahams provides this additional information:

  Remember that simple.empty is created by calling the metatype, 
so
  it's created on the heap, but it's Python's internal type creation
  mechanisms which do it -- we're not setting the flags manually.

Thanks!
        Ralf


----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2003-02-05 06:10

Message:
Logged In: YES 
user_id=6656

There has certainly been a change here.

Look at typeobject.c:type_module() (about 100 lines in).

If you can tell me what you want that function to do, I can
have a go at it.

Is simple.empty a HEAPTYPE?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=680429&group_id=5470


From noreply@sourceforge.net  Thu Feb  6 19:40:32 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 06 Feb 2003 11:40:32 -0800
Subject: [Python-bugs-list] [ python-Bugs-471893 ] Security review of pickle/marshal docs
Message-ID: <E18grsu-0003nT-00@sc8-sf-web4.sourceforge.net>

Bugs item #471893, was opened at 2001-10-16 18:42
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=471893&group_id=5470

Category: Extension Modules
Group: None
Status: Open
Resolution: Fixed
Priority: 7
Submitted By: Tim Peters (tim_one)
Assigned to: A.M. Kuchling (akuchling)
Summary: Security review of pickle/marshal docs

Initial Comment:
Paul Rubin points out that the security implications 
of using marshal and/or pickle aren't clear from the 
docs.  Assigning to Jeremy as he's more sensitive to 
such issues than I am; maybe Barry would like to get 
paranoid too <wink>.

A specific example:  the pickle docs say that pickle 
doesn't support code objects, and "at least this 
avoids the possibility of smuggling Trojan horses into 
a program".  However,

1) The marshal docs don't mention this vulnerability 
at all.

while

2) The pickle docs don't spell out possible dangers 
due to things pickle does that marshal doesn't (like 
importing modules, and running class constructors).

----------------------------------------------------------------------

>Comment By: Tim Peters (tim_one)
Date: 2003-02-06 14:40

Message:
Logged In: YES 
user_id=31435

I think there are several reasons to override these methods.  
The one most relevant to this bug report is that, while Python 
has stopped pretending that pickles are secure by default, 
the choke points are still there, and motivated users can still 
expolit them.

For example, search pickle.py for __import__.  The only 
occurrence of __import__ in the Unpickler class is in method 
find_class(), and that's by design.  If a user overrides 
find_class(), the only imports the Unpickler *can* do are 
those the user explicitly performs in their own find_class() 
implementation.  So if that's a notion of "security" a user is 
happy with, they can still have it.  The docs trying to describe 
this are still valid.  It's only the "by magic" safety checks that 
have gone away (and they were buggy anyway, so no loss).

----------------------------------------------------------------------

Comment By: A.M. Kuchling (akuchling)
Date: 2003-02-06 09:29

Message:
Logged In: YES 
user_id=11375

The Cookie classes that use pickle have DeprecationWarnings in 
2.3, and should disappear in 2.4.  If {find,load}_global are 
still going to be supported, though, then they should still be documented (though, given that you shouldn't be unpickling untrusted data, why would you ever bother overriding find_global?)


----------------------------------------------------------------------

Comment By: Barry A. Warsaw (bwarsaw)
Date: 2003-02-05 18:03

Message:
Logged In: YES 
user_id=12800

I'll just mention that anybody using anything other than
Cookie.SimpleCookie is insane, and even using the Cookie
module at all to parse real-world cookies is mildly nuts
because it's way too strict.  I prefer m&m's in my cookies.

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2003-02-05 17:57

Message:
Logged In: YES 
user_id=31435

Andrew, didn't you go overboard in deleting text here?  The 
__safe_for_unpickling__ check is gone in 2.3, but most of the 
rest of the text still applies.  In particular, Cookie is still a 
problem for people inclined to worry about this, and overriding 
methods like find_global and load_global is still valuable stuff 
(the unpickler still never imports anything that doesn't come 
from an opcode triggering one of these methods).  

----------------------------------------------------------------------

Comment By: Barry A. Warsaw (bwarsaw)
Date: 2003-02-05 16:29

Message:
Logged In: YES 
user_id=12800

Karmically (no, not comically) reassigning to Tim who
started this whole thing, and who happens to be dumping his
pickles these days.

----------------------------------------------------------------------

Comment By: A.M. Kuchling (akuchling)
Date: 2003-02-05 15:40

Message:
Logged In: YES 
user_id=11375

Re-opening.  With the changes to pickling in Python 2.3, 
the security material should be replaced with a warning to 
not unpickle untrusted data.  Patch attached.



----------------------------------------------------------------------

Comment By: Barry A. Warsaw (bwarsaw)
Date: 2001-11-18 11:08

Message:
Logged In: YES 
user_id=12800

You're right of course.  I meant to fix that and forgot. 
Will do so asap.  Glad you like the rest of it, though! :)

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2001-11-18 01:56

Message:
Logged In: NO 

Barry - the new docs just went up and they're a big
improvement over the old.  However the sentence 
"The issue here is usually not that a class's constructor
will get called -- it won't by the unpickler -- but that the
class's destructor (i.e. its __del__() method) might get
called when the object is garbage collected." isn't
correct.  

There's a flag in the pickle stream that tells
the unpickler that the pickler found a __getinitargs__
method at pickling time.  If the flag is set in the pickle,
then the unpickler calls the class constructor, whether
there's a __getinitargs__ method in the class or not.

The sample exploit that I posted earlier on,
<http://www.nightsong.com/phr/python/pickletest.py>,
is an example of an artificially concocted pickle that
makes a class constructor get called.


----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2001-11-17 17:07

Message:
Logged In: NO 

Are the new docs downloadable from somewhere?  thanx  --Paul


----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2001-11-15 19:43

Message:
Logged In: NO 

Barry, can you also do something about the Cookie module,
or at least about its documentation?  If Cookie is aliased
to SmartCookie, I think that needs mention.
 --Paul

----------------------------------------------------------------------

Comment By: Barry A. Warsaw (bwarsaw)
Date: 2001-11-15 19:12

Message:
Logged In: YES 
user_id=12800

I have rewritten the pickle documentation, and updated the
marshal, cPickle, and copy_reg documentation.  I believe all
the documentation issues raised here have been fixed.  

Implementation issues will be pushed to Python 2.3, and the
plan is to write a PEP to plan future work.

----------------------------------------------------------------------

Comment By: Barry A. Warsaw (bwarsaw)
Date: 2001-11-13 16:39

Message:
Logged In: YES 
user_id=12800

Assigning to Jeremy so he'll remember to forward me Jim's
email.  Jeremy can ping-pong it to Fred if he wants, and
then reassign to me after forwarding the email.

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2001-11-13 01:18

Message:
Logged In: NO 

The find_global variable sounds encouraging and if it
fixes this problem, that's great.  I'd still feel better
if the eval call goes away.  Is removing it allowed
under the 2.2 feature freeze?  It doesn't affect anything
visible, so no tests would have to change, etc.

Paul


----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2001-11-12 19:38

Message:
Logged In: NO 

It's possible that the eval is safe, but validating that
takes a line by line code review of all the paths that
evaling a string can go through.  It's also brittle in
that maybe someone will change the evaluator (say to
support Perl-like interpolation) in a future Python version
and not remember to change the unpickler.  Something like
that has already happened with the Cookie module.  My
guess is that it happened with the pickle module--the
whole point of that quoted string check is that the 
original pickle implementer didn't trust the input. 
The stuff about unpickling class instances was added
later (maybe by another person) without remembering the
security issue. 

Using eval this way is like storing a vat of cyanide in
your child's bedroom.  Sure, maybe if you check the seals
and locks on the vat carefully enough, you can convince
yourself that your child won't be able to get to the
cyanide.  But wouldn't you feel safer just not having the
vat there at all?  That's basic safety-consciousness.
Security consciousness works the same way.  Try to keep
dangerous ingredients and attackers as far away from each
other as possible.

Paul



----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2001-11-12 13:52

Message:
Logged In: YES 
user_id=31435

Why are people (Paul, Jeremy) concerned about eval'ing 
strings?  cPickle and pickle both check that they're 
properly quoted, and this isn't sh or Perl:  Python has 
no "dynamic" gimmicks buried in string literals.  All 
eval'ing a string literal can do is produce a binary blob 
via interpeting simple escape sequences.  They're like C 
strings this way -- maybe we'll run out of memory, but 
that's it.

I would agree that Python should be refactored internally 
to supply a clean function for changing string literals 
into binary blobs, but that would be for clarity and 
efficiency, not security.

----------------------------------------------------------------------

Comment By: Jim Fulton (dcjim)
Date: 2001-11-12 13:14

Message:
Logged In: YES 
user_id=73023

It sounds like there are some documentation bugs:

- The security ramifications are not discussed, nor are the
remedies.

- The cPickle module isn't documented very well. I submitted
some
   documentation a long time ago that never got incorporated
AFAIK.
   I wish I still had it. :)

- cPickle has a feature for turning off instance support and
for 
   restricting which classes can be unpickled. You can set
the find_global
   attribute on a cPickle.Unpickler. The find_global
attribute can be 
   a function or None.  If it is None, then no instances can
be 
   unpickled. If it is a function, then it should accept a
module and name
   and return the corresponding global. It is responsible
for looking
   up the global and can raise an error to prevent a
global's use.

   See the ZEO storage server implementation for an example
   of using this hook.


  



----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2001-11-12 01:33

Message:
Logged In: NO 

I like marshal and think it's much cleaner than pickle.
I'd like to see its format cleaned up and documented
so it can be a portable transport format.  Even the
recursion issue isn't so compelling--there's only a danger
on marshalling, not unmarshalling, and the application
has control over what it marshals.  So I think the idea
of documenting marshal (bug #467384) should be kept alive.
However, it shouldn't be changed in 2.2.  

If the Cookie class is still really aliased by default to
SmartCookie, that's a bad bug and should definitely be
fixed ASAP.  The Cookie docs clearly say not to use
SmartCookie or SerialCookie.  In fact I think SmartCookie
and SerialCookie should be disabled altogether.  Any
applications using them on the open internet have a security
but and should be fixed immediately.

Here's a possible temporary approach to fixing SmartCookie
and SerialCookie: append a cryptographic message
authentication code (MAC) to each pickled cookie.  That
means the application has some secret string K as a
configuration parameter (it should be 10 or more random
8-bit characters, or 20 or more random hex digits, etc). 
Then a smart cookie would be a pickle p, followed by
SHA(p+K).  The Cookie module would validate the MAC before
calling unpickle, and raise an exception if the MAC wasn't
valid.

The security of this scheme rests on K being kept secret
from attackers, which is generally not an easy thing to
manage, but it's something to think about.

Paul

----------------------------------------------------------------------

Comment By: Jeremy Hylton (jhylton)
Date: 2001-11-10 17:07

Message:
Logged In: YES 
user_id=31392

I don't think we should be doing anything about marshal.  
Maybe we should name in pyclib or something <0.9 wink>.  It 
works fine for .pyc files, but I don't see a reason for it 
to do anymore than is necessary for that purpose.

I think the notion of an unpickler that only handles 
builtin datatypes is the most attractive option you offer, 
but Paul has a good point about eval for strings.  (It 
currently has some hacks that address specific exploits, 
but I doubt they are sufficient.)  I also wonder how hard 
it is to handle builtin types and avoid subclasses of 
builtin types.

If there are any changes to pickle, I think we need to be 
careful about how it is described.  If we claim that an 
unpickler is safe for untrusted pickles, we've made a 
fairly strong claim.  I still think such a design change 
requires a PEP that includes some requirements and use 
cases and a thorough analysis of potential exploits. 


----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2001-11-10 14:42

Message:
Logged In: NO 

See bug #467384 for discussion about marshal.  Besides the
recursion issue, marshal's format is explicitly undocumented
and subject to change--you can't rely on it to interoperate
between two different Python versions, so it's no good as
an RPC serializer.  The format has kludges (e.g. the
representation of long ints) that make it undesirable to
freeze and document it and force future versions to be
backward compatible.

Adding a pickle.loads flag to prevent instance unpickling
isn't perfect but is probably the best alternative on 
your list.  Perhaps the flag can have a value that allows
unpickling the instances by restoring the instance 
attributes rather than calling the initializer.  That's
not always the right way to unpickle an instance (that's
why the unpickler no longer works that way) but it's good
enough a lot of the time.  

There's another issue with pickle/cPickle which is that they
unpickle quoted strings by evaling them.  This is scary.
While I don't see an immediate exploit, I also haven't
examined the 1000's of lines of code I'd need to examine
to convince myself that there's NOT an exploit.  I think
the unpickler should be changed to never call eval but just
parse the string as it needs to.  

Guido seemed to think pickle might have other possible
exploits.  I don't know what he had in mind but before
declaring it safe for untrusted data I think it needs to
be gone over with a fine toothed comb.

Paul


----------------------------------------------------------------------

Comment By: Barry A. Warsaw (bwarsaw)
Date: 2001-11-10 12:37

Message:
Logged In: YES 
user_id=12800

I'm going to agree with Paul that this is a problem needing
fixing, however there are really several issues.

1. Cookie module makes it too easy to code exploits.  Cookie
exports a class, also called Cookie, which is aliased to
SmartCookie, so that a naive program will simply pass cookie
data to Cookie.Cookie() and you're screwed.  So, Cookie
module's defaults should be for more security rather than
less, and Cookie.Cookie should be aliased to SimpleCookie
instead.

2. There is no built-in safe mechanism for de-serializing
data from untrusted sources.  You can't use pickle without
overloading a "magic" method.  You can't use cPickle because
you can't do the overloading trick.  You can't use marshal
because it isn't bulletproof against recursive
datastructures.  So how /do/ you do it?

I think it just may be serious enough to deal with in Python
2.2, and I volunteer to address it (so I'll steal this bug
report).  Without looking at the code, or the level of
effort necessary, I would make the following suggestions:

1. Add to the public interface of pickle and cPickle, a flag
that either disables the unpickling of instances altogether,
or disables calling any code with unpickled data, e.g.
constructors.

2. Fix marshal to be bulletproof against recursive
datastructures.
3. Update the docs for both pickle/cPickle and marshal to
explain how to safely write de-serializers of untrusted strings.

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2001-11-09 04:21

Message:
Logged In: NO 

Well, Guido and Tim agree with you that it's not a pickle
bug.  I still feel it is one, because its docs currently
make you think you can securely load untrusted pickles, and
because it's a natural, non-obscure thing to want to do
(see pyro and the cookie module), but whatever.  If it's
not a code bug then I feel it's a significant functionality
shortcoming in the python library.

Pyro uses pickle to serialize data for RPC calls over the
internet.  A malicious client could make a hostile pickle
take over the server.  The cookie module lets web
applications store user session state in browser cookies.
Its SerialCookie and SmartCookie classes let you put
arbitrary Python objects into the user session, and
serializes them when pickle.  Again, a malicious client
can make a hostile pickle, send it in a cookie header to
the http server, and take over the server when the 
application unpickles the cookie.

The current documentation for the pickle module makes it
very clear to me that the doc writer thought it was safe
to unpickle untrusted cookies.  If pickle wasn't designed
for that, then there was a communication failure between
the designer and the doc writer.

Yes, I'm willing to help with a PEP for fixing this
situation.

Paul


----------------------------------------------------------------------

Comment By: Jeremy Hylton (jhylton)
Date: 2001-11-08 12:37

Message:
Logged In: YES 
user_id=31392

I don't think of the issue you describe as a bug in the
code.  You're suggesting a new feature for pickle.  As far
as I can tell, the original design requirements for pickle
did not include the ability to securely load a pickle from
an untrusted source.

It may be a legitimate feature request, but it's too late to
make it into Python 2.2.  I suggest we look at the design
issues for Python 2.3 and decide if it's a feature we want
to support.  I imagine a PEP may be necessary to lay out the
issues and the solution.  Do you want to have a hand in that
PEP?

I still don't understand what it means that Pyro and cookie
were bit by a bug.  It sounds like they were using pickle in
ways that pickle was not intended to support.  A careful
analysis of how those two applications use pickle would be
helpful for generating requirements.


----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2001-11-07 18:54

Message:
Logged In: NO 

IMO it's a code bug that you can't unpickle strings from
untrusted sources.  Pyro and the cookie module are examples
of programs that got bitten by this bug.  Whether it's
really a bug is a matter of opinion--I had a big email
exchange with Guido and Tim about it, and they felt it
was enough to fix the pickle documentation.

Pickle has the same problem as cPickle, but with pickle
you can subclass the pickler and override the method that
unpickles class objects, and work around the (IMO) bug.
The workaround doesn't help cPickle since cPickle can't
be subclassed.  See bug #467384 for some related discussion.

Paul


----------------------------------------------------------------------

Comment By: Jeremy Hylton (jhylton)
Date: 2001-11-07 17:02

Message:
Logged In: YES 
user_id=31392

What's the code bug?  Your last message has a lot of gloom
and doom <wink>, but I'm not sure what specific problem
you'd like to see fixed.  Are you saying that something
needs to be fixed in cPickle and not in pickle?

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2001-11-07 15:08

Message:
Logged In: NO 

Irmen de Jong points out that the standard cookie module
uses pickling for serial and smart cookies.  The 2.1.1
cookie module docs explicitly say not to use those
classes because of the security hole--that they're provided
for backward compatibility only (but with what?!  Any
server that uses those classes on open ports needs to be
changed right away).

Irmen's library, http://pyro.sourceforge.net, also uses
unpickle insecurely (he's aware of this now and is figuring
out a fix).

IMO this is really a code bug rather than a documentation
bug, and should be fixed in the code rather than the docs.
Documenting the bug rather than fixing it leaves a
deficiency in the Python library: obvious uses of pickling,
like Pyro and the cookie module, can't be implemented
using cPickle and have to resort to a slower Python
deserializer, or use marshal and have possible compatibility
problems between versions (and not be able to serialize
class instances).

Paul



----------------------------------------------------------------------

Comment By: paul rubin (phr)
Date: 2001-10-16 19:06

Message:
Logged In: YES 
user_id=72053

Certainly anyone unserializing potentially malicious data
with pickle, marshal, or anything else, should check the
results before doing anything dangerous with them (like
executing code).  However, unpickle can potentially do
damage before it even returns, by creating loading modules
and creating initialized class instances.  So pickle.loads
should never be used on untrusted strings, except possibly
in a rexec wrapper (as proposed by Tim).  Another
possibility (also by Tim) is to override the load_inst
method of the Pickler class, though I don't think you can
do that for cPickle.

A sample exploit for unpickle can be found at
<http://www.nightsong.com/phr/python/pickletest.py>.
Unpickling the test string runs penguin.__init__ contrary
to the doc's saying no initialization unless there's a
__getinitargs__ method in the class def.

The "exploding penguin" class is artificial, but
applications are vulnerable if there's an unsafe
constructor anywhere in any class of the application or in
the python library (example: the NNTP constructor opens an
IP connection to an arbitrary address, so a malicious
imported string can send a message through your firewall
when you import it).


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=471893&group_id=5470


From noreply@sourceforge.net  Thu Feb  6 20:26:40 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 06 Feb 2003 12:26:40 -0800
Subject: [Python-bugs-list] [ python-Bugs-681902 ] Negative refcount abort
Message-ID: <E18gsbY-0006EC-00@sc8-sf-web4.sourceforge.net>

Bugs item #681902, was opened at 2003-02-06 14:26
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=681902&group_id=5470

Category: Python Interpreter Core
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Skip Montanaro (montanaro)
Assigned to: Nobody/Anonymous (nobody)
Summary: Negative refcount abort

Initial Comment:
I originally thought this was the same as #668433 (thinking there
could only be one negative refcount bug at a time).  Then I thought
perhaps it was the same as #681122, but removing dir() from the
test code seemed to have no effect.  So I conclude there is another
negative refcount bug (or a bug in the underlying extension
module) running around in the CVS HEAD, at least when configured with --with-pydebug.  My platform is Mac OS X, vanilla Unix-style
build.

I will attach a short test script.  Unfortunately, it's a unit test for
the still developing csv module.  I added a cvs tag so you can check
it out and install it easily enough though.

The underlying code is in .../head/nondist/sandbox/csv.  If you
check it out using the tag "refcount", you'll get the version which
barfs.

The symptom is that a reference to a classic class object appears to
be held by the module after the class has been released.  This class
is only manipulated from the Python level.  The underlying _csv
extension module doesn't refer to it as far as I know.

To tickle the problematic behavior, leave the code alone.  It contains
a typo.  The "restfields" keyowrd parameter ought to be "restfield".
Correct the error and the problem goes away.

(I guess the moral of the story is you should never write code with
bugs. ;-)

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=681902&group_id=5470


From noreply@sourceforge.net  Thu Feb  6 21:22:46 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 06 Feb 2003 13:22:46 -0800
Subject: [Python-bugs-list] [ python-Bugs-681934 ] bundlebuilder/zipimport errors
Message-ID: <E18gtTq-00009w-00@sc8-sf-web4.sourceforge.net>

Bugs item #681934, was opened at 2003-02-06 13:22
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=681934&group_id=5470

Category: Macintosh
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Robin Dunn (robind)
Assigned to: Jack Jansen (jackjansen)
Summary: bundlebuilder/zipimport errors

Initial Comment:
While testing the bundlebuilder patch that adds shared
library support I decided to try it with the CVS
version of bundlebuilder.py and was excited to see it
using the new zipimport feature.  However there is a
problem.

In the wxPython package there is a module named
__version__.py that is imported from __init__.py.  Both
correctly have .pyc files placed in Modules.zip under a
wxPython directory, but I get this error when trying to
run the app bundle:

Traceback (most recent call last):
  File
"/usr/local/projects/wx/wxPython/samples/doodle/SuperDoodle.app/Contents/Resources/superdoodle.py",
line 13, in ?
    from wxPython.wx import *
  File
"/Library/Frameworks/Python.framework/Versions/2.3/lib/python2.3/site-packages/wxPython/__init__.py",
line 14, in ?
    import __version__
ImportError: No module named __version__

(I assume that the pathnames are as they are (not
reflecting the Modules.zip path) because that is where
the .pyc's are compiled from ???)

If I force USE_ZIPIMPORT=0 in bundlebuilder.py then the
bundle works just fine.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=681934&group_id=5470


From noreply@sourceforge.net  Thu Feb  6 21:37:16 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 06 Feb 2003 13:37:16 -0800
Subject: [Python-bugs-list] [ python-Bugs-680789 ] repr() of large array objects takes quadratic time
Message-ID: <E18gths-00055w-00@sc8-sf-web1.sourceforge.net>

Bugs item #680789, was opened at 2003-02-05 12:00
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=680789&group_id=5470

Category: Python Library
Group: Python 2.2
Status: Open
Resolution: None
Priority: 5
Submitted By: Jurjen N.E. Bos (jneb)
>Assigned to: Tim Peters (tim_one)
>Summary: repr() of large array objects takes quadratic time

Initial Comment:
This is a bug and a partial patch.

If I debug a program that contains a ridiculously large array (8M entries in my case), the debugger takes forever.
It happens in Mac OS X, Python 2.2, but I found the bug in is the repr module, so it is probably universal.
The thing is, that after the fix below, it still doesn't work!
Did I miss something trivial (like repr is builtin, or something like that?). Would someone with Mac OS X experience help out here, please (Jack?).

Here's the diff to make repr.repr work with large arrays:
13a14
>       self.maxarray = 5
50a52,62
>     def repr_array(self, x, level):
>         n = len(x)
>       header = "array('"+x.typecode+"', ["
>         if n == 0: return header+"])"
>         if level <= 0: return header+"...])"
>         s = ''
>         for i in range(min(n, self.maxarray)):
>             if s: s = s + ', '
>             s = s + self.repr1(x[i], level-1)
>         if n > self.maxarray: s = s + ', ...'
>         return header + s + "])"

----------------------------------------------------------------------

>Comment By: Jack Jansen (jackjansen)
Date: 2003-02-06 22:37

Message:
Logged In: YES 
user_id=45365

Okay, so the real bug is that tp_repr of array objects takes quadratic time.

I'm changing the summary of this report then, and assigning back to you (Tim), on the basis that you did more checkins on arraymodule than I did. Feel free to pass the potato on:-)

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2003-02-06 00:08

Message:
Logged In: YES 
user_id=31435

pdb does import repr.py, but probably doesn't use it in 
whatever way Jurjen is using to display his big array.

WRT that, note that Jurjen is using array.array objects, not 
lists.  The internal array.array tp_repr slot is quadratic-time in 
the size of the array, while list's tp_repr is linear time.

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-02-05 23:40

Message:
Logged In: YES 
user_id=45365

The fix is fine (it works for me the same way as for Tim), but I think we're shooting past the problem here.

First, pdb doesn't use repr.repr(), it uses the normal builtin repr().

Second, I don't see any sluggishness in pdb with large arrays. I tried
debugging
def foo():
    a = range(8000000)
and there was no problem. Allocating the object took a bit of time, yes, and if you actually try to print it you'll stare at about 800K lines filled with digits scrolling over your screen, but that is to be expected.

Could it be your sluggishness is coming from something else? For example, MacOSX starts behaving *very* badly if your root disk is full, because then it can't allocate swap space, and due to its optimistic behaviour it comes to a grinding halt.

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2003-02-05 19:36

Message:
Logged In: YES 
user_id=31435

Nice to see you, Jurgen!  I checked this into current CVS, 
and it works fine for me in isolation:

>>> len(a)
11055060
>>> repr.repr(a)
"array('i', [0, 1, 2, 3, 4, ...])"
>>>

That goes in an eyeblink.  So more detail is needed about 
what "it still doesn't work!" means.  Assigned to Jack, and he 
can use current CVS to try it.

Lib/repr.py; new revision: 1.15
Lib/test/test_repr.py; new revision: 1.16
Misc/NEWS; new revision: 1.642

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=680789&group_id=5470


From noreply@sourceforge.net  Thu Feb  6 21:40:02 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 06 Feb 2003 13:40:02 -0800
Subject: [Python-bugs-list] [ python-Bugs-681934 ] bundlebuilder/zipimport errors
Message-ID: <E18gtkY-0005I3-00@sc8-sf-web1.sourceforge.net>

Bugs item #681934, was opened at 2003-02-06 22:22
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=681934&group_id=5470

Category: Macintosh
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Robin Dunn (robind)
>Assigned to: Just van Rossum (jvr)
Summary: bundlebuilder/zipimport errors

Initial Comment:
While testing the bundlebuilder patch that adds shared
library support I decided to try it with the CVS
version of bundlebuilder.py and was excited to see it
using the new zipimport feature.  However there is a
problem.

In the wxPython package there is a module named
__version__.py that is imported from __init__.py.  Both
correctly have .pyc files placed in Modules.zip under a
wxPython directory, but I get this error when trying to
run the app bundle:

Traceback (most recent call last):
  File
"/usr/local/projects/wx/wxPython/samples/doodle/SuperDoodle.app/Contents/Resources/superdoodle.py",
line 13, in ?
    from wxPython.wx import *
  File
"/Library/Frameworks/Python.framework/Versions/2.3/lib/python2.3/site-packages/wxPython/__init__.py",
line 14, in ?
    import __version__
ImportError: No module named __version__

(I assume that the pathnames are as they are (not
reflecting the Modules.zip path) because that is where
the .pyc's are compiled from ???)

If I force USE_ZIPIMPORT=0 in bundlebuilder.py then the
bundle works just fine.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=681934&group_id=5470


From noreply@sourceforge.net  Thu Feb  6 22:13:15 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 06 Feb 2003 14:13:15 -0800
Subject: [Python-bugs-list] [ python-Bugs-681934 ] bundlebuilder/zipimport errors
Message-ID: <E18guGh-000728-00@sc8-sf-web1.sourceforge.net>

Bugs item #681934, was opened at 2003-02-06 22:22
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=681934&group_id=5470

Category: Macintosh
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Robin Dunn (robind)
Assigned to: Just van Rossum (jvr)
Summary: bundlebuilder/zipimport errors

Initial Comment:
While testing the bundlebuilder patch that adds shared
library support I decided to try it with the CVS
version of bundlebuilder.py and was excited to see it
using the new zipimport feature.  However there is a
problem.

In the wxPython package there is a module named
__version__.py that is imported from __init__.py.  Both
correctly have .pyc files placed in Modules.zip under a
wxPython directory, but I get this error when trying to
run the app bundle:

Traceback (most recent call last):
  File
"/usr/local/projects/wx/wxPython/samples/doodle/SuperDoodle.app/Contents/Resources/superdoodle.py",
line 13, in ?
    from wxPython.wx import *
  File
"/Library/Frameworks/Python.framework/Versions/2.3/lib/python2.3/site-packages/wxPython/__init__.py",
line 14, in ?
    import __version__
ImportError: No module named __version__

(I assume that the pathnames are as they are (not
reflecting the Modules.zip path) because that is where
the .pyc's are compiled from ???)

If I force USE_ZIPIMPORT=0 in bundlebuilder.py then the
bundle works just fine.

----------------------------------------------------------------------

>Comment By: Just van Rossum (jvr)
Date: 2003-02-06 23:13

Message:
Logged In: YES 
user_id=92689

You don't happen to run 2.3a1 are you? It's zipimport support is unfortunately severly broken :-( If not, I'll investigate...

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=681934&group_id=5470


From noreply@sourceforge.net  Thu Feb  6 22:17:15 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 06 Feb 2003 14:17:15 -0800
Subject: [Python-bugs-list] [ python-Bugs-681960 ] Source encoding rules are extreme.
Message-ID: <E18guKZ-00075s-00@sc8-sf-web1.sourceforge.net>

Bugs item #681960, was opened at 2003-02-06 22:17
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=681960&group_id=5470

Category: Unicode
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Kirill Simonov (kirill_simonov)
Assigned to: M.-A. Lemburg (lemburg)
Summary: Source encoding rules are extreme.

Initial Comment:
According to the PEP 0263, a source code that contains
non-ASCII
characters (ord(ch)>127) and does not define an
encoding causes
DeprecationWarning. In the future, such code will cause
SyntaxError.

While I believe that the idea of defining source code
encoding is very
useful, I think that the current solution is
unnecessary extreme.

It is very unfriendly for beginners. Imagine a student that
types her first script:

name = raw_input("What's your name? ")   # russian
here, of course
print "Hi %s!" % name

Do not even try to convince me that she must define an
encoding
here. That feature would break any possibility to use
Python in schools.

Actually the source code encoding only affects Unicode
literals.
The above script works the same way with any defined
encoding,
so the warning for this code is unnecessary.

As a solution, I propose to issue DeprecationWarning
(or SyntaxError)
only when a non-ASCII character is contained in a
Unicode literal.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=681960&group_id=5470


From noreply@sourceforge.net  Thu Feb  6 22:45:22 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 06 Feb 2003 14:45:22 -0800
Subject: [Python-bugs-list] [ python-Bugs-681960 ] Source encoding rules are extreme.
Message-ID: <E18gulm-0006uZ-00@sc8-sf-web2.sourceforge.net>

Bugs item #681960, was opened at 2003-02-06 23:17
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=681960&group_id=5470

Category: Unicode
Group: Python 2.3
Status: Open
Resolution: None
>Priority: 3
Submitted By: Kirill Simonov (kirill_simonov)
Assigned to: M.-A. Lemburg (lemburg)
Summary: Source encoding rules are extreme.

Initial Comment:
According to the PEP 0263, a source code that contains
non-ASCII
characters (ord(ch)>127) and does not define an
encoding causes
DeprecationWarning. In the future, such code will cause
SyntaxError.

While I believe that the idea of defining source code
encoding is very
useful, I think that the current solution is
unnecessary extreme.

It is very unfriendly for beginners. Imagine a student that
types her first script:

name = raw_input("What's your name? ")   # russian
here, of course
print "Hi %s!" % name

Do not even try to convince me that she must define an
encoding
here. That feature would break any possibility to use
Python in schools.

Actually the source code encoding only affects Unicode
literals.
The above script works the same way with any defined
encoding,
so the warning for this code is unnecessary.

As a solution, I propose to issue DeprecationWarning
(or SyntaxError)
only when a non-ASCII character is contained in a
Unicode literal.


----------------------------------------------------------------------

>Comment By: M.-A. Lemburg (lemburg)
Date: 2003-02-06 23:45

Message:
Logged In: YES 
user_id=38388

Sorry, but the implementation we chose decodes the complete
file,
not only the Unicode literals, so if you want to use a specific 
encoding in the source code, you have to be explicit about it.

Python's source code was originally never meant to contain
non-ASCII characters. The PEP implementation now officially
allows this provided that you use an encoding marker, e.g.

"""
# -*- coding: windows-1251 -*-
name = raw_input("   ? ")
print " %s" % name
"""

Note that this is also needed in order to support UTF-16
file formats which use two bytes per character. Python
will automatically detect these files, so if you really don't
like the coding marker, simply write the file using a UTF-16
aware editor which prepends a UTF-16 BOM mark to the
file.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=681960&group_id=5470


From noreply@sourceforge.net  Thu Feb  6 23:06:37 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 06 Feb 2003 15:06:37 -0800
Subject: [Python-bugs-list] [ python-Bugs-681902 ] Negative refcount abort
Message-ID: <E18gv6L-00056t-00@sc8-sf-web4.sourceforge.net>

Bugs item #681902, was opened at 2003-02-06 15:26
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=681902&group_id=5470

Category: Python Interpreter Core
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Skip Montanaro (montanaro)
Assigned to: Nobody/Anonymous (nobody)
Summary: Negative refcount abort

Initial Comment:
I originally thought this was the same as #668433 (thinking there
could only be one negative refcount bug at a time).  Then I thought
perhaps it was the same as #681122, but removing dir() from the
test code seemed to have no effect.  So I conclude there is another
negative refcount bug (or a bug in the underlying extension
module) running around in the CVS HEAD, at least when configured with --with-pydebug.  My platform is Mac OS X, vanilla Unix-style
build.

I will attach a short test script.  Unfortunately, it's a unit test for
the still developing csv module.  I added a cvs tag so you can check
it out and install it easily enough though.

The underlying code is in .../head/nondist/sandbox/csv.  If you
check it out using the tag "refcount", you'll get the version which
barfs.

The symptom is that a reference to a classic class object appears to
be held by the module after the class has been released.  This class
is only manipulated from the Python level.  The underlying _csv
extension module doesn't refer to it as far as I know.

To tickle the problematic behavior, leave the code alone.  It contains
a typo.  The "restfields" keyowrd parameter ought to be "restfield".
Correct the error and the problem goes away.

(I guess the moral of the story is you should never write code with
bugs. ;-)

----------------------------------------------------------------------

>Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-06 18:06

Message:
Logged In: YES 
user_id=33168

My current guess is a problem in csv.  If you repeat this line:

  csv.reader("1", dialect="excel", fieldnames=["f1"],
restfields="_rest")

The ref count goes backwards  If I repeat this one line, the
interpreter crashes the third time.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=681902&group_id=5470


From noreply@sourceforge.net  Thu Feb  6 23:28:22 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 06 Feb 2003 15:28:22 -0800
Subject: [Python-bugs-list] [ python-Bugs-681934 ] bundlebuilder/zipimport errors
Message-ID: <E18gvRO-0001m6-00@sc8-sf-web3.sourceforge.net>

Bugs item #681934, was opened at 2003-02-06 13:22
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=681934&group_id=5470

Category: Macintosh
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Robin Dunn (robind)
Assigned to: Just van Rossum (jvr)
Summary: bundlebuilder/zipimport errors

Initial Comment:
While testing the bundlebuilder patch that adds shared
library support I decided to try it with the CVS
version of bundlebuilder.py and was excited to see it
using the new zipimport feature.  However there is a
problem.

In the wxPython package there is a module named
__version__.py that is imported from __init__.py.  Both
correctly have .pyc files placed in Modules.zip under a
wxPython directory, but I get this error when trying to
run the app bundle:

Traceback (most recent call last):
  File
"/usr/local/projects/wx/wxPython/samples/doodle/SuperDoodle.app/Contents/Resources/superdoodle.py",
line 13, in ?
    from wxPython.wx import *
  File
"/Library/Frameworks/Python.framework/Versions/2.3/lib/python2.3/site-packages/wxPython/__init__.py",
line 14, in ?
    import __version__
ImportError: No module named __version__

(I assume that the pathnames are as they are (not
reflecting the Modules.zip path) because that is where
the .pyc's are compiled from ???)

If I force USE_ZIPIMPORT=0 in bundlebuilder.py then the
bundle works just fine.

----------------------------------------------------------------------

>Comment By: Robin Dunn (robind)
Date: 2003-02-06 15:28

Message:
Logged In: YES 
user_id=53955

Yes, I do.  I wondered if it might be a problem with
zipimport rather than bundlebuilder, but didn't seen any
bugs listed that seemed the same. 

----------------------------------------------------------------------

Comment By: Just van Rossum (jvr)
Date: 2003-02-06 14:13

Message:
Logged In: YES 
user_id=92689

You don't happen to run 2.3a1 are you? It's zipimport support is unfortunately severly broken :-( If not, I'll investigate...

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=681934&group_id=5470


From noreply@sourceforge.net  Thu Feb  6 23:28:44 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 06 Feb 2003 15:28:44 -0800
Subject: [Python-bugs-list] [ python-Bugs-681960 ] Source encoding rules are extreme.
Message-ID: <E18gvRk-0008S2-00@sc8-sf-web2.sourceforge.net>

Bugs item #681960, was opened at 2003-02-06 22:17
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=681960&group_id=5470

Category: Unicode
Group: Python 2.3
Status: Open
Resolution: None
Priority: 3
Submitted By: Kirill Simonov (kirill_simonov)
Assigned to: M.-A. Lemburg (lemburg)
Summary: Source encoding rules are extreme.

Initial Comment:
According to the PEP 0263, a source code that contains
non-ASCII
characters (ord(ch)>127) and does not define an
encoding causes
DeprecationWarning. In the future, such code will cause
SyntaxError.

While I believe that the idea of defining source code
encoding is very
useful, I think that the current solution is
unnecessary extreme.

It is very unfriendly for beginners. Imagine a student that
types her first script:

name = raw_input("What's your name? ")   # russian
here, of course
print "Hi %s!" % name

Do not even try to convince me that she must define an
encoding
here. That feature would break any possibility to use
Python in schools.

Actually the source code encoding only affects Unicode
literals.
The above script works the same way with any defined
encoding,
so the warning for this code is unnecessary.

As a solution, I propose to issue DeprecationWarning
(or SyntaxError)
only when a non-ASCII character is contained in a
Unicode literal.


----------------------------------------------------------------------

>Comment By: Kirill Simonov (kirill_simonov)
Date: 2003-02-06 23:28

Message:
Logged In: YES 
user_id=36553

Hello,

Yes, I understand that the encoding is for the whole source
file.

But

1. The current implementation already assumes that one uses
ASCII-
compatible encoding. So we can make a step further and do
not use any
encoding while reading a source file. And then we'll
translate u"..." using
'ascii' encoding.

2. How do you want to support UTF-16 encoding? This will
completely
break ordinary string literals! "aa" is a source code would
become "a\x00a\x00" after compilation. Or do I miss something?

3. Do not forget that your change breaks billions of scripts
that use
non-ASCII characters even in comments!

4. I can write a patch. I would be forced to do this anyway.



----------------------------------------------------------------------

Comment By: M.-A. Lemburg (lemburg)
Date: 2003-02-06 22:45

Message:
Logged In: YES 
user_id=38388

Sorry, but the implementation we chose decodes the complete
file,
not only the Unicode literals, so if you want to use a specific 
encoding in the source code, you have to be explicit about it.

Python's source code was originally never meant to contain
non-ASCII characters. The PEP implementation now officially
allows this provided that you use an encoding marker, e.g.

"""
# -*- coding: windows-1251 -*-
name = raw_input("   ? ")
print " %s" % name
"""

Note that this is also needed in order to support UTF-16
file formats which use two bytes per character. Python
will automatically detect these files, so if you really don't
like the coding marker, simply write the file using a UTF-16
aware editor which prepends a UTF-16 BOM mark to the
file.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=681960&group_id=5470


From noreply@sourceforge.net  Fri Feb  7 00:34:30 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 06 Feb 2003 16:34:30 -0800
Subject: [Python-bugs-list] [ python-Bugs-681902 ] Negative refcount abort
Message-ID: <E18gwTO-0002Zj-00@sc8-sf-web2.sourceforge.net>

Bugs item #681902, was opened at 2003-02-06 15:26
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=681902&group_id=5470

Category: Python Interpreter Core
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Skip Montanaro (montanaro)
Assigned to: Nobody/Anonymous (nobody)
Summary: Negative refcount abort

Initial Comment:
I originally thought this was the same as #668433 (thinking there
could only be one negative refcount bug at a time).  Then I thought
perhaps it was the same as #681122, but removing dir() from the
test code seemed to have no effect.  So I conclude there is another
negative refcount bug (or a bug in the underlying extension
module) running around in the CVS HEAD, at least when configured with --with-pydebug.  My platform is Mac OS X, vanilla Unix-style
build.

I will attach a short test script.  Unfortunately, it's a unit test for
the still developing csv module.  I added a cvs tag so you can check
it out and install it easily enough though.

The underlying code is in .../head/nondist/sandbox/csv.  If you
check it out using the tag "refcount", you'll get the version which
barfs.

The symptom is that a reference to a classic class object appears to
be held by the module after the class has been released.  This class
is only manipulated from the Python level.  The underlying _csv
extension module doesn't refer to it as far as I know.

To tickle the problematic behavior, leave the code alone.  It contains
a typo.  The "restfields" keyowrd parameter ought to be "restfield".
Correct the error and the problem goes away.

(I guess the moral of the story is you should never write code with
bugs. ;-)

----------------------------------------------------------------------

>Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-06 19:34

Message:
Logged In: YES 
user_id=33168

This crashes (the ref count goes down inappropriately):
  _csv.parser(lineterminator='\r\n', restfields='_rest')

This is ok:
  _csv.parser(restfields='_rest')

Attaching output from valgrind.  It doesn't help me, but may
help someone else.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-06 18:06

Message:
Logged In: YES 
user_id=33168

My current guess is a problem in csv.  If you repeat this line:

  csv.reader("1", dialect="excel", fieldnames=["f1"],
restfields="_rest")

The ref count goes backwards  If I repeat this one line, the
interpreter crashes the third time.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=681902&group_id=5470


From noreply@sourceforge.net  Fri Feb  7 00:51:56 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 06 Feb 2003 16:51:56 -0800
Subject: [Python-bugs-list] [ python-Bugs-681902 ] Negative refcount abort
Message-ID: <E18gwkG-00043Z-00@sc8-sf-web1.sourceforge.net>

Bugs item #681902, was opened at 2003-02-06 14:26
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=681902&group_id=5470

Category: Python Interpreter Core
Group: Python 2.3
>Status: Closed
>Resolution: Invalid
Priority: 5
Submitted By: Skip Montanaro (montanaro)
Assigned to: Nobody/Anonymous (nobody)
Summary: Negative refcount abort

Initial Comment:
I originally thought this was the same as #668433 (thinking there
could only be one negative refcount bug at a time).  Then I thought
perhaps it was the same as #681122, but removing dir() from the
test code seemed to have no effect.  So I conclude there is another
negative refcount bug (or a bug in the underlying extension
module) running around in the CVS HEAD, at least when configured with --with-pydebug.  My platform is Mac OS X, vanilla Unix-style
build.

I will attach a short test script.  Unfortunately, it's a unit test for
the still developing csv module.  I added a cvs tag so you can check
it out and install it easily enough though.

The underlying code is in .../head/nondist/sandbox/csv.  If you
check it out using the tag "refcount", you'll get the version which
barfs.

The symptom is that a reference to a classic class object appears to
be held by the module after the class has been released.  This class
is only manipulated from the Python level.  The underlying _csv
extension module doesn't refer to it as far as I know.

To tickle the problematic behavior, leave the code alone.  It contains
a typo.  The "restfields" keyowrd parameter ought to be "restfield".
Correct the error and the problem goes away.

(I guess the moral of the story is you should never write code with
bugs. ;-)

----------------------------------------------------------------------

>Comment By: Skip Montanaro (montanaro)
Date: 2003-02-06 18:51

Message:
Logged In: YES 
user_id=44345

Sorry to have bothered you needlessly.  Andrew McNamara has done a fairly hefty rewrite of the underlying C module and says this problem doesn't
occur there, so the bug appeared to indeed be in the _csv module.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-06 18:34

Message:
Logged In: YES 
user_id=33168

This crashes (the ref count goes down inappropriately):
  _csv.parser(lineterminator='\r\n', restfields='_rest')

This is ok:
  _csv.parser(restfields='_rest')

Attaching output from valgrind.  It doesn't help me, but may
help someone else.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-06 17:06

Message:
Logged In: YES 
user_id=33168

My current guess is a problem in csv.  If you repeat this line:

  csv.reader("1", dialect="excel", fieldnames=["f1"],
restfields="_rest")

The ref count goes backwards  If I repeat this one line, the
interpreter crashes the third time.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=681902&group_id=5470


From noreply@sourceforge.net  Fri Feb  7 01:12:38 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 06 Feb 2003 17:12:38 -0800
Subject: [Python-bugs-list] [ python-Bugs-679520 ] long(3.1415) gives zero on Solaris 8
Message-ID: <E18gx4I-00040z-00@sc8-sf-web2.sourceforge.net>

Bugs item #679520, was opened at 2003-02-03 09:22
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=679520&group_id=5470

Category: Python Interpreter Core
Group: Platform-specific
Status: Open
>Resolution: Works For Me
Priority: 5
Submitted By: Mark Dickinson (marketdickinson)
Assigned to: Nobody/Anonymous (nobody)
Summary: long(3.1415) gives zero on Solaris 8

Initial Comment:
On Solaris 8, running Python 2.2.2, the built-in 
function long() gave me the following peculiar results
when applied to a floating-point argument.

(Unfortunately I don't have administrative access to
this machine, so couldn't test to see whether Python
2.3a has the same problems.)

s0100% uname -a
SunOS s0100.math.lsa.umich.edu 5.8 Generic_108528-14
sun4u sparc SUNW,Sun-Blade-1000
s0100% python
Python 2.2.2 (#22, Dec 23 2002, 12:02:55) 
[GCC 3.0.3] on sunos5
Type "help", "copyright", "credits" or "license" for
more information.
>>> long(3.1415)
0L
>>> _ == 0L
0
>>> long(6e20)
00000000000000000L
>>> _ - 1
37778931862957161709567L
>>> int(3.1415)
3


----------------------------------------------------------------------

>Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-06 20:12

Message:
Logged In: YES 
user_id=33168

Mark, can you verify if this is a compiler problem?  I also
don't have this problem on a sun?

----------------------------------------------------------------------

Comment By: Inyeol Lee (inyeol)
Date: 2003-02-03 14:41

Message:
Logged In: YES 
user_id=595280

My machine has similar configuration, but works fine;

$ uname -a
SunOS abbey 5.8 Generic_108528-18 sun4u sparc
SUNW,Sun-Blade-1000

$ python22
Python 2.2.2 (#1, Oct 15 2002, 14:20:16)
[GCC 2.95.3 20010315 (release)] on sunos5
>>> long(3.1415)
3L
>>> long(6e20)
600000000000000000000L
>>> _ - 1
599999999999999999999L

I think it's gcc/glibc optimization issue.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=679520&group_id=5470


From noreply@sourceforge.net  Fri Feb  7 01:16:00 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 06 Feb 2003 17:16:00 -0800
Subject: [Python-bugs-list] [ python-Bugs-680797 ] expat doesn't build on MacOS9
Message-ID: <E18gx7Y-0005Ey-00@sc8-sf-web3.sourceforge.net>

Bugs item #680797, was opened at 2003-02-05 06:11
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=680797&group_id=5470

Category: Extension Modules
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Jack Jansen (jackjansen)
>Assigned to: Fred L. Drake, Jr. (fdrake)
Summary: expat doesn't build on MacOS9

Initial Comment:
Expat doesn't build anymore on Mac OS 9. The problem is that the new expat.h has a couple of functions returning an "enum XML_Status", but that enum is declared only further down in expat.h and the Metrowerks compiler chokes on this.

It's easy enough to fix (either moving the routines down or moving the enum declaration up), but as I seem to recall that this file is imported from the expat distribution I'm not sure whether that's a good idea, or something else needs to be done.

----------------------------------------------------------------------

>Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-06 20:16

Message:
Logged In: YES 
user_id=33168

Fred can you look at this since Martin is on vacation?  I'm
also not sure if this is part of expat or pyexpat (or how to
tell).

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=680797&group_id=5470


From noreply@sourceforge.net  Fri Feb  7 01:17:48 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 06 Feb 2003 17:17:48 -0800
Subject: [Python-bugs-list] [ python-Bugs-678518 ] Another parsermodule validation error
Message-ID: <E18gx9I-0001Ll-00@sc8-sf-web4.sourceforge.net>

Bugs item #678518, was opened at 2003-01-31 21:23
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=678518&group_id=5470

Category: Parser/Compiler
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: logistix (logistix)
Assigned to: Michael Hudson (mwh)
Summary: Another parsermodule validation error

Initial Comment:
The following code generates an validation error.  It's an 
invalid ast tree where the global_stmt doesn't have the 
keyword 'global' as a child.

I tracked down the fix to validate_global_stmt() this time.

Python 2.3a1 (#38, Dec 31 2002, 17:53:59) [MSC 
v.1200 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more 
information.
>>> import symbol,token,parser
>>> ast = [symbol.file_input,
...  [symbol.stmt, [symbol.simple_stmt, 
[symbol.small_stmt, [symbol.global_stmt,
 [1, 'foo']]], [token.NEWLINE, '']]],
...  [token.NEWLINE, ''],
...  [token.ENDMARKER, '']]
>>> a = parser.sequence2ast(ast)
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
parser.ParserError: VALIDATION FAILURE: report this 
to the maintainer!
>>>

----------------------------------------------------------------------

>Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-06 20:17

Message:
Logged In: YES 
user_id=33168

Michael were these patches ok to check in?

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2003-02-03 06:00

Message:
Logged In: YES 
user_id=6656

Thanks for doing this.  I'll probably check these in after
I've finished crunching my way through monday morning email...

----------------------------------------------------------------------

Comment By: logistix (logistix)
Date: 2003-02-01 16:42

Message:
Logged In: YES 
user_id=699438

And I found a few more by doing a roundtrip 
ast2Tuple/tuple2ast on all files in pythonpath.  These actually 
affect valid AST's, so they're probably a bigger deal.

Valid assert statements were failing.  The patch was trivial, 
so its attached.

Also, files that took advantage of encoding_decl (from PEP 
263) were failing.  I couldn't tell if PEP 263 was offically 
closed, and didn't know what the offical grammar was, so I 
didn't do anything there.

I'll also attach the test script that turned up these errors.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=678518&group_id=5470


From noreply@sourceforge.net  Fri Feb  7 01:23:59 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 06 Feb 2003 17:23:59 -0800
Subject: [Python-bugs-list] [ python-Bugs-681902 ] Negative refcount abort
Message-ID: <E18gxFH-0004Qh-00@sc8-sf-web2.sourceforge.net>

Bugs item #681902, was opened at 2003-02-06 15:26
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=681902&group_id=5470

Category: Python Interpreter Core
Group: Python 2.3
Status: Closed
Resolution: Invalid
Priority: 5
Submitted By: Skip Montanaro (montanaro)
Assigned to: Nobody/Anonymous (nobody)
Summary: Negative refcount abort

Initial Comment:
I originally thought this was the same as #668433 (thinking there
could only be one negative refcount bug at a time).  Then I thought
perhaps it was the same as #681122, but removing dir() from the
test code seemed to have no effect.  So I conclude there is another
negative refcount bug (or a bug in the underlying extension
module) running around in the CVS HEAD, at least when configured with --with-pydebug.  My platform is Mac OS X, vanilla Unix-style
build.

I will attach a short test script.  Unfortunately, it's a unit test for
the still developing csv module.  I added a cvs tag so you can check
it out and install it easily enough though.

The underlying code is in .../head/nondist/sandbox/csv.  If you
check it out using the tag "refcount", you'll get the version which
barfs.

The symptom is that a reference to a classic class object appears to
be held by the module after the class has been released.  This class
is only manipulated from the Python level.  The underlying _csv
extension module doesn't refer to it as far as I know.

To tickle the problematic behavior, leave the code alone.  It contains
a typo.  The "restfields" keyowrd parameter ought to be "restfield".
Correct the error and the problem goes away.

(I guess the moral of the story is you should never write code with
bugs. ;-)

----------------------------------------------------------------------

>Comment By: Tim Peters (tim_one)
Date: 2003-02-06 20:23

Message:
Logged In: YES 
user_id=31435

I think csv_parser is too clevar.  If the PyArg_ParseTuple 
call fails, it may have already stored a borrowed reference 
into self->lineterminator, and then it's madness to decref 
that.in Parser_dealloc().

"The usual way" to allocate a new object is not to 
materialize self until *after* PyArg_ParseTuple succeeds.  
Then nothing delicate needs to be done to clean up, since 
nothing was done at all yet <wink>.

Good evidence:  adding the pure hack

Py_XINCREF(self->lineterminator);

before 

Py_DECREF(self);

stops the negative refcount errors in Neal's example.


----------------------------------------------------------------------

Comment By: Skip Montanaro (montanaro)
Date: 2003-02-06 19:51

Message:
Logged In: YES 
user_id=44345

Sorry to have bothered you needlessly.  Andrew McNamara has done a fairly hefty rewrite of the underlying C module and says this problem doesn't
occur there, so the bug appeared to indeed be in the _csv module.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-06 19:34

Message:
Logged In: YES 
user_id=33168

This crashes (the ref count goes down inappropriately):
  _csv.parser(lineterminator='\r\n', restfields='_rest')

This is ok:
  _csv.parser(restfields='_rest')

Attaching output from valgrind.  It doesn't help me, but may
help someone else.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-06 18:06

Message:
Logged In: YES 
user_id=33168

My current guess is a problem in csv.  If you repeat this line:

  csv.reader("1", dialect="excel", fieldnames=["f1"],
restfields="_rest")

The ref count goes backwards  If I repeat this one line, the
interpreter crashes the third time.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=681902&group_id=5470


From noreply@sourceforge.net  Fri Feb  7 01:28:25 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 06 Feb 2003 17:28:25 -0800
Subject: [Python-bugs-list] [ python-Bugs-675259 ] os.environ leaks under FreeBSD and Mac OS X
Message-ID: <E18gxJZ-0005fn-00@sc8-sf-web3.sourceforge.net>

Bugs item #675259, was opened at 2003-01-27 00:39
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=675259&group_id=5470

Category: Documentation
Group: Platform-specific
Status: Open
Resolution: None
Priority: 5
Submitted By: Stuart Bishop (zenzen)
Assigned to: Nobody/Anonymous (nobody)
Summary: os.environ leaks under FreeBSD and Mac OS X

Initial Comment:
If this sort if thing is documented anywhere, the putenv(3) method leaks under FreeBSD and Mac OS X (Mac OS X documents it as a bug they inherited from the FreeBSD codebase in the man page).

So don't set environment variables in a loop.

----------------------------------------------------------------------

>Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-06 20:28

Message:
Logged In: YES 
user_id=33168

Stuart have you tested this or are you just assuming that
putenv leaks in python?  I tested this on FreeBSD and the
process did not leak.  Note that python maintains a
dictionary of all the values set with putenv, if they are
set again, the memory will be cleared.  See posix_putenv()
in posixmodule.c and the posix_putenv_garbage variable.

----------------------------------------------------------------------

Comment By: Stuart Bishop (zenzen)
Date: 2003-01-28 19:15

Message:
Logged In: YES 
user_id=46639

Relevant URL's:

http://www.freebsd.org/cgi/query-pr.cgi?pr=misc/19406
http://www.freebsd.org/cgi/query-pr.cgi?pr=10341
http://www.freebsd.org/cgi/query-pr.cgi?pr=5604

An alternative may be to bypass the putenv(3) method on these platforms
and manipulate environ manually in which case this is not a documentation bug but a feature request.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=675259&group_id=5470


From noreply@sourceforge.net  Fri Feb  7 01:33:44 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 06 Feb 2003 17:33:44 -0800
Subject: [Python-bugs-list] [ python-Bugs-666958 ] repr.repr not always safe
Message-ID: <E18gxOi-0001nk-00@sc8-sf-web4.sourceforge.net>

Bugs item #666958, was opened at 2003-01-12 23:17
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=666958&group_id=5470

Category: Python Library
Group: Python 2.2
Status: Open
Resolution: None
Priority: 5
Submitted By: Bob Ippolito (etrepum)
Assigned to: Nobody/Anonymous (nobody)
Summary: repr.repr not always safe

Initial Comment:
>>> class A:
...   def __repr__(self):
...     raise Exception
...   def someMethod(self):
...     pass
... 
>>> A()
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "<stdin>", line 3, in __repr__
Exception
>>> repr.repr(A())
'<A instance at 237360>'
>>> repr.repr(A().someMethod)
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "/usr/lib/python2.2/repr.py", line 15, in repr
    return self.repr1(x, self.maxlevel)
  File "/usr/lib/python2.2/repr.py", line 24, in repr1
    s = `x`
  File "<stdin>", line

----------------------------------------------------------------------

>Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-06 20:33

Message:
Logged In: YES 
user_id=33168

Bob, I'm not sure I understand what you want (ie, what
behaviour you desire).  If you could provide a patch to the
repr module, that could help me.

----------------------------------------------------------------------

Comment By: Bob Ippolito (etrepum)
Date: 2003-02-03 15:27

Message:
Logged In: YES 
user_id=139309

Yes, of course repr.repr is for debugging use.. and it
*really* sucks if your debugger crashes!  I'm sure that's
why it's inconsistent with __builtins__.repr, the bug I'm
reporting is that it's inconsistent with itself.

Going back to the example of a list again.  If you're
debugging a list that has one or two items that raise an
exception, it's infinitely more useful to know *which* items
have a broken __repr__ (along with the context of the valid
objects), rather than knowing *one or more* items in the
list has a broken __repr__.  repr.repr is often called
recursively; if something bad happens far down in the call
stack it can really be a pain to track down where it
happened (all tracebacks to repr are going to look mostly
the same) unless repr.repr is fixed to handle this situation.

----------------------------------------------------------------------

Comment By: Christos Georgiou (tzot)
Date: 2003-01-21 15:58

Message:
Logged In: YES 
user_id=539787

Sorry, incomplete last post, I clicked in the wrong place (ie 
SUBMIT) to come back to IE.

The case is that repr.py is documented as for debugging 
use... you suggest that all exceptions are caught, but that 
would not be consistent with __builtins__.repr .

If the module was intended for generic use, I would suggest 
the opposite; no exceptions should be caught (for 
consistency).

----------------------------------------------------------------------

Comment By: Christos Georgiou (tzot)
Date: 2003-01-21 15:47

Message:
Logged In: YES 
user_id=539787

Do you suggest to remove the exception catching in 
Repr.repr_instance or adding a try clause in the s = `x` line of 
Repr.repr1 ?

----------------------------------------------------------------------

Comment By: Bob Ippolito (etrepum)
Date: 2003-01-20 09:11

Message:
Logged In: YES 
user_id=139309

It's not my module, it comes with Python (at least in my 2.2, and 2.3 CVS HEAD).  Look for yourself!

The issue has nothing to do with the fact that it has the same name as the builtin.

----------------------------------------------------------------------

Comment By: Christos Georgiou (tzot)
Date: 2003-01-20 00:28

Message:
Logged In: YES 
user_id=539787

Did you try to change the module name to, say, repr2 and 
then run it again?  Any specific reason you named the 
module as an internal function?
BTW, what is repr.repr supposed to do?  Please supply the 
repr.py script, along with the full traceback.

----------------------------------------------------------------------

Comment By: Bob Ippolito (etrepum)
Date: 2003-01-13 09:54

Message:
Logged In: YES 
user_id=139309

I say it's a bug because it's inconsistent behavior.  You
can repr the class instance that throws during a __repr__
just fine ('<A instance at 237360>' ), but you can't
__repr__ a bound method of the class instance without
throwing an exception.  In fact, if the __repr__ of the
class instance returns anything but a string, say it returns
None for example, then the exception will bubble all the way
up and you get no useful output from repr.repr.

It's very annoying if you're doing something equivalent to,
but more useful than:
repr.repr([1, 2, 3, 4, A().someMethod, 6, 7, 8, A()])

Note that repr.repr([1, 2, 3, 4, A(), 6, 7, 8]) does not
throw any exceptions.

There should be some sort of graceful way to do it, no?  I
think it ought to handle the bound method broken instance
case just like it handles the broken instance case directly.

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2003-01-12 23:34

Message:
Logged In: YES 
user_id=31435

Why do say this is a bug?  If you raise an exception, yes, 
you get an exception.  Most __repr__ *implementations* 
try hard not to raise any exceptions, but it's never 
guaranteed that they won't.  For example, just about any 
__repr__ implementation may run out of memory while 
trying to build its result string, so MemoryError is almost 
always a possible exception.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=666958&group_id=5470


From noreply@sourceforge.net  Fri Feb  7 01:40:41 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 06 Feb 2003 17:40:41 -0800
Subject: [Python-bugs-list] [ python-Bugs-680789 ] repr() of large array objects takes quadratic time
Message-ID: <E18gxVR-0004ut-00@sc8-sf-web2.sourceforge.net>

Bugs item #680789, was opened at 2003-02-05 06:00
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=680789&group_id=5470

Category: Python Library
Group: Python 2.2
Status: Open
Resolution: None
Priority: 5
Submitted By: Jurjen N.E. Bos (jneb)
>Assigned to: Nobody/Anonymous (nobody)
Summary: repr() of large array objects takes quadratic time

Initial Comment:
This is a bug and a partial patch.

If I debug a program that contains a ridiculously large array (8M entries in my case), the debugger takes forever.
It happens in Mac OS X, Python 2.2, but I found the bug in is the repr module, so it is probably universal.
The thing is, that after the fix below, it still doesn't work!
Did I miss something trivial (like repr is builtin, or something like that?). Would someone with Mac OS X experience help out here, please (Jack?).

Here's the diff to make repr.repr work with large arrays:
13a14
>       self.maxarray = 5
50a52,62
>     def repr_array(self, x, level):
>         n = len(x)
>       header = "array('"+x.typecode+"', ["
>         if n == 0: return header+"])"
>         if level <= 0: return header+"...])"
>         s = ''
>         for i in range(min(n, self.maxarray)):
>             if s: s = s + ', '
>             s = s + self.repr1(x[i], level-1)
>         if n > self.maxarray: s = s + ', ...'
>         return header + s + "])"

----------------------------------------------------------------------

>Comment By: Tim Peters (tim_one)
Date: 2003-02-06 20:40

Message:
Logged In: YES 
user_id=31435

I can't make time for this, so unassigned it.  It would make 
a good, brief project for someone -- the list and dict 
tp_reprs are linear-time, and tp_repr for array.array 
objects shouldn't be any harder than they were.

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-02-06 16:37

Message:
Logged In: YES 
user_id=45365

Okay, so the real bug is that tp_repr of array objects takes quadratic time.

I'm changing the summary of this report then, and assigning back to you (Tim), on the basis that you did more checkins on arraymodule than I did. Feel free to pass the potato on:-)

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2003-02-05 18:08

Message:
Logged In: YES 
user_id=31435

pdb does import repr.py, but probably doesn't use it in 
whatever way Jurjen is using to display his big array.

WRT that, note that Jurjen is using array.array objects, not 
lists.  The internal array.array tp_repr slot is quadratic-time in 
the size of the array, while list's tp_repr is linear time.

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-02-05 17:40

Message:
Logged In: YES 
user_id=45365

The fix is fine (it works for me the same way as for Tim), but I think we're shooting past the problem here.

First, pdb doesn't use repr.repr(), it uses the normal builtin repr().

Second, I don't see any sluggishness in pdb with large arrays. I tried
debugging
def foo():
    a = range(8000000)
and there was no problem. Allocating the object took a bit of time, yes, and if you actually try to print it you'll stare at about 800K lines filled with digits scrolling over your screen, but that is to be expected.

Could it be your sluggishness is coming from something else? For example, MacOSX starts behaving *very* badly if your root disk is full, because then it can't allocate swap space, and due to its optimistic behaviour it comes to a grinding halt.

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2003-02-05 13:36

Message:
Logged In: YES 
user_id=31435

Nice to see you, Jurgen!  I checked this into current CVS, 
and it works fine for me in isolation:

>>> len(a)
11055060
>>> repr.repr(a)
"array('i', [0, 1, 2, 3, 4, ...])"
>>>

That goes in an eyeblink.  So more detail is needed about 
what "it still doesn't work!" means.  Assigned to Jack, and he 
can use current CVS to try it.

Lib/repr.py; new revision: 1.15
Lib/test/test_repr.py; new revision: 1.16
Misc/NEWS; new revision: 1.642

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=680789&group_id=5470


From noreply@sourceforge.net  Fri Feb  7 01:45:25 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 06 Feb 2003 17:45:25 -0800
Subject: [Python-bugs-list] [ python-Bugs-675259 ] os.environ leaks under FreeBSD and Mac OS X
Message-ID: <E18gxa1-0006Gw-00@sc8-sf-web3.sourceforge.net>

Bugs item #675259, was opened at 2003-01-27 16:39
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=675259&group_id=5470

Category: Documentation
Group: Platform-specific
Status: Open
Resolution: None
Priority: 5
Submitted By: Stuart Bishop (zenzen)
Assigned to: Nobody/Anonymous (nobody)
Summary: os.environ leaks under FreeBSD and Mac OS X

Initial Comment:
If this sort if thing is documented anywhere, the putenv(3) method leaks under FreeBSD and Mac OS X (Mac OS X documents it as a bug they inherited from the FreeBSD codebase in the man page).

So don't set environment variables in a loop.

----------------------------------------------------------------------

>Comment By: Stuart Bishop (zenzen)
Date: 2003-02-07 12:45

Message:
Logged In: YES 
user_id=46639

The following code will demonstrate the issue under Mac OS X
(measured using 'top'). I havn't got access to a FreeBSD system 
to test it on. 

Python 2.3a1 (#1, Jan 30 2003, 16:43:39) 
[GCC 3.1 20020420 (prerelease)] on darwin
Type "help", "copyright", "credits" or "license" for more
information.
>>> import os
>>> while 1:
...     for e in ['some','strings','of','different','length']:
...             os.environ['TZ'] = e
...             del os.environ['TZ']
... 


Removing the 'del os.environ['TZ'] line makes no difference.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-07 12:28

Message:
Logged In: YES 
user_id=33168

Stuart have you tested this or are you just assuming that
putenv leaks in python?  I tested this on FreeBSD and the
process did not leak.  Note that python maintains a
dictionary of all the values set with putenv, if they are
set again, the memory will be cleared.  See posix_putenv()
in posixmodule.c and the posix_putenv_garbage variable.

----------------------------------------------------------------------

Comment By: Stuart Bishop (zenzen)
Date: 2003-01-29 11:15

Message:
Logged In: YES 
user_id=46639

Relevant URL's:

http://www.freebsd.org/cgi/query-pr.cgi?pr=misc/19406
http://www.freebsd.org/cgi/query-pr.cgi?pr=10341
http://www.freebsd.org/cgi/query-pr.cgi?pr=5604

An alternative may be to bypass the putenv(3) method on these platforms
and manipulate environ manually in which case this is not a documentation bug but a feature request.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=675259&group_id=5470


From noreply@sourceforge.net  Fri Feb  7 01:49:55 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 06 Feb 2003 17:49:55 -0800
Subject: [Python-bugs-list] [ python-Bugs-666444 ] 'help' makes linefeed only under Win32
Message-ID: <E18gxeN-0005DJ-00@sc8-sf-web2.sourceforge.net>

Bugs item #666444, was opened at 2003-01-11 20:04
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=666444&group_id=5470

>Category: Python Library
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Gregory Smith (gregsmith)
>Assigned to: Tim Peters (tim_one)
Summary: 'help' makes linefeed only under Win32

Initial Comment:
The 'help' command, under a win32 console app, separates
lines with \n instead of \r\n as it has in the past,
making its
output a little hard to follow. Possibly an unintended
side effect
of the 'universal newline' addition?

Python 2.3a1 (#38, Dec 31 2002, 17:53:59) [MSC v.1200
32 bit (Intel)] on win32

(windows 98)



----------------------------------------------------------------------

>Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-06 20:49

Message:
Logged In: YES 
user_id=33168

Looking at the code, I don't see any problems.  I'm guessing
this may be a windows issue.  Universal newlines only seems
to affect reading, so shouldn't be an issue.  Hopefully Tim
has time, interest, and a clue, since I don't.  Tim, do you
know if there is a difference between fopen() and fdopen()
on windows?

----------------------------------------------------------------------

Comment By: Gregory Smith (gregsmith)
Date: 2003-01-16 00:05

Message:
Logged In: YES 
user_id=292741

Tracked this down to a change in pydoc.py:
---------------- 2.2.2 -------------------------
def tempfilepager(text, cmd):
    import tempfile
    filename = tempfile.mktemp()
    file = open(filename, 'w')
    file.write(text)
    file.close()
------------------2.3 -------------------------
def tempfilepager(text, cmd):
    import tempfile
    (fd, filename) = tempfile.mkstemp()
    file = os.fdopen(fd, 'w')
    file.write(text)
    file.close()
------------------------------------------------
It seems the use of fdopen means that \n's don't
get  changed to \r\n when the file is written, even though
'w' is used. Whether this should be fixed by modding this
code, or whether this is a fdopen bug, I don't know.



----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=666444&group_id=5470


From noreply@sourceforge.net  Fri Feb  7 02:03:40 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 06 Feb 2003 18:03:40 -0800
Subject: [Python-bugs-list] [ python-Bugs-666444 ] 'help' makes linefeed only under Win32
Message-ID: <E18gxrg-0005iW-00@sc8-sf-web2.sourceforge.net>

Bugs item #666444, was opened at 2003-01-11 20:04
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=666444&group_id=5470

Category: Python Library
Group: Python 2.3
>Status: Closed
>Resolution: Fixed
Priority: 5
Submitted By: Gregory Smith (gregsmith)
Assigned to: Tim Peters (tim_one)
Summary: 'help' makes linefeed only under Win32

Initial Comment:
The 'help' command, under a win32 console app, separates
lines with \n instead of \r\n as it has in the past,
making its
output a little hard to follow. Possibly an unintended
side effect
of the 'universal newline' addition?

Python 2.3a1 (#38, Dec 31 2002, 17:53:59) [MSC v.1200
32 bit (Intel)] on win32

(windows 98)



----------------------------------------------------------------------

>Comment By: Tim Peters (tim_one)
Date: 2003-02-06 21:03

Message:
Logged In: YES 
user_id=31435

Neal, yes, it's not even possible for fdopen() to create a 
text-mode file on WIndows, and text mode is needed to get 
the \r\n line ends Windows needs for readability.  
Presumably life on Macs is also screwed by this.

Greg, I reverted the code to the "pre-security craze" 
version.  If someone thinks opening a help file for output is a 
security hole, they can waste their time scratching their 
own paranoid itches <wink>.

Lib/pydoc.py; new revision: 1.75

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-06 20:49

Message:
Logged In: YES 
user_id=33168

Looking at the code, I don't see any problems.  I'm guessing
this may be a windows issue.  Universal newlines only seems
to affect reading, so shouldn't be an issue.  Hopefully Tim
has time, interest, and a clue, since I don't.  Tim, do you
know if there is a difference between fopen() and fdopen()
on windows?

----------------------------------------------------------------------

Comment By: Gregory Smith (gregsmith)
Date: 2003-01-16 00:05

Message:
Logged In: YES 
user_id=292741

Tracked this down to a change in pydoc.py:
---------------- 2.2.2 -------------------------
def tempfilepager(text, cmd):
    import tempfile
    filename = tempfile.mktemp()
    file = open(filename, 'w')
    file.write(text)
    file.close()
------------------2.3 -------------------------
def tempfilepager(text, cmd):
    import tempfile
    (fd, filename) = tempfile.mkstemp()
    file = os.fdopen(fd, 'w')
    file.write(text)
    file.close()
------------------------------------------------
It seems the use of fdopen means that \n's don't
get  changed to \r\n when the file is written, even though
'w' is used. Whether this should be fixed by modding this
code, or whether this is a fdopen bug, I don't know.



----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=666444&group_id=5470


From noreply@sourceforge.net  Fri Feb  7 02:15:04 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 06 Feb 2003 18:15:04 -0800
Subject: [Python-bugs-list] [ python-Feature Requests-681533 ] Additional string stuff.
Message-ID: <E18gy2i-00031Y-00@sc8-sf-web4.sourceforge.net>

Feature Requests item #681533, was opened at 2003-02-06 03:14
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=681533&group_id=5470

Category: Python Library
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Jeremy Fincher (jemfinch)
Assigned to: Nobody/Anonymous (nobody)
Summary: Additional string stuff.

Initial Comment:
In a lot of my programs that mess with strings, I end up 
somewhere making a variable "ascii" via 
(string.maketrans('', '')).  It's just a 256-character string 
comprising the ascii character set. 
 
I use it, oftentimes, simply to be able to turn the 'translate' 
method on strings into a 'delete' method -- if I want to, 
say, remove all the spaces in aString, I'd do 
(aString.translate(ascii, string.whitespace)). 
 
Certainly an ascii variable in the string module couldn't 
hurt, would fit in with ascii_letters, etc. and would at least 
standarize the name of the full ascii set sure to be 
present in many Python programs. 
 
A little further out there, but I think just as useful, would 
be a delete method on strings.  So "foo bar 
baz".delete(string.whitespace) would return "foobarbaz".  
It would be equivalent to "foo bar baz".translate(ascii, 
string.whitespace), or the wildly inefficient: 
 
def delete(s, deleteChars): 
    l = [] 
    for c in s: 
        if c not in deleteChars: 
            l.append(c) 
    return ''.join(l) 
 
Anyway, that's all I can think of.  Do with it what you will. 
 
Jeremy 

----------------------------------------------------------------------

>Comment By: Tim Peters (tim_one)
Date: 2003-02-06 21:15

Message:
Logged In: YES 
user_id=31435

Just noting that you can pass None for sep if you want to 
explicitly ask for the default behavior.

>>> "      a  b   c".split(None, 1)
['a', 'b   c']
>>>


----------------------------------------------------------------------

Comment By: Jeremy Fincher (jemfinch)
Date: 2003-02-06 04:19

Message:
Logged In: YES 
user_id=99508

Let me also make one other suggestion:  the split method on a 
string object has, by default, behavior that can't be replicated 
by passing an argument as a separator.  That is, the default 
separated acts like re.split(r'\s+'), but it's impossible to pass 
any value into the method to achieve that same result. 
 
The problem arises when a user wants to use the maxsplit() 
parameter to the method.  Because maxsplit is a positional 
parameter instead of a keyword parameter, the user *must* 
declare a separate to split on, and thus loses his ability to split 
on whitespace-in-general.  If maxsplit was changed from being 
a positional parameter to being a keyword parameter, then a 
programmer wouldn't have to give up the default behavior of 
the split method in order to pass it a maxsplit. 
 
At present, negative maxsplit values don't differ in any way 
from split's default behavior (with no maxsplit parameter given).  
Thus, the keyword maxsplit could default to -1 with no 
breakage of code.  I can't see any place where changing 
maxsplit to a keyword parameter would break any existing 
code 
 
Jeremy 

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=681533&group_id=5470


From noreply@sourceforge.net  Fri Feb  7 02:22:40 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 06 Feb 2003 18:22:40 -0800
Subject: [Python-bugs-list] [ python-Bugs-680797 ] expat doesn't build on MacOS9
Message-ID: <E18gyA4-0003Fn-00@sc8-sf-web4.sourceforge.net>

Bugs item #680797, was opened at 2003-02-05 06:11
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=680797&group_id=5470

Category: Extension Modules
Group: None
>Status: Closed
>Resolution: Fixed
Priority: 5
Submitted By: Jack Jansen (jackjansen)
Assigned to: Fred L. Drake, Jr. (fdrake)
Summary: expat doesn't build on MacOS9

Initial Comment:
Expat doesn't build anymore on Mac OS 9. The problem is that the new expat.h has a couple of functions returning an "enum XML_Status", but that enum is declared only further down in expat.h and the Metrowerks compiler chokes on this.

It's easy enough to fix (either moving the routines down or moving the enum declaration up), but as I seem to recall that this file is imported from the expat distribution I'm not sure whether that's a good idea, or something else needs to be done.

----------------------------------------------------------------------

>Comment By: Fred L. Drake, Jr. (fdrake)
Date: 2003-02-06 21:22

Message:
Logged In: YES 
user_id=3066

This is a known bug in Expat 1.95.6 that affects some 
compilers and not others (it even varies across versions of 
GCC).  This has been fix in Expat's CVS repository; I've 
committed the same change to Python's CVS as 
Modules/expat/expat.h revision 1.5 (1.51 in Expat's CVS).

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-06 20:16

Message:
Logged In: YES 
user_id=33168

Fred can you look at this since Martin is on vacation?  I'm
also not sure if this is part of expat or pyexpat (or how to
tell).

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=680797&group_id=5470


From noreply@sourceforge.net  Fri Feb  7 02:35:17 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 06 Feb 2003 18:35:17 -0800
Subject: [Python-bugs-list] [ python-Bugs-675259 ] os.environ leaks under FreeBSD and Mac OS X
Message-ID: <E18gyMH-0006hh-00@sc8-sf-web2.sourceforge.net>

Bugs item #675259, was opened at 2003-01-27 00:39
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=675259&group_id=5470

Category: Documentation
Group: Platform-specific
>Status: Closed
>Resolution: Accepted
Priority: 5
Submitted By: Stuart Bishop (zenzen)
>Assigned to: Neal Norwitz (nnorwitz)
Summary: os.environ leaks under FreeBSD and Mac OS X

Initial Comment:
If this sort if thing is documented anywhere, the putenv(3) method leaks under FreeBSD and Mac OS X (Mac OS X documents it as a bug they inherited from the FreeBSD codebase in the man page).

So don't set environment variables in a loop.

----------------------------------------------------------------------

>Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-06 21:35

Message:
Logged In: YES 
user_id=33168

Ok, I tried simpler code on FreeBSD 4.7 and expected to see
the leak.  Thanks.

Checked in Doc/lib/libos.tex 1.111 and 1.74.2.1.2.10

----------------------------------------------------------------------

Comment By: Stuart Bishop (zenzen)
Date: 2003-02-06 20:45

Message:
Logged In: YES 
user_id=46639

The following code will demonstrate the issue under Mac OS X
(measured using 'top'). I havn't got access to a FreeBSD system 
to test it on. 

Python 2.3a1 (#1, Jan 30 2003, 16:43:39) 
[GCC 3.1 20020420 (prerelease)] on darwin
Type "help", "copyright", "credits" or "license" for more
information.
>>> import os
>>> while 1:
...     for e in ['some','strings','of','different','length']:
...             os.environ['TZ'] = e
...             del os.environ['TZ']
... 


Removing the 'del os.environ['TZ'] line makes no difference.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-06 20:28

Message:
Logged In: YES 
user_id=33168

Stuart have you tested this or are you just assuming that
putenv leaks in python?  I tested this on FreeBSD and the
process did not leak.  Note that python maintains a
dictionary of all the values set with putenv, if they are
set again, the memory will be cleared.  See posix_putenv()
in posixmodule.c and the posix_putenv_garbage variable.

----------------------------------------------------------------------

Comment By: Stuart Bishop (zenzen)
Date: 2003-01-28 19:15

Message:
Logged In: YES 
user_id=46639

Relevant URL's:

http://www.freebsd.org/cgi/query-pr.cgi?pr=misc/19406
http://www.freebsd.org/cgi/query-pr.cgi?pr=10341
http://www.freebsd.org/cgi/query-pr.cgi?pr=5604

An alternative may be to bypass the putenv(3) method on these platforms
and manipulate environ manually in which case this is not a documentation bug but a feature request.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=675259&group_id=5470


From noreply@sourceforge.net  Fri Feb  7 05:19:59 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 06 Feb 2003 21:19:59 -0800
Subject: [Python-bugs-list] [ python-Feature Requests-681533 ] Additional string stuff.
Message-ID: <E18h0vf-0004Dd-00@sc8-sf-web3.sourceforge.net>

Feature Requests item #681533, was opened at 2003-02-06 03:14
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=681533&group_id=5470

Category: Python Library
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Jeremy Fincher (jemfinch)
Assigned to: Nobody/Anonymous (nobody)
Summary: Additional string stuff.

Initial Comment:
In a lot of my programs that mess with strings, I end up 
somewhere making a variable "ascii" via 
(string.maketrans('', '')).  It's just a 256-character string 
comprising the ascii character set. 
 
I use it, oftentimes, simply to be able to turn the 'translate' 
method on strings into a 'delete' method -- if I want to, 
say, remove all the spaces in aString, I'd do 
(aString.translate(ascii, string.whitespace)). 
 
Certainly an ascii variable in the string module couldn't 
hurt, would fit in with ascii_letters, etc. and would at least 
standarize the name of the full ascii set sure to be 
present in many Python programs. 
 
A little further out there, but I think just as useful, would 
be a delete method on strings.  So "foo bar 
baz".delete(string.whitespace) would return "foobarbaz".  
It would be equivalent to "foo bar baz".translate(ascii, 
string.whitespace), or the wildly inefficient: 
 
def delete(s, deleteChars): 
    l = [] 
    for c in s: 
        if c not in deleteChars: 
            l.append(c) 
    return ''.join(l) 
 
Anyway, that's all I can think of.  Do with it what you will. 
 
Jeremy 

----------------------------------------------------------------------

>Comment By: Jeremy Fincher (jemfinch)
Date: 2003-02-07 00:19

Message:
Logged In: YES 
user_id=99508

I guess that's what I get for not reading the documentation :) 
 
Oh well, the other two suggestions stand :) 
 
Jeremy 

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2003-02-06 21:15

Message:
Logged In: YES 
user_id=31435

Just noting that you can pass None for sep if you want to 
explicitly ask for the default behavior.

>>> "      a  b   c".split(None, 1)
['a', 'b   c']
>>>


----------------------------------------------------------------------

Comment By: Jeremy Fincher (jemfinch)
Date: 2003-02-06 04:19

Message:
Logged In: YES 
user_id=99508

Let me also make one other suggestion:  the split method on a 
string object has, by default, behavior that can't be replicated 
by passing an argument as a separator.  That is, the default 
separated acts like re.split(r'\s+'), but it's impossible to pass 
any value into the method to achieve that same result. 
 
The problem arises when a user wants to use the maxsplit() 
parameter to the method.  Because maxsplit is a positional 
parameter instead of a keyword parameter, the user *must* 
declare a separate to split on, and thus loses his ability to split 
on whitespace-in-general.  If maxsplit was changed from being 
a positional parameter to being a keyword parameter, then a 
programmer wouldn't have to give up the default behavior of 
the split method in order to pass it a maxsplit. 
 
At present, negative maxsplit values don't differ in any way 
from split's default behavior (with no maxsplit parameter given).  
Thus, the keyword maxsplit could default to -1 with no 
breakage of code.  I can't see any place where changing 
maxsplit to a keyword parameter would break any existing 
code 
 
Jeremy 

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=681533&group_id=5470


From noreply@sourceforge.net  Fri Feb  7 05:44:36 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 06 Feb 2003 21:44:36 -0800
Subject: [Python-bugs-list] [ python-Bugs-681003 ] itertools issues
Message-ID: <E18h1JU-0003tu-00@sc8-sf-web2.sourceforge.net>

Bugs item #681003, was opened at 2003-02-05 10:52
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=681003&group_id=5470

Category: Python Library
Group: Python 2.3
>Status: Closed
>Resolution: Fixed
Priority: 5
Submitted By: Greg Chapman (glchapman)
Assigned to: Raymond Hettinger (rhettinger)
Summary: itertools issues

Initial Comment:
I've been checking out the new itertools module.  It's a great 
addition; I look forward to using it.  However, I did notice a 
few things:

1) I believe islice should worry about overflow when it adds 
step to next.  Given a call like islice(seq, 1, 10, sys.maxint), 
it looks like islice will not stop after retrieving item 1 
because this:

  lz->next += lz->step;

will result in lz->next becoming negative.

2) I note that starmap_next passes the args obtained from 
PyIter_Next directly to PyObject_Call.  PyObject_Call 
assumes that its arg param is a tuple (and so do the various 
tp_call slots).  However, there is nothing to ensure that the 
args object obtained from the iterator in starmap_next is in 
fact a tuple.  I believe either a PyTuple_Check or a 
PySequence_Tuple call is in order before passing the args 
to PyObject_Call.

3) I don't think it's safe for imap_next to reuse the same 
argtuple for all the calls.  This tuple is passed directly to C 
functions, and there's nothing to stop them from saving a 
reference to it someplace (unless it's documented 
somewhere that you shouldn't save references to the args 
tuple).

4) I think count_next should worry about overflow also 
(consider count(sys.maxint)).


----------------------------------------------------------------------

>Comment By: Raymond Hettinger (rhettinger)
Date: 2003-02-07 00:44

Message:
Logged In: YES 
user_id=80475

Committed changes as:
Modules/itertoolsmodule.c 1.2
Doc/lib/libitertools.tex 1.2
Lib/test/test_itertools.py 1.3

Documented the overflow behavior for count() but may 
still decide to add overflow handling at a later date.  All of 
the other ideas were implemented.



----------------------------------------------------------------------

Comment By: Christos Georgiou (tzot)
Date: 2003-02-06 12:19

Message:
Logged In: YES 
user_id=539787

Yes, itertools is great, and I wonder how Guido agreed to it 
(since he usually dislikes using FP stuff in Python).  I also 
think that the tuple way checking refcount will work.

Another point comes to mind: in many places, the code 
(Python in general, not only itertools) builds a tuple and calls 
PyEval_CallObject with it and without any keyword 
arguments; I believe in these cases the call should be 
PyObject_Call, since there's no use in checking every time if 
first arg is a tuple and then calling PyObject_Call...

----------------------------------------------------------------------

Comment By: Greg Chapman (glchapman)
Date: 2003-02-06 11:47

Message:
Logged In: YES 
user_id=86307

Just a quick note on imap.  I checked the 2.3a1 map function 
before posting (and just rechecked).  It creates the alist tuple at 
the beginning of the loop, and then eventually passes it to 
PyEval_CallObject before DECREFing it.  So it looks to me like 
the current version of map guards against someone saving a 
reference to alist.


----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2003-02-06 10:54

Message:
Logged In: YES 
user_id=80475

Glad you like the module.

Will implement the limit overflow checks and the tuple 
check.

I'm still thinking about the re-use of the arg tuple in 
imap_next().  The code was lifted directly from 
__builtins__.map() which has worked flawlessly since the 
early versions of Python.   But, if it does need fixing, then 
I'm looking at an approach that verifies the refcnt is one 
before re-using the tuple.  If that works out, then both the 
re-use and the guard can be applied to imap(), map(), izip
(), and enumerate().

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=681003&group_id=5470


From noreply@sourceforge.net  Fri Feb  7 06:47:43 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 06 Feb 2003 22:47:43 -0800
Subject: [Python-bugs-list] [ python-Feature Requests-681533 ] Additional string stuff.
Message-ID: <E18h2IZ-00027b-00@sc8-sf-web4.sourceforge.net>

Feature Requests item #681533, was opened at 2003-02-06 03:14
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=681533&group_id=5470

Category: Python Library
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Jeremy Fincher (jemfinch)
Assigned to: Nobody/Anonymous (nobody)
Summary: Additional string stuff.

Initial Comment:
In a lot of my programs that mess with strings, I end up 
somewhere making a variable "ascii" via 
(string.maketrans('', '')).  It's just a 256-character string 
comprising the ascii character set. 
 
I use it, oftentimes, simply to be able to turn the 'translate' 
method on strings into a 'delete' method -- if I want to, 
say, remove all the spaces in aString, I'd do 
(aString.translate(ascii, string.whitespace)). 
 
Certainly an ascii variable in the string module couldn't 
hurt, would fit in with ascii_letters, etc. and would at least 
standarize the name of the full ascii set sure to be 
present in many Python programs. 
 
A little further out there, but I think just as useful, would 
be a delete method on strings.  So "foo bar 
baz".delete(string.whitespace) would return "foobarbaz".  
It would be equivalent to "foo bar baz".translate(ascii, 
string.whitespace), or the wildly inefficient: 
 
def delete(s, deleteChars): 
    l = [] 
    for c in s: 
        if c not in deleteChars: 
            l.append(c) 
    return ''.join(l) 
 
Anyway, that's all I can think of.  Do with it what you will. 
 
Jeremy 

----------------------------------------------------------------------

>Comment By: Jeremy Fincher (jemfinch)
Date: 2003-02-07 01:47

Message:
Logged In: YES 
user_id=99508

I guess that's what I get for not reading the documentation :) 
 
Oh well, the other two suggestions stand :) 
 
Jeremy 

----------------------------------------------------------------------

Comment By: Jeremy Fincher (jemfinch)
Date: 2003-02-07 00:19

Message:
Logged In: YES 
user_id=99508

I guess that's what I get for not reading the documentation :) 
 
Oh well, the other two suggestions stand :) 
 
Jeremy 

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2003-02-06 21:15

Message:
Logged In: YES 
user_id=31435

Just noting that you can pass None for sep if you want to 
explicitly ask for the default behavior.

>>> "      a  b   c".split(None, 1)
['a', 'b   c']
>>>


----------------------------------------------------------------------

Comment By: Jeremy Fincher (jemfinch)
Date: 2003-02-06 04:19

Message:
Logged In: YES 
user_id=99508

Let me also make one other suggestion:  the split method on a 
string object has, by default, behavior that can't be replicated 
by passing an argument as a separator.  That is, the default 
separated acts like re.split(r'\s+'), but it's impossible to pass 
any value into the method to achieve that same result. 
 
The problem arises when a user wants to use the maxsplit() 
parameter to the method.  Because maxsplit is a positional 
parameter instead of a keyword parameter, the user *must* 
declare a separate to split on, and thus loses his ability to split 
on whitespace-in-general.  If maxsplit was changed from being 
a positional parameter to being a keyword parameter, then a 
programmer wouldn't have to give up the default behavior of 
the split method in order to pass it a maxsplit. 
 
At present, negative maxsplit values don't differ in any way 
from split's default behavior (with no maxsplit parameter given).  
Thus, the keyword maxsplit could default to -1 with no 
breakage of code.  I can't see any place where changing 
maxsplit to a keyword parameter would break any existing 
code 
 
Jeremy 

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=681533&group_id=5470


From noreply@sourceforge.net  Fri Feb  7 08:48:27 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Fri, 07 Feb 2003 00:48:27 -0800
Subject: [Python-bugs-list] [ python-Bugs-681934 ] bundlebuilder/zipimport errors
Message-ID: <E18h4BP-0005dI-00@sc8-sf-web4.sourceforge.net>

Bugs item #681934, was opened at 2003-02-06 22:22
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=681934&group_id=5470

Category: Macintosh
Group: Python 2.3
>Status: Closed
>Resolution: Fixed
Priority: 5
Submitted By: Robin Dunn (robind)
Assigned to: Just van Rossum (jvr)
Summary: bundlebuilder/zipimport errors

Initial Comment:
While testing the bundlebuilder patch that adds shared
library support I decided to try it with the CVS
version of bundlebuilder.py and was excited to see it
using the new zipimport feature.  However there is a
problem.

In the wxPython package there is a module named
__version__.py that is imported from __init__.py.  Both
correctly have .pyc files placed in Modules.zip under a
wxPython directory, but I get this error when trying to
run the app bundle:

Traceback (most recent call last):
  File
"/usr/local/projects/wx/wxPython/samples/doodle/SuperDoodle.app/Contents/Resources/superdoodle.py",
line 13, in ?
    from wxPython.wx import *
  File
"/Library/Frameworks/Python.framework/Versions/2.3/lib/python2.3/site-packages/wxPython/__init__.py",
line 14, in ?
    import __version__
ImportError: No module named __version__

(I assume that the pathnames are as they are (not
reflecting the Modules.zip path) because that is where
the .pyc's are compiled from ???)

If I force USE_ZIPIMPORT=0 in bundlebuilder.py then the
bundle works just fine.

----------------------------------------------------------------------

>Comment By: Just van Rossum (jvr)
Date: 2003-02-07 09:48

Message:
Logged In: YES 
user_id=92689

Yeah, sorry, I found that problem days after 2.3a1 was released when adding zipimport support to bundlebuilder :-/ So I simply fixed it, but there was never a bug report. I wondered back then if I should make one just for documentation purposes, and in hindsight I should've done just that ;-) Let this remark in this one serve that purpose for now. Closing.

----------------------------------------------------------------------

Comment By: Robin Dunn (robind)
Date: 2003-02-07 00:28

Message:
Logged In: YES 
user_id=53955

Yes, I do.  I wondered if it might be a problem with
zipimport rather than bundlebuilder, but didn't seen any
bugs listed that seemed the same. 

----------------------------------------------------------------------

Comment By: Just van Rossum (jvr)
Date: 2003-02-06 23:13

Message:
Logged In: YES 
user_id=92689

You don't happen to run 2.3a1 are you? It's zipimport support is unfortunately severly broken :-( If not, I'll investigate...

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=681934&group_id=5470


From noreply@sourceforge.net  Fri Feb  7 09:35:21 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Fri, 07 Feb 2003 01:35:21 -0800
Subject: [Python-bugs-list] [ python-Bugs-682215 ] Carbon.CF.CFString should require ASCII
Message-ID: <E18h4un-00070e-00@sc8-sf-web4.sourceforge.net>

Bugs item #682215, was opened at 2003-02-07 10:35
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=682215&group_id=5470

Category: Macintosh
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Jack Jansen (jackjansen)
Assigned to: Jack Jansen (jackjansen)
Summary: Carbon.CF.CFString should require ASCII

Initial Comment:
If a Python string is passed where a CFString is expected the bytes are used verbatim (using CFStringCreateWithCString()), but this is probably wrong. We should either allow only ASCII or use sys.getdefaultencoding() or something similar.

This is a copy of PyObjC bug <https://sourceforge.net/tracker/?func=detail&atid=114534&aid=681880&group_id=14534>.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=682215&group_id=5470


From noreply@sourceforge.net  Fri Feb  7 11:29:57 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Fri, 07 Feb 2003 03:29:57 -0800
Subject: [Python-bugs-list] [ python-Bugs-680429 ] __module__ broken for extension classes
Message-ID: <E18h6hh-0007j2-00@sc8-sf-web1.sourceforge.net>

Bugs item #680429, was opened at 2003-02-04 19:55
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=680429&group_id=5470

Category: Python Interpreter Core
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Ralf W. Grosse-Kunstleve (rwgk)
Assigned to: Michael Hudson (mwh)
Summary: __module__ broken for extension classes

Initial Comment:
We are having problems using Boost.Python with Python 
2.3a1 because
under some circumstances the result of __module__ is 
different compared
to earlier Python versions:

Python 2.2.1 (#2, Jun 17 2002, 12:06:51) 
[GCC 2.96 20000731 (Red Hat Linux 7.3 2.96-110)] on 
linux2
Type "help", "copyright", "credits" or "license" for more 
information.
>>> import simple
>>> simple.empty.__module__
'simple'
>>> 

Python 2.3a1 (#1, Jan  6 2003, 14:17:56) 
[GCC 2.96 20000731 (Red Hat Linux 7.3 2.96-110)] on 
linux2
Type "help", "copyright", "credits" or "license" for more 
information.
>>> import simple
>>> simple.empty.__module__
'__main__'
>>> 

Because of this we can no longer pickle our extension 
classes.
For your reference the code for the simple module is 
attached.
This is using Boost release 1.29.0 (www.boost.org).

We have done some debugging. Boost.Python's internal 
idea of the module
associated with an extension class is still correct even when 
using
Python 2.3a1. David Abrahams (main Boost.Python author) 
is telling me
that he "changed Boost.Python to work the way Guido 
suggested before
2.2.2." Therefore we suspect that the __module__ problem 
is due to
a change/bug in Python 2.3a1.

Ralf


#include <boost/python/module.hpp>
#include <boost/python/class.hpp>

namespace sandbx { namespace {

  struct empty {};

  void init_module()
  {
    using namespace boost::python;
    class_<empty>("empty");
  }

}} // namespace sandbx::<anonymous>

BOOST_PYTHON_MODULE(simple)
{
  sandbx::init_module();
}


----------------------------------------------------------------------

>Comment By: Michael Hudson (mwh)
Date: 2003-02-07 11:29

Message:
Logged In: YES 
user_id=6656

Ah, I start to remember the details now.

In 2.2, if tp_name contained a period, everything to the
left of the period was __module__, that to the right __name__.

However, at some point (maybe even by 2.3) we want

class X:
    class Y:
        pass
print X.Y.__name__

to print 'X.Y' (it prints 'Y' today).  Also we made __name__
assignable for 2.3 and this behaviour caused confusion.

So we made things so that the '__module__' entry in the
type's dict always wins.

In summary, if you poke a '__module__' key into the
boost::python::dict object you pass to the call of the
metatype and change the first argument to just 'name', I
think you'll be set.

I pretty sure this will also work with 2.2.x, but if I were
you I'd check.

HTH!

----------------------------------------------------------------------

Comment By: Ralf W. Grosse-Kunstleve (rwgk)
Date: 2003-02-06 18:36

Message:
Logged In: YES 
user_id=71407

> In 2.2.x, how are you telling Python that __module__ should
> be "simple"?  By setting tp_name to "simple.empty"?  What's
> simple.__name__?

Python 2.3a1 (#2, Feb  5 2003, 09:39:30) 
[GCC 2.96 20000731 (Red Hat Linux 7.3 2.96-110)] on linux2
Type "help", "copyright", "credits" or "license" for more 
information.
>>> import sandbx.simple
>>> sandbx.simple.__name__
'sandbx.simple'
>>> sandbx.simple.empty.__name__
'sandbx.simple.empty'
>>> sandbx.simple.empty.__module__
IS HEAPTYPE
type->tp_dict, __module__ = __main__
'__main__'
>>> 

> I think I know how __module__ is getting there, it's the
> chunk of code currently at typeobject.c:1750.

Yes, that's right. I've verified this by adding more print statements.

> Can you point me to the bit of the boost source that creates
> the typeobject?

Here is where it happens (file 
boost/libs/python/src/object/class.cpp):

object result = object(class_metatype())(module_prefix() + name, 
bases, d);

With print statements I've verified that module_prefix() + name is
correctly passed in as "sandbx.simple.empty" .

I am attaching the file class.cpp. Could it be that we have to
update our class_metatype_object?

Thanks!
        Ralf


----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2003-02-06 11:22

Message:
Logged In: YES 
user_id=6656

OK, thanks for the info.  More questions, I'm afraid:

In 2.2.x, how are you telling Python that __module__ should
be "simple"?  By setting tp_name to "simple.empty"?  What's
simple.__name__?

I think I know how __module__ is getting there, it's the
chunk of code currently at typeobject.c:1750.

Can you point me to the bit of the boost source that creates
the typeobject?  'cept sf's just fallen off the net.

----------------------------------------------------------------------

Comment By: Ralf W. Grosse-Kunstleve (rwgk)
Date: 2003-02-05 18:25

Message:
Logged In: YES 
user_id=71407

> Look at typeobject.c:type_module() (about 100 lines in).
>
> Is simple.empty a HEAPTYPE?

Yes, it is a HEAPTYPE. I've established this by adding print 
statements
in typeobject.c:type_module():

static PyObject *
type_module(PyTypeObject *type, void *context)
{
        PyObject *mod;
        char *s;

        if (type->tp_flags & Py_TPFLAGS_HEAPTYPE) {
printf("IS HEAPTYPE\n");
                mod = PyDict_GetItemString(type-
>tp_dict, "__module__");
                Py_XINCREF(mod);
                s = PyString_AsString(mod);
printf("type->tp_dict, __module__ = %s\n", s);
                return mod;
        }

Result:

Python 2.3a1 (#2, Feb  5 2003, 09:39:30) 
[GCC 2.96 20000731 (Red Hat Linux 7.3 2.96-110)] on linux2
Type "help", "copyright", "credits" or "license" for more 
information.
>>> import simple
>>> simple.empty.__module__
IS HEAPTYPE
type->tp_dict, __module__ = __main__
'__main__'
>>> 

> If you can tell me what you want that function to do, I can
> have a go at it.

We expect "simple" as the result of simple.empty.__module__.

Current result if simple.so is moved to a package:

Python 2.3a1 (#2, Feb  5 2003, 09:39:30) 
[GCC 2.96 20000731 (Red Hat Linux 7.3 2.96-110)] on linux2
Type "help", "copyright", "credits" or "license" for more 
information.
>>> from sandbx import simple
>>> simple.empty.__module__
IS HEAPTYPE
type->tp_dict, __module__ = __main__
'__main__'
>>> 

Here we expect "sandbx.simple" .
Python 2.2.x produces the expected results using the exact same
Boost.Python source code.

David Abrahams provides this additional information:

  Remember that simple.empty is created by calling the metatype, 
so
  it's created on the heap, but it's Python's internal type creation
  mechanisms which do it -- we're not setting the flags manually.

Thanks!
        Ralf


----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2003-02-05 14:10

Message:
Logged In: YES 
user_id=6656

There has certainly been a change here.

Look at typeobject.c:type_module() (about 100 lines in).

If you can tell me what you want that function to do, I can
have a go at it.

Is simple.empty a HEAPTYPE?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=680429&group_id=5470


From noreply@sourceforge.net  Fri Feb  7 11:58:48 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Fri, 07 Feb 2003 03:58:48 -0800
Subject: [Python-bugs-list] [ python-Bugs-678518 ] Another parsermodule validation error
Message-ID: <E18h79c-0004Rv-00@sc8-sf-web4.sourceforge.net>

Bugs item #678518, was opened at 2003-02-01 02:23
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=678518&group_id=5470

Category: Parser/Compiler
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: logistix (logistix)
Assigned to: Michael Hudson (mwh)
Summary: Another parsermodule validation error

Initial Comment:
The following code generates an validation error.  It's an 
invalid ast tree where the global_stmt doesn't have the 
keyword 'global' as a child.

I tracked down the fix to validate_global_stmt() this time.

Python 2.3a1 (#38, Dec 31 2002, 17:53:59) [MSC 
v.1200 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more 
information.
>>> import symbol,token,parser
>>> ast = [symbol.file_input,
...  [symbol.stmt, [symbol.simple_stmt, 
[symbol.small_stmt, [symbol.global_stmt,
 [1, 'foo']]], [token.NEWLINE, '']]],
...  [token.NEWLINE, ''],
...  [token.ENDMARKER, '']]
>>> a = parser.sequence2ast(ast)
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
parser.ParserError: VALIDATION FAILURE: report this 
to the maintainer!
>>>

----------------------------------------------------------------------

>Comment By: Michael Hudson (mwh)
Date: 2003-02-07 11:58

Message:
Logged In: YES 
user_id=6656

Patches as such are probably fine.  The reason I haven't
checked them in is a wierdness that I may or may not be
related, but is bothering me.  Consider:

->> import parser, sys,glob,os, traceback
/>> for path in sys.path:
|..     for fil in glob.glob(os.path.join(path,"*.py")):
|..       try:
|..         try:
|..             a = parser.suite(file(fil).read())
|..             b = parser.ast2tuple(a)
|..             c = parser.sequence2ast(b)
|..         except parser.ParserError:
|..             print "Parse of %s" % fil, "FAILED"
|..             traceback.print_exc()
|..             print '----------------------------'
|..       except:
|..          print "waaa?"
|.. else:
|..     print "what?"
\__ 
Parse of /home/mwh/src/sf/python/dist/src/Lib/tarfile.py FAILED
Traceback (most recent call last):
  File "<input>", line 7, in ?
ParserError
----------------------------
Parse of /home/mwh/src/sf/python/dist/src/Lib/heapq.py FAILED
Traceback (most recent call last):
  File "<input>", line 7, in ?
ParserError
----------------------------
Parse of /home/mwh/src/sf/python/dist/src/Lib/getopt.py FAILED
Traceback (most recent call last):
  File "<input>", line 7, in ?
ParserError
----------------------------
Parse of /home/mwh/src/sf/python/dist/src/Lib/inspect.py FAILED
Traceback (most recent call last):
  File "<input>", line 7, in ?
ParserError
----------------------------
Parse of /home/mwh/src/sf/python/dist/src/Lib/pydoc.py FAILED
Traceback (most recent call last):
  File "<input>", line 7, in ?
ParserError
----------------------------
Parse of /home/mwh/src/sf/python/dist/src/Lib/lib-tk/Tix.py
FAILED
Traceback (most recent call last):
  File "<input>", line 7, in ?
ParserError
----------------------------
Traceback (most recent call last):
  File "<input>", line 2, in ?
ParserError: Illegal terminal: expected "__assert__"
->> 

Notice that an exception seems to have made it through the
except clauses!  The scary bit is that a (seemingly) random
number of parser errors *will* caught, and then one will get
through.

The only explanation I have for this is some kind of
returning non-NULL with exception set kind of bug.

Off to play with a debug build, I guess.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-07 01:17

Message:
Logged In: YES 
user_id=33168

Michael were these patches ok to check in?

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2003-02-03 11:00

Message:
Logged In: YES 
user_id=6656

Thanks for doing this.  I'll probably check these in after
I've finished crunching my way through monday morning email...

----------------------------------------------------------------------

Comment By: logistix (logistix)
Date: 2003-02-01 21:42

Message:
Logged In: YES 
user_id=699438

And I found a few more by doing a roundtrip 
ast2Tuple/tuple2ast on all files in pythonpath.  These actually 
affect valid AST's, so they're probably a bigger deal.

Valid assert statements were failing.  The patch was trivial, 
so its attached.

Also, files that took advantage of encoding_decl (from PEP 
263) were failing.  I couldn't tell if PEP 263 was offically 
closed, and didn't know what the offical grammar was, so I 
didn't do anything there.

I'll also attach the test script that turned up these errors.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=678518&group_id=5470


From noreply@sourceforge.net  Fri Feb  7 12:08:23 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Fri, 07 Feb 2003 04:08:23 -0800
Subject: [Python-bugs-list] [ python-Bugs-678518 ] Another parsermodule validation error
Message-ID: <E18h7It-00027D-00@sc8-sf-web3.sourceforge.net>

Bugs item #678518, was opened at 2003-02-01 02:23
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=678518&group_id=5470

Category: Parser/Compiler
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: logistix (logistix)
Assigned to: Michael Hudson (mwh)
Summary: Another parsermodule validation error

Initial Comment:
The following code generates an validation error.  It's an 
invalid ast tree where the global_stmt doesn't have the 
keyword 'global' as a child.

I tracked down the fix to validate_global_stmt() this time.

Python 2.3a1 (#38, Dec 31 2002, 17:53:59) [MSC 
v.1200 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more 
information.
>>> import symbol,token,parser
>>> ast = [symbol.file_input,
...  [symbol.stmt, [symbol.simple_stmt, 
[symbol.small_stmt, [symbol.global_stmt,
 [1, 'foo']]], [token.NEWLINE, '']]],
...  [token.NEWLINE, ''],
...  [token.ENDMARKER, '']]
>>> a = parser.sequence2ast(ast)
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
parser.ParserError: VALIDATION FAILURE: report this 
to the maintainer!
>>>

----------------------------------------------------------------------

>Comment By: Michael Hudson (mwh)
Date: 2003-02-07 12:08

Message:
Logged In: YES 
user_id=6656

Pff, I was right, but logistix's patch fix it.

logistix, did you forget to attach the encoding patch?  I
still get errors in module's with a cookie.

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2003-02-07 11:58

Message:
Logged In: YES 
user_id=6656

Patches as such are probably fine.  The reason I haven't
checked them in is a wierdness that I may or may not be
related, but is bothering me.  Consider:

->> import parser, sys,glob,os, traceback
/>> for path in sys.path:
|..     for fil in glob.glob(os.path.join(path,"*.py")):
|..       try:
|..         try:
|..             a = parser.suite(file(fil).read())
|..             b = parser.ast2tuple(a)
|..             c = parser.sequence2ast(b)
|..         except parser.ParserError:
|..             print "Parse of %s" % fil, "FAILED"
|..             traceback.print_exc()
|..             print '----------------------------'
|..       except:
|..          print "waaa?"
|.. else:
|..     print "what?"
\__ 
Parse of /home/mwh/src/sf/python/dist/src/Lib/tarfile.py FAILED
Traceback (most recent call last):
  File "<input>", line 7, in ?
ParserError
----------------------------
Parse of /home/mwh/src/sf/python/dist/src/Lib/heapq.py FAILED
Traceback (most recent call last):
  File "<input>", line 7, in ?
ParserError
----------------------------
Parse of /home/mwh/src/sf/python/dist/src/Lib/getopt.py FAILED
Traceback (most recent call last):
  File "<input>", line 7, in ?
ParserError
----------------------------
Parse of /home/mwh/src/sf/python/dist/src/Lib/inspect.py FAILED
Traceback (most recent call last):
  File "<input>", line 7, in ?
ParserError
----------------------------
Parse of /home/mwh/src/sf/python/dist/src/Lib/pydoc.py FAILED
Traceback (most recent call last):
  File "<input>", line 7, in ?
ParserError
----------------------------
Parse of /home/mwh/src/sf/python/dist/src/Lib/lib-tk/Tix.py
FAILED
Traceback (most recent call last):
  File "<input>", line 7, in ?
ParserError
----------------------------
Traceback (most recent call last):
  File "<input>", line 2, in ?
ParserError: Illegal terminal: expected "__assert__"
->> 

Notice that an exception seems to have made it through the
except clauses!  The scary bit is that a (seemingly) random
number of parser errors *will* caught, and then one will get
through.

The only explanation I have for this is some kind of
returning non-NULL with exception set kind of bug.

Off to play with a debug build, I guess.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-07 01:17

Message:
Logged In: YES 
user_id=33168

Michael were these patches ok to check in?

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2003-02-03 11:00

Message:
Logged In: YES 
user_id=6656

Thanks for doing this.  I'll probably check these in after
I've finished crunching my way through monday morning email...

----------------------------------------------------------------------

Comment By: logistix (logistix)
Date: 2003-02-01 21:42

Message:
Logged In: YES 
user_id=699438

And I found a few more by doing a roundtrip 
ast2Tuple/tuple2ast on all files in pythonpath.  These actually 
affect valid AST's, so they're probably a bigger deal.

Valid assert statements were failing.  The patch was trivial, 
so its attached.

Also, files that took advantage of encoding_decl (from PEP 
263) were failing.  I couldn't tell if PEP 263 was offically 
closed, and didn't know what the offical grammar was, so I 
didn't do anything there.

I'll also attach the test script that turned up these errors.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=678518&group_id=5470


From noreply@sourceforge.net  Fri Feb  7 13:46:46 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Fri, 07 Feb 2003 05:46:46 -0800
Subject: [Python-bugs-list] [ python-Bugs-682317 ] plistlib.py selftest fails
Message-ID: <E18h8q6-0005nD-00@sc8-sf-web1.sourceforge.net>

Bugs item #682317, was opened at 2003-02-07 14:46
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=682317&group_id=5470

Category: Macintosh
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Jack Jansen (jackjansen)
Assigned to: Just van Rossum (jvr)
Summary: plistlib.py selftest fails

Initial Comment:
Running the plistlib selftest fails for me: it uses xml.utils.iso8601, which apparently isn't part of the standard library.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=682317&group_id=5470


From noreply@sourceforge.net  Fri Feb  7 14:23:51 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Fri, 07 Feb 2003 06:23:51 -0800
Subject: [Python-bugs-list] [ python-Bugs-678518 ] Another parsermodule validation error
Message-ID: <E18h9Pz-0007Ku-00@sc8-sf-web1.sourceforge.net>

Bugs item #678518, was opened at 2003-01-31 20:23
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=678518&group_id=5470

Category: Parser/Compiler
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: logistix (logistix)
Assigned to: Michael Hudson (mwh)
Summary: Another parsermodule validation error

Initial Comment:
The following code generates an validation error.  It's an 
invalid ast tree where the global_stmt doesn't have the 
keyword 'global' as a child.

I tracked down the fix to validate_global_stmt() this time.

Python 2.3a1 (#38, Dec 31 2002, 17:53:59) [MSC 
v.1200 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more 
information.
>>> import symbol,token,parser
>>> ast = [symbol.file_input,
...  [symbol.stmt, [symbol.simple_stmt, 
[symbol.small_stmt, [symbol.global_stmt,
 [1, 'foo']]], [token.NEWLINE, '']]],
...  [token.NEWLINE, ''],
...  [token.ENDMARKER, '']]
>>> a = parser.sequence2ast(ast)
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
parser.ParserError: VALIDATION FAILURE: report this 
to the maintainer!
>>>

----------------------------------------------------------------------

>Comment By: logistix (logistix)
Date: 2003-02-07 08:23

Message:
Logged In: YES 
user_id=699438

Michael,

At the time I didn't have an encoding patch and punted.  I did 
figure it out last night though.  I'll post it later today since I'm 
at the office now.

And yes, this is the error that'll print out "XXX unhandled 
exception" out of ceval.c when you run the test from debug 
mode.

It comes down to the bastardized way that encoding is 
implemented.  It doesn't fit the current definition of a token or 
a symbol.  There was some code added to add the encoding 
type in the ast2sequence functions, but not the 
sequence2ast functions, so when it hit scripts with 
encoding_decl's, it bombed big time.

Also, as a matter of style, should I add the stuff in my test 
script to the test suite or is that overkill? Should I just have it 
check4 or 5 random files?

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2003-02-07 06:08

Message:
Logged In: YES 
user_id=6656

Pff, I was right, but logistix's patch fix it.

logistix, did you forget to attach the encoding patch?  I
still get errors in module's with a cookie.

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2003-02-07 05:58

Message:
Logged In: YES 
user_id=6656

Patches as such are probably fine.  The reason I haven't
checked them in is a wierdness that I may or may not be
related, but is bothering me.  Consider:

->> import parser, sys,glob,os, traceback
/>> for path in sys.path:
|..     for fil in glob.glob(os.path.join(path,"*.py")):
|..       try:
|..         try:
|..             a = parser.suite(file(fil).read())
|..             b = parser.ast2tuple(a)
|..             c = parser.sequence2ast(b)
|..         except parser.ParserError:
|..             print "Parse of %s" % fil, "FAILED"
|..             traceback.print_exc()
|..             print '----------------------------'
|..       except:
|..          print "waaa?"
|.. else:
|..     print "what?"
\__ 
Parse of /home/mwh/src/sf/python/dist/src/Lib/tarfile.py FAILED
Traceback (most recent call last):
  File "<input>", line 7, in ?
ParserError
----------------------------
Parse of /home/mwh/src/sf/python/dist/src/Lib/heapq.py FAILED
Traceback (most recent call last):
  File "<input>", line 7, in ?
ParserError
----------------------------
Parse of /home/mwh/src/sf/python/dist/src/Lib/getopt.py FAILED
Traceback (most recent call last):
  File "<input>", line 7, in ?
ParserError
----------------------------
Parse of /home/mwh/src/sf/python/dist/src/Lib/inspect.py FAILED
Traceback (most recent call last):
  File "<input>", line 7, in ?
ParserError
----------------------------
Parse of /home/mwh/src/sf/python/dist/src/Lib/pydoc.py FAILED
Traceback (most recent call last):
  File "<input>", line 7, in ?
ParserError
----------------------------
Parse of /home/mwh/src/sf/python/dist/src/Lib/lib-tk/Tix.py
FAILED
Traceback (most recent call last):
  File "<input>", line 7, in ?
ParserError
----------------------------
Traceback (most recent call last):
  File "<input>", line 2, in ?
ParserError: Illegal terminal: expected "__assert__"
->> 

Notice that an exception seems to have made it through the
except clauses!  The scary bit is that a (seemingly) random
number of parser errors *will* caught, and then one will get
through.

The only explanation I have for this is some kind of
returning non-NULL with exception set kind of bug.

Off to play with a debug build, I guess.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-06 19:17

Message:
Logged In: YES 
user_id=33168

Michael were these patches ok to check in?

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2003-02-03 05:00

Message:
Logged In: YES 
user_id=6656

Thanks for doing this.  I'll probably check these in after
I've finished crunching my way through monday morning email...

----------------------------------------------------------------------

Comment By: logistix (logistix)
Date: 2003-02-01 15:42

Message:
Logged In: YES 
user_id=699438

And I found a few more by doing a roundtrip 
ast2Tuple/tuple2ast on all files in pythonpath.  These actually 
affect valid AST's, so they're probably a bigger deal.

Valid assert statements were failing.  The patch was trivial, 
so its attached.

Also, files that took advantage of encoding_decl (from PEP 
263) were failing.  I couldn't tell if PEP 263 was offically 
closed, and didn't know what the offical grammar was, so I 
didn't do anything there.

I'll also attach the test script that turned up these errors.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=678518&group_id=5470


From noreply@sourceforge.net  Fri Feb  7 14:45:25 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Fri, 07 Feb 2003 06:45:25 -0800
Subject: [Python-bugs-list] [ python-Bugs-679520 ] long(3.1415) gives zero on Solaris 8
Message-ID: <E18h9kr-0007yH-00@sc8-sf-web1.sourceforge.net>

Bugs item #679520, was opened at 2003-02-03 14:22
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=679520&group_id=5470

Category: Python Interpreter Core
Group: Platform-specific
Status: Open
Resolution: Works For Me
Priority: 5
Submitted By: Mark Dickinson (marketdickinson)
Assigned to: Nobody/Anonymous (nobody)
Summary: long(3.1415) gives zero on Solaris 8

Initial Comment:
On Solaris 8, running Python 2.2.2, the built-in 
function long() gave me the following peculiar results
when applied to a floating-point argument.

(Unfortunately I don't have administrative access to
this machine, so couldn't test to see whether Python
2.3a has the same problems.)

s0100% uname -a
SunOS s0100.math.lsa.umich.edu 5.8 Generic_108528-14
sun4u sparc SUNW,Sun-Blade-1000
s0100% python
Python 2.2.2 (#22, Dec 23 2002, 12:02:55) 
[GCC 3.0.3] on sunos5
Type "help", "copyright", "credits" or "license" for
more information.
>>> long(3.1415)
0L
>>> _ == 0L
0
>>> long(6e20)
00000000000000000L
>>> _ - 1
37778931862957161709567L
>>> int(3.1415)
3


----------------------------------------------------------------------

>Comment By: Mark Dickinson (marketdickinson)
Date: 2003-02-07 14:45

Message:
Logged In: YES 
user_id=703403

The sysadmin here is (very helpfully) looking into this; 
I'll report any findings.

I should probably also have pointed out in my original
report that this is a 64-bit compile of Python.  (Apologies
if this is obvious to everyone;  it wasn't to me.)





----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-07 01:12

Message:
Logged In: YES 
user_id=33168

Mark, can you verify if this is a compiler problem?  I also
don't have this problem on a sun?

----------------------------------------------------------------------

Comment By: Inyeol Lee (inyeol)
Date: 2003-02-03 19:41

Message:
Logged In: YES 
user_id=595280

My machine has similar configuration, but works fine;

$ uname -a
SunOS abbey 5.8 Generic_108528-18 sun4u sparc
SUNW,Sun-Blade-1000

$ python22
Python 2.2.2 (#1, Oct 15 2002, 14:20:16)
[GCC 2.95.3 20010315 (release)] on sunos5
>>> long(3.1415)
3L
>>> long(6e20)
600000000000000000000L
>>> _ - 1
599999999999999999999L

I think it's gcc/glibc optimization issue.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=679520&group_id=5470


From noreply@sourceforge.net  Fri Feb  7 15:04:07 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Fri, 07 Feb 2003 07:04:07 -0800
Subject: [Python-bugs-list] [ python-Bugs-682317 ] plistlib.py selftest fails
Message-ID: <E18hA2x-0008VI-00@sc8-sf-web2.sourceforge.net>

Bugs item #682317, was opened at 2003-02-07 14:46
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=682317&group_id=5470

Category: Macintosh
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Jack Jansen (jackjansen)
Assigned to: Just van Rossum (jvr)
Summary: plistlib.py selftest fails

Initial Comment:
Running the plistlib selftest fails for me: it uses xml.utils.iso8601, which apparently isn't part of the standard library.

----------------------------------------------------------------------

>Comment By: Just van Rossum (jvr)
Date: 2003-02-07 16:04

Message:
Logged In: YES 
user_id=92689

It's part of PyXML. The doc string explains that Date support depends on PyXML. The test is only an onformal test, would you like to formalize it? Or make Date testing optional?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=682317&group_id=5470


From noreply@sourceforge.net  Fri Feb  7 15:14:21 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Fri, 07 Feb 2003 07:14:21 -0800
Subject: [Python-bugs-list] [ python-Bugs-682215 ] Carbon.CF.CFString should require ASCII
Message-ID: <E18hACr-0002Jo-00@sc8-sf-web3.sourceforge.net>

Bugs item #682215, was opened at 2003-02-07 10:35
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=682215&group_id=5470

Category: Macintosh
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Jack Jansen (jackjansen)
Assigned to: Jack Jansen (jackjansen)
Summary: Carbon.CF.CFString should require ASCII

Initial Comment:
If a Python string is passed where a CFString is expected the bytes are used verbatim (using CFStringCreateWithCString()), but this is probably wrong. We should either allow only ASCII or use sys.getdefaultencoding() or something similar.

This is a copy of PyObjC bug <https://sourceforge.net/tracker/?func=detail&atid=114534&aid=681880&group_id=14534>.

----------------------------------------------------------------------

>Comment By: Just van Rossum (jvr)
Date: 2003-02-07 16:14

Message:
Logged In: YES 
user_id=92689

You can strike the "probably". Although the rest of Python uses the default encoding, I think it's actually wise to just use ASCII and nudge them towards unicode.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=682215&group_id=5470


From noreply@sourceforge.net  Fri Feb  7 15:35:28 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Fri, 07 Feb 2003 07:35:28 -0800
Subject: [Python-bugs-list] [ python-Bugs-679520 ] long(3.1415) gives zero on Solaris 8
Message-ID: <E18hAXI-0002Lp-00@sc8-sf-web1.sourceforge.net>

Bugs item #679520, was opened at 2003-02-03 09:22
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=679520&group_id=5470

Category: Python Interpreter Core
Group: Platform-specific
Status: Open
Resolution: Works For Me
Priority: 5
Submitted By: Mark Dickinson (marketdickinson)
Assigned to: Nobody/Anonymous (nobody)
Summary: long(3.1415) gives zero on Solaris 8

Initial Comment:
On Solaris 8, running Python 2.2.2, the built-in 
function long() gave me the following peculiar results
when applied to a floating-point argument.

(Unfortunately I don't have administrative access to
this machine, so couldn't test to see whether Python
2.3a has the same problems.)

s0100% uname -a
SunOS s0100.math.lsa.umich.edu 5.8 Generic_108528-14
sun4u sparc SUNW,Sun-Blade-1000
s0100% python
Python 2.2.2 (#22, Dec 23 2002, 12:02:55) 
[GCC 3.0.3] on sunos5
Type "help", "copyright", "credits" or "license" for
more information.
>>> long(3.1415)
0L
>>> _ == 0L
0
>>> long(6e20)
00000000000000000L
>>> _ - 1
37778931862957161709567L
>>> int(3.1415)
3


----------------------------------------------------------------------

>Comment By: Tim Peters (tim_one)
Date: 2003-02-07 10:35

Message:
Logged In: YES 
user_id=31435

Python runs on many 64-bit platforms (starting in 1993), so 
that's not likely a real problem.  A compiler optimization bug 
is much more likely, or perhaps a config problem when 
compiling Python.

----------------------------------------------------------------------

Comment By: Mark Dickinson (marketdickinson)
Date: 2003-02-07 09:45

Message:
Logged In: YES 
user_id=703403

The sysadmin here is (very helpfully) looking into this; 
I'll report any findings.

I should probably also have pointed out in my original
report that this is a 64-bit compile of Python.  (Apologies
if this is obvious to everyone;  it wasn't to me.)





----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-06 20:12

Message:
Logged In: YES 
user_id=33168

Mark, can you verify if this is a compiler problem?  I also
don't have this problem on a sun?

----------------------------------------------------------------------

Comment By: Inyeol Lee (inyeol)
Date: 2003-02-03 14:41

Message:
Logged In: YES 
user_id=595280

My machine has similar configuration, but works fine;

$ uname -a
SunOS abbey 5.8 Generic_108528-18 sun4u sparc
SUNW,Sun-Blade-1000

$ python22
Python 2.2.2 (#1, Oct 15 2002, 14:20:16)
[GCC 2.95.3 20010315 (release)] on sunos5
>>> long(3.1415)
3L
>>> long(6e20)
600000000000000000000L
>>> _ - 1
599999999999999999999L

I think it's gcc/glibc optimization issue.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=679520&group_id=5470


From noreply@sourceforge.net  Fri Feb  7 15:30:25 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Fri, 07 Feb 2003 07:30:25 -0800
Subject: [Python-bugs-list] [ python-Bugs-678352 ] list.append memory leak ?
Message-ID: <E18hASP-0006cX-00@sc8-sf-web4.sourceforge.net>

Bugs item #678352, was opened at 2003-01-31 15:18
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=678352&group_id=5470

Category: Python Interpreter Core
Group: None
>Status: Closed
>Resolution: Invalid
Priority: 5
Submitted By: thanos  vassilakis (thanos)
Assigned to: Nobody/Anonymous (nobody)
Summary: list.append memory leak ?

Initial Comment:
# this SEEMS to leak
l = xrange(10000)
while 1:
        a=[]
        map(a.append, l)
        del a

# this does not 
while 1:
       a=range(10000)
       del a



----------------------------------------------------------------------

>Comment By: thanos  vassilakis (thanos)
Date: 2003-02-07 10:30

Message:
Logged In: YES 
user_id=91277

My apologies:
gc was compiled disabled, and probably other things were non
standard.

On rebuilding python the problem went. 

Thanos


----------------------------------------------------------------------

Comment By: thanos  vassilakis (thanos)
Date: 2003-02-01 22:27

Message:
Logged In: YES 
user_id=91277

Good idea

I will try it out on my OSX mac tomorrow (after upgrading !)



----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-01 20:49

Message:
Logged In: YES 
user_id=33168

Was the python version 2.2 or 2.2.2?  If you are using 2.2,
you should probably try to upgrade to 2.2.2 since there have
been many bug fixes.

----------------------------------------------------------------------

Comment By: thanos  vassilakis (thanos)
Date: 2003-01-31 22:07

Message:
Logged In: YES 
user_id=91277

 We use HP UX 11 and BSD 4.7 but what were the origial build compilers and libraries  will have to wait till Monday - when I will be back at work and have access to the NYSE trading  floor systems.

I am chasing some memory leak that occurred when we moved from python 1.5.2 to 2.2


I don't think this is it, but...

Thanks for your quick response.




----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2003-01-31 20:40

Message:
Logged In: YES 
user_id=31435

Again, please identify the Python version and 
OS+compiler+C library in use.  I'm not arguing with you, I'm 
trying to help <wink>.  Since I don't see any evidence of 
anything like I leak when I try it (memory usage didn't 
increase *at all* when I tried it) , the answer to why you see 
it must lie in something you're using that I'm not -- only you 
can tell us what that might be.  I already told you the 
Python version, OS and C compiler/library version I used.

----------------------------------------------------------------------

Comment By: thanos  vassilakis (thanos)
Date: 2003-01-31 17:47

Message:
Logged In: YES 
user_id=91277

HP UX B.11.00
BSD 4.7
python 2.2

Its possible it is not a bug that is why I used the word SEEMS
but why does the second code fragment behave? 


----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2003-01-31 16:00

Message:
Logged In: YES 
user_id=31435

Please identify the Python version and OS+compiler+C 
library in use.  The process size remains steady for me under 
current CVS Python on Win2K w/ MSVC6.

A true leak is possible but unlikely.  More likely is that your 
platform C malloc()/free() decide not to return memory to the 
OS.  If so, that's outside of Python's control

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=678352&group_id=5470


From noreply@sourceforge.net  Fri Feb  7 15:55:09 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Fri, 07 Feb 2003 07:55:09 -0800
Subject: [Python-bugs-list] [ python-Bugs-682317 ] plistlib.py selftest fails
Message-ID: <E18hAqL-0003H1-00@sc8-sf-web1.sourceforge.net>

Bugs item #682317, was opened at 2003-02-07 14:46
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=682317&group_id=5470

Category: Macintosh
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Jack Jansen (jackjansen)
Assigned to: Just van Rossum (jvr)
Summary: plistlib.py selftest fails

Initial Comment:
Running the plistlib selftest fails for me: it uses xml.utils.iso8601, which apparently isn't part of the standard library.

----------------------------------------------------------------------

>Comment By: Jack Jansen (jackjansen)
Date: 2003-02-07 16:55

Message:
Logged In: YES 
user_id=45365

Ah, I see.

I'd like to turn it into a standard test (to go into test.test_plistlib). Then it can also easily test Date() only if the prerequisites are available.

Are you willing to give this a go? If not reassign to me and I'll get to it eventually.

----------------------------------------------------------------------

Comment By: Just van Rossum (jvr)
Date: 2003-02-07 16:04

Message:
Logged In: YES 
user_id=92689

It's part of PyXML. The doc string explains that Date support depends on PyXML. The test is only an onformal test, would you like to formalize it? Or make Date testing optional?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=682317&group_id=5470


From noreply@sourceforge.net  Fri Feb  7 16:04:31 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Fri, 07 Feb 2003 08:04:31 -0800
Subject: [Python-bugs-list] [ python-Bugs-682317 ] plistlib.py selftest fails
Message-ID: <E18hAzP-00050i-00@sc8-sf-web3.sourceforge.net>

Bugs item #682317, was opened at 2003-02-07 14:46
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=682317&group_id=5470

Category: Macintosh
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Jack Jansen (jackjansen)
Assigned to: Just van Rossum (jvr)
Summary: plistlib.py selftest fails

Initial Comment:
Running the plistlib selftest fails for me: it uses xml.utils.iso8601, which apparently isn't part of the standard library.

----------------------------------------------------------------------

>Comment By: Just van Rossum (jvr)
Date: 2003-02-07 17:04

Message:
Logged In: YES 
user_id=92689

I'd like to give this a go at _some_ point, but not now ;-)

Btw. I see xmlrpclib.py does writing of iso8601 dates by itself, but doesn't read them. It's been a while since I read xml.utils.iso8601.py, but if it's not all that much code, perhaps we should just duplicate it in plistlib.py. I have no great use for it, though, plistlib.py is/was mostly targeted to deal with Info.plist files, in which dates are fairly rare...

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-02-07 16:55

Message:
Logged In: YES 
user_id=45365

Ah, I see.

I'd like to turn it into a standard test (to go into test.test_plistlib). Then it can also easily test Date() only if the prerequisites are available.

Are you willing to give this a go? If not reassign to me and I'll get to it eventually.

----------------------------------------------------------------------

Comment By: Just van Rossum (jvr)
Date: 2003-02-07 16:04

Message:
Logged In: YES 
user_id=92689

It's part of PyXML. The doc string explains that Date support depends on PyXML. The test is only an onformal test, would you like to formalize it? Or make Date testing optional?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=682317&group_id=5470


From noreply@sourceforge.net  Fri Feb  7 17:44:14 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Fri, 07 Feb 2003 09:44:14 -0800
Subject: [Python-bugs-list] [ python-Bugs-680429 ] __module__ broken for extension classes
Message-ID: <E18hCXu-0001Fu-00@sc8-sf-web3.sourceforge.net>

Bugs item #680429, was opened at 2003-02-04 11:55
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=680429&group_id=5470

Category: Python Interpreter Core
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Ralf W. Grosse-Kunstleve (rwgk)
Assigned to: Michael Hudson (mwh)
Summary: __module__ broken for extension classes

Initial Comment:
We are having problems using Boost.Python with Python 
2.3a1 because
under some circumstances the result of __module__ is 
different compared
to earlier Python versions:

Python 2.2.1 (#2, Jun 17 2002, 12:06:51) 
[GCC 2.96 20000731 (Red Hat Linux 7.3 2.96-110)] on 
linux2
Type "help", "copyright", "credits" or "license" for more 
information.
>>> import simple
>>> simple.empty.__module__
'simple'
>>> 

Python 2.3a1 (#1, Jan  6 2003, 14:17:56) 
[GCC 2.96 20000731 (Red Hat Linux 7.3 2.96-110)] on 
linux2
Type "help", "copyright", "credits" or "license" for more 
information.
>>> import simple
>>> simple.empty.__module__
'__main__'
>>> 

Because of this we can no longer pickle our extension 
classes.
For your reference the code for the simple module is 
attached.
This is using Boost release 1.29.0 (www.boost.org).

We have done some debugging. Boost.Python's internal 
idea of the module
associated with an extension class is still correct even when 
using
Python 2.3a1. David Abrahams (main Boost.Python author) 
is telling me
that he "changed Boost.Python to work the way Guido 
suggested before
2.2.2." Therefore we suspect that the __module__ problem 
is due to
a change/bug in Python 2.3a1.

Ralf


#include <boost/python/module.hpp>
#include <boost/python/class.hpp>

namespace sandbx { namespace {

  struct empty {};

  void init_module()
  {
    using namespace boost::python;
    class_<empty>("empty");
  }

}} // namespace sandbx::<anonymous>

BOOST_PYTHON_MODULE(simple)
{
  sandbx::init_module();
}


----------------------------------------------------------------------

>Comment By: Ralf W. Grosse-Kunstleve (rwgk)
Date: 2003-02-07 09:44

Message:
Logged In: YES 
user_id=71407

> In summary, if you poke a '__module__' key into the
> boost::python::dict object you pass to the call of the
> metatype and change the first argument to just 'name', I
> think you'll be set.

Yes, that works!

> I pretty sure this will also work with 2.2.x, but if I were
> you I'd check.

I've tested with Python 2.2, 2.2.1, 2.2.2 and 2.3a1. The same
code works perfectly with any of these releases.

Thank you very much for your help!

Ralf


----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2003-02-07 03:29

Message:
Logged In: YES 
user_id=6656

Ah, I start to remember the details now.

In 2.2, if tp_name contained a period, everything to the
left of the period was __module__, that to the right __name__.

However, at some point (maybe even by 2.3) we want

class X:
    class Y:
        pass
print X.Y.__name__

to print 'X.Y' (it prints 'Y' today).  Also we made __name__
assignable for 2.3 and this behaviour caused confusion.

So we made things so that the '__module__' entry in the
type's dict always wins.

In summary, if you poke a '__module__' key into the
boost::python::dict object you pass to the call of the
metatype and change the first argument to just 'name', I
think you'll be set.

I pretty sure this will also work with 2.2.x, but if I were
you I'd check.

HTH!

----------------------------------------------------------------------

Comment By: Ralf W. Grosse-Kunstleve (rwgk)
Date: 2003-02-06 10:36

Message:
Logged In: YES 
user_id=71407

> In 2.2.x, how are you telling Python that __module__ should
> be "simple"?  By setting tp_name to "simple.empty"?  What's
> simple.__name__?

Python 2.3a1 (#2, Feb  5 2003, 09:39:30) 
[GCC 2.96 20000731 (Red Hat Linux 7.3 2.96-110)] on linux2
Type "help", "copyright", "credits" or "license" for more 
information.
>>> import sandbx.simple
>>> sandbx.simple.__name__
'sandbx.simple'
>>> sandbx.simple.empty.__name__
'sandbx.simple.empty'
>>> sandbx.simple.empty.__module__
IS HEAPTYPE
type->tp_dict, __module__ = __main__
'__main__'
>>> 

> I think I know how __module__ is getting there, it's the
> chunk of code currently at typeobject.c:1750.

Yes, that's right. I've verified this by adding more print statements.

> Can you point me to the bit of the boost source that creates
> the typeobject?

Here is where it happens (file 
boost/libs/python/src/object/class.cpp):

object result = object(class_metatype())(module_prefix() + name, 
bases, d);

With print statements I've verified that module_prefix() + name is
correctly passed in as "sandbx.simple.empty" .

I am attaching the file class.cpp. Could it be that we have to
update our class_metatype_object?

Thanks!
        Ralf


----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2003-02-06 03:22

Message:
Logged In: YES 
user_id=6656

OK, thanks for the info.  More questions, I'm afraid:

In 2.2.x, how are you telling Python that __module__ should
be "simple"?  By setting tp_name to "simple.empty"?  What's
simple.__name__?

I think I know how __module__ is getting there, it's the
chunk of code currently at typeobject.c:1750.

Can you point me to the bit of the boost source that creates
the typeobject?  'cept sf's just fallen off the net.

----------------------------------------------------------------------

Comment By: Ralf W. Grosse-Kunstleve (rwgk)
Date: 2003-02-05 10:25

Message:
Logged In: YES 
user_id=71407

> Look at typeobject.c:type_module() (about 100 lines in).
>
> Is simple.empty a HEAPTYPE?

Yes, it is a HEAPTYPE. I've established this by adding print 
statements
in typeobject.c:type_module():

static PyObject *
type_module(PyTypeObject *type, void *context)
{
        PyObject *mod;
        char *s;

        if (type->tp_flags & Py_TPFLAGS_HEAPTYPE) {
printf("IS HEAPTYPE\n");
                mod = PyDict_GetItemString(type-
>tp_dict, "__module__");
                Py_XINCREF(mod);
                s = PyString_AsString(mod);
printf("type->tp_dict, __module__ = %s\n", s);
                return mod;
        }

Result:

Python 2.3a1 (#2, Feb  5 2003, 09:39:30) 
[GCC 2.96 20000731 (Red Hat Linux 7.3 2.96-110)] on linux2
Type "help", "copyright", "credits" or "license" for more 
information.
>>> import simple
>>> simple.empty.__module__
IS HEAPTYPE
type->tp_dict, __module__ = __main__
'__main__'
>>> 

> If you can tell me what you want that function to do, I can
> have a go at it.

We expect "simple" as the result of simple.empty.__module__.

Current result if simple.so is moved to a package:

Python 2.3a1 (#2, Feb  5 2003, 09:39:30) 
[GCC 2.96 20000731 (Red Hat Linux 7.3 2.96-110)] on linux2
Type "help", "copyright", "credits" or "license" for more 
information.
>>> from sandbx import simple
>>> simple.empty.__module__
IS HEAPTYPE
type->tp_dict, __module__ = __main__
'__main__'
>>> 

Here we expect "sandbx.simple" .
Python 2.2.x produces the expected results using the exact same
Boost.Python source code.

David Abrahams provides this additional information:

  Remember that simple.empty is created by calling the metatype, 
so
  it's created on the heap, but it's Python's internal type creation
  mechanisms which do it -- we're not setting the flags manually.

Thanks!
        Ralf


----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2003-02-05 06:10

Message:
Logged In: YES 
user_id=6656

There has certainly been a change here.

Look at typeobject.c:type_module() (about 100 lines in).

If you can tell me what you want that function to do, I can
have a go at it.

Is simple.empty a HEAPTYPE?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=680429&group_id=5470


From noreply@sourceforge.net  Fri Feb  7 17:48:55 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Fri, 07 Feb 2003 09:48:55 -0800
Subject: [Python-bugs-list] [ python-Bugs-680429 ] __module__ broken for extension classes
Message-ID: <E18hCcR-0001Rj-00@sc8-sf-web3.sourceforge.net>

Bugs item #680429, was opened at 2003-02-04 19:55
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=680429&group_id=5470

>Category: Documentation
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Ralf W. Grosse-Kunstleve (rwgk)
Assigned to: Michael Hudson (mwh)
Summary: __module__ broken for extension classes

Initial Comment:
We are having problems using Boost.Python with Python 
2.3a1 because
under some circumstances the result of __module__ is 
different compared
to earlier Python versions:

Python 2.2.1 (#2, Jun 17 2002, 12:06:51) 
[GCC 2.96 20000731 (Red Hat Linux 7.3 2.96-110)] on 
linux2
Type "help", "copyright", "credits" or "license" for more 
information.
>>> import simple
>>> simple.empty.__module__
'simple'
>>> 

Python 2.3a1 (#1, Jan  6 2003, 14:17:56) 
[GCC 2.96 20000731 (Red Hat Linux 7.3 2.96-110)] on 
linux2
Type "help", "copyright", "credits" or "license" for more 
information.
>>> import simple
>>> simple.empty.__module__
'__main__'
>>> 

Because of this we can no longer pickle our extension 
classes.
For your reference the code for the simple module is 
attached.
This is using Boost release 1.29.0 (www.boost.org).

We have done some debugging. Boost.Python's internal 
idea of the module
associated with an extension class is still correct even when 
using
Python 2.3a1. David Abrahams (main Boost.Python author) 
is telling me
that he "changed Boost.Python to work the way Guido 
suggested before
2.2.2." Therefore we suspect that the __module__ problem 
is due to
a change/bug in Python 2.3a1.

Ralf


#include <boost/python/module.hpp>
#include <boost/python/class.hpp>

namespace sandbx { namespace {

  struct empty {};

  void init_module()
  {
    using namespace boost::python;
    class_<empty>("empty");
  }

}} // namespace sandbx::<anonymous>

BOOST_PYTHON_MODULE(simple)
{
  sandbx::init_module();
}


----------------------------------------------------------------------

>Comment By: Michael Hudson (mwh)
Date: 2003-02-07 17:48

Message:
Logged In: YES 
user_id=6656

No problem.  Glad it got sorted.

I'll leave this open, but mark it as a doc bug -- there
should probably be something in whatsnew about this, and
probably somewhere in the reference docs, but I've no idea
where.

----------------------------------------------------------------------

Comment By: Ralf W. Grosse-Kunstleve (rwgk)
Date: 2003-02-07 17:44

Message:
Logged In: YES 
user_id=71407

> In summary, if you poke a '__module__' key into the
> boost::python::dict object you pass to the call of the
> metatype and change the first argument to just 'name', I
> think you'll be set.

Yes, that works!

> I pretty sure this will also work with 2.2.x, but if I were
> you I'd check.

I've tested with Python 2.2, 2.2.1, 2.2.2 and 2.3a1. The same
code works perfectly with any of these releases.

Thank you very much for your help!

Ralf


----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2003-02-07 11:29

Message:
Logged In: YES 
user_id=6656

Ah, I start to remember the details now.

In 2.2, if tp_name contained a period, everything to the
left of the period was __module__, that to the right __name__.

However, at some point (maybe even by 2.3) we want

class X:
    class Y:
        pass
print X.Y.__name__

to print 'X.Y' (it prints 'Y' today).  Also we made __name__
assignable for 2.3 and this behaviour caused confusion.

So we made things so that the '__module__' entry in the
type's dict always wins.

In summary, if you poke a '__module__' key into the
boost::python::dict object you pass to the call of the
metatype and change the first argument to just 'name', I
think you'll be set.

I pretty sure this will also work with 2.2.x, but if I were
you I'd check.

HTH!

----------------------------------------------------------------------

Comment By: Ralf W. Grosse-Kunstleve (rwgk)
Date: 2003-02-06 18:36

Message:
Logged In: YES 
user_id=71407

> In 2.2.x, how are you telling Python that __module__ should
> be "simple"?  By setting tp_name to "simple.empty"?  What's
> simple.__name__?

Python 2.3a1 (#2, Feb  5 2003, 09:39:30) 
[GCC 2.96 20000731 (Red Hat Linux 7.3 2.96-110)] on linux2
Type "help", "copyright", "credits" or "license" for more 
information.
>>> import sandbx.simple
>>> sandbx.simple.__name__
'sandbx.simple'
>>> sandbx.simple.empty.__name__
'sandbx.simple.empty'
>>> sandbx.simple.empty.__module__
IS HEAPTYPE
type->tp_dict, __module__ = __main__
'__main__'
>>> 

> I think I know how __module__ is getting there, it's the
> chunk of code currently at typeobject.c:1750.

Yes, that's right. I've verified this by adding more print statements.

> Can you point me to the bit of the boost source that creates
> the typeobject?

Here is where it happens (file 
boost/libs/python/src/object/class.cpp):

object result = object(class_metatype())(module_prefix() + name, 
bases, d);

With print statements I've verified that module_prefix() + name is
correctly passed in as "sandbx.simple.empty" .

I am attaching the file class.cpp. Could it be that we have to
update our class_metatype_object?

Thanks!
        Ralf


----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2003-02-06 11:22

Message:
Logged In: YES 
user_id=6656

OK, thanks for the info.  More questions, I'm afraid:

In 2.2.x, how are you telling Python that __module__ should
be "simple"?  By setting tp_name to "simple.empty"?  What's
simple.__name__?

I think I know how __module__ is getting there, it's the
chunk of code currently at typeobject.c:1750.

Can you point me to the bit of the boost source that creates
the typeobject?  'cept sf's just fallen off the net.

----------------------------------------------------------------------

Comment By: Ralf W. Grosse-Kunstleve (rwgk)
Date: 2003-02-05 18:25

Message:
Logged In: YES 
user_id=71407

> Look at typeobject.c:type_module() (about 100 lines in).
>
> Is simple.empty a HEAPTYPE?

Yes, it is a HEAPTYPE. I've established this by adding print 
statements
in typeobject.c:type_module():

static PyObject *
type_module(PyTypeObject *type, void *context)
{
        PyObject *mod;
        char *s;

        if (type->tp_flags & Py_TPFLAGS_HEAPTYPE) {
printf("IS HEAPTYPE\n");
                mod = PyDict_GetItemString(type-
>tp_dict, "__module__");
                Py_XINCREF(mod);
                s = PyString_AsString(mod);
printf("type->tp_dict, __module__ = %s\n", s);
                return mod;
        }

Result:

Python 2.3a1 (#2, Feb  5 2003, 09:39:30) 
[GCC 2.96 20000731 (Red Hat Linux 7.3 2.96-110)] on linux2
Type "help", "copyright", "credits" or "license" for more 
information.
>>> import simple
>>> simple.empty.__module__
IS HEAPTYPE
type->tp_dict, __module__ = __main__
'__main__'
>>> 

> If you can tell me what you want that function to do, I can
> have a go at it.

We expect "simple" as the result of simple.empty.__module__.

Current result if simple.so is moved to a package:

Python 2.3a1 (#2, Feb  5 2003, 09:39:30) 
[GCC 2.96 20000731 (Red Hat Linux 7.3 2.96-110)] on linux2
Type "help", "copyright", "credits" or "license" for more 
information.
>>> from sandbx import simple
>>> simple.empty.__module__
IS HEAPTYPE
type->tp_dict, __module__ = __main__
'__main__'
>>> 

Here we expect "sandbx.simple" .
Python 2.2.x produces the expected results using the exact same
Boost.Python source code.

David Abrahams provides this additional information:

  Remember that simple.empty is created by calling the metatype, 
so
  it's created on the heap, but it's Python's internal type creation
  mechanisms which do it -- we're not setting the flags manually.

Thanks!
        Ralf


----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2003-02-05 14:10

Message:
Logged In: YES 
user_id=6656

There has certainly been a change here.

Look at typeobject.c:type_module() (about 100 lines in).

If you can tell me what you want that function to do, I can
have a go at it.

Is simple.empty a HEAPTYPE?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=680429&group_id=5470


From noreply@sourceforge.net  Fri Feb  7 19:06:12 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Fri, 07 Feb 2003 11:06:12 -0800
Subject: [Python-bugs-list] [ python-Bugs-682504 ] email: preamble must be \n terminated
Message-ID: <E18hDpE-0003g0-00@sc8-sf-web2.sourceforge.net>

Bugs item #682504, was opened at 2003-02-07 14:06
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=682504&group_id=5470

Category: Python Library
Group: Python 2.2.2
Status: Open
Resolution: None
Priority: 5
Submitted By: Andreas Jung (ajung)
Assigned to: Nobody/Anonymous (nobody)
Summary: email: preamble must be \n terminated

Initial Comment:
when setting the preamble attribute, one must ensure
that the attribute is terminated by "\n". Otherwise 
the boundary 
string of the first attachment is output directly
behind the
preample and all email programs ignore the first
attachment.

Solution: add a trailing "\n" to the preample

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=682504&group_id=5470


From noreply@sourceforge.net  Fri Feb  7 19:24:27 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Fri, 07 Feb 2003 11:24:27 -0800
Subject: [Python-bugs-list] [ python-Bugs-682504 ] email: preamble must be \n terminated
Message-ID: <E18hE6t-00048D-00@sc8-sf-web2.sourceforge.net>

Bugs item #682504, was opened at 2003-02-07 14:06
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=682504&group_id=5470

Category: Python Library
Group: Python 2.2.2
Status: Open
Resolution: None
Priority: 5
Submitted By: Andreas Jung (ajung)
>Assigned to: Barry A. Warsaw (bwarsaw)
Summary: email: preamble must be \n terminated

Initial Comment:
when setting the preamble attribute, one must ensure
that the attribute is terminated by "\n". Otherwise 
the boundary 
string of the first attachment is output directly
behind the
preample and all email programs ignore the first
attachment.

Solution: add a trailing "\n" to the preample

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=682504&group_id=5470


From noreply@sourceforge.net  Fri Feb  7 20:20:04 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Fri, 07 Feb 2003 12:20:04 -0800
Subject: [Python-bugs-list] [ python-Bugs-501716 ] "es#" parser marker leaks memory
Message-ID: <E18hEyi-0007Vc-00@sc8-sf-web1.sourceforge.net>

Bugs item #501716, was opened at 2002-01-10 10:39
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=501716&group_id=5470

Category: Unicode
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: M.-A. Lemburg (lemburg)
Assigned to: M.-A. Lemburg (lemburg)
Summary: "es#" parser marker leaks memory

Initial Comment:
... if subsequent parsing fails, a buffer which was possibly 
allocated by "es#" is not freed.


----------------------------------------------------------------------

Comment By: Michael Stone (mbrierst)
Date: 2003-02-07 20:20

Message:
Logged In: YES 
user_id=670441

In what case does this happen?  The only case
I can see is if the string is successfully encoded,
but the encoding results in a non-null terminated 
string.  That might cause problems, otherwise it
looks okay?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=501716&group_id=5470


From noreply@sourceforge.net  Fri Feb  7 20:57:44 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Fri, 07 Feb 2003 12:57:44 -0800
Subject: [Python-bugs-list] [ python-Bugs-642168 ] help() fails for some builtin topics
Message-ID: <E18hFZA-0008RE-00@sc8-sf-web2.sourceforge.net>

Bugs item #642168, was opened at 2002-11-22 02:54
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=642168&group_id=5470

Category: Documentation
Group: Python 2.3
>Status: Closed
>Resolution: Fixed
Priority: 5
Submitted By: Matthias Klose (doko)
>Assigned to: Neal Norwitz (nnorwitz)
Summary: help() fails for some builtin topics

Initial Comment:
[seen with 2.2.2 and current CVS]

Python's help() function can display some manual
sections, in addition 
to the docstrings of various modules and functions. 
However, for 
some (not all) things this fails. 

Python 2.2.2 (#4, Oct 15 2002, 04:21:28) 
[GCC 2.95.4 20011002 (Debian prerelease)] on linux2
Type "help", "copyright", "credits" or "license" for
more information.

help> and
could not read docs from
/usr/share/doc/python2.2-doc/html/ref/lambda.html

help> topics

Here is a list of available topics.  Enter any topic
name to get more
help.

ASSERTION           DEBUGGING           LITERALS
SEQUENCEMETHODS1
ASSIGNMENT          DELETION            LOOPING
SEQUENCEMETHODS2
ATTRIBUTEMETHODS    DICTIONARIES        MAPPINGMETHODS
     SEQUENCES
ATTRIBUTES          DICTIONARYLITERALS  MAPPINGS      
     SHIFTING
AUGMENTEDASSIGNMENT ELLIPSIS            METHODS       
     SLICINGS
BACKQUOTES          EXCEPTIONS          MODULES
SPECIALATTRIBUTES
BASICMETHODS        EXECUTION           NAMESPACES
SPECIALIDENTIFIERS
BINARY              EXPRESSIONS         NONE
SPECIALMETHODS
BITWISE             FILES               NUMBERMETHODS
STRINGMETHODS
BOOLEAN             FLOAT               NUMBERS       
     STRINGS
CALLABLEMETHODS     FORMATTING          OBJECTS       
     SUBSCRIPTS
CALLS               FRAMEOBJECTS        OPERATORS     
     TRACEBACKS
CLASSES             FRAMES              PACKAGES      
     TRUTHVALUE
CODEOBJECTS         FUNCTIONS           POWER
TUPLELITERALS
COERCIONS           IDENTIFIERS         PRECEDENCE    
     TUPLES
COMPARISON          IMPORTING           PRINTING      
     TYPEOBJECTS
COMPLEX             INTEGER             PRIVATENAMES  
     TYPES
CONDITIONAL         LISTLITERALS        RETURNING     
     UNARY
CONVERSIONS         LISTS               SCOPING       
     UNICODE

help> spam
no Python documentation found for 'spam'

help> TRUTHVALUE

Related help topics: if, while, and, or, not, BASICMETHODS

help> COERCIONS

help> UNICODE
could not read docs from
/usr/share/doc/python2.2-doc/html/ref/unicode.html




----------------------------------------------------------------------

>Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-07 15:57

Message:
Logged In: YES 
user_id=33168

Fixed for UNICODE (same as string), and/or/not (same as
BOOLEANS).  I don't know of any other problems, so I'm
closing this bug.

Checked in as:
  * Lib/pydoc.py 1.76 and 1.56.8.8

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=642168&group_id=5470


From noreply@sourceforge.net  Fri Feb  7 22:01:46 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Fri, 07 Feb 2003 14:01:46 -0800
Subject: [Python-bugs-list] [ python-Bugs-624860 ] help(UNICODE) -- reference page missing
Message-ID: <E18hGZ8-00048o-00@sc8-sf-web3.sourceforge.net>

Bugs item #624860, was opened at 2002-10-17 16:20
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=624860&group_id=5470

Category: Documentation
Group: Python 2.2.2
>Status: Closed
>Resolution: Fixed
Priority: 5
Submitted By: Scott David Daniels (scott_daniels)
>Assigned to: Neal Norwitz (nnorwitz)
Summary: help(UNICODE) -- reference page missing

Initial Comment:
Entering IDLE in 2.2.2 on Win2K:
    >>> help()
    help> topics
Lists (among many others), UNICODE.  Howver,
    help> UNICODE
elicits the error:
    could not read docs 
    from C:\PYTHON22\doc/ref/unicode.html

It seemed the only entry in topics that did fail,
but I am not certain this is so.


----------------------------------------------------------------------

>Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-07 17:01

Message:
Logged In: YES 
user_id=33168

This was fixed recently by change for bug #642168.

----------------------------------------------------------------------

Comment By: Fred L. Drake, Jr. (fdrake)
Date: 2002-10-24 17:15

Message:
Logged In: YES 
user_id=3066

I've looked into this briefly, and it looks like this is
caused by the pydoc module (which implements the interactive
help facility) having a static mapping of topic names to
HTML files.  Since this list is static in the code, and not
generated from the documentation in any way, it's out of
date.  That section (which was available for Python 2.1.x)
was never filled in with anything more than "XXX explain
more here...".

It's not at all clear what documentation this should point
to; I suspect it's material that still needs to be written.
 I'll open a separate bug on the matter that the list of
topics is not integrated with the documentation maintenance
-- it should be so we're more aware of this mapping and
don't break it accidentally.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=624860&group_id=5470


From noreply@sourceforge.net  Fri Feb  7 22:11:28 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Fri, 07 Feb 2003 14:11:28 -0800
Subject: [Python-bugs-list] [ python-Bugs-611052 ] SMTP.login() uses invalid base64 enc.
Message-ID: <E18hGiW-0000Fv-00@sc8-sf-web4.sourceforge.net>

Bugs item #611052, was opened at 2002-09-18 07:52
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=611052&group_id=5470

Category: Extension Modules
Group: Python 2.2.1
Status: Open
Resolution: None
Priority: 5
Submitted By: Ondrej Sury (ondrass)
>Assigned to: Gerhard Häring (ghaering)
Summary: SMTP.login() uses invalid base64 enc.

Initial Comment:
If you have veeery long username and/or password it encodes
it with '\n' after 80? characters.  This violates
(IMHO) smtp standard.
In following example look for \n characters.

O.

>>> from smtplib import SMTP
>>> s = SMTP('smtp.servery.cz')
>>> s.set_debuglevel(255)
>>>
s.login('veeeery.looooong.username@veeeeery.loooooong.domain',
'veeeeery.loooong.password')
send: 'ehlo shade.globe.cz\r\n'
reply: '250-smtp.cz\r\n'
reply: '250-AUTH LOGIN PLAIN\r\n'
reply: '250-AUTH=LOGIN PLAIN\r\n'
reply: '250-PIPELINING\r\n'
reply: '250-STARTTLS\r\n'
reply: '250 8BITMIME\r\n'
reply: retcode (250); Msg: smtp.cz
AUTH LOGIN PLAIN
AUTH=LOGIN PLAIN
PIPELINING
STARTTLS
8BITMIME
AuthMethod: PLAIN
send: 'AUTH PLAIN
dmVlZWVyeS5sb29vb29uZy51c2VybmFtZUB2ZWVlZWVyeS5sb29vb29vbmcuZG9tYWluAHZlZWVl\ncnkubG9vb29vbmcudXNlcm5hbWVAdmVlZWVlcnkubG9vb29vb25nLmRvbWFpbgB2ZWVlZWVyeS5s\nb29vb25nLnBhc3N3b3Jk\r\n'
reply: '501 malformed auth input (#5.5.4)\r\n'
reply: retcode (501); Msg: malformed auth input (#5.5.4)
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "/usr/lib/python2.2/smtplib.py", line 546, in login
    raise SMTPAuthenticationError(code, resp)
smtplib.SMTPAuthenticationError: (501, 'malformed auth
input (#5.5.4)')


----------------------------------------------------------------------

>Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-07 17:11

Message:
Logged In: YES 
user_id=33168

Gerhard, can this be closed now?

----------------------------------------------------------------------

Comment By: Gerhard Häring (ghaering)
Date: 2002-10-14 12:04

Message:
Logged In: YES 
user_id=163326

Thanks for the bug report. This has already been fixed in
Python 2.3-CVS and the 2.2 maintenance branch. So just
upgrade to Python 2.2.2 when it'll be out tomorrow :-)

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=611052&group_id=5470


From noreply@sourceforge.net  Fri Feb  7 22:14:08 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Fri, 07 Feb 2003 14:14:08 -0800
Subject: [Python-bugs-list] [ python-Bugs-594998 ] test_nis test fails on TRU64 5.1
Message-ID: <E18hGl6-0004c2-00@sc8-sf-web3.sourceforge.net>

Bugs item #594998, was opened at 2002-08-14 06:53
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=594998&group_id=5470

Category: Build
Group: Python 2.2.1
Status: Open
Resolution: None
Priority: 5
Submitted By: Patrick Finnerty (finnertyp)
Assigned to: Nobody/Anonymous (nobody)
Summary: test_nis test fails on TRU64 5.1

Initial Comment:
Platform: TRU64 v5.1a

Ran configure with --with-dec-threads options. 
Also used --without-gcc although this has no impact on
test failure.

The ouput from ./python ./Lib/test/test_nis.py is large
so I've included it in an attached file rather than
pasted here.

The machine that Python is being built on is a NIS
server as well as client.



----------------------------------------------------------------------

>Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-07 17:14

Message:
Logged In: YES 
user_id=33168

Patrick, are you able to test if this has been fixed?

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2002-12-14 20:49

Message:
Logged In: YES 
user_id=33168

I fixed a similar problem on Nov 4.  Can you test the latest
CVS versions from the head or for 2.2.2+?

----------------------------------------------------------------------

Comment By: Patrick Finnerty (finnertyp)
Date: 2002-08-15 09:06

Message:
Logged In: YES 
user_id=594846

On TRU64 this is doucmented as


       foreach(instatus, inkey, inkeylen, inval, invallen,
indata);
       int instatus;
       char *inkey;
       int inkeylen;
       char *inval;
       int invallen;
       char *indata;


----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2002-08-15 07:05

Message:
Logged In: YES 
user_id=21627

This sounds like some serious memory corruption is going on.
Unfortunately, I can't spot a problem, so somebody with
access to such a system would need to debug it.

My first guess is that the type of the callback function is
incorrect. Can you please find out what signatures are
involved in yp_all(3)? On Solaris, the callback signature is
not declared in a header, but only documented in
yp_clnt(3NSL), as

                foreach(int instatus, char *inkey,
                int inkeylen, char *inval,
                int invallen, char *indata);


----------------------------------------------------------------------

Comment By: Patrick Finnerty (finnertyp)
Date: 2002-08-15 06:02

Message:
Logged In: YES 
user_id=594846

Uploading the output file from ./python ./Lib/test/test_nis.py.

----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2002-08-15 05:01

Message:
Logged In: YES 
user_id=21627

There's no uploaded file!  You have to check the
checkbox labeled "Check to Upload & Attach File"
when you upload a file.

Please try again.

(This is a SourceForge annoyance that we can do
nothing about. :-( )

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=594998&group_id=5470


From noreply@sourceforge.net  Fri Feb  7 22:18:49 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Fri, 07 Feb 2003 14:18:49 -0800
Subject: [Python-bugs-list] [ python-Bugs-589427 ] standard include paths on command line
Message-ID: <E18hGpd-0003LQ-00@sc8-sf-web2.sourceforge.net>

Bugs item #589427, was opened at 2002-07-31 20:13
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=589427&group_id=5470

Category: Build
Group: Python 2.3
Status: Open
Resolution: Remind
Priority: 5
Submitted By: Matthias Klose (doko)
Assigned to: Skip Montanaro (montanaro)
Summary: standard include paths on command line

Initial Comment:
when compiling shared modules (using gcc-3.1.1), 
standard include paths are included (by setup.py?) in 
the include path, which can change the search order. 
gcc-3.1 warns about:

building 'xreadlines' extension
gcc-3.1 -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -
fPIC -I. -I/home/packages/pyt
hon2.3/python2.3-2.2.90/./Include -I/usr/include -
I/usr/local/include -I/home/pa
ckages/python2.3/python2.3-2.2.90/Include -
I/home/packages/python2.3/python2.3-2
.2.90/build-static -
c /home/packages/python2.3/python2.3-
2.2.90/Modules/xreadlin
esmodule.c -o build/temp.linux-i686-
2.3/xreadlinesmodule.o
gcc -shared build/temp.linux-i686-
2.3/xreadlinesmodule.o -L/usr/lib -L/usr/local
/lib -o build/lib.linux-i686-2.3/xreadlines.so
building 'array' extension
gcc-3.1 -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -
fPIC -I. -I/hocc1: warning: c
hanging search order for system 
directory "/usr/local/include"
cc1: warning:   as it has already been specified as a non-
system directory
cc1: warning: changing search order for system 
directory "/usr/include"
cc1: warning:   as it has already been specified as a non-
system directory
cc1: warning: changing search order for system 
directory "/usr/local/include"
cc1: warning:   as it has already been specified as a non-
system directory
cc1: warning: changing search order for system 
directory "/usr/include"
cc1: warning:   as it has already been specified as a non-
system directory
cc1: warning: changing search order for system 
directory "/usr/local/include"
cc1: warning:   as it has already been specified as a non-
system directory

In this case, /usr/include and /usr/local/include should 
not be included on the command line.


----------------------------------------------------------------------

>Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-07 17:18

Message:
Logged In: YES 
user_id=33168

Matthias, I seem to recall that the warnings were due to a
change in gcc or a bug.  I think I heard that it would be
fixed in gcc 3.2.2.  Do you still have any problems?

----------------------------------------------------------------------

Comment By: Matthias Klose (doko)
Date: 2002-08-20 17:32

Message:
Logged In: YES 
user_id=60903

Two more problems:

On Debian, the db headers are installed in /usr/include.

- This directory is not searched and bsddb not built.

- Adding '/usr/include' to setup.py in the db setup,
   the header is found, but /usr/include added to
   the command line.

One unrelated thing is that the bsddb setup adds
-R/usr/lib, which is not support on Linux by gcc-3.2.


----------------------------------------------------------------------

Comment By: Skip Montanaro (montanaro)
Date: 2002-08-13 22:29

Message:
Logged In: YES 
user_id=44345

Doko, are you still having problems?  I really have nothing to go on here.  I 
don't see /usr/include turning up on the command line and don't have a 
Debian system to try things out on. (Does the SF compile farm?)

Skip


----------------------------------------------------------------------

Comment By: Skip Montanaro (montanaro)
Date: 2002-08-05 11:25

Message:
Logged In: YES 
user_id=44345

Sorry guys, I'm coming in on this late.  

I'm done with the bsddb trickery.  In any case /usr/include is not added 
as a -I argument under any circumstances in the bsddb build section of 
setup.py.  Various Linux vendors allow you to install multiple versions of 
Berkeley DB simultaneously.  The only way you can guarantee that a 
particular version is compiled against and linked with is to add the 
appropriate -I and -L flags to the command line.  On RedHat-derived 
systems the include files are in /usr/include/db{1,2,3,4} if installed as part 
of a system install, while if you install from Sleepycat source I believe it's 
/usr/local/BerkeleyDB<version>/include.   I can't really get rid of the 
RH-related include directories. 

Wait a minute.  I take some of that back.  *If* you are foolish enough to 
uncomment the section of code which enables compilation and linkage 
with Berkeley DB 1.85, you might wind up with -I/usr/include on the 
command line to build the bsddb module.  I will correct that (make it so 
/usr/include isn't added to the -I flag) and also reorder the directory 
search so system include directories are scanned after non-system 
directories.

How's this going to help the OP?  I thought the problem he saw was 
more pervasive (xreadlines, _ssl, etc).



----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2002-08-05 10:51

Message:
Logged In: YES 
user_id=6656

Skip, are you still working on the bsddb related bits of
setup.py?  Can you look into how /usr/include is getting
onto the commandline?


----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2002-08-04 08:47

Message:
Logged In: YES 
user_id=21627

I can't reproduce this. With the current CVS, on Debian,
with --prefix=/usr, I get

gcc -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -fPIC
-DWITH_APPINIT=1 -I/usr/include/tcl8.3 -I/usr/X11R6/include
-I. -I/vol/marvin-vol8/loewis/python/dist/src/./Include
-I/usr/local/include
-I/vol/marvin-vol8/loewis/python/dist/src/Include
-I/vol/marvin-vol8/loewis/python/dist/src -c
/vol/marvin-vol8/loewis/python/dist/src/Modules/_tkinter.c
-o build/temp.linux-i686-2.3/_tkinter.o

So it won't add -I/usr/include to the command line options.

----------------------------------------------------------------------

Comment By: Matthias Klose (doko)
Date: 2002-08-03 12:30

Message:
Logged In: YES 
user_id=60903

No. It's:

 - bsddb
 - dbm
 - _tkinter

all modules, which are installed in /usr at least on Debian
GNU/Linux (but maybe on other distros as well).

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2002-08-03 12:19

Message:
Logged In: YES 
user_id=6656

It's only the _ssl module, right?  I've fixed that.

The -R/usr/lib and -L/usr/lib's come from various database
modules, the configuration of which is under review anyway.
 I hope those doing the reviewing have seen this bug traffic...

----------------------------------------------------------------------

Comment By: Matthias Klose (doko)
Date: 2002-08-03 11:56

Message:
Logged In: YES 
user_id=60903

build log attached. please tell me if you need an account on
a Debian machine (doko   debian org).
configure \
        --with-gcc=gcc-3.1 \
        --with-cxx=g++-3.1 \
        --prefix=/usr \
        --with-fpectl \
        --enable-ipv6


----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2002-08-03 11:12

Message:
Logged In: YES 
user_id=6656

Really?  Nuts.  They're not on my machine.

What arguments are you passing to configure?

----------------------------------------------------------------------

Comment By: Matthias Klose (doko)
Date: 2002-08-02 17:48

Message:
Logged In: YES 
user_id=60903

Thanks for the fix, however when building shared modules,
-I/usr/include and -L/usr/lib are still passed to the compiler.


----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2002-08-02 10:05

Message:
Logged In: YES 
user_id=21627

And indeed, we should not remove /usr/local/include, since
the system compiler may not be gcc.

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2002-08-02 09:56

Message:
Logged In: YES 
user_id=6656

OK, done, in

setup.py revision 1.100

I don't see us removing /usr/local/include from the command
line.

----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2002-08-02 09:49

Message:
Logged In: YES 
user_id=21627

The patch looks fine to me, please apply it.

I still think that the bogus warning should be removed from
gcc entirely.

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2002-08-02 08:54

Message:
Logged In: YES 
user_id=6656

OK, can you try the attached patch?

Martin, does this look OK to you?

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2002-08-02 04:53

Message:
Logged In: YES 
user_id=6656

Ah, sorry!

Hmm.  Now I don't see the problem, i.e. for me
-I/usr/include is not part of the command line passed to the
compiler.

Are you building from up-to-date CVS?  Though I don't recall
any recent changes in this area.  Ahh, you're running
".../configure --prefix=/usr", right?

I'll dig.

----------------------------------------------------------------------

Comment By: Matthias Klose (doko)
Date: 2002-08-01 16:23

Message:
Logged In: YES 
user_id=60903

I disagree. The discussion is on /usr/local/include. We are 
talking about /usr/include, which is a system include dir on 
almost all systems.

See the thread "gcc 3.2's cpp breaks configure scripts" at 
http://gcc.gnu.org/ml/gcc/2002-07/, i.e.:

http://gcc.gnu.org/ml/gcc/2002-07/msg01527.html

Inclusion of /usr/include can lead to errors, as explained in:
http://gcc.gnu.org/ml/gcc/2002-07/msg01541.html

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2002-08-01 05:08

Message:
Logged In: YES 
user_id=6656

This has been deemed a bug in gcc, see this thread:

http://mail.python.org/pipermail/python-dev/2002-June/025990.html

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=589427&group_id=5470


From noreply@sourceforge.net  Fri Feb  7 22:20:48 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Fri, 07 Feb 2003 14:20:48 -0800
Subject: [Python-bugs-list] [ python-Bugs-585923 ] macfs.FSSpec fails for "new" files
Message-ID: <E18hGrY-0004uu-00@sc8-sf-web3.sourceforge.net>

Bugs item #585923, was opened at 2002-07-24 09:35
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=585923&group_id=5470

Category: Macintosh
Group: Python 2.3
Status: Open
Resolution: None
Priority: 7
Submitted By: Just van Rossum (jvr)
Assigned to: Jack Jansen (jackjansen)
>Summary: macfs.FSSpec fails for "new" files

Initial Comment:
In the mach-o version of Python on MacOSX, the macfs.FSSpec() 
library function fails for files that don't yet exist.

The problem is caused by (internally) using an FSRef to construct 
the FSSpec: FSRefs can't hold references to non-existent files, 
whereas FSSpecs can.

----------------------------------------------------------------------

>Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-07 17:20

Message:
Logged In: YES 
user_id=33168

Can this be closed since macfs has been removed?

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2002-08-05 17:54

Message:
Logged In: YES 
user_id=45365

macostools.copy() has been fixed in rev. 1.16. mkalias()
still has the problem, though.

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2002-08-03 17:01

Message:
Logged In: YES 
user_id=45365

As the workaround is going to be both difficult to implement and incomplete I'm tempted to punt on this one. Difficult because we have to split the path in C, and we have to do /: substitutions on the last component; incomplete because we cannot cater for utf8/macroman differences and filenames > 32 characters.

If we punt on this we should make sure we phase out using fsspec's in the standard library in favor of the fsref calls.

Do you think this is an acceptable solution? How much work would it be?

----------------------------------------------------------------------

Comment By: Just van Rossum (jvr)
Date: 2002-07-29 17:56

Message:
Logged In: YES 
user_id=92689

This bug also cause macostools.copy() to fail, making the breakage 
larger than I thought :-(

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=585923&group_id=5470


From noreply@sourceforge.net  Fri Feb  7 22:22:11 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Fri, 07 Feb 2003 14:22:11 -0800
Subject: [Python-bugs-list] [ python-Bugs-584409 ] add way to detect bsddb version
Message-ID: <E18hGst-0003U4-00@sc8-sf-web2.sourceforge.net>

Bugs item #584409, was opened at 2002-07-21 03:29
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=584409&group_id=5470

Category: Extension Modules
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: paul rubin (phr)
Assigned to: Nobody/Anonymous (nobody)
Summary: add way to detect bsddb version

Initial Comment:
The bsddb module docs say that some Python
configurations use Berkeley db 1.85
and others use the incompatible 2.0.  Maybe by now
there are later versions as well.  There's no way
listed for a Python script to know which version of
bsddb is running underneath!  That's not so great,
since the versions don't interoperate and don't support
the same operations.

Proposed fix: please add a new function to the module,
bsddb.db_version().  This would
return a constant string like "1.85" or "2.0", built at
Python configuration time.


----------------------------------------------------------------------

>Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-07 17:22

Message:
Logged In: YES 
user_id=33168

bsddb has changed a lot since this bug report.  Have the
issues been resolved?

----------------------------------------------------------------------

Comment By: Skip Montanaro (montanaro)
Date: 2002-07-25 11:06

Message:
Logged In: YES 
user_id=44345

I can't comment on #504282 (I don't know what the problem is because 
the poster didn't provide enough information about the files and their 
names).  I attached a patch to #491888 which should solve that 
problem.

still-unconvinced-ly y'rs,

Skip


----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2002-07-25 05:08

Message:
Logged In: YES 
user_id=21627

It would solve bug #491888, and allow to give a better
diagnostic for #504282.

----------------------------------------------------------------------

Comment By: Skip Montanaro (montanaro)
Date: 2002-07-24 14:10

Message:
Logged In: YES 
user_id=44345

What would you have it report?  Dbhash is nothing more than a thin 
wrapper around bsddb.  Whichdb is a very fragile beast in my opinion, 
but it does already do some file content introspection, and if the file is 
some sort of Berkeley DB hash file, it will report it more-or-less correctly 
as "dbhash" (more correct in my opinion than returning None or "").  
This includes files created using the dbm module, if that module was 
linked with the dbm emulation API of Berkeley DB.

I still fail to see how any of this detection people propose would help.  If 
you have a version 5 hash file it doesn't matter how positive you are 
about it.  A later version of the Berkeley DB library which expects a 
version 7 hash file is still going to barf on the older file format.  To make 
things work again you're going to have to resort to running Sleepycat's 
tools to convert the file to the proper format.  It's not like you can 
detect file version differences and then plunge ahead along a different 
path without alerting the user to the problem.


----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2002-07-24 13:29

Message:
Logged In: YES 
user_id=21627

No, the main point would be that whichdb would not
incorrectly report the file format as 'dbhash', when it
isn't (because dbhash supports a different version).

----------------------------------------------------------------------

Comment By: Skip Montanaro (montanaro)
Date: 2002-07-24 10:54

Message:
Logged In: YES 
user_id=44345

This is precisely what Sleepycat's db_dump/db_load type tools take care 
of.  It's a one-time thing.  When you upgrade from one version of 
Berkeley DB to another you need to run these tools to make sure the file 
formats are up-to-date.  The only problem I see here with the current 
code is that the exception which is raised is rather mystical - something 
like a very large number followed by "invalid argument".  The most 
significant change I would see making here is to have the bsddb module 
recognize that weird error and raise an exception with a saner message.

I can't see the programmer or the user getting more information out of 
"expected hash file format version 7 but got hash file format version 5".


----------------------------------------------------------------------

Comment By: Barry A. Warsaw (bwarsaw)
Date: 2002-07-24 09:24

Message:
Logged In: YES 
user_id=12800

We could probably write a little utility to sniff file
version numbers based on the magic number as given in this doco:

http://www.sleepycat.com/docs/ref/install/magic.txt

----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2002-07-24 03:34

Message:
Logged In: YES 
user_id=21627

There is a bug report (somewhere) that whichdb incorrectly
determines the DB module. In that case, whichdb would
correctly find out that this is a Sleepycat database, and
suggest to use dbhash. In turn, dbhash would fail to open
the file, because the file version was incorrect. It would
have been correct to use the dbm module, since the dbm
library was also based on Sleepycat, but had a different
version than the bsddb library installed on the same system.

This problem can be solved if you can find out what file
version(s) your bsddb module supports.

The library version seems less useful to me, indeed.

----------------------------------------------------------------------

Comment By: Barry A. Warsaw (bwarsaw)
Date: 2002-07-23 22:53

Message:
Logged In: YES 
user_id=12800

It's useful if for no other reason than to figure out which
bugs you need to work around <wink>.

BTW, PyBSDDB does give you the ability to find out both the
version of the wrapper you've got and the version of the
underlying library.:

>>> import bsddb3
>>> bsddb3.__version__
'3.3.0'
>>> bsddb3._db.version()
(3, 3, 11)


You've also got DB_VERSION_STRING, DB_VERSION_MAJOR and
DB_VERSION_MINOR.

Note that if you're linking against a newer version of the
library using the 1.85 API, *that* might be a difficult
thing to figure out.  Off hand (and I can't check right
now), I don't know if that would give yo a different
bsddb3._db version constant or would otherwise be detectable.

----------------------------------------------------------------------

Comment By: Skip Montanaro (montanaro)
Date: 2002-07-23 20:15

Message:
Logged In: YES 
user_id=44345

I agree, if it's wanted badly enough, we can figure out what version was 
linked with the module code. The "define macros at configure time" idea 
is possible.  The "create a database and peek at it" idea won't work 
though.  There are library version numbers and file versions.  They don't 
always change in sync.

Like I said before, I'm skeptical a Python script would really need to 
know what version of the underlying library was linked with 
bsddbmodule.o.  Can you motivate things with a use case?

Skip


----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2002-07-23 19:35

Message:
Logged In: NO 

How can it be "impossible" to find out?  The build script
for the bsddb module can check what version is being linked,
and include a string reachable from Python.

At worst, there could be a routine added to the module that
actually creates a database, then examines the db file and
figures out from the bytes inside which version it is.

Paul


----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2002-07-23 17:50

Message:
Logged In: YES 
user_id=21627

I also believe that this problem should be fixed by importing 
pybsddb3.

On this issue itself: it turns out impossible to find out, 
programmatically, what version of Sleepycat DB you are 
running if all you have is the compatibility API: both the 
compile-time and the run-time version information is not 
available. Furthermore, you cannot include both new and old 
headers, since they conflict. So given the current code base, 
this problem cannot be solved.

----------------------------------------------------------------------

Comment By: Skip Montanaro (montanaro)
Date: 2002-07-22 21:18

Message:
Logged In: YES 
user_id=44345

Sorry for the lack of clarity.  What I should have said is that the code 
which implements the bsddb extension module only calls the 
1.85-compatible C API exposed when you configure the Berkeley DB 
code using the --enable-compat185 flag.  All the wonders and mysteries 
of the later parts of the API are lost on the bsddb code.

There are two levels of compatibility, the API level and the file format 
level.  All users of the bsddb module should care about is the file format 
level compatibility and handling that is a one-time problem dealt with 
using tools provided by Sleepycat as part of their distribution.

The topic of including bsddb3 in the standard distribution has been 
discussed before.  For one example, see:

  http://mail.python.org/pipermail/python-dev/2002-January/019261.html

I think the main stumbling block to incorporation is that it only works 
with versions 3 and 4 of the Berkeley DB library.  There is a more recent 
thread that currently escapes my feeble attempts to find it.



----------------------------------------------------------------------

Comment By: paul rubin (phr)
Date: 2002-07-22 16:26

Message:
Logged In: YES 
user_id=72053

OK, it looks like both the docs and Skip's note are a bit
unclear.  When you say only the 1.85 API is exposed, does
that mean the 1.85 file format is also used either way?  In
particular, if Python is linked with Berkeley DB 2.0 and I
create a db with it, will that db interoperate with another
application that's linked to Berkeley DB 1.85?

If it won't interoperate, then it's definitely worthwhile to
add some kind of call to the Python bsddb module to let
Python scripts find out which file format they're dealing
with.  

Also, I didn't realize only the 1.85 API was supported.  I
hope pybsddb3 can become part of the standard Python
distribution, since I'd like to use Sleepycat's transaction
features from Python scripts.

----------------------------------------------------------------------

Comment By: Skip Montanaro (montanaro)
Date: 2002-07-22 08:31

Message:
Logged In: YES 
user_id=44345

This is an interesting idea, but one that I think is less useful than you 
might believe.  The bsddb module exposes the same API based on the 
1.85 C API regardless what version of Berkeley DB you link with.  (I 
have linked it with versions 1.85 through 4.something.)  I've been using 
the bsddb module since its inclusion in Python and have never actually 
cared what version of the underlying C API the module what linked with. 
Someone programming to the C API *would* care about version 
differences, because the C API has grown richer over the years.  The 
bsddb module code just hasn't ever used any new functionality.  Note 
that the pybsddb3 module does use the new functionality in the version 
3 and 4 APIs.

What changes on you between versions are the file formats, and you 
should only care about that at the point where you upgrade from one 
version of Berkeley DB to another.  (Generally, you realize this when you 
start getting errors trying to open old databases.)  Sleepycat provides 
command line tools to help you convert from one file version to another, 
so once you realize your file formats have changed, you wind up poking 
around your disk looking for old format Berkeley DB files, run the tools 
on them, then go back to more interesting things, like writing 
stable sorts. ;-)


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=584409&group_id=5470


From noreply@sourceforge.net  Fri Feb  7 22:31:57 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Fri, 07 Feb 2003 14:31:57 -0800
Subject: [Python-bugs-list] [ python-Bugs-528990 ] bug? in PyImport_ImportModule under AIX
Message-ID: <E18hH2L-0003sH-00@sc8-sf-web2.sourceforge.net>

Bugs item #528990, was opened at 2002-03-12 11:19
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=528990&group_id=5470

Category: Python Interpreter Core
Group: Python 2.2
Status: Open
Resolution: None
Priority: 5
Submitted By: Fanny Wattez (wattez)
Assigned to: Nobody/Anonymous (nobody)
Summary: bug? in PyImport_ImportModule under AIX

Initial Comment:
Here is a description of the problem we encounter.

platform : AIX 4.3.3
Python 2.2 

environment variables : 
----------------------------------------
 
PATH=/home/soft/ccm_gdc7/bin:/bin:/usr/sbin:/usr/etc:/u
sr/lpp/X11/bin:/usr/wf/bin:/usr/local/bin:/usr/local/ad
min/etc:/lbin:/usr/bin/etc:/usr/ucb:/home/users/wattez/
lbin:/sbin:.:/usr/bin/X11:/sbin:/home/users/wattez/lbin
/fvwm-
exec:/tools/versant/6.0.0/rs6000/bin:/tools/views402/st
udio/rs6000:/tools/sp1.3.4/bin:/tools/imagemagick5.4.2/
bin:/tools/python2.2.debug/bin
PYTHON_ROOT=/tools/python2.2.debug
PYTHONPATH=/tools/python2.2.debug/lib/python2.2:/tools/
python2.2.debug/lib/python2.2/lib-
tk:/tools/xmlproc0.70:/tools/doctemplate2.2.1:/tools/py
thon2.2.debug/lib/python2.2/plat-
aix4:/tools/python2.2.debug/lib/python2.2/lib-
dynload:/tools/python2.2.debug/lib/python2.2/site-
packages

test program :
-------------------

#include <iostream.h>
#include <Python.h>

int main(int i__argc,char* i__argv[])
{

        Py_Initialize();

        cout << endl << "CALL of PyImport_ImportModule 
with argument " << getenv("NAME_MODULE") << endl;

        PyObject * l__obj = PyImport_ImportModule
(getenv("NAME_MODULE"));

        if (l__obj == NULL)
          cout << "importation of module " << getenv
("NAME_MODULE") << " does not work well!" << endl;

        return 1;
}


We ran this test program for different values of 
$NAME_MODULE.

$NAME_MODULE     	|    does the test program 
work well?
----------------------------------------------------
base64                  |    KO (Segmentation fault) 
at the call of PyImport_ImportModule
os                 	|    OK
strop            	|    KO (Segmentation fault) 
at the call of PyImport_ImportModule
string           	|    KO (Segmentation fault) 
at the call of PyImport_ImportModule

What do we see with dbx (Python 2.2 is compiled in 
debug mode)?
-------------------------------------
The last instructions in the stack are : 

stropmodule.split_whitespace(s = "strop", len = 
806723600, maxsplit = 806727672), line 80 
in "stropmodule.c"
initstrop(), line 1221 in "stropmodule.c"
_PyImport_LoadDynamicModule(0x2ff1d2b0, 0x2ff1cdc0, 
0xf0004d40), line 53 in "importdl.c"

with 0x2ff1d2b0 = "strop"
        0x2ff1cdc0 
= "/tools/python2.2.debug/lib/python2.2/lib-
dynload/strop.so"

Could you help us understand the problem?

Note : under the interactive Python interpreter, we 
have no problems importing each of these modules. The 
test program only crashes under AIX 4.3.3 not under 
Windows 2000.



----------------------------------------------------------------------

>Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-07 17:31

Message:
Logged In: YES 
user_id=33168

This problem may be fixed by adding a period (.) to the
first line of the Modules/python.exp file created.  The
first line should contain the following 3 characters:  #!.

This is documented in Misc/AIX-NOTES.  Fanny, if you are
there, let me know.  I'd like to try to close this bug report.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-01-13 17:40

Message:
Logged In: YES 
user_id=33168

Fanny, did this problem happen to occur when running in
64-bit mode?  There was another bug that was similar which
changed an (unsigned int) to a (char *).  The change was in
Python/dynload_aix.c Could you test with the CVS version of
2.2.2+ or 2.3a1?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=528990&group_id=5470


From noreply@sourceforge.net  Fri Feb  7 22:34:17 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Fri, 07 Feb 2003 14:34:17 -0800
Subject: [Python-bugs-list] [ python-Bugs-454086 ] distutils/mingw32 links to dbg libs
Message-ID: <E18hH4b-0001Tx-00@sc8-sf-web4.sourceforge.net>

Bugs item #454086, was opened at 2001-08-22 00:30
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=454086&group_id=5470

Category: Distutils
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Gerhard Häring (ghaering)
>Assigned to: Gerhard Häring (ghaering)
Summary: distutils/mingw32 links to dbg libs

Initial Comment:
When compiling extension modules on Windows with
debugging enabled for the native mode of the GNU
compilers ("--compiler=mingw32 --debug"), distutils
tries to link with the debugging version of the
libraries (python21_d.dll, ...). This may be useful for
Microsoft Visual C++, but for the GNU compilers, it
isn't. GNU tools have a different debugging symbol
format than MS tools, so there's no point in doing this.

----------------------------------------------------------------------

>Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-07 17:34

Message:
Logged In: YES 
user_id=33168

Gerhard, is this still a problem?  Can this be closed?

----------------------------------------------------------------------

Comment By: Gerhard Häring (ghaering)
Date: 2001-09-18 23:30

Message:
Logged In: YES 
user_id=163326

OK, see patch #462754.

----------------------------------------------------------------------

Comment By: A.M. Kuchling (akuchling)
Date: 2001-09-04 15:56

Message:
Logged In: YES 
user_id=11375

Hmm... I can't see any code in cygwinccompiler.py that 
adds the _d prefix in the current CVS.  Can you please 
track down the code that adds it, and submit a patch to 
fix the problem?



----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=454086&group_id=5470


From noreply@sourceforge.net  Fri Feb  7 22:51:50 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Fri, 07 Feb 2003 14:51:50 -0800
Subject: [Python-bugs-list] [ python-Bugs-678518 ] Another parsermodule validation error
Message-ID: <E18hHLa-00028U-00@sc8-sf-web4.sourceforge.net>

Bugs item #678518, was opened at 2003-01-31 20:23
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=678518&group_id=5470

Category: Parser/Compiler
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: logistix (logistix)
Assigned to: Michael Hudson (mwh)
Summary: Another parsermodule validation error

Initial Comment:
The following code generates an validation error.  It's an 
invalid ast tree where the global_stmt doesn't have the 
keyword 'global' as a child.

I tracked down the fix to validate_global_stmt() this time.

Python 2.3a1 (#38, Dec 31 2002, 17:53:59) [MSC 
v.1200 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more 
information.
>>> import symbol,token,parser
>>> ast = [symbol.file_input,
...  [symbol.stmt, [symbol.simple_stmt, 
[symbol.small_stmt, [symbol.global_stmt,
 [1, 'foo']]], [token.NEWLINE, '']]],
...  [token.NEWLINE, ''],
...  [token.ENDMARKER, '']]
>>> a = parser.sequence2ast(ast)
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
parser.ParserError: VALIDATION FAILURE: report this 
to the maintainer!
>>>

----------------------------------------------------------------------

>Comment By: logistix (logistix)
Date: 2003-02-07 16:51

Message:
Logged In: YES 
user_id=699438

You're right, the assert statement was the unhandled 
exception.  It was raising an error but returning true on 
the "__assert__" check.

Just posted the whole patch

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2003-02-07 08:38

Message:
Logged In: YES 
user_id=6656

> At the time I didn't have an encoding patch and punted.

Oh, I misread you then.

> I did figure it out last night though.  I'll post it later
today 
> since I'm at the office now.

Ta.

> And yes, this is the error that'll print out "XXX 
> unhandled exception" out of ceval.c when you run
> the test from debug mode.

Are you sure?  It seems that was the assert one.

> There was some code added to add the encoding 
> type in the ast2sequence functions,

By me :-)  I didn't really think about the sequence2ast 
functions as all I was doing was trying to get the compiler
package to work...

> Also, as a matter of style, should I add the stuff in 
> my test script to the test suite or is that overkill? Should
> I just have it check 4 or 5 random files?

It might be best to get it to check a few files we knew there
were problems with -- test_pep263, maybe.  Checking the
whole lot is probably overkill.

----------------------------------------------------------------------

Comment By: logistix (logistix)
Date: 2003-02-07 08:23

Message:
Logged In: YES 
user_id=699438

Michael,

At the time I didn't have an encoding patch and punted.  I did 
figure it out last night though.  I'll post it later today since I'm 
at the office now.

And yes, this is the error that'll print out "XXX unhandled 
exception" out of ceval.c when you run the test from debug 
mode.

It comes down to the bastardized way that encoding is 
implemented.  It doesn't fit the current definition of a token or 
a symbol.  There was some code added to add the encoding 
type in the ast2sequence functions, but not the 
sequence2ast functions, so when it hit scripts with 
encoding_decl's, it bombed big time.

Also, as a matter of style, should I add the stuff in my test 
script to the test suite or is that overkill? Should I just have it 
check4 or 5 random files?

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2003-02-07 06:08

Message:
Logged In: YES 
user_id=6656

Pff, I was right, but logistix's patch fix it.

logistix, did you forget to attach the encoding patch?  I
still get errors in module's with a cookie.

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2003-02-07 05:58

Message:
Logged In: YES 
user_id=6656

Patches as such are probably fine.  The reason I haven't
checked them in is a wierdness that I may or may not be
related, but is bothering me.  Consider:

->> import parser, sys,glob,os, traceback
/>> for path in sys.path:
|..     for fil in glob.glob(os.path.join(path,"*.py")):
|..       try:
|..         try:
|..             a = parser.suite(file(fil).read())
|..             b = parser.ast2tuple(a)
|..             c = parser.sequence2ast(b)
|..         except parser.ParserError:
|..             print "Parse of %s" % fil, "FAILED"
|..             traceback.print_exc()
|..             print '----------------------------'
|..       except:
|..          print "waaa?"
|.. else:
|..     print "what?"
\__ 
Parse of /home/mwh/src/sf/python/dist/src/Lib/tarfile.py FAILED
Traceback (most recent call last):
  File "<input>", line 7, in ?
ParserError
----------------------------
Parse of /home/mwh/src/sf/python/dist/src/Lib/heapq.py FAILED
Traceback (most recent call last):
  File "<input>", line 7, in ?
ParserError
----------------------------
Parse of /home/mwh/src/sf/python/dist/src/Lib/getopt.py FAILED
Traceback (most recent call last):
  File "<input>", line 7, in ?
ParserError
----------------------------
Parse of /home/mwh/src/sf/python/dist/src/Lib/inspect.py FAILED
Traceback (most recent call last):
  File "<input>", line 7, in ?
ParserError
----------------------------
Parse of /home/mwh/src/sf/python/dist/src/Lib/pydoc.py FAILED
Traceback (most recent call last):
  File "<input>", line 7, in ?
ParserError
----------------------------
Parse of /home/mwh/src/sf/python/dist/src/Lib/lib-tk/Tix.py
FAILED
Traceback (most recent call last):
  File "<input>", line 7, in ?
ParserError
----------------------------
Traceback (most recent call last):
  File "<input>", line 2, in ?
ParserError: Illegal terminal: expected "__assert__"
->> 

Notice that an exception seems to have made it through the
except clauses!  The scary bit is that a (seemingly) random
number of parser errors *will* caught, and then one will get
through.

The only explanation I have for this is some kind of
returning non-NULL with exception set kind of bug.

Off to play with a debug build, I guess.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-06 19:17

Message:
Logged In: YES 
user_id=33168

Michael were these patches ok to check in?

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2003-02-03 05:00

Message:
Logged In: YES 
user_id=6656

Thanks for doing this.  I'll probably check these in after
I've finished crunching my way through monday morning email...

----------------------------------------------------------------------

Comment By: logistix (logistix)
Date: 2003-02-01 15:42

Message:
Logged In: YES 
user_id=699438

And I found a few more by doing a roundtrip 
ast2Tuple/tuple2ast on all files in pythonpath.  These actually 
affect valid AST's, so they're probably a bigger deal.

Valid assert statements were failing.  The patch was trivial, 
so its attached.

Also, files that took advantage of encoding_decl (from PEP 
263) were failing.  I couldn't tell if PEP 263 was offically 
closed, and didn't know what the offical grammar was, so I 
didn't do anything there.

I'll also attach the test script that turned up these errors.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=678518&group_id=5470


From noreply@sourceforge.net  Fri Feb  7 22:57:38 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Fri, 07 Feb 2003 14:57:38 -0800
Subject: [Python-bugs-list] [ python-Bugs-584409 ] add way to detect bsddb version
Message-ID: <E18hHRC-0006ex-00@sc8-sf-web1.sourceforge.net>

Bugs item #584409, was opened at 2002-07-21 02:29
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=584409&group_id=5470

Category: Extension Modules
Group: None
>Status: Closed
>Resolution: Fixed
Priority: 5
Submitted By: paul rubin (phr)
Assigned to: Nobody/Anonymous (nobody)
Summary: add way to detect bsddb version

Initial Comment:
The bsddb module docs say that some Python
configurations use Berkeley db 1.85
and others use the incompatible 2.0.  Maybe by now
there are later versions as well.  There's no way
listed for a Python script to know which version of
bsddb is running underneath!  That's not so great,
since the versions don't interoperate and don't support
the same operations.

Proposed fix: please add a new function to the module,
bsddb.db_version().  This would
return a constant string like "1.85" or "2.0", built at
Python configuration time.


----------------------------------------------------------------------

>Comment By: Skip Montanaro (montanaro)
Date: 2003-02-07 16:57

Message:
Logged In: YES 
user_id=44345

In the new bsddb module you can call bsddb.db.version() to get the
version of the underlying library, so I'm going to close this.  I don't
know if it's worth it to add the same functionality to the old
bsddb185 module or not.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-07 16:22

Message:
Logged In: YES 
user_id=33168

bsddb has changed a lot since this bug report.  Have the
issues been resolved?

----------------------------------------------------------------------

Comment By: Skip Montanaro (montanaro)
Date: 2002-07-25 10:06

Message:
Logged In: YES 
user_id=44345

I can't comment on #504282 (I don't know what the problem is because 
the poster didn't provide enough information about the files and their 
names).  I attached a patch to #491888 which should solve that 
problem.

still-unconvinced-ly y'rs,

Skip


----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2002-07-25 04:08

Message:
Logged In: YES 
user_id=21627

It would solve bug #491888, and allow to give a better
diagnostic for #504282.

----------------------------------------------------------------------

Comment By: Skip Montanaro (montanaro)
Date: 2002-07-24 13:10

Message:
Logged In: YES 
user_id=44345

What would you have it report?  Dbhash is nothing more than a thin 
wrapper around bsddb.  Whichdb is a very fragile beast in my opinion, 
but it does already do some file content introspection, and if the file is 
some sort of Berkeley DB hash file, it will report it more-or-less correctly 
as "dbhash" (more correct in my opinion than returning None or "").  
This includes files created using the dbm module, if that module was 
linked with the dbm emulation API of Berkeley DB.

I still fail to see how any of this detection people propose would help.  If 
you have a version 5 hash file it doesn't matter how positive you are 
about it.  A later version of the Berkeley DB library which expects a 
version 7 hash file is still going to barf on the older file format.  To make 
things work again you're going to have to resort to running Sleepycat's 
tools to convert the file to the proper format.  It's not like you can 
detect file version differences and then plunge ahead along a different 
path without alerting the user to the problem.


----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2002-07-24 12:29

Message:
Logged In: YES 
user_id=21627

No, the main point would be that whichdb would not
incorrectly report the file format as 'dbhash', when it
isn't (because dbhash supports a different version).

----------------------------------------------------------------------

Comment By: Skip Montanaro (montanaro)
Date: 2002-07-24 09:54

Message:
Logged In: YES 
user_id=44345

This is precisely what Sleepycat's db_dump/db_load type tools take care 
of.  It's a one-time thing.  When you upgrade from one version of 
Berkeley DB to another you need to run these tools to make sure the file 
formats are up-to-date.  The only problem I see here with the current 
code is that the exception which is raised is rather mystical - something 
like a very large number followed by "invalid argument".  The most 
significant change I would see making here is to have the bsddb module 
recognize that weird error and raise an exception with a saner message.

I can't see the programmer or the user getting more information out of 
"expected hash file format version 7 but got hash file format version 5".


----------------------------------------------------------------------

Comment By: Barry A. Warsaw (bwarsaw)
Date: 2002-07-24 08:24

Message:
Logged In: YES 
user_id=12800

We could probably write a little utility to sniff file
version numbers based on the magic number as given in this doco:

http://www.sleepycat.com/docs/ref/install/magic.txt

----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2002-07-24 02:34

Message:
Logged In: YES 
user_id=21627

There is a bug report (somewhere) that whichdb incorrectly
determines the DB module. In that case, whichdb would
correctly find out that this is a Sleepycat database, and
suggest to use dbhash. In turn, dbhash would fail to open
the file, because the file version was incorrect. It would
have been correct to use the dbm module, since the dbm
library was also based on Sleepycat, but had a different
version than the bsddb library installed on the same system.

This problem can be solved if you can find out what file
version(s) your bsddb module supports.

The library version seems less useful to me, indeed.

----------------------------------------------------------------------

Comment By: Barry A. Warsaw (bwarsaw)
Date: 2002-07-23 21:53

Message:
Logged In: YES 
user_id=12800

It's useful if for no other reason than to figure out which
bugs you need to work around <wink>.

BTW, PyBSDDB does give you the ability to find out both the
version of the wrapper you've got and the version of the
underlying library.:

>>> import bsddb3
>>> bsddb3.__version__
'3.3.0'
>>> bsddb3._db.version()
(3, 3, 11)


You've also got DB_VERSION_STRING, DB_VERSION_MAJOR and
DB_VERSION_MINOR.

Note that if you're linking against a newer version of the
library using the 1.85 API, *that* might be a difficult
thing to figure out.  Off hand (and I can't check right
now), I don't know if that would give yo a different
bsddb3._db version constant or would otherwise be detectable.

----------------------------------------------------------------------

Comment By: Skip Montanaro (montanaro)
Date: 2002-07-23 19:15

Message:
Logged In: YES 
user_id=44345

I agree, if it's wanted badly enough, we can figure out what version was 
linked with the module code. The "define macros at configure time" idea 
is possible.  The "create a database and peek at it" idea won't work 
though.  There are library version numbers and file versions.  They don't 
always change in sync.

Like I said before, I'm skeptical a Python script would really need to 
know what version of the underlying library was linked with 
bsddbmodule.o.  Can you motivate things with a use case?

Skip


----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2002-07-23 18:35

Message:
Logged In: NO 

How can it be "impossible" to find out?  The build script
for the bsddb module can check what version is being linked,
and include a string reachable from Python.

At worst, there could be a routine added to the module that
actually creates a database, then examines the db file and
figures out from the bytes inside which version it is.

Paul


----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2002-07-23 16:50

Message:
Logged In: YES 
user_id=21627

I also believe that this problem should be fixed by importing 
pybsddb3.

On this issue itself: it turns out impossible to find out, 
programmatically, what version of Sleepycat DB you are 
running if all you have is the compatibility API: both the 
compile-time and the run-time version information is not 
available. Furthermore, you cannot include both new and old 
headers, since they conflict. So given the current code base, 
this problem cannot be solved.

----------------------------------------------------------------------

Comment By: Skip Montanaro (montanaro)
Date: 2002-07-22 20:18

Message:
Logged In: YES 
user_id=44345

Sorry for the lack of clarity.  What I should have said is that the code 
which implements the bsddb extension module only calls the 
1.85-compatible C API exposed when you configure the Berkeley DB 
code using the --enable-compat185 flag.  All the wonders and mysteries 
of the later parts of the API are lost on the bsddb code.

There are two levels of compatibility, the API level and the file format 
level.  All users of the bsddb module should care about is the file format 
level compatibility and handling that is a one-time problem dealt with 
using tools provided by Sleepycat as part of their distribution.

The topic of including bsddb3 in the standard distribution has been 
discussed before.  For one example, see:

  http://mail.python.org/pipermail/python-dev/2002-January/019261.html

I think the main stumbling block to incorporation is that it only works 
with versions 3 and 4 of the Berkeley DB library.  There is a more recent 
thread that currently escapes my feeble attempts to find it.



----------------------------------------------------------------------

Comment By: paul rubin (phr)
Date: 2002-07-22 15:26

Message:
Logged In: YES 
user_id=72053

OK, it looks like both the docs and Skip's note are a bit
unclear.  When you say only the 1.85 API is exposed, does
that mean the 1.85 file format is also used either way?  In
particular, if Python is linked with Berkeley DB 2.0 and I
create a db with it, will that db interoperate with another
application that's linked to Berkeley DB 1.85?

If it won't interoperate, then it's definitely worthwhile to
add some kind of call to the Python bsddb module to let
Python scripts find out which file format they're dealing
with.  

Also, I didn't realize only the 1.85 API was supported.  I
hope pybsddb3 can become part of the standard Python
distribution, since I'd like to use Sleepycat's transaction
features from Python scripts.

----------------------------------------------------------------------

Comment By: Skip Montanaro (montanaro)
Date: 2002-07-22 07:31

Message:
Logged In: YES 
user_id=44345

This is an interesting idea, but one that I think is less useful than you 
might believe.  The bsddb module exposes the same API based on the 
1.85 C API regardless what version of Berkeley DB you link with.  (I 
have linked it with versions 1.85 through 4.something.)  I've been using 
the bsddb module since its inclusion in Python and have never actually 
cared what version of the underlying C API the module what linked with. 
Someone programming to the C API *would* care about version 
differences, because the C API has grown richer over the years.  The 
bsddb module code just hasn't ever used any new functionality.  Note 
that the pybsddb3 module does use the new functionality in the version 
3 and 4 APIs.

What changes on you between versions are the file formats, and you 
should only care about that at the point where you upgrade from one 
version of Berkeley DB to another.  (Generally, you realize this when you 
start getting errors trying to open old databases.)  Sleepycat provides 
command line tools to help you convert from one file version to another, 
so once you realize your file formats have changed, you wind up poking 
around your disk looking for old format Berkeley DB files, run the tools 
on them, then go back to more interesting things, like writing 
stable sorts. ;-)


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=584409&group_id=5470


From noreply@sourceforge.net  Fri Feb  7 23:17:52 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Fri, 07 Feb 2003 15:17:52 -0800
Subject: [Python-bugs-list] [ python-Bugs-680577 ] urllib2 authentication problem
Message-ID: <E18hHkm-0006ik-00@sc8-sf-web3.sourceforge.net>

Bugs item #680577, was opened at 2003-02-05 01:22
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=680577&group_id=5470

Category: Python Library
>Group: Python 2.2
Status: Open
Resolution: None
Priority: 5
Submitted By: Gary Donovan (gazzadee)
Assigned to: Nobody/Anonymous (nobody)
Summary: urllib2 authentication problem

Initial Comment:
I've found a problem using the authentication in urllib2.

When matching up host-names in order to find a
password, then putting the protocol in the address
makes it seem like a different address. eg...

I create a HTTPBasicAuthHandler with a
HTTPPasswordMgrWithDefaultRealm, and add the tuple
(None, "http://proxy.blah.com:17828", "foo", "bar") to it.

I then setup the proxy to use
http://proxy.blah.com:17828 (which requires
authentication).

When I connect, the password lookup fails, because it
is trying to find a match for "proxy.blah.com:17828"
rather than "http://proxy.blah.com:17828"

This problem doesn't exist if I pass
"proxy.blah.com:17828" to the password manager.

There seems to be some stuff in HTTPPasswordMgr to deal
with variations on site names, but I guess it's not
working in this case (unless this is intentional).

Version Info:
Python 2.2 (#1, Feb 24 2002, 16:21:58)
[GCC 2.96 20000731 (Mandrake Linux 8.2 2.96-0.76mdk)]
on linux-i386


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=680577&group_id=5470


From noreply@sourceforge.net  Fri Feb  7 23:21:51 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Fri, 07 Feb 2003 15:21:51 -0800
Subject: [Python-bugs-list] [ python-Bugs-680577 ] urllib2 authentication problem
Message-ID: <E18hHod-0003uX-00@sc8-sf-web4.sourceforge.net>

Bugs item #680577, was opened at 2003-02-05 01:22
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=680577&group_id=5470

Category: Python Library
Group: Python 2.2
Status: Open
Resolution: None
Priority: 5
Submitted By: Gary Donovan (gazzadee)
Assigned to: Nobody/Anonymous (nobody)
Summary: urllib2 authentication problem

Initial Comment:
I've found a problem using the authentication in urllib2.

When matching up host-names in order to find a
password, then putting the protocol in the address
makes it seem like a different address. eg...

I create a HTTPBasicAuthHandler with a
HTTPPasswordMgrWithDefaultRealm, and add the tuple
(None, "http://proxy.blah.com:17828", "foo", "bar") to it.

I then setup the proxy to use
http://proxy.blah.com:17828 (which requires
authentication).

When I connect, the password lookup fails, because it
is trying to find a match for "proxy.blah.com:17828"
rather than "http://proxy.blah.com:17828"

This problem doesn't exist if I pass
"proxy.blah.com:17828" to the password manager.

There seems to be some stuff in HTTPPasswordMgr to deal
with variations on site names, but I guess it's not
working in this case (unless this is intentional).

Version Info:
Python 2.2 (#1, Feb 24 2002, 16:21:58)
[GCC 2.96 20000731 (Mandrake Linux 8.2 2.96-0.76mdk)]
on linux-i386


----------------------------------------------------------------------

>Comment By: Gerhard Häring (ghaering)
Date: 2003-02-08 00:21

Message:
Logged In: YES 
user_id=163326

Can you please retry with Python 2.2.2?

It seems that a related bug was fixed for 2.2.2:
http://python.org/2.2.2/NEWS.txt has an entry:

"""
- In urllib2.py: fix proxy config with user+pass
authentication.  [SF
  patch 527518]
"""

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=680577&group_id=5470


From noreply@sourceforge.net  Fri Feb  7 23:27:46 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Fri, 07 Feb 2003 15:27:46 -0800
Subject: [Python-bugs-list] [ python-Bugs-611052 ] SMTP.login() uses invalid base64 enc.
Message-ID: <E18hHuM-0004Ad-00@sc8-sf-web4.sourceforge.net>

Bugs item #611052, was opened at 2002-09-18 13:52
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=611052&group_id=5470

Category: Extension Modules
Group: Python 2.2.1
>Status: Closed
>Resolution: Fixed
Priority: 5
Submitted By: Ondrej Sury (ondrass)
Assigned to: Gerhard Häring (ghaering)
Summary: SMTP.login() uses invalid base64 enc.

Initial Comment:
If you have veeery long username and/or password it encodes
it with '\n' after 80? characters.  This violates
(IMHO) smtp standard.
In following example look for \n characters.

O.

>>> from smtplib import SMTP
>>> s = SMTP('smtp.servery.cz')
>>> s.set_debuglevel(255)
>>>
s.login('veeeery.looooong.username@veeeeery.loooooong.domain',
'veeeeery.loooong.password')
send: 'ehlo shade.globe.cz\r\n'
reply: '250-smtp.cz\r\n'
reply: '250-AUTH LOGIN PLAIN\r\n'
reply: '250-AUTH=LOGIN PLAIN\r\n'
reply: '250-PIPELINING\r\n'
reply: '250-STARTTLS\r\n'
reply: '250 8BITMIME\r\n'
reply: retcode (250); Msg: smtp.cz
AUTH LOGIN PLAIN
AUTH=LOGIN PLAIN
PIPELINING
STARTTLS
8BITMIME
AuthMethod: PLAIN
send: 'AUTH PLAIN
dmVlZWVyeS5sb29vb29uZy51c2VybmFtZUB2ZWVlZWVyeS5sb29vb29vbmcuZG9tYWluAHZlZWVl\ncnkubG9vb29vbmcudXNlcm5hbWVAdmVlZWVlcnkubG9vb29vb25nLmRvbWFpbgB2ZWVlZWVyeS5s\nb29vb25nLnBhc3N3b3Jk\r\n'
reply: '501 malformed auth input (#5.5.4)\r\n'
reply: retcode (501); Msg: malformed auth input (#5.5.4)
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "/usr/lib/python2.2/smtplib.py", line 546, in login
    raise SMTPAuthenticationError(code, resp)
smtplib.SMTPAuthenticationError: (501, 'malformed auth
input (#5.5.4)')


----------------------------------------------------------------------

>Comment By: Gerhard Häring (ghaering)
Date: 2003-02-08 00:27

Message:
Logged In: YES 
user_id=163326

Sure. Done.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-07 23:11

Message:
Logged In: YES 
user_id=33168

Gerhard, can this be closed now?

----------------------------------------------------------------------

Comment By: Gerhard Häring (ghaering)
Date: 2002-10-14 18:04

Message:
Logged In: YES 
user_id=163326

Thanks for the bug report. This has already been fixed in
Python 2.3-CVS and the 2.2 maintenance branch. So just
upgrade to Python 2.2.2 when it'll be out tomorrow :-)

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=611052&group_id=5470


From noreply@sourceforge.net  Sat Feb  8 00:05:45 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Fri, 07 Feb 2003 16:05:45 -0800
Subject: [Python-bugs-list] [ python-Bugs-682648 ] urllib2 HTTPDigestAuthHandler misnamed class attr
Message-ID: <E18hIV7-0000vm-00@sc8-sf-web1.sourceforge.net>

Bugs item #682648, was opened at 2003-02-07 19:05
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=682648&group_id=5470

Category: Python Library
Group: Python 2.2.2
Status: Open
Resolution: None
Priority: 5
Submitted By: steve steiner (ssteiner)
Assigned to: Nobody/Anonymous (nobody)
Summary: urllib2 HTTPDigestAuthHandler misnamed class attr

Initial Comment:
The class attribute 'header' should be named 'auth_header' as it is in HTTPBasicAuthHandler.  Same is true of ProxyDigestAuthHandler.  

These classes throw an AttributeError (shown below for HTTPDigestAuthHandler:

AttributeError: HTTPDigestAuthHandler instance has no attribute 'auth_header'

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=682648&group_id=5470


From noreply@sourceforge.net  Fri Feb  7 14:38:24 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Fri, 07 Feb 2003 06:38:24 -0800
Subject: [Python-bugs-list] [ python-Bugs-678518 ] Another parsermodule validation error
Message-ID: <E18h9e4-0003Yb-00@sc8-sf-web4.sourceforge.net>

Bugs item #678518, was opened at 2003-02-01 02:23
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=678518&group_id=5470

Category: Parser/Compiler
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: logistix (logistix)
Assigned to: Michael Hudson (mwh)
Summary: Another parsermodule validation error

Initial Comment:
The following code generates an validation error.  It's an 
invalid ast tree where the global_stmt doesn't have the 
keyword 'global' as a child.

I tracked down the fix to validate_global_stmt() this time.

Python 2.3a1 (#38, Dec 31 2002, 17:53:59) [MSC 
v.1200 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more 
information.
>>> import symbol,token,parser
>>> ast = [symbol.file_input,
...  [symbol.stmt, [symbol.simple_stmt, 
[symbol.small_stmt, [symbol.global_stmt,
 [1, 'foo']]], [token.NEWLINE, '']]],
...  [token.NEWLINE, ''],
...  [token.ENDMARKER, '']]
>>> a = parser.sequence2ast(ast)
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
parser.ParserError: VALIDATION FAILURE: report this 
to the maintainer!
>>>

----------------------------------------------------------------------

>Comment By: Michael Hudson (mwh)
Date: 2003-02-07 14:38

Message:
Logged In: YES 
user_id=6656

> At the time I didn't have an encoding patch and punted.

Oh, I misread you then.

> I did figure it out last night though.  I'll post it later
today 
> since I'm at the office now.

Ta.

> And yes, this is the error that'll print out "XXX 
> unhandled exception" out of ceval.c when you run
> the test from debug mode.

Are you sure?  It seems that was the assert one.

> There was some code added to add the encoding 
> type in the ast2sequence functions,

By me :-)  I didn't really think about the sequence2ast 
functions as all I was doing was trying to get the compiler
package to work...

> Also, as a matter of style, should I add the stuff in 
> my test script to the test suite or is that overkill? Should
> I just have it check 4 or 5 random files?

It might be best to get it to check a few files we knew there
were problems with -- test_pep263, maybe.  Checking the
whole lot is probably overkill.

----------------------------------------------------------------------

Comment By: logistix (logistix)
Date: 2003-02-07 14:23

Message:
Logged In: YES 
user_id=699438

Michael,

At the time I didn't have an encoding patch and punted.  I did 
figure it out last night though.  I'll post it later today since I'm 
at the office now.

And yes, this is the error that'll print out "XXX unhandled 
exception" out of ceval.c when you run the test from debug 
mode.

It comes down to the bastardized way that encoding is 
implemented.  It doesn't fit the current definition of a token or 
a symbol.  There was some code added to add the encoding 
type in the ast2sequence functions, but not the 
sequence2ast functions, so when it hit scripts with 
encoding_decl's, it bombed big time.

Also, as a matter of style, should I add the stuff in my test 
script to the test suite or is that overkill? Should I just have it 
check4 or 5 random files?

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2003-02-07 12:08

Message:
Logged In: YES 
user_id=6656

Pff, I was right, but logistix's patch fix it.

logistix, did you forget to attach the encoding patch?  I
still get errors in module's with a cookie.

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2003-02-07 11:58

Message:
Logged In: YES 
user_id=6656

Patches as such are probably fine.  The reason I haven't
checked them in is a wierdness that I may or may not be
related, but is bothering me.  Consider:

->> import parser, sys,glob,os, traceback
/>> for path in sys.path:
|..     for fil in glob.glob(os.path.join(path,"*.py")):
|..       try:
|..         try:
|..             a = parser.suite(file(fil).read())
|..             b = parser.ast2tuple(a)
|..             c = parser.sequence2ast(b)
|..         except parser.ParserError:
|..             print "Parse of %s" % fil, "FAILED"
|..             traceback.print_exc()
|..             print '----------------------------'
|..       except:
|..          print "waaa?"
|.. else:
|..     print "what?"
\__ 
Parse of /home/mwh/src/sf/python/dist/src/Lib/tarfile.py FAILED
Traceback (most recent call last):
  File "<input>", line 7, in ?
ParserError
----------------------------
Parse of /home/mwh/src/sf/python/dist/src/Lib/heapq.py FAILED
Traceback (most recent call last):
  File "<input>", line 7, in ?
ParserError
----------------------------
Parse of /home/mwh/src/sf/python/dist/src/Lib/getopt.py FAILED
Traceback (most recent call last):
  File "<input>", line 7, in ?
ParserError
----------------------------
Parse of /home/mwh/src/sf/python/dist/src/Lib/inspect.py FAILED
Traceback (most recent call last):
  File "<input>", line 7, in ?
ParserError
----------------------------
Parse of /home/mwh/src/sf/python/dist/src/Lib/pydoc.py FAILED
Traceback (most recent call last):
  File "<input>", line 7, in ?
ParserError
----------------------------
Parse of /home/mwh/src/sf/python/dist/src/Lib/lib-tk/Tix.py
FAILED
Traceback (most recent call last):
  File "<input>", line 7, in ?
ParserError
----------------------------
Traceback (most recent call last):
  File "<input>", line 2, in ?
ParserError: Illegal terminal: expected "__assert__"
->> 

Notice that an exception seems to have made it through the
except clauses!  The scary bit is that a (seemingly) random
number of parser errors *will* caught, and then one will get
through.

The only explanation I have for this is some kind of
returning non-NULL with exception set kind of bug.

Off to play with a debug build, I guess.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-07 01:17

Message:
Logged In: YES 
user_id=33168

Michael were these patches ok to check in?

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2003-02-03 11:00

Message:
Logged In: YES 
user_id=6656

Thanks for doing this.  I'll probably check these in after
I've finished crunching my way through monday morning email...

----------------------------------------------------------------------

Comment By: logistix (logistix)
Date: 2003-02-01 21:42

Message:
Logged In: YES 
user_id=699438

And I found a few more by doing a roundtrip 
ast2Tuple/tuple2ast on all files in pythonpath.  These actually 
affect valid AST's, so they're probably a bigger deal.

Valid assert statements were failing.  The patch was trivial, 
so its attached.

Also, files that took advantage of encoding_decl (from PEP 
263) were failing.  I couldn't tell if PEP 263 was offically 
closed, and didn't know what the offical grammar was, so I 
didn't do anything there.

I'll also attach the test script that turned up these errors.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=678518&group_id=5470


From noreply@sourceforge.net  Sat Feb  8 10:23:12 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sat, 08 Feb 2003 02:23:12 -0800
Subject: [Python-bugs-list] [ python-Bugs-682813 ] dircache.listdir doesn't signal error
Message-ID: <E18hS8e-0008UR-00@sc8-sf-web1.sourceforge.net>

Bugs item #682813, was opened at 2003-02-08 10:23
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=682813&group_id=5470

Category: Python Library
Group: Python 2.2
Status: Open
Resolution: None
Priority: 5
Submitted By: paul rubin (phr)
Assigned to: Nobody/Anonymous (nobody)
Summary: dircache.listdir doesn't signal error

Initial Comment:
dircache.listidir('some_non_existent_directory') simply
returns an empty list.  That's inconsistent with os.listdir
which raises an OSError exception.  I think
dircache.listdir
should also raise an exception if the directory can't
be read.
At minimum, though, the difference should be documented.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=682813&group_id=5470


From noreply@sourceforge.net  Sat Feb  8 10:27:58 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sat, 08 Feb 2003 02:27:58 -0800
Subject: [Python-bugs-list] [ python-Feature Requests-662923 ] iterator for dbm keys
Message-ID: <E18hSDG-000091-00@sc8-sf-web1.sourceforge.net>

Feature Requests item #662923, was opened at 2003-01-06 02:34
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=662923&group_id=5470

Category: Python Library
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: paul rubin (phr)
Assigned to: Nobody/Anonymous (nobody)
Summary: iterator for dbm keys

Initial Comment:
if d is a dbm (gdbm, bsddb, shelve, etc.) object, then
"for k in d"
should iterate through the keys of d, like you can do
with dicts.  Alternatively, add a new operation,
d.xkeys(), which returns an iterator that steps through
all the keys in d.  d.keys() is often inappropriate
since dbm databases can be extremely large and d.keys()
tries to read all the keys into memory at once.


----------------------------------------------------------------------

>Comment By: paul rubin (phr)
Date: 2003-02-08 10:27

Message:
Logged In: YES 
user_id=72053

That's nice about shelve, but really, all the dbm classes
need this operation.

----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2003-01-10 09:47

Message:
Logged In: YES 
user_id=80475

* for Py2.3, all shelve now inherits from UserDict.DictMixin 
which adds in the missing methods for a dictionary style 
interface.

* the appropriate name for d.xkeys() is d.iterkeys().

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=662923&group_id=5470


From noreply@sourceforge.net  Sat Feb  8 14:29:52 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sat, 08 Feb 2003 06:29:52 -0800
Subject: [Python-bugs-list] [ python-Bugs-501716 ] "es#" parser marker leaks memory
Message-ID: <E18hVzM-0005kw-00@sc8-sf-web2.sourceforge.net>

Bugs item #501716, was opened at 2002-01-10 11:39
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=501716&group_id=5470

Category: Unicode
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: M.-A. Lemburg (lemburg)
Assigned to: M.-A. Lemburg (lemburg)
>Summary: "es#" parser marker leaks memory

Initial Comment:
... if subsequent parsing fails, a buffer which was possibly 
allocated by "es#" is not freed.


----------------------------------------------------------------------

>Comment By: M.-A. Lemburg (lemburg)
Date: 2003-02-08 15:29

Message:
Logged In: YES 
user_id=38388

What I meant was that in the case of a parser marker
string like "es#iii" if the last "i" fails to convert, then the
buffer used for "es#" won't be freed up again.

----------------------------------------------------------------------

Comment By: Michael Stone (mbrierst)
Date: 2003-02-07 21:20

Message:
Logged In: YES 
user_id=670441

In what case does this happen?  The only case
I can see is if the string is successfully encoded,
but the encoding results in a non-null terminated 
string.  That might cause problems, otherwise it
looks okay?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=501716&group_id=5470


From noreply@sourceforge.net  Sat Feb  8 14:43:59 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sat, 08 Feb 2003 06:43:59 -0800
Subject: [Python-bugs-list] [ python-Bugs-682906 ] refcount leak in list +=
Message-ID: <E18hWD1-0007zV-00@sc8-sf-web1.sourceforge.net>

Bugs item #682906, was opened at 2003-02-08 15:43
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=682906&group_id=5470

Category: None
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Walter Dörwald (doerwalter)
Assigned to: Nobody/Anonymous (nobody)
Summary: refcount leak in list +=

Initial Comment:
The following script leaks references:

import sys

c = 100
L = []

def test():
·    L2 = L
·    L2 += [1]*c

for i in xrange(10):
·    test()
·    print sys.gettotalrefcount()

The number of references leaked is exactly the value of 
c.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=682906&group_id=5470


From noreply@sourceforge.net  Sat Feb  8 15:55:47 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sat, 08 Feb 2003 07:55:47 -0800
Subject: [Python-bugs-list] [ python-Bugs-682906 ] refcount leak in list +=
Message-ID: <E18hXKV-0000MQ-00@sc8-sf-web2.sourceforge.net>

Bugs item #682906, was opened at 2003-02-08 09:43
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=682906&group_id=5470

Category: None
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Walter Dörwald (doerwalter)
Assigned to: Nobody/Anonymous (nobody)
Summary: refcount leak in list +=

Initial Comment:
The following script leaks references:

import sys

c = 100
L = []

def test():
·    L2 = L
·    L2 += [1]*c

for i in xrange(10):
·    test()
·    print sys.gettotalrefcount()

The number of references leaked is exactly the value of 
c.

----------------------------------------------------------------------

>Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-08 10:55

Message:
Logged In: YES 
user_id=33168

I don't think this is a leak.  The ref count increases
because there are <c> references to 1.  If you change the
loop like so:

for j in xrange(10):
    L = []
    for i in xrange(10):
        test()
    del L
    print sys.gettotalrefcount()

The refcount is constant.  I also attached the file with the
code above.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=682906&group_id=5470


From noreply@sourceforge.net  Sat Feb  8 16:13:40 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sat, 08 Feb 2003 08:13:40 -0800
Subject: [Python-bugs-list] [ python-Bugs-682906 ] refcount leak in list +=
Message-ID: <E18hXbo-0000m6-00@sc8-sf-web3.sourceforge.net>

Bugs item #682906, was opened at 2003-02-08 15:43
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=682906&group_id=5470

Category: None
Group: Python 2.3
>Status: Deleted
>Resolution: Invalid
Priority: 5
Submitted By: Walter Dörwald (doerwalter)
Assigned to: Nobody/Anonymous (nobody)
Summary: refcount leak in list +=

Initial Comment:
The following script leaks references:

import sys

c = 100
L = []

def test():
·    L2 = L
·    L2 += [1]*c

for i in xrange(10):
·    test()
·    print sys.gettotalrefcount()

The number of references leaked is exactly the value of 
c.

----------------------------------------------------------------------

>Comment By: Walter Dörwald (doerwalter)
Date: 2003-02-08 17:13

Message:
Logged In: YES 
user_id=89016

Argl, you're right. test() *does* modify the global variable. 
Deleting the bug report.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-08 16:55

Message:
Logged In: YES 
user_id=33168

I don't think this is a leak.  The ref count increases
because there are <c> references to 1.  If you change the
loop like so:

for j in xrange(10):
    L = []
    for i in xrange(10):
        test()
    del L
    print sys.gettotalrefcount()

The refcount is constant.  I also attached the file with the
code above.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=682906&group_id=5470


From noreply@sourceforge.net  Sat Feb  8 18:12:43 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sat, 08 Feb 2003 10:12:43 -0800
Subject: [Python-bugs-list] [ python-Bugs-678518 ] Another parsermodule validation error
Message-ID: <E18hZT1-0005CM-00@sc8-sf-web2.sourceforge.net>

Bugs item #678518, was opened at 2003-02-01 02:23
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=678518&group_id=5470

Category: Parser/Compiler
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: logistix (logistix)
Assigned to: Michael Hudson (mwh)
Summary: Another parsermodule validation error

Initial Comment:
The following code generates an validation error.  It's an 
invalid ast tree where the global_stmt doesn't have the 
keyword 'global' as a child.

I tracked down the fix to validate_global_stmt() this time.

Python 2.3a1 (#38, Dec 31 2002, 17:53:59) [MSC 
v.1200 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more 
information.
>>> import symbol,token,parser
>>> ast = [symbol.file_input,
...  [symbol.stmt, [symbol.simple_stmt, 
[symbol.small_stmt, [symbol.global_stmt,
 [1, 'foo']]], [token.NEWLINE, '']]],
...  [token.NEWLINE, ''],
...  [token.ENDMARKER, '']]
>>> a = parser.sequence2ast(ast)
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
parser.ParserError: VALIDATION FAILURE: report this 
to the maintainer!
>>>

----------------------------------------------------------------------

>Comment By: Michael Hudson (mwh)
Date: 2003-02-08 18:12

Message:
Logged In: YES 
user_id=6656

OK, I've checked in revision 2.76 of Modules/parsermodule.c.

Do you have a real name?  For Misc/ACKS.

Also, would you be interested in putting an example of each
failing syntax into test_parser?

Thanks!

----------------------------------------------------------------------

Comment By: logistix (logistix)
Date: 2003-02-07 22:51

Message:
Logged In: YES 
user_id=699438

You're right, the assert statement was the unhandled 
exception.  It was raising an error but returning true on 
the "__assert__" check.

Just posted the whole patch

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2003-02-07 14:38

Message:
Logged In: YES 
user_id=6656

> At the time I didn't have an encoding patch and punted.

Oh, I misread you then.

> I did figure it out last night though.  I'll post it later
today 
> since I'm at the office now.

Ta.

> And yes, this is the error that'll print out "XXX 
> unhandled exception" out of ceval.c when you run
> the test from debug mode.

Are you sure?  It seems that was the assert one.

> There was some code added to add the encoding 
> type in the ast2sequence functions,

By me :-)  I didn't really think about the sequence2ast 
functions as all I was doing was trying to get the compiler
package to work...

> Also, as a matter of style, should I add the stuff in 
> my test script to the test suite or is that overkill? Should
> I just have it check 4 or 5 random files?

It might be best to get it to check a few files we knew there
were problems with -- test_pep263, maybe.  Checking the
whole lot is probably overkill.

----------------------------------------------------------------------

Comment By: logistix (logistix)
Date: 2003-02-07 14:23

Message:
Logged In: YES 
user_id=699438

Michael,

At the time I didn't have an encoding patch and punted.  I did 
figure it out last night though.  I'll post it later today since I'm 
at the office now.

And yes, this is the error that'll print out "XXX unhandled 
exception" out of ceval.c when you run the test from debug 
mode.

It comes down to the bastardized way that encoding is 
implemented.  It doesn't fit the current definition of a token or 
a symbol.  There was some code added to add the encoding 
type in the ast2sequence functions, but not the 
sequence2ast functions, so when it hit scripts with 
encoding_decl's, it bombed big time.

Also, as a matter of style, should I add the stuff in my test 
script to the test suite or is that overkill? Should I just have it 
check4 or 5 random files?

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2003-02-07 12:08

Message:
Logged In: YES 
user_id=6656

Pff, I was right, but logistix's patch fix it.

logistix, did you forget to attach the encoding patch?  I
still get errors in module's with a cookie.

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2003-02-07 11:58

Message:
Logged In: YES 
user_id=6656

Patches as such are probably fine.  The reason I haven't
checked them in is a wierdness that I may or may not be
related, but is bothering me.  Consider:

->> import parser, sys,glob,os, traceback
/>> for path in sys.path:
|..     for fil in glob.glob(os.path.join(path,"*.py")):
|..       try:
|..         try:
|..             a = parser.suite(file(fil).read())
|..             b = parser.ast2tuple(a)
|..             c = parser.sequence2ast(b)
|..         except parser.ParserError:
|..             print "Parse of %s" % fil, "FAILED"
|..             traceback.print_exc()
|..             print '----------------------------'
|..       except:
|..          print "waaa?"
|.. else:
|..     print "what?"
\__ 
Parse of /home/mwh/src/sf/python/dist/src/Lib/tarfile.py FAILED
Traceback (most recent call last):
  File "<input>", line 7, in ?
ParserError
----------------------------
Parse of /home/mwh/src/sf/python/dist/src/Lib/heapq.py FAILED
Traceback (most recent call last):
  File "<input>", line 7, in ?
ParserError
----------------------------
Parse of /home/mwh/src/sf/python/dist/src/Lib/getopt.py FAILED
Traceback (most recent call last):
  File "<input>", line 7, in ?
ParserError
----------------------------
Parse of /home/mwh/src/sf/python/dist/src/Lib/inspect.py FAILED
Traceback (most recent call last):
  File "<input>", line 7, in ?
ParserError
----------------------------
Parse of /home/mwh/src/sf/python/dist/src/Lib/pydoc.py FAILED
Traceback (most recent call last):
  File "<input>", line 7, in ?
ParserError
----------------------------
Parse of /home/mwh/src/sf/python/dist/src/Lib/lib-tk/Tix.py
FAILED
Traceback (most recent call last):
  File "<input>", line 7, in ?
ParserError
----------------------------
Traceback (most recent call last):
  File "<input>", line 2, in ?
ParserError: Illegal terminal: expected "__assert__"
->> 

Notice that an exception seems to have made it through the
except clauses!  The scary bit is that a (seemingly) random
number of parser errors *will* caught, and then one will get
through.

The only explanation I have for this is some kind of
returning non-NULL with exception set kind of bug.

Off to play with a debug build, I guess.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-07 01:17

Message:
Logged In: YES 
user_id=33168

Michael were these patches ok to check in?

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2003-02-03 11:00

Message:
Logged In: YES 
user_id=6656

Thanks for doing this.  I'll probably check these in after
I've finished crunching my way through monday morning email...

----------------------------------------------------------------------

Comment By: logistix (logistix)
Date: 2003-02-01 21:42

Message:
Logged In: YES 
user_id=699438

And I found a few more by doing a roundtrip 
ast2Tuple/tuple2ast on all files in pythonpath.  These actually 
affect valid AST's, so they're probably a bigger deal.

Valid assert statements were failing.  The patch was trivial, 
so its attached.

Also, files that took advantage of encoding_decl (from PEP 
263) were failing.  I couldn't tell if PEP 263 was offically 
closed, and didn't know what the offical grammar was, so I 
didn't do anything there.

I'll also attach the test script that turned up these errors.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=678518&group_id=5470


From noreply@sourceforge.net  Sat Feb  8 19:52:33 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sat, 08 Feb 2003 11:52:33 -0800
Subject: [Python-bugs-list] [ python-Bugs-683061 ] gzip module cannot write large files
Message-ID: <E18hb1d-0006jS-00@sc8-sf-web4.sourceforge.net>

Bugs item #683061, was opened at 2003-02-08 11:52
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=683061&group_id=5470

Category: Python Library
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Ben Escoto (bescoto)
Assigned to: Nobody/Anonymous (nobody)
Summary: gzip module cannot write large files

Initial Comment:
Something like this:

import gzip
fin = open("/dev/urandom", "rb")
fout = gzip.GzipFile("out", "wb")
for i in range(4500): fout.write(fin.read(1024*1024))
fout.close()

will result in an error like:

Exception 'long int too large to convert to int' raised
of class 'exceptions.OverflowError':
...
  File "/usr/local/lib/python2.2/gzip.py", line 253, in
close
    write32(self.fileobj, self.size)
  File "/usr/local/lib/python2.2/gzip.py", line 19, in
write32
    output.write(struct.pack("<l", value))

Tested in python 2.2.1


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=683061&group_id=5470


From noreply@sourceforge.net  Sat Feb  8 21:15:33 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sat, 08 Feb 2003 13:15:33 -0800
Subject: [Python-bugs-list] [ python-Bugs-678518 ] Another parsermodule validation error
Message-ID: <E18hcJx-0003AQ-00@sc8-sf-web2.sourceforge.net>

Bugs item #678518, was opened at 2003-01-31 20:23
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=678518&group_id=5470

Category: Parser/Compiler
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: logistix (logistix)
Assigned to: Michael Hudson (mwh)
Summary: Another parsermodule validation error

Initial Comment:
The following code generates an validation error.  It's an 
invalid ast tree where the global_stmt doesn't have the 
keyword 'global' as a child.

I tracked down the fix to validate_global_stmt() this time.

Python 2.3a1 (#38, Dec 31 2002, 17:53:59) [MSC 
v.1200 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more 
information.
>>> import symbol,token,parser
>>> ast = [symbol.file_input,
...  [symbol.stmt, [symbol.simple_stmt, 
[symbol.small_stmt, [symbol.global_stmt,
 [1, 'foo']]], [token.NEWLINE, '']]],
...  [token.NEWLINE, ''],
...  [token.ENDMARKER, '']]
>>> a = parser.sequence2ast(ast)
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
parser.ParserError: VALIDATION FAILURE: report this 
to the maintainer!
>>>

----------------------------------------------------------------------

>Comment By: logistix (logistix)
Date: 2003-02-08 15:15

Message:
Logged In: YES 
user_id=699438

I attached some changes for the testsuite.

My name is Grant Olson.

Looks like we can close this ticket out ;-)

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2003-02-08 12:12

Message:
Logged In: YES 
user_id=6656

OK, I've checked in revision 2.76 of Modules/parsermodule.c.

Do you have a real name?  For Misc/ACKS.

Also, would you be interested in putting an example of each
failing syntax into test_parser?

Thanks!

----------------------------------------------------------------------

Comment By: logistix (logistix)
Date: 2003-02-07 16:51

Message:
Logged In: YES 
user_id=699438

You're right, the assert statement was the unhandled 
exception.  It was raising an error but returning true on 
the "__assert__" check.

Just posted the whole patch

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2003-02-07 08:38

Message:
Logged In: YES 
user_id=6656

> At the time I didn't have an encoding patch and punted.

Oh, I misread you then.

> I did figure it out last night though.  I'll post it later
today 
> since I'm at the office now.

Ta.

> And yes, this is the error that'll print out "XXX 
> unhandled exception" out of ceval.c when you run
> the test from debug mode.

Are you sure?  It seems that was the assert one.

> There was some code added to add the encoding 
> type in the ast2sequence functions,

By me :-)  I didn't really think about the sequence2ast 
functions as all I was doing was trying to get the compiler
package to work...

> Also, as a matter of style, should I add the stuff in 
> my test script to the test suite or is that overkill? Should
> I just have it check 4 or 5 random files?

It might be best to get it to check a few files we knew there
were problems with -- test_pep263, maybe.  Checking the
whole lot is probably overkill.

----------------------------------------------------------------------

Comment By: logistix (logistix)
Date: 2003-02-07 08:23

Message:
Logged In: YES 
user_id=699438

Michael,

At the time I didn't have an encoding patch and punted.  I did 
figure it out last night though.  I'll post it later today since I'm 
at the office now.

And yes, this is the error that'll print out "XXX unhandled 
exception" out of ceval.c when you run the test from debug 
mode.

It comes down to the bastardized way that encoding is 
implemented.  It doesn't fit the current definition of a token or 
a symbol.  There was some code added to add the encoding 
type in the ast2sequence functions, but not the 
sequence2ast functions, so when it hit scripts with 
encoding_decl's, it bombed big time.

Also, as a matter of style, should I add the stuff in my test 
script to the test suite or is that overkill? Should I just have it 
check4 or 5 random files?

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2003-02-07 06:08

Message:
Logged In: YES 
user_id=6656

Pff, I was right, but logistix's patch fix it.

logistix, did you forget to attach the encoding patch?  I
still get errors in module's with a cookie.

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2003-02-07 05:58

Message:
Logged In: YES 
user_id=6656

Patches as such are probably fine.  The reason I haven't
checked them in is a wierdness that I may or may not be
related, but is bothering me.  Consider:

->> import parser, sys,glob,os, traceback
/>> for path in sys.path:
|..     for fil in glob.glob(os.path.join(path,"*.py")):
|..       try:
|..         try:
|..             a = parser.suite(file(fil).read())
|..             b = parser.ast2tuple(a)
|..             c = parser.sequence2ast(b)
|..         except parser.ParserError:
|..             print "Parse of %s" % fil, "FAILED"
|..             traceback.print_exc()
|..             print '----------------------------'
|..       except:
|..          print "waaa?"
|.. else:
|..     print "what?"
\__ 
Parse of /home/mwh/src/sf/python/dist/src/Lib/tarfile.py FAILED
Traceback (most recent call last):
  File "<input>", line 7, in ?
ParserError
----------------------------
Parse of /home/mwh/src/sf/python/dist/src/Lib/heapq.py FAILED
Traceback (most recent call last):
  File "<input>", line 7, in ?
ParserError
----------------------------
Parse of /home/mwh/src/sf/python/dist/src/Lib/getopt.py FAILED
Traceback (most recent call last):
  File "<input>", line 7, in ?
ParserError
----------------------------
Parse of /home/mwh/src/sf/python/dist/src/Lib/inspect.py FAILED
Traceback (most recent call last):
  File "<input>", line 7, in ?
ParserError
----------------------------
Parse of /home/mwh/src/sf/python/dist/src/Lib/pydoc.py FAILED
Traceback (most recent call last):
  File "<input>", line 7, in ?
ParserError
----------------------------
Parse of /home/mwh/src/sf/python/dist/src/Lib/lib-tk/Tix.py
FAILED
Traceback (most recent call last):
  File "<input>", line 7, in ?
ParserError
----------------------------
Traceback (most recent call last):
  File "<input>", line 2, in ?
ParserError: Illegal terminal: expected "__assert__"
->> 

Notice that an exception seems to have made it through the
except clauses!  The scary bit is that a (seemingly) random
number of parser errors *will* caught, and then one will get
through.

The only explanation I have for this is some kind of
returning non-NULL with exception set kind of bug.

Off to play with a debug build, I guess.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-06 19:17

Message:
Logged In: YES 
user_id=33168

Michael were these patches ok to check in?

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2003-02-03 05:00

Message:
Logged In: YES 
user_id=6656

Thanks for doing this.  I'll probably check these in after
I've finished crunching my way through monday morning email...

----------------------------------------------------------------------

Comment By: logistix (logistix)
Date: 2003-02-01 15:42

Message:
Logged In: YES 
user_id=699438

And I found a few more by doing a roundtrip 
ast2Tuple/tuple2ast on all files in pythonpath.  These actually 
affect valid AST's, so they're probably a bigger deal.

Valid assert statements were failing.  The patch was trivial, 
so its attached.

Also, files that took advantage of encoding_decl (from PEP 
263) were failing.  I couldn't tell if PEP 263 was offically 
closed, and didn't know what the offical grammar was, so I 
didn't do anything there.

I'll also attach the test script that turned up these errors.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=678518&group_id=5470


From noreply@sourceforge.net  Sat Feb  8 23:18:55 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sat, 08 Feb 2003 15:18:55 -0800
Subject: [Python-bugs-list] [ python-Bugs-585923 ] macfs.FSSpec and Carbon.File.FSSpec fail for "new" files
Message-ID: <E18heFL-0005Dj-00@sc8-sf-web4.sourceforge.net>

Bugs item #585923, was opened at 2002-07-24 15:35
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=585923&group_id=5470

Category: Macintosh
Group: Python 2.3
Status: Open
Resolution: None
>Priority: 4
Submitted By: Just van Rossum (jvr)
Assigned to: Jack Jansen (jackjansen)
>Summary: macfs.FSSpec and Carbon.File.FSSpec fail for "new" files

Initial Comment:
In the mach-o version of Python on MacOSX, the macfs.FSSpec() 
library function fails for files that don't yet exist.

The problem is caused by (internally) using an FSRef to construct 
the FSSpec: FSRefs can't hold references to non-existent files, 
whereas FSSpecs can.

----------------------------------------------------------------------

>Comment By: Jack Jansen (jackjansen)
Date: 2003-02-09 00:18

Message:
Logged In: YES 
user_id=45365

No, the problem still exists, also in Carbon.File, and probably is solvable. It has become less urgent, though, as FSSpecs are phased out in the core.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-07 23:20

Message:
Logged In: YES 
user_id=33168

Can this be closed since macfs has been removed?

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2002-08-05 23:54

Message:
Logged In: YES 
user_id=45365

macostools.copy() has been fixed in rev. 1.16. mkalias()
still has the problem, though.

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2002-08-03 23:01

Message:
Logged In: YES 
user_id=45365

As the workaround is going to be both difficult to implement and incomplete I'm tempted to punt on this one. Difficult because we have to split the path in C, and we have to do /: substitutions on the last component; incomplete because we cannot cater for utf8/macroman differences and filenames > 32 characters.

If we punt on this we should make sure we phase out using fsspec's in the standard library in favor of the fsref calls.

Do you think this is an acceptable solution? How much work would it be?

----------------------------------------------------------------------

Comment By: Just van Rossum (jvr)
Date: 2002-07-29 23:56

Message:
Logged In: YES 
user_id=92689

This bug also cause macostools.copy() to fail, making the breakage 
larger than I thought :-(

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=585923&group_id=5470


From noreply@sourceforge.net  Sat Feb  8 23:50:22 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sat, 08 Feb 2003 15:50:22 -0800
Subject: [Python-bugs-list] [ python-Bugs-683160 ] Reading while writing-only permissions
Message-ID: <E18hejm-0001a0-00@sc8-sf-web1.sourceforge.net>

Bugs item #683160, was opened at 2003-02-09 00:50
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=683160&group_id=5470

Category: Python Library
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Pablo de la Rosa (quevedo)
Assigned to: Nobody/Anonymous (nobody)
Summary: Reading while writing-only permissions

Initial Comment:
Hello world.




Well, let's see. When you open a file with "write-only" 
permission, then you write some in and after that you 
print the text contained in the file, you get your code (or 
almost all) and some other characters by output (See 1 
). 




- 1:




a) Code:


#!/usr/bin/python


file = open("test.txt", "w")


file.write("We love Python")


text = file.read()


print text




b) Output:


ÿÿ    





pen("test.txt", "w")


file.write("We love Python")


text = file.read()


print text


ext




ufferTypes   tuples	   TupleTypes   lists   
ListTypes   dicts   DictTypes   DictionaryTypes   
_fs   FunctionTypes


   LambdaTypes	   func_codes   CodeTypes   
RuntimeErrors
   gs


   GeneratorTypes   _Cs	   ClassTypes   _ms   
UnboundMethodTypes   _xs   InstanceTypes


   MethodTypes   lens   BuiltinFunctionTypes   
appends   BuiltinMethodTypes


   ModuleTypes   files   FileTypes   xranges


   XRangeTypes	   TypeErrors   exc_infos   tbs


   TracebackTypes   tb_frames	   FrameTypes   
AttributeErrors   slices	   SliceTypes   Ellipsiss   
EllipsisTypes   __dict__s


   DictProxyType((   s   IntTypes   tbs   
BuiltinFunctionTypes   BooleanTypes   _Cs   
UnboundMethodTypes


   StringTypes   BuiltinMethodTypes	   FloatTypes   
DictionaryTypes   TypeTypes


   DictProxyTypes   _fs


   GeneratorTypes   InstanceTypes


   ObjectTypes   DictTypes   FileTypes   syss   
EllipsisTypes   Strý%  Lÿh HNh  ListTypes


   MethodTypes	   TupleTypes


   ModuleTypes	   FrameTypes   LongTypes


   BufferTypes


   TracebackTypes
   gs   CodeTypes	   
ClassTypes   _xs   UnicodeTypes	   SliceTypes   
ComplexTypes


   LambdaTypes   FunctionTypes


   XRangeTypes   NoneType(    (    s   
C:\PYTHON23\lib\types.pys
   ?   sr   

	




















	




	


	

















    s   |  
i
 i ?  Sd  S(
   N(   s   selfs   datas


   itervalues(
   s   self(    (    s   C




---------------------------------------------------




But if you try this without writing to the file you get just 
an error (See 2).




- 2:




a) Code:


#!/usr/bin/python


file = open("test.txt", "w")


text = file.read()


print text




b) Output:


Traceback (most recent call last):


  File "tralara.py", line 3, in ?


    text = file.read()


IOError: [Errno 9] Bad file descriptor




------------------------------------------------




It's stupid, I know. Why somenone would want to read a 
file after writing to it ? Don't know, I've discovered this 
by error ;)




Tested in Python 2.3 Beta (Win32).




Happy coding friends.




----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=683160&group_id=5470


From noreply@sourceforge.net  Sun Feb  9 00:31:28 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sat, 08 Feb 2003 16:31:28 -0800
Subject: [Python-bugs-list] [ python-Bugs-683160 ] Reading while writing-only permissions
Message-ID: <E18hfNY-00034E-00@sc8-sf-web1.sourceforge.net>

Bugs item #683160, was opened at 2003-02-08 18:50
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=683160&group_id=5470

Category: Python Library
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Pablo de la Rosa (quevedo)
>Assigned to: Tim Peters (tim_one)
Summary: Reading while writing-only permissions

Initial Comment:
Hello world.




Well, let's see. When you open a file with "write-only" 
permission, then you write some in and after that you 
print the text contained in the file, you get your code (or 
almost all) and some other characters by output (See 1 
). 




- 1:




a) Code:


#!/usr/bin/python


file = open("test.txt", "w")


file.write("We love Python")


text = file.read()


print text




b) Output:


ÿÿ    





pen("test.txt", "w")


file.write("We love Python")


text = file.read()


print text


ext




ufferTypes   tuples	   TupleTypes   lists   
ListTypes   dicts   DictTypes   DictionaryTypes   
_fs   FunctionTypes


   LambdaTypes	   func_codes   CodeTypes   
RuntimeErrors
   gs


   GeneratorTypes   _Cs	   ClassTypes   _ms   
UnboundMethodTypes   _xs   InstanceTypes


   MethodTypes   lens   BuiltinFunctionTypes   
appends   BuiltinMethodTypes


   ModuleTypes   files   FileTypes   xranges


   XRangeTypes	   TypeErrors   exc_infos   tbs


   TracebackTypes   tb_frames	   FrameTypes   
AttributeErrors   slices	   SliceTypes   Ellipsiss   
EllipsisTypes   __dict__s


   DictProxyType((   s   IntTypes   tbs   
BuiltinFunctionTypes   BooleanTypes   _Cs   
UnboundMethodTypes


   StringTypes   BuiltinMethodTypes	   FloatTypes   
DictionaryTypes   TypeTypes


   DictProxyTypes   _fs


   GeneratorTypes   InstanceTypes


   ObjectTypes   DictTypes   FileTypes   syss   
EllipsisTypes   Strý%  Lÿh HNh  ListTypes


   MethodTypes	   TupleTypes


   ModuleTypes	   FrameTypes   LongTypes


   BufferTypes


   TracebackTypes
   gs   CodeTypes	   
ClassTypes   _xs   UnicodeTypes	   SliceTypes   
ComplexTypes


   LambdaTypes   FunctionTypes


   XRangeTypes   NoneType(    (    s   
C:\PYTHON23\lib\types.pys
   ?   sr   

	




















	




	


	

















    s   |  
i
 i ?  Sd  S(
   N(   s   selfs   datas


   itervalues(
   s   self(    (    s   C




---------------------------------------------------




But if you try this without writing to the file you get just 
an error (See 2).




- 2:




a) Code:


#!/usr/bin/python


file = open("test.txt", "w")


text = file.read()


print text




b) Output:


Traceback (most recent call last):


  File "tralara.py", line 3, in ?


    text = file.read()


IOError: [Errno 9] Bad file descriptor




------------------------------------------------




It's stupid, I know. Why somenone would want to read a 
file after writing to it ? Don't know, I've discovered this 
by error ;)




Tested in Python 2.3 Beta (Win32).




Happy coding friends.




----------------------------------------------------------------------

>Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-08 19:31

Message:
Logged In: YES 
user_id=33168

I cannot reproduce on Linux, I'm guessing this is a windows bug.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=683160&group_id=5470


From noreply@sourceforge.net  Sun Feb  9 00:47:22 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sat, 08 Feb 2003 16:47:22 -0800
Subject: [Python-bugs-list] [ python-Bugs-683160 ] Reading while writing-only permissions
Message-ID: <E18hfcw-00008t-00@sc8-sf-web3.sourceforge.net>

Bugs item #683160, was opened at 2003-02-08 18:50
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=683160&group_id=5470

Category: Python Library
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Pablo de la Rosa (quevedo)
Assigned to: Tim Peters (tim_one)
Summary: Reading while writing-only permissions

Initial Comment:
Hello world.




Well, let's see. When you open a file with "write-only" 
permission, then you write some in and after that you 
print the text contained in the file, you get your code (or 
almost all) and some other characters by output (See 1 
). 




- 1:




a) Code:


#!/usr/bin/python


file = open("test.txt", "w")


file.write("We love Python")


text = file.read()


print text




b) Output:


ÿÿ    





pen("test.txt", "w")


file.write("We love Python")


text = file.read()


print text


ext




ufferTypes   tuples	   TupleTypes   lists   
ListTypes   dicts   DictTypes   DictionaryTypes   
_fs   FunctionTypes


   LambdaTypes	   func_codes   CodeTypes   
RuntimeErrors
   gs


   GeneratorTypes   _Cs	   ClassTypes   _ms   
UnboundMethodTypes   _xs   InstanceTypes


   MethodTypes   lens   BuiltinFunctionTypes   
appends   BuiltinMethodTypes


   ModuleTypes   files   FileTypes   xranges


   XRangeTypes	   TypeErrors   exc_infos   tbs


   TracebackTypes   tb_frames	   FrameTypes   
AttributeErrors   slices	   SliceTypes   Ellipsiss   
EllipsisTypes   __dict__s


   DictProxyType((   s   IntTypes   tbs   
BuiltinFunctionTypes   BooleanTypes   _Cs   
UnboundMethodTypes


   StringTypes   BuiltinMethodTypes	   FloatTypes   
DictionaryTypes   TypeTypes


   DictProxyTypes   _fs


   GeneratorTypes   InstanceTypes


   ObjectTypes   DictTypes   FileTypes   syss   
EllipsisTypes   Strý%  Lÿh HNh  ListTypes


   MethodTypes	   TupleTypes


   ModuleTypes	   FrameTypes   LongTypes


   BufferTypes


   TracebackTypes
   gs   CodeTypes	   
ClassTypes   _xs   UnicodeTypes	   SliceTypes   
ComplexTypes


   LambdaTypes   FunctionTypes


   XRangeTypes   NoneType(    (    s   
C:\PYTHON23\lib\types.pys
   ?   sr   

	




















	




	


	

















    s   |  
i
 i ?  Sd  S(
   N(   s   selfs   datas


   itervalues(
   s   self(    (    s   C




---------------------------------------------------




But if you try this without writing to the file you get just 
an error (See 2).




- 2:




a) Code:


#!/usr/bin/python


file = open("test.txt", "w")


text = file.read()


print text




b) Output:


Traceback (most recent call last):


  File "tralara.py", line 3, in ?


    text = file.read()


IOError: [Errno 9] Bad file descriptor




------------------------------------------------




It's stupid, I know. Why somenone would want to read a 
file after writing to it ? Don't know, I've discovered this 
by error ;)




Tested in Python 2.3 Beta (Win32).




Happy coding friends.




----------------------------------------------------------------------

>Comment By: Tim Peters (tim_one)
Date: 2003-02-08 19:47

Message:
Logged In: YES 
user_id=31435

Python doesn't implement files, the operating system and 
platform C libraries do that.  If the same thing happens in a 
C program, there's no hope.  Which version of Windows 
were you running (their file implementations aren't the 
same, especially not 95/98/ME vs NT/2000/XP)?

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-08 19:31

Message:
Logged In: YES 
user_id=33168

I cannot reproduce on Linux, I'm guessing this is a windows bug.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=683160&group_id=5470


From noreply@sourceforge.net  Sun Feb  9 01:06:28 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sat, 08 Feb 2003 17:06:28 -0800
Subject: [Python-bugs-list] [ python-Bugs-683160 ] Reading while writing-only permissions
Message-ID: <E18hfvQ-0008Fw-00@sc8-sf-web4.sourceforge.net>

Bugs item #683160, was opened at 2003-02-09 00:50
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=683160&group_id=5470

Category: Python Library
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Pablo de la Rosa (quevedo)
Assigned to: Tim Peters (tim_one)
Summary: Reading while writing-only permissions

Initial Comment:
Hello world.




Well, let's see. When you open a file with "write-only" 
permission, then you write some in and after that you 
print the text contained in the file, you get your code (or 
almost all) and some other characters by output (See 1 
). 




- 1:




a) Code:


#!/usr/bin/python


file = open("test.txt", "w")


file.write("We love Python")


text = file.read()


print text




b) Output:


ÿÿ    





pen("test.txt", "w")


file.write("We love Python")


text = file.read()


print text


ext




ufferTypes   tuples	   TupleTypes   lists   
ListTypes   dicts   DictTypes   DictionaryTypes   
_fs   FunctionTypes


   LambdaTypes	   func_codes   CodeTypes   
RuntimeErrors
   gs


   GeneratorTypes   _Cs	   ClassTypes   _ms   
UnboundMethodTypes   _xs   InstanceTypes


   MethodTypes   lens   BuiltinFunctionTypes   
appends   BuiltinMethodTypes


   ModuleTypes   files   FileTypes   xranges


   XRangeTypes	   TypeErrors   exc_infos   tbs


   TracebackTypes   tb_frames	   FrameTypes   
AttributeErrors   slices	   SliceTypes   Ellipsiss   
EllipsisTypes   __dict__s


   DictProxyType((   s   IntTypes   tbs   
BuiltinFunctionTypes   BooleanTypes   _Cs   
UnboundMethodTypes


   StringTypes   BuiltinMethodTypes	   FloatTypes   
DictionaryTypes   TypeTypes


   DictProxyTypes   _fs


   GeneratorTypes   InstanceTypes


   ObjectTypes   DictTypes   FileTypes   syss   
EllipsisTypes   Strý%  Lÿh HNh  ListTypes


   MethodTypes	   TupleTypes


   ModuleTypes	   FrameTypes   LongTypes


   BufferTypes


   TracebackTypes
   gs   CodeTypes	   
ClassTypes   _xs   UnicodeTypes	   SliceTypes   
ComplexTypes


   LambdaTypes   FunctionTypes


   XRangeTypes   NoneType(    (    s   
C:\PYTHON23\lib\types.pys
   ?   sr   

	




















	




	


	

















    s   |  
i
 i ?  Sd  S(
   N(   s   selfs   datas


   itervalues(
   s   self(    (    s   C




---------------------------------------------------




But if you try this without writing to the file you get just 
an error (See 2).




- 2:




a) Code:


#!/usr/bin/python


file = open("test.txt", "w")


text = file.read()


print text




b) Output:


Traceback (most recent call last):


  File "tralara.py", line 3, in ?


    text = file.read()


IOError: [Errno 9] Bad file descriptor




------------------------------------------------




It's stupid, I know. Why somenone would want to read a 
file after writing to it ? Don't know, I've discovered this 
by error ;)




Tested in Python 2.3 Beta (Win32).




Happy coding friends.




----------------------------------------------------------------------

>Comment By: Pablo de la Rosa (quevedo)
Date: 2003-02-09 02:06

Message:
Logged In: YES 
user_id=707875

Windows 98 Second Edition ;)


----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2003-02-09 01:47

Message:
Logged In: YES 
user_id=31435

Python doesn't implement files, the operating system and 
platform C libraries do that.  If the same thing happens in a 
C program, there's no hope.  Which version of Windows 
were you running (their file implementations aren't the 
same, especially not 95/98/ME vs NT/2000/XP)?

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-09 01:31

Message:
Logged In: YES 
user_id=33168

I cannot reproduce on Linux, I'm guessing this is a windows bug.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=683160&group_id=5470


From noreply@sourceforge.net  Sun Feb  9 03:20:11 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sat, 08 Feb 2003 19:20:11 -0800
Subject: [Python-bugs-list] [ python-Bugs-683160 ] Reading while writing-only permissions
Message-ID: <E18hi0p-0003Xu-00@sc8-sf-web4.sourceforge.net>

Bugs item #683160, was opened at 2003-02-08 18:50
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=683160&group_id=5470

>Category: Windows
>Group: 3rd Party
>Status: Closed
>Resolution: Wont Fix
Priority: 5
Submitted By: Pablo de la Rosa (quevedo)
Assigned to: Tim Peters (tim_one)
Summary: Reading while writing-only permissions

Initial Comment:
Hello world.




Well, let's see. When you open a file with "write-only" 
permission, then you write some in and after that you 
print the text contained in the file, you get your code (or 
almost all) and some other characters by output (See 1 
). 




- 1:




a) Code:


#!/usr/bin/python


file = open("test.txt", "w")


file.write("We love Python")


text = file.read()


print text




b) Output:


ÿÿ    





pen("test.txt", "w")


file.write("We love Python")


text = file.read()


print text


ext




ufferTypes   tuples	   TupleTypes   lists   
ListTypes   dicts   DictTypes   DictionaryTypes   
_fs   FunctionTypes


   LambdaTypes	   func_codes   CodeTypes   
RuntimeErrors
   gs


   GeneratorTypes   _Cs	   ClassTypes   _ms   
UnboundMethodTypes   _xs   InstanceTypes


   MethodTypes   lens   BuiltinFunctionTypes   
appends   BuiltinMethodTypes


   ModuleTypes   files   FileTypes   xranges


   XRangeTypes	   TypeErrors   exc_infos   tbs


   TracebackTypes   tb_frames	   FrameTypes   
AttributeErrors   slices	   SliceTypes   Ellipsiss   
EllipsisTypes   __dict__s


   DictProxyType((   s   IntTypes   tbs   
BuiltinFunctionTypes   BooleanTypes   _Cs   
UnboundMethodTypes


   StringTypes   BuiltinMethodTypes	   FloatTypes   
DictionaryTypes   TypeTypes


   DictProxyTypes   _fs


   GeneratorTypes   InstanceTypes


   ObjectTypes   DictTypes   FileTypes   syss   
EllipsisTypes   Strý%  Lÿh HNh  ListTypes


   MethodTypes	   TupleTypes


   ModuleTypes	   FrameTypes   LongTypes


   BufferTypes


   TracebackTypes
   gs   CodeTypes	   
ClassTypes   _xs   UnicodeTypes	   SliceTypes   
ComplexTypes


   LambdaTypes   FunctionTypes


   XRangeTypes   NoneType(    (    s   
C:\PYTHON23\lib\types.pys
   ?   sr   

	




















	




	


	

















    s   |  
i
 i ?  Sd  S(
   N(   s   selfs   datas


   itervalues(
   s   self(    (    s   C




---------------------------------------------------




But if you try this without writing to the file you get just 
an error (See 2).




- 2:




a) Code:


#!/usr/bin/python


file = open("test.txt", "w")


text = file.read()


print text




b) Output:


Traceback (most recent call last):


  File "tralara.py", line 3, in ?


    text = file.read()


IOError: [Errno 9] Bad file descriptor




------------------------------------------------




It's stupid, I know. Why somenone would want to read a 
file after writing to it ? Don't know, I've discovered this 
by error ;)




Tested in Python 2.3 Beta (Win32).




Happy coding friends.




----------------------------------------------------------------------

>Comment By: Tim Peters (tim_one)
Date: 2003-02-08 22:20

Message:
Logged In: YES 
user_id=31435

Figures <wink>.  I tried this C program on Win98SE, using 
Microsoft's MSVC 6:

#include <stdio.h>
#include <stdlib.h>

void main()
{
    char buf[1000];
    size_t n;
    FILE *f = fopen("temp.txt", "w");

    fprintf(f, "%s\n", "Hello, world!");
    memset(buf, ' ', sizeof(buf));
    n = fread(buf, 1, sizeof(buf), f);
    printf("read %d chars, ferror is %d, feof is %d\n",
           n, ferror(f), feof(f));
}

It didn't complain.  The output was

read 1000 chars, ferror is 0, feof is 0

Since the OS doesn't complain, there's really nothing 
Python can do about it short of writing our own file 
implementation, and that's a huge project.

When I boosted the buf size in the above to 10000, it said 
it read 4082 characters, suggesting it's just reading 
whatever bits were left on the disk and sucked into its 
internal buffer.  File temp.txt was 4097 bytes when the 
program ended.

Since Python isn't doing any of this, I'm closing this as 
Windows, 3rdParty, and WontFix.  BTW, I use Win98SE 
too at home, but I don't expect it to act like a real 
operating system <wink>.

----------------------------------------------------------------------

Comment By: Pablo de la Rosa (quevedo)
Date: 2003-02-08 20:06

Message:
Logged In: YES 
user_id=707875

Windows 98 Second Edition ;)


----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2003-02-08 19:47

Message:
Logged In: YES 
user_id=31435

Python doesn't implement files, the operating system and 
platform C libraries do that.  If the same thing happens in a 
C program, there's no hope.  Which version of Windows 
were you running (their file implementations aren't the 
same, especially not 95/98/ME vs NT/2000/XP)?

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-08 19:31

Message:
Logged In: YES 
user_id=33168

I cannot reproduce on Linux, I'm guessing this is a windows bug.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=683160&group_id=5470


From noreply@sourceforge.net  Sun Feb  9 04:05:41 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sat, 08 Feb 2003 20:05:41 -0800
Subject: [Python-bugs-list] [ python-Bugs-580952 ] import lock should be exposed
Message-ID: <E18hiir-0000R1-00@sc8-sf-web1.sourceforge.net>

Bugs item #580952, was opened at 2002-07-13 08:06
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=580952&group_id=5470

Category: Python Interpreter Core
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Gordon B. McMillan (gmcm)
Assigned to: Guido van Rossum (gvanrossum)
Summary: import lock should be exposed

Initial Comment:
See
http://mail.python.org/pipermail/import-sig/2002-July/000179.html

In brief, a good import hook needs to use an import
lock. But if it creates it's own import lock, you have the
possibility of a deadlock, since PyImport_ImportModuleEx
still uses the core lock. So one thread can end up
acquiring the hook's lock, then seeking the core lock.
While another holds the core lock, but  does an import
(routed through the hook) and seeks the hook's lock.

Seems to me exposing the core lock through
imp.acquire_lock() and imp.release_lock() would
allow a properly coded hook  to avoid this.

----------------------------------------------------------------------

Comment By: Jason Hildebrand (jdhildeb)
Date: 2003-02-08 22:05

Message:
Logged In: YES 
user_id=173690

For what it's worth, Webware for Python users are exposed to
this bug, too, as Webware is multithreaded and uses an
import hook.  This has caused me some amount of grief, and
I've been happy to figure out the cause (the import
collisions were causing really weird things to happen).

I've attached a patch to import.c which exposes the
acquire_lock() and release_lock() functions.  Since
modifying Webware's import routine to use this these
functions, I have been completely unable to reproduce the
buggy behaviour (which had previously been easy to reproduce).

It would be great to have this merged in for Python 2.3, as
projects using both custom import hooks and threads have no
other reliable way of solving this problem.


----------------------------------------------------------------------

Comment By: Gordon B. McMillan (gmcm)
Date: 2002-07-14 16:34

Message:
Logged In: YES 
user_id=4923

Yes, I would just like to expose the current lock_import and 
unlock_import to C and Python.

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2002-07-14 16:06

Message:
Logged In: YES 
user_id=31435

I don't understand how imports work, at least not at this 
level.  I note that the current lock_import() and 
unlock_import() arrange to implement a reentrant lock:  in 
the absence of you supplying a patch, are these the 
functions you would like to see exposed?  Given the games 
being played here, it seems too dangerous to expose the 
*underlying* lock's acquire and release methods directly.

Assigned to Guido for cogitation.

----------------------------------------------------------------------

Comment By: Gordon B. McMillan (gmcm)
Date: 2002-07-13 08:10

Message:
Logged In: YES 
user_id=4923

Assigning to Tim for initial review.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=580952&group_id=5470


From noreply@sourceforge.net  Sun Feb  9 04:32:32 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sat, 08 Feb 2003 20:32:32 -0800
Subject: [Python-bugs-list] [ python-Bugs-580952 ] import lock should be exposed
Message-ID: <E18hj8q-0005a9-00@sc8-sf-web3.sourceforge.net>

Bugs item #580952, was opened at 2002-07-13 08:06
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=580952&group_id=5470

Category: Python Interpreter Core
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Gordon B. McMillan (gmcm)
Assigned to: Guido van Rossum (gvanrossum)
Summary: import lock should be exposed

Initial Comment:
See
http://mail.python.org/pipermail/import-sig/2002-July/000179.html

In brief, a good import hook needs to use an import
lock. But if it creates it's own import lock, you have the
possibility of a deadlock, since PyImport_ImportModuleEx
still uses the core lock. So one thread can end up
acquiring the hook's lock, then seeking the core lock.
While another holds the core lock, but  does an import
(routed through the hook) and seeks the hook's lock.

Seems to me exposing the core lock through
imp.acquire_lock() and imp.release_lock() would
allow a properly coded hook  to avoid this.

----------------------------------------------------------------------

Comment By: Jason Hildebrand (jdhildeb)
Date: 2003-02-08 22:32

Message:
Logged In: YES 
user_id=173690

I've attached the patches against both 2.3-2.2.99 and 2.2.2;
they are available in patch # 683257.


----------------------------------------------------------------------

Comment By: Jason Hildebrand (jdhildeb)
Date: 2003-02-08 22:05

Message:
Logged In: YES 
user_id=173690

For what it's worth, Webware for Python users are exposed to
this bug, too, as Webware is multithreaded and uses an
import hook.  This has caused me some amount of grief, and
I've been happy to figure out the cause (the import
collisions were causing really weird things to happen).

I've attached a patch to import.c which exposes the
acquire_lock() and release_lock() functions.  Since
modifying Webware's import routine to use this these
functions, I have been completely unable to reproduce the
buggy behaviour (which had previously been easy to reproduce).

It would be great to have this merged in for Python 2.3, as
projects using both custom import hooks and threads have no
other reliable way of solving this problem.


----------------------------------------------------------------------

Comment By: Gordon B. McMillan (gmcm)
Date: 2002-07-14 16:34

Message:
Logged In: YES 
user_id=4923

Yes, I would just like to expose the current lock_import and 
unlock_import to C and Python.

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2002-07-14 16:06

Message:
Logged In: YES 
user_id=31435

I don't understand how imports work, at least not at this 
level.  I note that the current lock_import() and 
unlock_import() arrange to implement a reentrant lock:  in 
the absence of you supplying a patch, are these the 
functions you would like to see exposed?  Given the games 
being played here, it seems too dangerous to expose the 
*underlying* lock's acquire and release methods directly.

Assigned to Guido for cogitation.

----------------------------------------------------------------------

Comment By: Gordon B. McMillan (gmcm)
Date: 2002-07-13 08:10

Message:
Logged In: YES 
user_id=4923

Assigning to Tim for initial review.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=580952&group_id=5470


From noreply@sourceforge.net  Sun Feb  9 14:52:29 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sun, 09 Feb 2003 06:52:29 -0800
Subject: [Python-bugs-list] [ python-Bugs-683416 ] Whats new: Obligatory source code encodings if not ASCII
Message-ID: <E18hson-0003B9-00@sc8-sf-web1.sourceforge.net>

Bugs item #683416, was opened at 2003-02-09 14:52
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=683416&group_id=5470

Category: Documentation
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Roman Suzi (rnd0110)
Assigned to: Nobody/Anonymous (nobody)
Summary: Whats new: Obligatory source code encodings if not ASCII

Initial Comment:
As per PEP 263, source code encoding cookie is a must
for source code file. This fact  is not stressed enough in
the documentation.

I think, these sections must be changed to reflect PEP263:

What's new in Python 2.3

Section 3: PEP 263 Source Code Encoding
 must inform the reader on this:

Source files which use non-ASCII characters and do not
have encoding declaration are phased out. Python 2.3 
gives DeprecationWarnings. Later versions will give
SyntraxError
instead.


13.1 String changes

Unicode literals can now contain characters of the
source file encoding if such was specified (see PEP 263).



18 Porting to Python 2.3 section needs addition:

* to avoid DeprecationWarnings, add encoding
declaration to all source files according to PEP 263,
if they are using non-ASCII characters. After that,
Unicode strings could be specified with symbols from
specified encoding.

(Please correct my English, as it is not native)

(I'll look into other docs and submit corrections to
them separately)


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=683416&group_id=5470


From noreply@sourceforge.net  Sun Feb  9 15:31:18 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sun, 09 Feb 2003 07:31:18 -0800
Subject: [Python-bugs-list] [ python-Bugs-683416 ] Whats new: Obligatory source code encodings if not ASCII
Message-ID: <E18htQM-0000T1-00@sc8-sf-web4.sourceforge.net>

Bugs item #683416, was opened at 2003-02-09 09:52
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=683416&group_id=5470

Category: Documentation
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Roman Suzi (rnd0110)
>Assigned to: A.M. Kuchling (akuchling)
Summary: Whats new: Obligatory source code encodings if not ASCII

Initial Comment:
As per PEP 263, source code encoding cookie is a must
for source code file. This fact  is not stressed enough in
the documentation.

I think, these sections must be changed to reflect PEP263:

What's new in Python 2.3

Section 3: PEP 263 Source Code Encoding
 must inform the reader on this:

Source files which use non-ASCII characters and do not
have encoding declaration are phased out. Python 2.3 
gives DeprecationWarnings. Later versions will give
SyntraxError
instead.


13.1 String changes

Unicode literals can now contain characters of the
source file encoding if such was specified (see PEP 263).



18 Porting to Python 2.3 section needs addition:

* to avoid DeprecationWarnings, add encoding
declaration to all source files according to PEP 263,
if they are using non-ASCII characters. After that,
Unicode strings could be specified with symbols from
specified encoding.

(Please correct my English, as it is not native)

(I'll look into other docs and submit corrections to
them separately)


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=683416&group_id=5470


From noreply@sourceforge.net  Sun Feb  9 16:59:02 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sun, 09 Feb 2003 08:59:02 -0800
Subject: [Python-bugs-list] [ python-Bugs-683467 ] 'int' ability to generate longs not inherited
Message-ID: <E18hunG-00030R-00@sc8-sf-web3.sourceforge.net>

Bugs item #683467, was opened at 2003-02-09 16:59
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=683467&group_id=5470

Category: Type/class unification
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Gerrit Holl (gerrit)
Assigned to: Nobody/Anonymous (nobody)
Summary: 'int' ability to generate longs not inherited

Initial Comment:
[cvs update, ./configure, make]
$ ./python
Python 2.3a1 (#2, Feb  9 2003, 17:48:14)
[GCC 2.96 20000731 (Red Hat Linux 7.3 2.96-110)] on linux2
Type "help", "copyright", "credits" or "license" for
more information.
  0 >>> import sys
  1 >>> class A(int): pass
  1 ...
  2 >>> int(sys.maxint+1)
2147483648L
  3 >>> A(sys.maxint+1)
3
$ uname -a
Linux stopcontact 2.4.17 #1 Sat Jan 19 16:45:33 CET
2002 i686 unknown


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=683467&group_id=5470


From noreply@sourceforge.net  Sun Feb  9 17:13:30 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sun, 09 Feb 2003 09:13:30 -0800
Subject: [Python-bugs-list] [ python-Bugs-683467 ] 'int' ability to generate longs not inherited
Message-ID: <E18hv1G-0003Xj-00@sc8-sf-web3.sourceforge.net>

Bugs item #683467, was opened at 2003-02-09 11:59
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=683467&group_id=5470

Category: Type/class unification
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Gerrit Holl (gerrit)
Assigned to: Nobody/Anonymous (nobody)
Summary: 'int' ability to generate longs not inherited

Initial Comment:
[cvs update, ./configure, make]
$ ./python
Python 2.3a1 (#2, Feb  9 2003, 17:48:14)
[GCC 2.96 20000731 (Red Hat Linux 7.3 2.96-110)] on linux2
Type "help", "copyright", "credits" or "license" for
more information.
  0 >>> import sys
  1 >>> class A(int): pass
  1 ...
  2 >>> int(sys.maxint+1)
2147483648L
  3 >>> A(sys.maxint+1)
3
$ uname -a
Linux stopcontact 2.4.17 #1 Sat Jan 19 16:45:33 CET
2002 i686 unknown


----------------------------------------------------------------------

>Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-09 12:13

Message:
Logged In: YES 
user_id=33168

Wait, it gets better with a debug build:

python: Objects/intobject.c:844: int_subtype_new: Assertion
`((tmp)->ob_type == (&PyInt_Type) ||
PyType_IsSubtype((tmp)->ob_type, (&PyInt_Type)))' failed.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=683467&group_id=5470


From noreply@sourceforge.net  Sun Feb  9 17:49:11 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sun, 09 Feb 2003 09:49:11 -0800
Subject: [Python-bugs-list] [ python-Bugs-683467 ] 'int' ability to generate longs not inherited
Message-ID: <E18hvZn-0005Ph-00@sc8-sf-web2.sourceforge.net>

Bugs item #683467, was opened at 2003-02-09 11:59
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=683467&group_id=5470

Category: Type/class unification
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Gerrit Holl (gerrit)
>Assigned to: Guido van Rossum (gvanrossum)
Summary: 'int' ability to generate longs not inherited

Initial Comment:
[cvs update, ./configure, make]
$ ./python
Python 2.3a1 (#2, Feb  9 2003, 17:48:14)
[GCC 2.96 20000731 (Red Hat Linux 7.3 2.96-110)] on linux2
Type "help", "copyright", "credits" or "license" for
more information.
  0 >>> import sys
  1 >>> class A(int): pass
  1 ...
  2 >>> int(sys.maxint+1)
2147483648L
  3 >>> A(sys.maxint+1)
3
$ uname -a
Linux stopcontact 2.4.17 #1 Sat Jan 19 16:45:33 CET
2002 i686 unknown


----------------------------------------------------------------------

>Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-09 12:49

Message:
Logged In: YES 
user_id=33168

Attached is a patch which fixes this problem.  If the value
is a PyLong, try to get the int out of it (int values
proceed as before).  Otherwise, raise a ValueError, "value
must convertable to an int".  Not sure if that exception is
really reachable, since it depends on what int_new() returns
(usually return PyNumber_Int).

If this is acceptable, I'll update NEWS too.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-09 12:13

Message:
Logged In: YES 
user_id=33168

Wait, it gets better with a debug build:

python: Objects/intobject.c:844: int_subtype_new: Assertion
`((tmp)->ob_type == (&PyInt_Type) ||
PyType_IsSubtype((tmp)->ob_type, (&PyInt_Type)))' failed.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=683467&group_id=5470


From noreply@sourceforge.net  Sun Feb  9 17:50:28 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sun, 09 Feb 2003 09:50:28 -0800
Subject: [Python-bugs-list] [ python-Bugs-683486 ] Tutorial: info on Source Code Encoding is missing
Message-ID: <E18hvb2-0005Rb-00@sc8-sf-web2.sourceforge.net>

Bugs item #683486, was opened at 2003-02-09 17:50
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=683486&group_id=5470

Category: Documentation
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Roman Suzi (rnd0110)
Assigned to: Nobody/Anonymous (nobody)
Summary: Tutorial: info on Source Code Encoding is missing

Initial Comment:
(I propose to add information into Tutorial on how
to specify source code encoding. Please correct
my English and maybe there is better way to 
get a list of possible encodings)



2.2.2  Executable Python Scripts 

...

2.2.3 Source Code Encoding

   It is possible to use encodings different than ASCII
in Python
   source files. The best way to do it is to put one more
   special comment line right after #!-line making
proper encoding
   declaration:
  
# -*- coding: latin-1 -*-

   After that, all characters in the source file will
be treated
   as belonging to latin-1 encoding, and it will be
possible
   to directly write Unicode strings in the selected
encoding.

   List of possible encodings could be found in the
encodings
   subdirectory of Python directory.

   Significant advantage could be had if using utf-8
encoding,
   as it allows to use letters from many languages in
the same
   file.

2.2.4

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=683486&group_id=5470


From noreply@sourceforge.net  Sun Feb  9 17:52:57 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sun, 09 Feb 2003 09:52:57 -0800
Subject: [Python-bugs-list] [ python-Bugs-683467 ] 'int' ability to generate longs not inherited
Message-ID: <E18hvdR-0005XC-00@sc8-sf-web2.sourceforge.net>

Bugs item #683467, was opened at 2003-02-09 16:59
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=683467&group_id=5470

Category: Type/class unification
Group: Python 2.3
Status: Open
Resolution: None
>Priority: 6
Submitted By: Gerrit Holl (gerrit)
>Assigned to: Nobody/Anonymous (nobody)
Summary: 'int' ability to generate longs not inherited

Initial Comment:
[cvs update, ./configure, make]
$ ./python
Python 2.3a1 (#2, Feb  9 2003, 17:48:14)
[GCC 2.96 20000731 (Red Hat Linux 7.3 2.96-110)] on linux2
Type "help", "copyright", "credits" or "license" for
more information.
  0 >>> import sys
  1 >>> class A(int): pass
  1 ...
  2 >>> int(sys.maxint+1)
2147483648L
  3 >>> A(sys.maxint+1)
3
$ uname -a
Linux stopcontact 2.4.17 #1 Sat Jan 19 16:45:33 CET
2002 i686 unknown


----------------------------------------------------------------------

>Comment By: Gerrit Holl (gerrit)
Date: 2003-02-09 17:52

Message:
Logged In: YES 
user_id=13298

See also:
http://mail.python.org/pipermail/python-list/2003-February/145798.html

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-09 17:49

Message:
Logged In: YES 
user_id=33168

Attached is a patch which fixes this problem.  If the value
is a PyLong, try to get the int out of it (int values
proceed as before).  Otherwise, raise a ValueError, "value
must convertable to an int".  Not sure if that exception is
really reachable, since it depends on what int_new() returns
(usually return PyNumber_Int).

If this is acceptable, I'll update NEWS too.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-09 17:13

Message:
Logged In: YES 
user_id=33168

Wait, it gets better with a debug build:

python: Objects/intobject.c:844: int_subtype_new: Assertion
`((tmp)->ob_type == (&PyInt_Type) ||
PyType_IsSubtype((tmp)->ob_type, (&PyInt_Type)))' failed.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=683467&group_id=5470


From noreply@sourceforge.net  Sun Feb  9 17:58:13 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sun, 09 Feb 2003 09:58:13 -0800
Subject: [Python-bugs-list] [ python-Bugs-683467 ] 'int' ability to generate longs not inherited
Message-ID: <E18hviX-00056M-00@sc8-sf-web3.sourceforge.net>

Bugs item #683467, was opened at 2003-02-09 16:59
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=683467&group_id=5470

Category: Type/class unification
Group: Python 2.3
Status: Open
Resolution: None
Priority: 6
Submitted By: Gerrit Holl (gerrit)
Assigned to: Nobody/Anonymous (nobody)
Summary: 'int' ability to generate longs not inherited

Initial Comment:
[cvs update, ./configure, make]
$ ./python
Python 2.3a1 (#2, Feb  9 2003, 17:48:14)
[GCC 2.96 20000731 (Red Hat Linux 7.3 2.96-110)] on linux2
Type "help", "copyright", "credits" or "license" for
more information.
  0 >>> import sys
  1 >>> class A(int): pass
  1 ...
  2 >>> int(sys.maxint+1)
2147483648L
  3 >>> A(sys.maxint+1)
3
$ uname -a
Linux stopcontact 2.4.17 #1 Sat Jan 19 16:45:33 CET
2002 i686 unknown


----------------------------------------------------------------------

>Comment By: Gerrit Holl (gerrit)
Date: 2003-02-09 17:58

Message:
Logged In: YES 
user_id=13298

See also:
http://mail.python.org/pipermail/python-list/2003-February/145798.html

----------------------------------------------------------------------

Comment By: Gerrit Holl (gerrit)
Date: 2003-02-09 17:52

Message:
Logged In: YES 
user_id=13298

See also:
http://mail.python.org/pipermail/python-list/2003-February/145798.html

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-09 17:49

Message:
Logged In: YES 
user_id=33168

Attached is a patch which fixes this problem.  If the value
is a PyLong, try to get the int out of it (int values
proceed as before).  Otherwise, raise a ValueError, "value
must convertable to an int".  Not sure if that exception is
really reachable, since it depends on what int_new() returns
(usually return PyNumber_Int).

If this is acceptable, I'll update NEWS too.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-09 17:13

Message:
Logged In: YES 
user_id=33168

Wait, it gets better with a debug build:

python: Objects/intobject.c:844: int_subtype_new: Assertion
`((tmp)->ob_type == (&PyInt_Type) ||
PyType_IsSubtype((tmp)->ob_type, (&PyInt_Type)))' failed.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=683467&group_id=5470


From noreply@sourceforge.net  Sun Feb  9 20:46:18 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sun, 09 Feb 2003 12:46:18 -0800
Subject: [Python-bugs-list] [ python-Bugs-679880 ] 'compile' refuses BOM.
Message-ID: <E18hyLC-0003me-00@sc8-sf-web4.sourceforge.net>

Bugs item #679880, was opened at 2003-02-04 00:05
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=679880&group_id=5470

Category: Unicode
Group: Python 2.3
>Status: Closed
>Resolution: Fixed
Priority: 5
Submitted By: Kirill Simonov (kirill_simonov)
Assigned to: Martin v. Löwis (loewis)
Summary: 'compile' refuses BOM.

Initial Comment:
The builtin 'compile' function refuses source files
with UTF-8 BOM marks.

For example,

$ python2.3 testutf8.py

works correctly, but

$ python2.3
>>> compile(open('testutf8.py').read(), 'testutf8.py',
'exec')

fails.



----------------------------------------------------------------------

>Comment By: Just van Rossum (jvr)
Date: 2003-02-09 21:46

Message:
Logged In: YES 
user_id=92689

Fixed in rev. 2.72 of tokenize.c. Patch #680474 has been applied.

----------------------------------------------------------------------

Comment By: Kirill Simonov (kirill_simonov)
Date: 2003-02-04 22:24

Message:
Logged In: YES 
user_id=36553

I have uploaded a patch to fix this bug: #680474.
Hope somebody find time to apply it.

----------------------------------------------------------------------

Comment By: M.-A. Lemburg (lemburg)
Date: 2003-02-04 10:18

Message:
Logged In: YES 
user_id=38388

No time for this. Assigning to Martin. Feel free to pass it on.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=679880&group_id=5470


From noreply@sourceforge.net  Sun Feb  9 23:17:50 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sun, 09 Feb 2003 15:17:50 -0800
Subject: [Python-bugs-list] [ python-Bugs-680577 ] urllib2 authentication problem
Message-ID: <E18i0hq-0004k9-00@sc8-sf-web1.sourceforge.net>

Bugs item #680577, was opened at 2003-02-05 00:22
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=680577&group_id=5470

Category: Python Library
>Group: Python 2.2.2
Status: Open
Resolution: None
Priority: 5
Submitted By: Gary Donovan (gazzadee)
Assigned to: Nobody/Anonymous (nobody)
Summary: urllib2 authentication problem

Initial Comment:
I've found a problem using the authentication in urllib2.

When matching up host-names in order to find a
password, then putting the protocol in the address
makes it seem like a different address. eg...

I create a HTTPBasicAuthHandler with a
HTTPPasswordMgrWithDefaultRealm, and add the tuple
(None, "http://proxy.blah.com:17828", "foo", "bar") to it.

I then setup the proxy to use
http://proxy.blah.com:17828 (which requires
authentication).

When I connect, the password lookup fails, because it
is trying to find a match for "proxy.blah.com:17828"
rather than "http://proxy.blah.com:17828"

This problem doesn't exist if I pass
"proxy.blah.com:17828" to the password manager.

There seems to be some stuff in HTTPPasswordMgr to deal
with variations on site names, but I guess it's not
working in this case (unless this is intentional).

Version Info:
Python 2.2 (#1, Feb 24 2002, 16:21:58)
[GCC 2.96 20000731 (Mandrake Linux 8.2 2.96-0.76mdk)]
on linux-i386


----------------------------------------------------------------------

>Comment By: Gary Donovan (gazzadee)
Date: 2003-02-09 23:17

Message:
Logged In: YES 
user_id=693152

Okay, the same problem crops up in Python 2.2.2 running
under cygwin on Win XP

Version Info:
Python 2.2.2 (#1, Dec 31 2002, 12:24:34) 
[GCC 3.2 20020927 (prerelease)] on cygwin

Here's the pertinent section of my test file (passwords and
URL changed to protect the innocent):


    # Setup proxy
    proxy_handler = ProxyHandler({"http" :
"http://blah.com:17828"})
    
    # Setup authentication
    pass_mgr = HTTPPasswordMgrWithDefaultRealm()
    for passwd in [ \
                   (None, "http://blah.com:17828", "foo",
"bar"), \
#                   (None, "blah.com:17828", "foo",
"bar"), \	# Works if this line is uncommented
                   (None, "blah.com", "foo", "bar"), \
                  ]:
        print("Adding password set (%s, %s, %s, %s)" % passwd)
        pass_mgr.add_password(*passwd)
    auth_handler = HTTPBasicAuthHandler(pass_mgr)
    proxy_auth_handler = ProxyBasicAuthHandler(pass_mgr)
    
    # Now build a new URL opener and install it
    opener = build_opener(proxy_handler, proxy_auth_handler,
auth_handler, HTTPHandler)
    install_opener(opener)
    
    # Now try to open a file and see what happens
    request = Request("http://www.google.com")
    try:
        remotefile = urlopen(request)
    except HTTPError, ex:
        print("Unable to download file due to HTTP Error %d
(%s)." % (ex.code, ex.msg))
        return



----------------------------------------------------------------------

Comment By: Gerhard Häring (ghaering)
Date: 2003-02-07 23:21

Message:
Logged In: YES 
user_id=163326

Can you please retry with Python 2.2.2?

It seems that a related bug was fixed for 2.2.2:
http://python.org/2.2.2/NEWS.txt has an entry:

"""
- In urllib2.py: fix proxy config with user+pass
authentication.  [SF
  patch 527518]
"""

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=680577&group_id=5470


From noreply@sourceforge.net  Mon Feb 10 00:26:08 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sun, 09 Feb 2003 16:26:08 -0800
Subject: [Python-bugs-list] [ python-Bugs-683658 ] PyErr_Warn may cause import deadlock
Message-ID: <E18i1lw-00037x-00@sc8-sf-web2.sourceforge.net>

Bugs item #683658, was opened at 2003-02-10 11:26
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=683658&group_id=5470

Category: Python Interpreter Core
Group: Python 2.3
Status: Open
Resolution: None
Priority: 6
Submitted By: Mark Hammond (mhammond)
Assigned to: Nobody/Anonymous (nobody)
Summary: PyErr_Warn may cause import deadlock

Initial Comment:
PyErr_Warn() does an implicit import.  Thus, if
PyErr_Warn() is called on a thread while the main
thread holds the import lock, and the main thread then
subsequently waits for the child thread, Python will
deadlock.  The builtin 'apply' now calls PyErr_Warn(),
so simply calling 'apply' on another thread may cause
the deadlock.

Attaching a sample case.  Executing 'python -c "import
hang_apply"' will cause Python to deadlock.  Commenting
out the call to "apply" in that file will prevent the
deadlock (even though the apply() in question is
effectively a no-op)

The example is a little contrived, but is extracted
from real code of mine that saw this hang.  The code
path is:

* Main thread acquires import lock to import the module.
* Module import spawns a new thread.  This new thread
calls apply()
* builtin_apply() calls PyErr_Warn
* PyErr_Warn does an import of the warnings module, and
attempts to acquire the import lock.
* Main thread still waiting for child thread to
complete, but still holds import lock.

Note that removing the call to PyErr_Warn() in
builtin_apply also solves this particular hang -
however, it seems like this is a potential time bomb. 
A potential solution would be to prevent PyErr_Warn
from doing an import - this would mean importing
'warnings' at startup, and keeping a static reference
in errors.c.  Other thoughts welcome.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=683658&group_id=5470


From noreply@sourceforge.net  Mon Feb 10 01:17:37 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sun, 09 Feb 2003 17:17:37 -0800
Subject: [Python-bugs-list] [ python-Bugs-678518 ] Another parsermodule validation error
Message-ID: <E18i2Zl-00006x-00@sc8-sf-web1.sourceforge.net>

Bugs item #678518, was opened at 2003-01-31 21:23
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=678518&group_id=5470

Category: Parser/Compiler
Group: Python 2.3
>Status: Closed
>Resolution: Fixed
Priority: 5
Submitted By: logistix (logistix)
Assigned to: Michael Hudson (mwh)
Summary: Another parsermodule validation error

Initial Comment:
The following code generates an validation error.  It's an 
invalid ast tree where the global_stmt doesn't have the 
keyword 'global' as a child.

I tracked down the fix to validate_global_stmt() this time.

Python 2.3a1 (#38, Dec 31 2002, 17:53:59) [MSC 
v.1200 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more 
information.
>>> import symbol,token,parser
>>> ast = [symbol.file_input,
...  [symbol.stmt, [symbol.simple_stmt, 
[symbol.small_stmt, [symbol.global_stmt,
 [1, 'foo']]], [token.NEWLINE, '']]],
...  [token.NEWLINE, ''],
...  [token.ENDMARKER, '']]
>>> a = parser.sequence2ast(ast)
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
parser.ParserError: VALIDATION FAILURE: report this 
to the maintainer!
>>>

----------------------------------------------------------------------

>Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-09 20:17

Message:
Logged In: YES 
user_id=33168

I added Grant to ACKS 1.227 and removed a duplicate test I'm
guessing was introduced by patching parsermodule.c twice. 
Closing.

----------------------------------------------------------------------

Comment By: logistix (logistix)
Date: 2003-02-08 16:15

Message:
Logged In: YES 
user_id=699438

I attached some changes for the testsuite.

My name is Grant Olson.

Looks like we can close this ticket out ;-)

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2003-02-08 13:12

Message:
Logged In: YES 
user_id=6656

OK, I've checked in revision 2.76 of Modules/parsermodule.c.

Do you have a real name?  For Misc/ACKS.

Also, would you be interested in putting an example of each
failing syntax into test_parser?

Thanks!

----------------------------------------------------------------------

Comment By: logistix (logistix)
Date: 2003-02-07 17:51

Message:
Logged In: YES 
user_id=699438

You're right, the assert statement was the unhandled 
exception.  It was raising an error but returning true on 
the "__assert__" check.

Just posted the whole patch

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2003-02-07 09:38

Message:
Logged In: YES 
user_id=6656

> At the time I didn't have an encoding patch and punted.

Oh, I misread you then.

> I did figure it out last night though.  I'll post it later
today 
> since I'm at the office now.

Ta.

> And yes, this is the error that'll print out "XXX 
> unhandled exception" out of ceval.c when you run
> the test from debug mode.

Are you sure?  It seems that was the assert one.

> There was some code added to add the encoding 
> type in the ast2sequence functions,

By me :-)  I didn't really think about the sequence2ast 
functions as all I was doing was trying to get the compiler
package to work...

> Also, as a matter of style, should I add the stuff in 
> my test script to the test suite or is that overkill? Should
> I just have it check 4 or 5 random files?

It might be best to get it to check a few files we knew there
were problems with -- test_pep263, maybe.  Checking the
whole lot is probably overkill.

----------------------------------------------------------------------

Comment By: logistix (logistix)
Date: 2003-02-07 09:23

Message:
Logged In: YES 
user_id=699438

Michael,

At the time I didn't have an encoding patch and punted.  I did 
figure it out last night though.  I'll post it later today since I'm 
at the office now.

And yes, this is the error that'll print out "XXX unhandled 
exception" out of ceval.c when you run the test from debug 
mode.

It comes down to the bastardized way that encoding is 
implemented.  It doesn't fit the current definition of a token or 
a symbol.  There was some code added to add the encoding 
type in the ast2sequence functions, but not the 
sequence2ast functions, so when it hit scripts with 
encoding_decl's, it bombed big time.

Also, as a matter of style, should I add the stuff in my test 
script to the test suite or is that overkill? Should I just have it 
check4 or 5 random files?

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2003-02-07 07:08

Message:
Logged In: YES 
user_id=6656

Pff, I was right, but logistix's patch fix it.

logistix, did you forget to attach the encoding patch?  I
still get errors in module's with a cookie.

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2003-02-07 06:58

Message:
Logged In: YES 
user_id=6656

Patches as such are probably fine.  The reason I haven't
checked them in is a wierdness that I may or may not be
related, but is bothering me.  Consider:

->> import parser, sys,glob,os, traceback
/>> for path in sys.path:
|..     for fil in glob.glob(os.path.join(path,"*.py")):
|..       try:
|..         try:
|..             a = parser.suite(file(fil).read())
|..             b = parser.ast2tuple(a)
|..             c = parser.sequence2ast(b)
|..         except parser.ParserError:
|..             print "Parse of %s" % fil, "FAILED"
|..             traceback.print_exc()
|..             print '----------------------------'
|..       except:
|..          print "waaa?"
|.. else:
|..     print "what?"
\__ 
Parse of /home/mwh/src/sf/python/dist/src/Lib/tarfile.py FAILED
Traceback (most recent call last):
  File "<input>", line 7, in ?
ParserError
----------------------------
Parse of /home/mwh/src/sf/python/dist/src/Lib/heapq.py FAILED
Traceback (most recent call last):
  File "<input>", line 7, in ?
ParserError
----------------------------
Parse of /home/mwh/src/sf/python/dist/src/Lib/getopt.py FAILED
Traceback (most recent call last):
  File "<input>", line 7, in ?
ParserError
----------------------------
Parse of /home/mwh/src/sf/python/dist/src/Lib/inspect.py FAILED
Traceback (most recent call last):
  File "<input>", line 7, in ?
ParserError
----------------------------
Parse of /home/mwh/src/sf/python/dist/src/Lib/pydoc.py FAILED
Traceback (most recent call last):
  File "<input>", line 7, in ?
ParserError
----------------------------
Parse of /home/mwh/src/sf/python/dist/src/Lib/lib-tk/Tix.py
FAILED
Traceback (most recent call last):
  File "<input>", line 7, in ?
ParserError
----------------------------
Traceback (most recent call last):
  File "<input>", line 2, in ?
ParserError: Illegal terminal: expected "__assert__"
->> 

Notice that an exception seems to have made it through the
except clauses!  The scary bit is that a (seemingly) random
number of parser errors *will* caught, and then one will get
through.

The only explanation I have for this is some kind of
returning non-NULL with exception set kind of bug.

Off to play with a debug build, I guess.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-06 20:17

Message:
Logged In: YES 
user_id=33168

Michael were these patches ok to check in?

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2003-02-03 06:00

Message:
Logged In: YES 
user_id=6656

Thanks for doing this.  I'll probably check these in after
I've finished crunching my way through monday morning email...

----------------------------------------------------------------------

Comment By: logistix (logistix)
Date: 2003-02-01 16:42

Message:
Logged In: YES 
user_id=699438

And I found a few more by doing a roundtrip 
ast2Tuple/tuple2ast on all files in pythonpath.  These actually 
affect valid AST's, so they're probably a bigger deal.

Valid assert statements were failing.  The patch was trivial, 
so its attached.

Also, files that took advantage of encoding_decl (from PEP 
263) were failing.  I couldn't tell if PEP 263 was offically 
closed, and didn't know what the offical grammar was, so I 
didn't do anything there.

I'll also attach the test script that turned up these errors.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=678518&group_id=5470


From noreply@sourceforge.net  Mon Feb 10 01:31:16 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sun, 09 Feb 2003 17:31:16 -0800
Subject: [Python-bugs-list] [ python-Bugs-683467 ] 'int' ability to generate longs not inherited
Message-ID: <E18i2my-0005wC-00@sc8-sf-web2.sourceforge.net>

Bugs item #683467, was opened at 2003-02-09 11:59
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=683467&group_id=5470

Category: Type/class unification
Group: Python 2.3
Status: Open
>Resolution: Accepted
>Priority: 7
Submitted By: Gerrit Holl (gerrit)
>Assigned to: Neal Norwitz (nnorwitz)
Summary: 'int' ability to generate longs not inherited

Initial Comment:
[cvs update, ./configure, make]
$ ./python
Python 2.3a1 (#2, Feb  9 2003, 17:48:14)
[GCC 2.96 20000731 (Red Hat Linux 7.3 2.96-110)] on linux2
Type "help", "copyright", "credits" or "license" for
more information.
  0 >>> import sys
  1 >>> class A(int): pass
  1 ...
  2 >>> int(sys.maxint+1)
2147483648L
  3 >>> A(sys.maxint+1)
3
$ uname -a
Linux stopcontact 2.4.17 #1 Sat Jan 19 16:45:33 CET
2002 i686 unknown


----------------------------------------------------------------------

>Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-09 20:31

Message:
Logged In: YES 
user_id=6380

Looks good.

----------------------------------------------------------------------

Comment By: Gerrit Holl (gerrit)
Date: 2003-02-09 12:58

Message:
Logged In: YES 
user_id=13298

See also:
http://mail.python.org/pipermail/python-list/2003-February/145798.html

----------------------------------------------------------------------

Comment By: Gerrit Holl (gerrit)
Date: 2003-02-09 12:52

Message:
Logged In: YES 
user_id=13298

See also:
http://mail.python.org/pipermail/python-list/2003-February/145798.html

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-09 12:49

Message:
Logged In: YES 
user_id=33168

Attached is a patch which fixes this problem.  If the value
is a PyLong, try to get the int out of it (int values
proceed as before).  Otherwise, raise a ValueError, "value
must convertable to an int".  Not sure if that exception is
really reachable, since it depends on what int_new() returns
(usually return PyNumber_Int).

If this is acceptable, I'll update NEWS too.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-09 12:13

Message:
Logged In: YES 
user_id=33168

Wait, it gets better with a debug build:

python: Objects/intobject.c:844: int_subtype_new: Assertion
`((tmp)->ob_type == (&PyInt_Type) ||
PyType_IsSubtype((tmp)->ob_type, (&PyInt_Type)))' failed.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=683467&group_id=5470


From noreply@sourceforge.net  Mon Feb 10 02:05:16 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sun, 09 Feb 2003 18:05:16 -0800
Subject: [Python-bugs-list] [ python-Bugs-678518 ] Another parsermodule validation error
Message-ID: <E18i3Js-0001ii-00@sc8-sf-web1.sourceforge.net>

Bugs item #678518, was opened at 2003-01-31 21:23
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=678518&group_id=5470

Category: Parser/Compiler
Group: Python 2.3
Status: Closed
Resolution: Fixed
Priority: 5
Submitted By: logistix (logistix)
Assigned to: Michael Hudson (mwh)
Summary: Another parsermodule validation error

Initial Comment:
The following code generates an validation error.  It's an 
invalid ast tree where the global_stmt doesn't have the 
keyword 'global' as a child.

I tracked down the fix to validate_global_stmt() this time.

Python 2.3a1 (#38, Dec 31 2002, 17:53:59) [MSC 
v.1200 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more 
information.
>>> import symbol,token,parser
>>> ast = [symbol.file_input,
...  [symbol.stmt, [symbol.simple_stmt, 
[symbol.small_stmt, [symbol.global_stmt,
 [1, 'foo']]], [token.NEWLINE, '']]],
...  [token.NEWLINE, ''],
...  [token.ENDMARKER, '']]
>>> a = parser.sequence2ast(ast)
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
parser.ParserError: VALIDATION FAILURE: report this 
to the maintainer!
>>>

----------------------------------------------------------------------

>Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-09 21:05

Message:
Logged In: YES 
user_id=33168

Added test_parser.py changes as 1.15
Also added NEWS entry 1.647 and backported the assert and
global changes as:
 test_parser.py 1.10.12.2
 parsermodule.c 2.68.6.2

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-09 20:17

Message:
Logged In: YES 
user_id=33168

I added Grant to ACKS 1.227 and removed a duplicate test I'm
guessing was introduced by patching parsermodule.c twice. 
Closing.

----------------------------------------------------------------------

Comment By: logistix (logistix)
Date: 2003-02-08 16:15

Message:
Logged In: YES 
user_id=699438

I attached some changes for the testsuite.

My name is Grant Olson.

Looks like we can close this ticket out ;-)

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2003-02-08 13:12

Message:
Logged In: YES 
user_id=6656

OK, I've checked in revision 2.76 of Modules/parsermodule.c.

Do you have a real name?  For Misc/ACKS.

Also, would you be interested in putting an example of each
failing syntax into test_parser?

Thanks!

----------------------------------------------------------------------

Comment By: logistix (logistix)
Date: 2003-02-07 17:51

Message:
Logged In: YES 
user_id=699438

You're right, the assert statement was the unhandled 
exception.  It was raising an error but returning true on 
the "__assert__" check.

Just posted the whole patch

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2003-02-07 09:38

Message:
Logged In: YES 
user_id=6656

> At the time I didn't have an encoding patch and punted.

Oh, I misread you then.

> I did figure it out last night though.  I'll post it later
today 
> since I'm at the office now.

Ta.

> And yes, this is the error that'll print out "XXX 
> unhandled exception" out of ceval.c when you run
> the test from debug mode.

Are you sure?  It seems that was the assert one.

> There was some code added to add the encoding 
> type in the ast2sequence functions,

By me :-)  I didn't really think about the sequence2ast 
functions as all I was doing was trying to get the compiler
package to work...

> Also, as a matter of style, should I add the stuff in 
> my test script to the test suite or is that overkill? Should
> I just have it check 4 or 5 random files?

It might be best to get it to check a few files we knew there
were problems with -- test_pep263, maybe.  Checking the
whole lot is probably overkill.

----------------------------------------------------------------------

Comment By: logistix (logistix)
Date: 2003-02-07 09:23

Message:
Logged In: YES 
user_id=699438

Michael,

At the time I didn't have an encoding patch and punted.  I did 
figure it out last night though.  I'll post it later today since I'm 
at the office now.

And yes, this is the error that'll print out "XXX unhandled 
exception" out of ceval.c when you run the test from debug 
mode.

It comes down to the bastardized way that encoding is 
implemented.  It doesn't fit the current definition of a token or 
a symbol.  There was some code added to add the encoding 
type in the ast2sequence functions, but not the 
sequence2ast functions, so when it hit scripts with 
encoding_decl's, it bombed big time.

Also, as a matter of style, should I add the stuff in my test 
script to the test suite or is that overkill? Should I just have it 
check4 or 5 random files?

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2003-02-07 07:08

Message:
Logged In: YES 
user_id=6656

Pff, I was right, but logistix's patch fix it.

logistix, did you forget to attach the encoding patch?  I
still get errors in module's with a cookie.

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2003-02-07 06:58

Message:
Logged In: YES 
user_id=6656

Patches as such are probably fine.  The reason I haven't
checked them in is a wierdness that I may or may not be
related, but is bothering me.  Consider:

->> import parser, sys,glob,os, traceback
/>> for path in sys.path:
|..     for fil in glob.glob(os.path.join(path,"*.py")):
|..       try:
|..         try:
|..             a = parser.suite(file(fil).read())
|..             b = parser.ast2tuple(a)
|..             c = parser.sequence2ast(b)
|..         except parser.ParserError:
|..             print "Parse of %s" % fil, "FAILED"
|..             traceback.print_exc()
|..             print '----------------------------'
|..       except:
|..          print "waaa?"
|.. else:
|..     print "what?"
\__ 
Parse of /home/mwh/src/sf/python/dist/src/Lib/tarfile.py FAILED
Traceback (most recent call last):
  File "<input>", line 7, in ?
ParserError
----------------------------
Parse of /home/mwh/src/sf/python/dist/src/Lib/heapq.py FAILED
Traceback (most recent call last):
  File "<input>", line 7, in ?
ParserError
----------------------------
Parse of /home/mwh/src/sf/python/dist/src/Lib/getopt.py FAILED
Traceback (most recent call last):
  File "<input>", line 7, in ?
ParserError
----------------------------
Parse of /home/mwh/src/sf/python/dist/src/Lib/inspect.py FAILED
Traceback (most recent call last):
  File "<input>", line 7, in ?
ParserError
----------------------------
Parse of /home/mwh/src/sf/python/dist/src/Lib/pydoc.py FAILED
Traceback (most recent call last):
  File "<input>", line 7, in ?
ParserError
----------------------------
Parse of /home/mwh/src/sf/python/dist/src/Lib/lib-tk/Tix.py
FAILED
Traceback (most recent call last):
  File "<input>", line 7, in ?
ParserError
----------------------------
Traceback (most recent call last):
  File "<input>", line 2, in ?
ParserError: Illegal terminal: expected "__assert__"
->> 

Notice that an exception seems to have made it through the
except clauses!  The scary bit is that a (seemingly) random
number of parser errors *will* caught, and then one will get
through.

The only explanation I have for this is some kind of
returning non-NULL with exception set kind of bug.

Off to play with a debug build, I guess.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-06 20:17

Message:
Logged In: YES 
user_id=33168

Michael were these patches ok to check in?

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2003-02-03 06:00

Message:
Logged In: YES 
user_id=6656

Thanks for doing this.  I'll probably check these in after
I've finished crunching my way through monday morning email...

----------------------------------------------------------------------

Comment By: logistix (logistix)
Date: 2003-02-01 16:42

Message:
Logged In: YES 
user_id=699438

And I found a few more by doing a roundtrip 
ast2Tuple/tuple2ast on all files in pythonpath.  These actually 
affect valid AST's, so they're probably a bigger deal.

Valid assert statements were failing.  The patch was trivial, 
so its attached.

Also, files that took advantage of encoding_decl (from PEP 
263) were failing.  I couldn't tell if PEP 263 was offically 
closed, and didn't know what the offical grammar was, so I 
didn't do anything there.

I'll also attach the test script that turned up these errors.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=678518&group_id=5470


From noreply@sourceforge.net  Mon Feb 10 02:22:19 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sun, 09 Feb 2003 18:22:19 -0800
Subject: [Python-bugs-list] [ python-Bugs-683467 ] 'int' ability to generate longs not inherited
Message-ID: <E18i3aN-0006eN-00@sc8-sf-web4.sourceforge.net>

Bugs item #683467, was opened at 2003-02-09 11:59
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=683467&group_id=5470

Category: Type/class unification
Group: Python 2.3
>Status: Closed
Resolution: Accepted
Priority: 7
Submitted By: Gerrit Holl (gerrit)
Assigned to: Neal Norwitz (nnorwitz)
Summary: 'int' ability to generate longs not inherited

Initial Comment:
[cvs update, ./configure, make]
$ ./python
Python 2.3a1 (#2, Feb  9 2003, 17:48:14)
[GCC 2.96 20000731 (Red Hat Linux 7.3 2.96-110)] on linux2
Type "help", "copyright", "credits" or "license" for
more information.
  0 >>> import sys
  1 >>> class A(int): pass
  1 ...
  2 >>> int(sys.maxint+1)
2147483648L
  3 >>> A(sys.maxint+1)
3
$ uname -a
Linux stopcontact 2.4.17 #1 Sat Jan 19 16:45:33 CET
2002 i686 unknown


----------------------------------------------------------------------

>Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-09 21:22

Message:
Logged In: YES 
user_id=33168

Thanks!

Checked in as:
 Objects/intobject.c 2.101
 Lib/test/test_descr.py 1.179
 Misc/NEWS 1.648

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-09 20:31

Message:
Logged In: YES 
user_id=6380

Looks good.

----------------------------------------------------------------------

Comment By: Gerrit Holl (gerrit)
Date: 2003-02-09 12:58

Message:
Logged In: YES 
user_id=13298

See also:
http://mail.python.org/pipermail/python-list/2003-February/145798.html

----------------------------------------------------------------------

Comment By: Gerrit Holl (gerrit)
Date: 2003-02-09 12:52

Message:
Logged In: YES 
user_id=13298

See also:
http://mail.python.org/pipermail/python-list/2003-February/145798.html

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-09 12:49

Message:
Logged In: YES 
user_id=33168

Attached is a patch which fixes this problem.  If the value
is a PyLong, try to get the int out of it (int values
proceed as before).  Otherwise, raise a ValueError, "value
must convertable to an int".  Not sure if that exception is
really reachable, since it depends on what int_new() returns
(usually return PyNumber_Int).

If this is acceptable, I'll update NEWS too.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-09 12:13

Message:
Logged In: YES 
user_id=33168

Wait, it gets better with a debug build:

python: Objects/intobject.c:844: int_subtype_new: Assertion
`((tmp)->ob_type == (&PyInt_Type) ||
PyType_IsSubtype((tmp)->ob_type, (&PyInt_Type)))' failed.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=683467&group_id=5470


From noreply@sourceforge.net  Mon Feb 10 02:59:24 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sun, 09 Feb 2003 18:59:24 -0800
Subject: [Python-bugs-list] [ python-Bugs-682648 ] urllib2 HTTPDigestAuthHandler misnamed class attr
Message-ID: <E18i4AG-0007jd-00@sc8-sf-web4.sourceforge.net>

Bugs item #682648, was opened at 2003-02-07 19:05
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=682648&group_id=5470

Category: Python Library
Group: Python 2.2.2
Status: Open
Resolution: None
Priority: 5
Submitted By: steve steiner (ssteiner)
Assigned to: Nobody/Anonymous (nobody)
Summary: urllib2 HTTPDigestAuthHandler misnamed class attr

Initial Comment:
The class attribute 'header' should be named 'auth_header' as it is in HTTPBasicAuthHandler.  Same is true of ProxyDigestAuthHandler.  

These classes throw an AttributeError (shown below for HTTPDigestAuthHandler:

AttributeError: HTTPDigestAuthHandler instance has no attribute 'auth_header'

----------------------------------------------------------------------

>Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-09 21:59

Message:
Logged In: YES 
user_id=33168

Sure looks right to me (ie, changing header to auth_header).
 Can you supply tests?  Thanks.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=682648&group_id=5470


From noreply@sourceforge.net  Mon Feb 10 03:21:21 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sun, 09 Feb 2003 19:21:21 -0800
Subject: [Python-bugs-list] [ python-Bugs-683726 ] METH_CLASS allows binding to the wrong class
Message-ID: <E18i4VV-0003oo-00@sc8-sf-web1.sourceforge.net>

Bugs item #683726, was opened at 2003-02-09 18:21
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=683726&group_id=5470

Category: Python Interpreter Core
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Greg Chapman (glchapman)
Assigned to: Nobody/Anonymous (nobody)
Summary: METH_CLASS allows binding to the wrong class

Initial Comment:
I just noticed that this doesn't raise an exception:

>>> cmeth = dict.__dict__['fromkeys'].__get__(None, 
list)
>>> cmeth
<built-in method fromkeys of type object at 
0x1E0BB2E0>
>>> cmeth.__self__
<type 'list'>

Looking over the various METH_CLASS 
implementations, it doesn't look like they depend on the 
first (cls) parameter being of any particular type, but it 
might be nice to flag something like the above with an 
exception.

Also, there's this:

>>> cmeth2 = dict.__dict__['fromkeys'].__get__(None, 
42)
>>> cmeth2
<built-in method fromkeys of int object at 0x0074FD90>
>>> print cmeth2.__self__
42

Above, 42 gets passed through to classmethod_get in 
the type parameter, despite the fact that this parameter 
is typed as a PyTypeObject *. Perhpas wrap_descr_get 
should check that its second parameter (if it gets one) is 
a type?
 

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=683726&group_id=5470


From noreply@sourceforge.net  Mon Feb 10 03:38:36 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sun, 09 Feb 2003 19:38:36 -0800
Subject: [Python-bugs-list] [ python-Bugs-683726 ] METH_CLASS allows binding to the wrong class
Message-ID: <E18i4mC-0004Rj-00@sc8-sf-web1.sourceforge.net>

Bugs item #683726, was opened at 2003-02-09 18:21
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=683726&group_id=5470

Category: Python Interpreter Core
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Greg Chapman (glchapman)
Assigned to: Nobody/Anonymous (nobody)
Summary: METH_CLASS allows binding to the wrong class

Initial Comment:
I just noticed that this doesn't raise an exception:

>>> cmeth = dict.__dict__['fromkeys'].__get__(None, 
list)
>>> cmeth
<built-in method fromkeys of type object at 
0x1E0BB2E0>
>>> cmeth.__self__
<type 'list'>

Looking over the various METH_CLASS 
implementations, it doesn't look like they depend on the 
first (cls) parameter being of any particular type, but it 
might be nice to flag something like the above with an 
exception.

Also, there's this:

>>> cmeth2 = dict.__dict__['fromkeys'].__get__(None, 
42)
>>> cmeth2
<built-in method fromkeys of int object at 0x0074FD90>
>>> print cmeth2.__self__
42

Above, 42 gets passed through to classmethod_get in 
the type parameter, despite the fact that this parameter 
is typed as a PyTypeObject *. Perhpas wrap_descr_get 
should check that its second parameter (if it gets one) is 
a type?
 

----------------------------------------------------------------------

>Comment By: Greg Chapman (glchapman)
Date: 2003-02-09 18:38

Message:
Logged In: YES 
user_id=86307

I just realized that the above can be used to crash python, so 
I believe some sort of checking has to be added to 
classmethod_get:

>>> cmeth = dict.__dict__['fromkeys'].__get__(None)
>>> cmeth(4)

(above causes and access violation because the cls object 
passed to the fromkeys method is NULL).


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=683726&group_id=5470


From noreply@sourceforge.net  Mon Feb 10 03:42:11 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sun, 09 Feb 2003 19:42:11 -0800
Subject: [Python-bugs-list] [ python-Bugs-683726 ] METH_CLASS allows binding to the wrong class
Message-ID: <E18i4pf-0000NX-00@sc8-sf-web4.sourceforge.net>

Bugs item #683726, was opened at 2003-02-09 22:21
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=683726&group_id=5470

Category: Python Interpreter Core
Group: Python 2.3
Status: Open
Resolution: None
>Priority: 7
Submitted By: Greg Chapman (glchapman)
>Assigned to: Guido van Rossum (gvanrossum)
Summary: METH_CLASS allows binding to the wrong class

Initial Comment:
I just noticed that this doesn't raise an exception:

>>> cmeth = dict.__dict__['fromkeys'].__get__(None, 
list)
>>> cmeth
<built-in method fromkeys of type object at 
0x1E0BB2E0>
>>> cmeth.__self__
<type 'list'>

Looking over the various METH_CLASS 
implementations, it doesn't look like they depend on the 
first (cls) parameter being of any particular type, but it 
might be nice to flag something like the above with an 
exception.

Also, there's this:

>>> cmeth2 = dict.__dict__['fromkeys'].__get__(None, 
42)
>>> cmeth2
<built-in method fromkeys of int object at 0x0074FD90>
>>> print cmeth2.__self__
42

Above, 42 gets passed through to classmethod_get in 
the type parameter, despite the fact that this parameter 
is typed as a PyTypeObject *. Perhpas wrap_descr_get 
should check that its second parameter (if it gets one) is 
a type?
 

----------------------------------------------------------------------

Comment By: Greg Chapman (glchapman)
Date: 2003-02-09 22:38

Message:
Logged In: YES 
user_id=86307

I just realized that the above can be used to crash python, so 
I believe some sort of checking has to be added to 
classmethod_get:

>>> cmeth = dict.__dict__['fromkeys'].__get__(None)
>>> cmeth(4)

(above causes and access violation because the cls object 
passed to the fromkeys method is NULL).


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=683726&group_id=5470


From noreply@sourceforge.net  Mon Feb 10 09:43:53 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Mon, 10 Feb 2003 01:43:53 -0800
Subject: [Python-bugs-list] [ python-Bugs-681960 ] Source encoding rules are extreme.
Message-ID: <E18iATh-0005EP-00@sc8-sf-web2.sourceforge.net>

Bugs item #681960, was opened at 2003-02-06 23:17
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=681960&group_id=5470

Category: Unicode
Group: Python 2.3
>Status: Closed
Resolution: None
Priority: 3
Submitted By: Kirill Simonov (kirill_simonov)
Assigned to: M.-A. Lemburg (lemburg)
Summary: Source encoding rules are extreme.

Initial Comment:
According to the PEP 0263, a source code that contains
non-ASCII
characters (ord(ch)>127) and does not define an
encoding causes
DeprecationWarning. In the future, such code will cause
SyntaxError.

While I believe that the idea of defining source code
encoding is very
useful, I think that the current solution is
unnecessary extreme.

It is very unfriendly for beginners. Imagine a student that
types her first script:

name = raw_input("What's your name? ")   # russian
here, of course
print "Hi %s!" % name

Do not even try to convince me that she must define an
encoding
here. That feature would break any possibility to use
Python in schools.

Actually the source code encoding only affects Unicode
literals.
The above script works the same way with any defined
encoding,
so the warning for this code is unnecessary.

As a solution, I propose to issue DeprecationWarning
(or SyntaxError)
only when a non-ASCII character is contained in a
Unicode literal.


----------------------------------------------------------------------

>Comment By: M.-A. Lemburg (lemburg)
Date: 2003-02-10 10:43

Message:
Logged In: YES 
user_id=38388

I've had a private discussion with Guido and Roman Suzi:

We'll add a way to set the source code default encoding via the
site.py/sitecustomize.py files. This should then allow anyone
wishing to customize the default behaviour to do so.

----------------------------------------------------------------------

Comment By: Kirill Simonov (kirill_simonov)
Date: 2003-02-07 00:28

Message:
Logged In: YES 
user_id=36553

Hello,

Yes, I understand that the encoding is for the whole source
file.

But

1. The current implementation already assumes that one uses
ASCII-
compatible encoding. So we can make a step further and do
not use any
encoding while reading a source file. And then we'll
translate u"..." using
'ascii' encoding.

2. How do you want to support UTF-16 encoding? This will
completely
break ordinary string literals! "aa" is a source code would
become "a\x00a\x00" after compilation. Or do I miss something?

3. Do not forget that your change breaks billions of scripts
that use
non-ASCII characters even in comments!

4. I can write a patch. I would be forced to do this anyway.



----------------------------------------------------------------------

Comment By: M.-A. Lemburg (lemburg)
Date: 2003-02-06 23:45

Message:
Logged In: YES 
user_id=38388

Sorry, but the implementation we chose decodes the complete
file,
not only the Unicode literals, so if you want to use a specific 
encoding in the source code, you have to be explicit about it.

Python's source code was originally never meant to contain
non-ASCII characters. The PEP implementation now officially
allows this provided that you use an encoding marker, e.g.

"""
# -*- coding: windows-1251 -*-
name = raw_input("   ? ")
print " %s" % name
"""

Note that this is also needed in order to support UTF-16
file formats which use two bytes per character. Python
will automatically detect these files, so if you really don't
like the coding marker, simply write the file using a UTF-16
aware editor which prepends a UTF-16 BOM mark to the
file.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=681960&group_id=5470


From noreply@sourceforge.net  Mon Feb 10 12:08:46 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Mon, 10 Feb 2003 04:08:46 -0800
Subject: [Python-bugs-list] [ python-Bugs-683883 ] Numeric Python >22 dimensions trouble
Message-ID: <E18iCju-0001i0-00@sc8-sf-web3.sourceforge.net>

Bugs item #683883, was opened at 2003-02-10 12:08
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=683883&group_id=5470

Category: Extension Modules
Group: 3rd Party
Status: Open
Resolution: None
Priority: 5
Submitted By: Jurjen N.E. Bos (jneb)
Assigned to: Nobody/Anonymous (nobody)
Summary: Numeric Python >22 dimensions trouble

Initial Comment:
Platform: Mac OS X
Python 2.2.2 (#138, Oct 25 2002, 23:10:42)  [CW CARBON GUSI2 THREADS GC]

Numeric Python is preinstalled here. My machine has plenty of RAM.
I did some research myself: I hope this helps figuring out where the bug is.

>>> from Numeric import zeroes
>>> size(zeros((1<<25,),'b')+0)
33554432
>>> size(zeros((2,)*25,'b'))
33554432
>>> size(zeros((2,)*21,'b')+0)
2097152
>>> size(zeros((2,)*22,'b')+0)
**BOOM**

The interpreter crashes after the last statement (not earlier). I assume this is not intended behaviour :-)
To make it more interesting:
>>> zeros((1,)*22,'b')+0
array([...]
>>> zeros((1,)*23,'b')+0
**BOOM**


- Jurjen

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=683883&group_id=5470


From noreply@sourceforge.net  Mon Feb 10 13:27:00 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Mon, 10 Feb 2003 05:27:00 -0800
Subject: [Python-bugs-list] [ python-Feature Requests-683910 ] zipfile should have a tarfile-like API
Message-ID: <E18iDxc-0006FT-00@sc8-sf-web2.sourceforge.net>

Feature Requests item #683910, was opened at 2003-02-11 00:26
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=683910&group_id=5470

>Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Anthony Baxter (anthonybaxter)
>Assigned to: Nobody/Anonymous (nobody)
Summary: zipfile should have a tarfile-like API

Initial Comment:
zipfile.ZipFile() should have an interface similar to
tarfile.TarFile() for extracting files. At it's
simplest, this can
be written:

tf = tarfile.open(self.filename, 'r')
for member in tf.getmembers():
     tf.extract(member)

This does "all the right things". This is a much more
pleasant
interface to use than the ZipFile equivalent.

Assigning to self as a reminder to do this.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=683910&group_id=5470


From noreply@sourceforge.net  Mon Feb 10 13:27:31 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Mon, 10 Feb 2003 05:27:31 -0800
Subject: [Python-bugs-list] [ python-Bugs-665835 ] filter() treatment of str and tuple inconsistent
Message-ID: <E18iDy7-0004oH-00@sc8-sf-web4.sourceforge.net>

Bugs item #665835, was opened at 2003-01-10 17:36
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=665835&group_id=5470

Category: Python Interpreter Core
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Walter Dörwald (doerwalter)
Assigned to: Raymond Hettinger (rhettinger)
Summary: filter() treatment of str and tuple inconsistent

Initial Comment:
class tuple2(tuple):
·  def __getitem__(self, index):
·  ·  return 2*tuple.__getitem__(self, index)

class str2(str):
·  def __getitem__(self, index):
·  ·  return chr(ord(str.__getitem__(self, index))+1)

print filter(lambda x: x>1, tuple2((1, 2)))
print filter(lambda x: x>"a", str2("ab"))

this prints:
(2,)
bc

i.e. the overwritten __getitem__ is ignored in the
first case, but honored in the second.


----------------------------------------------------------------------

>Comment By: Walter Dörwald (doerwalter)
Date: 2003-02-10 14:27

Message:
Logged In: YES 
user_id=89016

OK, this has been fixed: function==None and function==lambda x:x
now behave the same (for str and unicode, for tuples it's
still broken, because PyTuple_GetItem() is used. (Checked in
as Python/bltinmodule.c 2.278 and  Lib/test/test_builtin.py
1.12)

Why can't we simply replace PyTuple_GetItem() with
tp_as_sequence->sq_item in filtertuple()?

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-04 22:06

Message:
Logged In: YES 
user_id=6380

So it does. I guess the special shortcut for None should
only be taken
when it's a proper str instance.

----------------------------------------------------------------------

Comment By: Walter Dörwald (doerwalter)
Date: 2003-02-04 21:34

Message:
Logged In: YES 
user_id=89016

The subclass problem has been fixed in:
Python/bltinmodule.c 2.275
Lib/test/test_builtin.py 1.9

But now something strange happens:
---
class badstr(str):
···def __getitem__(self, index):
······return str.__getitem__(self, index).upper()

print filter(None, badstr("abc"))
print filter(lambda x: x, badstr("abc"))
---
This prints
---
abc
ABC
---
although according to the filter docstring they should be
the same.


----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-04 18:33

Message:
Logged In: YES 
user_id=6380

Yes, the special treatment of tuple, str and unicode is
problematic. :-(
I wish filter() had always returned a list for all input types.
But it's too late to change that.

However, I don't think that filter() should ever return a
*subclass* of tuple, str or unicode. Note that slicing a
subclass of these also doesn't return a subclass instance,
unless the subclass specifically overrides __getslice__.

I note that filter() of a tuple *almost* implements what I
think it should, except that if it receives an empty tuple
subclass, it returns it unchanged. The slicing and other
methods (e.g. lower()) have all been modified to make a copy
whose type is the base class; I think filter() should follow
suit.

Similar for filter() of strings and unicode.

----------------------------------------------------------------------

Comment By: Walter Dörwald (doerwalter)
Date: 2003-02-04 18:15

Message:
Logged In: YES 
user_id=89016

OK, the problem of __getitem__ not returning str or unicode
is fixed. Unfortunately the result is rather ugly. With the
following class:

class u(unicode):
   def __getitem__(self, index):
      return u(2*unicode.__getitem__(self, index))

filter neither returns a list nor an u object, but a unicode
object, defeating the whole purpose of the special treatment
of str/unicode. If we remove the special treatment this
problem would go away, furthermore __getitem__ returning
objects that are not str/unicode instances wouldn't be
problem any longer.


----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-03 23:36

Message:
Logged In: YES 
user_id=6380

Walter: if you can fix the bug in your latest message here,
go ahead and check it in. Seems like a case of a missing test.

Raymond: it turns out that the iterator in Python 2.2 has
the same problem with lists -- it special-cases lists. But
for tuples, the iterator uses PySequence_GetItem; the fast
tuple iterator in Python 2.3 introduces the problem for
tuples though.

I actually don't think there would be much disagreement that
this behavior (ignoring __getitem__) is a bug. There may be
disagreement over how important it is to fix it. Personally,
I've generally been on the side of "it needn't be fixed if
it slows down the common case", as long as a workaround
(like overriding __iter__ alongside the __getitem__
override) exists.

But I draw the line at being backwards incompatible with
Python 2.2. There fore I think the tuple iterator (and
probably also the string iterator) needs to be fixed, and I
still think that it would be best if the list iterator were
also fixed. One way to do this would be for the tp_iter
implementation to check whether
self->ob_type->tp_as_sequence->sq_item is not equal to the
list_item function (this is a good check to detect a
__getitem__ override) and then return an instance of the
generic sequence iterator instead of the list-specific iterator.

----------------------------------------------------------------------

Comment By: Walter Dörwald (doerwalter)
Date: 2003-01-27 13:24

Message:
Logged In: YES 
user_id=89016

Another problem with filter() is that filterstring() (and
the new filterunicode()) blindly assume that
tp_as_sequence->sq_item returns a str or unicode object with
len==1. This might fail with str or unicode subclasses:
----
class badstr(str):
   def __getitem__(self, index):
      return 42

s = filter(lambda x: x>=42, badstr("1234"))
print len(s), repr(s)
----
This prints
4 '\x00\x00\x00\x00'

----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2003-01-27 02:13

Message:
Logged In: YES 
user_id=80475

One other thought:  A major reason for implementing 
__iter__ in the first place is that objects were overriding 
__getitem__ and disregarding the index -- the __getitem__ 
interface just didn't make sense for iteration in some 
situations.  __iter__ was supposed to provide enormous 
flexibility in various ways to loop over a collection (inorder, 
preorder, postorder, priorityorder, sortedorder, hashorder, 
randomorder, etc).  Making iter() default to using 
__getitem__ was only supposed to be an expedient for 
backwards compatability.  Always using __getitem__ 
diminishes the flexibility and speed advantages.

Maybe the discussion belongs on python-dev. I'm sure a 
number of people feel strongly one way or the other.  The 
question might as well be addressed head-on before 2.3 
goes out the door. 

----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2003-01-27 01:54

Message:
Logged In: YES 
user_id=80475

I understand.  Ideally, *all* methods would respond to a 
single overridden method, but I think this is just a fact of 
life in object oriented programming.

I can't remember where you gave an example of a 
d.__getitem__() subclass override, but you were careful to 
point out that other methods, like d.get() also needed to 
be overridden so that the modified access applied 
everywhere.  Likewise, __iter__() or any other object 
access method must be assumed to access the underlying 
data structure directly and must be overridden.   For 
instance, creating a dictionary with case insensitive 
lookups entails overriding __getitem__(k), get(k,default), 
and pop(k) -- no one of them can be presumed to inform 
the others.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-01-27 01:17

Message:
Logged In: YES 
user_id=6380

Hm... that means that iter() of *amy* built-in type subclass
overriding __getitem__ bypasses the override, unless the
subclass also overrides __iter__. This sounds like a step in
the wrong direction. I think the built-in iterators should
be aware of subclasses overriding __getitem__ one way or
another. I hadn't realized this when we started the trend of
creating faster iterators for built-in types. :-(

----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2003-01-25 17:45

Message:
Logged In: YES 
user_id=80475

None of the existing iterators (incl dicts, lists, tuples, and 
files) use __getitem__.  Most likely, user defined iterators 
also access the data structure directly (for flexiblity and 
speed). Also, anything that uses PyTuple_GET_ITEM 
bypasses __getitem__.

If string/unicode iterators are added, they should also go
directly to the underlying data; otherwise, there is no point 
to it.

Also, the proposal to change filtertuple(), doesn't solve
inconsistencies within filterstring() which uses __getitem__ 
when there is a function call, but bypasses it when the 
function parameter is Py_None.

I think the right answer is to change filterstring() to use an 
iterator and to implement string/unicode iterators that 
access the data directly (not using __getitem__).

FYI for Tim:  MvL noticed and fixed the unicode vs string 
difference.  His patch, SF #636005, has not been applied 
yet.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-01-25 14:51

Message:
Logged In: YES 
user_id=6380

(But in addition th that, I don't mind having a custom
string iterator -- as long as it calls __getitem__ properly.
Hm, shouldn't the tuple iterator call __getitem__ properly too?)


----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2003-01-25 14:45

Message:
Logged In: YES 
user_id=31435

Just noting that filter() is unique in special-casing the type 
of the input.  It's always been surprising that way, and, 
e.g., filtering a string produces a string, but filtering a 
Unicode string produces a list.

map() and reduce() don't play games like that, and always 
use the iteration protocol to march over their inputs.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-01-25 14:36

Message:
Logged In: YES 
user_id=6380

I don't know which Python sources Raymond has been reading,
but in the sources I've got in front of me, there are
special cases for strings and tuples, and these *don't* use
iter(). It so happens that the tuple special-case calls
PyTuple_GetItem(), which doesn't call your __getitem__,
while the string special-case calls the sq_item slot
function, which (in your case) will be a wrapper that calls
your __getitem__.

A minimal fix would be to only call filtertuple for strict
tuples -- although this changes the output type, but I don't
think one should count on filter() of a tuple subclass
returning a tuple (and it can't be made to return an
instance of the subclass either -- we don't know the
constructor signature).

Similar fixes probably need to be made to map() and maybe
reduce().

----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2003-01-25 04:47

Message:
Logged In: YES 
user_id=80475

The problem isn't with filter() which correctly calls iter() in 
both cases.

Tuple object have their own iterator which loops over 
elements directly and has no intervening calls to 
__getitem__().

String objects do not define a custom iterator, so iter() 
wraps itself around consecutive calls to __getitem__().

The resolution is to provide string objects with their own 
iterator. As a side benefit, iteration will run just a tiny bit 
faster.  The same applies to unicode objects.

Guido, do you care about this and want me to fix it or 
would you like to close it as "won't fix".


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=665835&group_id=5470


From noreply@sourceforge.net  Mon Feb 10 13:26:34 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Mon, 10 Feb 2003 05:26:34 -0800
Subject: [Python-bugs-list] [ python-Bugs-683910 ] zipfile should have a tarfile-like API
Message-ID: <E18iDxC-0006Ea-00@sc8-sf-web2.sourceforge.net>

Bugs item #683910, was opened at 2003-02-11 00:26
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=683910&group_id=5470

Category: Python Library
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Anthony Baxter (anthonybaxter)
Assigned to: Anthony Baxter (anthonybaxter)
Summary: zipfile should have a tarfile-like API

Initial Comment:
zipfile.ZipFile() should have an interface similar to
tarfile.TarFile() for extracting files. At it's
simplest, this can
be written:

tf = tarfile.open(self.filename, 'r')
for member in tf.getmembers():
     tf.extract(member)

This does "all the right things". This is a much more
pleasant
interface to use than the ZipFile equivalent.

Assigning to self as a reminder to do this.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=683910&group_id=5470


From noreply@sourceforge.net  Mon Feb 10 14:06:41 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Mon, 10 Feb 2003 06:06:41 -0800
Subject: [Python-bugs-list] [ python-Bugs-683883 ] Numeric Python >22 dimensions trouble
Message-ID: <E18iEa1-00077j-00@sc8-sf-web4.sourceforge.net>

Bugs item #683883, was opened at 2003-02-10 12:08
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=683883&group_id=5470

Category: Extension Modules
Group: 3rd Party
>Status: Closed
>Resolution: Invalid
Priority: 5
Submitted By: Jurjen N.E. Bos (jneb)
>Assigned to: Michael Hudson (mwh)
Summary: Numeric Python >22 dimensions trouble

Initial Comment:
Platform: Mac OS X
Python 2.2.2 (#138, Oct 25 2002, 23:10:42)  [CW CARBON GUSI2 THREADS GC]

Numeric Python is preinstalled here. My machine has plenty of RAM.
I did some research myself: I hope this helps figuring out where the bug is.

>>> from Numeric import zeroes
>>> size(zeros((1<<25,),'b')+0)
33554432
>>> size(zeros((2,)*25,'b'))
33554432
>>> size(zeros((2,)*21,'b')+0)
2097152
>>> size(zeros((2,)*22,'b')+0)
**BOOM**

The interpreter crashes after the last statement (not earlier). I assume this is not intended behaviour :-)
To make it more interesting:
>>> zeros((1,)*22,'b')+0
array([...]
>>> zeros((1,)*23,'b')+0
**BOOM**


- Jurjen

----------------------------------------------------------------------

>Comment By: Michael Hudson (mwh)
Date: 2003-02-10 14:06

Message:
Logged In: YES 
user_id=6656

Numeric Python is not maintained by us!  You need to report
this to the numpy project:

http://sourceforge.net/projects/numpy

(unless it turns out to be a bug in Python, of course, but
that seems unlikely).

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=683883&group_id=5470


From noreply@sourceforge.net  Mon Feb 10 14:57:40 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Mon, 10 Feb 2003 06:57:40 -0800
Subject: [Python-bugs-list] [ python-Bugs-683938 ] HTMLParser attribute parsing bug
Message-ID: <E18iFNM-00049V-00@sc8-sf-web1.sourceforge.net>

Bugs item #683938, was opened at 2003-02-10 09:57
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=683938&group_id=5470

Category: Python Library
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Fred L. Drake, Jr. (fdrake)
Assigned to: Fred L. Drake, Jr. (fdrake)
Summary: HTMLParser attribute parsing bug

Initial Comment:
HTMLParser (reportedly)  fails to parse this construct:

<a href="http://ss"title="pe">P</a>

(Note that a required space between the two attributes
of the "a" tag has been omitted).  The W3C validator
appearantly treats this differently, so there's no
point in arguing the letter of the law.

Assigned to me.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=683938&group_id=5470


From noreply@sourceforge.net  Mon Feb 10 15:20:07 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Mon, 10 Feb 2003 07:20:07 -0800
Subject: [Python-bugs-list] [ python-Bugs-665835 ] filter() treatment of str and tuple inconsistent
Message-ID: <E18iFj5-0003is-00@sc8-sf-web2.sourceforge.net>

Bugs item #665835, was opened at 2003-01-10 11:36
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=665835&group_id=5470

Category: Python Interpreter Core
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Walter Dörwald (doerwalter)
Assigned to: Raymond Hettinger (rhettinger)
Summary: filter() treatment of str and tuple inconsistent

Initial Comment:
class tuple2(tuple):
·  def __getitem__(self, index):
·  ·  return 2*tuple.__getitem__(self, index)

class str2(str):
·  def __getitem__(self, index):
·  ·  return chr(ord(str.__getitem__(self, index))+1)

print filter(lambda x: x>1, tuple2((1, 2)))
print filter(lambda x: x>"a", str2("ab"))

this prints:
(2,)
bc

i.e. the overwritten __getitem__ is ignored in the
first case, but honored in the second.


----------------------------------------------------------------------

>Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-10 10:20

Message:
Logged In: YES 
user_id=6380

Feel free to fix filtertuple() too. Just note that
tp_as_sequence might be NULL, or ...->sq_item might be NULL.
I'm not 100% sure that those can never be NULL for a tuple
subclass.

----------------------------------------------------------------------

Comment By: Walter Dörwald (doerwalter)
Date: 2003-02-10 08:27

Message:
Logged In: YES 
user_id=89016

OK, this has been fixed: function==None and function==lambda x:x
now behave the same (for str and unicode, for tuples it's
still broken, because PyTuple_GetItem() is used. (Checked in
as Python/bltinmodule.c 2.278 and  Lib/test/test_builtin.py
1.12)

Why can't we simply replace PyTuple_GetItem() with
tp_as_sequence->sq_item in filtertuple()?

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-04 16:06

Message:
Logged In: YES 
user_id=6380

So it does. I guess the special shortcut for None should
only be taken
when it's a proper str instance.

----------------------------------------------------------------------

Comment By: Walter Dörwald (doerwalter)
Date: 2003-02-04 15:34

Message:
Logged In: YES 
user_id=89016

The subclass problem has been fixed in:
Python/bltinmodule.c 2.275
Lib/test/test_builtin.py 1.9

But now something strange happens:
---
class badstr(str):
···def __getitem__(self, index):
······return str.__getitem__(self, index).upper()

print filter(None, badstr("abc"))
print filter(lambda x: x, badstr("abc"))
---
This prints
---
abc
ABC
---
although according to the filter docstring they should be
the same.


----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-04 12:33

Message:
Logged In: YES 
user_id=6380

Yes, the special treatment of tuple, str and unicode is
problematic. :-(
I wish filter() had always returned a list for all input types.
But it's too late to change that.

However, I don't think that filter() should ever return a
*subclass* of tuple, str or unicode. Note that slicing a
subclass of these also doesn't return a subclass instance,
unless the subclass specifically overrides __getslice__.

I note that filter() of a tuple *almost* implements what I
think it should, except that if it receives an empty tuple
subclass, it returns it unchanged. The slicing and other
methods (e.g. lower()) have all been modified to make a copy
whose type is the base class; I think filter() should follow
suit.

Similar for filter() of strings and unicode.

----------------------------------------------------------------------

Comment By: Walter Dörwald (doerwalter)
Date: 2003-02-04 12:15

Message:
Logged In: YES 
user_id=89016

OK, the problem of __getitem__ not returning str or unicode
is fixed. Unfortunately the result is rather ugly. With the
following class:

class u(unicode):
   def __getitem__(self, index):
      return u(2*unicode.__getitem__(self, index))

filter neither returns a list nor an u object, but a unicode
object, defeating the whole purpose of the special treatment
of str/unicode. If we remove the special treatment this
problem would go away, furthermore __getitem__ returning
objects that are not str/unicode instances wouldn't be
problem any longer.


----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-03 17:36

Message:
Logged In: YES 
user_id=6380

Walter: if you can fix the bug in your latest message here,
go ahead and check it in. Seems like a case of a missing test.

Raymond: it turns out that the iterator in Python 2.2 has
the same problem with lists -- it special-cases lists. But
for tuples, the iterator uses PySequence_GetItem; the fast
tuple iterator in Python 2.3 introduces the problem for
tuples though.

I actually don't think there would be much disagreement that
this behavior (ignoring __getitem__) is a bug. There may be
disagreement over how important it is to fix it. Personally,
I've generally been on the side of "it needn't be fixed if
it slows down the common case", as long as a workaround
(like overriding __iter__ alongside the __getitem__
override) exists.

But I draw the line at being backwards incompatible with
Python 2.2. There fore I think the tuple iterator (and
probably also the string iterator) needs to be fixed, and I
still think that it would be best if the list iterator were
also fixed. One way to do this would be for the tp_iter
implementation to check whether
self->ob_type->tp_as_sequence->sq_item is not equal to the
list_item function (this is a good check to detect a
__getitem__ override) and then return an instance of the
generic sequence iterator instead of the list-specific iterator.

----------------------------------------------------------------------

Comment By: Walter Dörwald (doerwalter)
Date: 2003-01-27 07:24

Message:
Logged In: YES 
user_id=89016

Another problem with filter() is that filterstring() (and
the new filterunicode()) blindly assume that
tp_as_sequence->sq_item returns a str or unicode object with
len==1. This might fail with str or unicode subclasses:
----
class badstr(str):
   def __getitem__(self, index):
      return 42

s = filter(lambda x: x>=42, badstr("1234"))
print len(s), repr(s)
----
This prints
4 '\x00\x00\x00\x00'

----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2003-01-26 20:13

Message:
Logged In: YES 
user_id=80475

One other thought:  A major reason for implementing 
__iter__ in the first place is that objects were overriding 
__getitem__ and disregarding the index -- the __getitem__ 
interface just didn't make sense for iteration in some 
situations.  __iter__ was supposed to provide enormous 
flexibility in various ways to loop over a collection (inorder, 
preorder, postorder, priorityorder, sortedorder, hashorder, 
randomorder, etc).  Making iter() default to using 
__getitem__ was only supposed to be an expedient for 
backwards compatability.  Always using __getitem__ 
diminishes the flexibility and speed advantages.

Maybe the discussion belongs on python-dev. I'm sure a 
number of people feel strongly one way or the other.  The 
question might as well be addressed head-on before 2.3 
goes out the door. 

----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2003-01-26 19:54

Message:
Logged In: YES 
user_id=80475

I understand.  Ideally, *all* methods would respond to a 
single overridden method, but I think this is just a fact of 
life in object oriented programming.

I can't remember where you gave an example of a 
d.__getitem__() subclass override, but you were careful to 
point out that other methods, like d.get() also needed to 
be overridden so that the modified access applied 
everywhere.  Likewise, __iter__() or any other object 
access method must be assumed to access the underlying 
data structure directly and must be overridden.   For 
instance, creating a dictionary with case insensitive 
lookups entails overriding __getitem__(k), get(k,default), 
and pop(k) -- no one of them can be presumed to inform 
the others.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-01-26 19:17

Message:
Logged In: YES 
user_id=6380

Hm... that means that iter() of *amy* built-in type subclass
overriding __getitem__ bypasses the override, unless the
subclass also overrides __iter__. This sounds like a step in
the wrong direction. I think the built-in iterators should
be aware of subclasses overriding __getitem__ one way or
another. I hadn't realized this when we started the trend of
creating faster iterators for built-in types. :-(

----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2003-01-25 11:45

Message:
Logged In: YES 
user_id=80475

None of the existing iterators (incl dicts, lists, tuples, and 
files) use __getitem__.  Most likely, user defined iterators 
also access the data structure directly (for flexiblity and 
speed). Also, anything that uses PyTuple_GET_ITEM 
bypasses __getitem__.

If string/unicode iterators are added, they should also go
directly to the underlying data; otherwise, there is no point 
to it.

Also, the proposal to change filtertuple(), doesn't solve
inconsistencies within filterstring() which uses __getitem__ 
when there is a function call, but bypasses it when the 
function parameter is Py_None.

I think the right answer is to change filterstring() to use an 
iterator and to implement string/unicode iterators that 
access the data directly (not using __getitem__).

FYI for Tim:  MvL noticed and fixed the unicode vs string 
difference.  His patch, SF #636005, has not been applied 
yet.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-01-25 08:51

Message:
Logged In: YES 
user_id=6380

(But in addition th that, I don't mind having a custom
string iterator -- as long as it calls __getitem__ properly.
Hm, shouldn't the tuple iterator call __getitem__ properly too?)


----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2003-01-25 08:45

Message:
Logged In: YES 
user_id=31435

Just noting that filter() is unique in special-casing the type 
of the input.  It's always been surprising that way, and, 
e.g., filtering a string produces a string, but filtering a 
Unicode string produces a list.

map() and reduce() don't play games like that, and always 
use the iteration protocol to march over their inputs.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-01-25 08:36

Message:
Logged In: YES 
user_id=6380

I don't know which Python sources Raymond has been reading,
but in the sources I've got in front of me, there are
special cases for strings and tuples, and these *don't* use
iter(). It so happens that the tuple special-case calls
PyTuple_GetItem(), which doesn't call your __getitem__,
while the string special-case calls the sq_item slot
function, which (in your case) will be a wrapper that calls
your __getitem__.

A minimal fix would be to only call filtertuple for strict
tuples -- although this changes the output type, but I don't
think one should count on filter() of a tuple subclass
returning a tuple (and it can't be made to return an
instance of the subclass either -- we don't know the
constructor signature).

Similar fixes probably need to be made to map() and maybe
reduce().

----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2003-01-24 22:47

Message:
Logged In: YES 
user_id=80475

The problem isn't with filter() which correctly calls iter() in 
both cases.

Tuple object have their own iterator which loops over 
elements directly and has no intervening calls to 
__getitem__().

String objects do not define a custom iterator, so iter() 
wraps itself around consecutive calls to __getitem__().

The resolution is to provide string objects with their own 
iterator. As a side benefit, iteration will run just a tiny bit 
faster.  The same applies to unicode objects.

Guido, do you care about this and want me to fix it or 
would you like to close it as "won't fix".


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=665835&group_id=5470


From noreply@sourceforge.net  Mon Feb 10 15:39:48 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Mon, 10 Feb 2003 07:39:48 -0800
Subject: [Python-bugs-list] [ python-Bugs-681960 ] Source encoding rules are extreme.
Message-ID: <E18iG28-0003Ix-00@sc8-sf-web3.sourceforge.net>

Bugs item #681960, was opened at 2003-02-06 22:17
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=681960&group_id=5470

Category: Unicode
Group: Python 2.3
Status: Closed
Resolution: None
Priority: 3
Submitted By: Kirill Simonov (kirill_simonov)
Assigned to: M.-A. Lemburg (lemburg)
Summary: Source encoding rules are extreme.

Initial Comment:
According to the PEP 0263, a source code that contains
non-ASCII
characters (ord(ch)>127) and does not define an
encoding causes
DeprecationWarning. In the future, such code will cause
SyntaxError.

While I believe that the idea of defining source code
encoding is very
useful, I think that the current solution is
unnecessary extreme.

It is very unfriendly for beginners. Imagine a student that
types her first script:

name = raw_input("What's your name? ")   # russian
here, of course
print "Hi %s!" % name

Do not even try to convince me that she must define an
encoding
here. That feature would break any possibility to use
Python in schools.

Actually the source code encoding only affects Unicode
literals.
The above script works the same way with any defined
encoding,
so the warning for this code is unnecessary.

As a solution, I propose to issue DeprecationWarning
(or SyntaxError)
only when a non-ASCII character is contained in a
Unicode literal.


----------------------------------------------------------------------

>Comment By: Kirill Simonov (kirill_simonov)
Date: 2003-02-10 15:39

Message:
Logged In: YES 
user_id=36553

I like this. Thanks.

----------------------------------------------------------------------

Comment By: M.-A. Lemburg (lemburg)
Date: 2003-02-10 09:43

Message:
Logged In: YES 
user_id=38388

I've had a private discussion with Guido and Roman Suzi:

We'll add a way to set the source code default encoding via the
site.py/sitecustomize.py files. This should then allow anyone
wishing to customize the default behaviour to do so.

----------------------------------------------------------------------

Comment By: Kirill Simonov (kirill_simonov)
Date: 2003-02-06 23:28

Message:
Logged In: YES 
user_id=36553

Hello,

Yes, I understand that the encoding is for the whole source
file.

But

1. The current implementation already assumes that one uses
ASCII-
compatible encoding. So we can make a step further and do
not use any
encoding while reading a source file. And then we'll
translate u"..." using
'ascii' encoding.

2. How do you want to support UTF-16 encoding? This will
completely
break ordinary string literals! "aa" is a source code would
become "a\x00a\x00" after compilation. Or do I miss something?

3. Do not forget that your change breaks billions of scripts
that use
non-ASCII characters even in comments!

4. I can write a patch. I would be forced to do this anyway.



----------------------------------------------------------------------

Comment By: M.-A. Lemburg (lemburg)
Date: 2003-02-06 22:45

Message:
Logged In: YES 
user_id=38388

Sorry, but the implementation we chose decodes the complete
file,
not only the Unicode literals, so if you want to use a specific 
encoding in the source code, you have to be explicit about it.

Python's source code was originally never meant to contain
non-ASCII characters. The PEP implementation now officially
allows this provided that you use an encoding marker, e.g.

"""
# -*- coding: windows-1251 -*-
name = raw_input("   ? ")
print " %s" % name
"""

Note that this is also needed in order to support UTF-16
file formats which use two bytes per character. Python
will automatically detect these files, so if you really don't
like the coding marker, simply write the file using a UTF-16
aware editor which prepends a UTF-16 BOM mark to the
file.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=681960&group_id=5470


From noreply@sourceforge.net  Mon Feb 10 16:54:06 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Mon, 10 Feb 2003 08:54:06 -0800
Subject: [Python-bugs-list] [ python-Bugs-684022 ] extended slice strangeness
Message-ID: <E18iHC2-0007Qu-00@sc8-sf-web3.sourceforge.net>

Bugs item #684022, was opened at 2003-02-10 11:54
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=684022&group_id=5470

Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Robert Roy (rjroy)
Assigned to: Nobody/Anonymous (nobody)
Summary: extended slice strangeness

Initial Comment:
I was playing with extended slices and wanted to see if 
it would return a reversed slice. The behaviour was not 
quite what I expected. See the following


Python 2.3a1 (#38, Dec 31 2002, 17:53:59) [MSC 
v.1200 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more 
information.
>>> l = range(12)
>>> l
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11]
## this works as expected
>>> l[0:5:2]
[0, 2, 4]

## this returns empty list ??
>>> l[0:5:-1]
[]

## this works as expected
>>> l[::-1]
[11, 10, 9, 8, 7, 6, 5, 4, 3, 2, 1, 0]

## I would expect this to return same as above
>>> l[0::-1]
[0]

## would expect empty list since l[5:0] == []
>>> l[5:0:-1]
[5, 4, 3, 2, 1]

## would expect [11, 10, 9, 8, 7, 6,5]
>>> l[5::-1]
[5, 4, 3, 2, 1, 0]
>>>


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=684022&group_id=5470


From noreply@sourceforge.net  Mon Feb 10 16:57:44 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Mon, 10 Feb 2003 08:57:44 -0800
Subject: [Python-bugs-list] [ python-Bugs-602444 ] non greedy match bug
Message-ID: <E18iHFY-0007qC-00@sc8-sf-web4.sourceforge.net>

Bugs item #602444, was opened at 2002-08-30 10:44
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=602444&group_id=5470

Category: Regular Expressions
>Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Robert Roy (rjroy)
Assigned to: Fredrik Lundh (effbot)
Summary: non greedy match bug

Initial Comment:
When using the following re to extract all objects from a 
PDF file, I get a maximum recursion limit exceeded error.

Attached is a pdf file that will reproduce the error.

If I do import pre as re, it works fine.

platform is Win2k, Python 2.2.1 build #34

#######
import re

GETOBJECT = re.compile(r'\d+\s+\d+\s+obj.+?endobj', 
re.I|re.S|re.M)

pdf = open('userguide.pdf', 'rb').read()
all = GETOBJECT.findall(pdf)
print len(all)


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=602444&group_id=5470


From noreply@sourceforge.net  Mon Feb 10 16:59:50 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Mon, 10 Feb 2003 08:59:50 -0800
Subject: [Python-bugs-list] [ python-Bugs-683726 ] METH_CLASS allows binding to the wrong class
Message-ID: <E18iHHa-0007hr-00@sc8-sf-web3.sourceforge.net>

Bugs item #683726, was opened at 2003-02-09 18:21
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=683726&group_id=5470

Category: Python Interpreter Core
Group: Python 2.3
Status: Open
Resolution: None
Priority: 7
Submitted By: Greg Chapman (glchapman)
Assigned to: Guido van Rossum (gvanrossum)
Summary: METH_CLASS allows binding to the wrong class

Initial Comment:
I just noticed that this doesn't raise an exception:

>>> cmeth = dict.__dict__['fromkeys'].__get__(None, 
list)
>>> cmeth
<built-in method fromkeys of type object at 
0x1E0BB2E0>
>>> cmeth.__self__
<type 'list'>

Looking over the various METH_CLASS 
implementations, it doesn't look like they depend on the 
first (cls) parameter being of any particular type, but it 
might be nice to flag something like the above with an 
exception.

Also, there's this:

>>> cmeth2 = dict.__dict__['fromkeys'].__get__(None, 
42)
>>> cmeth2
<built-in method fromkeys of int object at 0x0074FD90>
>>> print cmeth2.__self__
42

Above, 42 gets passed through to classmethod_get in 
the type parameter, despite the fact that this parameter 
is typed as a PyTypeObject *. Perhpas wrap_descr_get 
should check that its second parameter (if it gets one) is 
a type?
 

----------------------------------------------------------------------

>Comment By: Greg Chapman (glchapman)
Date: 2003-02-10 07:59

Message:
Logged In: YES 
user_id=86307

I'm not entirely sure what the desired semantics for 
classmethod_get should be, but it seems to me they should 
be something like the following:

1) descr.__get__(None)
   descr.__get__(None, None)

This case would construct a PyCFunction using descr's 
d_type.  (What happens if both obj and type are NULL (as 
opposed to None)?  I think that should be a BadInternalCall, 
but perhaps it should also return a function with d_type.)

2) descr.__get__(notNone)
    descr.__get__(notNone, anything)

I think this case should be handled as for other descriptors.  
notNone would be checked to see if it is an instance of 
d_type; if so, a PyCFunction would be constructed using 
type(notNone).

3) descr.__get__(None, notNone)

Here notNone would be checked to ensure it is a type object, 
and then checked to see if it is a subtype of d_type.  If it 
passes these checks, then a PyCFunction is constructed 
using notNone.  (This branch would also handle the case 
where the obj == NULL in classmethod_get).

Alternatively, if it is always safe to assume that obj == NULL 
means that classmethod_get was called by the core library, 
and so the type parameter is correct, there could be a 0'th 
branch for obj == NULL which returns a PyCFunction 
constructed using type.


----------------------------------------------------------------------

Comment By: Greg Chapman (glchapman)
Date: 2003-02-09 18:38

Message:
Logged In: YES 
user_id=86307

I just realized that the above can be used to crash python, so 
I believe some sort of checking has to be added to 
classmethod_get:

>>> cmeth = dict.__dict__['fromkeys'].__get__(None)
>>> cmeth(4)

(above causes and access violation because the cls object 
passed to the fromkeys method is NULL).


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=683726&group_id=5470


From noreply@sourceforge.net  Mon Feb 10 16:59:25 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Mon, 10 Feb 2003 08:59:25 -0800
Subject: [Python-bugs-list] [ python-Bugs-684022 ] extended slice strangeness
Message-ID: <E18iHHB-0000mO-00@sc8-sf-web2.sourceforge.net>

Bugs item #684022, was opened at 2003-02-10 11:54
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=684022&group_id=5470

>Category: Python Interpreter Core
>Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Robert Roy (rjroy)
Assigned to: Nobody/Anonymous (nobody)
Summary: extended slice strangeness

Initial Comment:
I was playing with extended slices and wanted to see if 
it would return a reversed slice. The behaviour was not 
quite what I expected. See the following


Python 2.3a1 (#38, Dec 31 2002, 17:53:59) [MSC 
v.1200 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more 
information.
>>> l = range(12)
>>> l
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11]
## this works as expected
>>> l[0:5:2]
[0, 2, 4]

## this returns empty list ??
>>> l[0:5:-1]
[]

## this works as expected
>>> l[::-1]
[11, 10, 9, 8, 7, 6, 5, 4, 3, 2, 1, 0]

## I would expect this to return same as above
>>> l[0::-1]
[0]

## would expect empty list since l[5:0] == []
>>> l[5:0:-1]
[5, 4, 3, 2, 1]

## would expect [11, 10, 9, 8, 7, 6,5]
>>> l[5::-1]
[5, 4, 3, 2, 1, 0]
>>>


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=684022&group_id=5470


From noreply@sourceforge.net  Mon Feb 10 17:44:22 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Mon, 10 Feb 2003 09:44:22 -0800
Subject: [Python-bugs-list] [ python-Bugs-665835 ] filter() treatment of str and tuple inconsistent
Message-ID: <E18iHyg-0003ZP-00@sc8-sf-web2.sourceforge.net>

Bugs item #665835, was opened at 2003-01-10 17:36
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=665835&group_id=5470

Category: Python Interpreter Core
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Walter Dörwald (doerwalter)
Assigned to: Raymond Hettinger (rhettinger)
Summary: filter() treatment of str and tuple inconsistent

Initial Comment:
class tuple2(tuple):
·  def __getitem__(self, index):
·  ·  return 2*tuple.__getitem__(self, index)

class str2(str):
·  def __getitem__(self, index):
·  ·  return chr(ord(str.__getitem__(self, index))+1)

print filter(lambda x: x>1, tuple2((1, 2)))
print filter(lambda x: x>"a", str2("ab"))

this prints:
(2,)
bc

i.e. the overwritten __getitem__ is ignored in the
first case, but honored in the second.


----------------------------------------------------------------------

>Comment By: Walter Dörwald (doerwalter)
Date: 2003-02-10 18:44

Message:
Logged In: YES 
user_id=89016

Checked in as:
Lib/test/test_builtin.py 1.13
Python/bltinmodule.c 2.280


----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-10 16:20

Message:
Logged In: YES 
user_id=6380

Feel free to fix filtertuple() too. Just note that
tp_as_sequence might be NULL, or ...->sq_item might be NULL.
I'm not 100% sure that those can never be NULL for a tuple
subclass.

----------------------------------------------------------------------

Comment By: Walter Dörwald (doerwalter)
Date: 2003-02-10 14:27

Message:
Logged In: YES 
user_id=89016

OK, this has been fixed: function==None and function==lambda x:x
now behave the same (for str and unicode, for tuples it's
still broken, because PyTuple_GetItem() is used. (Checked in
as Python/bltinmodule.c 2.278 and  Lib/test/test_builtin.py
1.12)

Why can't we simply replace PyTuple_GetItem() with
tp_as_sequence->sq_item in filtertuple()?

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-04 22:06

Message:
Logged In: YES 
user_id=6380

So it does. I guess the special shortcut for None should
only be taken
when it's a proper str instance.

----------------------------------------------------------------------

Comment By: Walter Dörwald (doerwalter)
Date: 2003-02-04 21:34

Message:
Logged In: YES 
user_id=89016

The subclass problem has been fixed in:
Python/bltinmodule.c 2.275
Lib/test/test_builtin.py 1.9

But now something strange happens:
---
class badstr(str):
···def __getitem__(self, index):
······return str.__getitem__(self, index).upper()

print filter(None, badstr("abc"))
print filter(lambda x: x, badstr("abc"))
---
This prints
---
abc
ABC
---
although according to the filter docstring they should be
the same.


----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-04 18:33

Message:
Logged In: YES 
user_id=6380

Yes, the special treatment of tuple, str and unicode is
problematic. :-(
I wish filter() had always returned a list for all input types.
But it's too late to change that.

However, I don't think that filter() should ever return a
*subclass* of tuple, str or unicode. Note that slicing a
subclass of these also doesn't return a subclass instance,
unless the subclass specifically overrides __getslice__.

I note that filter() of a tuple *almost* implements what I
think it should, except that if it receives an empty tuple
subclass, it returns it unchanged. The slicing and other
methods (e.g. lower()) have all been modified to make a copy
whose type is the base class; I think filter() should follow
suit.

Similar for filter() of strings and unicode.

----------------------------------------------------------------------

Comment By: Walter Dörwald (doerwalter)
Date: 2003-02-04 18:15

Message:
Logged In: YES 
user_id=89016

OK, the problem of __getitem__ not returning str or unicode
is fixed. Unfortunately the result is rather ugly. With the
following class:

class u(unicode):
   def __getitem__(self, index):
      return u(2*unicode.__getitem__(self, index))

filter neither returns a list nor an u object, but a unicode
object, defeating the whole purpose of the special treatment
of str/unicode. If we remove the special treatment this
problem would go away, furthermore __getitem__ returning
objects that are not str/unicode instances wouldn't be
problem any longer.


----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-03 23:36

Message:
Logged In: YES 
user_id=6380

Walter: if you can fix the bug in your latest message here,
go ahead and check it in. Seems like a case of a missing test.

Raymond: it turns out that the iterator in Python 2.2 has
the same problem with lists -- it special-cases lists. But
for tuples, the iterator uses PySequence_GetItem; the fast
tuple iterator in Python 2.3 introduces the problem for
tuples though.

I actually don't think there would be much disagreement that
this behavior (ignoring __getitem__) is a bug. There may be
disagreement over how important it is to fix it. Personally,
I've generally been on the side of "it needn't be fixed if
it slows down the common case", as long as a workaround
(like overriding __iter__ alongside the __getitem__
override) exists.

But I draw the line at being backwards incompatible with
Python 2.2. There fore I think the tuple iterator (and
probably also the string iterator) needs to be fixed, and I
still think that it would be best if the list iterator were
also fixed. One way to do this would be for the tp_iter
implementation to check whether
self->ob_type->tp_as_sequence->sq_item is not equal to the
list_item function (this is a good check to detect a
__getitem__ override) and then return an instance of the
generic sequence iterator instead of the list-specific iterator.

----------------------------------------------------------------------

Comment By: Walter Dörwald (doerwalter)
Date: 2003-01-27 13:24

Message:
Logged In: YES 
user_id=89016

Another problem with filter() is that filterstring() (and
the new filterunicode()) blindly assume that
tp_as_sequence->sq_item returns a str or unicode object with
len==1. This might fail with str or unicode subclasses:
----
class badstr(str):
   def __getitem__(self, index):
      return 42

s = filter(lambda x: x>=42, badstr("1234"))
print len(s), repr(s)
----
This prints
4 '\x00\x00\x00\x00'

----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2003-01-27 02:13

Message:
Logged In: YES 
user_id=80475

One other thought:  A major reason for implementing 
__iter__ in the first place is that objects were overriding 
__getitem__ and disregarding the index -- the __getitem__ 
interface just didn't make sense for iteration in some 
situations.  __iter__ was supposed to provide enormous 
flexibility in various ways to loop over a collection (inorder, 
preorder, postorder, priorityorder, sortedorder, hashorder, 
randomorder, etc).  Making iter() default to using 
__getitem__ was only supposed to be an expedient for 
backwards compatability.  Always using __getitem__ 
diminishes the flexibility and speed advantages.

Maybe the discussion belongs on python-dev. I'm sure a 
number of people feel strongly one way or the other.  The 
question might as well be addressed head-on before 2.3 
goes out the door. 

----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2003-01-27 01:54

Message:
Logged In: YES 
user_id=80475

I understand.  Ideally, *all* methods would respond to a 
single overridden method, but I think this is just a fact of 
life in object oriented programming.

I can't remember where you gave an example of a 
d.__getitem__() subclass override, but you were careful to 
point out that other methods, like d.get() also needed to 
be overridden so that the modified access applied 
everywhere.  Likewise, __iter__() or any other object 
access method must be assumed to access the underlying 
data structure directly and must be overridden.   For 
instance, creating a dictionary with case insensitive 
lookups entails overriding __getitem__(k), get(k,default), 
and pop(k) -- no one of them can be presumed to inform 
the others.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-01-27 01:17

Message:
Logged In: YES 
user_id=6380

Hm... that means that iter() of *amy* built-in type subclass
overriding __getitem__ bypasses the override, unless the
subclass also overrides __iter__. This sounds like a step in
the wrong direction. I think the built-in iterators should
be aware of subclasses overriding __getitem__ one way or
another. I hadn't realized this when we started the trend of
creating faster iterators for built-in types. :-(

----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2003-01-25 17:45

Message:
Logged In: YES 
user_id=80475

None of the existing iterators (incl dicts, lists, tuples, and 
files) use __getitem__.  Most likely, user defined iterators 
also access the data structure directly (for flexiblity and 
speed). Also, anything that uses PyTuple_GET_ITEM 
bypasses __getitem__.

If string/unicode iterators are added, they should also go
directly to the underlying data; otherwise, there is no point 
to it.

Also, the proposal to change filtertuple(), doesn't solve
inconsistencies within filterstring() which uses __getitem__ 
when there is a function call, but bypasses it when the 
function parameter is Py_None.

I think the right answer is to change filterstring() to use an 
iterator and to implement string/unicode iterators that 
access the data directly (not using __getitem__).

FYI for Tim:  MvL noticed and fixed the unicode vs string 
difference.  His patch, SF #636005, has not been applied 
yet.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-01-25 14:51

Message:
Logged In: YES 
user_id=6380

(But in addition th that, I don't mind having a custom
string iterator -- as long as it calls __getitem__ properly.
Hm, shouldn't the tuple iterator call __getitem__ properly too?)


----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2003-01-25 14:45

Message:
Logged In: YES 
user_id=31435

Just noting that filter() is unique in special-casing the type 
of the input.  It's always been surprising that way, and, 
e.g., filtering a string produces a string, but filtering a 
Unicode string produces a list.

map() and reduce() don't play games like that, and always 
use the iteration protocol to march over their inputs.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-01-25 14:36

Message:
Logged In: YES 
user_id=6380

I don't know which Python sources Raymond has been reading,
but in the sources I've got in front of me, there are
special cases for strings and tuples, and these *don't* use
iter(). It so happens that the tuple special-case calls
PyTuple_GetItem(), which doesn't call your __getitem__,
while the string special-case calls the sq_item slot
function, which (in your case) will be a wrapper that calls
your __getitem__.

A minimal fix would be to only call filtertuple for strict
tuples -- although this changes the output type, but I don't
think one should count on filter() of a tuple subclass
returning a tuple (and it can't be made to return an
instance of the subclass either -- we don't know the
constructor signature).

Similar fixes probably need to be made to map() and maybe
reduce().

----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2003-01-25 04:47

Message:
Logged In: YES 
user_id=80475

The problem isn't with filter() which correctly calls iter() in 
both cases.

Tuple object have their own iterator which loops over 
elements directly and has no intervening calls to 
__getitem__().

String objects do not define a custom iterator, so iter() 
wraps itself around consecutive calls to __getitem__().

The resolution is to provide string objects with their own 
iterator. As a side benefit, iteration will run just a tiny bit 
faster.  The same applies to unicode objects.

Guido, do you care about this and want me to fix it or 
would you like to close it as "won't fix".


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=665835&group_id=5470


From noreply@sourceforge.net  Mon Feb 10 17:50:09 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Mon, 10 Feb 2003 09:50:09 -0800
Subject: [Python-bugs-list] [ python-Bugs-684022 ] extended slice strangeness
Message-ID: <E18iI4H-0001iC-00@sc8-sf-web3.sourceforge.net>

Bugs item #684022, was opened at 2003-02-10 16:54
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=684022&group_id=5470

Category: Python Interpreter Core
Group: Python 2.3
>Status: Closed
>Resolution: Rejected
Priority: 5
Submitted By: Robert Roy (rjroy)
Assigned to: Michael Hudson (mwh)
Summary: extended slice strangeness

Initial Comment:
I was playing with extended slices and wanted to see if 
it would return a reversed slice. The behaviour was not 
quite what I expected. See the following


Python 2.3a1 (#38, Dec 31 2002, 17:53:59) [MSC 
v.1200 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more 
information.
>>> l = range(12)
>>> l
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11]
## this works as expected
>>> l[0:5:2]
[0, 2, 4]

## this returns empty list ??
>>> l[0:5:-1]
[]

## this works as expected
>>> l[::-1]
[11, 10, 9, 8, 7, 6, 5, 4, 3, 2, 1, 0]

## I would expect this to return same as above
>>> l[0::-1]
[0]

## would expect empty list since l[5:0] == []
>>> l[5:0:-1]
[5, 4, 3, 2, 1]

## would expect [11, 10, 9, 8, 7, 6,5]
>>> l[5::-1]
[5, 4, 3, 2, 1, 0]
>>>


----------------------------------------------------------------------

>Comment By: Michael Hudson (mwh)
Date: 2003-02-10 17:50

Message:
Logged In: YES 
user_id=6656

the interpretation of 

l[start:stop:step]

is similar to that of 

range(start, stop, end)

An omitted an index means the endpoint and what that means
changes with the sign of step.  You have to have this to
make l[::-1] do what you want.

So I'm going to reject this.  Feel free to reopen if you
disagree wildly, but you'll have to talk fast.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-10 17:12

Message:
Logged In: YES 
user_id=33168

I'm not sure what I would expect.  Your suggestions seem
reasonable.  Michael?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=684022&group_id=5470


From noreply@sourceforge.net  Mon Feb 10 17:52:03 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Mon, 10 Feb 2003 09:52:03 -0800
Subject: [Python-bugs-list] [ python-Bugs-665835 ] filter() treatment of str and tuple inconsistent
Message-ID: <E18iI67-0004eW-00@sc8-sf-web1.sourceforge.net>

Bugs item #665835, was opened at 2003-01-10 11:36
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=665835&group_id=5470

Category: Python Interpreter Core
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Walter Dörwald (doerwalter)
Assigned to: Raymond Hettinger (rhettinger)
Summary: filter() treatment of str and tuple inconsistent

Initial Comment:
class tuple2(tuple):
·  def __getitem__(self, index):
·  ·  return 2*tuple.__getitem__(self, index)

class str2(str):
·  def __getitem__(self, index):
·  ·  return chr(ord(str.__getitem__(self, index))+1)

print filter(lambda x: x>1, tuple2((1, 2)))
print filter(lambda x: x>"a", str2("ab"))

this prints:
(2,)
bc

i.e. the overwritten __getitem__ is ignored in the
first case, but honored in the second.


----------------------------------------------------------------------

>Comment By: Raymond Hettinger (rhettinger)
Date: 2003-02-10 12:52

Message:
Logged In: YES 
user_id=80475

I'm still working on fixing the iterators so that __getitem__ 
overrides are recognized by __iter__.

Hope that simplifies your changes.

----------------------------------------------------------------------

Comment By: Walter Dörwald (doerwalter)
Date: 2003-02-10 12:44

Message:
Logged In: YES 
user_id=89016

Checked in as:
Lib/test/test_builtin.py 1.13
Python/bltinmodule.c 2.280


----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-10 10:20

Message:
Logged In: YES 
user_id=6380

Feel free to fix filtertuple() too. Just note that
tp_as_sequence might be NULL, or ...->sq_item might be NULL.
I'm not 100% sure that those can never be NULL for a tuple
subclass.

----------------------------------------------------------------------

Comment By: Walter Dörwald (doerwalter)
Date: 2003-02-10 08:27

Message:
Logged In: YES 
user_id=89016

OK, this has been fixed: function==None and function==lambda x:x
now behave the same (for str and unicode, for tuples it's
still broken, because PyTuple_GetItem() is used. (Checked in
as Python/bltinmodule.c 2.278 and  Lib/test/test_builtin.py
1.12)

Why can't we simply replace PyTuple_GetItem() with
tp_as_sequence->sq_item in filtertuple()?

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-04 16:06

Message:
Logged In: YES 
user_id=6380

So it does. I guess the special shortcut for None should
only be taken
when it's a proper str instance.

----------------------------------------------------------------------

Comment By: Walter Dörwald (doerwalter)
Date: 2003-02-04 15:34

Message:
Logged In: YES 
user_id=89016

The subclass problem has been fixed in:
Python/bltinmodule.c 2.275
Lib/test/test_builtin.py 1.9

But now something strange happens:
---
class badstr(str):
···def __getitem__(self, index):
······return str.__getitem__(self, index).upper()

print filter(None, badstr("abc"))
print filter(lambda x: x, badstr("abc"))
---
This prints
---
abc
ABC
---
although according to the filter docstring they should be
the same.


----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-04 12:33

Message:
Logged In: YES 
user_id=6380

Yes, the special treatment of tuple, str and unicode is
problematic. :-(
I wish filter() had always returned a list for all input types.
But it's too late to change that.

However, I don't think that filter() should ever return a
*subclass* of tuple, str or unicode. Note that slicing a
subclass of these also doesn't return a subclass instance,
unless the subclass specifically overrides __getslice__.

I note that filter() of a tuple *almost* implements what I
think it should, except that if it receives an empty tuple
subclass, it returns it unchanged. The slicing and other
methods (e.g. lower()) have all been modified to make a copy
whose type is the base class; I think filter() should follow
suit.

Similar for filter() of strings and unicode.

----------------------------------------------------------------------

Comment By: Walter Dörwald (doerwalter)
Date: 2003-02-04 12:15

Message:
Logged In: YES 
user_id=89016

OK, the problem of __getitem__ not returning str or unicode
is fixed. Unfortunately the result is rather ugly. With the
following class:

class u(unicode):
   def __getitem__(self, index):
      return u(2*unicode.__getitem__(self, index))

filter neither returns a list nor an u object, but a unicode
object, defeating the whole purpose of the special treatment
of str/unicode. If we remove the special treatment this
problem would go away, furthermore __getitem__ returning
objects that are not str/unicode instances wouldn't be
problem any longer.


----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-03 17:36

Message:
Logged In: YES 
user_id=6380

Walter: if you can fix the bug in your latest message here,
go ahead and check it in. Seems like a case of a missing test.

Raymond: it turns out that the iterator in Python 2.2 has
the same problem with lists -- it special-cases lists. But
for tuples, the iterator uses PySequence_GetItem; the fast
tuple iterator in Python 2.3 introduces the problem for
tuples though.

I actually don't think there would be much disagreement that
this behavior (ignoring __getitem__) is a bug. There may be
disagreement over how important it is to fix it. Personally,
I've generally been on the side of "it needn't be fixed if
it slows down the common case", as long as a workaround
(like overriding __iter__ alongside the __getitem__
override) exists.

But I draw the line at being backwards incompatible with
Python 2.2. There fore I think the tuple iterator (and
probably also the string iterator) needs to be fixed, and I
still think that it would be best if the list iterator were
also fixed. One way to do this would be for the tp_iter
implementation to check whether
self->ob_type->tp_as_sequence->sq_item is not equal to the
list_item function (this is a good check to detect a
__getitem__ override) and then return an instance of the
generic sequence iterator instead of the list-specific iterator.

----------------------------------------------------------------------

Comment By: Walter Dörwald (doerwalter)
Date: 2003-01-27 07:24

Message:
Logged In: YES 
user_id=89016

Another problem with filter() is that filterstring() (and
the new filterunicode()) blindly assume that
tp_as_sequence->sq_item returns a str or unicode object with
len==1. This might fail with str or unicode subclasses:
----
class badstr(str):
   def __getitem__(self, index):
      return 42

s = filter(lambda x: x>=42, badstr("1234"))
print len(s), repr(s)
----
This prints
4 '\x00\x00\x00\x00'

----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2003-01-26 20:13

Message:
Logged In: YES 
user_id=80475

One other thought:  A major reason for implementing 
__iter__ in the first place is that objects were overriding 
__getitem__ and disregarding the index -- the __getitem__ 
interface just didn't make sense for iteration in some 
situations.  __iter__ was supposed to provide enormous 
flexibility in various ways to loop over a collection (inorder, 
preorder, postorder, priorityorder, sortedorder, hashorder, 
randomorder, etc).  Making iter() default to using 
__getitem__ was only supposed to be an expedient for 
backwards compatability.  Always using __getitem__ 
diminishes the flexibility and speed advantages.

Maybe the discussion belongs on python-dev. I'm sure a 
number of people feel strongly one way or the other.  The 
question might as well be addressed head-on before 2.3 
goes out the door. 

----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2003-01-26 19:54

Message:
Logged In: YES 
user_id=80475

I understand.  Ideally, *all* methods would respond to a 
single overridden method, but I think this is just a fact of 
life in object oriented programming.

I can't remember where you gave an example of a 
d.__getitem__() subclass override, but you were careful to 
point out that other methods, like d.get() also needed to 
be overridden so that the modified access applied 
everywhere.  Likewise, __iter__() or any other object 
access method must be assumed to access the underlying 
data structure directly and must be overridden.   For 
instance, creating a dictionary with case insensitive 
lookups entails overriding __getitem__(k), get(k,default), 
and pop(k) -- no one of them can be presumed to inform 
the others.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-01-26 19:17

Message:
Logged In: YES 
user_id=6380

Hm... that means that iter() of *amy* built-in type subclass
overriding __getitem__ bypasses the override, unless the
subclass also overrides __iter__. This sounds like a step in
the wrong direction. I think the built-in iterators should
be aware of subclasses overriding __getitem__ one way or
another. I hadn't realized this when we started the trend of
creating faster iterators for built-in types. :-(

----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2003-01-25 11:45

Message:
Logged In: YES 
user_id=80475

None of the existing iterators (incl dicts, lists, tuples, and 
files) use __getitem__.  Most likely, user defined iterators 
also access the data structure directly (for flexiblity and 
speed). Also, anything that uses PyTuple_GET_ITEM 
bypasses __getitem__.

If string/unicode iterators are added, they should also go
directly to the underlying data; otherwise, there is no point 
to it.

Also, the proposal to change filtertuple(), doesn't solve
inconsistencies within filterstring() which uses __getitem__ 
when there is a function call, but bypasses it when the 
function parameter is Py_None.

I think the right answer is to change filterstring() to use an 
iterator and to implement string/unicode iterators that 
access the data directly (not using __getitem__).

FYI for Tim:  MvL noticed and fixed the unicode vs string 
difference.  His patch, SF #636005, has not been applied 
yet.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-01-25 08:51

Message:
Logged In: YES 
user_id=6380

(But in addition th that, I don't mind having a custom
string iterator -- as long as it calls __getitem__ properly.
Hm, shouldn't the tuple iterator call __getitem__ properly too?)


----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2003-01-25 08:45

Message:
Logged In: YES 
user_id=31435

Just noting that filter() is unique in special-casing the type 
of the input.  It's always been surprising that way, and, 
e.g., filtering a string produces a string, but filtering a 
Unicode string produces a list.

map() and reduce() don't play games like that, and always 
use the iteration protocol to march over their inputs.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-01-25 08:36

Message:
Logged In: YES 
user_id=6380

I don't know which Python sources Raymond has been reading,
but in the sources I've got in front of me, there are
special cases for strings and tuples, and these *don't* use
iter(). It so happens that the tuple special-case calls
PyTuple_GetItem(), which doesn't call your __getitem__,
while the string special-case calls the sq_item slot
function, which (in your case) will be a wrapper that calls
your __getitem__.

A minimal fix would be to only call filtertuple for strict
tuples -- although this changes the output type, but I don't
think one should count on filter() of a tuple subclass
returning a tuple (and it can't be made to return an
instance of the subclass either -- we don't know the
constructor signature).

Similar fixes probably need to be made to map() and maybe
reduce().

----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2003-01-24 22:47

Message:
Logged In: YES 
user_id=80475

The problem isn't with filter() which correctly calls iter() in 
both cases.

Tuple object have their own iterator which loops over 
elements directly and has no intervening calls to 
__getitem__().

String objects do not define a custom iterator, so iter() 
wraps itself around consecutive calls to __getitem__().

The resolution is to provide string objects with their own 
iterator. As a side benefit, iteration will run just a tiny bit 
faster.  The same applies to unicode objects.

Guido, do you care about this and want me to fix it or 
would you like to close it as "won't fix".


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=665835&group_id=5470


From noreply@sourceforge.net  Mon Feb 10 18:11:40 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Mon, 10 Feb 2003 10:11:40 -0800
Subject: [Python-bugs-list] [ python-Bugs-684022 ] extended slice strangeness
Message-ID: <E18iIP6-0005rw-00@sc8-sf-web1.sourceforge.net>

Bugs item #684022, was opened at 2003-02-10 11:54
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=684022&group_id=5470

Category: Python Interpreter Core
Group: Python 2.3
Status: Closed
Resolution: Rejected
Priority: 5
Submitted By: Robert Roy (rjroy)
Assigned to: Michael Hudson (mwh)
Summary: extended slice strangeness

Initial Comment:
I was playing with extended slices and wanted to see if 
it would return a reversed slice. The behaviour was not 
quite what I expected. See the following


Python 2.3a1 (#38, Dec 31 2002, 17:53:59) [MSC 
v.1200 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more 
information.
>>> l = range(12)
>>> l
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11]
## this works as expected
>>> l[0:5:2]
[0, 2, 4]

## this returns empty list ??
>>> l[0:5:-1]
[]

## this works as expected
>>> l[::-1]
[11, 10, 9, 8, 7, 6, 5, 4, 3, 2, 1, 0]

## I would expect this to return same as above
>>> l[0::-1]
[0]

## would expect empty list since l[5:0] == []
>>> l[5:0:-1]
[5, 4, 3, 2, 1]

## would expect [11, 10, 9, 8, 7, 6,5]
>>> l[5::-1]
[5, 4, 3, 2, 1, 0]
>>>


----------------------------------------------------------------------

>Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-10 13:11

Message:
Logged In: YES 
user_id=33168

I don't have a problem with your answer.  I looked through
ref3.tex and didn't see anything in particular about negative
steps.  Should something be added?  If so, is ref3 the
correct place?

There's a comment in libconsts:
  % XXX Someone who understands extended slicing should fill
in here.


----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2003-02-10 12:50

Message:
Logged In: YES 
user_id=6656

the interpretation of 

l[start:stop:step]

is similar to that of 

range(start, stop, end)

An omitted an index means the endpoint and what that means
changes with the sign of step.  You have to have this to
make l[::-1] do what you want.

So I'm going to reject this.  Feel free to reopen if you
disagree wildly, but you'll have to talk fast.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-10 12:12

Message:
Logged In: YES 
user_id=33168

I'm not sure what I would expect.  Your suggestions seem
reasonable.  Michael?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=684022&group_id=5470


From noreply@sourceforge.net  Mon Feb 10 18:19:51 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Mon, 10 Feb 2003 10:19:51 -0800
Subject: [Python-bugs-list] [ python-Bugs-684022 ] extended slice strangeness
Message-ID: <E18iIX1-0005iV-00@sc8-sf-web2.sourceforge.net>

Bugs item #684022, was opened at 2003-02-10 16:54
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=684022&group_id=5470

Category: Python Interpreter Core
Group: Python 2.3
Status: Closed
Resolution: Rejected
Priority: 5
Submitted By: Robert Roy (rjroy)
Assigned to: Michael Hudson (mwh)
Summary: extended slice strangeness

Initial Comment:
I was playing with extended slices and wanted to see if 
it would return a reversed slice. The behaviour was not 
quite what I expected. See the following


Python 2.3a1 (#38, Dec 31 2002, 17:53:59) [MSC 
v.1200 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more 
information.
>>> l = range(12)
>>> l
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11]
## this works as expected
>>> l[0:5:2]
[0, 2, 4]

## this returns empty list ??
>>> l[0:5:-1]
[]

## this works as expected
>>> l[::-1]
[11, 10, 9, 8, 7, 6, 5, 4, 3, 2, 1, 0]

## I would expect this to return same as above
>>> l[0::-1]
[0]

## would expect empty list since l[5:0] == []
>>> l[5:0:-1]
[5, 4, 3, 2, 1]

## would expect [11, 10, 9, 8, 7, 6,5]
>>> l[5::-1]
[5, 4, 3, 2, 1, 0]
>>>


----------------------------------------------------------------------

>Comment By: Michael Hudson (mwh)
Date: 2003-02-10 18:19

Message:
Logged In: YES 
user_id=6656

Hmm.  The thing is, it's more or less up to the types as to
what they do with the indices.  ref3.tex refers to them as
start:stop:step and that seems good enough for me.

The libconsts thing is about Ellipsis; I know nothing about
that except that it's to do with multi-rank "sequences". 
Ask the numpy folks, if you care.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-10 18:11

Message:
Logged In: YES 
user_id=33168

I don't have a problem with your answer.  I looked through
ref3.tex and didn't see anything in particular about negative
steps.  Should something be added?  If so, is ref3 the
correct place?

There's a comment in libconsts:
  % XXX Someone who understands extended slicing should fill
in here.


----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2003-02-10 17:50

Message:
Logged In: YES 
user_id=6656

the interpretation of 

l[start:stop:step]

is similar to that of 

range(start, stop, end)

An omitted an index means the endpoint and what that means
changes with the sign of step.  You have to have this to
make l[::-1] do what you want.

So I'm going to reject this.  Feel free to reopen if you
disagree wildly, but you'll have to talk fast.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-10 17:12

Message:
Logged In: YES 
user_id=33168

I'm not sure what I would expect.  Your suggestions seem
reasonable.  Michael?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=684022&group_id=5470


From noreply@sourceforge.net  Mon Feb 10 17:12:24 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Mon, 10 Feb 2003 09:12:24 -0800
Subject: [Python-bugs-list] [ python-Bugs-684022 ] extended slice strangeness
Message-ID: <E18iHTk-0002Hp-00@sc8-sf-web1.sourceforge.net>

Bugs item #684022, was opened at 2003-02-10 11:54
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=684022&group_id=5470

Category: Python Interpreter Core
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Robert Roy (rjroy)
>Assigned to: Michael Hudson (mwh)
Summary: extended slice strangeness

Initial Comment:
I was playing with extended slices and wanted to see if 
it would return a reversed slice. The behaviour was not 
quite what I expected. See the following


Python 2.3a1 (#38, Dec 31 2002, 17:53:59) [MSC 
v.1200 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more 
information.
>>> l = range(12)
>>> l
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11]
## this works as expected
>>> l[0:5:2]
[0, 2, 4]

## this returns empty list ??
>>> l[0:5:-1]
[]

## this works as expected
>>> l[::-1]
[11, 10, 9, 8, 7, 6, 5, 4, 3, 2, 1, 0]

## I would expect this to return same as above
>>> l[0::-1]
[0]

## would expect empty list since l[5:0] == []
>>> l[5:0:-1]
[5, 4, 3, 2, 1]

## would expect [11, 10, 9, 8, 7, 6,5]
>>> l[5::-1]
[5, 4, 3, 2, 1, 0]
>>>


----------------------------------------------------------------------

>Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-10 12:12

Message:
Logged In: YES 
user_id=33168

I'm not sure what I would expect.  Your suggestions seem
reasonable.  Michael?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=684022&group_id=5470


From noreply@sourceforge.net  Mon Feb 10 20:37:10 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Mon, 10 Feb 2003 12:37:10 -0800
Subject: [Python-bugs-list] [ python-Bugs-501716 ] "es#" parser marker leaks memory
Message-ID: <E18iKfu-0000Xu-00@sc8-sf-web3.sourceforge.net>

Bugs item #501716, was opened at 2002-01-10 10:39
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=501716&group_id=5470

Category: Unicode
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: M.-A. Lemburg (lemburg)
Assigned to: M.-A. Lemburg (lemburg)
Summary: "es#" parser marker leaks memory

Initial Comment:
... if subsequent parsing fails, a buffer which was possibly 
allocated by "es#" is not freed.


----------------------------------------------------------------------

Comment By: Michael Stone (mbrierst)
Date: 2003-02-10 20:37

Message:
Logged In: YES 
user_id=670441

Ahhhh... thank you for your reply.  I understand now.
I have a patch that gets rid of this problem by putting allocated
memory into a list of CObjects which are freed when exceptions occur.
The list is only created when necessary, to hopefully preserve efficiency
in most cases.  It has to be passed around, so several function calls had
to be changed, and to do the cleanup on return I made a macro CLEANRETURN.

This patch also fixes a memory leak when the buffer_len pointer is NULL.
This isn't a big deal since the documentation says never to pass in a NULL
buffer_len.

And it fixes another leak when a string with encoded NULL's is passed in
(again not a big deal)

If you want, I have C code that can be called from python to demonstrate
any of these leaks.  Let me know what you think of the patch/approach.

Index: Python/getargs.c
===================================================================
RCS file: /cvsroot/python/python/dist/src/Python/getargs.c,v
retrieving revision 2.95
diff -c -r2.95 getargs.c
*** Python/getargs.c	24 Jan 2003 22:15:21 -0000	2.95
--- Python/getargs.c	10 Feb 2003 20:28:52 -0000
***************
*** 17,26 ****
  static int vgetargs1(PyObject *, char *, va_list *, int);
  static void seterror(int, char *, int *, char *, char *);
  static char *convertitem(PyObject *, char **, va_list *, int *, char *, 
! 			 size_t);
  static char *converttuple(PyObject *, char **, va_list *,
! 			  int *, char *, size_t, int);
! static char *convertsimple(PyObject *, char **, va_list *, char *, size_t);
  static int convertbuffer(PyObject *, void **p, char **);
  
  static int vgetargskeywords(PyObject *, PyObject *,
--- 17,27 ----
  static int vgetargs1(PyObject *, char *, va_list *, int);
  static void seterror(int, char *, int *, char *, char *);
  static char *convertitem(PyObject *, char **, va_list *, int *, char *, 
! 			 size_t, PyObject **, int *);
  static char *converttuple(PyObject *, char **, va_list *,
! 			  int *, char *, size_t, int, PyObject **, int *);
! static char *convertsimple(PyObject *, char **, va_list *, char *,
! 			   size_t, PyObject **, int *);
  static int convertbuffer(PyObject *, void **p, char **);
  
  static int vgetargskeywords(PyObject *, PyObject *,
***************
*** 72,77 ****
--- 73,120 ----
  }
  
  
+ /* Handle cleanup of allocated memory in case of exception */
+ 
+ static void
+ docleanup(void *ptr, void *dofree)
+ {
+ 	if (*(int *)dofree)
+ 		PyMem_FREE(ptr);
+ }
+ 
+ static int
+ addcleanup(void *ptr, PyObject **freelist, int *dofree)
+ {
+ 	PyObject *cobj;
+ 	if (!*freelist) {
+ 		*freelist = PyList_New(0);
+ 		if (!*freelist) {
+ 			PyMem_FREE(ptr);
+ 			return -1;
+ 		}
+ 	}
+ 	cobj = PyCObject_FromVoidPtrAndDesc(ptr, dofree, docleanup);
+ 	if (!cobj) {
+ 		PyMem_FREE(ptr);
+ 		return -1;
+ 	}
+ 	if(PyList_Append(*freelist, cobj)) {
+ 		*dofree = 1;
+ 		Py_DECREF(cobj);
+ 		return -1;
+ 	}
+         Py_DECREF(cobj);
+ 	return 0;
+ }
+ 
+ #define CLEANRETURN(retval, freelist, dofree) do	\
+ {							\
+ 	dofree = ((retval) == 0);			\
+ 	Py_XDECREF(freelist);				\
+ 	return (retval);				\
+ } while(0)
+ 
+ 
  static int
  vgetargs1(PyObject *args, char *format, va_list *p_va, int compat)
  {
***************
*** 86,91 ****
--- 129,136 ----
  	char *formatsave = format;
  	int i, len;
  	char *msg;
+ 	PyObject *freelist = NULL;
+ 	int dofree = 0;
  	
  	assert(compat || (args != (PyObject*)NULL));
  
***************
*** 157,167 ****
  				return 0;
  			}
  			msg = convertitem(args, &format, p_va, levels, msgbuf,
! 					  sizeof(msgbuf));
  			if (msg == NULL)
! 				return 1;
  			seterror(levels[0], msg, levels+1, fname, message);
! 			return 0;
  		}
  		else {
  			PyErr_SetString(PyExc_SystemError,
--- 202,212 ----
  				return 0;
  			}
  			msg = convertitem(args, &format, p_va, levels, msgbuf,
! 					  sizeof(msgbuf), &freelist, &dofree);
  			if (msg == NULL)
! 				CLEANRETURN(1, freelist, dofree);
  			seterror(levels[0], msg, levels+1, fname, message);
! 			CLEANRETURN(0, freelist, dofree);
  		}
  		else {
  			PyErr_SetString(PyExc_SystemError,
***************
*** 200,209 ****
  		if (*format == '|')
  			format++;
  		msg = convertitem(PyTuple_GET_ITEM(args, i), &format, p_va,
! 				  levels, msgbuf, sizeof(msgbuf));
  		if (msg) {
  			seterror(i+1, msg, levels, fname, message);
! 			return 0;
  		}
  	}
  
--- 245,254 ----
  		if (*format == '|')
  			format++;
  		msg = convertitem(PyTuple_GET_ITEM(args, i), &format, p_va,
! 				  levels, msgbuf, sizeof(msgbuf), &freelist, &dofree);
  		if (msg) {
  			seterror(i+1, msg, levels, fname, message);
! 			CLEANRETURN(0, freelist, dofree);
  		}
  	}
  
***************
*** 212,221 ****
  	    *format != '|' && *format != ':' && *format != ';') {
  		PyErr_Format(PyExc_SystemError,
  			     "bad format string: %.200s", formatsave);
! 		return 0;
  	}
  	
! 	return 1;
  }
  
  
--- 257,266 ----
  	    *format != '|' && *format != ':' && *format != ';') {
  		PyErr_Format(PyExc_SystemError,
  			     "bad format string: %.200s", formatsave);
! 		CLEANRETURN(0, freelist, dofree);
  	}
  	
! 	CLEANRETURN(1, freelist, dofree);
  }
  
  
***************
*** 277,283 ****
  
  static char *
  converttuple(PyObject *arg, char **p_format, va_list *p_va, int *levels,
! 	     char *msgbuf, size_t bufsize, int toplevel)
  {
  	int level = 0;
  	int n = 0;
--- 322,329 ----
  
  static char *
  converttuple(PyObject *arg, char **p_format, va_list *p_va, int *levels,
! 	     char *msgbuf, size_t bufsize, int toplevel,
! 	     PyObject **freelist, int *dofree)
  {
  	int level = 0;
  	int n = 0;
***************
*** 327,333 ****
  		PyObject *item;
  		item = PySequence_GetItem(arg, i);
  		msg = convertitem(item, &format, p_va, levels+1, msgbuf,
! 				  bufsize);
  		/* PySequence_GetItem calls tp->sq_item, which INCREFs */
  		Py_XDECREF(item);
  		if (msg != NULL) {
--- 373,379 ----
  		PyObject *item;
  		item = PySequence_GetItem(arg, i);
  		msg = convertitem(item, &format, p_va, levels+1, msgbuf,
! 				  bufsize, freelist, dofree);
  		/* PySequence_GetItem calls tp->sq_item, which INCREFs */
  		Py_XDECREF(item);
  		if (msg != NULL) {
***************
*** 345,351 ****
  
  static char *
  convertitem(PyObject *arg, char **p_format, va_list *p_va, int *levels,
! 	    char *msgbuf, size_t bufsize)
  {
  	char *msg;
  	char *format = *p_format;
--- 391,397 ----
  
  static char *
  convertitem(PyObject *arg, char **p_format, va_list *p_va, int *levels,
! 	    char *msgbuf, size_t bufsize, PyObject **freelist, int *dofree)
  {
  	char *msg;
  	char *format = *p_format;
***************
*** 353,364 ****
  	if (*format == '(' /* ')' */) {
  		format++;
  		msg = converttuple(arg, &format, p_va, levels, msgbuf, 
! 				   bufsize, 0);
  		if (msg == NULL)
  			format++;
  	}
  	else {
! 		msg = convertsimple(arg, &format, p_va, msgbuf, bufsize);
  		if (msg != NULL)
  			levels[0] = 0;
  	}
--- 399,411 ----
  	if (*format == '(' /* ')' */) {
  		format++;
  		msg = converttuple(arg, &format, p_va, levels, msgbuf, 
! 				   bufsize, 0, freelist, dofree);
  		if (msg == NULL)
  			format++;
  	}
  	else {
! 		msg = convertsimple(arg, &format, p_va, msgbuf, bufsize,
! 				    freelist, dofree);
  		if (msg != NULL)
  			levels[0] = 0;
  	}
***************
*** 396,402 ****
  
  static char *
  convertsimple(PyObject *arg, char **p_format, va_list *p_va, char *msgbuf,
! 	      size_t bufsize)
  {
  	char *format = *p_format;
  	char c = *format++;
--- 443,449 ----
  
  static char *
  convertsimple(PyObject *arg, char **p_format, va_list *p_va, char *msgbuf,
! 	      size_t bufsize, PyObject **freelist, int *dofree)
  {
  	char *format = *p_format;
  	char c = *format++;
***************
*** 801,810 ****
  			int *buffer_len = va_arg(*p_va, int *);
  
  			format++;
! 			if (buffer_len == NULL)
  				return converterr(
  					"(buffer_len is NULL)",
  					arg, msgbuf, bufsize);
  			if (*buffer == NULL) {
  				*buffer = PyMem_NEW(char, size + 1);
  				if (*buffer == NULL) {
--- 848,859 ----
  			int *buffer_len = va_arg(*p_va, int *);
  
  			format++;
! 			if (buffer_len == NULL) {
! 				Py_DECREF(s);
  				return converterr(
  					"(buffer_len is NULL)",
  					arg, msgbuf, bufsize);
+ 			}
  			if (*buffer == NULL) {
  				*buffer = PyMem_NEW(char, size + 1);
  				if (*buffer == NULL) {
***************
*** 813,818 ****
--- 862,873 ----
  						"(memory error)",
  						arg, msgbuf, bufsize);
  				}
+ 				if(addcleanup(*buffer, freelist, dofree)) {
+ 					Py_DECREF(s);
+ 					return converterr(
+ 						"(cleanup problem)",
+ 						arg, msgbuf, bufsize);
+ 				}
  			} else {
  				if (size + 1 > *buffer_len) {
  					Py_DECREF(s);
***************
*** 839,854 ****
  			   PyMem_Free()ing it after usage
  
  			*/
! 			if ((int)strlen(PyString_AS_STRING(s)) != size)
  				return converterr(
  					"(encoded string without NULL bytes)",
  					arg, msgbuf, bufsize);
  			*buffer = PyMem_NEW(char, size + 1);
  			if (*buffer == NULL) {
  				Py_DECREF(s);
  				return converterr("(memory error)",
  						  arg, msgbuf, bufsize);
  			}
  			memcpy(*buffer,
  			       PyString_AS_STRING(s),
  			       size + 1);
--- 894,916 ----
  			   PyMem_Free()ing it after usage
  
  			*/
! 			if ((int)strlen(PyString_AS_STRING(s)) != size) {
! 				Py_DECREF(s);
  				return converterr(
  					"(encoded string without NULL bytes)",
  					arg, msgbuf, bufsize);
+ 			}
  			*buffer = PyMem_NEW(char, size + 1);
  			if (*buffer == NULL) {
  				Py_DECREF(s);
  				return converterr("(memory error)",
  						  arg, msgbuf, bufsize);
  			}
+ 			if(addcleanup(*buffer, freelist, dofree)) {
+ 				Py_DECREF(s);
+ 				return converterr("(cleanup problem)",
+ 						arg, msgbuf, bufsize);
+ 			}
  			memcpy(*buffer,
  			       PyString_AS_STRING(s),
  			       size + 1);
***************
*** 1068,1073 ****
--- 1130,1137 ----
  	char *formatsave;
  	int i, len, nargs, nkeywords;
  	char *msg, **p;
+ 	PyObject *freelist = NULL;
+ 	int dofree = 0;
  
  	assert(args != NULL && PyTuple_Check(args));
  	assert(keywords == NULL || PyDict_Check(keywords));
***************
*** 1192,1207 ****
  		if (*format == '|')
  			format++;
  		msg = convertitem(PyTuple_GET_ITEM(args, i), &format, p_va,
! 				 levels, msgbuf, sizeof(msgbuf));
  		if (msg) {
  			seterror(i+1, msg, levels, fname, message);
! 			return 0;
  		}
  	}
  
  	/* handle no keyword parameters in call */	
  	if (nkeywords == 0)
! 		return 1; 
  
  	/* convert the keyword arguments; this uses the format 
  	   string where it was left after processing args */
--- 1256,1272 ----
  		if (*format == '|')
  			format++;
  		msg = convertitem(PyTuple_GET_ITEM(args, i), &format, p_va,
! 				  levels, msgbuf, sizeof(msgbuf), &freelist,
! 				  &dofree);
  		if (msg) {
  			seterror(i+1, msg, levels, fname, message);
! 			CLEANRETURN(0, freelist, dofree);
  		}
  	}
  
  	/* handle no keyword parameters in call */	
  	if (nkeywords == 0)
! 		CLEANRETURN(1, freelist, dofree);
  
  	/* convert the keyword arguments; this uses the format 
  	   string where it was left after processing args */
***************
*** 1213,1235 ****
  		if (item != NULL) {
  			Py_INCREF(item);
  			msg = convertitem(item, &format, p_va, levels, msgbuf,
! 					  sizeof(msgbuf));
  			Py_DECREF(item);
  			if (msg) {
  				seterror(i+1, msg, levels, fname, message);
! 				return 0;
  			}
  			--nkeywords;
  			if (nkeywords == 0)
  				break;
  		}
  		else if (PyErr_Occurred())
! 			return 0;
  		else {
  			msg = skipitem(&format, p_va);
  			if (msg) {
  				seterror(i+1, msg, levels, fname, message);
! 				return 0;
  			}
  		}
  	}
--- 1278,1300 ----
  		if (item != NULL) {
  			Py_INCREF(item);
  			msg = convertitem(item, &format, p_va, levels, msgbuf,
! 					  sizeof(msgbuf), &freelist, &dofree);
  			Py_DECREF(item);
  			if (msg) {
  				seterror(i+1, msg, levels, fname, message);
! 				CLEANRETURN(0, freelist, dofree);
  			}
  			--nkeywords;
  			if (nkeywords == 0)
  				break;
  		}
  		else if (PyErr_Occurred())
! 			CLEANRETURN(0, freelist, dofree);
  		else {
  			msg = skipitem(&format, p_va);
  			if (msg) {
  				seterror(i+1, msg, levels, fname, message);
! 				CLEANRETURN(0, freelist, dofree);
  			}
  		}
  	}
***************
*** 1244,1250 ****
  			if (!PyString_Check(key)) {
  				PyErr_SetString(PyExc_TypeError, 
  					        "keywords must be strings");
! 				return 0;
  			}
  			ks = PyString_AsString(key);
  			for (i = 0; i < max; i++) {
--- 1309,1315 ----
  			if (!PyString_Check(key)) {
  				PyErr_SetString(PyExc_TypeError, 
  					        "keywords must be strings");
! 				CLEANRETURN(0, freelist, dofree);
  			}
  			ks = PyString_AsString(key);
  			for (i = 0; i < max; i++) {
***************
*** 1258,1269 ****
  					     "'%s' is an invalid keyword "
  					     "argument for this function",
  					     ks);
! 				return 0;
  			}
  		}
  	}
  
! 	return 1;
  }
  
  
--- 1323,1334 ----
  					     "'%s' is an invalid keyword "
  					     "argument for this function",
  					     ks);
! 				CLEANRETURN(0, freelist, dofree);
  			}
  		}
  	}
  
! 	CLEANRETURN(1, freelist, dofree);
  }
  
  


----------------------------------------------------------------------

Comment By: M.-A. Lemburg (lemburg)
Date: 2003-02-08 14:29

Message:
Logged In: YES 
user_id=38388

What I meant was that in the case of a parser marker
string like "es#iii" if the last "i" fails to convert, then the
buffer used for "es#" won't be freed up again.

----------------------------------------------------------------------

Comment By: Michael Stone (mbrierst)
Date: 2003-02-07 20:20

Message:
Logged In: YES 
user_id=670441

In what case does this happen?  The only case
I can see is if the string is successfully encoded,
but the encoding results in a non-null terminated 
string.  That might cause problems, otherwise it
looks okay?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=501716&group_id=5470


From noreply@sourceforge.net  Mon Feb 10 22:16:13 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Mon, 10 Feb 2003 14:16:13 -0800
Subject: [Python-bugs-list] [ python-Bugs-683726 ] METH_CLASS allows binding to the wrong class
Message-ID: <E18iMDl-00056G-00@sc8-sf-web3.sourceforge.net>

Bugs item #683726, was opened at 2003-02-09 22:21
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=683726&group_id=5470

Category: Python Interpreter Core
Group: Python 2.3
Status: Open
Resolution: None
Priority: 7
Submitted By: Greg Chapman (glchapman)
Assigned to: Guido van Rossum (gvanrossum)
Summary: METH_CLASS allows binding to the wrong class

Initial Comment:
I just noticed that this doesn't raise an exception:

>>> cmeth = dict.__dict__['fromkeys'].__get__(None, 
list)
>>> cmeth
<built-in method fromkeys of type object at 
0x1E0BB2E0>
>>> cmeth.__self__
<type 'list'>

Looking over the various METH_CLASS 
implementations, it doesn't look like they depend on the 
first (cls) parameter being of any particular type, but it 
might be nice to flag something like the above with an 
exception.

Also, there's this:

>>> cmeth2 = dict.__dict__['fromkeys'].__get__(None, 
42)
>>> cmeth2
<built-in method fromkeys of int object at 0x0074FD90>
>>> print cmeth2.__self__
42

Above, 42 gets passed through to classmethod_get in 
the type parameter, despite the fact that this parameter 
is typed as a PyTypeObject *. Perhpas wrap_descr_get 
should check that its second parameter (if it gets one) is 
a type?
 

----------------------------------------------------------------------

>Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-10 17:16

Message:
Logged In: YES 
user_id=6380

Looking into this now...


----------------------------------------------------------------------

Comment By: Greg Chapman (glchapman)
Date: 2003-02-10 11:59

Message:
Logged In: YES 
user_id=86307

I'm not entirely sure what the desired semantics for 
classmethod_get should be, but it seems to me they should 
be something like the following:

1) descr.__get__(None)
   descr.__get__(None, None)

This case would construct a PyCFunction using descr's 
d_type.  (What happens if both obj and type are NULL (as 
opposed to None)?  I think that should be a BadInternalCall, 
but perhaps it should also return a function with d_type.)

2) descr.__get__(notNone)
    descr.__get__(notNone, anything)

I think this case should be handled as for other descriptors.  
notNone would be checked to see if it is an instance of 
d_type; if so, a PyCFunction would be constructed using 
type(notNone).

3) descr.__get__(None, notNone)

Here notNone would be checked to ensure it is a type object, 
and then checked to see if it is a subtype of d_type.  If it 
passes these checks, then a PyCFunction is constructed 
using notNone.  (This branch would also handle the case 
where the obj == NULL in classmethod_get).

Alternatively, if it is always safe to assume that obj == NULL 
means that classmethod_get was called by the core library, 
and so the type parameter is correct, there could be a 0'th 
branch for obj == NULL which returns a PyCFunction 
constructed using type.


----------------------------------------------------------------------

Comment By: Greg Chapman (glchapman)
Date: 2003-02-09 22:38

Message:
Logged In: YES 
user_id=86307

I just realized that the above can be used to crash python, so 
I believe some sort of checking has to be added to 
classmethod_get:

>>> cmeth = dict.__dict__['fromkeys'].__get__(None)
>>> cmeth(4)

(above causes and access violation because the cls object 
passed to the fromkeys method is NULL).


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=683726&group_id=5470


From noreply@sourceforge.net  Tue Feb 11 06:25:10 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Mon, 10 Feb 2003 22:25:10 -0800
Subject: [Python-bugs-list] [ python-Bugs-635929 ] 2.2.2 build on Solaris
Message-ID: <E18iTqw-0001JQ-00@sc8-sf-web1.sourceforge.net>

Bugs item #635929, was opened at 2002-11-09 09:40
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=635929&group_id=5470

Category: Build
Group: Platform-specific
Status: Closed
Resolution: Works For Me
Priority: 5
Submitted By: Guido van Rossum (gvanrossum)
Assigned to: Nobody/Anonymous (nobody)
Summary: 2.2.2 build on Solaris

Initial Comment:
Attached is a message that concludes an exchange
between David LeVine and me. He built Python using the
gcc 3.2 from sunfreeware.com.

Running ./python, he got this error:

ld.so.1: ./python: fatal: libstdc++.so.5: open failed:
No such file or directory

I suggested adding a -R somewhere and he got it to work
by patching configure. Perhaps a real fix can be made
out of this?


----------------------------------------------------------------------

Comment By: Ben Taylor (h1ghlander)
Date: 2003-02-11 01:25

Message:
Logged In: YES 
user_id=709528

Since you can determine the location of libstdc++.so by invoking
g++ --print-file-name libstdc++.so, you can calculate what
the linker directory is, and therefore what the library path
flags.

Here's my patch:

--- tmp/Python-2.2.2/configure	2002-10-10 11:26:41.000000000
-0400
+++ work/Python-2.2.2/configure	2003-02-11
00:10:36.084484000 -0500
@@ -3310,6 +3310,14 @@
 		    if $CC -Xlinker --help 2>&1 | grep export-dynamic
>/dev/null 		    then
 			LINKFORSHARED="-Xlinker --export-dynamic"
+			LIBSTDCPLUSPLUSPATH=`c++ --print-file-name
libstdc++.so`			LIBSTDCPLUSPLUSDIR="`/usr/bin/dirname
$LIBSTDCPLUSPLUSPATH`"
+			STDCPLUSLIBPATH="`cd $LIBSTDPLUSPLUSDIR;pwd`"
+			LINKFORSHARED="$LINKFORSHARED -R$STDCPLUSLIBPATH
-L$STDCPLUSLIBPATH"
+		    else
+			LIBSTDCPLUSPLUSPATH=`c++ --print-file-name
libstdc++.so`			LIBSTDCPLUSPLUSDIR="`/usr/bin/dirname
$LIBSTDCPLUSPLUSPATH`"
+			STDCPLUSLIBPATH="`cd $LIBSTDCPLUSPLUSDIR;pwd`"
+			LINKFORSHARED="-R$STDCPLUSLIBPATH -L$STDCPLUSLIBPATH"
+
 		    fi;;
 		  esac;;
 	esac


----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2002-11-15 02:28

Message:
Logged In: YES 
user_id=21627

Ok, added a note to README 1.136.4.13 and 1.158.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2002-11-14 18:23

Message:
Logged In: YES 
user_id=6380

Feel free to close it then. It seemed obscure to me at best.
Maybe a note can be added to README with the solution?

----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2002-11-09 13:02

Message:
Logged In: YES 
user_id=21627

I'm not sure that this should be fixed in Python, there are
several complications:

1. It is not certain that python is linked with libstdc++.
If not, you 
    only need to worry about libgcc_s.so, if gcc is gcc 3.2.

2. It is not certain whether /usr/local is the right prefix; 
    gcc might be installed with a different prefix.

3. It is not certain whether libstdc++ is installed in
<gccprefix>/lib, gcc also supports installing it into
<gccprefix>/lib/gcc-lib/<arch>/<version>

4. Randomly adding -R options might shoot back, since python
may now pick up the wrong libraries, for libraries used in
extension modules.

We usually fix this in the gcc installation, not in all
affected applications, by adding proper -R options into the
installed specs file.

I would rather "fix" this by pointing out that gcc 3 users
will have to set LD_RUN_PATH on Solaris, to include the
directories containing libgcc_s.so.1, and possibly
libstdc++.so.xy

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=635929&group_id=5470


From noreply@sourceforge.net  Tue Feb 11 12:26:14 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 11 Feb 2003 04:26:14 -0800
Subject: [Python-bugs-list] [ python-Feature Requests-684542 ] dict setdefault lazy evaluation
Message-ID: <E18iZUM-0007SE-00@sc8-sf-web1.sourceforge.net>

Feature Requests item #684542, was opened at 2003-02-11 13:26
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=684542&group_id=5470

Category: Python Interpreter Core
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Jesús Cea Avión (jcea)
Assigned to: Nobody/Anonymous (nobody)
Summary: dict setdefault lazy evaluation

Initial Comment:
Reading "python cookbook" I reach a very intelligent
dictionary construction: to use dictionary "setdefault"
to avoid a "get" or a "try...except".

Nevertheless current "setdefault" behaviour is
questionable, since it evaluates the default value when
it is not necessary. In my mind, "setdefault" should
evaluate the default value when it need to use it, not
ALWAYS. Example:

>>> a={}
>>> def b() :
...   print "hi!"
...   return 1
... 
>>> a.setdefault("hola","hello")
hello
>>> print a
{'hola': 'hello'}
>>> a.setdefault("hola",b())
hi!
'hello'

In the example, the "setdefault" evaluation is not
necessary since 'a["hola"]' exists. But the default
function is called, although its return value will be
ignored.

This behaviour is not intuitive and I think it should
be changed or, at least, documented. The change would
be very welcome when the "setdefault" default code is
expensive, like my application. In any case, the change
would be ALWAYS more efficient, even in the tipical
"[]" case.

Perhaps a "form __future__ import setdefaultNG" would
be necessary to help to update legacy code?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=684542&group_id=5470


From noreply@sourceforge.net  Tue Feb 11 14:27:44 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 11 Feb 2003 06:27:44 -0800
Subject: [Python-bugs-list] [ python-Bugs-680429 ] __module__ broken for extension classes
Message-ID: <E18ibNw-0007FB-00@sc8-sf-web3.sourceforge.net>

Bugs item #680429, was opened at 2003-02-04 19:55
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=680429&group_id=5470

Category: Documentation
Group: Python 2.3
>Status: Closed
>Resolution: Fixed
Priority: 5
Submitted By: Ralf W. Grosse-Kunstleve (rwgk)
Assigned to: Michael Hudson (mwh)
Summary: __module__ broken for extension classes

Initial Comment:
We are having problems using Boost.Python with Python 
2.3a1 because
under some circumstances the result of __module__ is 
different compared
to earlier Python versions:

Python 2.2.1 (#2, Jun 17 2002, 12:06:51) 
[GCC 2.96 20000731 (Red Hat Linux 7.3 2.96-110)] on 
linux2
Type "help", "copyright", "credits" or "license" for more 
information.
>>> import simple
>>> simple.empty.__module__
'simple'
>>> 

Python 2.3a1 (#1, Jan  6 2003, 14:17:56) 
[GCC 2.96 20000731 (Red Hat Linux 7.3 2.96-110)] on 
linux2
Type "help", "copyright", "credits" or "license" for more 
information.
>>> import simple
>>> simple.empty.__module__
'__main__'
>>> 

Because of this we can no longer pickle our extension 
classes.
For your reference the code for the simple module is 
attached.
This is using Boost release 1.29.0 (www.boost.org).

We have done some debugging. Boost.Python's internal 
idea of the module
associated with an extension class is still correct even when 
using
Python 2.3a1. David Abrahams (main Boost.Python author) 
is telling me
that he "changed Boost.Python to work the way Guido 
suggested before
2.2.2." Therefore we suspect that the __module__ problem 
is due to
a change/bug in Python 2.3a1.

Ralf


#include <boost/python/module.hpp>
#include <boost/python/class.hpp>

namespace sandbx { namespace {

  struct empty {};

  void init_module()
  {
    using namespace boost::python;
    class_<empty>("empty");
  }

}} // namespace sandbx::<anonymous>

BOOST_PYTHON_MODULE(simple)
{
  sandbx::init_module();
}


----------------------------------------------------------------------

>Comment By: Michael Hudson (mwh)
Date: 2003-02-11 14:27

Message:
Logged In: YES 
user_id=6656

OK, I hope this is adequately documented in

revision 1.21 of Doc/api/newtypes.tex
revision 1.118 of Doc/whatsnew/whatsnew23.tex

Closing.

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2003-02-07 17:48

Message:
Logged In: YES 
user_id=6656

No problem.  Glad it got sorted.

I'll leave this open, but mark it as a doc bug -- there
should probably be something in whatsnew about this, and
probably somewhere in the reference docs, but I've no idea
where.

----------------------------------------------------------------------

Comment By: Ralf W. Grosse-Kunstleve (rwgk)
Date: 2003-02-07 17:44

Message:
Logged In: YES 
user_id=71407

> In summary, if you poke a '__module__' key into the
> boost::python::dict object you pass to the call of the
> metatype and change the first argument to just 'name', I
> think you'll be set.

Yes, that works!

> I pretty sure this will also work with 2.2.x, but if I were
> you I'd check.

I've tested with Python 2.2, 2.2.1, 2.2.2 and 2.3a1. The same
code works perfectly with any of these releases.

Thank you very much for your help!

Ralf


----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2003-02-07 11:29

Message:
Logged In: YES 
user_id=6656

Ah, I start to remember the details now.

In 2.2, if tp_name contained a period, everything to the
left of the period was __module__, that to the right __name__.

However, at some point (maybe even by 2.3) we want

class X:
    class Y:
        pass
print X.Y.__name__

to print 'X.Y' (it prints 'Y' today).  Also we made __name__
assignable for 2.3 and this behaviour caused confusion.

So we made things so that the '__module__' entry in the
type's dict always wins.

In summary, if you poke a '__module__' key into the
boost::python::dict object you pass to the call of the
metatype and change the first argument to just 'name', I
think you'll be set.

I pretty sure this will also work with 2.2.x, but if I were
you I'd check.

HTH!

----------------------------------------------------------------------

Comment By: Ralf W. Grosse-Kunstleve (rwgk)
Date: 2003-02-06 18:36

Message:
Logged In: YES 
user_id=71407

> In 2.2.x, how are you telling Python that __module__ should
> be "simple"?  By setting tp_name to "simple.empty"?  What's
> simple.__name__?

Python 2.3a1 (#2, Feb  5 2003, 09:39:30) 
[GCC 2.96 20000731 (Red Hat Linux 7.3 2.96-110)] on linux2
Type "help", "copyright", "credits" or "license" for more 
information.
>>> import sandbx.simple
>>> sandbx.simple.__name__
'sandbx.simple'
>>> sandbx.simple.empty.__name__
'sandbx.simple.empty'
>>> sandbx.simple.empty.__module__
IS HEAPTYPE
type->tp_dict, __module__ = __main__
'__main__'
>>> 

> I think I know how __module__ is getting there, it's the
> chunk of code currently at typeobject.c:1750.

Yes, that's right. I've verified this by adding more print statements.

> Can you point me to the bit of the boost source that creates
> the typeobject?

Here is where it happens (file 
boost/libs/python/src/object/class.cpp):

object result = object(class_metatype())(module_prefix() + name, 
bases, d);

With print statements I've verified that module_prefix() + name is
correctly passed in as "sandbx.simple.empty" .

I am attaching the file class.cpp. Could it be that we have to
update our class_metatype_object?

Thanks!
        Ralf


----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2003-02-06 11:22

Message:
Logged In: YES 
user_id=6656

OK, thanks for the info.  More questions, I'm afraid:

In 2.2.x, how are you telling Python that __module__ should
be "simple"?  By setting tp_name to "simple.empty"?  What's
simple.__name__?

I think I know how __module__ is getting there, it's the
chunk of code currently at typeobject.c:1750.

Can you point me to the bit of the boost source that creates
the typeobject?  'cept sf's just fallen off the net.

----------------------------------------------------------------------

Comment By: Ralf W. Grosse-Kunstleve (rwgk)
Date: 2003-02-05 18:25

Message:
Logged In: YES 
user_id=71407

> Look at typeobject.c:type_module() (about 100 lines in).
>
> Is simple.empty a HEAPTYPE?

Yes, it is a HEAPTYPE. I've established this by adding print 
statements
in typeobject.c:type_module():

static PyObject *
type_module(PyTypeObject *type, void *context)
{
        PyObject *mod;
        char *s;

        if (type->tp_flags & Py_TPFLAGS_HEAPTYPE) {
printf("IS HEAPTYPE\n");
                mod = PyDict_GetItemString(type-
>tp_dict, "__module__");
                Py_XINCREF(mod);
                s = PyString_AsString(mod);
printf("type->tp_dict, __module__ = %s\n", s);
                return mod;
        }

Result:

Python 2.3a1 (#2, Feb  5 2003, 09:39:30) 
[GCC 2.96 20000731 (Red Hat Linux 7.3 2.96-110)] on linux2
Type "help", "copyright", "credits" or "license" for more 
information.
>>> import simple
>>> simple.empty.__module__
IS HEAPTYPE
type->tp_dict, __module__ = __main__
'__main__'
>>> 

> If you can tell me what you want that function to do, I can
> have a go at it.

We expect "simple" as the result of simple.empty.__module__.

Current result if simple.so is moved to a package:

Python 2.3a1 (#2, Feb  5 2003, 09:39:30) 
[GCC 2.96 20000731 (Red Hat Linux 7.3 2.96-110)] on linux2
Type "help", "copyright", "credits" or "license" for more 
information.
>>> from sandbx import simple
>>> simple.empty.__module__
IS HEAPTYPE
type->tp_dict, __module__ = __main__
'__main__'
>>> 

Here we expect "sandbx.simple" .
Python 2.2.x produces the expected results using the exact same
Boost.Python source code.

David Abrahams provides this additional information:

  Remember that simple.empty is created by calling the metatype, 
so
  it's created on the heap, but it's Python's internal type creation
  mechanisms which do it -- we're not setting the flags manually.

Thanks!
        Ralf


----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2003-02-05 14:10

Message:
Logged In: YES 
user_id=6656

There has certainly been a change here.

Look at typeobject.c:type_module() (about 100 lines in).

If you can tell me what you want that function to do, I can
have a go at it.

Is simple.empty a HEAPTYPE?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=680429&group_id=5470


From noreply@sourceforge.net  Tue Feb 11 15:34:09 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 11 Feb 2003 07:34:09 -0800
Subject: [Python-bugs-list] [ python-Bugs-684667 ] Modules/selectmodule.c returns NULL without exception set
Message-ID: <E18icQD-0008IZ-00@sc8-sf-web2.sourceforge.net>

Bugs item #684667, was opened at 2003-02-11 16:34
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=684667&group_id=5470

Category: Extension Modules
Group: Python 2.2.2
Status: Open
Resolution: None
Priority: 5
Submitted By: Troels Walsted Hansen (troels)
Assigned to: Nobody/Anonymous (nobody)
Summary: Modules/selectmodule.c returns NULL without exception set

Initial Comment:
The "return NULL" statement below should read "return 
PyErr_NoMemory()".

#ifdef SELECT_USES_HEAP
        /* Allocate memory for the lists */
        rfd2obj = PyMem_NEW(pylist, FD_SETSIZE + 1);
        wfd2obj = PyMem_NEW(pylist, FD_SETSIZE + 1);
        efd2obj = PyMem_NEW(pylist, FD_SETSIZE + 1);
        if (rfd2obj == NULL || wfd2obj == NULL || efd2obj 
== NULL) {
                if (rfd2obj) PyMem_DEL(rfd2obj);
                if (wfd2obj) PyMem_DEL(wfd2obj);
                if (efd2obj) PyMem_DEL(efd2obj);
                return NULL;
        }
#endif /* SELECT_USES_HEAP */


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=684667&group_id=5470


From noreply@sourceforge.net  Tue Feb 11 16:01:23 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 11 Feb 2003 08:01:23 -0800
Subject: [Python-bugs-list] [ python-Bugs-684679 ] Freeze can not be made to exclude implicits
Message-ID: <E18icqZ-0000xl-00@sc8-sf-web2.sourceforge.net>

Bugs item #684679, was opened at 2003-02-11 16:01
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=684679&group_id=5470

Category: Demos and Tools
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Lawrence Hudson (lhudson)
Assigned to: Nobody/Anonymous (nobody)
Summary: Freeze can not be made to exclude implicits

Initial Comment:
Freeze always includes site and exceptions in the
resulting binary.  This is undesirable when creating an
extension module that contains frozen code;in sucha
situation the extension module should import the site
and exceptions (and all other standard library modules)
from the target system at runtime.

Patch 684677 fixes this bug.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=684679&group_id=5470


From noreply@sourceforge.net  Tue Feb 11 17:26:37 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 11 Feb 2003 09:26:37 -0800
Subject: [Python-bugs-list] [ python-Bugs-684667 ] Modules/selectmodule.c returns NULL without exception set
Message-ID: <E18ieB3-00020D-00@sc8-sf-web4.sourceforge.net>

Bugs item #684667, was opened at 2003-02-11 10:34
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=684667&group_id=5470

Category: Extension Modules
Group: Python 2.2.2
>Status: Closed
>Resolution: Fixed
Priority: 5
Submitted By: Troels Walsted Hansen (troels)
>Assigned to: Tim Peters (tim_one)
Summary: Modules/selectmodule.c returns NULL without exception set

Initial Comment:
The "return NULL" statement below should read "return 
PyErr_NoMemory()".

#ifdef SELECT_USES_HEAP
        /* Allocate memory for the lists */
        rfd2obj = PyMem_NEW(pylist, FD_SETSIZE + 1);
        wfd2obj = PyMem_NEW(pylist, FD_SETSIZE + 1);
        efd2obj = PyMem_NEW(pylist, FD_SETSIZE + 1);
        if (rfd2obj == NULL || wfd2obj == NULL || efd2obj 
== NULL) {
                if (rfd2obj) PyMem_DEL(rfd2obj);
                if (wfd2obj) PyMem_DEL(wfd2obj);
                if (efd2obj) PyMem_DEL(efd2obj);
                return NULL;
        }
#endif /* SELECT_USES_HEAP */


----------------------------------------------------------------------

>Comment By: Tim Peters (tim_one)
Date: 2003-02-11 12:26

Message:
Logged In: YES 
user_id=31435

Good eye!  Fixed in Modules/selectmodule.c, rev 2.73.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=684667&group_id=5470


From noreply@sourceforge.net  Tue Feb 11 19:33:23 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 11 Feb 2003 11:33:23 -0800
Subject: [Python-bugs-list] [ python-Bugs-683726 ] METH_CLASS allows binding to the wrong class
Message-ID: <E18ig9j-0004uH-00@sc8-sf-web3.sourceforge.net>

Bugs item #683726, was opened at 2003-02-09 22:21
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=683726&group_id=5470

Category: Python Interpreter Core
Group: Python 2.3
>Status: Closed
>Resolution: Fixed
Priority: 7
Submitted By: Greg Chapman (glchapman)
Assigned to: Guido van Rossum (gvanrossum)
Summary: METH_CLASS allows binding to the wrong class

Initial Comment:
I just noticed that this doesn't raise an exception:

>>> cmeth = dict.__dict__['fromkeys'].__get__(None, 
list)
>>> cmeth
<built-in method fromkeys of type object at 
0x1E0BB2E0>
>>> cmeth.__self__
<type 'list'>

Looking over the various METH_CLASS 
implementations, it doesn't look like they depend on the 
first (cls) parameter being of any particular type, but it 
might be nice to flag something like the above with an 
exception.

Also, there's this:

>>> cmeth2 = dict.__dict__['fromkeys'].__get__(None, 
42)
>>> cmeth2
<built-in method fromkeys of int object at 0x0074FD90>
>>> print cmeth2.__self__
42

Above, 42 gets passed through to classmethod_get in 
the type parameter, despite the fact that this parameter 
is typed as a PyTypeObject *. Perhpas wrap_descr_get 
should check that its second parameter (if it gets one) is 
a type?
 

----------------------------------------------------------------------

>Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-11 14:33

Message:
Logged In: YES 
user_id=6380

Thanks! The classmethod_get() function was in serious need
of some type checking. All of the examples you show now
raise TypeError in CVS.



----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-10 17:16

Message:
Logged In: YES 
user_id=6380

Looking into this now...


----------------------------------------------------------------------

Comment By: Greg Chapman (glchapman)
Date: 2003-02-10 11:59

Message:
Logged In: YES 
user_id=86307

I'm not entirely sure what the desired semantics for 
classmethod_get should be, but it seems to me they should 
be something like the following:

1) descr.__get__(None)
   descr.__get__(None, None)

This case would construct a PyCFunction using descr's 
d_type.  (What happens if both obj and type are NULL (as 
opposed to None)?  I think that should be a BadInternalCall, 
but perhaps it should also return a function with d_type.)

2) descr.__get__(notNone)
    descr.__get__(notNone, anything)

I think this case should be handled as for other descriptors.  
notNone would be checked to see if it is an instance of 
d_type; if so, a PyCFunction would be constructed using 
type(notNone).

3) descr.__get__(None, notNone)

Here notNone would be checked to ensure it is a type object, 
and then checked to see if it is a subtype of d_type.  If it 
passes these checks, then a PyCFunction is constructed 
using notNone.  (This branch would also handle the case 
where the obj == NULL in classmethod_get).

Alternatively, if it is always safe to assume that obj == NULL 
means that classmethod_get was called by the core library, 
and so the type parameter is correct, there could be a 0'th 
branch for obj == NULL which returns a PyCFunction 
constructed using type.


----------------------------------------------------------------------

Comment By: Greg Chapman (glchapman)
Date: 2003-02-09 22:38

Message:
Logged In: YES 
user_id=86307

I just realized that the above can be used to crash python, so 
I believe some sort of checking has to be added to 
classmethod_get:

>>> cmeth = dict.__dict__['fromkeys'].__get__(None)
>>> cmeth(4)

(above causes and access violation because the cls object 
passed to the fromkeys method is NULL).


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=683726&group_id=5470


From noreply@sourceforge.net  Tue Feb 11 20:48:39 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 11 Feb 2003 12:48:39 -0800
Subject: [Python-bugs-list] [ python-Bugs-681367 ] C subtypes do not inherit tp_as_buffer
Message-ID: <E18ihKZ-0007UL-00@sc8-sf-web2.sourceforge.net>

Bugs item #681367, was opened at 2003-02-05 19:23
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=681367&group_id=5470

Category: Python Interpreter Core
Group: Python 2.3
>Status: Closed
>Resolution: Accepted
Priority: 5
Submitted By: Greg Chapman (glchapman)
Assigned to: Guido van Rossum (gvanrossum)
Summary: C subtypes do not inherit tp_as_buffer

Initial Comment:
Subtypes of str (for example) implemented in C do not 
inherit str's tp_as_buffer funcs unless the subtypes provide 
a pointer to a tp_as_buffer structure in their type struct.  
This is different from how the other method suites 
(tp_as_sequence, etc.) are handled; near the end of 
PyType_Ready the other method suites are copied from 
the base into the subtype if the subtype's slot is NULL.  

It looks like the tp_as_buffer omission is simply an 
oversight, since the flag manipulations in inherit_special 
appear to be preparing the way for copying the base 
pointer into the subtype.  So I've attached a small patch 
(against the 2.3a1 version of typeobject.c) to fix this.


----------------------------------------------------------------------

>Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-11 15:48

Message:
Logged In: YES 
user_id=6380

Accepted and checked in. I'm not 100% sure, but I can't see
anything wrong with it, so here it goes.

----------------------------------------------------------------------

Comment By: Greg Chapman (glchapman)
Date: 2003-02-05 19:25

Message:
Logged In: YES 
user_id=86307

aargh -- the patch didn't get attached; trying again.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=681367&group_id=5470


From noreply@sourceforge.net  Tue Feb 11 21:02:29 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 11 Feb 2003 13:02:29 -0800
Subject: [Python-bugs-list] [ python-Bugs-684903 ] socket module on solaris
Message-ID: <E18ihXx-0000sX-00@sc8-sf-web3.sourceforge.net>

Bugs item #684903, was opened at 2003-02-11 16:02
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=684903&group_id=5470

Category: Build
Group: Python 2.2.2
Status: Open
Resolution: None
Priority: 5
Submitted By: RIck Bradshaw (rabrads)
Assigned to: Nobody/Anonymous (nobody)
Summary: socket module on solaris

Initial Comment:
I just built python 2.2.2 on solaris 7 or 2.7 to be
technically correct.  I then fire up the interpreter
and import test.testall. It fails for socket, which is
very import to me and my users. I then import _socket
and get the following info

Python 2.2.2 (#4, Feb  6 2003, 17:17:38) [C] on sunos5
Type "help", "copyright", "credits" or "license" for
more information.
>>> import _socket
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
ImportError: ld.so.1:
/soft/apps/packages/Python-2.2.2/bin/python: fatal:
relocation error: file
/soft/apps/packages/Python-2.2.2/lib/python2.2/_socket.so:
symbol __eprintf: referenced symbol not found

Thanks for any help.
Rick

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=684903&group_id=5470


From noreply@sourceforge.net  Tue Feb 11 21:08:35 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 11 Feb 2003 13:08:35 -0800
Subject: [Python-bugs-list] [ python-Bugs-663701 ] sets module review
Message-ID: <E18ihdr-0001cP-00@sc8-sf-web1.sourceforge.net>

Bugs item #663701, was opened at 2003-01-07 09:08
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=663701&group_id=5470

Category: Python Library
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Sebastien Keim (s_keim)
>Assigned to: Raymond Hettinger (rhettinger)
Summary: sets module review

Initial Comment:
* the ^ operator doesnt print well in pdf generated
documentation (both in what's new and in library reference)

* shouldn't the _as_immutable and
_as_temporaly_immutable be spelled __as_immutable__ and
__as_temporaly_immutable__ for consistency with other
hook methods?

* cmp() suck:
<pre>
bash-2.05$ ./python 
Python 2.3a1 (#1, Jan  4 2003, 10:17:56) 
[GCC 2.95.3 20010315 (release)] on linux2
Type "help", "copyright", "credits" or "license" for
more information.
>>> import sets
>>> s = sets.Set
>>> a = s([0])
>>> b = s([1])
>>> cmp(a,b)
1
>>> 
bash-2.05$ ./python 
Python 2.3a1 (#1, Jan  4 2003, 10:17:56) 
[GCC 2.95.3 20010315 (release)] on linux2
Type "help", "copyright", "credits" or "license" for
more information.
>>> import sets
>>> a=sets.Set([0])
>>> b=sets.Set([1])
>>> cmp(a,b)
-1
</pre>

* Because we can have set1!=set2 and both
(set1<set2)==False and (set2<set1)==False what will be
the behavhior of: [set1, set2].sort().
Since inclusion can't  define a full ordering relation,
wouldn't it be better to remove the equivalence between
issubset and __lt__ (and other comparaison methods) and
to use an equivalent to the dictionary protocol to
define the < and > behavior?




----------------------------------------------------------------------

>Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-11 16:08

Message:
Logged In: YES 
user_id=6380

Assigning to Raymond Hettinger. Maybe there are some
concrete to-do items above; after that I think it can be
closed. Maybe using __xxxx__ names is a good idea (since
when somebody else reuses these names, we can blame it on
them violating the rule not to use __xxxx__ names except as
documented).

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2003-01-13 00:28

Message:
Logged In: YES 
user_id=31435

try_rich_to_3way_compare gives up after LT, EQ and GT all 
return false.  The comparison then falls back to the default 
comparison by object id (addresses).  So, e.g., I can build 
an a, b, and c on my box such that:

>>> a, b, c
(Set([0]), Set([1]), Set([1]))
>>> cmp(b, c)   # b equals c
0
>>> cmp(a, b)   # but a "greater than" b while
1
>>> cmp(a, c)   # a "less than" c, despite that b == c
-1
>>>

This is simply because a is at a lower address than c but at 
a higher address than b:

>>> id(a) < id(c)
True
>>> id(a) > id(b)
True
>>>

I don't think <= for subset is too cute -- sets do have a 
natural and useful partial order.  I don't really care happens 
if you pass them to cmp(), though.  An exception would be 
fine -- sets weren't intended to be "generally" comparable.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-01-12 09:28

Message:
Logged In: YES 
user_id=6380

It's unlikely that the built-in type will behave exactly the
same, so the name difference is a good way to distinguish
the module from a future builtin.

Regarding cmp(): hmm, we may need to think about this more.
Currently comparison operators are used to express subset
relationships, e.g. <= means subset, < means strict subset.
The result of this is that cmp() between sets that are
neither subset nor set gets a useless outcome. (I can
reproduce your example but can't explain it.) Maybe we
should implement __cmp__ to raise an exception? Or maybe <=
for subset is too cute?

Assigning for Tim to ask his opinion.

----------------------------------------------------------------------

Comment By: Terry J. Reedy (tjreedy)
Date: 2003-01-07 17:44

Message:
Logged In: YES 
user_id=593130

If/when sets become builtin, would type object be 
called 'set' or break current convention by being 
called 'Set'?  If the former, I think classSet should be 'set' 
now so one can write 'from sets import set' and have rest 
of code ready for the future.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=663701&group_id=5470


From noreply@sourceforge.net  Tue Feb 11 21:10:12 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 11 Feb 2003 13:10:12 -0800
Subject: [Python-bugs-list] [ python-Bugs-639611 ] crash (SEGV) in Py_EndInterpreter()
Message-ID: <E18ihfQ-0001hl-00@sc8-sf-web1.sourceforge.net>

Bugs item #639611, was opened at 2002-11-17 04:38
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=639611&group_id=5470

Category: Python Interpreter Core
Group: Python 2.2.2
Status: Open
Resolution: None
>Priority: 3
Submitted By: Gernot Hillier (yoda_gh)
Assigned to: Guido van Rossum (gvanrossum)
Summary: crash (SEGV) in Py_EndInterpreter()

Initial Comment:
I experience a 80% reproducable SIGSEGV in a multithreaded app I  
wrote using different embedded sub interpreters in its threads.  
  
I have a C++ class (PClass) which encapsulates the sub-interpreters  
somehow - it creates a new one in its constructor and deletes it in  
the destructor (see below).  
  
When 2 objects of this class are destroyed at nearly the same time  
thus resulting in subsequent calls to Py_EndInterpreter() (but for the 
different threads), I get the following SIGSEGV in most cases:  
  
> gdb application core.31889  
[...]  
Program terminated with signal 11, Segmentation fault.  
[...]  
Reading symbols from /lib/ld-linux.so.2...done.  
Loaded symbols for /lib/ld-linux.so.2  
Reading symbols from  
/usr/local/lib/python2.2/lib-dynload/time.so...done.  
Loaded symbols for /usr/local/lib/python2.2/lib-dynload/time.so  
#0  0x08078763 in class_dealloc (op=0x81a8624) at  
Objects/classobject.c:169  
169             _PyObject_GC_UNTRACK(op);  
(gdb) bt  
#0  0x08078763 in class_dealloc (op=0x81a8624) at  
Objects/classobject.c:169  
#1  0x080daefa in PyThreadState_Clear (tstate=0x816b7f0) at   
Python/pystate.c:190  
#2  0x080dab89 in PyInterpreterState_Clear (interp=0x81427e8) at   
Python/pystate.c:77  
#3  0x080dce1e in Py_EndInterpreter (tstate=0x816b7f0) at   
Python/pythonrun.c:381  
#4  0x0805c287 in ~PClass (this=0x81421d0) at pclass.cpp:123  
  
When the 2nd object is destroyed some seconds later, everything  
seems to be fine. It only crashes when the 2 objects are deleted  
within a short period of time.  
  
I've tried this with the SuSE RPMs of Python 2.2.1 and a self-built   
Python-2.2.2 with the same result. :-(  
  
Here's a very short snippet of the python related code in my class:  
  
// Constructor, initializes Python sub-interpreter  
PClass::PClass()   
{  
        PyEval_AcquireLock();  
        py_state=Py_NewInterpreter();  
        PyEval_SaveThread();  
}  
  
// destructor, kills the Python sub-interpreter  
PClass::~PClass()  
{  
        PyEval_RestoreThread(py_state);   
        Py_EndInterpreter(py_state);  
        py_state=NULL;  
        PyEval_ReleaseLock(); // release lock  
}  
  
// thread code, runs Python function   
void  
PClass::run()  
{  
        PyEval_RestoreThread(py_state);   
        PyObject_CallFunction(...)   
        PyEval_SaveThread();  
}  

----------------------------------------------------------------------

>Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-11 16:10

Message:
Logged In: YES 
user_id=6380

No time for this now, lowering priority. If someone has a
patch, feel free to submit to SF.

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2002-12-31 16:35

Message:
Logged In: YES 
user_id=31435

Guido should look at it -- subinterpreters were his idea 
<wink>.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2002-12-31 08:19

Message:
Logged In: YES 
user_id=33168

Tim, it was suggested you see this.  Is this documented
somewhere?  What should be done?

----------------------------------------------------------------------

Comment By: Grzegorz Makarewicz (makaron)
Date: 2002-11-21 04:31

Message:
Logged In: YES 
user_id=115179


I think it should be documented and additional check
performed before killing thread state or interpereter.

Leave it open - Tim Peters should see this.


----------------------------------------------------------------------

Comment By: Gernot Hillier (yoda_gh)
Date: 2002-11-21 01:09

Message:
Logged In: YES 
user_id=633130

Ok, after cleaning up my exception handling a little bit (especially preventing  
exceptions to be triggered from outside after run() has finished) I can't 
reproduce the problems any more.  
  
So the bug is worked around for me. I leave it open if you consider this to 
be a real bug which has to be fixed. 
 
If you don't fix it and an user must always call PyErr_Clear() before 
Py_EndInterpreter() this should be documented IMHO. 
 
TIA! 

----------------------------------------------------------------------

Comment By: Gernot Hillier (yoda_gh)
Date: 2002-11-20 03:12

Message:
Logged In: YES 
user_id=633130

It really seems to have some connection with my exception handling. 
 
I'm investigating further. Please stand by... 

----------------------------------------------------------------------

Comment By: Grzegorz Makarewicz (makaron)
Date: 2002-11-18 09:00

Message:
Logged In: YES 
user_id=115179


I have found it (perhaps), while an python exception is
pending and  PyEval_RestoreThread is called, my Python
anwalys bumps.

After adding PyErr_Clear or  PyErr_Print just before this
call makes python happy.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=639611&group_id=5470


From noreply@sourceforge.net  Tue Feb 11 22:46:28 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 11 Feb 2003 14:46:28 -0800
Subject: [Python-bugs-list] [ python-Bugs-620190 ] inspect and object instances
Message-ID: <E18ijAa-0007lX-00@sc8-sf-web4.sourceforge.net>

Bugs item #620190, was opened at 2002-10-08 12:26
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=620190&group_id=5470

Category: Python Library
Group: Python 2.2.2
Status: Open
Resolution: None
Priority: 5
Submitted By: Alexander Schmolck (aschmolck)
Assigned to: Nobody/Anonymous (nobody)
Summary: inspect and object instances

Initial Comment:
inspect.getargspec(NewKlass.aMethod)  fails (which
might technically be OK because the docs only mention
functions, not methods) but I also vaguely seem to
remember that some other operations in the inspect
module only worked for oldstyle classes under 2.2.1.

----------------------------------------------------------------------

Comment By: Michael Stone (mbrierst)
Date: 2003-02-11 22:46

Message:
Logged In: YES 
user_id=670441

Well I think this (one line)
patch will make getargspec work with
methods, if anyone wants to apply it.
Did you find any other operations that
don't work?

patch:
Index: Lib/inspect.py
===================================================================
RCS file: /cvsroot/python/python/dist/src/Lib/inspect.py,v
retrieving revision 1.41
diff -c -r1.41 inspect.py
*** Lib/inspect.py	19 Jan 2003 13:21:20 -0000	1.41
--- Lib/inspect.py	11 Feb 2003 22:35:41 -0000
***************
*** 16,22 ****
      getmodule() - determine the module that an object came from
      getclasstree() - arrange classes so as to represent their hierarchy
  
!     getargspec(), getargvalues() - get info about function arguments
      formatargspec(), formatargvalues() - format an argument spec
      getouterframes(), getinnerframes() - get info about frames
      currentframe() - get the current stack frame
--- 16,22 ----
      getmodule() - determine the module that an object came from
      getclasstree() - arrange classes so as to represent their hierarchy
  
!     getargspec(), getargvalues() - get info about function or method's arguments
      formatargspec(), formatargvalues() - format an argument spec
      getouterframes(), getinnerframes() - get info about frames
      currentframe() - get the current stack frame
***************
*** 625,636 ****
      return args, varargs, varkw
  
  def getargspec(func):
!     """Get the names and default values of a function's arguments.
  
      A tuple of four things is returned: (args, varargs, varkw, defaults).
      'args' is a list of the argument names (it may contain nested lists).
      'varargs' and 'varkw' are the names of the * and ** arguments or None.
      'defaults' is an n-tuple of the default values of the last n arguments."""
      if not isfunction(func): raise TypeError, 'arg is not a Python function'
      args, varargs, varkw = getargs(func.func_code)
      return args, varargs, varkw, func.func_defaults
--- 625,637 ----
      return args, varargs, varkw
  
  def getargspec(func):
!     """Get the names and default values of a function or method's arguments.
  
      A tuple of four things is returned: (args, varargs, varkw, defaults).
      'args' is a list of the argument names (it may contain nested lists).
      'varargs' and 'varkw' are the names of the * and ** arguments or None.
      'defaults' is an n-tuple of the default values of the last n arguments."""
+     if ismethod(func): func = func.im_func
      if not isfunction(func): raise TypeError, 'arg is not a Python function'
      args, varargs, varkw = getargs(func.func_code)
      return args, varargs, varkw, func.func_defaults


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=620190&group_id=5470


From noreply@sourceforge.net  Tue Feb 11 23:01:31 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 11 Feb 2003 15:01:31 -0800
Subject: [Python-bugs-list] [ python-Bugs-660098 ] New style classes and __hash__
Message-ID: <E18ijP9-0005Ui-00@sc8-sf-web3.sourceforge.net>

Bugs item #660098, was opened at 2002-12-30 13:39
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=660098&group_id=5470

Category: None
Group: None
Status: Open
Resolution: None
>Priority: 3
Submitted By: Thomas Heller (theller)
Assigned to: Guido van Rossum (gvanrossum)
Summary: New style classes and __hash__

Initial Comment:
New style classes obviously inherit a __hash__ 
implementation from object which returns the id. Per 
default this allows using instances as dictionary keys, 
but usually with the wrong behaviour, because most 
often user classes are mutable, and their contained data 
should be used to calculate the hash value.

IMO one possible solution would be to change 
typeobject.c:object_hash() to raise TypeError, and 
change all the immutable (core) Python objects to use 
_Py_HashPointer in their tp_hash slot.

----------------------------------------------------------------------

>Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-11 18:01

Message:
Logged In: YES 
user_id=6380

I spent an afternoon looking into this, and I can't see an
easy solution. The idea of only inheriting __hash__ together
with certain other slots is really flawed; it may be better
if object DIDN'T define a default implementation for
__hash__, comparisons (both flavors), and other things, or
maybe the default __hash__ should raise an exception when
the comparisons are not those inherited from object, or
maybe PyType_Ready should insert a dummy __hash__ when it
sees that you redefine __eq__, or...  I really don't know.
I'm going to sleep on this some more, and lower the
priority. You can always get the right behavior by
explicitly defining __hash__.

----------------------------------------------------------------------

Comment By: Thomas Heller (theller)
Date: 2002-12-30 13:50

Message:
Logged In: YES 
user_id=11105

You mean at the end of the inherit_slots() function?
For my extension which I'm currently debugging, tp_compare, 
tp_richcompare, and tp_hash are inherited from base, but 
only tp_hash is != NULL there.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2002-12-30 13:44

Message:
Logged In: YES 
user_id=6380

There seems to be code that tries to inherit tp_hash only
when tp_compare and tp_richcompare are also inherited, but
it seems to be failing.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=660098&group_id=5470


From noreply@sourceforge.net  Tue Feb 11 23:01:38 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 11 Feb 2003 15:01:38 -0800
Subject: [Python-bugs-list] [ python-Bugs-633930 ] Nested class __name__
Message-ID: <E18ijPG-0005gQ-00@sc8-sf-web2.sourceforge.net>

Bugs item #633930, was opened at 2002-11-05 12:56
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=633930&group_id=5470

Category: Python Interpreter Core
Group: None
Status: Open
Resolution: None
>Priority: 3
Submitted By: Guido van Rossum (gvanrossum)
Assigned to: Guido van Rossum (gvanrossum)
Summary: Nested class __name__

Initial Comment:
The __name__ attribute of a nested class should be set
to 'outer.inner', both for classic and for new-style
classes. E.g.

>>> class C:
...     class C1: pass
... 
>>> C.C1.__name__
'C.C1'
>>> 


----------------------------------------------------------------------

>Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-11 18:01

Message:
Logged In: YES 
user_id=6380

I'm less sure I even want this now, and not at all sure how
to do it any more, so lowering priority.


----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2002-11-14 18:13

Message:
Logged In: YES 
user_id=6380

Hm, but should this also be done for functions inside
classes?  E.g.

class C:
 def foo(self): pass

assert C.foo.__name__ == "C.foo"
assert C.__dict__["foo"].__name__ == "C.foo"

And what about classes inside functions?

def f():
  class C: pass
  return C

assert f().__name__ == "f.C"

???


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=633930&group_id=5470


From noreply@sourceforge.net  Tue Feb 11 23:03:56 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 11 Feb 2003 15:03:56 -0800
Subject: [Python-bugs-list] [ python-Bugs-684975 ] IDE asks for attention when quitting
Message-ID: <E18ijRU-0007VF-00@sc8-sf-web1.sourceforge.net>

Bugs item #684975, was opened at 2003-02-12 00:03
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=684975&group_id=5470

Category: Macintosh
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Jack Jansen (jackjansen)
Assigned to: Just van Rossum (jvr)
Summary: IDE asks for attention when quitting

Initial Comment:
When you quit the IDE (in MacPython-OSX) via the popup menu in the dock it somehow asks for attention (the icon jumps up and down enthusiastically). When you the bring it to front it exits normally, without showing any message or anything.

I've looked at this a couple of times but never managed to find what is going on. Do you have any idea?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=684975&group_id=5470


From noreply@sourceforge.net  Tue Feb 11 23:11:25 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 11 Feb 2003 15:11:25 -0800
Subject: [Python-bugs-list] [ python-Bugs-501716 ] "es#" parser marker leaks memory
Message-ID: <E18ijYj-0007qC-00@sc8-sf-web1.sourceforge.net>

Bugs item #501716, was opened at 2002-01-10 10:39
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=501716&group_id=5470

Category: Unicode
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: M.-A. Lemburg (lemburg)
Assigned to: M.-A. Lemburg (lemburg)
Summary: "es#" parser marker leaks memory

Initial Comment:
... if subsequent parsing fails, a buffer which was possibly 
allocated by "es#" is not freed.


----------------------------------------------------------------------

Comment By: Michael Stone (mbrierst)
Date: 2003-02-11 23:11

Message:
Logged In: YES 
user_id=670441

I submitted this as patch 684981 so it can be
attached to the report instead of pasted in.
Also, the patch there is nicer than the one here.

----------------------------------------------------------------------

Comment By: Michael Stone (mbrierst)
Date: 2003-02-10 20:37

Message:
Logged In: YES 
user_id=670441

Ahhhh... thank you for your reply.  I understand now.
I have a patch that gets rid of this problem by putting allocated
memory into a list of CObjects which are freed when exceptions occur.
The list is only created when necessary, to hopefully preserve efficiency
in most cases.  It has to be passed around, so several function calls had
to be changed, and to do the cleanup on return I made a macro CLEANRETURN.

This patch also fixes a memory leak when the buffer_len pointer is NULL.
This isn't a big deal since the documentation says never to pass in a NULL
buffer_len.

And it fixes another leak when a string with encoded NULL's is passed in
(again not a big deal)

If you want, I have C code that can be called from python to demonstrate
any of these leaks.  Let me know what you think of the patch/approach.

Index: Python/getargs.c
===================================================================
RCS file: /cvsroot/python/python/dist/src/Python/getargs.c,v
retrieving revision 2.95
diff -c -r2.95 getargs.c
*** Python/getargs.c	24 Jan 2003 22:15:21 -0000	2.95
--- Python/getargs.c	10 Feb 2003 20:28:52 -0000
***************
*** 17,26 ****
  static int vgetargs1(PyObject *, char *, va_list *, int);
  static void seterror(int, char *, int *, char *, char *);
  static char *convertitem(PyObject *, char **, va_list *, int *, char *, 
! 			 size_t);
  static char *converttuple(PyObject *, char **, va_list *,
! 			  int *, char *, size_t, int);
! static char *convertsimple(PyObject *, char **, va_list *, char *, size_t);
  static int convertbuffer(PyObject *, void **p, char **);
  
  static int vgetargskeywords(PyObject *, PyObject *,
--- 17,27 ----
  static int vgetargs1(PyObject *, char *, va_list *, int);
  static void seterror(int, char *, int *, char *, char *);
  static char *convertitem(PyObject *, char **, va_list *, int *, char *, 
! 			 size_t, PyObject **, int *);
  static char *converttuple(PyObject *, char **, va_list *,
! 			  int *, char *, size_t, int, PyObject **, int *);
! static char *convertsimple(PyObject *, char **, va_list *, char *,
! 			   size_t, PyObject **, int *);
  static int convertbuffer(PyObject *, void **p, char **);
  
  static int vgetargskeywords(PyObject *, PyObject *,
***************
*** 72,77 ****
--- 73,120 ----
  }
  
  
+ /* Handle cleanup of allocated memory in case of exception */
+ 
+ static void
+ docleanup(void *ptr, void *dofree)
+ {
+ 	if (*(int *)dofree)
+ 		PyMem_FREE(ptr);
+ }
+ 
+ static int
+ addcleanup(void *ptr, PyObject **freelist, int *dofree)
+ {
+ 	PyObject *cobj;
+ 	if (!*freelist) {
+ 		*freelist = PyList_New(0);
+ 		if (!*freelist) {
+ 			PyMem_FREE(ptr);
+ 			return -1;
+ 		}
+ 	}
+ 	cobj = PyCObject_FromVoidPtrAndDesc(ptr, dofree, docleanup);
+ 	if (!cobj) {
+ 		PyMem_FREE(ptr);
+ 		return -1;
+ 	}
+ 	if(PyList_Append(*freelist, cobj)) {
+ 		*dofree = 1;
+ 		Py_DECREF(cobj);
+ 		return -1;
+ 	}
+         Py_DECREF(cobj);
+ 	return 0;
+ }
+ 
+ #define CLEANRETURN(retval, freelist, dofree) do	\
+ {							\
+ 	dofree = ((retval) == 0);			\
+ 	Py_XDECREF(freelist);				\
+ 	return (retval);				\
+ } while(0)
+ 
+ 
  static int
  vgetargs1(PyObject *args, char *format, va_list *p_va, int compat)
  {
***************
*** 86,91 ****
--- 129,136 ----
  	char *formatsave = format;
  	int i, len;
  	char *msg;
+ 	PyObject *freelist = NULL;
+ 	int dofree = 0;
  	
  	assert(compat || (args != (PyObject*)NULL));
  
***************
*** 157,167 ****
  				return 0;
  			}
  			msg = convertitem(args, &format, p_va, levels, msgbuf,
! 					  sizeof(msgbuf));
  			if (msg == NULL)
! 				return 1;
  			seterror(levels[0], msg, levels+1, fname, message);
! 			return 0;
  		}
  		else {
  			PyErr_SetString(PyExc_SystemError,
--- 202,212 ----
  				return 0;
  			}
  			msg = convertitem(args, &format, p_va, levels, msgbuf,
! 					  sizeof(msgbuf), &freelist, &dofree);
  			if (msg == NULL)
! 				CLEANRETURN(1, freelist, dofree);
  			seterror(levels[0], msg, levels+1, fname, message);
! 			CLEANRETURN(0, freelist, dofree);
  		}
  		else {
  			PyErr_SetString(PyExc_SystemError,
***************
*** 200,209 ****
  		if (*format == '|')
  			format++;
  		msg = convertitem(PyTuple_GET_ITEM(args, i), &format, p_va,
! 				  levels, msgbuf, sizeof(msgbuf));
  		if (msg) {
  			seterror(i+1, msg, levels, fname, message);
! 			return 0;
  		}
  	}
  
--- 245,254 ----
  		if (*format == '|')
  			format++;
  		msg = convertitem(PyTuple_GET_ITEM(args, i), &format, p_va,
! 				  levels, msgbuf, sizeof(msgbuf), &freelist, &dofree);
  		if (msg) {
  			seterror(i+1, msg, levels, fname, message);
! 			CLEANRETURN(0, freelist, dofree);
  		}
  	}
  
***************
*** 212,221 ****
  	    *format != '|' && *format != ':' && *format != ';') {
  		PyErr_Format(PyExc_SystemError,
  			     "bad format string: %.200s", formatsave);
! 		return 0;
  	}
  	
! 	return 1;
  }
  
  
--- 257,266 ----
  	    *format != '|' && *format != ':' && *format != ';') {
  		PyErr_Format(PyExc_SystemError,
  			     "bad format string: %.200s", formatsave);
! 		CLEANRETURN(0, freelist, dofree);
  	}
  	
! 	CLEANRETURN(1, freelist, dofree);
  }
  
  
***************
*** 277,283 ****
  
  static char *
  converttuple(PyObject *arg, char **p_format, va_list *p_va, int *levels,
! 	     char *msgbuf, size_t bufsize, int toplevel)
  {
  	int level = 0;
  	int n = 0;
--- 322,329 ----
  
  static char *
  converttuple(PyObject *arg, char **p_format, va_list *p_va, int *levels,
! 	     char *msgbuf, size_t bufsize, int toplevel,
! 	     PyObject **freelist, int *dofree)
  {
  	int level = 0;
  	int n = 0;
***************
*** 327,333 ****
  		PyObject *item;
  		item = PySequence_GetItem(arg, i);
  		msg = convertitem(item, &format, p_va, levels+1, msgbuf,
! 				  bufsize);
  		/* PySequence_GetItem calls tp->sq_item, which INCREFs */
  		Py_XDECREF(item);
  		if (msg != NULL) {
--- 373,379 ----
  		PyObject *item;
  		item = PySequence_GetItem(arg, i);
  		msg = convertitem(item, &format, p_va, levels+1, msgbuf,
! 				  bufsize, freelist, dofree);
  		/* PySequence_GetItem calls tp->sq_item, which INCREFs */
  		Py_XDECREF(item);
  		if (msg != NULL) {
***************
*** 345,351 ****
  
  static char *
  convertitem(PyObject *arg, char **p_format, va_list *p_va, int *levels,
! 	    char *msgbuf, size_t bufsize)
  {
  	char *msg;
  	char *format = *p_format;
--- 391,397 ----
  
  static char *
  convertitem(PyObject *arg, char **p_format, va_list *p_va, int *levels,
! 	    char *msgbuf, size_t bufsize, PyObject **freelist, int *dofree)
  {
  	char *msg;
  	char *format = *p_format;
***************
*** 353,364 ****
  	if (*format == '(' /* ')' */) {
  		format++;
  		msg = converttuple(arg, &format, p_va, levels, msgbuf, 
! 				   bufsize, 0);
  		if (msg == NULL)
  			format++;
  	}
  	else {
! 		msg = convertsimple(arg, &format, p_va, msgbuf, bufsize);
  		if (msg != NULL)
  			levels[0] = 0;
  	}
--- 399,411 ----
  	if (*format == '(' /* ')' */) {
  		format++;
  		msg = converttuple(arg, &format, p_va, levels, msgbuf, 
! 				   bufsize, 0, freelist, dofree);
  		if (msg == NULL)
  			format++;
  	}
  	else {
! 		msg = convertsimple(arg, &format, p_va, msgbuf, bufsize,
! 				    freelist, dofree);
  		if (msg != NULL)
  			levels[0] = 0;
  	}
***************
*** 396,402 ****
  
  static char *
  convertsimple(PyObject *arg, char **p_format, va_list *p_va, char *msgbuf,
! 	      size_t bufsize)
  {
  	char *format = *p_format;
  	char c = *format++;
--- 443,449 ----
  
  static char *
  convertsimple(PyObject *arg, char **p_format, va_list *p_va, char *msgbuf,
! 	      size_t bufsize, PyObject **freelist, int *dofree)
  {
  	char *format = *p_format;
  	char c = *format++;
***************
*** 801,810 ****
  			int *buffer_len = va_arg(*p_va, int *);
  
  			format++;
! 			if (buffer_len == NULL)
  				return converterr(
  					"(buffer_len is NULL)",
  					arg, msgbuf, bufsize);
  			if (*buffer == NULL) {
  				*buffer = PyMem_NEW(char, size + 1);
  				if (*buffer == NULL) {
--- 848,859 ----
  			int *buffer_len = va_arg(*p_va, int *);
  
  			format++;
! 			if (buffer_len == NULL) {
! 				Py_DECREF(s);
  				return converterr(
  					"(buffer_len is NULL)",
  					arg, msgbuf, bufsize);
+ 			}
  			if (*buffer == NULL) {
  				*buffer = PyMem_NEW(char, size + 1);
  				if (*buffer == NULL) {
***************
*** 813,818 ****
--- 862,873 ----
  						"(memory error)",
  						arg, msgbuf, bufsize);
  				}
+ 				if(addcleanup(*buffer, freelist, dofree)) {
+ 					Py_DECREF(s);
+ 					return converterr(
+ 						"(cleanup problem)",
+ 						arg, msgbuf, bufsize);
+ 				}
  			} else {
  				if (size + 1 > *buffer_len) {
  					Py_DECREF(s);
***************
*** 839,854 ****
  			   PyMem_Free()ing it after usage
  
  			*/
! 			if ((int)strlen(PyString_AS_STRING(s)) != size)
  				return converterr(
  					"(encoded string without NULL bytes)",
  					arg, msgbuf, bufsize);
  			*buffer = PyMem_NEW(char, size + 1);
  			if (*buffer == NULL) {
  				Py_DECREF(s);
  				return converterr("(memory error)",
  						  arg, msgbuf, bufsize);
  			}
  			memcpy(*buffer,
  			       PyString_AS_STRING(s),
  			       size + 1);
--- 894,916 ----
  			   PyMem_Free()ing it after usage
  
  			*/
! 			if ((int)strlen(PyString_AS_STRING(s)) != size) {
! 				Py_DECREF(s);
  				return converterr(
  					"(encoded string without NULL bytes)",
  					arg, msgbuf, bufsize);
+ 			}
  			*buffer = PyMem_NEW(char, size + 1);
  			if (*buffer == NULL) {
  				Py_DECREF(s);
  				return converterr("(memory error)",
  						  arg, msgbuf, bufsize);
  			}
+ 			if(addcleanup(*buffer, freelist, dofree)) {
+ 				Py_DECREF(s);
+ 				return converterr("(cleanup problem)",
+ 						arg, msgbuf, bufsize);
+ 			}
  			memcpy(*buffer,
  			       PyString_AS_STRING(s),
  			       size + 1);
***************
*** 1068,1073 ****
--- 1130,1137 ----
  	char *formatsave;
  	int i, len, nargs, nkeywords;
  	char *msg, **p;
+ 	PyObject *freelist = NULL;
+ 	int dofree = 0;
  
  	assert(args != NULL && PyTuple_Check(args));
  	assert(keywords == NULL || PyDict_Check(keywords));
***************
*** 1192,1207 ****
  		if (*format == '|')
  			format++;
  		msg = convertitem(PyTuple_GET_ITEM(args, i), &format, p_va,
! 				 levels, msgbuf, sizeof(msgbuf));
  		if (msg) {
  			seterror(i+1, msg, levels, fname, message);
! 			return 0;
  		}
  	}
  
  	/* handle no keyword parameters in call */	
  	if (nkeywords == 0)
! 		return 1; 
  
  	/* convert the keyword arguments; this uses the format 
  	   string where it was left after processing args */
--- 1256,1272 ----
  		if (*format == '|')
  			format++;
  		msg = convertitem(PyTuple_GET_ITEM(args, i), &format, p_va,
! 				  levels, msgbuf, sizeof(msgbuf), &freelist,
! 				  &dofree);
  		if (msg) {
  			seterror(i+1, msg, levels, fname, message);
! 			CLEANRETURN(0, freelist, dofree);
  		}
  	}
  
  	/* handle no keyword parameters in call */	
  	if (nkeywords == 0)
! 		CLEANRETURN(1, freelist, dofree);
  
  	/* convert the keyword arguments; this uses the format 
  	   string where it was left after processing args */
***************
*** 1213,1235 ****
  		if (item != NULL) {
  			Py_INCREF(item);
  			msg = convertitem(item, &format, p_va, levels, msgbuf,
! 					  sizeof(msgbuf));
  			Py_DECREF(item);
  			if (msg) {
  				seterror(i+1, msg, levels, fname, message);
! 				return 0;
  			}
  			--nkeywords;
  			if (nkeywords == 0)
  				break;
  		}
  		else if (PyErr_Occurred())
! 			return 0;
  		else {
  			msg = skipitem(&format, p_va);
  			if (msg) {
  				seterror(i+1, msg, levels, fname, message);
! 				return 0;
  			}
  		}
  	}
--- 1278,1300 ----
  		if (item != NULL) {
  			Py_INCREF(item);
  			msg = convertitem(item, &format, p_va, levels, msgbuf,
! 					  sizeof(msgbuf), &freelist, &dofree);
  			Py_DECREF(item);
  			if (msg) {
  				seterror(i+1, msg, levels, fname, message);
! 				CLEANRETURN(0, freelist, dofree);
  			}
  			--nkeywords;
  			if (nkeywords == 0)
  				break;
  		}
  		else if (PyErr_Occurred())
! 			CLEANRETURN(0, freelist, dofree);
  		else {
  			msg = skipitem(&format, p_va);
  			if (msg) {
  				seterror(i+1, msg, levels, fname, message);
! 				CLEANRETURN(0, freelist, dofree);
  			}
  		}
  	}
***************
*** 1244,1250 ****
  			if (!PyString_Check(key)) {
  				PyErr_SetString(PyExc_TypeError, 
  					        "keywords must be strings");
! 				return 0;
  			}
  			ks = PyString_AsString(key);
  			for (i = 0; i < max; i++) {
--- 1309,1315 ----
  			if (!PyString_Check(key)) {
  				PyErr_SetString(PyExc_TypeError, 
  					        "keywords must be strings");
! 				CLEANRETURN(0, freelist, dofree);
  			}
  			ks = PyString_AsString(key);
  			for (i = 0; i < max; i++) {
***************
*** 1258,1269 ****
  					     "'%s' is an invalid keyword "
  					     "argument for this function",
  					     ks);
! 				return 0;
  			}
  		}
  	}
  
! 	return 1;
  }
  
  
--- 1323,1334 ----
  					     "'%s' is an invalid keyword "
  					     "argument for this function",
  					     ks);
! 				CLEANRETURN(0, freelist, dofree);
  			}
  		}
  	}
  
! 	CLEANRETURN(1, freelist, dofree);
  }
  
  


----------------------------------------------------------------------

Comment By: M.-A. Lemburg (lemburg)
Date: 2003-02-08 14:29

Message:
Logged In: YES 
user_id=38388

What I meant was that in the case of a parser marker
string like "es#iii" if the last "i" fails to convert, then the
buffer used for "es#" won't be freed up again.

----------------------------------------------------------------------

Comment By: Michael Stone (mbrierst)
Date: 2003-02-07 20:20

Message:
Logged In: YES 
user_id=670441

In what case does this happen?  The only case
I can see is if the string is successfully encoded,
but the encoding results in a non-null terminated 
string.  That might cause problems, otherwise it
looks okay?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=501716&group_id=5470


From noreply@sourceforge.net  Tue Feb 11 23:23:02 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 11 Feb 2003 15:23:02 -0800
Subject: [Python-bugs-list] [ python-Bugs-683658 ] PyErr_Warn may cause import deadlock
Message-ID: <E18ijjy-00011k-00@sc8-sf-web4.sourceforge.net>

Bugs item #683658, was opened at 2003-02-09 19:26
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=683658&group_id=5470

Category: Python Interpreter Core
Group: Python 2.3
Status: Open
Resolution: None
Priority: 6
Submitted By: Mark Hammond (mhammond)
Assigned to: Nobody/Anonymous (nobody)
Summary: PyErr_Warn may cause import deadlock

Initial Comment:
PyErr_Warn() does an implicit import.  Thus, if
PyErr_Warn() is called on a thread while the main
thread holds the import lock, and the main thread then
subsequently waits for the child thread, Python will
deadlock.  The builtin 'apply' now calls PyErr_Warn(),
so simply calling 'apply' on another thread may cause
the deadlock.

Attaching a sample case.  Executing 'python -c "import
hang_apply"' will cause Python to deadlock.  Commenting
out the call to "apply" in that file will prevent the
deadlock (even though the apply() in question is
effectively a no-op)

The example is a little contrived, but is extracted
from real code of mine that saw this hang.  The code
path is:

* Main thread acquires import lock to import the module.
* Module import spawns a new thread.  This new thread
calls apply()
* builtin_apply() calls PyErr_Warn
* PyErr_Warn does an import of the warnings module, and
attempts to acquire the import lock.
* Main thread still waiting for child thread to
complete, but still holds import lock.

Note that removing the call to PyErr_Warn() in
builtin_apply also solves this particular hang -
however, it seems like this is a potential time bomb. 
A potential solution would be to prevent PyErr_Warn
from doing an import - this would mean importing
'warnings' at startup, and keeping a static reference
in errors.c.  Other thoughts welcome.


----------------------------------------------------------------------

>Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-11 18:23

Message:
Logged In: YES 
user_id=33168

I can't think of any other/better solution.  Any idea if
there are there other ticking bombs like this?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=683658&group_id=5470


From noreply@sourceforge.net  Tue Feb 11 23:25:45 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 11 Feb 2003 15:25:45 -0800
Subject: [Python-bugs-list] [ python-Bugs-684903 ] socket module on solaris
Message-ID: <E18ijmb-0006RN-00@sc8-sf-web2.sourceforge.net>

Bugs item #684903, was opened at 2003-02-11 16:02
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=684903&group_id=5470

Category: Build
Group: Python 2.2.2
Status: Open
Resolution: None
Priority: 5
Submitted By: RIck Bradshaw (rabrads)
Assigned to: Nobody/Anonymous (nobody)
Summary: socket module on solaris

Initial Comment:
I just built python 2.2.2 on solaris 7 or 2.7 to be
technically correct.  I then fire up the interpreter
and import test.testall. It fails for socket, which is
very import to me and my users. I then import _socket
and get the following info

Python 2.2.2 (#4, Feb  6 2003, 17:17:38) [C] on sunos5
Type "help", "copyright", "credits" or "license" for
more information.
>>> import _socket
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
ImportError: ld.so.1:
/soft/apps/packages/Python-2.2.2/bin/python: fatal:
relocation error: file
/soft/apps/packages/Python-2.2.2/lib/python2.2/_socket.so:
symbol __eprintf: referenced symbol not found

Thanks for any help.
Rick

----------------------------------------------------------------------

>Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-11 18:25

Message:
Logged In: YES 
user_id=33168

This error looks familiar, but I don't recall the cause. 
Did you have a working version of python on this machine? 
What compiler and version are you using?  Is __eprintf in a
library (libc.a)?  Is that not getting linked?  Is it using
the correct linker and library locations?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=684903&group_id=5470


From noreply@sourceforge.net  Wed Feb 12 00:34:15 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 11 Feb 2003 16:34:15 -0800
Subject: [Python-bugs-list] [ python-Bugs-685011 ] calendar module overflow handling
Message-ID: <E18ikqt-0000bK-00@sc8-sf-web3.sourceforge.net>

Bugs item #685011, was opened at 2003-02-12 11:34
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=685011&group_id=5470

Category: Python Library
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Richard Jones (richard)
Assigned to: Nobody/Anonymous (nobody)
Summary: calendar module overflow handling

Initial Comment:
The calendar module has lost a significant area of functionality in 
Python 2.3  
- that of handling overflows in the tuples passed to timegm(). 
 
Python 2.2.2 (#1, Oct 28 2002, 16:39:08) 
[GCC 2.96 20000731 (Red Hat Linux 7.1 2.96-98)] on linux2 
Type "help", "copyright", "credits" or "license" for more information. 
>>> import time,calendar 
>>> time.gmtime(calendar.timegm((2003,1,30,0,0,0,0,0,0))) 
(2003, 1, 30, 0, 0, 0, 3, 30, 0) 
>>> time.gmtime(calendar.timegm((2003,2,30,0,0,0,0,0,0))) 
(2003, 3, 2, 0, 0, 0, 6, 61, 0) 
>>> 
 
Python 2.3a1 (#1, Feb  6 2003, 14:54:16) 
[GCC 2.96 20000731 (Red Hat Linux 7.1 2.96-98)] on linux2 
Type "help", "copyright", "credits" or "license" for more information. 
>>> import time,calendar 
>>> time.gmtime(calendar.timegm((2003,1,30,0,0,0,0,0,0))) 
(2003, 1, 30, 0, 0, 0, 3, 30, 0) 
>>> time.gmtime(calendar.timegm((2003,2,30,0,0,0,0,0,0))) 
Traceback (most recent call last): 
  File "<stdin>", line 1, in ? 
  File "/usr/lib/python2.3/calendar.py", line 216, in timegm 
    days = datetime.date(year, month, day).toordinal() - _EPOCH_ORD 
ValueError: day is out of range for month 
>>> 
 
 
Is this functionality likely to be reinstated for 2.3's release? 
 

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=685011&group_id=5470


From noreply@sourceforge.net  Wed Feb 12 00:54:23 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 11 Feb 2003 16:54:23 -0800
Subject: [Python-bugs-list] [ python-Bugs-685011 ] calendar module overflow handling
Message-ID: <E18ilAN-0000vF-00@sc8-sf-web2.sourceforge.net>

Bugs item #685011, was opened at 2003-02-11 19:34
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=685011&group_id=5470

Category: Python Library
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Richard Jones (richard)
>Assigned to: Raymond Hettinger (rhettinger)
Summary: calendar module overflow handling

Initial Comment:
The calendar module has lost a significant area of functionality in 
Python 2.3  
- that of handling overflows in the tuples passed to timegm(). 
 
Python 2.2.2 (#1, Oct 28 2002, 16:39:08) 
[GCC 2.96 20000731 (Red Hat Linux 7.1 2.96-98)] on linux2 
Type "help", "copyright", "credits" or "license" for more information. 
>>> import time,calendar 
>>> time.gmtime(calendar.timegm((2003,1,30,0,0,0,0,0,0))) 
(2003, 1, 30, 0, 0, 0, 3, 30, 0) 
>>> time.gmtime(calendar.timegm((2003,2,30,0,0,0,0,0,0))) 
(2003, 3, 2, 0, 0, 0, 6, 61, 0) 
>>> 
 
Python 2.3a1 (#1, Feb  6 2003, 14:54:16) 
[GCC 2.96 20000731 (Red Hat Linux 7.1 2.96-98)] on linux2 
Type "help", "copyright", "credits" or "license" for more information. 
>>> import time,calendar 
>>> time.gmtime(calendar.timegm((2003,1,30,0,0,0,0,0,0))) 
(2003, 1, 30, 0, 0, 0, 3, 30, 0) 
>>> time.gmtime(calendar.timegm((2003,2,30,0,0,0,0,0,0))) 
Traceback (most recent call last): 
  File "<stdin>", line 1, in ? 
  File "/usr/lib/python2.3/calendar.py", line 216, in timegm 
    days = datetime.date(year, month, day).toordinal() - _EPOCH_ORD 
ValueError: day is out of range for month 
>>> 
 
 
Is this functionality likely to be reinstated for 2.3's release? 
 

----------------------------------------------------------------------

>Comment By: Tim Peters (tim_one)
Date: 2003-02-11 19:54

Message:
Logged In: YES 
user_id=31435

Cute.  My best reading of the code is that you were relying 
on an undocumented accident.  The timegm code in 2.2 did 
do bounds-checking on the year and month, it looks like it 
was just too lazy to check the day.

Assigning to Raymond to teach him a lesson <wink>.  If you 
want to restore this accident, pass 1 instead of days to the 
datetime constructor, and restore the old

    days = days + day - 1

line after that returns.  It should still be lot faster than the 
2.2 code.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=685011&group_id=5470


From noreply@sourceforge.net  Wed Feb 12 01:43:39 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 11 Feb 2003 17:43:39 -0800
Subject: [Python-bugs-list] [ python-Bugs-680789 ] repr() of large array objects takes quadratic time
Message-ID: <E18ilw3-00068Y-00@sc8-sf-web4.sourceforge.net>

Bugs item #680789, was opened at 2003-02-05 05:00
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=680789&group_id=5470

Category: Python Library
Group: Python 2.2
Status: Open
Resolution: None
Priority: 5
Submitted By: Jurjen N.E. Bos (jneb)
Assigned to: Nobody/Anonymous (nobody)
Summary: repr() of large array objects takes quadratic time

Initial Comment:
This is a bug and a partial patch.

If I debug a program that contains a ridiculously large array (8M entries in my case), the debugger takes forever.
It happens in Mac OS X, Python 2.2, but I found the bug in is the repr module, so it is probably universal.
The thing is, that after the fix below, it still doesn't work!
Did I miss something trivial (like repr is builtin, or something like that?). Would someone with Mac OS X experience help out here, please (Jack?).

Here's the diff to make repr.repr work with large arrays:
13a14
>       self.maxarray = 5
50a52,62
>     def repr_array(self, x, level):
>         n = len(x)
>       header = "array('"+x.typecode+"', ["
>         if n == 0: return header+"])"
>         if level <= 0: return header+"...])"
>         s = ''
>         for i in range(min(n, self.maxarray)):
>             if s: s = s + ', '
>             s = s + self.repr1(x[i], level-1)
>         if n > self.maxarray: s = s + ', ...'
>         return header + s + "])"

----------------------------------------------------------------------

Comment By: logistix (logistix)
Date: 2003-02-11 19:43

Message:
Logged In: YES 
user_id=699438

arraymodule's repr used "string += ',' + el" for each element in 
the array.  Lists and dicts did a string.join to build the repr.

Attached patch builds a tuple of array elements and then 
joins them.  (actually for some reason I can't attach now, I'll 
post the patch in patch manager)

This fixes the time issue, but I don't know enough about how 
you guys manage memory in each case to tell what impact 
that'll have on really, really big arrays (although I imagine it 
takes more memory).



----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2003-02-06 19:40

Message:
Logged In: YES 
user_id=31435

I can't make time for this, so unassigned it.  It would make 
a good, brief project for someone -- the list and dict 
tp_reprs are linear-time, and tp_repr for array.array 
objects shouldn't be any harder than they were.

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-02-06 15:37

Message:
Logged In: YES 
user_id=45365

Okay, so the real bug is that tp_repr of array objects takes quadratic time.

I'm changing the summary of this report then, and assigning back to you (Tim), on the basis that you did more checkins on arraymodule than I did. Feel free to pass the potato on:-)

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2003-02-05 17:08

Message:
Logged In: YES 
user_id=31435

pdb does import repr.py, but probably doesn't use it in 
whatever way Jurjen is using to display his big array.

WRT that, note that Jurjen is using array.array objects, not 
lists.  The internal array.array tp_repr slot is quadratic-time in 
the size of the array, while list's tp_repr is linear time.

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-02-05 16:40

Message:
Logged In: YES 
user_id=45365

The fix is fine (it works for me the same way as for Tim), but I think we're shooting past the problem here.

First, pdb doesn't use repr.repr(), it uses the normal builtin repr().

Second, I don't see any sluggishness in pdb with large arrays. I tried
debugging
def foo():
    a = range(8000000)
and there was no problem. Allocating the object took a bit of time, yes, and if you actually try to print it you'll stare at about 800K lines filled with digits scrolling over your screen, but that is to be expected.

Could it be your sluggishness is coming from something else? For example, MacOSX starts behaving *very* badly if your root disk is full, because then it can't allocate swap space, and due to its optimistic behaviour it comes to a grinding halt.

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2003-02-05 12:36

Message:
Logged In: YES 
user_id=31435

Nice to see you, Jurgen!  I checked this into current CVS, 
and it works fine for me in isolation:

>>> len(a)
11055060
>>> repr.repr(a)
"array('i', [0, 1, 2, 3, 4, ...])"
>>>

That goes in an eyeblink.  So more detail is needed about 
what "it still doesn't work!" means.  Assigned to Jack, and he 
can use current CVS to try it.

Lib/repr.py; new revision: 1.15
Lib/test/test_repr.py; new revision: 1.16
Misc/NEWS; new revision: 1.642

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=680789&group_id=5470


From noreply@sourceforge.net  Wed Feb 12 03:41:06 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 11 Feb 2003 19:41:06 -0800
Subject: [Python-bugs-list] [ python-Bugs-532767 ] isinstance() should respect __class__
Message-ID: <E18inli-0006aQ-00@sc8-sf-web3.sourceforge.net>

Bugs item #532767, was opened at 2002-03-20 17:22
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=532767&group_id=5470

Category: Type/class unification
Group: Python 2.3
>Status: Closed
>Resolution: Accepted
Priority: 6
Submitted By: Steve Alexander (stevea_zope)
Assigned to: Guido van Rossum (gvanrossum)
Summary: isinstance() should respect __class__

Initial Comment:
isinstance(obj, class_or_type_or_tuple) should compare
using obj.__class__ when obj is an instance of a type
or a new-style class.

This is important for using weak references and other
kinds of proxy wrappers, where you want to pass a proxy
to some code, which might query its type using isinstance.

issubclass may need a similar treatment.

----------------------------------------------------------------------

>Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-11 22:41

Message:
Logged In: YES 
user_id=6380

OK, checked in. When X is a new-style class, isinstance(x,
X) is now defined as "issubclass(type(x), X) or
issubclass(x.__class__, X)". That seems a very reasonable
definition to me.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2002-03-21 09:52

Message:
Logged In: YES 
user_id=6380

I'm not sure I like SteveA's suggestion of making
isinstance() respond both to the wrapper type and to the
wrapped type. Although there's a precedent:

>>> class C: pass # classic class
... 
>>> c = C()
>>> isinstance(c, C)
1
>>> import types
>>> isinstance(c, types.InstanceType)
1
>>> 

I'm also not sure I agree with Martin's assertion that
type(o) is X and isinstance(o, X) should always be
equivalent. Especially with new-style user-defined classes,
we could also require that o.__class__ == X and
isinstance(o, X) should be equivalent (modulo subclassing),
and that would require isinstance() to prefer __class__.

I guess we'll have to look at actual use cases of
isinstance() and wrappers (both weakrefs and Zope3's
transparent wrappers). My expectation is that, since
wrappers try hard to pretend to be the wrapped object,
extending this to the isinstance() test is more useful than
the stricter interpretation.

----------------------------------------------------------------------

Comment By: Steve Alexander (stevea_zope)
Date: 2002-03-21 08:57

Message:
Logged In: YES 
user_id=492001

oops... please ignore my unsupported use of "is" to compare
small ints.

I meant:

from Zope.ContextWrapper import Wrapper

wl = Wrapper([])
assert isinstance(wl, list) == 1
assert isinstance(wl, Wrapper) == 1



----------------------------------------------------------------------

Comment By: Steve Alexander (stevea_zope)
Date: 2002-03-21 08:55

Message:
Logged In: YES 
user_id=492001

A new isinstance can maintain and extend the semantic Martin
describes.

Let's say object wl is a wrapped list:

from Zope.ContextWrapper import Wrapper

wl = Wrapper([])
assert isinstance(wl, list) is 1
assert isinstance(wl, Wrapper) is 1

So, your semantics are maintained. With the proposed change,
the property you describe need not be given up.

----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2002-03-21 08:47

Message:
Logged In: YES 
user_id=21627

There has been a long-standing guarantee that 'type(o) is X'
implies 'isinstance(o, X)', or, more compact,
'isinstance(o,type(o))' for all objects o. In fact, people
have been advised to change the explicit test for type() to
isinstance calls. With the proposed change, this property
will be given up.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2002-03-21 08:16

Message:
Logged In: YES 
user_id=6380

I give this a +1. To refute Martin's -1: The use case that
prompts this is passing a wrapper to wrapper-ignorant code.
This may be 3rd party code that you can't afford to make
wrapper-aware. If I pass you a wrapper to an X where you
expect an X, your isinstance(x, X) call should succeed.
Especially since x.__class__ already returns X. Also,
isinstance(x, X) succeeds if X is a classic class and x is a
wrapped X instance.

If you want to know if something is a wrapper, you have to
use type().

----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2002-03-21 03:59

Message:
Logged In: YES 
user_id=21627

-1. That means that you can't use isinstance anymore to
determine whether something is a weak reference, or other
kind of proxy wrapper.

If you need a function that unwraps wrappers, write one
yourself.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=532767&group_id=5470


From noreply@sourceforge.net  Wed Feb 12 13:13:39 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Wed, 12 Feb 2003 05:13:39 -0800
Subject: [Python-bugs-list] [ python-Bugs-663074 ] codec registry and Python embedding problem
Message-ID: <E18iwhn-0002wH-00@sc8-sf-web3.sourceforge.net>

Bugs item #663074, was opened at 2003-01-06 12:17
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=663074&group_id=5470

Category: Python Interpreter Core
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: M.-A. Lemburg (lemburg)
>Assigned to: Gustavo Niemeyer (niemeyer)
Summary: codec registry and Python embedding problem

Initial Comment:
Found by Hartmut Ring:

#include <Python.h>
int main() {
  for (int i=0; i<2; i++) {
    Py_Initialize();
    PyRun_SimpleString("a = u'\xe4'.encode('Latin-1')");
    Py_Finalize();
  }
  return 0;
}

First try runs fine, second try gives:

Traceback (most recent call last):
  File "<string>", line 1, in ?
LookupError: no codec search functions registered:
can't find encoding



----------------------------------------------------------------------

>Comment By: Gustavo Niemeyer (niemeyer)
Date: 2003-02-12 13:13

Message:
Logged In: YES 
user_id=7887

As discussed in python-dev, I'll work on that.

----------------------------------------------------------------------

Comment By: M.-A. Lemburg (lemburg)
Date: 2003-01-22 12:12

Message:
Logged In: YES 
user_id=38388

Good suggestion. Do you have time to cook up a patch ?

----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2003-01-22 10:07

Message:
Logged In: YES 
user_id=21627

I think the various global variables maintained in codecs.c
(_PyCodec_SearchCache, import_encodings_called, ...) need to
become part of the interpreter state. Otherwise, a codec
registered in one interpreter will be found in another.

Also, it appears that import_encodings_called can be
eliminated. Instead, a NULL value of _PyCodec_SearchPath
could be used to indicate whether initialization has taken
place. 

----------------------------------------------------------------------

Comment By: M.-A. Lemburg (lemburg)
Date: 2003-01-06 12:18

Message:
Logged In: YES 
user_id=38388

This is due to a flag used in codecs.c:

/* Flag used for lazy import of the standard encodings
package */
static int import_encodings_called = 0;

The solution is to reset this flag in Py_Finalize().

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=663074&group_id=5470


From noreply@sourceforge.net  Wed Feb 12 14:16:41 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Wed, 12 Feb 2003 06:16:41 -0800
Subject: [Python-bugs-list] [ python-Bugs-684903 ] socket module on solaris
Message-ID: <E18ixgn-0007Cz-00@sc8-sf-web4.sourceforge.net>

Bugs item #684903, was opened at 2003-02-11 21:02
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=684903&group_id=5470

Category: Build
Group: Python 2.2.2
Status: Open
Resolution: None
Priority: 5
Submitted By: RIck Bradshaw (rabrads)
Assigned to: Nobody/Anonymous (nobody)
Summary: socket module on solaris

Initial Comment:
I just built python 2.2.2 on solaris 7 or 2.7 to be
technically correct.  I then fire up the interpreter
and import test.testall. It fails for socket, which is
very import to me and my users. I then import _socket
and get the following info

Python 2.2.2 (#4, Feb  6 2003, 17:17:38) [C] on sunos5
Type "help", "copyright", "credits" or "license" for
more information.
>>> import _socket
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
ImportError: ld.so.1:
/soft/apps/packages/Python-2.2.2/bin/python: fatal:
relocation error: file
/soft/apps/packages/Python-2.2.2/lib/python2.2/_socket.so:
symbol __eprintf: referenced symbol not found

Thanks for any help.
Rick

----------------------------------------------------------------------

>Comment By: Michael Hudson (mwh)
Date: 2003-02-12 14:16

Message:
Logged In: YES 
user_id=6656

IIRC this usually happens when you compile with gcc and link
with the system ld or vice versa or something like that.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-11 23:25

Message:
Logged In: YES 
user_id=33168

This error looks familiar, but I don't recall the cause. 
Did you have a working version of python on this machine? 
What compiler and version are you using?  Is __eprintf in a
library (libc.a)?  Is that not getting linked?  Is it using
the correct linker and library locations?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=684903&group_id=5470


From noreply@sourceforge.net  Wed Feb 12 14:36:07 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Wed, 12 Feb 2003 06:36:07 -0800
Subject: [Python-bugs-list] [ python-Bugs-681960 ] Source encoding rules are extreme.
Message-ID: <E18ixzb-0008P1-00@sc8-sf-web4.sourceforge.net>

Bugs item #681960, was opened at 2003-02-07 01:17
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=681960&group_id=5470

Category: Unicode
Group: Python 2.3
Status: Closed
Resolution: None
Priority: 3
Submitted By: Kirill Simonov (kirill_simonov)
Assigned to: M.-A. Lemburg (lemburg)
Summary: Source encoding rules are extreme.

Initial Comment:
According to the PEP 0263, a source code that contains
non-ASCII
characters (ord(ch)>127) and does not define an
encoding causes
DeprecationWarning. In the future, such code will cause
SyntaxError.

While I believe that the idea of defining source code
encoding is very
useful, I think that the current solution is
unnecessary extreme.

It is very unfriendly for beginners. Imagine a student that
types her first script:

name = raw_input("What's your name? ")   # russian
here, of course
print "Hi %s!" % name

Do not even try to convince me that she must define an
encoding
here. That feature would break any possibility to use
Python in schools.

Actually the source code encoding only affects Unicode
literals.
The above script works the same way with any defined
encoding,
so the warning for this code is unnecessary.

As a solution, I propose to issue DeprecationWarning
(or SyntaxError)
only when a non-ASCII character is contained in a
Unicode literal.


----------------------------------------------------------------------

Comment By: Denis S. Otkidach (ods)
Date: 2003-02-12 17:36

Message:
Logged In: YES 
user_id=63454

8-bit string in Python is just a stream of bytes now. Why should I specify 
encoding for inline image data for instance? And what encoding should I 
use?

----------------------------------------------------------------------

Comment By: Kirill Simonov (kirill_simonov)
Date: 2003-02-10 18:39

Message:
Logged In: YES 
user_id=36553

I like this. Thanks.

----------------------------------------------------------------------

Comment By: M.-A. Lemburg (lemburg)
Date: 2003-02-10 12:43

Message:
Logged In: YES 
user_id=38388

I've had a private discussion with Guido and Roman Suzi:

We'll add a way to set the source code default encoding via the
site.py/sitecustomize.py files. This should then allow anyone
wishing to customize the default behaviour to do so.

----------------------------------------------------------------------

Comment By: Kirill Simonov (kirill_simonov)
Date: 2003-02-07 02:28

Message:
Logged In: YES 
user_id=36553

Hello,

Yes, I understand that the encoding is for the whole source
file.

But

1. The current implementation already assumes that one uses
ASCII-
compatible encoding. So we can make a step further and do
not use any
encoding while reading a source file. And then we'll
translate u"..." using
'ascii' encoding.

2. How do you want to support UTF-16 encoding? This will
completely
break ordinary string literals! "aa" is a source code would
become "a\x00a\x00" after compilation. Or do I miss something?

3. Do not forget that your change breaks billions of scripts
that use
non-ASCII characters even in comments!

4. I can write a patch. I would be forced to do this anyway.



----------------------------------------------------------------------

Comment By: M.-A. Lemburg (lemburg)
Date: 2003-02-07 01:45

Message:
Logged In: YES 
user_id=38388

Sorry, but the implementation we chose decodes the complete
file,
not only the Unicode literals, so if you want to use a specific 
encoding in the source code, you have to be explicit about it.

Python's source code was originally never meant to contain
non-ASCII characters. The PEP implementation now officially
allows this provided that you use an encoding marker, e.g.

"""
# -*- coding: windows-1251 -*-
name = raw_input("   ? ")
print " %s" % name
"""

Note that this is also needed in order to support UTF-16
file formats which use two bytes per character. Python
will automatically detect these files, so if you really don't
like the coding marker, simply write the file using a UTF-16
aware editor which prepends a UTF-16 BOM mark to the
file.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=681960&group_id=5470


From noreply@sourceforge.net  Wed Feb 12 14:49:04 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Wed, 12 Feb 2003 06:49:04 -0800
Subject: [Python-bugs-list] [ python-Bugs-681960 ] Source encoding rules are extreme.
Message-ID: <E18iyC8-0004yW-00@sc8-sf-web2.sourceforge.net>

Bugs item #681960, was opened at 2003-02-06 23:17
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=681960&group_id=5470

Category: Unicode
Group: Python 2.3
Status: Closed
Resolution: None
Priority: 3
Submitted By: Kirill Simonov (kirill_simonov)
Assigned to: M.-A. Lemburg (lemburg)
Summary: Source encoding rules are extreme.

Initial Comment:
According to the PEP 0263, a source code that contains
non-ASCII
characters (ord(ch)>127) and does not define an
encoding causes
DeprecationWarning. In the future, such code will cause
SyntaxError.

While I believe that the idea of defining source code
encoding is very
useful, I think that the current solution is
unnecessary extreme.

It is very unfriendly for beginners. Imagine a student that
types her first script:

name = raw_input("What's your name? ")   # russian
here, of course
print "Hi %s!" % name

Do not even try to convince me that she must define an
encoding
here. That feature would break any possibility to use
Python in schools.

Actually the source code encoding only affects Unicode
literals.
The above script works the same way with any defined
encoding,
so the warning for this code is unnecessary.

As a solution, I propose to issue DeprecationWarning
(or SyntaxError)
only when a non-ASCII character is contained in a
Unicode literal.


----------------------------------------------------------------------

>Comment By: M.-A. Lemburg (lemburg)
Date: 2003-02-12 15:49

Message:
Logged In: YES 
user_id=38388

Encode the 8-bit data as base64 value and put that into the
source code.

----------------------------------------------------------------------

Comment By: Denis S. Otkidach (ods)
Date: 2003-02-12 15:36

Message:
Logged In: YES 
user_id=63454

8-bit string in Python is just a stream of bytes now. Why should I specify 
encoding for inline image data for instance? And what encoding should I 
use?

----------------------------------------------------------------------

Comment By: Kirill Simonov (kirill_simonov)
Date: 2003-02-10 16:39

Message:
Logged In: YES 
user_id=36553

I like this. Thanks.

----------------------------------------------------------------------

Comment By: M.-A. Lemburg (lemburg)
Date: 2003-02-10 10:43

Message:
Logged In: YES 
user_id=38388

I've had a private discussion with Guido and Roman Suzi:

We'll add a way to set the source code default encoding via the
site.py/sitecustomize.py files. This should then allow anyone
wishing to customize the default behaviour to do so.

----------------------------------------------------------------------

Comment By: Kirill Simonov (kirill_simonov)
Date: 2003-02-07 00:28

Message:
Logged In: YES 
user_id=36553

Hello,

Yes, I understand that the encoding is for the whole source
file.

But

1. The current implementation already assumes that one uses
ASCII-
compatible encoding. So we can make a step further and do
not use any
encoding while reading a source file. And then we'll
translate u"..." using
'ascii' encoding.

2. How do you want to support UTF-16 encoding? This will
completely
break ordinary string literals! "aa" is a source code would
become "a\x00a\x00" after compilation. Or do I miss something?

3. Do not forget that your change breaks billions of scripts
that use
non-ASCII characters even in comments!

4. I can write a patch. I would be forced to do this anyway.



----------------------------------------------------------------------

Comment By: M.-A. Lemburg (lemburg)
Date: 2003-02-06 23:45

Message:
Logged In: YES 
user_id=38388

Sorry, but the implementation we chose decodes the complete
file,
not only the Unicode literals, so if you want to use a specific 
encoding in the source code, you have to be explicit about it.

Python's source code was originally never meant to contain
non-ASCII characters. The PEP implementation now officially
allows this provided that you use an encoding marker, e.g.

"""
# -*- coding: windows-1251 -*-
name = raw_input("   ? ")
print " %s" % name
"""

Note that this is also needed in order to support UTF-16
file formats which use two bytes per character. Python
will automatically detect these files, so if you really don't
like the coding marker, simply write the file using a UTF-16
aware editor which prepends a UTF-16 BOM mark to the
file.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=681960&group_id=5470


From noreply@sourceforge.net  Wed Feb 12 15:05:11 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Wed, 12 Feb 2003 07:05:11 -0800
Subject: [Python-bugs-list] [ python-Bugs-681960 ] Source encoding rules are extreme.
Message-ID: <E18iyRj-0005xn-00@sc8-sf-web2.sourceforge.net>

Bugs item #681960, was opened at 2003-02-07 01:17
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=681960&group_id=5470

Category: Unicode
Group: Python 2.3
Status: Closed
Resolution: None
Priority: 3
Submitted By: Kirill Simonov (kirill_simonov)
Assigned to: M.-A. Lemburg (lemburg)
Summary: Source encoding rules are extreme.

Initial Comment:
According to the PEP 0263, a source code that contains
non-ASCII
characters (ord(ch)>127) and does not define an
encoding causes
DeprecationWarning. In the future, such code will cause
SyntaxError.

While I believe that the idea of defining source code
encoding is very
useful, I think that the current solution is
unnecessary extreme.

It is very unfriendly for beginners. Imagine a student that
types her first script:

name = raw_input("What's your name? ")   # russian
here, of course
print "Hi %s!" % name

Do not even try to convince me that she must define an
encoding
here. That feature would break any possibility to use
Python in schools.

Actually the source code encoding only affects Unicode
literals.
The above script works the same way with any defined
encoding,
so the warning for this code is unnecessary.

As a solution, I propose to issue DeprecationWarning
(or SyntaxError)
only when a non-ASCII character is contained in a
Unicode literal.


----------------------------------------------------------------------

Comment By: Denis S. Otkidach (ods)
Date: 2003-02-12 18:05

Message:
Logged In: YES 
user_id=63454

Hmm... There no type for byte streams in Python anymore? Too much to 
change in existing code.  Base64 is not the best solution - too many 
unwanted and slow operations. There are too many areas where we need 
literals for binary data. One more example: translation tables for different 
encodings. Yea, I know about unicode/encode/decode etc, but they are 
_very_ slow for many applications. Use map(ord, [...list of ints...])?

----------------------------------------------------------------------

Comment By: M.-A. Lemburg (lemburg)
Date: 2003-02-12 17:49

Message:
Logged In: YES 
user_id=38388

Encode the 8-bit data as base64 value and put that into the
source code.

----------------------------------------------------------------------

Comment By: Denis S. Otkidach (ods)
Date: 2003-02-12 17:36

Message:
Logged In: YES 
user_id=63454

8-bit string in Python is just a stream of bytes now. Why should I specify 
encoding for inline image data for instance? And what encoding should I 
use?

----------------------------------------------------------------------

Comment By: Kirill Simonov (kirill_simonov)
Date: 2003-02-10 18:39

Message:
Logged In: YES 
user_id=36553

I like this. Thanks.

----------------------------------------------------------------------

Comment By: M.-A. Lemburg (lemburg)
Date: 2003-02-10 12:43

Message:
Logged In: YES 
user_id=38388

I've had a private discussion with Guido and Roman Suzi:

We'll add a way to set the source code default encoding via the
site.py/sitecustomize.py files. This should then allow anyone
wishing to customize the default behaviour to do so.

----------------------------------------------------------------------

Comment By: Kirill Simonov (kirill_simonov)
Date: 2003-02-07 02:28

Message:
Logged In: YES 
user_id=36553

Hello,

Yes, I understand that the encoding is for the whole source
file.

But

1. The current implementation already assumes that one uses
ASCII-
compatible encoding. So we can make a step further and do
not use any
encoding while reading a source file. And then we'll
translate u"..." using
'ascii' encoding.

2. How do you want to support UTF-16 encoding? This will
completely
break ordinary string literals! "aa" is a source code would
become "a\x00a\x00" after compilation. Or do I miss something?

3. Do not forget that your change breaks billions of scripts
that use
non-ASCII characters even in comments!

4. I can write a patch. I would be forced to do this anyway.



----------------------------------------------------------------------

Comment By: M.-A. Lemburg (lemburg)
Date: 2003-02-07 01:45

Message:
Logged In: YES 
user_id=38388

Sorry, but the implementation we chose decodes the complete
file,
not only the Unicode literals, so if you want to use a specific 
encoding in the source code, you have to be explicit about it.

Python's source code was originally never meant to contain
non-ASCII characters. The PEP implementation now officially
allows this provided that you use an encoding marker, e.g.

"""
# -*- coding: windows-1251 -*-
name = raw_input("   ? ")
print " %s" % name
"""

Note that this is also needed in order to support UTF-16
file formats which use two bytes per character. Python
will automatically detect these files, so if you really don't
like the coding marker, simply write the file using a UTF-16
aware editor which prepends a UTF-16 BOM mark to the
file.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=681960&group_id=5470


From noreply@sourceforge.net  Wed Feb 12 15:28:26 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Wed, 12 Feb 2003 07:28:26 -0800
Subject: [Python-bugs-list] [ python-Bugs-681960 ] Source encoding rules are extreme.
Message-ID: <E18iyoE-00072i-00@sc8-sf-web2.sourceforge.net>

Bugs item #681960, was opened at 2003-02-06 23:17
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=681960&group_id=5470

Category: Unicode
Group: Python 2.3
Status: Closed
Resolution: None
Priority: 3
Submitted By: Kirill Simonov (kirill_simonov)
Assigned to: M.-A. Lemburg (lemburg)
Summary: Source encoding rules are extreme.

Initial Comment:
According to the PEP 0263, a source code that contains
non-ASCII
characters (ord(ch)>127) and does not define an
encoding causes
DeprecationWarning. In the future, such code will cause
SyntaxError.

While I believe that the idea of defining source code
encoding is very
useful, I think that the current solution is
unnecessary extreme.

It is very unfriendly for beginners. Imagine a student that
types her first script:

name = raw_input("What's your name? ")   # russian
here, of course
print "Hi %s!" % name

Do not even try to convince me that she must define an
encoding
here. That feature would break any possibility to use
Python in schools.

Actually the source code encoding only affects Unicode
literals.
The above script works the same way with any defined
encoding,
so the warning for this code is unnecessary.

As a solution, I propose to issue DeprecationWarning
(or SyntaxError)
only when a non-ASCII character is contained in a
Unicode literal.


----------------------------------------------------------------------

>Comment By: M.-A. Lemburg (lemburg)
Date: 2003-02-12 16:28

Message:
Logged In: YES 
user_id=38388

You shouldn't put binary data into Python source files 
to begin with. If you absolutely must, then base64 provides
a good start for an ASCII-encoding. The other alternative
is using Python octal escapes. Both are fast.

I don't know where you get the idea from that 
encode/decode are slow. They are certainly faster than
first building a list of ints in memory and then applying
map() to the list.

----------------------------------------------------------------------

Comment By: Denis S. Otkidach (ods)
Date: 2003-02-12 16:05

Message:
Logged In: YES 
user_id=63454

Hmm... There no type for byte streams in Python anymore? Too much to 
change in existing code.  Base64 is not the best solution - too many 
unwanted and slow operations. There are too many areas where we need 
literals for binary data. One more example: translation tables for different 
encodings. Yea, I know about unicode/encode/decode etc, but they are 
_very_ slow for many applications. Use map(ord, [...list of ints...])?

----------------------------------------------------------------------

Comment By: M.-A. Lemburg (lemburg)
Date: 2003-02-12 15:49

Message:
Logged In: YES 
user_id=38388

Encode the 8-bit data as base64 value and put that into the
source code.

----------------------------------------------------------------------

Comment By: Denis S. Otkidach (ods)
Date: 2003-02-12 15:36

Message:
Logged In: YES 
user_id=63454

8-bit string in Python is just a stream of bytes now. Why should I specify 
encoding for inline image data for instance? And what encoding should I 
use?

----------------------------------------------------------------------

Comment By: Kirill Simonov (kirill_simonov)
Date: 2003-02-10 16:39

Message:
Logged In: YES 
user_id=36553

I like this. Thanks.

----------------------------------------------------------------------

Comment By: M.-A. Lemburg (lemburg)
Date: 2003-02-10 10:43

Message:
Logged In: YES 
user_id=38388

I've had a private discussion with Guido and Roman Suzi:

We'll add a way to set the source code default encoding via the
site.py/sitecustomize.py files. This should then allow anyone
wishing to customize the default behaviour to do so.

----------------------------------------------------------------------

Comment By: Kirill Simonov (kirill_simonov)
Date: 2003-02-07 00:28

Message:
Logged In: YES 
user_id=36553

Hello,

Yes, I understand that the encoding is for the whole source
file.

But

1. The current implementation already assumes that one uses
ASCII-
compatible encoding. So we can make a step further and do
not use any
encoding while reading a source file. And then we'll
translate u"..." using
'ascii' encoding.

2. How do you want to support UTF-16 encoding? This will
completely
break ordinary string literals! "aa" is a source code would
become "a\x00a\x00" after compilation. Or do I miss something?

3. Do not forget that your change breaks billions of scripts
that use
non-ASCII characters even in comments!

4. I can write a patch. I would be forced to do this anyway.



----------------------------------------------------------------------

Comment By: M.-A. Lemburg (lemburg)
Date: 2003-02-06 23:45

Message:
Logged In: YES 
user_id=38388

Sorry, but the implementation we chose decodes the complete
file,
not only the Unicode literals, so if you want to use a specific 
encoding in the source code, you have to be explicit about it.

Python's source code was originally never meant to contain
non-ASCII characters. The PEP implementation now officially
allows this provided that you use an encoding marker, e.g.

"""
# -*- coding: windows-1251 -*-
name = raw_input("   ? ")
print " %s" % name
"""

Note that this is also needed in order to support UTF-16
file formats which use two bytes per character. Python
will automatically detect these files, so if you really don't
like the coding marker, simply write the file using a UTF-16
aware editor which prepends a UTF-16 BOM mark to the
file.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=681960&group_id=5470


From noreply@sourceforge.net  Wed Feb 12 15:50:35 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Wed, 12 Feb 2003 07:50:35 -0800
Subject: [Python-bugs-list] [ python-Bugs-625728 ] sys.executable in applets
Message-ID: <E18iz9f-0001zS-00@sc8-sf-web3.sourceforge.net>

Bugs item #625728, was opened at 2002-10-19 23:18
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=625728&group_id=5470

Category: Macintosh
Group: None
>Status: Closed
>Resolution: Fixed
Priority: 5
Submitted By: Jack Jansen (jackjansen)
Assigned to: Just van Rossum (jvr)
Summary: sys.executable in applets

Initial Comment:
In applets sys.executable is the interpreter in the applet. Running this will launch a second copy of the applet. As sys.executable is apparently often used to fire up external scripts with system() or popen() this isn't very useful right now (especially within the IDE!).

It would be nice if sys.executable could be set to an interpreter that can be used for starting external scripts.

----------------------------------------------------------------------

>Comment By: Jack Jansen (jackjansen)
Date: 2003-02-12 16:50

Message:
Logged In: YES 
user_id=45365

This has been fixed in CVS.

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-01-04 22:23

Message:
Logged In: YES 
user_id=45365

Agreed. I'm assigning it to you, then you can close the bug report once you've modified BuildApplet to use bundlebuilder in MacPython-OSX. :-)

----------------------------------------------------------------------

Comment By: Just van Rossum (jvr)
Date: 2003-01-04 11:32

Message:
Logged In: YES 
user_id=92689

This becomes a non-issue once we build applets with
bundlebuilder.py, as the "executable" in the applet is a
shell script, and sys.executable will actually be the proper
executable. For standalone apps built with bundlebuilder.py
sys.executable points to the (still vanilla) interpreter
that was copied to the bundle, and can be used as a normal
interpreter without invoking bundle magic.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=625728&group_id=5470


From noreply@sourceforge.net  Wed Feb 12 15:51:56 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Wed, 12 Feb 2003 07:51:56 -0800
Subject: [Python-bugs-list] [ python-Bugs-685373 ] bundlebuilder and spaces in filenames
Message-ID: <E18izAy-00025d-00@sc8-sf-web3.sourceforge.net>

Bugs item #685373, was opened at 2003-02-12 16:51
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=685373&group_id=5470

Category: Macintosh
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Jack Jansen (jackjansen)
Assigned to: Just van Rossum (jvr)
Summary: bundlebuilder and spaces in filenames

Initial Comment:
If there is a space in your main program filename then bundlebuilder creates a non-functional applet.

Easiest seen by doing "save as applet" in the IDE on an unnamed script.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=685373&group_id=5470


From noreply@sourceforge.net  Wed Feb 12 15:58:47 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Wed, 12 Feb 2003 07:58:47 -0800
Subject: [Python-bugs-list] [ python-Bugs-660455 ] -hex/oct const generates wrong code
Message-ID: <E18izHb-0000bY-00@sc8-sf-web2.sourceforge.net>

Bugs item #660455, was opened at 2002-12-31 13:11
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=660455&group_id=5470

Category: Parser/Compiler
Group: Python 2.3
Status: Open
Resolution: None
Priority: 7
Submitted By: Guido van Rossum (gvanrossum)
>Assigned to: Guido van Rossum (gvanrossum)
Summary: -hex/oct const generates wrong code

Initial Comment:
Since Python 2.2, there's code in com_factor() in
compile.c that expands negative constants inline. For
example, the expression -1 generates a single
LOAD_CONST opcode for -1, rather than (as in Python 2.1
and before) generating a LOAD_CONST for +1 followed by
a UNARY_NEGATIVE opcode.

Unfortunately there's an end case where this causes
surprising behavior: when the constant is an octal or
hexadecimal constant that already has a negative value
(despite having no sign), the generated code yields a
different result than expected.  For example:

>>> print -(0xffffffff)
1
>>> x = 0xffffffff 
>>> print -x
1
>>> print -0xffffffff 
-4294967295
>>> 

Despite the fact that in Python 2.4 (!) all of these
will print the third outcome, in Python 2.2 and 2.3, I
think the third outcome is wrong and unexpected.

No time to fix this for 2.3a1, but should fix it before
2.3a2.

----------------------------------------------------------------------

>Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-12 10:58

Message:
Logged In: YES 
user_id=6380

I've decided to fix this to be compatible with Python 2.1;
I'll hvae to backport the fix to 2.2.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-05 13:24

Message:
Logged In: YES 
user_id=6380

Hmm...  There's a philosophical issue here. I'll post to
python-dev (if anyone still reads that).

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2003-01-17 05:47

Message:
Logged In: YES 
user_id=6656

Oh, so maybe the fact that Lib/compiler has a problem in the
same place *isn't* a problem with the compiler package... 

What ever gets done here should probably coincide with
whatever happens in patch 661536.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-01-16 14:53

Message:
Logged In: YES 
user_id=6380

I believe that test relies on the bug. :-(

So the test should be fixed.

The value ought to be equal to -(020000000000L).

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-01-16 14:45

Message:
Logged In: YES 
user_id=33168

The test fails because of line test_grammar.py:40:

  if -2147483647-1 != -020000000000:

What should -020000000000 be?
Without the -, it is:  -(2**31)-1, so it can't be
represented in an int (on 32-bit arches).
With the patch, it is: 2147483648L.

Later, I'll take a look at doing what Guido suggested.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-01-16 13:38

Message:
Logged In: YES 
user_id=33168

mwh pointed out that my patch breaks test_grammar.  I'll
take another look later, unless Raymond beats me to it.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-01-16 12:32

Message:
Logged In: YES 
user_id=6380

I'd like to keep the optimization too.

I think you have to do something different. The code
currently tries to save some time by prepending a minus sign
to the literal. I think it should just evaluate the literal
and then apply PyNumber_Negative() to the result, all the
while catching exceptions and falling back to unoptimized
code if there are exceptions.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-01-16 10:18

Message:
Logged In: YES 
user_id=33168

Ideally, I'd like to keep the code, rather than rip it out.
 Is it possible to keep the optimization, but if it is -
followed by a 0, then fallback to the original technique
(ie, UNARY_NEGATIVE, followed by the value)?

I have attached a patch which seems to implement this.

----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2003-01-12 06:35

Message:
Logged In: YES 
user_id=80475

See attached patch.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=660455&group_id=5470


From noreply@sourceforge.net  Wed Feb 12 16:24:54 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Wed, 12 Feb 2003 08:24:54 -0800
Subject: [Python-bugs-list] [ python-Bugs-681960 ] Source encoding rules are extreme.
Message-ID: <E18izgs-0001tH-00@sc8-sf-web2.sourceforge.net>

Bugs item #681960, was opened at 2003-02-07 01:17
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=681960&group_id=5470

Category: Unicode
Group: Python 2.3
Status: Closed
Resolution: None
Priority: 3
Submitted By: Kirill Simonov (kirill_simonov)
Assigned to: M.-A. Lemburg (lemburg)
Summary: Source encoding rules are extreme.

Initial Comment:
According to the PEP 0263, a source code that contains
non-ASCII
characters (ord(ch)>127) and does not define an
encoding causes
DeprecationWarning. In the future, such code will cause
SyntaxError.

While I believe that the idea of defining source code
encoding is very
useful, I think that the current solution is
unnecessary extreme.

It is very unfriendly for beginners. Imagine a student that
types her first script:

name = raw_input("What's your name? ")   # russian
here, of course
print "Hi %s!" % name

Do not even try to convince me that she must define an
encoding
here. That feature would break any possibility to use
Python in schools.

Actually the source code encoding only affects Unicode
literals.
The above script works the same way with any defined
encoding,
so the warning for this code is unnecessary.

As a solution, I propose to issue DeprecationWarning
(or SyntaxError)
only when a non-ASCII character is contained in a
Unicode literal.


----------------------------------------------------------------------

Comment By: Denis S. Otkidach (ods)
Date: 2003-02-12 19:24

Message:
Logged In: YES 
user_id=63454

encode/decode is slow compared to translate. Octal/hexadecimal escapes 
are OK. I've noticed that defining arbitrary encoding of source allows 
arbitrary binary data in stings (a bit ugly, but is OK when this setting is 
hidden in site.py), so there is no problem even for old code.

----------------------------------------------------------------------

Comment By: M.-A. Lemburg (lemburg)
Date: 2003-02-12 18:28

Message:
Logged In: YES 
user_id=38388

You shouldn't put binary data into Python source files 
to begin with. If you absolutely must, then base64 provides
a good start for an ASCII-encoding. The other alternative
is using Python octal escapes. Both are fast.

I don't know where you get the idea from that 
encode/decode are slow. They are certainly faster than
first building a list of ints in memory and then applying
map() to the list.

----------------------------------------------------------------------

Comment By: Denis S. Otkidach (ods)
Date: 2003-02-12 18:05

Message:
Logged In: YES 
user_id=63454

Hmm... There no type for byte streams in Python anymore? Too much to 
change in existing code.  Base64 is not the best solution - too many 
unwanted and slow operations. There are too many areas where we need 
literals for binary data. One more example: translation tables for different 
encodings. Yea, I know about unicode/encode/decode etc, but they are 
_very_ slow for many applications. Use map(ord, [...list of ints...])?

----------------------------------------------------------------------

Comment By: M.-A. Lemburg (lemburg)
Date: 2003-02-12 17:49

Message:
Logged In: YES 
user_id=38388

Encode the 8-bit data as base64 value and put that into the
source code.

----------------------------------------------------------------------

Comment By: Denis S. Otkidach (ods)
Date: 2003-02-12 17:36

Message:
Logged In: YES 
user_id=63454

8-bit string in Python is just a stream of bytes now. Why should I specify 
encoding for inline image data for instance? And what encoding should I 
use?

----------------------------------------------------------------------

Comment By: Kirill Simonov (kirill_simonov)
Date: 2003-02-10 18:39

Message:
Logged In: YES 
user_id=36553

I like this. Thanks.

----------------------------------------------------------------------

Comment By: M.-A. Lemburg (lemburg)
Date: 2003-02-10 12:43

Message:
Logged In: YES 
user_id=38388

I've had a private discussion with Guido and Roman Suzi:

We'll add a way to set the source code default encoding via the
site.py/sitecustomize.py files. This should then allow anyone
wishing to customize the default behaviour to do so.

----------------------------------------------------------------------

Comment By: Kirill Simonov (kirill_simonov)
Date: 2003-02-07 02:28

Message:
Logged In: YES 
user_id=36553

Hello,

Yes, I understand that the encoding is for the whole source
file.

But

1. The current implementation already assumes that one uses
ASCII-
compatible encoding. So we can make a step further and do
not use any
encoding while reading a source file. And then we'll
translate u"..." using
'ascii' encoding.

2. How do you want to support UTF-16 encoding? This will
completely
break ordinary string literals! "aa" is a source code would
become "a\x00a\x00" after compilation. Or do I miss something?

3. Do not forget that your change breaks billions of scripts
that use
non-ASCII characters even in comments!

4. I can write a patch. I would be forced to do this anyway.



----------------------------------------------------------------------

Comment By: M.-A. Lemburg (lemburg)
Date: 2003-02-07 01:45

Message:
Logged In: YES 
user_id=38388

Sorry, but the implementation we chose decodes the complete
file,
not only the Unicode literals, so if you want to use a specific 
encoding in the source code, you have to be explicit about it.

Python's source code was originally never meant to contain
non-ASCII characters. The PEP implementation now officially
allows this provided that you use an encoding marker, e.g.

"""
# -*- coding: windows-1251 -*-
name = raw_input("   ? ")
print " %s" % name
"""

Note that this is also needed in order to support UTF-16
file formats which use two bytes per character. Python
will automatically detect these files, so if you really don't
like the coding marker, simply write the file using a UTF-16
aware editor which prepends a UTF-16 BOM mark to the
file.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=681960&group_id=5470


From noreply@sourceforge.net  Wed Feb 12 16:27:24 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Wed, 12 Feb 2003 08:27:24 -0800
Subject: [Python-bugs-list] [ python-Bugs-685373 ] bundlebuilder and spaces in filenames
Message-ID: <E18izjI-0006D8-00@sc8-sf-web4.sourceforge.net>

Bugs item #685373, was opened at 2003-02-12 16:51
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=685373&group_id=5470

Category: Macintosh
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Jack Jansen (jackjansen)
Assigned to: Just van Rossum (jvr)
Summary: bundlebuilder and spaces in filenames

Initial Comment:
If there is a space in your main program filename then bundlebuilder creates a non-functional applet.

Easiest seen by doing "save as applet" in the IDE on an unnamed script.

----------------------------------------------------------------------

>Comment By: Just van Rossum (jvr)
Date: 2003-02-12 17:27

Message:
Logged In: YES 
user_id=92689

Darn, I thought I fixed this, apparently not thoroughly enough... Can you check whether current CVS works for you now?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=685373&group_id=5470


From noreply@sourceforge.net  Wed Feb 12 17:34:49 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Wed, 12 Feb 2003 09:34:49 -0800
Subject: [Python-bugs-list] [ python-Bugs-660455 ] -hex/oct const generates wrong code
Message-ID: <E18j0mX-0006lp-00@sc8-sf-web3.sourceforge.net>

Bugs item #660455, was opened at 2002-12-31 13:11
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=660455&group_id=5470

Category: Parser/Compiler
>Group: Python 2.2.2
Status: Open
Resolution: None
>Priority: 5
Submitted By: Guido van Rossum (gvanrossum)
Assigned to: Guido van Rossum (gvanrossum)
Summary: -hex/oct const generates wrong code

Initial Comment:
Since Python 2.2, there's code in com_factor() in
compile.c that expands negative constants inline. For
example, the expression -1 generates a single
LOAD_CONST opcode for -1, rather than (as in Python 2.1
and before) generating a LOAD_CONST for +1 followed by
a UNARY_NEGATIVE opcode.

Unfortunately there's an end case where this causes
surprising behavior: when the constant is an octal or
hexadecimal constant that already has a negative value
(despite having no sign), the generated code yields a
different result than expected.  For example:

>>> print -(0xffffffff)
1
>>> x = 0xffffffff 
>>> print -x
1
>>> print -0xffffffff 
-4294967295
>>> 

Despite the fact that in Python 2.4 (!) all of these
will print the third outcome, in Python 2.2 and 2.3, I
think the third outcome is wrong and unexpected.

No time to fix this for 2.3a1, but should fix it before
2.3a2.

----------------------------------------------------------------------

>Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-12 12:34

Message:
Logged In: YES 
user_id=6380

All checked in.

Keeping this open because it needs to be backported to 2.2.


----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-12 10:58

Message:
Logged In: YES 
user_id=6380

I've decided to fix this to be compatible with Python 2.1;
I'll hvae to backport the fix to 2.2.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-05 13:24

Message:
Logged In: YES 
user_id=6380

Hmm...  There's a philosophical issue here. I'll post to
python-dev (if anyone still reads that).

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2003-01-17 05:47

Message:
Logged In: YES 
user_id=6656

Oh, so maybe the fact that Lib/compiler has a problem in the
same place *isn't* a problem with the compiler package... 

What ever gets done here should probably coincide with
whatever happens in patch 661536.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-01-16 14:53

Message:
Logged In: YES 
user_id=6380

I believe that test relies on the bug. :-(

So the test should be fixed.

The value ought to be equal to -(020000000000L).

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-01-16 14:45

Message:
Logged In: YES 
user_id=33168

The test fails because of line test_grammar.py:40:

  if -2147483647-1 != -020000000000:

What should -020000000000 be?
Without the -, it is:  -(2**31)-1, so it can't be
represented in an int (on 32-bit arches).
With the patch, it is: 2147483648L.

Later, I'll take a look at doing what Guido suggested.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-01-16 13:38

Message:
Logged In: YES 
user_id=33168

mwh pointed out that my patch breaks test_grammar.  I'll
take another look later, unless Raymond beats me to it.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-01-16 12:32

Message:
Logged In: YES 
user_id=6380

I'd like to keep the optimization too.

I think you have to do something different. The code
currently tries to save some time by prepending a minus sign
to the literal. I think it should just evaluate the literal
and then apply PyNumber_Negative() to the result, all the
while catching exceptions and falling back to unoptimized
code if there are exceptions.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-01-16 10:18

Message:
Logged In: YES 
user_id=33168

Ideally, I'd like to keep the code, rather than rip it out.
 Is it possible to keep the optimization, but if it is -
followed by a 0, then fallback to the original technique
(ie, UNARY_NEGATIVE, followed by the value)?

I have attached a patch which seems to implement this.

----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2003-01-12 06:35

Message:
Logged In: YES 
user_id=80475

See attached patch.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=660455&group_id=5470


From noreply@sourceforge.net  Wed Feb 12 18:58:26 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Wed, 12 Feb 2003 10:58:26 -0800
Subject: [Python-bugs-list] [ python-Bugs-685465 ] python-mode loops on if/else
Message-ID: <E18j25S-0003uy-00@sc8-sf-web1.sourceforge.net>

Bugs item #685465, was opened at 2003-02-12 13:58
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=685465&group_id=5470

Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Guido van Rossum (gvanrossum)
Assigned to: Barry A. Warsaw (bwarsaw)
Summary: python-mode loops on if/else

Initial Comment:
Here's how to reproduce.  Create a .py file in XEmacs
containing this text:

x = (if 1: 2
 ____else: 3)

(except that the ____ really means four spaces).

Now position your cursor somewhere inside those 4
spaces, and hit TAB. XEmacs freezes until you hit ^G.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=685465&group_id=5470


From noreply@sourceforge.net  Wed Feb 12 19:18:05 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Wed, 12 Feb 2003 11:18:05 -0800
Subject: [Python-bugs-list] [ python-Bugs-660455 ] -hex/oct const generates wrong code
Message-ID: <E18j2OT-0001mc-00@sc8-sf-web2.sourceforge.net>

Bugs item #660455, was opened at 2002-12-31 13:11
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=660455&group_id=5470

Category: Parser/Compiler
Group: Python 2.2.2
>Status: Closed
>Resolution: Fixed
Priority: 5
Submitted By: Guido van Rossum (gvanrossum)
Assigned to: Guido van Rossum (gvanrossum)
Summary: -hex/oct const generates wrong code

Initial Comment:
Since Python 2.2, there's code in com_factor() in
compile.c that expands negative constants inline. For
example, the expression -1 generates a single
LOAD_CONST opcode for -1, rather than (as in Python 2.1
and before) generating a LOAD_CONST for +1 followed by
a UNARY_NEGATIVE opcode.

Unfortunately there's an end case where this causes
surprising behavior: when the constant is an octal or
hexadecimal constant that already has a negative value
(despite having no sign), the generated code yields a
different result than expected.  For example:

>>> print -(0xffffffff)
1
>>> x = 0xffffffff 
>>> print -x
1
>>> print -0xffffffff 
-4294967295
>>> 

Despite the fact that in Python 2.4 (!) all of these
will print the third outcome, in Python 2.2 and 2.3, I
think the third outcome is wrong and unexpected.

No time to fix this for 2.3a1, but should fix it before
2.3a2.

----------------------------------------------------------------------

>Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-12 14:18

Message:
Logged In: YES 
user_id=6380

OK, backported to 2.2.3.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-12 12:34

Message:
Logged In: YES 
user_id=6380

All checked in.

Keeping this open because it needs to be backported to 2.2.


----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-12 10:58

Message:
Logged In: YES 
user_id=6380

I've decided to fix this to be compatible with Python 2.1;
I'll hvae to backport the fix to 2.2.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-05 13:24

Message:
Logged In: YES 
user_id=6380

Hmm...  There's a philosophical issue here. I'll post to
python-dev (if anyone still reads that).

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2003-01-17 05:47

Message:
Logged In: YES 
user_id=6656

Oh, so maybe the fact that Lib/compiler has a problem in the
same place *isn't* a problem with the compiler package... 

What ever gets done here should probably coincide with
whatever happens in patch 661536.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-01-16 14:53

Message:
Logged In: YES 
user_id=6380

I believe that test relies on the bug. :-(

So the test should be fixed.

The value ought to be equal to -(020000000000L).

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-01-16 14:45

Message:
Logged In: YES 
user_id=33168

The test fails because of line test_grammar.py:40:

  if -2147483647-1 != -020000000000:

What should -020000000000 be?
Without the -, it is:  -(2**31)-1, so it can't be
represented in an int (on 32-bit arches).
With the patch, it is: 2147483648L.

Later, I'll take a look at doing what Guido suggested.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-01-16 13:38

Message:
Logged In: YES 
user_id=33168

mwh pointed out that my patch breaks test_grammar.  I'll
take another look later, unless Raymond beats me to it.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-01-16 12:32

Message:
Logged In: YES 
user_id=6380

I'd like to keep the optimization too.

I think you have to do something different. The code
currently tries to save some time by prepending a minus sign
to the literal. I think it should just evaluate the literal
and then apply PyNumber_Negative() to the result, all the
while catching exceptions and falling back to unoptimized
code if there are exceptions.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-01-16 10:18

Message:
Logged In: YES 
user_id=33168

Ideally, I'd like to keep the code, rather than rip it out.
 Is it possible to keep the optimization, but if it is -
followed by a 0, then fallback to the original technique
(ie, UNARY_NEGATIVE, followed by the value)?

I have attached a patch which seems to implement this.

----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2003-01-12 06:35

Message:
Logged In: YES 
user_id=80475

See attached patch.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=660455&group_id=5470


From noreply@sourceforge.net  Wed Feb 12 20:45:30 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Wed, 12 Feb 2003 12:45:30 -0800
Subject: [Python-bugs-list] [ python-Bugs-413135 ] urllib2 fails with proxy requiring auth
Message-ID: <E18j3l4-00028p-00@sc8-sf-web4.sourceforge.net>

Bugs item #413135, was opened at 2001-04-02 15:14
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=413135&group_id=5470

Category: Python Library
Group: None
Status: Closed
Resolution: Fixed
Priority: 5
Submitted By: Paul Moore (pmoore)
Assigned to: Moshe Zadka (moshez)
Summary: urllib2 fails with proxy requiring auth

Initial Comment:
The following program:

import urllib2

proxy_info = {
    'user' : 'my_name', 'pass' : 'my_pass',
    'host' : "my-proxy", 'port' : 80
}
 
# build a new opener that uses a proxy requiring
# authorization
proxy_support = urllib2.ProxyHandler(
    {"http" :
     "http://%(user)s:%(pass)s@%(host)s:%(port)d"
     % proxy_info})
opener = urllib2.build_opener(proxy_support, 
urllib2.HTTPHandler)
 
# install it
urllib2.install_opener(opener)
 
f = urllib2.urlopen('http://www.python.org/')
print f.headers
print f.read()


fails with the following error on Python 2.1b2 (on 
Windows)

C:\Data>python21 proxy_auth.py
Traceback (most recent call last):
  File "proxy_auth.py", line 18, in ?
    f = urllib2.urlopen('http://www.python.org/')
  File "c:\applications\python21\lib\urllib2.py", line 
135, in urlopen
    return _opener.open(url, data)
  File "c:\applications\python21\lib\urllib2.py", line 
318, in open
    '_open', req)
  File "c:\applications\python21\lib\urllib2.py", line 
297, in _call_chain
    result = func(*args)
  File "c:\applications\python21\lib\urllib2.py", line 
823, in http_open
    return self.do_open(httplib.HTTP, req)
  File "c:\applications\python21\lib\urllib2.py", line 
801, in do_open
    raise URLError(err)
urllib2.URLError: <urlopen error host not found>

A similar error occurred in beta 1, but this was 
reported as bug 406683. The fix is in beta 2. I 
applied the fix manually in beta 1, and it worked, so 
I can only assume that something else changed in the 
transition from beta 1 to beta 2, which broke this 
again.


----------------------------------------------------------------------

Comment By: John J Lee (jjlee)
Date: 2003-02-12 20:45

Message:
Logged In: YES 
user_id=261020

Well, I believe Paul when he says his proxy was doing this, but 
AFAICS, RFC 2616 says this is incorrect: 
 
(RFC 2616, p. 128): 
 
   The Host request-header field specifies the Internet host and port 
   number of the resource being requested, as obtained from the original 
   URI given by the user or referring resource (generally an HTTP URL, 
   as described in section 3.2.2). The Host field value MUST represent 
   the naming authority of the origin server or gateway given by the 
   original URL. This allows the origin server or gateway to 
   differentiate between internally-ambiguous URLs, such as the root "/" 
   URL of a server for multiple host names on a single IP address. 
 
Can't find anything to say this doesn't apply to requests going via a 
proxy. 
 
Of course (well, I say of course -- I *think* I understand fully), the 
Host: header is redundant for proxies, because you have to send the 
full absoluteURI (with host) anyway, but I think Python should follow 
the RFC rather some random, and presumably broken, proxy. 
 
[well, to clarify: Host is redundant when using a proxy, but still 
required to be present by the RFC -- don't know why] 
 
BTW, urllib.py (in 2.2.1, anyway) *does* still follow RFC 2616, so 
urllib now differs from Moshe's 'fixed' urllib2. 
 
I think the patch to urllib2 should be reversed. 
 
 
John 
 

----------------------------------------------------------------------

Comment By: Moshe Zadka (moshez)
Date: 2001-04-11 08:45

Message:
Logged In: YES 
user_id=11645

You're right!
I've fixed this in urllib2.py v 1.12

----------------------------------------------------------------------

Comment By: Paul Moore (pmoore)
Date: 2001-04-10 10:46

Message:
Logged In: YES 
user_id=113328

I found the problem. In urllib2.py, class 
AbstractHTTPHandler, method do_open, the first line is now 

        host = urlparse.urlparse(req.get_full_url())[1]

It used to be

        host = req.get_host()

With the old version, the code works (with my proxy). With 
the new version it doesn't, as it passes the destination 
host, rather than the proxy name (and so loses the proxy 
info totally).

Paul.


----------------------------------------------------------------------

Comment By: Paul Moore (pmoore)
Date: 2001-04-10 10:45

Message:
Logged In: YES 
user_id=113328

I found the problem. In urllib2.py, class 
AbstractHTTPHandler, method do_open, the first line is now 

        host = urlparse.urlparse(req.get_full_url())[1]

It used to be

        host = req.get_host()

With the old version, the code works (with my proxy). With 
the new version it doesn't, as it passes the destination 
host, rather than the proxy name (and so loses the proxy 
info totally).

Paul.


----------------------------------------------------------------------

Comment By: Moshe Zadka (moshez)
Date: 2001-04-09 15:11

Message:
Logged In: YES 
user_id=11645

I've just tested with my installation of Python 2.1b2 and it
works. So I cannot reproduce the problem, and I need more
information from you: can you insert prints in the correct
places (e.g. do_open) to see what host urllib2 *thinks* it
is trying to access?
Thanks.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=413135&group_id=5470


From noreply@sourceforge.net  Wed Feb 12 21:54:04 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Wed, 12 Feb 2003 13:54:04 -0800
Subject: [Python-bugs-list] [ python-Bugs-580952 ] import lock should be exposed
Message-ID: <E18j4pQ-0003fj-00@sc8-sf-web1.sourceforge.net>

Bugs item #580952, was opened at 2002-07-13 09:06
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=580952&group_id=5470

Category: Python Interpreter Core
Group: None
>Status: Closed
>Resolution: Fixed
Priority: 5
Submitted By: Gordon B. McMillan (gmcm)
Assigned to: Guido van Rossum (gvanrossum)
Summary: import lock should be exposed

Initial Comment:
See
http://mail.python.org/pipermail/import-sig/2002-July/000179.html

In brief, a good import hook needs to use an import
lock. But if it creates it's own import lock, you have the
possibility of a deadlock, since PyImport_ImportModuleEx
still uses the core lock. So one thread can end up
acquiring the hook's lock, then seeking the core lock.
While another holds the core lock, but  does an import
(routed through the hook) and seeks the hook's lock.

Seems to me exposing the core lock through
imp.acquire_lock() and imp.release_lock() would
allow a properly coded hook  to avoid this.

----------------------------------------------------------------------

>Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-12 16:54

Message:
Logged In: YES 
user_id=6380

OK, this is fixed in CVS for Python 2.3. See #683257 for a
question about backporting.

----------------------------------------------------------------------

Comment By: Jason Hildebrand (jdhildeb)
Date: 2003-02-08 23:32

Message:
Logged In: YES 
user_id=173690

I've attached the patches against both 2.3-2.2.99 and 2.2.2;
they are available in patch # 683257.


----------------------------------------------------------------------

Comment By: Jason Hildebrand (jdhildeb)
Date: 2003-02-08 23:05

Message:
Logged In: YES 
user_id=173690

For what it's worth, Webware for Python users are exposed to
this bug, too, as Webware is multithreaded and uses an
import hook.  This has caused me some amount of grief, and
I've been happy to figure out the cause (the import
collisions were causing really weird things to happen).

I've attached a patch to import.c which exposes the
acquire_lock() and release_lock() functions.  Since
modifying Webware's import routine to use this these
functions, I have been completely unable to reproduce the
buggy behaviour (which had previously been easy to reproduce).

It would be great to have this merged in for Python 2.3, as
projects using both custom import hooks and threads have no
other reliable way of solving this problem.


----------------------------------------------------------------------

Comment By: Gordon B. McMillan (gmcm)
Date: 2002-07-14 17:34

Message:
Logged In: YES 
user_id=4923

Yes, I would just like to expose the current lock_import and 
unlock_import to C and Python.

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2002-07-14 17:06

Message:
Logged In: YES 
user_id=31435

I don't understand how imports work, at least not at this 
level.  I note that the current lock_import() and 
unlock_import() arrange to implement a reentrant lock:  in 
the absence of you supplying a patch, are these the 
functions you would like to see exposed?  Given the games 
being played here, it seems too dangerous to expose the 
*underlying* lock's acquire and release methods directly.

Assigned to Guido for cogitation.

----------------------------------------------------------------------

Comment By: Gordon B. McMillan (gmcm)
Date: 2002-07-13 09:10

Message:
Logged In: YES 
user_id=4923

Assigning to Tim for initial review.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=580952&group_id=5470


From noreply@sourceforge.net  Wed Feb 12 22:19:56 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Wed, 12 Feb 2003 14:19:56 -0800
Subject: [Python-bugs-list] [ python-Bugs-614555 ] Rewrite _reduce and _reconstructor in C
Message-ID: <E18j5ES-0004sN-00@sc8-sf-web1.sourceforge.net>

Bugs item #614555, was opened at 2002-09-25 14:15
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=614555&group_id=5470

Category: Python Interpreter Core
Group: Python 2.3
Status: Open
Resolution: None
>Priority: 2
Submitted By: Guido van Rossum (gvanrossum)
Assigned to: Guido van Rossum (gvanrossum)
Summary: Rewrite _reduce and _reconstructor in C

Initial Comment:
The copy_reg defines two functions, _reduce() and
_reconstructor(), that are used for the pickling and
unpickling of new-style classes. These were originally
written in Python because the right implementation
wasn't entirely clear; _reduce() is actually called
from a C-level __reduce__ method defined in 'object',
and it references _reconstructor() in its return tuple.
It is now time to move both back into C for efficiency.


----------------------------------------------------------------------

>Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-12 17:19

Message:
Logged In: YES 
user_id=6380

Lowering priority; when you use pickle protocol 2, none of
this Python code is used any more.

Jim's comment has been addressed by __newobj__ and
__getnewargs__ (see PEP 307).

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2002-11-14 11:46

Message:
Logged In: YES 
user_id=6380

I'd like to add some comments from Jim Fulton on this (or a
related) issue so they don't get lost.

"""
I still need to think of a good way to handle this. 
Currently, the pattern is a reduce return value of the form:

   reconstructor, (some_class, object, None), some_state

but there is a more general case where an uninitialized
object can be gotten by calling "type.__new__(some_class)",
or even: "some_class.__new__(some_class)".

There are really two issues:

- It's too hard to write __reduce__ functions for this case, and

- Too much data needs to be stored in pickles.

A new pickling code would handler the later, but I also want
to solve the former problem.

One thought is to return: (copy_reg.new, (some_class, ),
some_state) where copy_reg.new is:

   def new(class_): return class.__new__(class_)

The pickler could easily spot reduce returns with
copy_reg.new as the first value and generate a special
pickle code.
"""


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=614555&group_id=5470


From noreply@sourceforge.net  Wed Feb 12 22:36:43 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Wed, 12 Feb 2003 14:36:43 -0800
Subject: [Python-bugs-list] [ python-Bugs-638610 ] Instantiation init-less class with param
Message-ID: <E18j5Uh-00047X-00@sc8-sf-web3.sourceforge.net>

Bugs item #638610, was opened at 2002-11-14 14:53
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=638610&group_id=5470

Category: Type/class unification
Group: Python 2.3
>Status: Closed
>Resolution: Wont Fix
>Priority: 5
Submitted By: Shalabh Chaturvedi (shalabh)
Assigned to: Nobody/Anonymous (nobody)
Summary: Instantiation init-less class with param

Initial Comment:
Instantiation new style __init__-less class with
parameters does not raise exception. Old style classes
raise TypeError: this constructor takes no arguments.

Python 2.2.2 (#37, Oct 14 2002, 17:02:34) [MSC 32 bit
(Intel)] on win32
>>> class C:
...     pass
...
>>> c = C('extra', 'params')
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
TypeError: this constructor takes no arguments
>>>
>>> class C(object):
...     pass
...
>>> c = C('whos','eating','my', 'params')
>>> c
<__main__.C object at 0x007A49B8>
>>>

Who?


----------------------------------------------------------------------

>Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-12 17:36

Message:
Logged In: YES 
user_id=6380

This is a feature. You inherit both a __new__ and a __init__
from object that ignore their arguments; this is so that you
can write a class that defines one but not the other, and
its signature then forces the class's signature.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-01-20 13:29

Message:
Logged In: YES 
user_id=33168

Changed to 2.3 and bumped priority in the hopes that this
can/will be fixed for 2.3.

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2002-11-15 05:14

Message:
Logged In: YES 
user_id=6656

This is known behaviour (at least, I knew about it <wink>).

This is why:

>>> object(1,2,3)
<object object at 0x818dfb8>

I seem to recall discussion that this had to be left alone
in 2.2.X but should be fixed in 2.3...

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2002-11-14 18:26

Message:
Logged In: YES 
user_id=33168

Confirmed the behaviour in 2.2.2 and 2.3.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=638610&group_id=5470


From noreply@sourceforge.net  Wed Feb 12 23:15:50 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Wed, 12 Feb 2003 15:15:50 -0800
Subject: [Python-bugs-list] [ python-Bugs-643005 ] socket.inet_aton("255.255.255.255")
Message-ID: <E18j66Y-0004Xz-00@sc8-sf-web2.sourceforge.net>

Bugs item #643005, was opened at 2002-11-24 04:15
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=643005&group_id=5470

Category: Python Library
Group: Platform-specific
>Status: Closed
>Resolution: Fixed
Priority: 6
Submitted By: Jp Calderone (kuran)
Assigned to: Nobody/Anonymous (nobody)
>Summary: socket.inet_aton("255.255.255.255")

Initial Comment:
This comes up as an error on platforms with a 32 bit
in_addr_t type, because 0xFFFFFFFF -> -1 -> INADDR_ANY
-> error condition.  socketmodule's inet_aton() should
probably be changed to use inet_aton if available (I'm
not sure how to make the appropriate platform checks,
or I'd submit a patch).  It isn't clear to me the best
way to work around this on platforms lacking
inet_aton(), short of re-implementing its functionality.


----------------------------------------------------------------------

>Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-12 18:15

Message:
Logged In: YES 
user_id=6380

Fixed in Python 2.3 CVS by using inet_aton().

Does this need to be backported to 2.2.3?


----------------------------------------------------------------------

Comment By: Christos Georgiou (tzot)
Date: 2003-01-13 04:52

Message:
Logged In: YES 
user_id=539787

I'm in the process of trying to make a patch, since I found this 
relating to a mistaken 8-byte result in 64-bit machines.  
When I come up with an answer, I will post a link to the patch 
here.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=643005&group_id=5470


From noreply@sourceforge.net  Thu Feb 13 05:28:05 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Wed, 12 Feb 2003 21:28:05 -0800
Subject: [Python-bugs-list] [ python-Bugs-638610 ] Instantiation init-less class with param
Message-ID: <E18jBun-00026Q-00@sc8-sf-web3.sourceforge.net>

Bugs item #638610, was opened at 2002-11-14 19:53
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=638610&group_id=5470

Category: Type/class unification
Group: Python 2.3
Status: Closed
Resolution: Wont Fix
Priority: 5
Submitted By: Shalabh Chaturvedi (shalabh)
Assigned to: Nobody/Anonymous (nobody)
Summary: Instantiation init-less class with param

Initial Comment:
Instantiation new style __init__-less class with
parameters does not raise exception. Old style classes
raise TypeError: this constructor takes no arguments.

Python 2.2.2 (#37, Oct 14 2002, 17:02:34) [MSC 32 bit
(Intel)] on win32
>>> class C:
...     pass
...
>>> c = C('extra', 'params')
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
TypeError: this constructor takes no arguments
>>>
>>> class C(object):
...     pass
...
>>> c = C('whos','eating','my', 'params')
>>> c
<__main__.C object at 0x007A49B8>
>>>

Who?


----------------------------------------------------------------------

Comment By: Michael Stone (mbrierst)
Date: 2003-02-13 05:28

Message:
Logged In: YES 
user_id=670441

I agree that having separate __new__ and __init__ objects is a feature, but this can be preserved while also throwing an exception in the example given.  When no __new__ OR __init__ has overriden the base object's new and init, arguments passed to them will be silently ignored.  It seems preferable in that case to throw a type exception.  If either one has been overridden, it will determine the class signature and throw an exception when appropriate.

I submitted a patch to implement this as #685738.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-12 22:36

Message:
Logged In: YES 
user_id=6380

This is a feature. You inherit both a __new__ and a __init__
from object that ignore their arguments; this is so that you
can write a class that defines one but not the other, and
its signature then forces the class's signature.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-01-20 18:29

Message:
Logged In: YES 
user_id=33168

Changed to 2.3 and bumped priority in the hopes that this
can/will be fixed for 2.3.

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2002-11-15 10:14

Message:
Logged In: YES 
user_id=6656

This is known behaviour (at least, I knew about it <wink>).

This is why:

>>> object(1,2,3)
<object object at 0x818dfb8>

I seem to recall discussion that this had to be left alone
in 2.2.X but should be fixed in 2.3...

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2002-11-14 23:26

Message:
Logged In: YES 
user_id=33168

Confirmed the behaviour in 2.2.2 and 2.3.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=638610&group_id=5470


From noreply@sourceforge.net  Thu Feb 13 05:57:00 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Wed, 12 Feb 2003 21:57:00 -0800
Subject: [Python-bugs-list] [ python-Bugs-514345 ] pty.fork problem
Message-ID: <E18jCMm-0003Ba-00@sc8-sf-web1.sourceforge.net>

Bugs item #514345, was opened at 2002-02-07 16:21
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=514345&group_id=5470

Category: Python Library
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: M.-A. Lemburg (lemburg)
Assigned to: Fred L. Drake, Jr. (fdrake)
Summary: pty.fork problem

Initial Comment:
Subject: 
         Python bugreport, pty.fork problem
    Date: 
         Thu, 07 Feb 2002 07:30:08 -0800
   From: 
         Ronald Oussoren <oussoren@cistron.nl>
      To: 
         mal@lemburg.com


Sorry about the e-mail, but the bugtracker on SF doesn't 
accept my bugreport (I don't have a SF account).

The following script never returns:

----------------- start of script -------------
import pty
import os
import sys

def test():
        pid, fd = pty.fork()

        if pid == 0:
                print "1"
                print "2"
                print "3"
        else:
                fp = os.fdopen(fd, 'r')
                ln = fp.readline()
                while ln:
                        print '-->', ln
                        ln = fp.readline()
                print '-->', ln

test()
------------------ end of script -----------------
It prints '-->1' to '-->3' and then blocks. 

I've tested this with python 2.1 on Solaris 8. On Solaris 
pty.open seems to use 'openpty' instead of 'os.openpty'. A 
2-line change fixed the problem for me, but not for this 
demo-script: Close 'slave_fd' when pid != CHILD.




----------------------------------------------------------------------

Comment By: Michael Stone (mbrierst)
Date: 2003-02-13 05:57

Message:
Logged In: YES 
user_id=670441

I think I understand the problem.  I'm assuming Ronald was running the script from the interpreter.  In that case I also see the program block.  What's going on here is that python is continuing to run an interpreter in the forked process.  It's actually waiting for input from the user!  If you use read on the file descriptor to read in small quantities, you can actually see it print out the >>> prompt.  You can add sys.exit() to the end of the child fork to get guido's behavior.

The behavior Guido sees, I see if I run the script not in the interpreter.  It seems to be a real problem with forkpty/openpty.  When you close the slave fd, the master fd does not see EOF, it sees IOError.  This can be observed in C code as well.  This might be because of strange terminal settings in the new session, but I checked them and they look pretty normal.

This 'bug' in python was introduced by revision 2.117 of fileobject.c, when errors on close of file were raised instead of ignored.  (The checkin notes say there was no way to test it!)  I can't think of a good way to fix this.  Is there a unix guru who at least knows why it happens?  I tried newsgroups with no luck.  Maybe it should just be a documented problem.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2002-02-08 21:58

Message:
Logged In: YES 
user_id=6380

Assigning to Fred, who appears to have hacked this module
before.

I cannot reproduce the problem, but I see a different
problem that may be hinting at the same issue:

Under Python 2.1 or before, on Red Hat Linux 7.2, the test
program for me prints this:

--> 1

--> 2

--> 3

--> 

and exits. But with Python 2.2, it prints:

--> 1

--> 2

--> 3

Traceback (most recent call last):
  File "/tmp/tpty.py", line 20, in ?
    test() 
  File "/tmp/tpty.py", line 17, in test
    ln = fp.readline() 
IOError: [Errno 5] Input/output error

How can the difference be explained? I like the pre-2.2
behavior better!

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=514345&group_id=5470


From noreply@sourceforge.net  Thu Feb 13 07:00:01 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Wed, 12 Feb 2003 23:00:01 -0800
Subject: [Python-bugs-list] [ python-Bugs-685773 ] 2 (more) bugs in turtle
Message-ID: <E18jDLl-00056k-00@sc8-sf-web3.sourceforge.net>

Bugs item #685773, was opened at 2003-02-13 01:00
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=685773&group_id=5470

Category: Python Library
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Christopher Smith (smichr)
Assigned to: Nobody/Anonymous (nobody)
Summary: 2 (more) bugs in turtle

Initial Comment:
1) After initiating filling with the fill(1) command, the next fill 
command (e.g. fill(0)) does not cause the filling to take place.  
FIlling does not occur until the next draw statement occurs.

SOLUTION:
###
At the end of the IF block in fill(), put the following lines as 
delimited below with the #-- comment:

    def fill(self, flag):
        if self._filling:
 <cut>
                        self._items.append(item)
                        
                #--cps Addition to force filling.  Filling doesn't occur until 
                #--a move command is issued, so a "move" to the 
                #--present position is being issued to force the 
                #--filling to occur.
                x,y=self._position
                self._goto(x,y)
                #--
        self._path = []
###

2)  The last line of the goto() (not the _goto()) function incorrectly 
computes the x coordinate as x-x0.  You can verify this by issuing 
a goto(0,0) command as the first turtle command:  the turtle 
wanders off of the origin.

SOLUTION The coordinate should be computed as x0+x as shown 
below (again, this is the last line of the goto() function):

        self._goto(x0+x, y0-y)



/c

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=685773&group_id=5470


From noreply@sourceforge.net  Thu Feb 13 07:10:00 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Wed, 12 Feb 2003 23:10:00 -0800
Subject: [Python-bugs-list] [ python-Bugs-685775 ] turtle circle() documentation error
Message-ID: <E18jDVQ-0005Ij-00@sc8-sf-web1.sourceforge.net>

Bugs item #685775, was opened at 2003-02-13 01:10
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=685775&group_id=5470

Category: Documentation
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Christopher Smith (smichr)
Assigned to: Nobody/Anonymous (nobody)
Summary: turtle circle() documentation error

Initial Comment:
The following refer to doc page 
http://www.python.org/doc/current/lib/module-turtle.html

1) The "circle()" function of the turtle module does not behave as 
indicated in the documentation.  The doc's say that this command 
should...

"Draw a circle with radius radius whose center-point is where the 
pen would be if a forward(radius) were called."

In fact, it draws a circle centered on the point that is radius units to 
the left of the turtle.

2) Further information is given in the second paragraph that starts:

"If extent is not a full circle, one endpoint of the arc is the current 
pen position."

It might be good to add "...and the angle of the turtle is changed by 
extent."  i.e., if it started at 0 degrees and you drew 90 degrees of 
a circle, the turtle is now facing at 90 degrees.

/c


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=685775&group_id=5470


From noreply@sourceforge.net  Thu Feb 13 08:26:33 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 13 Feb 2003 00:26:33 -0800
Subject: [Python-bugs-list] [ python-Bugs-684903 ] socket module on solaris
Message-ID: <E18jEhV-00089X-00@sc8-sf-web3.sourceforge.net>

Bugs item #684903, was opened at 2003-02-12 08:02
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=684903&group_id=5470

Category: Build
Group: Python 2.2.2
Status: Open
Resolution: None
Priority: 5
Submitted By: RIck Bradshaw (rabrads)
Assigned to: Nobody/Anonymous (nobody)
Summary: socket module on solaris

Initial Comment:
I just built python 2.2.2 on solaris 7 or 2.7 to be
technically correct.  I then fire up the interpreter
and import test.testall. It fails for socket, which is
very import to me and my users. I then import _socket
and get the following info

Python 2.2.2 (#4, Feb  6 2003, 17:17:38) [C] on sunos5
Type "help", "copyright", "credits" or "license" for
more information.
>>> import _socket
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
ImportError: ld.so.1:
/soft/apps/packages/Python-2.2.2/bin/python: fatal:
relocation error: file
/soft/apps/packages/Python-2.2.2/lib/python2.2/_socket.so:
symbol __eprintf: referenced symbol not found

Thanks for any help.
Rick

----------------------------------------------------------------------

>Comment By: Anthony Baxter (anthonybaxter)
Date: 2003-02-13 19:26

Message:
Logged In: YES 
user_id=29957

This means something didn't get linked with -lgcc. As mwh
guessed, this is usually because Solaris' 'ld' was used
instead of the GNU one. Does the build machine have a
working install of binutils as well as gcc?


----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2003-02-13 01:16

Message:
Logged In: YES 
user_id=6656

IIRC this usually happens when you compile with gcc and link
with the system ld or vice versa or something like that.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-12 10:25

Message:
Logged In: YES 
user_id=33168

This error looks familiar, but I don't recall the cause. 
Did you have a working version of python on this machine? 
What compiler and version are you using?  Is __eprintf in a
library (libc.a)?  Is that not getting linked?  Is it using
the correct linker and library locations?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=684903&group_id=5470


From noreply@sourceforge.net  Thu Feb 13 10:11:27 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 13 Feb 2003 02:11:27 -0800
Subject: [Python-bugs-list] [ python-Bugs-685846 ] raw_input defers alarm signal
Message-ID: <E18jGL1-0004Ec-00@sc8-sf-web3.sourceforge.net>

Bugs item #685846, was opened at 2003-02-13 10:11
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=685846&group_id=5470

Category: Python Library
Group: Python 2.2
Status: Open
Resolution: None
Priority: 5
Submitted By: paul rubin (phr)
Assigned to: Nobody/Anonymous (nobody)
Summary: raw_input defers alarm signal

Initial Comment:
import signal
def handle(a,b):
   print "received signal"
signal.signal(signal.SIGALRM, handle)
signal.alarm(5)
name = raw_input('Please enter your name within 5
seconds: ')


waits for input forever instead of getting the signal
thrown
after 5 seconds.  If you wait more than 5 seconds before
typing your name, the signal does get handled after you
finally enter your input.

The workaround of calling sys.stdin.readline() instead of
raw_input does the right thing, so something is funny about
the raw_input function.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=685846&group_id=5470


From noreply@sourceforge.net  Thu Feb 13 11:46:16 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 13 Feb 2003 03:46:16 -0800
Subject: [Python-bugs-list] [ python-Bugs-685846 ] raw_input defers alarm signal
Message-ID: <E18jHom-0003jh-00@sc8-sf-web2.sourceforge.net>

Bugs item #685846, was opened at 2003-02-13 10:11
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=685846&group_id=5470

Category: Python Library
Group: Python 2.2
Status: Open
Resolution: None
Priority: 5
Submitted By: paul rubin (phr)
Assigned to: Nobody/Anonymous (nobody)
Summary: raw_input defers alarm signal

Initial Comment:
import signal
def handle(a,b):
   print "received signal"
signal.signal(signal.SIGALRM, handle)
signal.alarm(5)
name = raw_input('Please enter your name within 5
seconds: ')


waits for input forever instead of getting the signal
thrown
after 5 seconds.  If you wait more than 5 seconds before
typing your name, the signal does get handled after you
finally enter your input.

The workaround of calling sys.stdin.readline() instead of
raw_input does the right thing, so something is funny about
the raw_input function.


----------------------------------------------------------------------

>Comment By: Michael Hudson (mwh)
Date: 2003-02-13 11:46

Message:
Logged In: YES 
user_id=6656

I'm 99% sure this is readline's fault.

You might be able to test this by hiding the readline.so
from the interpreter.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=685846&group_id=5470


From noreply@sourceforge.net  Thu Feb 13 16:25:55 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 13 Feb 2003 08:25:55 -0800
Subject: [Python-bugs-list] [ python-Bugs-643005 ] socket.inet_aton("255.255.255.255")
Message-ID: <E18jMBP-0000LR-00@sc8-sf-web2.sourceforge.net>

Bugs item #643005, was opened at 2002-11-24 11:15
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=643005&group_id=5470

Category: Python Library
Group: Platform-specific
Status: Closed
Resolution: Fixed
Priority: 6
Submitted By: Jp Calderone (kuran)
Assigned to: Nobody/Anonymous (nobody)
Summary: socket.inet_aton("255.255.255.255")

Initial Comment:
This comes up as an error on platforms with a 32 bit
in_addr_t type, because 0xFFFFFFFF -> -1 -> INADDR_ANY
-> error condition.  socketmodule's inet_aton() should
probably be changed to use inet_aton if available (I'm
not sure how to make the appropriate platform checks,
or I'd submit a patch).  It isn't clear to me the best
way to work around this on platforms lacking
inet_aton(), short of re-implementing its functionality.


----------------------------------------------------------------------

Comment By: Christos Georgiou (tzot)
Date: 2003-02-13 18:25

Message:
Logged In: YES 
user_id=539787

I believe all users having used the old inet_aton have used 
workarounds if 0xffffffff could be used as an argument (I know I 
have).  It sure would be nice, though (and make 2.2.3 
the "industrial strength" version :)

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-13 01:15

Message:
Logged In: YES 
user_id=6380

Fixed in Python 2.3 CVS by using inet_aton().

Does this need to be backported to 2.2.3?


----------------------------------------------------------------------

Comment By: Christos Georgiou (tzot)
Date: 2003-01-13 11:52

Message:
Logged In: YES 
user_id=539787

I'm in the process of trying to make a patch, since I found this 
relating to a mistaken 8-byte result in 64-bit machines.  
When I come up with an answer, I will post a link to the patch 
here.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=643005&group_id=5470


From noreply@sourceforge.net  Thu Feb 13 16:39:49 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 13 Feb 2003 08:39:49 -0800
Subject: [Python-bugs-list] [ python-Bugs-643005 ] socket.inet_aton("255.255.255.255")
Message-ID: <E18jMOr-0005XQ-00@sc8-sf-web3.sourceforge.net>

Bugs item #643005, was opened at 2002-11-24 04:15
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=643005&group_id=5470

Category: Python Library
Group: Platform-specific
Status: Closed
Resolution: Fixed
Priority: 6
Submitted By: Jp Calderone (kuran)
Assigned to: Nobody/Anonymous (nobody)
>Summary: socket.inet_aton("255.255.255.255")

Initial Comment:
This comes up as an error on platforms with a 32 bit
in_addr_t type, because 0xFFFFFFFF -> -1 -> INADDR_ANY
-> error condition.  socketmodule's inet_aton() should
probably be changed to use inet_aton if available (I'm
not sure how to make the appropriate platform checks,
or I'd submit a patch).  It isn't clear to me the best
way to work around this on platforms lacking
inet_aton(), short of re-implementing its functionality.


----------------------------------------------------------------------

>Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-13 11:39

Message:
Logged In: YES 
user_id=6380

The fix needed configuration work because Windows doesn't
have inet_aton(). :-)

I'm calling for a volunteer to backport this to 2.2.3.

----------------------------------------------------------------------

Comment By: Christos Georgiou (tzot)
Date: 2003-02-13 11:25

Message:
Logged In: YES 
user_id=539787

I believe all users having used the old inet_aton have used 
workarounds if 0xffffffff could be used as an argument (I know I 
have).  It sure would be nice, though (and make 2.2.3 
the "industrial strength" version :)

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-12 18:15

Message:
Logged In: YES 
user_id=6380

Fixed in Python 2.3 CVS by using inet_aton().

Does this need to be backported to 2.2.3?


----------------------------------------------------------------------

Comment By: Christos Georgiou (tzot)
Date: 2003-01-13 04:52

Message:
Logged In: YES 
user_id=539787

I'm in the process of trying to make a patch, since I found this 
relating to a mistaken 8-byte result in 64-bit machines.  
When I come up with an answer, I will post a link to the patch 
here.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=643005&group_id=5470


From noreply@sourceforge.net  Thu Feb 13 16:41:13 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 13 Feb 2003 08:41:13 -0800
Subject: [Python-bugs-list] [ python-Bugs-638610 ] Instantiation init-less class with param
Message-ID: <E18jMQD-0005cS-00@sc8-sf-web3.sourceforge.net>

Bugs item #638610, was opened at 2002-11-14 14:53
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=638610&group_id=5470

Category: Type/class unification
Group: Python 2.3
Status: Closed
>Resolution: Fixed
Priority: 5
Submitted By: Shalabh Chaturvedi (shalabh)
Assigned to: Nobody/Anonymous (nobody)
Summary: Instantiation init-less class with param

Initial Comment:
Instantiation new style __init__-less class with
parameters does not raise exception. Old style classes
raise TypeError: this constructor takes no arguments.

Python 2.2.2 (#37, Oct 14 2002, 17:02:34) [MSC 32 bit
(Intel)] on win32
>>> class C:
...     pass
...
>>> c = C('extra', 'params')
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
TypeError: this constructor takes no arguments
>>>
>>> class C(object):
...     pass
...
>>> c = C('whos','eating','my', 'params')
>>> c
<__main__.C object at 0x007A49B8>
>>>

Who?


----------------------------------------------------------------------

>Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-13 11:41

Message:
Logged In: YES 
user_id=6380

This is nevertheless fixed now in CVS, thanks to SF fix
685738. :-)

----------------------------------------------------------------------

Comment By: Michael Stone (mbrierst)
Date: 2003-02-13 00:28

Message:
Logged In: YES 
user_id=670441

I agree that having separate __new__ and __init__ objects is a feature, but this can be preserved while also throwing an exception in the example given.  When no __new__ OR __init__ has overriden the base object's new and init, arguments passed to them will be silently ignored.  It seems preferable in that case to throw a type exception.  If either one has been overridden, it will determine the class signature and throw an exception when appropriate.

I submitted a patch to implement this as #685738.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-12 17:36

Message:
Logged In: YES 
user_id=6380

This is a feature. You inherit both a __new__ and a __init__
from object that ignore their arguments; this is so that you
can write a class that defines one but not the other, and
its signature then forces the class's signature.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-01-20 13:29

Message:
Logged In: YES 
user_id=33168

Changed to 2.3 and bumped priority in the hopes that this
can/will be fixed for 2.3.

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2002-11-15 05:14

Message:
Logged In: YES 
user_id=6656

This is known behaviour (at least, I knew about it <wink>).

This is why:

>>> object(1,2,3)
<object object at 0x818dfb8>

I seem to recall discussion that this had to be left alone
in 2.2.X but should be fixed in 2.3...

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2002-11-14 18:26

Message:
Logged In: YES 
user_id=33168

Confirmed the behaviour in 2.2.2 and 2.3.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=638610&group_id=5470


From noreply@sourceforge.net  Thu Feb 13 16:51:52 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 13 Feb 2003 08:51:52 -0800
Subject: [Python-bugs-list] [ python-Bugs-643005 ] socket.inet_aton("255.255.255.255")
Message-ID: <E18jMaW-0001ft-00@sc8-sf-web4.sourceforge.net>

Bugs item #643005, was opened at 2002-11-24 11:15
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=643005&group_id=5470

Category: Python Library
Group: Platform-specific
Status: Closed
Resolution: Fixed
Priority: 6
Submitted By: Jp Calderone (kuran)
Assigned to: Nobody/Anonymous (nobody)
Summary: socket.inet_aton("255.255.255.255")

Initial Comment:
This comes up as an error on platforms with a 32 bit
in_addr_t type, because 0xFFFFFFFF -> -1 -> INADDR_ANY
-> error condition.  socketmodule's inet_aton() should
probably be changed to use inet_aton if available (I'm
not sure how to make the appropriate platform checks,
or I'd submit a patch).  It isn't clear to me the best
way to work around this on platforms lacking
inet_aton(), short of re-implementing its functionality.


----------------------------------------------------------------------

Comment By: Christos Georgiou (tzot)
Date: 2003-02-13 18:51

Message:
Logged In: YES 
user_id=539787

If none other volunteers, I'll give it a shot during the weekend (I 
don't have VC++ but a friend has, and I need to download the 
current 2.2.3 tree).

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-13 18:39

Message:
Logged In: YES 
user_id=6380

The fix needed configuration work because Windows doesn't
have inet_aton(). :-)

I'm calling for a volunteer to backport this to 2.2.3.

----------------------------------------------------------------------

Comment By: Christos Georgiou (tzot)
Date: 2003-02-13 18:25

Message:
Logged In: YES 
user_id=539787

I believe all users having used the old inet_aton have used 
workarounds if 0xffffffff could be used as an argument (I know I 
have).  It sure would be nice, though (and make 2.2.3 
the "industrial strength" version :)

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-13 01:15

Message:
Logged In: YES 
user_id=6380

Fixed in Python 2.3 CVS by using inet_aton().

Does this need to be backported to 2.2.3?


----------------------------------------------------------------------

Comment By: Christos Georgiou (tzot)
Date: 2003-01-13 11:52

Message:
Logged In: YES 
user_id=539787

I'm in the process of trying to make a patch, since I found this 
relating to a mistaken 8-byte result in 64-bit machines.  
When I come up with an answer, I will post a link to the patch 
here.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=643005&group_id=5470


From guido@python.org  Thu Feb 13 17:52:12 2003
From: guido@python.org (Guido van Rossum)
Date: Thu, 13 Feb 2003 12:52:12 -0500
Subject: [Python-bugs-list] Changing SF tracker email preferences
Message-ID: <200302131752.h1DHqCi00785@odiug.zope.com>

When a new bug or patch is posted to a SF tracker, an email gets sent
to python-bugs-list@python.org or patches@python.org.

But...  when anything is changed or added to those trackers, an email
is *also* sent to those lists.

I would like to receive the first kind of traffic (new tracker issues)
but not the second kind, except for those tracker issues in which I am
interested (which I can get by explicit turning on monitoring of those
issues, or by following up).

The SF tracker admin has an option to allow this: all I have to do is
uncheck the box that says "Send email on all changes".  I would really
like to do this.

Would anybody who is currently receiving bugs or patches traffic be
bothered if I made this change?

--Guido van Rossum (home page: http://www.python.org/~guido/)


From noreply@sourceforge.net  Thu Feb 13 19:19:12 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 13 Feb 2003 11:19:12 -0800
Subject: [Python-bugs-list] [ python-Bugs-514345 ] pty.fork problem
Message-ID: <E18jOt6-0005Dt-00@sc8-sf-web3.sourceforge.net>

Bugs item #514345, was opened at 2002-02-07 11:21
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=514345&group_id=5470

Category: Python Library
Group: None
>Status: Closed
>Resolution: Wont Fix
Priority: 5
Submitted By: M.-A. Lemburg (lemburg)
>Assigned to: Guido van Rossum (gvanrossum)
Summary: pty.fork problem

Initial Comment:
Subject: 
         Python bugreport, pty.fork problem
    Date: 
         Thu, 07 Feb 2002 07:30:08 -0800
   From: 
         Ronald Oussoren <oussoren@cistron.nl>
      To: 
         mal@lemburg.com


Sorry about the e-mail, but the bugtracker on SF doesn't 
accept my bugreport (I don't have a SF account).

The following script never returns:

----------------- start of script -------------
import pty
import os
import sys

def test():
        pid, fd = pty.fork()

        if pid == 0:
                print "1"
                print "2"
                print "3"
        else:
                fp = os.fdopen(fd, 'r')
                ln = fp.readline()
                while ln:
                        print '-->', ln
                        ln = fp.readline()
                print '-->', ln

test()
------------------ end of script -----------------
It prints '-->1' to '-->3' and then blocks. 

I've tested this with python 2.1 on Solaris 8. On Solaris 
pty.open seems to use 'openpty' instead of 'os.openpty'. A 
2-line change fixed the problem for me, but not for this 
demo-script: Close 'slave_fd' when pid != CHILD.




----------------------------------------------------------------------

>Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-13 14:19

Message:
Logged In: YES 
user_id=6380

Thanks for the explanation, Michael!

Raising an exception on a close() error is the right thing
to do, IMO.

I don't know why the error happens either, but it's probably
part of the design of PTYs; it's not a big deal I think --
PTYs should only be used by experts who know what they are
doing.

I'm not sure what should be documented; I'll just close this
bug now.

----------------------------------------------------------------------

Comment By: Michael Stone (mbrierst)
Date: 2003-02-13 00:57

Message:
Logged In: YES 
user_id=670441

I think I understand the problem.  I'm assuming Ronald was running the script from the interpreter.  In that case I also see the program block.  What's going on here is that python is continuing to run an interpreter in the forked process.  It's actually waiting for input from the user!  If you use read on the file descriptor to read in small quantities, you can actually see it print out the >>> prompt.  You can add sys.exit() to the end of the child fork to get guido's behavior.

The behavior Guido sees, I see if I run the script not in the interpreter.  It seems to be a real problem with forkpty/openpty.  When you close the slave fd, the master fd does not see EOF, it sees IOError.  This can be observed in C code as well.  This might be because of strange terminal settings in the new session, but I checked them and they look pretty normal.

This 'bug' in python was introduced by revision 2.117 of fileobject.c, when errors on close of file were raised instead of ignored.  (The checkin notes say there was no way to test it!)  I can't think of a good way to fix this.  Is there a unix guru who at least knows why it happens?  I tried newsgroups with no luck.  Maybe it should just be a documented problem.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2002-02-08 16:58

Message:
Logged In: YES 
user_id=6380

Assigning to Fred, who appears to have hacked this module
before.

I cannot reproduce the problem, but I see a different
problem that may be hinting at the same issue:

Under Python 2.1 or before, on Red Hat Linux 7.2, the test
program for me prints this:

--> 1

--> 2

--> 3

--> 

and exits. But with Python 2.2, it prints:

--> 1

--> 2

--> 3

Traceback (most recent call last):
  File "/tmp/tpty.py", line 20, in ?
    test() 
  File "/tmp/tpty.py", line 17, in test
    ln = fp.readline() 
IOError: [Errno 5] Input/output error

How can the difference be explained? I like the pre-2.2
behavior better!

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=514345&group_id=5470


From noreply@sourceforge.net  Thu Feb 13 22:21:06 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 13 Feb 2003 14:21:06 -0800
Subject: [Python-bugs-list] [ python-Bugs-501622 ] Missing trailing newline should not raise SyntaxError
Message-ID: <E18jRj8-0005TG-00@sc8-sf-web3.sourceforge.net>

Bugs item #501622, was opened at 2002-01-09 23:01
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=501622&group_id=5470

Category: Parser/Compiler
Group: None
>Status: Closed
>Resolution: Fixed
Priority: 5
Submitted By: David Bolen (db3l)
Assigned to: Guido van Rossum (gvanrossum)
Summary: Missing trailing newline should not raise SyntaxError

Initial Comment:
If you have a module that you wish to compile using 
the builtin compile() function (in 'exec' mode), it 
will fail with a SyntaxError if that module does not 
have a newline as its final token.

The same module can be executed directly by the 
interpreter, or imported by another module, and Python 
can properly compile and save a pyc for the module.

I believe the difference is rooted in the fact that 
the tokenizer (tokenizer.c, in tok_nextc()) 
will "fake" a newline at the end of a file if it 
doesn't find one, but it will not do so when 
tokenizing a string buffer.

What I'm not certain of is whether faking such a token 
for strings as well won't break something else (such 
as when parsing a string for an expression rather than 
a full module).  But without such a change, you have a 
state where a module that works (and compiles) in 
other circumstances cannot be read into memory and 
compiled with the compile() builtin.

This came up while tracking down a problem with 
failures using Gordan McMillan's Installer package 
which compiles modules using compile() before 
including them in the archive.

I believe this is true for all releases since at least 
1.5.2.

-- David

----------------------------------------------------------------------

>Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-13 17:21

Message:
Logged In: YES 
user_id=6380

Fixed. This was simpler than I thought (only 4 lines in
parsetok.c) but also harder (codeop.py depend on the broken
behavior!) Most of the changes checked in had to do with
adding a flag to enable the old behavior. :-(

I note that Tim's second comment is visionary in this respect.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2002-03-22 19:35

Message:
Logged In: YES 
user_id=6380

Hm, adding it to builtin_compile isn't enough. We'd have to
add it to exec as well.

I think the lexer and/or parser should take care of this --
just as it should take care of accepting \r as well as \n as
well as \r\n.

Yes, it's hard to find. But there's got to be a way.

----------------------------------------------------------------------

Comment By: Neil Schemenauer (nascheme)
Date: 2002-03-22 18:14

Message:
Logged In: YES 
user_id=35752

I'm +1 on builtin_compile adding the newline.  It's the
lazy way out and it's better than every person hacking
with the parser stumbling into it and coming up with their
own work around.

Guido?

----------------------------------------------------------------------

Comment By: David Bolen (db3l)
Date: 2002-03-22 18:06

Message:
Logged In: YES 
user_id=53196

If compile() is being used in exec mode with a non-
terminated multi-line string, it's not going to work unless 
the application generates that copy itself in any event.  
So without an interpreter fix, I'd think the string copy is 
inevitable, and it might simplify things to have the 
builtin function take care of it.  It's something easy to 
overlook at the application level and could thus be fixed 
in one place rather than at each point of use.

On the other hand, I also noticed something I overlooked 
when first encountering the problem - the 2.2 docs added 
some text to compile() talking about this need for 
termination.  So it could be argued that it's now a 
documented restriction, and should the newline append (with 
any requisite string duplication) be needed, it leaves it 
to the individual applications rather than forcing it in 
the builtin.

Not to mention a documentation solution could thus be 
declared already done.


----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2002-03-22 18:01

Message:
Logged In: YES 
user_id=31435

Well, the user can't append an '\n' inplace either.  The 
question is whether we do that for them, or let it blow 
up.  OTOH, codeop.py has a lot of fun <wink> now trying to 
compile as-is, tben with one '\n' tacked on, then with two 
of 'em.  It would take me a long time to figure out exactly 
why it's doing all that, and to guess exactly how it would 
break.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2002-03-22 17:46

Message:
Logged In: YES 
user_id=6380

Probably, unless the start symbol is "expr" (which doesn't
need a newline).

But it would mean copying a potentially huge string -- we
can't append the \n in place.

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2002-03-22 17:41

Message:
Logged In: YES 
user_id=31435

Would it make sense for builtin_compile() to append a 
newline itself (say, if one weren't already present)?

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2002-03-22 17:20

Message:
Logged In: YES 
user_id=6380

> the tok_nextc code is hairy and whatever
> I tried broke something else.

That's exactly what happened to me when I tried to fix this
myself long ago. :-(

The workaround is simple enough: whoever calls compile()
should append a newline to the string just to be sure.

----------------------------------------------------------------------

Comment By: Neil Schemenauer (nascheme)
Date: 2002-03-22 17:07

Message:
Logged In: YES 
user_id=35752

I ran into this bug myself when writing the PTL compiler.
Here's a test case:

code = "def foo():\n  pass"
open("bug.py", "w").write(code)
import bug # works
compile(code, "<string>", "exec") # doesn't work

I traced this bug to tok_nextc.  If the input is coming from
a file and the last bit of input doesn't end with a newline
then one is faked.  This doesn't happen if the input is
coming from a string.  I spent time trying to figure out
how to fix it but the tok_nextc code is hairy and whatever
I tried broke something else.

----------------------------------------------------------------------

Comment By: Neil Schemenauer (nascheme)
Date: 2002-03-22 17:07

Message:
Logged In: YES 
user_id=35752

I ran into this bug myself when writing the PTL compiler.
Here's a test case:

code = "def foo():\n  pass"
open("bug.py", "w").write(code)
import bug # works
compile(code, "<string>", "exec") # doesn't work

I traced this bug to tok_nextc.  If the input is coming from
a file and the last bit of input doesn't end with a newline
then one is faked.  This doesn't happen if the input is
coming from a string.  I spent time trying to figure out
how to fix it but the tok_nextc code is hairy and whatever
I tried broke something else.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=501622&group_id=5470


From noreply@sourceforge.net  Thu Feb 13 23:06:20 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 13 Feb 2003 15:06:20 -0800
Subject: [Python-bugs-list] [ python-Bugs-685011 ] calendar module overflow handling
Message-ID: <E18jSQu-0002kB-00@sc8-sf-web4.sourceforge.net>

Bugs item #685011, was opened at 2003-02-11 19:34
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=685011&group_id=5470

Category: Python Library
Group: Python 2.3
>Status: Closed
>Resolution: Fixed
Priority: 5
Submitted By: Richard Jones (richard)
Assigned to: Raymond Hettinger (rhettinger)
Summary: calendar module overflow handling

Initial Comment:
The calendar module has lost a significant area of functionality in 
Python 2.3  
- that of handling overflows in the tuples passed to timegm(). 
 
Python 2.2.2 (#1, Oct 28 2002, 16:39:08) 
[GCC 2.96 20000731 (Red Hat Linux 7.1 2.96-98)] on linux2 
Type "help", "copyright", "credits" or "license" for more information. 
>>> import time,calendar 
>>> time.gmtime(calendar.timegm((2003,1,30,0,0,0,0,0,0))) 
(2003, 1, 30, 0, 0, 0, 3, 30, 0) 
>>> time.gmtime(calendar.timegm((2003,2,30,0,0,0,0,0,0))) 
(2003, 3, 2, 0, 0, 0, 6, 61, 0) 
>>> 
 
Python 2.3a1 (#1, Feb  6 2003, 14:54:16) 
[GCC 2.96 20000731 (Red Hat Linux 7.1 2.96-98)] on linux2 
Type "help", "copyright", "credits" or "license" for more information. 
>>> import time,calendar 
>>> time.gmtime(calendar.timegm((2003,1,30,0,0,0,0,0,0))) 
(2003, 1, 30, 0, 0, 0, 3, 30, 0) 
>>> time.gmtime(calendar.timegm((2003,2,30,0,0,0,0,0,0))) 
Traceback (most recent call last): 
  File "<stdin>", line 1, in ? 
  File "/usr/lib/python2.3/calendar.py", line 216, in timegm 
    days = datetime.date(year, month, day).toordinal() - _EPOCH_ORD 
ValueError: day is out of range for month 
>>> 
 
 
Is this functionality likely to be reinstated for 2.3's release? 
 

----------------------------------------------------------------------

>Comment By: Raymond Hettinger (rhettinger)
Date: 2003-02-13 18:06

Message:
Logged In: YES 
user_id=80475

Restored behavior as requested.

See Lib/calendar.py 1.32

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2003-02-11 19:54

Message:
Logged In: YES 
user_id=31435

Cute.  My best reading of the code is that you were relying 
on an undocumented accident.  The timegm code in 2.2 did 
do bounds-checking on the year and month, it looks like it 
was just too lazy to check the day.

Assigning to Raymond to teach him a lesson <wink>.  If you 
want to restore this accident, pass 1 instead of days to the 
datetime constructor, and restore the old

    days = days + day - 1

line after that returns.  It should still be lot faster than the 
2.2 code.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=685011&group_id=5470


From noreply@sourceforge.net  Thu Feb 13 23:13:18 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 13 Feb 2003 15:13:18 -0800
Subject: [Python-bugs-list] [ python-Bugs-685775 ] turtle circle() documentation error
Message-ID: <E18jSXe-000725-00@sc8-sf-web1.sourceforge.net>

Bugs item #685775, was opened at 2003-02-13 02:10
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=685775&group_id=5470

Category: Documentation
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Christopher Smith (smichr)
>Assigned to: Raymond Hettinger (rhettinger)
Summary: turtle circle() documentation error

Initial Comment:
The following refer to doc page 
http://www.python.org/doc/current/lib/module-turtle.html

1) The "circle()" function of the turtle module does not behave as 
indicated in the documentation.  The doc's say that this command 
should...

"Draw a circle with radius radius whose center-point is where the 
pen would be if a forward(radius) were called."

In fact, it draws a circle centered on the point that is radius units to 
the left of the turtle.

2) Further information is given in the second paragraph that starts:

"If extent is not a full circle, one endpoint of the arc is the current 
pen position."

It might be good to add "...and the angle of the turtle is changed by 
extent."  i.e., if it started at 0 degrees and you drew 90 degrees of 
a circle, the turtle is now facing at 90 degrees.

/c


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=685775&group_id=5470


From noreply@sourceforge.net  Fri Feb 14 01:13:46 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 13 Feb 2003 17:13:46 -0800
Subject: [Python-bugs-list] [ python-Bugs-686323 ] Minor array module enhancements
Message-ID: <E18jUQE-0003m7-00@sc8-sf-web3.sourceforge.net>

Bugs item #686323, was opened at 2003-02-14 01:13
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=686323&group_id=5470

Category: Python Library
Group: Feature Request
Status: Open
Resolution: None
Priority: 5
Submitted By: paul rubin (phr)
Assigned to: Nobody/Anonymous (nobody)
Summary: Minor array module enhancements

Initial Comment:
1) I notice that array('B', (2,3,4)) throws an error
saying the
2nd arg must be a list or string.  That is, array('B',
[2,3,4]) is
legal but using the tuple (2,3,4) is not.  The
limitation doesn't
seem harmful, but I also don't see any good reason for
it.  May
as well remove it.

2) Of more usefulness (and maybe of more controversy), I
think the byte array methods should accept string
arguments,
for example
   a = array('B', 'abc')
   a.append('def')
should do the obvious thing.  That gives a natural way to
build up a string in pieces instead of making a list of
strings
and doing the counterintuitive ''.join(x) maneuver. 
Appending
to byte arrays is the usual way to build up a string in
Java,
if that matters.  So I favor this change too.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=686323&group_id=5470


From noreply@sourceforge.net  Fri Feb 14 02:27:06 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 13 Feb 2003 18:27:06 -0800
Subject: [Python-bugs-list] [ python-Bugs-614555 ] Rewrite _reduce and _reconstructor in C
Message-ID: <E18jVZC-0001UF-00@sc8-sf-web4.sourceforge.net>

Bugs item #614555, was opened at 2002-09-25 14:15
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=614555&group_id=5470

Category: Python Interpreter Core
Group: Python 2.3
Status: Open
Resolution: None
>Priority: 7
Submitted By: Guido van Rossum (gvanrossum)
>Assigned to: Tim Peters (tim_one)
Summary: Rewrite _reduce and _reconstructor in C

Initial Comment:
The copy_reg defines two functions, _reduce() and
_reconstructor(), that are used for the pickling and
unpickling of new-style classes. These were originally
written in Python because the right implementation
wasn't entirely clear; _reduce() is actually called
from a C-level __reduce__ method defined in 'object',
and it references _reconstructor() in its return tuple.
It is now time to move both back into C for efficiency.


----------------------------------------------------------------------

>Comment By: Tim Peters (tim_one)
Date: 2003-02-13 21:27

Message:
Logged In: YES 
user_id=31435

Except that the new Python code _better_reduce() is used 
then.  As discussed, I intend to rewrite _reduce and 
_better_reduce in C, inside object's __reduce__ 
implementation, so I assigned this to me and boosted the 
priority.  I don't intend to recode _reconstructor in C, 
though.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-12 17:19

Message:
Logged In: YES 
user_id=6380

Lowering priority; when you use pickle protocol 2, none of
this Python code is used any more.

Jim's comment has been addressed by __newobj__ and
__getnewargs__ (see PEP 307).

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2002-11-14 11:46

Message:
Logged In: YES 
user_id=6380

I'd like to add some comments from Jim Fulton on this (or a
related) issue so they don't get lost.

"""
I still need to think of a good way to handle this. 
Currently, the pattern is a reduce return value of the form:

   reconstructor, (some_class, object, None), some_state

but there is a more general case where an uninitialized
object can be gotten by calling "type.__new__(some_class)",
or even: "some_class.__new__(some_class)".

There are really two issues:

- It's too hard to write __reduce__ functions for this case, and

- Too much data needs to be stored in pickles.

A new pickling code would handler the later, but I also want
to solve the former problem.

One thought is to return: (copy_reg.new, (some_class, ),
some_state) where copy_reg.new is:

   def new(class_): return class.__new__(class_)

The pickler could easily spot reduce returns with
copy_reg.new as the first value and generate a special
pickle code.
"""


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=614555&group_id=5470


From noreply@sourceforge.net  Fri Feb 14 03:52:36 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 13 Feb 2003 19:52:36 -0800
Subject: [Python-bugs-list] [ python-Bugs-663701 ] sets module review
Message-ID: <E18jWtw-0000Bi-00@sc8-sf-web1.sourceforge.net>

Bugs item #663701, was opened at 2003-01-07 09:08
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=663701&group_id=5470

Category: Python Library
Group: Python 2.3
>Status: Closed
>Resolution: Fixed
Priority: 5
Submitted By: Sebastien Keim (s_keim)
Assigned to: Raymond Hettinger (rhettinger)
Summary: sets module review

Initial Comment:
* the ^ operator doesnt print well in pdf generated
documentation (both in what's new and in library reference)

* shouldn't the _as_immutable and
_as_temporaly_immutable be spelled __as_immutable__ and
__as_temporaly_immutable__ for consistency with other
hook methods?

* cmp() suck:
<pre>
bash-2.05$ ./python 
Python 2.3a1 (#1, Jan  4 2003, 10:17:56) 
[GCC 2.95.3 20010315 (release)] on linux2
Type "help", "copyright", "credits" or "license" for
more information.
>>> import sets
>>> s = sets.Set
>>> a = s([0])
>>> b = s([1])
>>> cmp(a,b)
1
>>> 
bash-2.05$ ./python 
Python 2.3a1 (#1, Jan  4 2003, 10:17:56) 
[GCC 2.95.3 20010315 (release)] on linux2
Type "help", "copyright", "credits" or "license" for
more information.
>>> import sets
>>> a=sets.Set([0])
>>> b=sets.Set([1])
>>> cmp(a,b)
-1
</pre>

* Because we can have set1!=set2 and both
(set1<set2)==False and (set2<set1)==False what will be
the behavhior of: [set1, set2].sort().
Since inclusion can't  define a full ordering relation,
wouldn't it be better to remove the equivalence between
issubset and __lt__ (and other comparaison methods) and
to use an equivalent to the dictionary protocol to
define the < and > behavior?




----------------------------------------------------------------------

>Comment By: Raymond Hettinger (rhettinger)
Date: 2003-02-13 22:52

Message:
Logged In: YES 
user_id=80475

Done.
Committed as:  Lib/sets.py 1.41 and 1.42
                          Doc/lib/libsets.tex 1.11


----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-11 16:08

Message:
Logged In: YES 
user_id=6380

Assigning to Raymond Hettinger. Maybe there are some
concrete to-do items above; after that I think it can be
closed. Maybe using __xxxx__ names is a good idea (since
when somebody else reuses these names, we can blame it on
them violating the rule not to use __xxxx__ names except as
documented).

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2003-01-13 00:28

Message:
Logged In: YES 
user_id=31435

try_rich_to_3way_compare gives up after LT, EQ and GT all 
return false.  The comparison then falls back to the default 
comparison by object id (addresses).  So, e.g., I can build 
an a, b, and c on my box such that:

>>> a, b, c
(Set([0]), Set([1]), Set([1]))
>>> cmp(b, c)   # b equals c
0
>>> cmp(a, b)   # but a "greater than" b while
1
>>> cmp(a, c)   # a "less than" c, despite that b == c
-1
>>>

This is simply because a is at a lower address than c but at 
a higher address than b:

>>> id(a) < id(c)
True
>>> id(a) > id(b)
True
>>>

I don't think <= for subset is too cute -- sets do have a 
natural and useful partial order.  I don't really care happens 
if you pass them to cmp(), though.  An exception would be 
fine -- sets weren't intended to be "generally" comparable.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-01-12 09:28

Message:
Logged In: YES 
user_id=6380

It's unlikely that the built-in type will behave exactly the
same, so the name difference is a good way to distinguish
the module from a future builtin.

Regarding cmp(): hmm, we may need to think about this more.
Currently comparison operators are used to express subset
relationships, e.g. <= means subset, < means strict subset.
The result of this is that cmp() between sets that are
neither subset nor set gets a useless outcome. (I can
reproduce your example but can't explain it.) Maybe we
should implement __cmp__ to raise an exception? Or maybe <=
for subset is too cute?

Assigning for Tim to ask his opinion.

----------------------------------------------------------------------

Comment By: Terry J. Reedy (tjreedy)
Date: 2003-01-07 17:44

Message:
Logged In: YES 
user_id=593130

If/when sets become builtin, would type object be 
called 'set' or break current convention by being 
called 'Set'?  If the former, I think classSet should be 'set' 
now so one can write 'from sets import set' and have rest 
of code ready for the future.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=663701&group_id=5470


From noreply@sourceforge.net  Fri Feb 14 04:34:29 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 13 Feb 2003 20:34:29 -0800
Subject: [Python-bugs-list] [ python-Feature Requests-686323 ] Minor array module enhancements
Message-ID: <E18jXYT-0000x2-00@sc8-sf-web3.sourceforge.net>

Feature Requests item #686323, was opened at 2003-02-13 20:13
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=686323&group_id=5470

>Category: None
>Group: None
Status: Open
Resolution: None
>Priority: 4
Submitted By: paul rubin (phr)
Assigned to: Nobody/Anonymous (nobody)
Summary: Minor array module enhancements

Initial Comment:
1) I notice that array('B', (2,3,4)) throws an error
saying the
2nd arg must be a list or string.  That is, array('B',
[2,3,4]) is
legal but using the tuple (2,3,4) is not.  The
limitation doesn't
seem harmful, but I also don't see any good reason for
it.  May
as well remove it.

2) Of more usefulness (and maybe of more controversy), I
think the byte array methods should accept string
arguments,
for example
   a = array('B', 'abc')
   a.append('def')
should do the obvious thing.  That gives a natural way to
build up a string in pieces instead of making a list of
strings
and doing the counterintuitive ''.join(x) maneuver. 
Appending
to byte arrays is the usual way to build up a string in
Java,
if that matters.  So I favor this change too.

----------------------------------------------------------------------

>Comment By: Raymond Hettinger (rhettinger)
Date: 2003-02-13 23:34

Message:
Logged In: YES 
user_id=80475

The first request is reasonable.

The second is not the natural meaning of append().  In 
python, multiple appends are handled with extend().  The 
array module already provides extend.  So, for the 
example given above, the code is:

>>> a = array('B', 'abc')
>>> a.extend(array('B', 'def'))
>>> a
array('B', [97, 98, 99, 100, 101, 102])

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=686323&group_id=5470


From noreply@sourceforge.net  Fri Feb 14 04:37:41 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 13 Feb 2003 20:37:41 -0800
Subject: [Python-bugs-list] [ python-Feature Requests-686323 ] Minor array module enhancements
Message-ID: <E18jXbZ-000148-00@sc8-sf-web3.sourceforge.net>

Feature Requests item #686323, was opened at 2003-02-13 20:13
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=686323&group_id=5470

>Category: Extension Modules
Group: None
Status: Open
Resolution: None
Priority: 4
Submitted By: paul rubin (phr)
>Assigned to: Raymond Hettinger (rhettinger)
Summary: Minor array module enhancements

Initial Comment:
1) I notice that array('B', (2,3,4)) throws an error
saying the
2nd arg must be a list or string.  That is, array('B',
[2,3,4]) is
legal but using the tuple (2,3,4) is not.  The
limitation doesn't
seem harmful, but I also don't see any good reason for
it.  May
as well remove it.

2) Of more usefulness (and maybe of more controversy), I
think the byte array methods should accept string
arguments,
for example
   a = array('B', 'abc')
   a.append('def')
should do the obvious thing.  That gives a natural way to
build up a string in pieces instead of making a list of
strings
and doing the counterintuitive ''.join(x) maneuver. 
Appending
to byte arrays is the usual way to build up a string in
Java,
if that matters.  So I favor this change too.

----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2003-02-13 23:34

Message:
Logged In: YES 
user_id=80475

The first request is reasonable.

The second is not the natural meaning of append().  In 
python, multiple appends are handled with extend().  The 
array module already provides extend.  So, for the 
example given above, the code is:

>>> a = array('B', 'abc')
>>> a.extend(array('B', 'def'))
>>> a
array('B', [97, 98, 99, 100, 101, 102])

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=686323&group_id=5470


From noreply@sourceforge.net  Fri Feb 14 04:42:21 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 13 Feb 2003 20:42:21 -0800
Subject: [Python-bugs-list] [ python-Feature Requests-686323 ] Minor array module enhancements
Message-ID: <E18jXg5-0001C7-00@sc8-sf-web3.sourceforge.net>

Feature Requests item #686323, was opened at 2003-02-14 01:13
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=686323&group_id=5470

Category: Extension Modules
Group: None
Status: Open
Resolution: None
Priority: 4
Submitted By: paul rubin (phr)
Assigned to: Raymond Hettinger (rhettinger)
Summary: Minor array module enhancements

Initial Comment:
1) I notice that array('B', (2,3,4)) throws an error
saying the
2nd arg must be a list or string.  That is, array('B',
[2,3,4]) is
legal but using the tuple (2,3,4) is not.  The
limitation doesn't
seem harmful, but I also don't see any good reason for
it.  May
as well remove it.

2) Of more usefulness (and maybe of more controversy), I
think the byte array methods should accept string
arguments,
for example
   a = array('B', 'abc')
   a.append('def')
should do the obvious thing.  That gives a natural way to
build up a string in pieces instead of making a list of
strings
and doing the counterintuitive ''.join(x) maneuver. 
Appending
to byte arrays is the usual way to build up a string in
Java,
if that matters.  So I favor this change too.

----------------------------------------------------------------------

>Comment By: paul rubin (phr)
Date: 2003-02-14 04:42

Message:
Logged In: YES 
user_id=72053

You're correct, second request should be for extend rather
than append.

That is,

a = array('B', 'abc')
a.extend('def')

should do the obvious thing.


----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2003-02-14 04:34

Message:
Logged In: YES 
user_id=80475

The first request is reasonable.

The second is not the natural meaning of append().  In 
python, multiple appends are handled with extend().  The 
array module already provides extend.  So, for the 
example given above, the code is:

>>> a = array('B', 'abc')
>>> a.extend(array('B', 'def'))
>>> a
array('B', [97, 98, 99, 100, 101, 102])

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=686323&group_id=5470


From noreply@sourceforge.net  Fri Feb 14 04:47:45 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 13 Feb 2003 20:47:45 -0800
Subject: [Python-bugs-list] [ python-Feature Requests-500698 ] Taint a la Perl?
Message-ID: <E18jXlJ-0005Hn-00@sc8-sf-web2.sourceforge.net>

Feature Requests item #500698, was opened at 2002-01-08 02:48
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=500698&group_id=5470

Category: Python Interpreter Core
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Peter Scott (sketerpot)
Assigned to: Nobody/Anonymous (nobody)
Summary: Taint a la Perl?

Initial Comment:
This might just add unnecessary bloat, but since Python is being 
used in CGI scripts, it can be used to narrow a security hole. One way 
of breaking security is for a naiive programmer (don't try to deny 
their existance) to run an arbitrary command from the page 
viewer.

Perl has developed an interesting mechanism for 
helping with this: taint. The way it works is, when something comes 
directly from the user, like a key in a form, it is considered to have 
taint unless specifically untainted. Things like os.exec() would 
create a warning message if you passed tainted strings to 
them.

As I said, this might just add unnecessary bloat, but for 
an option that can be left out for most builds of Python I think it 
would be pretty nice.

----------------------------------------------------------------------

Comment By: paul rubin (phr)
Date: 2003-02-14 04:47

Message:
Logged In: YES 
user_id=72053

With new-style classes, maybe this can be done by
subclassing string somehow.  There would be a subclass for
tainted strings and trying to do most things with them would
raise an exception.  With taint checking enabled, functions
like os.getenv and cgi.FieldStorage would make objects
containing tainted strings.  You'd untaint them by passing
them to re.search or re.match and pulling out the match
variables, like in Per.

----------------------------------------------------------------------

Comment By: Skip Montanaro (montanaro)
Date: 2003-01-03 01:25

Message:
Logged In: YES 
user_id=44345

Took awhile for a response to this feature request. ;-)

Perl's heavy integration of regular expressions with its
taint facility probably wouldn't work all that well in
Python.  For one, Python has more ways of searching
strings than with regular expressions.  Second, regular
expressions are not nearly as tightly wound into Python
as they are in Perl.  I think you'd have to add a taint
attribute to strings and just rely on the programmer to
properly clear that attribute.

I think a first cut at an implementation would go much
further toward getting the concept seriously considered
for addition to Python.


----------------------------------------------------------------------

Comment By: Neal McBurnett (nealmcb)
Date: 2003-01-02 21:20

Message:
Logged In: YES 
user_id=105956

I really like taint mode.
I think this would make Python a better choice for CGI scripts.

See http://www.perldoc.com/perl5.8.0/pod/perlsec.html
and http://gunther.web66.com/FAQS/taintmode.html
for more background.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=500698&group_id=5470


From noreply@sourceforge.net  Fri Feb 14 05:11:11 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 13 Feb 2003 21:11:11 -0800
Subject: [Python-bugs-list] [ python-Bugs-686380 ] errors trying to get help on os attributes
Message-ID: <E18jY7z-0005f1-00@sc8-sf-web4.sourceforge.net>

Bugs item #686380, was opened at 2003-02-13 23:11
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=686380&group_id=5470

Category: Python Library
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Skip Montanaro (montanaro)
Assigned to: Nobody/Anonymous (nobody)
Summary: errors trying to get help on os attributes

Initial Comment:
I noticed some very weird behavior w/ 2.3 when asking for
help about some data attributes in the os module.  Consider:

% python
Python 2.3a1 (#13, Feb  4 2003, 09:26:29) 
[GCC 3.1 20020420 (prerelease)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> help(os.sep)
no Python documentation found for '/'

>>> help(os.extsep)
problem in  - ValueError: Empty module name

>>> help(os.pathsep)
no Python documentation found for ':'

>>> help(os.curdir)
problem in  - ValueError: Empty module name

Now the same feat of prestidigation using 2.2.2 (on Mac OSX):

% python2.2 
Python 2.2.2 (#7, Oct 14 2002, 18:07:17) 
[GCC 3.1 20020420 (prerelease)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> help(os.sep)
no Python documentation found for '/'

>>> help(os.extsep)
problem in  - ValueError: Empty module name

>>> help(os.pathsep)
no Python documentation found for ':'

>>> help(os.curdir)
problem in  - ValueError: Empty module name

Finally, the same example using 2.2.2 under Mandrake Linux:

% python
Python 2.2.2 (#1, Nov  8 2002, 08:50:21) 
[GCC 2.96 20000731 (Mandrake Linux 8.1 2.96-0.62mdk)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> help(os.sep)
str(object) -> string

Return a nice string representation of the object.
If the argument is a string, the return value is the same object.
>>> help(os.extsep)
str(object) -> string

Return a nice string representation of the object.
If the argument is a string, the return value is the same object.
>>> help(os.pathsep)
str(object) -> string

Return a nice string representation of the object.
If the argument is a string, the return value is the same object.
>>> help(os.curdir)
str(object) -> string

Return a nice string representation of the object.
If the argument is a string, the return value is the same object.

Now, I can understand that maybe I've hosed my CVS tree,
which would affect the 2.3 build, but the 2.2.2 build on both
Mac OS X and Mandrake Linux is I believe straight from the
source with no CVS monkey business.

Can anyone else reproduce this?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=686380&group_id=5470


From noreply@sourceforge.net  Fri Feb 14 05:16:10 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 13 Feb 2003 21:16:10 -0800
Subject: [Python-bugs-list] [ python-Bugs-683658 ] PyErr_Warn may cause import deadlock
Message-ID: <E18jYCo-0002cq-00@sc8-sf-web1.sourceforge.net>

Bugs item #683658, was opened at 2003-02-10 11:26
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=683658&group_id=5470

Category: Python Interpreter Core
Group: Python 2.3
Status: Open
Resolution: None
Priority: 6
Submitted By: Mark Hammond (mhammond)
>Assigned to: Mark Hammond (mhammond)
Summary: PyErr_Warn may cause import deadlock

Initial Comment:
PyErr_Warn() does an implicit import.  Thus, if
PyErr_Warn() is called on a thread while the main
thread holds the import lock, and the main thread then
subsequently waits for the child thread, Python will
deadlock.  The builtin 'apply' now calls PyErr_Warn(),
so simply calling 'apply' on another thread may cause
the deadlock.

Attaching a sample case.  Executing 'python -c "import
hang_apply"' will cause Python to deadlock.  Commenting
out the call to "apply" in that file will prevent the
deadlock (even though the apply() in question is
effectively a no-op)

The example is a little contrived, but is extracted
from real code of mine that saw this hang.  The code
path is:

* Main thread acquires import lock to import the module.
* Module import spawns a new thread.  This new thread
calls apply()
* builtin_apply() calls PyErr_Warn
* PyErr_Warn does an import of the warnings module, and
attempts to acquire the import lock.
* Main thread still waiting for child thread to
complete, but still holds import lock.

Note that removing the call to PyErr_Warn() in
builtin_apply also solves this particular hang -
however, it seems like this is a potential time bomb. 
A potential solution would be to prevent PyErr_Warn
from doing an import - this would mean importing
'warnings' at startup, and keeping a static reference
in errors.c.  Other thoughts welcome.


----------------------------------------------------------------------

>Comment By: Mark Hammond (mhammond)
Date: 2003-02-14 16:16

Message:
Logged In: YES 
user_id=14198

This simple strategy doesn't work - avoiding the import of
'warnings' works fine, until 'warnings' imports 'linecache'!

I'll look at how we can simply throw the warning away if a
deadlock would occur.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-12 10:23

Message:
Logged In: YES 
user_id=33168

I can't think of any other/better solution.  Any idea if
there are there other ticking bombs like this?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=683658&group_id=5470


From noreply@sourceforge.net  Fri Feb 14 13:23:31 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Fri, 14 Feb 2003 05:23:31 -0800
Subject: [Python-bugs-list] [ python-Bugs-686536 ] raw_input without echo
Message-ID: <E18jfoR-0006UY-00@sc8-sf-web4.sourceforge.net>

Bugs item #686536, was opened at 2003-02-14 13:23
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=686536&group_id=5470

Category: None
Group: Feature Request
Status: Open
Resolution: None
Priority: 5
Submitted By: a.hofkamp (axhlkhb)
Assigned to: Nobody/Anonymous (nobody)
Summary: raw_input without echo

Initial Comment:
Hello,

When entering data from the keyboard that should not
become known (like passwords), it is not possible to
prevent raw_input from echo-ing the entered data.

By adding an 'echo' flag (which defaults to 1) to
raw_input() such data can be queried from the keyboard
easily (safely is another matter, but then again that
is mainly a matter of how paranoid you are).

On systems that support more secure input methods, one
could even switch to these moethods (e.g. using
/dev/tty on a Unix system).

Albert


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=686536&group_id=5470


From noreply@sourceforge.net  Fri Feb 14 13:51:51 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Fri, 14 Feb 2003 05:51:51 -0800
Subject: [Python-bugs-list] [ python-Bugs-686536 ] raw_input without echo
Message-ID: <E18jgFr-0003ZS-00@sc8-sf-web3.sourceforge.net>

Bugs item #686536, was opened at 2003-02-14 13:23
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=686536&group_id=5470

Category: None
Group: Feature Request
Status: Open
Resolution: None
Priority: 5
Submitted By: a.hofkamp (axhlkhb)
Assigned to: Nobody/Anonymous (nobody)
Summary: raw_input without echo

Initial Comment:
Hello,

When entering data from the keyboard that should not
become known (like passwords), it is not possible to
prevent raw_input from echo-ing the entered data.

By adding an 'echo' flag (which defaults to 1) to
raw_input() such data can be queried from the keyboard
easily (safely is another matter, but then again that
is mainly a matter of how paranoid you are).

On systems that support more secure input methods, one
could even switch to these moethods (e.g. using
/dev/tty on a Unix system).

Albert


----------------------------------------------------------------------

>Comment By: Michael Hudson (mwh)
Date: 2003-02-14 13:51

Message:
Logged In: YES 
user_id=6656

Err, do you know about the getpass module?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=686536&group_id=5470


From noreply@sourceforge.net  Fri Feb 14 14:06:22 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Fri, 14 Feb 2003 06:06:22 -0800
Subject: [Python-bugs-list] [ python-Bugs-686536 ] raw_input without echo
Message-ID: <E18jgTu-0006xu-00@sc8-sf-web2.sourceforge.net>

Bugs item #686536, was opened at 2003-02-14 13:23
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=686536&group_id=5470

Category: None
Group: Feature Request
Status: Open
Resolution: None
Priority: 5
Submitted By: a.hofkamp (axhlkhb)
Assigned to: Nobody/Anonymous (nobody)
Summary: raw_input without echo

Initial Comment:
Hello,

When entering data from the keyboard that should not
become known (like passwords), it is not possible to
prevent raw_input from echo-ing the entered data.

By adding an 'echo' flag (which defaults to 1) to
raw_input() such data can be queried from the keyboard
easily (safely is another matter, but then again that
is mainly a matter of how paranoid you are).

On systems that support more secure input methods, one
could even switch to these moethods (e.g. using
/dev/tty on a Unix system).

Albert


----------------------------------------------------------------------

>Comment By: a.hofkamp (axhlkhb)
Date: 2003-02-14 14:06

Message:
Logged In: YES 
user_id=438586

No, I didn't know about getpass.

Let me rephrase the question then
Why is there getpass.getpass() when almost the same
functionality exists as raw_input()

Fragmentation of functionality is bound to throw users
off-track, having them search at the wrong place, and
submitting unneeded feature requests :-)

Albert


----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2003-02-14 13:51

Message:
Logged In: YES 
user_id=6656

Err, do you know about the getpass module?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=686536&group_id=5470


From noreply@sourceforge.net  Fri Feb 14 15:00:12 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Fri, 14 Feb 2003 07:00:12 -0800
Subject: [Python-bugs-list] [ python-Bugs-686536 ] raw_input without echo
Message-ID: <E18jhK0-0006qX-00@sc8-sf-web3.sourceforge.net>

Bugs item #686536, was opened at 2003-02-14 13:23
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=686536&group_id=5470

Category: None
Group: Feature Request
>Status: Closed
>Resolution: Rejected
Priority: 5
Submitted By: a.hofkamp (axhlkhb)
Assigned to: Nobody/Anonymous (nobody)
Summary: raw_input without echo

Initial Comment:
Hello,

When entering data from the keyboard that should not
become known (like passwords), it is not possible to
prevent raw_input from echo-ing the entered data.

By adding an 'echo' flag (which defaults to 1) to
raw_input() such data can be queried from the keyboard
easily (safely is another matter, but then again that
is mainly a matter of how paranoid you are).

On systems that support more secure input methods, one
could even switch to these moethods (e.g. using
/dev/tty on a Unix system).

Albert


----------------------------------------------------------------------

>Comment By: Michael Hudson (mwh)
Date: 2003-02-14 15:00

Message:
Logged In: YES 
user_id=6656

I don't think it's possible to add non-echoing functionality
to raw_input() in a sane manner.  It's also not IMHO a
natural extensions of the interface -- a non-echoing input
isn't a common enough requirement to justify messing with a
builtin.  So I'm rejecting this.

----------------------------------------------------------------------

Comment By: a.hofkamp (axhlkhb)
Date: 2003-02-14 14:06

Message:
Logged In: YES 
user_id=438586

No, I didn't know about getpass.

Let me rephrase the question then
Why is there getpass.getpass() when almost the same
functionality exists as raw_input()

Fragmentation of functionality is bound to throw users
off-track, having them search at the wrong place, and
submitting unneeded feature requests :-)

Albert


----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2003-02-14 13:51

Message:
Logged In: YES 
user_id=6656

Err, do you know about the getpass module?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=686536&group_id=5470


From noreply@sourceforge.net  Fri Feb 14 17:11:41 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Fri, 14 Feb 2003 09:11:41 -0800
Subject: [Python-bugs-list] [ python-Bugs-686380 ] errors trying to get help on os attributes
Message-ID: <E18jjNF-0007IJ-00@sc8-sf-web2.sourceforge.net>

Bugs item #686380, was opened at 2003-02-13 23:11
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=686380&group_id=5470

Category: Python Library
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Skip Montanaro (montanaro)
Assigned to: Nobody/Anonymous (nobody)
Summary: errors trying to get help on os attributes

Initial Comment:
I noticed some very weird behavior w/ 2.3 when asking for
help about some data attributes in the os module.  Consider:

% python
Python 2.3a1 (#13, Feb  4 2003, 09:26:29) 
[GCC 3.1 20020420 (prerelease)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> help(os.sep)
no Python documentation found for '/'

>>> help(os.extsep)
problem in  - ValueError: Empty module name

>>> help(os.pathsep)
no Python documentation found for ':'

>>> help(os.curdir)
problem in  - ValueError: Empty module name

Now the same feat of prestidigation using 2.2.2 (on Mac OSX):

% python2.2 
Python 2.2.2 (#7, Oct 14 2002, 18:07:17) 
[GCC 3.1 20020420 (prerelease)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> help(os.sep)
no Python documentation found for '/'

>>> help(os.extsep)
problem in  - ValueError: Empty module name

>>> help(os.pathsep)
no Python documentation found for ':'

>>> help(os.curdir)
problem in  - ValueError: Empty module name

Finally, the same example using 2.2.2 under Mandrake Linux:

% python
Python 2.2.2 (#1, Nov  8 2002, 08:50:21) 
[GCC 2.96 20000731 (Mandrake Linux 8.1 2.96-0.62mdk)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> help(os.sep)
str(object) -> string

Return a nice string representation of the object.
If the argument is a string, the return value is the same object.
>>> help(os.extsep)
str(object) -> string

Return a nice string representation of the object.
If the argument is a string, the return value is the same object.
>>> help(os.pathsep)
str(object) -> string

Return a nice string representation of the object.
If the argument is a string, the return value is the same object.
>>> help(os.curdir)
str(object) -> string

Return a nice string representation of the object.
If the argument is a string, the return value is the same object.

Now, I can understand that maybe I've hosed my CVS tree,
which would affect the 2.3 build, but the 2.2.2 build on both
Mac OS X and Mandrake Linux is I believe straight from the
source with no CVS monkey business.

Can anyone else reproduce this?

----------------------------------------------------------------------

Comment By: logistix (logistix)
Date: 2003-02-14 11:11

Message:
Logged In: YES 
user_id=699438

I think you're overthinking things.  A string instance won't 
have it's own specific docstring.

Python 2.3a1 (#38, Dec 31 2002, 17:53:59) [MSC v.1200 32 
bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more 
information.
>>> import os
>>> help(os.sep)
no Python documentation found for '\'

>>> type(os.sep)
<type 'str'>
>>>

Maybe the versions you migrate into os.path should be 
functions (or will that break too much?)

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=686380&group_id=5470


From noreply@sourceforge.net  Fri Feb 14 17:21:11 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Fri, 14 Feb 2003 09:21:11 -0800
Subject: [Python-bugs-list] [ python-Feature Requests-500698 ] Taint a la Perl?
Message-ID: <E18jjWR-0001W2-00@sc8-sf-web4.sourceforge.net>

Feature Requests item #500698, was opened at 2002-01-07 19:48
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=500698&group_id=5470

Category: Python Interpreter Core
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Peter Scott (sketerpot)
Assigned to: Nobody/Anonymous (nobody)
Summary: Taint a la Perl?

Initial Comment:
This might just add unnecessary bloat, but since Python is being 
used in CGI scripts, it can be used to narrow a security hole. One way 
of breaking security is for a naiive programmer (don't try to deny 
their existance) to run an arbitrary command from the page 
viewer.

Perl has developed an interesting mechanism for 
helping with this: taint. The way it works is, when something comes 
directly from the user, like a key in a form, it is considered to have 
taint unless specifically untainted. Things like os.exec() would 
create a warning message if you passed tainted strings to 
them.

As I said, this might just add unnecessary bloat, but for 
an option that can be left out for most builds of Python I think it 
would be pretty nice.

----------------------------------------------------------------------

>Comment By: Peter Scott (sketerpot)
Date: 2003-02-14 10:21

Message:
Logged In: YES 
user_id=252564

Thanks for the idea, phr. I wrote a small class called 
TaintString, derived from string, that has a taint attribute. This 
is probably the least difficult part. The difficult part will be in 
modifying functions like os.system() to raise warnings or 
exceptions when tainted strings are passed to them. I'm 
currently thinking of making wrapper modules with names like 
taint.os, or taint.cgi, but the problem with this is that you 
have to manually use taint.* for certain functions. If anybody 
can think of something that can simplify this, please post it.

----------------------------------------------------------------------

Comment By: paul rubin (phr)
Date: 2003-02-13 21:47

Message:
Logged In: YES 
user_id=72053

With new-style classes, maybe this can be done by
subclassing string somehow.  There would be a subclass for
tainted strings and trying to do most things with them would
raise an exception.  With taint checking enabled, functions
like os.getenv and cgi.FieldStorage would make objects
containing tainted strings.  You'd untaint them by passing
them to re.search or re.match and pulling out the match
variables, like in Per.

----------------------------------------------------------------------

Comment By: Skip Montanaro (montanaro)
Date: 2003-01-02 18:25

Message:
Logged In: YES 
user_id=44345

Took awhile for a response to this feature request. ;-)

Perl's heavy integration of regular expressions with its
taint facility probably wouldn't work all that well in
Python.  For one, Python has more ways of searching
strings than with regular expressions.  Second, regular
expressions are not nearly as tightly wound into Python
as they are in Perl.  I think you'd have to add a taint
attribute to strings and just rely on the programmer to
properly clear that attribute.

I think a first cut at an implementation would go much
further toward getting the concept seriously considered
for addition to Python.


----------------------------------------------------------------------

Comment By: Neal McBurnett (nealmcb)
Date: 2003-01-02 14:20

Message:
Logged In: YES 
user_id=105956

I really like taint mode.
I think this would make Python a better choice for CGI scripts.

See http://www.perldoc.com/perl5.8.0/pod/perlsec.html
and http://gunther.web66.com/FAQS/taintmode.html
for more background.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=500698&group_id=5470


From noreply@sourceforge.net  Fri Feb 14 17:26:47 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Fri, 14 Feb 2003 09:26:47 -0800
Subject: [Python-bugs-list] [ python-Bugs-686380 ] errors trying to get help on os attributes
Message-ID: <E18jjbr-0006Q0-00@sc8-sf-web1.sourceforge.net>

Bugs item #686380, was opened at 2003-02-13 23:11
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=686380&group_id=5470

Category: Python Library
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Skip Montanaro (montanaro)
Assigned to: Nobody/Anonymous (nobody)
Summary: errors trying to get help on os attributes

Initial Comment:
I noticed some very weird behavior w/ 2.3 when asking for
help about some data attributes in the os module.  Consider:

% python
Python 2.3a1 (#13, Feb  4 2003, 09:26:29) 
[GCC 3.1 20020420 (prerelease)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> help(os.sep)
no Python documentation found for '/'

>>> help(os.extsep)
problem in  - ValueError: Empty module name

>>> help(os.pathsep)
no Python documentation found for ':'

>>> help(os.curdir)
problem in  - ValueError: Empty module name

Now the same feat of prestidigation using 2.2.2 (on Mac OSX):

% python2.2 
Python 2.2.2 (#7, Oct 14 2002, 18:07:17) 
[GCC 3.1 20020420 (prerelease)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> help(os.sep)
no Python documentation found for '/'

>>> help(os.extsep)
problem in  - ValueError: Empty module name

>>> help(os.pathsep)
no Python documentation found for ':'

>>> help(os.curdir)
problem in  - ValueError: Empty module name

Finally, the same example using 2.2.2 under Mandrake Linux:

% python
Python 2.2.2 (#1, Nov  8 2002, 08:50:21) 
[GCC 2.96 20000731 (Mandrake Linux 8.1 2.96-0.62mdk)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> help(os.sep)
str(object) -> string

Return a nice string representation of the object.
If the argument is a string, the return value is the same object.
>>> help(os.extsep)
str(object) -> string

Return a nice string representation of the object.
If the argument is a string, the return value is the same object.
>>> help(os.pathsep)
str(object) -> string

Return a nice string representation of the object.
If the argument is a string, the return value is the same object.
>>> help(os.curdir)
str(object) -> string

Return a nice string representation of the object.
If the argument is a string, the return value is the same object.

Now, I can understand that maybe I've hosed my CVS tree,
which would affect the 2.3 build, but the 2.2.2 build on both
Mac OS X and Mandrake Linux is I believe straight from the
source with no CVS monkey business.

Can anyone else reproduce this?

----------------------------------------------------------------------

>Comment By: Skip Montanaro (montanaro)
Date: 2003-02-14 11:26

Message:
Logged In: YES 
user_id=44345

I can understand the "no docstring" message, but that doesn't explain
the "problem in  - ValueError: Empty module name" error from help().
Also, the distinction I was trying to make between the 2.2.2 on Linux
and MacOSX was that on the former, the docstrings were consistent
(and what you'd expect for a string), while on the latter there were
two different responses, neither of which matched the response I
saw on Linux.

----------------------------------------------------------------------

Comment By: logistix (logistix)
Date: 2003-02-14 11:11

Message:
Logged In: YES 
user_id=699438

I think you're overthinking things.  A string instance won't 
have it's own specific docstring.

Python 2.3a1 (#38, Dec 31 2002, 17:53:59) [MSC v.1200 32 
bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more 
information.
>>> import os
>>> help(os.sep)
no Python documentation found for '\'

>>> type(os.sep)
<type 'str'>
>>>

Maybe the versions you migrate into os.path should be 
functions (or will that break too much?)

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=686380&group_id=5470


From noreply@sourceforge.net  Fri Feb 14 17:47:18 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Fri, 14 Feb 2003 09:47:18 -0800
Subject: [Python-bugs-list] [ python-Bugs-686667 ] os.spawnv(P_WAIT, ...) on Linux doesn't handle EINTR
Message-ID: <E18jjvi-0005pn-00@sc8-sf-web3.sourceforge.net>

Bugs item #686667, was opened at 2003-02-14 18:47
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=686667&group_id=5470

Category: Python Library
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Bernhard Herzog (bernhard)
Assigned to: Nobody/Anonymous (nobody)
Summary: os.spawnv(P_WAIT, ...) on Linux doesn't handle EINTR

Initial Comment:
The implementation of os.spawnv when called with the
P_WAIT flag calls waitpid to wait for the subprocess.
If this function is aborted early because of a signal,
i.e. if it raises OSError with EINTR, it should be
called again.

I ran across this bug when trying to write a test case
for a script that stops another process. Both the
script and the other process are executed as
subprocesses of the test program. The stop script is
executed with os.spawnv(P_WAIT,  ...) to wait until the
script completed. Unfortunately when it stops the other
process a SIGCHLD is sent to the test program which
then aborts the waitpid with an exception.

Tested with Python 2.1.3, 2.2 and CVS from 2003-02-13
Platform: Debian GNU/Linux, Kernel 2.4.20

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=686667&group_id=5470


From noreply@sourceforge.net  Fri Feb 14 18:50:48 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Fri, 14 Feb 2003 10:50:48 -0800
Subject: [Python-bugs-list] [ python-Bugs-686380 ] errors trying to get help on os attributes
Message-ID: <E18jkvA-0000S0-00@sc8-sf-web3.sourceforge.net>

Bugs item #686380, was opened at 2003-02-13 23:11
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=686380&group_id=5470

Category: Python Library
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Skip Montanaro (montanaro)
Assigned to: Nobody/Anonymous (nobody)
Summary: errors trying to get help on os attributes

Initial Comment:
I noticed some very weird behavior w/ 2.3 when asking for
help about some data attributes in the os module.  Consider:

% python
Python 2.3a1 (#13, Feb  4 2003, 09:26:29) 
[GCC 3.1 20020420 (prerelease)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> help(os.sep)
no Python documentation found for '/'

>>> help(os.extsep)
problem in  - ValueError: Empty module name

>>> help(os.pathsep)
no Python documentation found for ':'

>>> help(os.curdir)
problem in  - ValueError: Empty module name

Now the same feat of prestidigation using 2.2.2 (on Mac OSX):

% python2.2 
Python 2.2.2 (#7, Oct 14 2002, 18:07:17) 
[GCC 3.1 20020420 (prerelease)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> help(os.sep)
no Python documentation found for '/'

>>> help(os.extsep)
problem in  - ValueError: Empty module name

>>> help(os.pathsep)
no Python documentation found for ':'

>>> help(os.curdir)
problem in  - ValueError: Empty module name

Finally, the same example using 2.2.2 under Mandrake Linux:

% python
Python 2.2.2 (#1, Nov  8 2002, 08:50:21) 
[GCC 2.96 20000731 (Mandrake Linux 8.1 2.96-0.62mdk)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> help(os.sep)
str(object) -> string

Return a nice string representation of the object.
If the argument is a string, the return value is the same object.
>>> help(os.extsep)
str(object) -> string

Return a nice string representation of the object.
If the argument is a string, the return value is the same object.
>>> help(os.pathsep)
str(object) -> string

Return a nice string representation of the object.
If the argument is a string, the return value is the same object.
>>> help(os.curdir)
str(object) -> string

Return a nice string representation of the object.
If the argument is a string, the return value is the same object.

Now, I can understand that maybe I've hosed my CVS tree,
which would affect the 2.3 build, but the 2.2.2 build on both
Mac OS X and Mandrake Linux is I believe straight from the
source with no CVS monkey business.

Can anyone else reproduce this?

----------------------------------------------------------------------

Comment By: logistix (logistix)
Date: 2003-02-14 12:50

Message:
Logged In: YES 
user_id=699438

DOH! I'm an idiot.

You get the same value errors in Win32 in 2.2.2 and 2.3.1.  It 
looks it's the "locate" function in pydoc.  It tries to split a 
string by '.' in an attempt to resolve namespaces.  When you 
run that on a string like '.', '..', you get some empty strings 
and bad things happen.

I've got a patch that looks like it works.  It just uses a list 
comp to filter out null strings in the above situation.  I'll post it 
after running the test suite.

I don't know why Mandrake produced different results.

----------------------------------------------------------------------

Comment By: Skip Montanaro (montanaro)
Date: 2003-02-14 11:26

Message:
Logged In: YES 
user_id=44345

I can understand the "no docstring" message, but that doesn't explain
the "problem in  - ValueError: Empty module name" error from help().
Also, the distinction I was trying to make between the 2.2.2 on Linux
and MacOSX was that on the former, the docstrings were consistent
(and what you'd expect for a string), while on the latter there were
two different responses, neither of which matched the response I
saw on Linux.

----------------------------------------------------------------------

Comment By: logistix (logistix)
Date: 2003-02-14 11:11

Message:
Logged In: YES 
user_id=699438

I think you're overthinking things.  A string instance won't 
have it's own specific docstring.

Python 2.3a1 (#38, Dec 31 2002, 17:53:59) [MSC v.1200 32 
bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more 
information.
>>> import os
>>> help(os.sep)
no Python documentation found for '\'

>>> type(os.sep)
<type 'str'>
>>>

Maybe the versions you migrate into os.path should be 
functions (or will that break too much?)

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=686380&group_id=5470


From noreply@sourceforge.net  Fri Feb 14 18:56:24 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Fri, 14 Feb 2003 10:56:24 -0800
Subject: [Python-bugs-list] [ python-Bugs-602444 ] non greedy match bug
Message-ID: <E18jl0a-0001z7-00@sc8-sf-web1.sourceforge.net>

Bugs item #602444, was opened at 2002-08-30 10:44
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=602444&group_id=5470

Category: Regular Expressions
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Robert Roy (rjroy)
Assigned to: Fredrik Lundh (effbot)
Summary: non greedy match bug

Initial Comment:
When using the following re to extract all objects from a 
PDF file, I get a maximum recursion limit exceeded error.

Attached is a pdf file that will reproduce the error.

If I do import pre as re, it works fine.

platform is Win2k, Python 2.2.1 build #34

#######
import re

GETOBJECT = re.compile(r'\d+\s+\d+\s+obj.+?endobj', 
re.I|re.S|re.M)

pdf = open('userguide.pdf', 'rb').read()
all = GETOBJECT.findall(pdf)
print len(all)


----------------------------------------------------------------------

Comment By: Robert Roy (rjroy)
Date: 2003-02-14 13:56

Message:
Logged In: YES 
user_id=352797

The max recursion limit problem in the re module is well-known.  
Until this limitation in the implementation is removed, to work 
around it check

http://www.python.org/dev/doc/devel/lib/module-re.html
http://python/org/sf/493252

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=602444&group_id=5470


From noreply@sourceforge.net  Fri Feb 14 20:57:06 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Fri, 14 Feb 2003 12:57:06 -0800
Subject: [Python-bugs-list] [ python-Bugs-686380 ] errors trying to get help on os attributes
Message-ID: <E18jmtO-0003Zw-00@sc8-sf-web4.sourceforge.net>

Bugs item #686380, was opened at 2003-02-13 23:11
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=686380&group_id=5470

Category: Python Library
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Skip Montanaro (montanaro)
Assigned to: Nobody/Anonymous (nobody)
Summary: errors trying to get help on os attributes

Initial Comment:
I noticed some very weird behavior w/ 2.3 when asking for
help about some data attributes in the os module.  Consider:

% python
Python 2.3a1 (#13, Feb  4 2003, 09:26:29) 
[GCC 3.1 20020420 (prerelease)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> help(os.sep)
no Python documentation found for '/'

>>> help(os.extsep)
problem in  - ValueError: Empty module name

>>> help(os.pathsep)
no Python documentation found for ':'

>>> help(os.curdir)
problem in  - ValueError: Empty module name

Now the same feat of prestidigation using 2.2.2 (on Mac OSX):

% python2.2 
Python 2.2.2 (#7, Oct 14 2002, 18:07:17) 
[GCC 3.1 20020420 (prerelease)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> help(os.sep)
no Python documentation found for '/'

>>> help(os.extsep)
problem in  - ValueError: Empty module name

>>> help(os.pathsep)
no Python documentation found for ':'

>>> help(os.curdir)
problem in  - ValueError: Empty module name

Finally, the same example using 2.2.2 under Mandrake Linux:

% python
Python 2.2.2 (#1, Nov  8 2002, 08:50:21) 
[GCC 2.96 20000731 (Mandrake Linux 8.1 2.96-0.62mdk)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> help(os.sep)
str(object) -> string

Return a nice string representation of the object.
If the argument is a string, the return value is the same object.
>>> help(os.extsep)
str(object) -> string

Return a nice string representation of the object.
If the argument is a string, the return value is the same object.
>>> help(os.pathsep)
str(object) -> string

Return a nice string representation of the object.
If the argument is a string, the return value is the same object.
>>> help(os.curdir)
str(object) -> string

Return a nice string representation of the object.
If the argument is a string, the return value is the same object.

Now, I can understand that maybe I've hosed my CVS tree,
which would affect the 2.3 build, but the 2.2.2 build on both
Mac OS X and Mandrake Linux is I believe straight from the
source with no CVS monkey business.

Can anyone else reproduce this?

----------------------------------------------------------------------

Comment By: logistix (logistix)
Date: 2003-02-14 14:57

Message:
Logged In: YES 
user_id=699438

SF Patch #686771 posted.

----------------------------------------------------------------------

Comment By: logistix (logistix)
Date: 2003-02-14 12:50

Message:
Logged In: YES 
user_id=699438

DOH! I'm an idiot.

You get the same value errors in Win32 in 2.2.2 and 2.3.1.  It 
looks it's the "locate" function in pydoc.  It tries to split a 
string by '.' in an attempt to resolve namespaces.  When you 
run that on a string like '.', '..', you get some empty strings 
and bad things happen.

I've got a patch that looks like it works.  It just uses a list 
comp to filter out null strings in the above situation.  I'll post it 
after running the test suite.

I don't know why Mandrake produced different results.

----------------------------------------------------------------------

Comment By: Skip Montanaro (montanaro)
Date: 2003-02-14 11:26

Message:
Logged In: YES 
user_id=44345

I can understand the "no docstring" message, but that doesn't explain
the "problem in  - ValueError: Empty module name" error from help().
Also, the distinction I was trying to make between the 2.2.2 on Linux
and MacOSX was that on the former, the docstrings were consistent
(and what you'd expect for a string), while on the latter there were
two different responses, neither of which matched the response I
saw on Linux.

----------------------------------------------------------------------

Comment By: logistix (logistix)
Date: 2003-02-14 11:11

Message:
Logged In: YES 
user_id=699438

I think you're overthinking things.  A string instance won't 
have it's own specific docstring.

Python 2.3a1 (#38, Dec 31 2002, 17:53:59) [MSC v.1200 32 
bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more 
information.
>>> import os
>>> help(os.sep)
no Python documentation found for '\'

>>> type(os.sep)
<type 'str'>
>>>

Maybe the versions you migrate into os.path should be 
functions (or will that break too much?)

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=686380&group_id=5470


From noreply@sourceforge.net  Sat Feb 15 05:02:43 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Fri, 14 Feb 2003 21:02:43 -0800
Subject: [Python-bugs-list] [ python-Bugs-683658 ] PyErr_Warn may cause import deadlock
Message-ID: <E18juTL-0004aH-00@sc8-sf-web1.sourceforge.net>

Bugs item #683658, was opened at 2003-02-10 11:26
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=683658&group_id=5470

Category: Python Interpreter Core
Group: Python 2.3
Status: Open
Resolution: None
Priority: 6
Submitted By: Mark Hammond (mhammond)
Assigned to: Mark Hammond (mhammond)
Summary: PyErr_Warn may cause import deadlock

Initial Comment:
PyErr_Warn() does an implicit import.  Thus, if
PyErr_Warn() is called on a thread while the main
thread holds the import lock, and the main thread then
subsequently waits for the child thread, Python will
deadlock.  The builtin 'apply' now calls PyErr_Warn(),
so simply calling 'apply' on another thread may cause
the deadlock.

Attaching a sample case.  Executing 'python -c "import
hang_apply"' will cause Python to deadlock.  Commenting
out the call to "apply" in that file will prevent the
deadlock (even though the apply() in question is
effectively a no-op)

The example is a little contrived, but is extracted
from real code of mine that saw this hang.  The code
path is:

* Main thread acquires import lock to import the module.
* Module import spawns a new thread.  This new thread
calls apply()
* builtin_apply() calls PyErr_Warn
* PyErr_Warn does an import of the warnings module, and
attempts to acquire the import lock.
* Main thread still waiting for child thread to
complete, but still holds import lock.

Note that removing the call to PyErr_Warn() in
builtin_apply also solves this particular hang -
however, it seems like this is a potential time bomb. 
A potential solution would be to prevent PyErr_Warn
from doing an import - this would mean importing
'warnings' at startup, and keeping a static reference
in errors.c.  Other thoughts welcome.


----------------------------------------------------------------------

>Comment By: Mark Hammond (mhammond)
Date: 2003-02-15 16:02

Message:
Logged In: YES 
user_id=14198

The best I can come up with here is exposing the import lock
to the C API, with a "wait" flag.

This will allow a C function to reliably determine if an
"import" work block, or acquire the lock if not.  It can
then complete its import before releasing the lock.  If the
import would block, then it can take whatever action is
necessary - in the case of PyErr_Warn, it dumps the warning
to stdout.

Attaching a patch.  It exposes (to the core, but not in
headers) two functions:

extern int PyImport_LockImport(int wait);
extern void PyImport_UnlockImport(void);

PyErr_Warn then uses these.

If we do go this route, it makes sense to make these
functions truly public (ie, add them to the headers), and
cleanup import.c appropriately.  I didn't do this in the
interests of keeping the patch small so it can be easily
digested.

Comments?  Other ideas?

----------------------------------------------------------------------

Comment By: Mark Hammond (mhammond)
Date: 2003-02-14 16:16

Message:
Logged In: YES 
user_id=14198

This simple strategy doesn't work - avoiding the import of
'warnings' works fine, until 'warnings' imports 'linecache'!

I'll look at how we can simply throw the warning away if a
deadlock would occur.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-12 10:23

Message:
Logged In: YES 
user_id=33168

I can't think of any other/better solution.  Any idea if
there are there other ticking bombs like this?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=683658&group_id=5470


From noreply@sourceforge.net  Sat Feb 15 15:46:36 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sat, 15 Feb 2003 07:46:36 -0800
Subject: [Python-bugs-list] [ python-Bugs-683658 ] PyErr_Warn may cause import deadlock
Message-ID: <E18k4WS-00079O-00@sc8-sf-web2.sourceforge.net>

Bugs item #683658, was opened at 2003-02-09 19:26
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=683658&group_id=5470

Category: Python Interpreter Core
Group: Python 2.3
Status: Open
Resolution: None
Priority: 6
Submitted By: Mark Hammond (mhammond)
Assigned to: Mark Hammond (mhammond)
Summary: PyErr_Warn may cause import deadlock

Initial Comment:
PyErr_Warn() does an implicit import.  Thus, if
PyErr_Warn() is called on a thread while the main
thread holds the import lock, and the main thread then
subsequently waits for the child thread, Python will
deadlock.  The builtin 'apply' now calls PyErr_Warn(),
so simply calling 'apply' on another thread may cause
the deadlock.

Attaching a sample case.  Executing 'python -c "import
hang_apply"' will cause Python to deadlock.  Commenting
out the call to "apply" in that file will prevent the
deadlock (even though the apply() in question is
effectively a no-op)

The example is a little contrived, but is extracted
from real code of mine that saw this hang.  The code
path is:

* Main thread acquires import lock to import the module.
* Module import spawns a new thread.  This new thread
calls apply()
* builtin_apply() calls PyErr_Warn
* PyErr_Warn does an import of the warnings module, and
attempts to acquire the import lock.
* Main thread still waiting for child thread to
complete, but still holds import lock.

Note that removing the call to PyErr_Warn() in
builtin_apply also solves this particular hang -
however, it seems like this is a potential time bomb. 
A potential solution would be to prevent PyErr_Warn
from doing an import - this would mean importing
'warnings' at startup, and keeping a static reference
in errors.c.  Other thoughts welcome.


----------------------------------------------------------------------

>Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-15 10:46

Message:
Logged In: YES 
user_id=6380

No time to review everything here, but maybe PyErr_Warn
should not do an import? The import could be done at startup
time (when site.py is also imported) and saved in the
interpreter state.

----------------------------------------------------------------------

Comment By: Mark Hammond (mhammond)
Date: 2003-02-15 00:02

Message:
Logged In: YES 
user_id=14198

The best I can come up with here is exposing the import lock
to the C API, with a "wait" flag.

This will allow a C function to reliably determine if an
"import" work block, or acquire the lock if not.  It can
then complete its import before releasing the lock.  If the
import would block, then it can take whatever action is
necessary - in the case of PyErr_Warn, it dumps the warning
to stdout.

Attaching a patch.  It exposes (to the core, but not in
headers) two functions:

extern int PyImport_LockImport(int wait);
extern void PyImport_UnlockImport(void);

PyErr_Warn then uses these.

If we do go this route, it makes sense to make these
functions truly public (ie, add them to the headers), and
cleanup import.c appropriately.  I didn't do this in the
interests of keeping the patch small so it can be easily
digested.

Comments?  Other ideas?

----------------------------------------------------------------------

Comment By: Mark Hammond (mhammond)
Date: 2003-02-14 00:16

Message:
Logged In: YES 
user_id=14198

This simple strategy doesn't work - avoiding the import of
'warnings' works fine, until 'warnings' imports 'linecache'!

I'll look at how we can simply throw the warning away if a
deadlock would occur.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-11 18:23

Message:
Logged In: YES 
user_id=33168

I can't think of any other/better solution.  Any idea if
there are there other ticking bombs like this?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=683658&group_id=5470


From noreply@sourceforge.net  Sat Feb 15 21:01:41 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sat, 15 Feb 2003 13:01:41 -0800
Subject: [Python-bugs-list] [ python-Bugs-683658 ] PyErr_Warn may cause import deadlock
Message-ID: <E18k9RN-0000xh-00@sc8-sf-web2.sourceforge.net>

Bugs item #683658, was opened at 2003-02-09 19:26
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=683658&group_id=5470

Category: Python Interpreter Core
Group: Python 2.3
Status: Open
Resolution: None
Priority: 6
Submitted By: Mark Hammond (mhammond)
Assigned to: Mark Hammond (mhammond)
Summary: PyErr_Warn may cause import deadlock

Initial Comment:
PyErr_Warn() does an implicit import.  Thus, if
PyErr_Warn() is called on a thread while the main
thread holds the import lock, and the main thread then
subsequently waits for the child thread, Python will
deadlock.  The builtin 'apply' now calls PyErr_Warn(),
so simply calling 'apply' on another thread may cause
the deadlock.

Attaching a sample case.  Executing 'python -c "import
hang_apply"' will cause Python to deadlock.  Commenting
out the call to "apply" in that file will prevent the
deadlock (even though the apply() in question is
effectively a no-op)

The example is a little contrived, but is extracted
from real code of mine that saw this hang.  The code
path is:

* Main thread acquires import lock to import the module.
* Module import spawns a new thread.  This new thread
calls apply()
* builtin_apply() calls PyErr_Warn
* PyErr_Warn does an import of the warnings module, and
attempts to acquire the import lock.
* Main thread still waiting for child thread to
complete, but still holds import lock.

Note that removing the call to PyErr_Warn() in
builtin_apply also solves this particular hang -
however, it seems like this is a potential time bomb. 
A potential solution would be to prevent PyErr_Warn
from doing an import - this would mean importing
'warnings' at startup, and keeping a static reference
in errors.c.  Other thoughts welcome.


----------------------------------------------------------------------

>Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-15 16:01

Message:
Logged In: YES 
user_id=6380

Oh, and the import of linecache by warnings can be put at
the top of warnings, if that's an issue.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-15 10:46

Message:
Logged In: YES 
user_id=6380

No time to review everything here, but maybe PyErr_Warn
should not do an import? The import could be done at startup
time (when site.py is also imported) and saved in the
interpreter state.

----------------------------------------------------------------------

Comment By: Mark Hammond (mhammond)
Date: 2003-02-15 00:02

Message:
Logged In: YES 
user_id=14198

The best I can come up with here is exposing the import lock
to the C API, with a "wait" flag.

This will allow a C function to reliably determine if an
"import" work block, or acquire the lock if not.  It can
then complete its import before releasing the lock.  If the
import would block, then it can take whatever action is
necessary - in the case of PyErr_Warn, it dumps the warning
to stdout.

Attaching a patch.  It exposes (to the core, but not in
headers) two functions:

extern int PyImport_LockImport(int wait);
extern void PyImport_UnlockImport(void);

PyErr_Warn then uses these.

If we do go this route, it makes sense to make these
functions truly public (ie, add them to the headers), and
cleanup import.c appropriately.  I didn't do this in the
interests of keeping the patch small so it can be easily
digested.

Comments?  Other ideas?

----------------------------------------------------------------------

Comment By: Mark Hammond (mhammond)
Date: 2003-02-14 00:16

Message:
Logged In: YES 
user_id=14198

This simple strategy doesn't work - avoiding the import of
'warnings' works fine, until 'warnings' imports 'linecache'!

I'll look at how we can simply throw the warning away if a
deadlock would occur.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-11 18:23

Message:
Logged In: YES 
user_id=33168

I can't think of any other/better solution.  Any idea if
there are there other ticking bombs like this?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=683658&group_id=5470


From noreply@sourceforge.net  Sat Feb 15 23:17:58 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sat, 15 Feb 2003 15:17:58 -0800
Subject: [Python-bugs-list] [ python-Bugs-683658 ] PyErr_Warn may cause import deadlock
Message-ID: <E18kBZG-0005hB-00@sc8-sf-web2.sourceforge.net>

Bugs item #683658, was opened at 2003-02-10 11:26
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=683658&group_id=5470

Category: Python Interpreter Core
Group: Python 2.3
Status: Open
Resolution: None
Priority: 6
Submitted By: Mark Hammond (mhammond)
Assigned to: Mark Hammond (mhammond)
Summary: PyErr_Warn may cause import deadlock

Initial Comment:
PyErr_Warn() does an implicit import.  Thus, if
PyErr_Warn() is called on a thread while the main
thread holds the import lock, and the main thread then
subsequently waits for the child thread, Python will
deadlock.  The builtin 'apply' now calls PyErr_Warn(),
so simply calling 'apply' on another thread may cause
the deadlock.

Attaching a sample case.  Executing 'python -c "import
hang_apply"' will cause Python to deadlock.  Commenting
out the call to "apply" in that file will prevent the
deadlock (even though the apply() in question is
effectively a no-op)

The example is a little contrived, but is extracted
from real code of mine that saw this hang.  The code
path is:

* Main thread acquires import lock to import the module.
* Module import spawns a new thread.  This new thread
calls apply()
* builtin_apply() calls PyErr_Warn
* PyErr_Warn does an import of the warnings module, and
attempts to acquire the import lock.
* Main thread still waiting for child thread to
complete, but still holds import lock.

Note that removing the call to PyErr_Warn() in
builtin_apply also solves this particular hang -
however, it seems like this is a potential time bomb. 
A potential solution would be to prevent PyErr_Warn
from doing an import - this would mean importing
'warnings' at startup, and keeping a static reference
in errors.c.  Other thoughts welcome.


----------------------------------------------------------------------

>Comment By: Mark Hammond (mhammond)
Date: 2003-02-16 10:17

Message:
Logged In: YES 
user_id=14198

Yes, the linecache import was the problem.  I abandonded
that approach mainly to avoid yet another import at startup.
 Should 'warnings' get new features, all required imports in
the future will also need to be imported at process startup.
 It also struck me as a time bomb.

I will make an alternative patch that moves the imports.


----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-16 08:01

Message:
Logged In: YES 
user_id=6380

Oh, and the import of linecache by warnings can be put at
the top of warnings, if that's an issue.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-16 02:46

Message:
Logged In: YES 
user_id=6380

No time to review everything here, but maybe PyErr_Warn
should not do an import? The import could be done at startup
time (when site.py is also imported) and saved in the
interpreter state.

----------------------------------------------------------------------

Comment By: Mark Hammond (mhammond)
Date: 2003-02-15 16:02

Message:
Logged In: YES 
user_id=14198

The best I can come up with here is exposing the import lock
to the C API, with a "wait" flag.

This will allow a C function to reliably determine if an
"import" work block, or acquire the lock if not.  It can
then complete its import before releasing the lock.  If the
import would block, then it can take whatever action is
necessary - in the case of PyErr_Warn, it dumps the warning
to stdout.

Attaching a patch.  It exposes (to the core, but not in
headers) two functions:

extern int PyImport_LockImport(int wait);
extern void PyImport_UnlockImport(void);

PyErr_Warn then uses these.

If we do go this route, it makes sense to make these
functions truly public (ie, add them to the headers), and
cleanup import.c appropriately.  I didn't do this in the
interests of keeping the patch small so it can be easily
digested.

Comments?  Other ideas?

----------------------------------------------------------------------

Comment By: Mark Hammond (mhammond)
Date: 2003-02-14 16:16

Message:
Logged In: YES 
user_id=14198

This simple strategy doesn't work - avoiding the import of
'warnings' works fine, until 'warnings' imports 'linecache'!

I'll look at how we can simply throw the warning away if a
deadlock would occur.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-12 10:23

Message:
Logged In: YES 
user_id=33168

I can't think of any other/better solution.  Any idea if
there are there other ticking bombs like this?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=683658&group_id=5470


From noreply@sourceforge.net  Sun Feb 16 01:20:59 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sat, 15 Feb 2003 17:20:59 -0800
Subject: [Python-bugs-list] [ python-Bugs-686380 ] errors trying to get help on os attributes
Message-ID: <E18kDUJ-0000Tp-00@sc8-sf-web2.sourceforge.net>

Bugs item #686380, was opened at 2003-02-14 00:11
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=686380&group_id=5470

Category: Python Library
Group: Python 2.3
>Status: Closed
>Resolution: Fixed
Priority: 5
Submitted By: Skip Montanaro (montanaro)
Assigned to: Nobody/Anonymous (nobody)
Summary: errors trying to get help on os attributes

Initial Comment:
I noticed some very weird behavior w/ 2.3 when asking for
help about some data attributes in the os module.  Consider:

% python
Python 2.3a1 (#13, Feb  4 2003, 09:26:29) 
[GCC 3.1 20020420 (prerelease)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> help(os.sep)
no Python documentation found for '/'

>>> help(os.extsep)
problem in  - ValueError: Empty module name

>>> help(os.pathsep)
no Python documentation found for ':'

>>> help(os.curdir)
problem in  - ValueError: Empty module name

Now the same feat of prestidigation using 2.2.2 (on Mac OSX):

% python2.2 
Python 2.2.2 (#7, Oct 14 2002, 18:07:17) 
[GCC 3.1 20020420 (prerelease)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> help(os.sep)
no Python documentation found for '/'

>>> help(os.extsep)
problem in  - ValueError: Empty module name

>>> help(os.pathsep)
no Python documentation found for ':'

>>> help(os.curdir)
problem in  - ValueError: Empty module name

Finally, the same example using 2.2.2 under Mandrake Linux:

% python
Python 2.2.2 (#1, Nov  8 2002, 08:50:21) 
[GCC 2.96 20000731 (Mandrake Linux 8.1 2.96-0.62mdk)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> help(os.sep)
str(object) -> string

Return a nice string representation of the object.
If the argument is a string, the return value is the same object.
>>> help(os.extsep)
str(object) -> string

Return a nice string representation of the object.
If the argument is a string, the return value is the same object.
>>> help(os.pathsep)
str(object) -> string

Return a nice string representation of the object.
If the argument is a string, the return value is the same object.
>>> help(os.curdir)
str(object) -> string

Return a nice string representation of the object.
If the argument is a string, the return value is the same object.

Now, I can understand that maybe I've hosed my CVS tree,
which would affect the 2.3 build, but the 2.2.2 build on both
Mac OS X and Mandrake Linux is I believe straight from the
source with no CVS monkey business.

Can anyone else reproduce this?

----------------------------------------------------------------------

>Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-15 20:20

Message:
Logged In: YES 
user_id=6380

Patch applied.

----------------------------------------------------------------------

Comment By: logistix (logistix)
Date: 2003-02-14 15:57

Message:
Logged In: YES 
user_id=699438

SF Patch #686771 posted.

----------------------------------------------------------------------

Comment By: logistix (logistix)
Date: 2003-02-14 13:50

Message:
Logged In: YES 
user_id=699438

DOH! I'm an idiot.

You get the same value errors in Win32 in 2.2.2 and 2.3.1.  It 
looks it's the "locate" function in pydoc.  It tries to split a 
string by '.' in an attempt to resolve namespaces.  When you 
run that on a string like '.', '..', you get some empty strings 
and bad things happen.

I've got a patch that looks like it works.  It just uses a list 
comp to filter out null strings in the above situation.  I'll post it 
after running the test suite.

I don't know why Mandrake produced different results.

----------------------------------------------------------------------

Comment By: Skip Montanaro (montanaro)
Date: 2003-02-14 12:26

Message:
Logged In: YES 
user_id=44345

I can understand the "no docstring" message, but that doesn't explain
the "problem in  - ValueError: Empty module name" error from help().
Also, the distinction I was trying to make between the 2.2.2 on Linux
and MacOSX was that on the former, the docstrings were consistent
(and what you'd expect for a string), while on the latter there were
two different responses, neither of which matched the response I
saw on Linux.

----------------------------------------------------------------------

Comment By: logistix (logistix)
Date: 2003-02-14 12:11

Message:
Logged In: YES 
user_id=699438

I think you're overthinking things.  A string instance won't 
have it's own specific docstring.

Python 2.3a1 (#38, Dec 31 2002, 17:53:59) [MSC v.1200 32 
bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more 
information.
>>> import os
>>> help(os.sep)
no Python documentation found for '\'

>>> type(os.sep)
<type 'str'>
>>>

Maybe the versions you migrate into os.path should be 
functions (or will that break too much?)

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=686380&group_id=5470


From noreply@sourceforge.net  Sun Feb 16 01:41:48 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sat, 15 Feb 2003 17:41:48 -0800
Subject: [Python-bugs-list] [ python-Bugs-687297 ] Profilier hooked into SystemExit
Message-ID: <E18kDoS-0006nE-00@sc8-sf-web3.sourceforge.net>

Bugs item #687297, was opened at 2003-02-16 03:41
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=687297&group_id=5470

Category: Demos and Tools
Group: Python 2.2.2
Status: Open
Resolution: None
Priority: 5
Submitted By: Dylan Shell (dshell)
Assigned to: Nobody/Anonymous (nobody)
Summary: Profilier hooked into SystemExit

Initial Comment:
I've been attempting to profile code that uses the
PyOpenGL bindings. Essentially I've got a program with
that calls glutMainLoop - which is said to never return.

The problem is that since this really envokes some C
code that calls "exit" the profiler does not catch a
"SystemExit" exception, because this is never thrown.

If there was a way to get the profiler to dump state on
demand, I could do this in an "onExit" event handler,
and then restart python with the pstats module to have
a look-see.

Alternatively the profiler could use some OS provided
exit handler - or something simliar.

Also, running the main loop in a thread does not seem
to work (the video memory used by GLUT) is corrupted.
So, this isn't a fair test on which to profile.

I suspect that the ability to dump profile stats to
disk could also be useful for other folks.




----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=687297&group_id=5470


From noreply@sourceforge.net  Sun Feb 16 18:37:30 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sun, 16 Feb 2003 10:37:30 -0800
Subject: [Python-bugs-list] [ python-Bugs-684679 ] Freeze can not be made to exclude implicits
Message-ID: <E18kTfO-00083r-00@sc8-sf-web1.sourceforge.net>

Bugs item #684679, was opened at 2003-02-11 17:01
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=684679&group_id=5470

Category: Demos and Tools
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Lawrence Hudson (lhudson)
>Assigned to: Just van Rossum (jvr)
Summary: Freeze can not be made to exclude implicits

Initial Comment:
Freeze always includes site and exceptions in the
resulting binary.  This is undesirable when creating an
extension module that contains frozen code;in sucha
situation the extension module should import the site
and exceptions (and all other standard library modules)
from the target system at runtime.

Patch 684677 fixes this bug.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=684679&group_id=5470


From noreply@sourceforge.net  Sun Feb 16 18:38:08 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sun, 16 Feb 2003 10:38:08 -0800
Subject: [Python-bugs-list] [ python-Bugs-685373 ] bundlebuilder and spaces in filenames
Message-ID: <E18kTg0-000859-00@sc8-sf-web1.sourceforge.net>

Bugs item #685373, was opened at 2003-02-12 16:51
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=685373&group_id=5470

Category: Macintosh
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Jack Jansen (jackjansen)
Assigned to: Just van Rossum (jvr)
Summary: bundlebuilder and spaces in filenames

Initial Comment:
If there is a space in your main program filename then bundlebuilder creates a non-functional applet.

Easiest seen by doing "save as applet" in the IDE on an unnamed script.

----------------------------------------------------------------------

>Comment By: Just van Rossum (jvr)
Date: 2003-02-16 19:38

Message:
Logged In: YES 
user_id=92689

Jack, can you double check my fix so I can close this item?

----------------------------------------------------------------------

Comment By: Just van Rossum (jvr)
Date: 2003-02-12 17:27

Message:
Logged In: YES 
user_id=92689

Darn, I thought I fixed this, apparently not thoroughly enough... Can you check whether current CVS works for you now?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=685373&group_id=5470


From noreply@sourceforge.net  Sun Feb 16 20:11:19 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sun, 16 Feb 2003 12:11:19 -0800
Subject: [Python-bugs-list] [ python-Bugs-668662 ] Compiling C sources with absolute path bug
Message-ID: <E18kV8B-0002FT-00@sc8-sf-web4.sourceforge.net>

Bugs item #668662, was opened at 2003-01-15 20:59
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=668662&group_id=5470

Category: Distutils
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Pearu Peterson (pearu)
Assigned to: A.M. Kuchling (akuchling)
Summary: Compiling C sources with absolute path bug

Initial Comment:
Py-2.3 distutils is broken in the following situation
when building extension modules:
If a file in C sources list is given with absolute
path, say
/path/to/foo.c then Py-2.3 distutils tries to create
the corresponding    
object file to /path/to/foo.o (earlier versions would
create it to, say, build/temp.linux-i686-2.2/foo.o).
This causes problems when
1) an user does not have permissions to write to
/path/to/, e.g. in multiuser systems
2) an user builds extension modules, say, using
different compilers or flags. Then /path/to/foo.o will
be in a way for subsequent builds.
IMHO, distutils should not create any files outside the
given
build directory, unless explicitely specified using
--build-temp.

Fix:
The problematic code seems to be in the method
ccompiler.object_filenames in file distutils/ccompiler.py.
As a fix, I suggest using the following object name for
a C source if given with absolute path: 
  ./build/temp.linux-i686-2.2/path/to/foo.o.

This can be achived by inserting the following line
just after
the line `base, ext = os.path.splitext(src_name)`:

 base = base[os.path.isabs(base):]

Pearu

----------------------------------------------------------------------

>Comment By: Pearu Peterson (pearu)
Date: 2003-02-16 22:11

Message:
Logged In: YES 
user_id=88489

Attached patch maps the base of a source file with absolute
path (and 
drive if present) to relative path before composing the
corresponding object file. This avoids creation of object
files out of the build directory. 

Pearu

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2003-01-16 15:28

Message:
Logged In: YES 
user_id=6656

Actually, I guessed after posting that something like that
could happen.

BTW, I strongly suspect this is related to the fix for this bug:

[ 599248 ] ext module generation problem

Can you come up with a patch for your suggested fix?  I'm
sure Andrew would appreciate it :-)

----------------------------------------------------------------------

Comment By: Pearu Peterson (pearu)
Date: 2003-01-15 21:48

Message:
Logged In: YES 
user_id=88489

FYI, distutils does not make this absolute path, but one can
specify
absolute paths in setup.py files, for instance.
Also, tools that create extension modules (e.g. f2py), may
ship with C sources that are installed as data files and
compiled
for each generated extension module separately (possibly
with different
compilers and flags so that pre-compilation of such sources
would not be an option).


----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2003-01-15 21:23

Message:
Logged In: YES 
user_id=6656

I'm a little boggled by how distutils would get an absolute
path, but I'm going to assign this to amk anyway :)

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=668662&group_id=5470


From noreply@sourceforge.net  Sun Feb 16 20:28:59 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sun, 16 Feb 2003 12:28:59 -0800
Subject: [Python-bugs-list] [ python-Bugs-687607 ] In some OSes BROWSER env var not used as webbrowser expects
Message-ID: <E18kVPH-00044J-00@sc8-sf-web1.sourceforge.net>

Bugs item #687607, was opened at 2003-02-16 14:28
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=687607&group_id=5470

Category: Python Library
Group: Python 2.2.2
Status: Open
Resolution: None
Priority: 5
Submitted By: Matthew Cowles (mdcowles)
Assigned to: Nobody/Anonymous (nobody)
Summary: In some OSes BROWSER env var not used as webbrowser expects

Initial Comment:
[From a post on python-help]

It appears that in at least some installations of KDE
under Mandrake, the BROWSER environment variable is set
but doesn't contain names of executable browsers. Since
BROWSER overrides everything that webbrowser.py has
found, the module doesn't work in that case. I believe
that it would be sufficient to change

    _tryorder = os.environ["BROWSER"].split(os.pathsep)

to

    _tryorder =
os.environ["BROWSER"].split(os.pathsep)+_tryorder

to make it work for those users.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=687607&group_id=5470


From noreply@sourceforge.net  Sun Feb 16 21:30:37 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sun, 16 Feb 2003 13:30:37 -0800
Subject: [Python-bugs-list] [ python-Bugs-685373 ] bundlebuilder and spaces in filenames
Message-ID: <E18kWMv-0001Hx-00@sc8-sf-web3.sourceforge.net>

Bugs item #685373, was opened at 2003-02-12 16:51
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=685373&group_id=5470

Category: Macintosh
Group: None
>Status: Closed
>Resolution: Fixed
Priority: 5
Submitted By: Jack Jansen (jackjansen)
Assigned to: Just van Rossum (jvr)
Summary: bundlebuilder and spaces in filenames

Initial Comment:
If there is a space in your main program filename then bundlebuilder creates a non-functional applet.

Easiest seen by doing "save as applet" in the IDE on an unnamed script.

----------------------------------------------------------------------

>Comment By: Jack Jansen (jackjansen)
Date: 2003-02-16 22:30

Message:
Logged In: YES 
user_id=45365

Yup, works fine!

----------------------------------------------------------------------

Comment By: Just van Rossum (jvr)
Date: 2003-02-16 19:38

Message:
Logged In: YES 
user_id=92689

Jack, can you double check my fix so I can close this item?

----------------------------------------------------------------------

Comment By: Just van Rossum (jvr)
Date: 2003-02-12 17:27

Message:
Logged In: YES 
user_id=92689

Darn, I thought I fixed this, apparently not thoroughly enough... Can you check whether current CVS works for you now?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=685373&group_id=5470


From noreply@sourceforge.net  Sun Feb 16 22:20:16 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sun, 16 Feb 2003 14:20:16 -0800
Subject: [Python-bugs-list] [ python-Bugs-687644 ] Functions does not accept keyword arguments
Message-ID: <E18kX8y-0000Uw-00@sc8-sf-web2.sourceforge.net>

Bugs item #687644, was opened at 2003-02-16 23:20
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=687644&group_id=5470

Category: Parser/Compiler
Group: Python 2.2.1
Status: Open
Resolution: None
Priority: 5
Submitted By: Viktor Ferenczi (complex)
Assigned to: Nobody/Anonymous (nobody)
Summary: Functions does not accept keyword arguments

Initial Comment:
Functions does not accept keyword arguments if argument list specified. Interactive session:

ActivePython 2.2.1 Build 222 (ActiveState Corp.) based on
Python 2.2.1 (#34, Apr 15 2002, 09:51:39) [MSC 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> def fn(*args,**kw): print 'args=%r, kw=%r'%(args,kw)
...
>>> fn(1,2,a=3,b=4)
args=(1, 2), kw={'a': 3, 'b': 4}
>>> al=[1,2]
>>> kd={'a':3,'b':4}
>>> fn(*al,**kd)
args=(1, 2), kw={'a': 3, 'b': 4}
>>> fn(1,2,**kd)
args=(1, 2), kw={'a': 3, 'b': 4}
>>> fn(*al,a=3,b=4)
  File "<stdin>", line 1
    fn(*al,a=3,b=4)
           ^
SyntaxError: invalid syntax

It's valid to apply a keyword dictionary and an argument list at the same time. Keyword dictionary can be applied with real arguments. Python should allow applying an argument list while passing real keyword arguments. As far as I know, it should cause no ambiguous syntax.

- Complex -

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=687644&group_id=5470


From noreply@sourceforge.net  Sun Feb 16 22:27:52 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sun, 16 Feb 2003 14:27:52 -0800
Subject: [Python-bugs-list] [ python-Bugs-687648 ] classic division in demos/ directory
Message-ID: <E18kXGK-0007Fj-00@sc8-sf-web4.sourceforge.net>

Bugs item #687648, was opened at 2003-02-16 17:27
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=687648&group_id=5470

Category: Demos and Tools
Group: Not a Bug
Status: Open
Resolution: None
Priority: 5
Submitted By: Jp Calderone (kuran)
Assigned to: Nobody/Anonymous (nobody)
Summary: classic division in demos/ directory

Initial Comment:
PEP 238 states:

     - The standard library will use the future
division statement and
       the // operator when appropriate, so as to
completely avoid
       classic division.

  While the demos/ directory is not technically part of
the standard library, it does contain code that should
work, and that newbies may examine in the course of
learning Python.  Python source there should follow the
same rules as anyplace else.

  I'll volunteer to make the changes and submit a
patch, if it is agreed that the changes should indeed
be made.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=687648&group_id=5470


From noreply@sourceforge.net  Sun Feb 16 22:41:54 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sun, 16 Feb 2003 14:41:54 -0800
Subject: [Python-bugs-list] [ python-Bugs-687654 ] Future division breaks mpz
Message-ID: <E18kXTu-0007gR-00@sc8-sf-web4.sourceforge.net>

Bugs item #687654, was opened at 2003-02-16 17:41
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=687654&group_id=5470

Category: None
Group: Python 2.2.2
Status: Open
Resolution: None
Priority: 5
Submitted By: Jp Calderone (kuran)
Assigned to: Nobody/Anonymous (nobody)
Summary: Future division breaks mpz

Initial Comment:
When future division is enabled in Python 2.2.2,
division of mpz objects no longer functions.

>>> from mpz import mpz
>>> mpz(12) / mpz(4)
mpz(3)
>>> from __future__ import division
>>> mpz(12) / mpz(4)
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
TypeError: unsupported operand type(s) for /: 'mpz.mpz'
and 'mpz.mpz'
>>> mpz(12) // mpz(4)
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
TypeError: unsupported operand type(s) for //:
'mpz.mpz' and 'mpz.mpz'


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=687654&group_id=5470


From noreply@sourceforge.net  Sun Feb 16 22:49:41 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sun, 16 Feb 2003 14:49:41 -0800
Subject: [Python-bugs-list] [ python-Bugs-687655 ] String formatting conversions misleading
Message-ID: <E18kXbR-0004Mr-00@sc8-sf-web3.sourceforge.net>

Bugs item #687655, was opened at 2003-02-16 23:49
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=687655&group_id=5470

Category: Documentation
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Beat Bolli (bolli)
Assigned to: Nobody/Anonymous (nobody)
Summary: String formatting conversions misleading

Initial Comment:
Chapter 2.2.6.2, "String Formatting":

The "conversion flag 
characters" table should mention that the 0 flag only applies to 
numeric values, not to strings.

This bug is the result of a 
short discussion on c.l.python on Feb 13, 2003.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=687655&group_id=5470


From noreply@sourceforge.net  Mon Feb 17 03:31:33 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sun, 16 Feb 2003 19:31:33 -0800
Subject: [Python-bugs-list] [ python-Bugs-687607 ] In some OSes BROWSER env var not used as webbrowser expects
Message-ID: <E18kc0D-00052h-00@sc8-sf-web3.sourceforge.net>

Bugs item #687607, was opened at 2003-02-16 14:28
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=687607&group_id=5470

Category: Python Library
Group: Python 2.2.2
>Status: Deleted
>Resolution: Invalid
Priority: 5
Submitted By: Matthew Cowles (mdcowles)
Assigned to: Nobody/Anonymous (nobody)
Summary: In some OSes BROWSER env var not used as webbrowser expects

Initial Comment:
[From a post on python-help]

It appears that in at least some installations of KDE
under Mandrake, the BROWSER environment variable is set
but doesn't contain names of executable browsers. Since
BROWSER overrides everything that webbrowser.py has
found, the module doesn't work in that case. I believe
that it would be sufficient to change

    _tryorder = os.environ["BROWSER"].split(os.pathsep)

to

    _tryorder =
os.environ["BROWSER"].split(os.pathsep)+_tryorder

to make it work for those users.

----------------------------------------------------------------------

>Comment By: Matthew Cowles (mdcowles)
Date: 2003-02-16 21:31

Message:
Logged In: YES 
user_id=198518

Sorry, I misunderstood what was going on with the user's
system. The environment variable does contain a browser, but
_iscommand() doesn't recognize it. I'll file that separately.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=687607&group_id=5470


From noreply@sourceforge.net  Mon Feb 17 04:17:07 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sun, 16 Feb 2003 20:17:07 -0800
Subject: [Python-bugs-list] [ python-Bugs-687747 ] _iscommand() in webbrowser module
Message-ID: <E18kciJ-000286-00@sc8-sf-web2.sourceforge.net>

Bugs item #687747, was opened at 2003-02-16 22:17
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=687747&group_id=5470

Category: Python Library
Group: Python 2.2.2
Status: Open
Resolution: None
Priority: 5
Submitted By: Matthew Cowles (mdcowles)
Assigned to: Nobody/Anonymous (nobody)
Summary: _iscommand() in webbrowser module

Initial Comment:
[From a post to python-help]

Under KDE under Mandrake Linux the BROWSER environment
variable is set to

kfmclient openProfile webbrowsing

The problem is that _iscommand() in the webbrowser
module doesn't realize that only the first thing there
is the name of the executable. It looks for an
executable with that whole thing as its name and
doesn't find one. Since the module doesn't use any
browser it has found if BROWSER is set, it raises an
error rather than opening the page.

The possible fixes that are obvious to me all have
potential disadvantages:

One solution would be to assume that the name of the
executable ran only up to the first space. But
executables can have spaces in their names, especially
on a Macintosh, I think.

Another possibility would be not to call _iscommand()
on anything in BROWSER. That would have the additional
advantage of not requiring that the executable
specified there be in a directory that's in the user's
PATH. The problem with doing things this way is that it
would be impossible to tell which of the browsers
specified in BROWSER should be used until the user
called open().

I'm prepared to work on a fix if given some guidance
about the best way to go.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=687747&group_id=5470


From noreply@sourceforge.net  Mon Feb 17 04:33:34 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sun, 16 Feb 2003 20:33:34 -0800
Subject: [Python-bugs-list] [ python-Bugs-687644 ] Functions does not accept keyword arguments
Message-ID: <E18kcyE-0006hY-00@sc8-sf-web3.sourceforge.net>

Bugs item #687644, was opened at 2003-02-16 17:20
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=687644&group_id=5470

Category: Parser/Compiler
Group: Python 2.2.1
>Status: Closed
>Resolution: Invalid
Priority: 5
Submitted By: Viktor Ferenczi (complex)
>Assigned to: Neal Norwitz (nnorwitz)
Summary: Functions does not accept keyword arguments

Initial Comment:
Functions does not accept keyword arguments if argument list specified. Interactive session:

ActivePython 2.2.1 Build 222 (ActiveState Corp.) based on
Python 2.2.1 (#34, Apr 15 2002, 09:51:39) [MSC 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> def fn(*args,**kw): print 'args=%r, kw=%r'%(args,kw)
...
>>> fn(1,2,a=3,b=4)
args=(1, 2), kw={'a': 3, 'b': 4}
>>> al=[1,2]
>>> kd={'a':3,'b':4}
>>> fn(*al,**kd)
args=(1, 2), kw={'a': 3, 'b': 4}
>>> fn(1,2,**kd)
args=(1, 2), kw={'a': 3, 'b': 4}
>>> fn(*al,a=3,b=4)
  File "<stdin>", line 1
    fn(*al,a=3,b=4)
           ^
SyntaxError: invalid syntax

It's valid to apply a keyword dictionary and an argument list at the same time. Keyword dictionary can be applied with real arguments. Python should allow applying an argument list while passing real keyword arguments. As far as I know, it should cause no ambiguous syntax.

- Complex -

----------------------------------------------------------------------

>Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-16 23:33

Message:
Logged In: YES 
user_id=33168

This isn't a bug, although the behaviour seems a bit odd at
first.  The order for an argument list is: parameters,
named/keyword parameters, parameters (sequences) expanded
with '*', and finally parameters (mappings) expanded with '**'.

If you reorder the parameters, it works:

>>> fn(a=3, b=4, *al)
args=(1, 2), kw={'a': 3, 'b': 4}
>>> fn(c=3, d=4, *al, **kd)
args=(1, 2), kw={'a': 3, 'c': 3, 'b': 4, 'd': 4}

This is shown in section 5.3.4 of the reference manual:
http://www.python.org/doc/current/ref/calls.html

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=687644&group_id=5470


From noreply@sourceforge.net  Mon Feb 17 14:21:19 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Mon, 17 Feb 2003 06:21:19 -0800
Subject: [Python-bugs-list] [ python-Bugs-688007 ] Bundlebuilder needs to pre-convert resource files
Message-ID: <E18km91-00040R-00@sc8-sf-web3.sourceforge.net>

Bugs item #688007, was opened at 2003-02-17 15:21
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=688007&group_id=5470

Category: Macintosh
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Jack Jansen (jackjansen)
Assigned to: Jack Jansen (jackjansen)
Summary: Bundlebuilder needs to pre-convert resource files

Initial Comment:
If bundlebuilder encounters an applesingle-encoded resource file it should pre-convert this to a datafork-based resource file.

At the moment the file is copied to the Resources directory verbatim, and this can cause problems if you use "sudo make frameworkinstall" to install Python: the resource folder will be mode 755 owned by root, and any normal user trying to run the applet will get a crash when the applet tries to unpack the applesingle .rsrc file into the datafork resource file .rsrc.df.rsrc.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=688007&group_id=5470


From noreply@sourceforge.net  Mon Feb 17 14:23:32 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Mon, 17 Feb 2003 06:23:32 -0800
Subject: [Python-bugs-list] [ python-Bugs-688011 ] macresource should handle readonly applesingle files
Message-ID: <E18kmBA-00048Z-00@sc8-sf-web3.sourceforge.net>

Bugs item #688011, was opened at 2003-02-17 15:23
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=688011&group_id=5470

Category: Macintosh
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Jack Jansen (jackjansen)
Assigned to: Jack Jansen (jackjansen)
Summary: macresource should handle readonly applesingle files

Initial Comment:
If macresource.need() encounters an applesingle file it will unpack it and store the resource data into a .rsrc.df.rsrc file. If the directory
is not writeable this fails.

Macresource should in stead issue a warning and use a temporary file.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=688011&group_id=5470


From noreply@sourceforge.net  Mon Feb 17 15:46:00 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Mon, 17 Feb 2003 07:46:00 -0800
Subject: [Python-bugs-list] [ python-Bugs-687655 ] String formatting conversions misleading
Message-ID: <E18knSy-0003Nh-00@sc8-sf-web2.sourceforge.net>

Bugs item #687655, was opened at 2003-02-16 17:49
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=687655&group_id=5470

Category: Documentation
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Beat Bolli (bolli)
Assigned to: Nobody/Anonymous (nobody)
Summary: String formatting conversions misleading

Initial Comment:
Chapter 2.2.6.2, "String Formatting":

The "conversion flag 
characters" table should mention that the 0 flag only applies to 
numeric values, not to strings.

This bug is the result of a 
short discussion on c.l.python on Feb 13, 2003.


----------------------------------------------------------------------

>Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-17 10:46

Message:
Logged In: YES 
user_id=33168

Currently the text reads:

 * The conversion will be zero padded.

I suggest:
 * The conversion will be zero padded for integer
conversions (d, i, o, u, x, X).
Does this address the problem?  If not, can you suggest
wording?  

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=687655&group_id=5470


From noreply@sourceforge.net  Mon Feb 17 15:48:38 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Mon, 17 Feb 2003 07:48:38 -0800
Subject: [Python-bugs-list] [ python-Bugs-676358 ] Have PythonLauncher use #! line
Message-ID: <E18knVW-00085D-00@sc8-sf-web3.sourceforge.net>

Bugs item #676358, was opened at 2003-01-28 22:17
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=676358&group_id=5470

Category: Macintosh
Group: Feature Request
>Status: Closed
>Resolution: Accepted
Priority: 5
Submitted By: Christopher Barker (chrishbarker)
Assigned to: Jack Jansen (jackjansen)
Summary: Have PythonLauncher use #! line

Initial Comment:
As discussed on the mailing list, it would be great it
the PythonLauncher applet had a preference:

Honour #! lines in scripts

That way, one could have different versions of Python
on a machine, and any script could have an appropriate
#! line, and it would get launched with the correct
python.

-thanks, Chris

----------------------------------------------------------------------

>Comment By: Jack Jansen (jackjansen)
Date: 2003-02-17 16:48

Message:
Logged In: YES 
user_id=45365

Implemented in CVS.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=676358&group_id=5470


From noreply@sourceforge.net  Mon Feb 17 16:56:54 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Mon, 17 Feb 2003 08:56:54 -0800
Subject: [Python-bugs-list] [ python-Bugs-688007 ] Bundlebuilder needs to pre-convert resource files
Message-ID: <E18koZa-00071t-00@sc8-sf-web2.sourceforge.net>

Bugs item #688007, was opened at 2003-02-17 15:21
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=688007&group_id=5470

Category: Macintosh
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Jack Jansen (jackjansen)
>Assigned to: Just van Rossum (jvr)
Summary: Bundlebuilder needs to pre-convert resource files

Initial Comment:
If bundlebuilder encounters an applesingle-encoded resource file it should pre-convert this to a datafork-based resource file.

At the moment the file is copied to the Resources directory verbatim, and this can cause problems if you use "sudo make frameworkinstall" to install Python: the resource folder will be mode 755 owned by root, and any normal user trying to run the applet will get a crash when the applet tries to unpack the applesingle .rsrc file into the datafork resource file .rsrc.df.rsrc.

----------------------------------------------------------------------

>Comment By: Jack Jansen (jackjansen)
Date: 2003-02-17 17:56

Message:
Logged In: YES 
user_id=45365

Fixed in bundlebuilder.py rev 1.8, macresource rev. 1.3.

Just, I'm assigning this to you because I'm not sure whether you still keep bundlebuilder 2.2-compatible. If so you may need to disable this bit of magic if macresource.install isn't available. Please close the report when you're happy.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=688007&group_id=5470


From noreply@sourceforge.net  Mon Feb 17 18:55:31 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Mon, 17 Feb 2003 10:55:31 -0800
Subject: [Python-bugs-list] [ python-Bugs-687655 ] String formatting conversions misleading
Message-ID: <E18kqQN-0004lh-00@sc8-sf-web2.sourceforge.net>

Bugs item #687655, was opened at 2003-02-16 23:49
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=687655&group_id=5470

Category: Documentation
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Beat Bolli (bolli)
Assigned to: Nobody/Anonymous (nobody)
Summary: String formatting conversions misleading

Initial Comment:
Chapter 2.2.6.2, "String Formatting":

The "conversion flag 
characters" table should mention that the 0 flag only applies to 
numeric values, not to strings.

This bug is the result of a 
short discussion on c.l.python on Feb 13, 2003.


----------------------------------------------------------------------

>Comment By: Beat Bolli (bolli)
Date: 2003-02-17 19:55

Message:
Logged In: YES 
user_id=7310

> I suggest: * The conversion will be zero padded for 
integer conversions (d, i, o, u, x, X).
> Does this address the problem?  If not, can you suggest
wording?
I propose

* The conversion will be zero-padded for numeric values

This is more accurate, since floats can also be zero-padded.


----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-17 16:46

Message:
Logged In: YES 
user_id=33168

Currently the text reads:

 * The conversion will be zero padded.

I suggest:
 * The conversion will be zero padded for integer
conversions (d, i, o, u, x, X).
Does this address the problem?  If not, can you suggest
wording?  

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=687655&group_id=5470


From noreply@sourceforge.net  Mon Feb 17 19:08:47 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Mon, 17 Feb 2003 11:08:47 -0800
Subject: [Python-bugs-list] [ python-Bugs-687655 ] String formatting conversions misleading
Message-ID: <E18kqdD-0000Vz-00@sc8-sf-web3.sourceforge.net>

Bugs item #687655, was opened at 2003-02-16 17:49
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=687655&group_id=5470

Category: Documentation
Group: None
>Status: Closed
>Resolution: Fixed
Priority: 5
Submitted By: Beat Bolli (bolli)
>Assigned to: Neal Norwitz (nnorwitz)
Summary: String formatting conversions misleading

Initial Comment:
Chapter 2.2.6.2, "String Formatting":

The "conversion flag 
characters" table should mention that the 0 flag only applies to 
numeric values, not to strings.

This bug is the result of a 
short discussion on c.l.python on Feb 13, 2003.


----------------------------------------------------------------------

>Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-17 14:08

Message:
Logged In: YES 
user_id=33168

Your version was checked in as:
  Doc/lib/libstdtypes.tex; 1.118 and 1.80.6.19

Thanks.

----------------------------------------------------------------------

Comment By: Beat Bolli (bolli)
Date: 2003-02-17 13:55

Message:
Logged In: YES 
user_id=7310

> I suggest: * The conversion will be zero padded for 
integer conversions (d, i, o, u, x, X).
> Does this address the problem?  If not, can you suggest
wording?
I propose

* The conversion will be zero-padded for numeric values

This is more accurate, since floats can also be zero-padded.


----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-17 10:46

Message:
Logged In: YES 
user_id=33168

Currently the text reads:

 * The conversion will be zero padded.

I suggest:
 * The conversion will be zero padded for integer
conversions (d, i, o, u, x, X).
Does this address the problem?  If not, can you suggest
wording?  

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=687655&group_id=5470


From noreply@sourceforge.net  Mon Feb 17 21:37:13 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Mon, 17 Feb 2003 13:37:13 -0800
Subject: [Python-bugs-list] [ python-Bugs-688261 ] (Very) Small Documentation Error
Message-ID: <E18kswr-0008Q1-00@sc8-sf-web3.sourceforge.net>

Bugs item #688261, was opened at 2003-02-17 21:37
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=688261&group_id=5470

Category: Documentation
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Chris Lambert (stallion)
Assigned to: Nobody/Anonymous (nobody)
Summary: (Very) Small Documentation Error

Initial Comment:
In the "What's new" section:
http://www.python.org/doc/2.3a1/whatsnew/node15.html

$ ./python opt.py --input=data --length=4
<Values at 0x400cad2c: {'input': 'data', 'length': 4}>
['arg1']
$

shouldn't print
['arg1']

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=688261&group_id=5470


From noreply@sourceforge.net  Mon Feb 17 21:47:20 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Mon, 17 Feb 2003 13:47:20 -0800
Subject: [Python-bugs-list] [ python-Bugs-688266 ] IDLE does not work on Mac OS X
Message-ID: <E18kt6e-0003tc-00@sc8-sf-web2.sourceforge.net>

Bugs item #688266, was opened at 2003-02-17 21:47
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=688266&group_id=5470

Category: IDLE
Group: Platform-specific
Status: Open
Resolution: None
Priority: 5
Submitted By: Tony Lownds (tonylownds)
Assigned to: Nobody/Anonymous (nobody)
Summary: IDLE does not work on Mac OS X

Initial Comment:
When building a framework build of python, the IDLE application that is made does not work anymore.

Here is the command that is causing trouble:

/Library/Frameworks/../../Library/Frameworks/Python.framework/Versions/2.3/Resources/Python.app/Contents/MacOS/python ../python/dist/src/Mac/scripts/BuildApplet.py --output /Library/Frameworks/../../Applications/MacPython-2.3/IDLE.app --extra ../python/dist/src/Tools/idle ../python/dist/src/Tools/idle/idle

or essentially:

python Mac/scripts/BuildApplet.py --output /Applications/MacPython-2.3/IDLE.app --extra Tools/idle Tools/idle/idle

The semantics of BuildApplet.py (well, buildtools.py) changed slightly, in a way that upset IDLE's build - it used to copy its argument (the main program file) into Resources/ under a new name - __main__.py. This changed so that the main program file keeps the same file name.

For IDLE, the main program is Tools/idle/idle but the Tools/idle directory needs to be copied into the bundle as well; before the slight semantic change in buildtools.py this worked OK but now two things named "idle" are getting copied to Resources/ and its not working.

Its hard to argue that BuildApplet really should support an --extra argument with the same basename as the main program file. Perhaps the best fix is to integrate the latest idlefork with Python 2.3 - this would eliminate this problem because the main file for mac applets is not called idle in idlefork.

Another way to fix this would be to extend BuildApplet and buildtools and bundlebuilder so that it is possible to specify a new name for the --extra argument, something like:

python Mac/scripts/BuildApplet.py --output IDLE.app --extra Tools/idle --rename-extra idlelib Tools/idle/idle


But the first fix option seems way more palateable.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=688266&group_id=5470


From noreply@sourceforge.net  Mon Feb 17 22:33:59 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Mon, 17 Feb 2003 14:33:59 -0800
Subject: [Python-bugs-list] [ python-Bugs-688266 ] IDLE does not work on Mac OS X
Message-ID: <E18ktpn-0002X5-00@sc8-sf-web3.sourceforge.net>

Bugs item #688266, was opened at 2003-02-17 21:47
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=688266&group_id=5470

Category: IDLE
Group: Platform-specific
Status: Open
Resolution: None
Priority: 5
Submitted By: Tony Lownds (tonylownds)
Assigned to: Nobody/Anonymous (nobody)
Summary: IDLE does not work on Mac OS X

Initial Comment:
When building a framework build of python, the IDLE application that is made does not work anymore.

Here is the command that is causing trouble:

/Library/Frameworks/../../Library/Frameworks/Python.framework/Versions/2.3/Resources/Python.app/Contents/MacOS/python ../python/dist/src/Mac/scripts/BuildApplet.py --output /Library/Frameworks/../../Applications/MacPython-2.3/IDLE.app --extra ../python/dist/src/Tools/idle ../python/dist/src/Tools/idle/idle

or essentially:

python Mac/scripts/BuildApplet.py --output /Applications/MacPython-2.3/IDLE.app --extra Tools/idle Tools/idle/idle

The semantics of BuildApplet.py (well, buildtools.py) changed slightly, in a way that upset IDLE's build - it used to copy its argument (the main program file) into Resources/ under a new name - __main__.py. This changed so that the main program file keeps the same file name.

For IDLE, the main program is Tools/idle/idle but the Tools/idle directory needs to be copied into the bundle as well; before the slight semantic change in buildtools.py this worked OK but now two things named "idle" are getting copied to Resources/ and its not working.

Its hard to argue that BuildApplet really should support an --extra argument with the same basename as the main program file. Perhaps the best fix is to integrate the latest idlefork with Python 2.3 - this would eliminate this problem because the main file for mac applets is not called idle in idlefork.

Another way to fix this would be to extend BuildApplet and buildtools and bundlebuilder so that it is possible to specify a new name for the --extra argument, something like:

python Mac/scripts/BuildApplet.py --output IDLE.app --extra Tools/idle --rename-extra idlelib Tools/idle/idle


But the first fix option seems way more palateable.

----------------------------------------------------------------------

>Comment By: Tony Lownds (tonylownds)
Date: 2003-02-17 22:33

Message:
Logged In: YES 
user_id=24100

I think I have a found a good solution. Instead of using Tools/idle/idle as the main program argument to BuildApplet.py, the patch to Mac/OSX/Makefile below uses Tools/idle/macosx_main.py, which is attached separately.

The macosx_main.py file I provide is a copy of Tools/idle/idle with some code to work around another recent problem with running IDLE on Mac OS X. The new code removes some funny arguments that Mac OS X adds when GUI applications are launched. This same code can be found in Lib/plat-mac/argvemulator.py. I would have used argvemulator.py directly but I don't know if that would have other consequences. 

The patch to Mac/OSX/Makefile will be necessary when idlefork is integrated back into python. That is the reason for naming the new file macosx_main.py

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=688266&group_id=5470


From noreply@sourceforge.net  Tue Feb 18 00:51:26 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Mon, 17 Feb 2003 16:51:26 -0800
Subject: [Python-bugs-list] [ python-Bugs-683416 ] Whats new: Obligatory source code encodings if not ASCII
Message-ID: <E18kvyo-0003VN-00@sc8-sf-web2.sourceforge.net>

Bugs item #683416, was opened at 2003-02-09 09:52
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=683416&group_id=5470

Category: Documentation
Group: Python 2.3
>Status: Closed
>Resolution: Fixed
Priority: 5
Submitted By: Roman Suzi (rnd0110)
Assigned to: A.M. Kuchling (akuchling)
Summary: Whats new: Obligatory source code encodings if not ASCII

Initial Comment:
As per PEP 263, source code encoding cookie is a must
for source code file. This fact  is not stressed enough in
the documentation.

I think, these sections must be changed to reflect PEP263:

What's new in Python 2.3

Section 3: PEP 263 Source Code Encoding
 must inform the reader on this:

Source files which use non-ASCII characters and do not
have encoding declaration are phased out. Python 2.3 
gives DeprecationWarnings. Later versions will give
SyntraxError
instead.


13.1 String changes

Unicode literals can now contain characters of the
source file encoding if such was specified (see PEP 263).



18 Porting to Python 2.3 section needs addition:

* to avoid DeprecationWarnings, add encoding
declaration to all source files according to PEP 263,
if they are using non-ASCII characters. After that,
Unicode strings could be specified with symbols from
specified encoding.

(Please correct my English, as it is not native)

(I'll look into other docs and submit corrections to
them separately)


----------------------------------------------------------------------

>Comment By: A.M. Kuchling (akuchling)
Date: 2003-02-17 19:51

Message:
Logged In: YES 
user_id=11375

Updated; thanks!

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=683416&group_id=5470


From noreply@sourceforge.net  Tue Feb 18 00:56:24 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Mon, 17 Feb 2003 16:56:24 -0800
Subject: [Python-bugs-list] [ python-Bugs-688261 ] (Very) Small Documentation Error
Message-ID: <E18kw3c-0008Ma-00@sc8-sf-web1.sourceforge.net>

Bugs item #688261, was opened at 2003-02-17 16:37
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=688261&group_id=5470

Category: Documentation
Group: Python 2.3
>Status: Closed
>Resolution: Fixed
Priority: 5
Submitted By: Chris Lambert (stallion)
>Assigned to: A.M. Kuchling (akuchling)
Summary: (Very) Small Documentation Error

Initial Comment:
In the "What's new" section:
http://www.python.org/doc/2.3a1/whatsnew/node15.html

$ ./python opt.py --input=data --length=4
<Values at 0x400cad2c: {'input': 'data', 'length': 4}>
['arg1']
$

shouldn't print
['arg1']

----------------------------------------------------------------------

>Comment By: A.M. Kuchling (akuchling)
Date: 2003-02-17 19:56

Message:
Logged In: YES 
user_id=11375

Fixed; thanks!

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=688261&group_id=5470


From noreply@sourceforge.net  Tue Feb 18 00:59:01 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Mon, 17 Feb 2003 16:59:01 -0800
Subject: [Python-bugs-list] [ python-Bugs-687654 ] Future division breaks mpz
Message-ID: <E18kw69-0007qY-00@sc8-sf-web3.sourceforge.net>

Bugs item #687654, was opened at 2003-02-16 17:41
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=687654&group_id=5470

Category: None
Group: Python 2.2.2
Status: Open
Resolution: None
Priority: 5
Submitted By: Jp Calderone (kuran)
Assigned to: Nobody/Anonymous (nobody)
Summary: Future division breaks mpz

Initial Comment:
When future division is enabled in Python 2.2.2,
division of mpz objects no longer functions.

>>> from mpz import mpz
>>> mpz(12) / mpz(4)
mpz(3)
>>> from __future__ import division
>>> mpz(12) / mpz(4)
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
TypeError: unsupported operand type(s) for /: 'mpz.mpz'
and 'mpz.mpz'
>>> mpz(12) // mpz(4)
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
TypeError: unsupported operand type(s) for //:
'mpz.mpz' and 'mpz.mpz'


----------------------------------------------------------------------

>Comment By: A.M. Kuchling (akuchling)
Date: 2003-02-17 19:59

Message:
Logged In: YES 
user_id=11375

According to the docs for Python 2.2, the mpz module is deprecated and will be removed in Python 2.3, so it doesn't seem worth fixing this for a 
2.2.3 release.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=687654&group_id=5470


From noreply@sourceforge.net  Tue Feb 18 04:03:15 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Mon, 17 Feb 2003 20:03:15 -0800
Subject: [Python-bugs-list] [ python-Bugs-688424 ] 64-bit test failures
Message-ID: <E18kyyR-0005DZ-00@sc8-sf-web3.sourceforge.net>

Bugs item #688424, was opened at 2003-02-17 23:03
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=688424&group_id=5470

Category: Python Interpreter Core
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Neal Norwitz (nnorwitz)
Assigned to: Guido van Rossum (gvanrossum)
Summary: 64-bit test failures

Initial Comment:
The 3 failures are:  test_compile test_hexoct test_sys.
------------------
test_compile failed -- eval('0xffffffff') gave
4294967295, but expected -1

 - Should this case be special cased for 32 vs 64 bit
archs?
------------------
test_hexoct fails in the unsigned tests:

test_hex_unsigned (test.test_hexoct.TextHexOct) ... FAIL
test_oct_unsigned (test.test_hexoct.TextHexOct) ... FAIL

AssertionError: 2147483648 != -2147483648L
------------------
test_sys is specifically in _getframe(sys.maxint):
        int depth = -1;

        if (!PyArg_ParseTuple(args, "|i:_getframe",
&depth))
                return NULL;
This raises an OverflowError instead of a ValueError
which is expected.
Should all "i"s in sysmodule be changed to "l"s with
corresponding changes to long for the C variable?  Or
is the test bogus and we should use a value smaller
than maxint?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=688424&group_id=5470


From noreply@sourceforge.net  Tue Feb 18 10:45:28 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 18 Feb 2003 02:45:28 -0800
Subject: [Python-bugs-list] [ python-Bugs-688581 ] sys.stdout.flush() in regrtest
Message-ID: <E18l5Fg-0006OI-00@sc8-sf-web2.sourceforge.net>

Bugs item #688581, was opened at 2003-02-18 11:45
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=688581&group_id=5470

Category: Python Library
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Jack Jansen (jackjansen)
Assigned to: Guido van Rossum (gvanrossum)
Summary: sys.stdout.flush() in regrtest

Initial Comment:
regrtest.py got some sys.stdout.flush() calls in rev. 1.96 (and fred added more in 1.122). But according to the documentation of the sys module stdout can be "any object that has a write method" and nothing is said about flush.

This bug is in 2.2.2 also (but not in 2.2.1), so a fix will have to be backported.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=688581&group_id=5470


From noreply@sourceforge.net  Tue Feb 18 12:32:03 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 18 Feb 2003 04:32:03 -0800
Subject: [Python-bugs-list] [ python-Bugs-688007 ] Bundlebuilder needs to pre-convert resource files
Message-ID: <E18l6up-0007v5-00@sc8-sf-web4.sourceforge.net>

Bugs item #688007, was opened at 2003-02-17 15:21
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=688007&group_id=5470

Category: Macintosh
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Jack Jansen (jackjansen)
Assigned to: Just van Rossum (jvr)
Summary: Bundlebuilder needs to pre-convert resource files

Initial Comment:
If bundlebuilder encounters an applesingle-encoded resource file it should pre-convert this to a datafork-based resource file.

At the moment the file is copied to the Resources directory verbatim, and this can cause problems if you use "sudo make frameworkinstall" to install Python: the resource folder will be mode 755 owned by root, and any normal user trying to run the applet will get a crash when the applet tries to unpack the applesingle .rsrc file into the datafork resource file .rsrc.df.rsrc.

----------------------------------------------------------------------

>Comment By: Just van Rossum (jvr)
Date: 2003-02-18 13:32

Message:
Logged In: YES 
user_id=92689

Yes, it must be compatible with 2.2 (it also ships with PyObjC). bundlebuilder.py was deliberately written to have very few dependencies: plistlib.py is the only dependency outside the std lib.

I'm not sure I like this patch to begin with: for one it seems it's doing this resource processing at the wrong abstraction level, it adds a special case for something that I don't like being treated specially within bundlebuilder. It would be much better if this preprocessing was done elsewhere.

In other words: I think the bug is in attempting to write the converted file to begin with, not with bundlebuilder not doing the conversion.

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-02-17 17:56

Message:
Logged In: YES 
user_id=45365

Fixed in bundlebuilder.py rev 1.8, macresource rev. 1.3.

Just, I'm assigning this to you because I'm not sure whether you still keep bundlebuilder 2.2-compatible. If so you may need to disable this bit of magic if macresource.install isn't available. Please close the report when you're happy.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=688007&group_id=5470


From noreply@sourceforge.net  Tue Feb 18 12:43:17 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 18 Feb 2003 04:43:17 -0800
Subject: [Python-bugs-list] [ python-Bugs-688424 ] 64-bit test failures
Message-ID: <E18l75h-0006CD-00@sc8-sf-web3.sourceforge.net>

Bugs item #688424, was opened at 2003-02-17 23:03
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=688424&group_id=5470

Category: Python Interpreter Core
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Neal Norwitz (nnorwitz)
>Assigned to: Neal Norwitz (nnorwitz)
Summary: 64-bit test failures

Initial Comment:
The 3 failures are:  test_compile test_hexoct test_sys.
------------------
test_compile failed -- eval('0xffffffff') gave
4294967295, but expected -1

 - Should this case be special cased for 32 vs 64 bit
archs?
------------------
test_hexoct fails in the unsigned tests:

test_hex_unsigned (test.test_hexoct.TextHexOct) ... FAIL
test_oct_unsigned (test.test_hexoct.TextHexOct) ... FAIL

AssertionError: 2147483648 != -2147483648L
------------------
test_sys is specifically in _getframe(sys.maxint):
        int depth = -1;

        if (!PyArg_ParseTuple(args, "|i:_getframe",
&depth))
                return NULL;
This raises an OverflowError instead of a ValueError
which is expected.
Should all "i"s in sysmodule be changed to "l"s with
corresponding changes to long for the C variable?  Or
is the test bogus and we should use a value smaller
than maxint?

----------------------------------------------------------------------

>Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-18 07:43

Message:
Logged In: YES 
user_id=6380

Thanks for checking!

test_compile.py, eval('0xffffffff'): yes, on a 64-bit
machine this should test eval('0xffffffffffffffff') and
expect -1.

Ditto for test_hex/oct_unsigned (use 0x800000000000 etc.)

test_sys: since I don't expect anyone to ever have 2 billion
frames, change the test -- using an explicit 2000000000
would be fine IMO, as would be an explicit 0x7ffffff.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=688424&group_id=5470


From noreply@sourceforge.net  Tue Feb 18 13:00:54 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 18 Feb 2003 05:00:54 -0800
Subject: [Python-bugs-list] [ python-Bugs-688581 ] sys.stdout.flush() in regrtest
Message-ID: <E18l7Mk-0001PJ-00@sc8-sf-web1.sourceforge.net>

Bugs item #688581, was opened at 2003-02-18 05:45
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=688581&group_id=5470

Category: Python Library
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Jack Jansen (jackjansen)
Assigned to: Guido van Rossum (gvanrossum)
Summary: sys.stdout.flush() in regrtest

Initial Comment:
regrtest.py got some sys.stdout.flush() calls in rev. 1.96 (and fred added more in 1.122). But according to the documentation of the sys module stdout can be "any object that has a write method" and nothing is said about flush.

This bug is in 2.2.2 also (but not in 2.2.1), so a fix will have to be backported.

----------------------------------------------------------------------

>Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-18 08:00

Message:
Logged In: YES 
user_id=6380

What's your use case? Since regrtest mostly controls its own
stdout, I'm not sure that the documented requirement applies.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=688581&group_id=5470


From noreply@sourceforge.net  Tue Feb 18 14:04:19 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 18 Feb 2003 06:04:19 -0800
Subject: [Python-bugs-list] [ python-Bugs-688007 ] Bundlebuilder needs to pre-convert resource files
Message-ID: <E18l8M7-0004kO-00@sc8-sf-web4.sourceforge.net>

Bugs item #688007, was opened at 2003-02-17 15:21
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=688007&group_id=5470

Category: Macintosh
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Jack Jansen (jackjansen)
Assigned to: Just van Rossum (jvr)
Summary: Bundlebuilder needs to pre-convert resource files

Initial Comment:
If bundlebuilder encounters an applesingle-encoded resource file it should pre-convert this to a datafork-based resource file.

At the moment the file is copied to the Resources directory verbatim, and this can cause problems if you use "sudo make frameworkinstall" to install Python: the resource folder will be mode 755 owned by root, and any normal user trying to run the applet will get a crash when the applet tries to unpack the applesingle .rsrc file into the datafork resource file .rsrc.df.rsrc.

----------------------------------------------------------------------

>Comment By: Jack Jansen (jackjansen)
Date: 2003-02-18 15:04

Message:
Logged In: YES 
user_id=45365

I agree on a theoretical level, but not on a practical one. The .rsrc files in the CVS repository are AppleSingle encoded, and I can't think of another way to keep them useable on both MacOS9 and MacOSX.

Any suggestions?

----------------------------------------------------------------------

Comment By: Just van Rossum (jvr)
Date: 2003-02-18 13:32

Message:
Logged In: YES 
user_id=92689

Yes, it must be compatible with 2.2 (it also ships with PyObjC). bundlebuilder.py was deliberately written to have very few dependencies: plistlib.py is the only dependency outside the std lib.

I'm not sure I like this patch to begin with: for one it seems it's doing this resource processing at the wrong abstraction level, it adds a special case for something that I don't like being treated specially within bundlebuilder. It would be much better if this preprocessing was done elsewhere.

In other words: I think the bug is in attempting to write the converted file to begin with, not with bundlebuilder not doing the conversion.

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-02-17 17:56

Message:
Logged In: YES 
user_id=45365

Fixed in bundlebuilder.py rev 1.8, macresource rev. 1.3.

Just, I'm assigning this to you because I'm not sure whether you still keep bundlebuilder 2.2-compatible. If so you may need to disable this bit of magic if macresource.install isn't available. Please close the report when you're happy.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=688007&group_id=5470


From noreply@sourceforge.net  Tue Feb 18 15:08:02 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 18 Feb 2003 07:08:02 -0800
Subject: [Python-bugs-list] [ python-Bugs-688007 ] Bundlebuilder needs to pre-convert resource files
Message-ID: <E18l9Lm-0007Xn-00@sc8-sf-web1.sourceforge.net>

Bugs item #688007, was opened at 2003-02-17 15:21
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=688007&group_id=5470

Category: Macintosh
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Jack Jansen (jackjansen)
Assigned to: Just van Rossum (jvr)
Summary: Bundlebuilder needs to pre-convert resource files

Initial Comment:
If bundlebuilder encounters an applesingle-encoded resource file it should pre-convert this to a datafork-based resource file.

At the moment the file is copied to the Resources directory verbatim, and this can cause problems if you use "sudo make frameworkinstall" to install Python: the resource folder will be mode 755 owned by root, and any normal user trying to run the applet will get a crash when the applet tries to unpack the applesingle .rsrc file into the datafork resource file .rsrc.df.rsrc.

----------------------------------------------------------------------

>Comment By: Just van Rossum (jvr)
Date: 2003-02-18 16:08

Message:
Logged In: YES 
user_id=92689

Aren't datafork res files supported in OS9? If so, this might be an opportunity to get rid of dataforks (as well as macresource.py) altogether...

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-02-18 15:04

Message:
Logged In: YES 
user_id=45365

I agree on a theoretical level, but not on a practical one. The .rsrc files in the CVS repository are AppleSingle encoded, and I can't think of another way to keep them useable on both MacOS9 and MacOSX.

Any suggestions?

----------------------------------------------------------------------

Comment By: Just van Rossum (jvr)
Date: 2003-02-18 13:32

Message:
Logged In: YES 
user_id=92689

Yes, it must be compatible with 2.2 (it also ships with PyObjC). bundlebuilder.py was deliberately written to have very few dependencies: plistlib.py is the only dependency outside the std lib.

I'm not sure I like this patch to begin with: for one it seems it's doing this resource processing at the wrong abstraction level, it adds a special case for something that I don't like being treated specially within bundlebuilder. It would be much better if this preprocessing was done elsewhere.

In other words: I think the bug is in attempting to write the converted file to begin with, not with bundlebuilder not doing the conversion.

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-02-17 17:56

Message:
Logged In: YES 
user_id=45365

Fixed in bundlebuilder.py rev 1.8, macresource rev. 1.3.

Just, I'm assigning this to you because I'm not sure whether you still keep bundlebuilder 2.2-compatible. If so you may need to disable this bit of magic if macresource.install isn't available. Please close the report when you're happy.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=688007&group_id=5470


From noreply@sourceforge.net  Tue Feb 18 15:31:09 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 18 Feb 2003 07:31:09 -0800
Subject: [Python-bugs-list] [ python-Bugs-688424 ] 64-bit test failures
Message-ID: <E18l9i9-0004TS-00@sc8-sf-web2.sourceforge.net>

Bugs item #688424, was opened at 2003-02-17 23:03
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=688424&group_id=5470

Category: Python Interpreter Core
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Neal Norwitz (nnorwitz)
Assigned to: Neal Norwitz (nnorwitz)
Summary: 64-bit test failures

Initial Comment:
The 3 failures are:  test_compile test_hexoct test_sys.
------------------
test_compile failed -- eval('0xffffffff') gave
4294967295, but expected -1

 - Should this case be special cased for 32 vs 64 bit
archs?
------------------
test_hexoct fails in the unsigned tests:

test_hex_unsigned (test.test_hexoct.TextHexOct) ... FAIL
test_oct_unsigned (test.test_hexoct.TextHexOct) ... FAIL

AssertionError: 2147483648 != -2147483648L
------------------
test_sys is specifically in _getframe(sys.maxint):
        int depth = -1;

        if (!PyArg_ParseTuple(args, "|i:_getframe",
&depth))
                return NULL;
This raises an OverflowError instead of a ValueError
which is expected.
Should all "i"s in sysmodule be changed to "l"s with
corresponding changes to long for the C variable?  Or
is the test bogus and we should use a value smaller
than maxint?

----------------------------------------------------------------------

>Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-18 10:31

Message:
Logged In: YES 
user_id=33168

Checked in as:
 Lib/test/test_compile.py 1.17
 Lib/test/test_sys.py 1.3
(still need to fix test_hexoct)

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-18 07:43

Message:
Logged In: YES 
user_id=6380

Thanks for checking!

test_compile.py, eval('0xffffffff'): yes, on a 64-bit
machine this should test eval('0xffffffffffffffff') and
expect -1.

Ditto for test_hex/oct_unsigned (use 0x800000000000 etc.)

test_sys: since I don't expect anyone to ever have 2 billion
frames, change the test -- using an explicit 2000000000
would be fine IMO, as would be an explicit 0x7ffffff.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=688424&group_id=5470


From noreply@sourceforge.net  Tue Feb 18 15:50:14 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 18 Feb 2003 07:50:14 -0800
Subject: [Python-bugs-list] [ python-Bugs-688007 ] Bundlebuilder needs to pre-convert resource files
Message-ID: <E18lA0c-0001MV-00@sc8-sf-web4.sourceforge.net>

Bugs item #688007, was opened at 2003-02-17 15:21
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=688007&group_id=5470

Category: Macintosh
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Jack Jansen (jackjansen)
Assigned to: Just van Rossum (jvr)
Summary: Bundlebuilder needs to pre-convert resource files

Initial Comment:
If bundlebuilder encounters an applesingle-encoded resource file it should pre-convert this to a datafork-based resource file.

At the moment the file is copied to the Resources directory verbatim, and this can cause problems if you use "sudo make frameworkinstall" to install Python: the resource folder will be mode 755 owned by root, and any normal user trying to run the applet will get a crash when the applet tries to unpack the applesingle .rsrc file into the datafork resource file .rsrc.df.rsrc.

----------------------------------------------------------------------

>Comment By: Jack Jansen (jackjansen)
Date: 2003-02-18 16:50

Message:
Logged In: YES 
user_id=45365

The FSRef APIs for data fork stuff is available under OS9, but (a) this won't work for applets, (b) ResEdit and Resorcerer don't understand this and (c) I don't think CodeWarrior handles it. So until OS9 support is deprecated wer're stuck with resource forks:-(

----------------------------------------------------------------------

Comment By: Just van Rossum (jvr)
Date: 2003-02-18 16:08

Message:
Logged In: YES 
user_id=92689

Aren't datafork res files supported in OS9? If so, this might be an opportunity to get rid of dataforks (as well as macresource.py) altogether...

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-02-18 15:04

Message:
Logged In: YES 
user_id=45365

I agree on a theoretical level, but not on a practical one. The .rsrc files in the CVS repository are AppleSingle encoded, and I can't think of another way to keep them useable on both MacOS9 and MacOSX.

Any suggestions?

----------------------------------------------------------------------

Comment By: Just van Rossum (jvr)
Date: 2003-02-18 13:32

Message:
Logged In: YES 
user_id=92689

Yes, it must be compatible with 2.2 (it also ships with PyObjC). bundlebuilder.py was deliberately written to have very few dependencies: plistlib.py is the only dependency outside the std lib.

I'm not sure I like this patch to begin with: for one it seems it's doing this resource processing at the wrong abstraction level, it adds a special case for something that I don't like being treated specially within bundlebuilder. It would be much better if this preprocessing was done elsewhere.

In other words: I think the bug is in attempting to write the converted file to begin with, not with bundlebuilder not doing the conversion.

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-02-17 17:56

Message:
Logged In: YES 
user_id=45365

Fixed in bundlebuilder.py rev 1.8, macresource rev. 1.3.

Just, I'm assigning this to you because I'm not sure whether you still keep bundlebuilder 2.2-compatible. If so you may need to disable this bit of magic if macresource.install isn't available. Please close the report when you're happy.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=688007&group_id=5470


From noreply@sourceforge.net  Tue Feb 18 15:54:16 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 18 Feb 2003 07:54:16 -0800
Subject: [Python-bugs-list] [ python-Bugs-688424 ] 64-bit test failures
Message-ID: <E18lA4W-0005mv-00@sc8-sf-web2.sourceforge.net>

Bugs item #688424, was opened at 2003-02-17 23:03
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=688424&group_id=5470

Category: Python Interpreter Core
Group: Python 2.3
>Status: Closed
>Resolution: Fixed
Priority: 5
Submitted By: Neal Norwitz (nnorwitz)
Assigned to: Neal Norwitz (nnorwitz)
Summary: 64-bit test failures

Initial Comment:
The 3 failures are:  test_compile test_hexoct test_sys.
------------------
test_compile failed -- eval('0xffffffff') gave
4294967295, but expected -1

 - Should this case be special cased for 32 vs 64 bit
archs?
------------------
test_hexoct fails in the unsigned tests:

test_hex_unsigned (test.test_hexoct.TextHexOct) ... FAIL
test_oct_unsigned (test.test_hexoct.TextHexOct) ... FAIL

AssertionError: 2147483648 != -2147483648L
------------------
test_sys is specifically in _getframe(sys.maxint):
        int depth = -1;

        if (!PyArg_ParseTuple(args, "|i:_getframe",
&depth))
                return NULL;
This raises an OverflowError instead of a ValueError
which is expected.
Should all "i"s in sysmodule be changed to "l"s with
corresponding changes to long for the C variable?  Or
is the test bogus and we should use a value smaller
than maxint?

----------------------------------------------------------------------

>Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-18 10:54

Message:
Logged In: YES 
user_id=33168

Checked in as:
 Lib/test/test_hexoct.py 1.3

That's all for now.


----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-18 10:31

Message:
Logged In: YES 
user_id=33168

Checked in as:
 Lib/test/test_compile.py 1.17
 Lib/test/test_sys.py 1.3
(still need to fix test_hexoct)

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-18 07:43

Message:
Logged In: YES 
user_id=6380

Thanks for checking!

test_compile.py, eval('0xffffffff'): yes, on a 64-bit
machine this should test eval('0xffffffffffffffff') and
expect -1.

Ditto for test_hex/oct_unsigned (use 0x800000000000 etc.)

test_sys: since I don't expect anyone to ever have 2 billion
frames, change the test -- using an explicit 2000000000
would be fine IMO, as would be an explicit 0x7ffffff.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=688424&group_id=5470


From noreply@sourceforge.net  Tue Feb 18 17:50:17 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 18 Feb 2003 09:50:17 -0800
Subject: [Python-bugs-list] [ python-Bugs-628842 ] Deprecate PyNumber_Check
Message-ID: <E18lBsn-0003g0-00@sc8-sf-web2.sourceforge.net>

Bugs item #628842, was opened at 2002-10-25 21:35
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=628842&group_id=5470

Category: Python Interpreter Core
Group: None
Status: Open
Resolution: None
Priority: 4
Submitted By: Neil Schemenauer (nascheme)
Assigned to: Neil Schemenauer (nascheme)
Summary: Deprecate PyNumber_Check

Initial Comment:
I think PyNumber_Check should be deprecated.  An type
having tp_as_number does not mean it is a number.  If we
don't depreciate it, perhaps we should modify the test.
 For
example, we could check for a __int__ method.

Guido, if you decide what to do, assign the bug back to
me and I will work on a patch.

----------------------------------------------------------------------

>Comment By: Neil Schemenauer (nascheme)
Date: 2003-02-18 17:50

Message:
Logged In: YES 
user_id=35752

Progress update:

Guido has changed PyNumber_Check() to check for
nb_int and nb_float.  I improved the PyFloat_AsDouble()
exception message.  Still need to check the usage of
PySequence_Check() and PyMapping_Check().

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2002-10-29 23:02

Message:
Logged In: YES 
user_id=6380

Adding a test for nb_int *or* nb_float would seem an
adequate redefinition of PyNumber_Check(). But it's still
not perfect -- a class instance *always* passes this test,
for example, and you can get select() to report strange
errors by passing a class instance. (And it also doesn't
help for complex.) The problem is, what to use in the select
module instead of PyNumber_Check()? There really isn't a
decent alternative; we don't want to call PyInt_Check(),
PyLong_Check() and PyFloat_Check() directly, and
PyFloat_AsDouble() reports an unhelpful "bad argument type
for built-in operation" when given a non-number. Hm, maybe
PyFloat_AsDouble() should be changed to report a better error???

I've also noticed that at least some of the calls to
PySequence_Check() seem bogus -- e.g. the ones in
PySequence_List() and in list_fill() (which look like the
same code BTW, as do several others). In fact, I think most
(all?) uses of this should not be there, since they are only
designed to give "nice" error messages, but they are neither
necessary nor sufficient to catch all cases. (They aren't
necessary because all the other PySequence_XXX() APIs do
proper checking anyway.)

I also noticed that PyMapping_Check() again gives a false
sense of security in e.g. posix_execve().

----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2002-10-28 13:10

Message:
Logged In: YES 
user_id=80475

A grep of Modules, Objects, and Python shows that 
PyNumber_Check is only used in selectmodule.c

The sequence and mapping checks are used much more 
frequently but not naively.  They are used as guards before 
the slot is referenced.



----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2002-10-28 02:39

Message:
Logged In: YES 
user_id=6380

I agree that checking for tp_as_number is bogus. (Ditto for
sequence and mapping -- are these similarly naive?)

Is there any code that currently uses PyNumber_Check? 
Depending on what it is used for, I'd make a different decision.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=628842&group_id=5470


From noreply@sourceforge.net  Tue Feb 18 18:20:23 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 18 Feb 2003 10:20:23 -0800
Subject: [Python-bugs-list] [ python-Feature Requests-688805 ] gzlib: request iterator
Message-ID: <E18lCLv-0001U9-00@sc8-sf-web4.sourceforge.net>

Feature Requests item #688805, was opened at 2003-02-18 10:20
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=688805&group_id=5470

Category: Python Library
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Russell Owen (reowen)
Assigned to: Nobody/Anonymous (nobody)
Summary: gzlib: request iterator

Initial Comment:
I'd like to request an iterator for gzlib, so that GzipFile objects can be treated like regular files, e.g.:
gfile = gzlib.GzipFile(filename, 'r')
for aline in gfile:
   # do stuff

Here is a simple workaround that I use; this could be implemented in the gzlib itself, though I suspect there's a better way to do it (and if anybody cares to tell me what it is, I'd be happy to hear; I'm still finding my way around with generators and iterators):

class FixGZip(gzip.GzipFile):
	"""the GzipFile class normally does not have an iterator; add one"""
	def __iter__(self):
		def stopIter():
			raise StopIteration
		while True:
			yield self.readline() or stopIter()


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=688805&group_id=5470


From noreply@sourceforge.net  Tue Feb 18 18:40:45 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 18 Feb 2003 10:40:45 -0800
Subject: [Python-bugs-list] [ python-Bugs-688907 ] Applet support is broken
Message-ID: <E18lCfd-0002WH-00@sc8-sf-web4.sourceforge.net>

Bugs item #688907, was opened at 2003-02-18 19:40
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=688907&group_id=5470

Category: Macintosh
Group: None
Status: Open
Resolution: None
Priority: 8
Submitted By: Jack Jansen (jackjansen)
Assigned to: Jack Jansen (jackjansen)
Summary: Applet support is broken

Initial Comment:
Applet support is broken in two ways:
- there is no sys.argv emulation anymore, and applets always see the -psnXXXX argument when started from the finder.
- when an applet is started from the commandline it is non-functional (no window manager connection).

Also, the building of the Idle applet doesn't work due to name conflicts.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=688907&group_id=5470


From noreply@sourceforge.net  Tue Feb 18 19:17:48 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 18 Feb 2003 11:17:48 -0800
Subject: [Python-bugs-list] [ python-Bugs-688007 ] Bundlebuilder needs to pre-convert resource files
Message-ID: <E18lDFU-0003lY-00@sc8-sf-web4.sourceforge.net>

Bugs item #688007, was opened at 2003-02-17 15:21
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=688007&group_id=5470

Category: Macintosh
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Jack Jansen (jackjansen)
Assigned to: Just van Rossum (jvr)
Summary: Bundlebuilder needs to pre-convert resource files

Initial Comment:
If bundlebuilder encounters an applesingle-encoded resource file it should pre-convert this to a datafork-based resource file.

At the moment the file is copied to the Resources directory verbatim, and this can cause problems if you use "sudo make frameworkinstall" to install Python: the resource folder will be mode 755 owned by root, and any normal user trying to run the applet will get a crash when the applet tries to unpack the applesingle .rsrc file into the datafork resource file .rsrc.df.rsrc.

----------------------------------------------------------------------

>Comment By: Just van Rossum (jvr)
Date: 2003-02-18 20:17

Message:
Logged In: YES 
user_id=92689

Ok, next try: can't BuildApplet and/or buildtools.py do the conversion?

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-02-18 16:50

Message:
Logged In: YES 
user_id=45365

The FSRef APIs for data fork stuff is available under OS9, but (a) this won't work for applets, (b) ResEdit and Resorcerer don't understand this and (c) I don't think CodeWarrior handles it. So until OS9 support is deprecated wer're stuck with resource forks:-(

----------------------------------------------------------------------

Comment By: Just van Rossum (jvr)
Date: 2003-02-18 16:08

Message:
Logged In: YES 
user_id=92689

Aren't datafork res files supported in OS9? If so, this might be an opportunity to get rid of dataforks (as well as macresource.py) altogether...

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-02-18 15:04

Message:
Logged In: YES 
user_id=45365

I agree on a theoretical level, but not on a practical one. The .rsrc files in the CVS repository are AppleSingle encoded, and I can't think of another way to keep them useable on both MacOS9 and MacOSX.

Any suggestions?

----------------------------------------------------------------------

Comment By: Just van Rossum (jvr)
Date: 2003-02-18 13:32

Message:
Logged In: YES 
user_id=92689

Yes, it must be compatible with 2.2 (it also ships with PyObjC). bundlebuilder.py was deliberately written to have very few dependencies: plistlib.py is the only dependency outside the std lib.

I'm not sure I like this patch to begin with: for one it seems it's doing this resource processing at the wrong abstraction level, it adds a special case for something that I don't like being treated specially within bundlebuilder. It would be much better if this preprocessing was done elsewhere.

In other words: I think the bug is in attempting to write the converted file to begin with, not with bundlebuilder not doing the conversion.

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-02-17 17:56

Message:
Logged In: YES 
user_id=45365

Fixed in bundlebuilder.py rev 1.8, macresource rev. 1.3.

Just, I'm assigning this to you because I'm not sure whether you still keep bundlebuilder 2.2-compatible. If so you may need to disable this bit of magic if macresource.install isn't available. Please close the report when you're happy.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=688007&group_id=5470


From noreply@sourceforge.net  Tue Feb 18 19:18:18 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 18 Feb 2003 11:18:18 -0800
Subject: [Python-bugs-list] [ python-Bugs-595026 ] Support for masks in getargs.c
Message-ID: <E18lDFy-0008GC-00@sc8-sf-web3.sourceforge.net>

Bugs item #595026, was opened at 2002-08-14 14:26
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=595026&group_id=5470

Category: Python Interpreter Core
Group: Python 2.3
Status: Open
Resolution: None
Priority: 6
Submitted By: Guido van Rossum (gvanrossum)
Assigned to: Guido van Rossum (gvanrossum)
Summary: Support for masks in getargs.c

Initial Comment:
We need this implemented:

> How about the following counterproposal.  This also
changes some of
> the other format codes to be a little more regular.
>
> Code    C type          	Range check
>
> b	unsigned char		0..UCHAR_MAX
> B	unsigned char		none **
> h	unsigned short		0..USHRT_MAX
> H	unsigned short		none **
> i	int
		INT_MIN..INT_MAX
> I *	unsigned int		0..UINT_MAX
> l	long
		LONG_MIN..LONG_MAX
> k *	unsigned long		none
> L	long long		LLONG_MIN..LLONG_MAX
> K *	unsigned long long	none
>
> Notes:
>
> * New format codes.
>
> ** Changed from previous "range-and-a-half" to
"none"; the
>    range-and-a-half checking wasn't particularly useful.

Plus a C API or two, e.g. PyInt_AsLongMask() ->
unsigned long and PyInt_AsLongLongMask() -> unsigned
long long (if that exists).

----------------------------------------------------------------------

>Comment By: Thomas Heller (theller)
Date: 2003-02-18 20:18

Message:
Logged In: YES 
user_id=11105

If nobody else comes up, I can do this. I also had similar
checks for the struct module, but Tim killed it because of
backward compatibility.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=595026&group_id=5470


From noreply@sourceforge.net  Tue Feb 18 19:50:13 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 18 Feb 2003 11:50:13 -0800
Subject: [Python-bugs-list] [ python-Bugs-595026 ] Support for masks in getargs.c
Message-ID: <E18lDkr-000587-00@sc8-sf-web1.sourceforge.net>

Bugs item #595026, was opened at 2002-08-14 08:26
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=595026&group_id=5470

Category: Python Interpreter Core
Group: Python 2.3
Status: Open
Resolution: None
Priority: 6
Submitted By: Guido van Rossum (gvanrossum)
Assigned to: Guido van Rossum (gvanrossum)
Summary: Support for masks in getargs.c

Initial Comment:
We need this implemented:

> How about the following counterproposal.  This also
changes some of
> the other format codes to be a little more regular.
>
> Code    C type          	Range check
>
> b	unsigned char		0..UCHAR_MAX
> B	unsigned char		none **
> h	unsigned short		0..USHRT_MAX
> H	unsigned short		none **
> i	int
		INT_MIN..INT_MAX
> I *	unsigned int		0..UINT_MAX
> l	long
		LONG_MIN..LONG_MAX
> k *	unsigned long		none
> L	long long		LLONG_MIN..LLONG_MAX
> K *	unsigned long long	none
>
> Notes:
>
> * New format codes.
>
> ** Changed from previous "range-and-a-half" to
"none"; the
>    range-and-a-half checking wasn't particularly useful.

Plus a C API or two, e.g. PyInt_AsLongMask() ->
unsigned long and PyInt_AsLongLongMask() -> unsigned
long long (if that exists).

----------------------------------------------------------------------

>Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-18 14:50

Message:
Logged In: YES 
user_id=6380

Thomas: that would be great! (As long it isn't killed by bw
compat. :-)


----------------------------------------------------------------------

Comment By: Thomas Heller (theller)
Date: 2003-02-18 14:18

Message:
Logged In: YES 
user_id=11105

If nobody else comes up, I can do this. I also had similar
checks for the struct module, but Tim killed it because of
backward compatibility.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=595026&group_id=5470


From noreply@sourceforge.net  Tue Feb 18 20:16:06 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 18 Feb 2003 12:16:06 -0800
Subject: [Python-bugs-list] [ python-Bugs-595026 ] Support for masks in getargs.c
Message-ID: <E18lE9u-0006fz-00@sc8-sf-web1.sourceforge.net>

Bugs item #595026, was opened at 2002-08-14 14:26
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=595026&group_id=5470

Category: Python Interpreter Core
Group: Python 2.3
Status: Open
Resolution: None
Priority: 6
Submitted By: Guido van Rossum (gvanrossum)
Assigned to: Guido van Rossum (gvanrossum)
Summary: Support for masks in getargs.c

Initial Comment:
We need this implemented:

> How about the following counterproposal.  This also
changes some of
> the other format codes to be a little more regular.
>
> Code    C type          	Range check
>
> b	unsigned char		0..UCHAR_MAX
> B	unsigned char		none **
> h	unsigned short		0..USHRT_MAX
> H	unsigned short		none **
> i	int
		INT_MIN..INT_MAX
> I *	unsigned int		0..UINT_MAX
> l	long
		LONG_MIN..LONG_MAX
> k *	unsigned long		none
> L	long long		LLONG_MIN..LLONG_MAX
> K *	unsigned long long	none
>
> Notes:
>
> * New format codes.
>
> ** Changed from previous "range-and-a-half" to
"none"; the
>    range-and-a-half checking wasn't particularly useful.

Plus a C API or two, e.g. PyInt_AsLongMask() ->
unsigned long and PyInt_AsLongLongMask() -> unsigned
long long (if that exists).

----------------------------------------------------------------------

>Comment By: Thomas Heller (theller)
Date: 2003-02-18 21:16

Message:
Logged In: YES 
user_id=11105

But it would probably take a few days.

What about these changes to the format codes, they are now
more in sync with the struct module (I hope the formatting
is kept):

Code    C type          	Range check

b	unsigned char		0..UCHAR_MAX
B	unsigned char		none **
h	unsigned short		0..USHRT_MAX
H	unsigned short		none **
i	int			INT_MIN..INT_MAX
I *	unsigned int		0..UINT_MAX
l	long			LONG_MIN..LONG_MAX
L *	unsigned long		none
q	long long		LLONG_MIN..LLONG_MAX
Q *	unsigned long long	none


----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-18 20:50

Message:
Logged In: YES 
user_id=6380

Thomas: that would be great! (As long it isn't killed by bw
compat. :-)


----------------------------------------------------------------------

Comment By: Thomas Heller (theller)
Date: 2003-02-18 20:18

Message:
Logged In: YES 
user_id=11105

If nobody else comes up, I can do this. I also had similar
checks for the struct module, but Tim killed it because of
backward compatibility.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=595026&group_id=5470


From noreply@sourceforge.net  Tue Feb 18 20:18:56 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 18 Feb 2003 12:18:56 -0800
Subject: [Python-bugs-list] [ python-Bugs-595026 ] Support for masks in getargs.c
Message-ID: <E18lECe-0002QU-00@sc8-sf-web3.sourceforge.net>

Bugs item #595026, was opened at 2002-08-14 08:26
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=595026&group_id=5470

Category: Python Interpreter Core
Group: Python 2.3
Status: Open
Resolution: None
Priority: 6
Submitted By: Guido van Rossum (gvanrossum)
Assigned to: Guido van Rossum (gvanrossum)
Summary: Support for masks in getargs.c

Initial Comment:
We need this implemented:

> How about the following counterproposal.  This also
changes some of
> the other format codes to be a little more regular.
>
> Code    C type          	Range check
>
> b	unsigned char		0..UCHAR_MAX
> B	unsigned char		none **
> h	unsigned short		0..USHRT_MAX
> H	unsigned short		none **
> i	int
		INT_MIN..INT_MAX
> I *	unsigned int		0..UINT_MAX
> l	long
		LONG_MIN..LONG_MAX
> k *	unsigned long		none
> L	long long		LLONG_MIN..LLONG_MAX
> K *	unsigned long long	none
>
> Notes:
>
> * New format codes.
>
> ** Changed from previous "range-and-a-half" to
"none"; the
>    range-and-a-half checking wasn't particularly useful.

Plus a C API or two, e.g. PyInt_AsLongMask() ->
unsigned long and PyInt_AsLongLongMask() -> unsigned
long long (if that exists).

----------------------------------------------------------------------

>Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-18 15:18

Message:
Logged In: YES 
user_id=6380

A few days is fine (this doesn't need to make it into 2.3a2).

But the proposal here is not backwards compatible, is it?
Those codes already mean something different now.

I think we'll need to invent new format codes, or a new
modifier.

----------------------------------------------------------------------

Comment By: Thomas Heller (theller)
Date: 2003-02-18 15:16

Message:
Logged In: YES 
user_id=11105

But it would probably take a few days.

What about these changes to the format codes, they are now
more in sync with the struct module (I hope the formatting
is kept):

Code    C type          	Range check

b	unsigned char		0..UCHAR_MAX
B	unsigned char		none **
h	unsigned short		0..USHRT_MAX
H	unsigned short		none **
i	int			INT_MIN..INT_MAX
I *	unsigned int		0..UINT_MAX
l	long			LONG_MIN..LONG_MAX
L *	unsigned long		none
q	long long		LLONG_MIN..LLONG_MAX
Q *	unsigned long long	none


----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-18 14:50

Message:
Logged In: YES 
user_id=6380

Thomas: that would be great! (As long it isn't killed by bw
compat. :-)


----------------------------------------------------------------------

Comment By: Thomas Heller (theller)
Date: 2003-02-18 14:18

Message:
Logged In: YES 
user_id=11105

If nobody else comes up, I can do this. I also had similar
checks for the struct module, but Tim killed it because of
backward compatibility.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=595026&group_id=5470


From noreply@sourceforge.net  Tue Feb 18 20:33:43 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 18 Feb 2003 12:33:43 -0800
Subject: [Python-bugs-list] [ python-Bugs-595026 ] Support for masks in getargs.c
Message-ID: <E18lEQx-00081Y-00@sc8-sf-web1.sourceforge.net>

Bugs item #595026, was opened at 2002-08-14 14:26
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=595026&group_id=5470

Category: Python Interpreter Core
Group: Python 2.3
Status: Open
Resolution: None
Priority: 6
Submitted By: Guido van Rossum (gvanrossum)
Assigned to: Guido van Rossum (gvanrossum)
Summary: Support for masks in getargs.c

Initial Comment:
We need this implemented:

> How about the following counterproposal.  This also
changes some of
> the other format codes to be a little more regular.
>
> Code    C type          	Range check
>
> b	unsigned char		0..UCHAR_MAX
> B	unsigned char		none **
> h	unsigned short		0..USHRT_MAX
> H	unsigned short		none **
> i	int
		INT_MIN..INT_MAX
> I *	unsigned int		0..UINT_MAX
> l	long
		LONG_MIN..LONG_MAX
> k *	unsigned long		none
> L	long long		LLONG_MIN..LLONG_MAX
> K *	unsigned long long	none
>
> Notes:
>
> * New format codes.
>
> ** Changed from previous "range-and-a-half" to
"none"; the
>    range-and-a-half checking wasn't particularly useful.

Plus a C API or two, e.g. PyInt_AsLongMask() ->
unsigned long and PyInt_AsLongLongMask() -> unsigned
long long (if that exists).

----------------------------------------------------------------------

>Comment By: Thomas Heller (theller)
Date: 2003-02-18 21:33

Message:
Logged In: YES 
user_id=11105

Ok, I'll use your codes.

Here is my suggestion for the C api functions:

int PyInt_AsLongMask(PyObject *v, unsigned long *pval);
int PyInt_AsLongLongMask(PyObject *v, unsigned LONG_LONG *pval);

return -1 and set exception on error, return 0 otherwise and
store the result in pval. This saves the PyErr_Occurred() in
case the value is -1.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-18 21:18

Message:
Logged In: YES 
user_id=6380

A few days is fine (this doesn't need to make it into 2.3a2).

But the proposal here is not backwards compatible, is it?
Those codes already mean something different now.

I think we'll need to invent new format codes, or a new
modifier.

----------------------------------------------------------------------

Comment By: Thomas Heller (theller)
Date: 2003-02-18 21:16

Message:
Logged In: YES 
user_id=11105

But it would probably take a few days.

What about these changes to the format codes, they are now
more in sync with the struct module (I hope the formatting
is kept):

Code    C type          	Range check

b	unsigned char		0..UCHAR_MAX
B	unsigned char		none **
h	unsigned short		0..USHRT_MAX
H	unsigned short		none **
i	int			INT_MIN..INT_MAX
I *	unsigned int		0..UINT_MAX
l	long			LONG_MIN..LONG_MAX
L *	unsigned long		none
q	long long		LLONG_MIN..LLONG_MAX
Q *	unsigned long long	none


----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-18 20:50

Message:
Logged In: YES 
user_id=6380

Thomas: that would be great! (As long it isn't killed by bw
compat. :-)


----------------------------------------------------------------------

Comment By: Thomas Heller (theller)
Date: 2003-02-18 20:18

Message:
Logged In: YES 
user_id=11105

If nobody else comes up, I can do this. I also had similar
checks for the struct module, but Tim killed it because of
backward compatibility.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=595026&group_id=5470


From noreply@sourceforge.net  Tue Feb 18 20:59:03 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 18 Feb 2003 12:59:03 -0800
Subject: [Python-bugs-list] [ python-Bugs-595026 ] Support for masks in getargs.c
Message-ID: <E18lEpT-0000yX-00@sc8-sf-web4.sourceforge.net>

Bugs item #595026, was opened at 2002-08-14 14:26
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=595026&group_id=5470

Category: Python Interpreter Core
Group: Python 2.3
Status: Open
Resolution: None
Priority: 6
Submitted By: Guido van Rossum (gvanrossum)
Assigned to: Guido van Rossum (gvanrossum)
Summary: Support for masks in getargs.c

Initial Comment:
We need this implemented:

> How about the following counterproposal.  This also
changes some of
> the other format codes to be a little more regular.
>
> Code    C type          	Range check
>
> b	unsigned char		0..UCHAR_MAX
> B	unsigned char		none **
> h	unsigned short		0..USHRT_MAX
> H	unsigned short		none **
> i	int
		INT_MIN..INT_MAX
> I *	unsigned int		0..UINT_MAX
> l	long
		LONG_MIN..LONG_MAX
> k *	unsigned long		none
> L	long long		LLONG_MIN..LLONG_MAX
> K *	unsigned long long	none
>
> Notes:
>
> * New format codes.
>
> ** Changed from previous "range-and-a-half" to
"none"; the
>    range-and-a-half checking wasn't particularly useful.

Plus a C API or two, e.g. PyInt_AsLongMask() ->
unsigned long and PyInt_AsLongLongMask() -> unsigned
long long (if that exists).

----------------------------------------------------------------------

>Comment By: Thomas Heller (theller)
Date: 2003-02-18 21:59

Message:
Logged In: YES 
user_id=11105

Currently the h code means signed short
(SHRT_MIN..SHRT_MAX), do you really want to change that to
unsigned short (0..USHRT_MAX)?

----------------------------------------------------------------------

Comment By: Thomas Heller (theller)
Date: 2003-02-18 21:33

Message:
Logged In: YES 
user_id=11105

Ok, I'll use your codes.

Here is my suggestion for the C api functions:

int PyInt_AsLongMask(PyObject *v, unsigned long *pval);
int PyInt_AsLongLongMask(PyObject *v, unsigned LONG_LONG *pval);

return -1 and set exception on error, return 0 otherwise and
store the result in pval. This saves the PyErr_Occurred() in
case the value is -1.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-18 21:18

Message:
Logged In: YES 
user_id=6380

A few days is fine (this doesn't need to make it into 2.3a2).

But the proposal here is not backwards compatible, is it?
Those codes already mean something different now.

I think we'll need to invent new format codes, or a new
modifier.

----------------------------------------------------------------------

Comment By: Thomas Heller (theller)
Date: 2003-02-18 21:16

Message:
Logged In: YES 
user_id=11105

But it would probably take a few days.

What about these changes to the format codes, they are now
more in sync with the struct module (I hope the formatting
is kept):

Code    C type          	Range check

b	unsigned char		0..UCHAR_MAX
B	unsigned char		none **
h	unsigned short		0..USHRT_MAX
H	unsigned short		none **
i	int			INT_MIN..INT_MAX
I *	unsigned int		0..UINT_MAX
l	long			LONG_MIN..LONG_MAX
L *	unsigned long		none
q	long long		LLONG_MIN..LLONG_MAX
Q *	unsigned long long	none


----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-18 20:50

Message:
Logged In: YES 
user_id=6380

Thomas: that would be great! (As long it isn't killed by bw
compat. :-)


----------------------------------------------------------------------

Comment By: Thomas Heller (theller)
Date: 2003-02-18 20:18

Message:
Logged In: YES 
user_id=11105

If nobody else comes up, I can do this. I also had similar
checks for the struct module, but Tim killed it because of
backward compatibility.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=595026&group_id=5470


From noreply@sourceforge.net  Tue Feb 18 22:08:41 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 18 Feb 2003 14:08:41 -0800
Subject: [Python-bugs-list] [ python-Bugs-595026 ] Support for masks in getargs.c
Message-ID: <E18lFur-0004dG-00@sc8-sf-web1.sourceforge.net>

Bugs item #595026, was opened at 2002-08-14 14:26
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=595026&group_id=5470

Category: Python Interpreter Core
Group: Python 2.3
Status: Open
Resolution: None
Priority: 6
Submitted By: Guido van Rossum (gvanrossum)
Assigned to: Guido van Rossum (gvanrossum)
Summary: Support for masks in getargs.c

Initial Comment:
We need this implemented:

> How about the following counterproposal.  This also
changes some of
> the other format codes to be a little more regular.
>
> Code    C type          	Range check
>
> b	unsigned char		0..UCHAR_MAX
> B	unsigned char		none **
> h	unsigned short		0..USHRT_MAX
> H	unsigned short		none **
> i	int
		INT_MIN..INT_MAX
> I *	unsigned int		0..UINT_MAX
> l	long
		LONG_MIN..LONG_MAX
> k *	unsigned long		none
> L	long long		LLONG_MIN..LLONG_MAX
> K *	unsigned long long	none
>
> Notes:
>
> * New format codes.
>
> ** Changed from previous "range-and-a-half" to
"none"; the
>    range-and-a-half checking wasn't particularly useful.

Plus a C API or two, e.g. PyInt_AsLongMask() ->
unsigned long and PyInt_AsLongLongMask() -> unsigned
long long (if that exists).

----------------------------------------------------------------------

>Comment By: Jack Jansen (jackjansen)
Date: 2003-02-18 23:08

Message:
Logged In: YES 
user_id=45365

Guido,
I would be happy with the no-rangecheck unsigned long formatcode. One request, though: if this is implemented can it be done ASAP after 2.3a2, so there's still some time to shake out the bugs before 2.3b1 comes out? Especially the hand-written code will have to be examined to see where l needs to be turned into k.

----------------------------------------------------------------------

Comment By: Thomas Heller (theller)
Date: 2003-02-18 21:59

Message:
Logged In: YES 
user_id=11105

Currently the h code means signed short
(SHRT_MIN..SHRT_MAX), do you really want to change that to
unsigned short (0..USHRT_MAX)?

----------------------------------------------------------------------

Comment By: Thomas Heller (theller)
Date: 2003-02-18 21:33

Message:
Logged In: YES 
user_id=11105

Ok, I'll use your codes.

Here is my suggestion for the C api functions:

int PyInt_AsLongMask(PyObject *v, unsigned long *pval);
int PyInt_AsLongLongMask(PyObject *v, unsigned LONG_LONG *pval);

return -1 and set exception on error, return 0 otherwise and
store the result in pval. This saves the PyErr_Occurred() in
case the value is -1.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-18 21:18

Message:
Logged In: YES 
user_id=6380

A few days is fine (this doesn't need to make it into 2.3a2).

But the proposal here is not backwards compatible, is it?
Those codes already mean something different now.

I think we'll need to invent new format codes, or a new
modifier.

----------------------------------------------------------------------

Comment By: Thomas Heller (theller)
Date: 2003-02-18 21:16

Message:
Logged In: YES 
user_id=11105

But it would probably take a few days.

What about these changes to the format codes, they are now
more in sync with the struct module (I hope the formatting
is kept):

Code    C type          	Range check

b	unsigned char		0..UCHAR_MAX
B	unsigned char		none **
h	unsigned short		0..USHRT_MAX
H	unsigned short		none **
i	int			INT_MIN..INT_MAX
I *	unsigned int		0..UINT_MAX
l	long			LONG_MIN..LONG_MAX
L *	unsigned long		none
q	long long		LLONG_MIN..LLONG_MAX
Q *	unsigned long long	none


----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-18 20:50

Message:
Logged In: YES 
user_id=6380

Thomas: that would be great! (As long it isn't killed by bw
compat. :-)


----------------------------------------------------------------------

Comment By: Thomas Heller (theller)
Date: 2003-02-18 20:18

Message:
Logged In: YES 
user_id=11105

If nobody else comes up, I can do this. I also had similar
checks for the struct module, but Tim killed it because of
backward compatibility.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=595026&group_id=5470


From noreply@sourceforge.net  Tue Feb 18 23:42:22 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 18 Feb 2003 15:42:22 -0800
Subject: [Python-bugs-list] [ python-Bugs-688007 ] Bundlebuilder needs to pre-convert resource files
Message-ID: <E18lHNW-0003aF-00@sc8-sf-web3.sourceforge.net>

Bugs item #688007, was opened at 2003-02-17 15:21
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=688007&group_id=5470

Category: Macintosh
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Jack Jansen (jackjansen)
Assigned to: Just van Rossum (jvr)
Summary: Bundlebuilder needs to pre-convert resource files

Initial Comment:
If bundlebuilder encounters an applesingle-encoded resource file it should pre-convert this to a datafork-based resource file.

At the moment the file is copied to the Resources directory verbatim, and this can cause problems if you use "sudo make frameworkinstall" to install Python: the resource folder will be mode 755 owned by root, and any normal user trying to run the applet will get a crash when the applet tries to unpack the applesingle .rsrc file into the datafork resource file .rsrc.df.rsrc.

----------------------------------------------------------------------

>Comment By: Jack Jansen (jackjansen)
Date: 2003-02-19 00:42

Message:
Logged In: YES 
user_id=45365

With some hacking, yes. buildtools would have to create the datafork-based resource file, but it would have to keep its original name, so it should be copied to a temp directory somewhere....

Can this wait until after 2.3a2?

----------------------------------------------------------------------

Comment By: Just van Rossum (jvr)
Date: 2003-02-18 20:17

Message:
Logged In: YES 
user_id=92689

Ok, next try: can't BuildApplet and/or buildtools.py do the conversion?

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-02-18 16:50

Message:
Logged In: YES 
user_id=45365

The FSRef APIs for data fork stuff is available under OS9, but (a) this won't work for applets, (b) ResEdit and Resorcerer don't understand this and (c) I don't think CodeWarrior handles it. So until OS9 support is deprecated wer're stuck with resource forks:-(

----------------------------------------------------------------------

Comment By: Just van Rossum (jvr)
Date: 2003-02-18 16:08

Message:
Logged In: YES 
user_id=92689

Aren't datafork res files supported in OS9? If so, this might be an opportunity to get rid of dataforks (as well as macresource.py) altogether...

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-02-18 15:04

Message:
Logged In: YES 
user_id=45365

I agree on a theoretical level, but not on a practical one. The .rsrc files in the CVS repository are AppleSingle encoded, and I can't think of another way to keep them useable on both MacOS9 and MacOSX.

Any suggestions?

----------------------------------------------------------------------

Comment By: Just van Rossum (jvr)
Date: 2003-02-18 13:32

Message:
Logged In: YES 
user_id=92689

Yes, it must be compatible with 2.2 (it also ships with PyObjC). bundlebuilder.py was deliberately written to have very few dependencies: plistlib.py is the only dependency outside the std lib.

I'm not sure I like this patch to begin with: for one it seems it's doing this resource processing at the wrong abstraction level, it adds a special case for something that I don't like being treated specially within bundlebuilder. It would be much better if this preprocessing was done elsewhere.

In other words: I think the bug is in attempting to write the converted file to begin with, not with bundlebuilder not doing the conversion.

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-02-17 17:56

Message:
Logged In: YES 
user_id=45365

Fixed in bundlebuilder.py rev 1.8, macresource rev. 1.3.

Just, I'm assigning this to you because I'm not sure whether you still keep bundlebuilder 2.2-compatible. If so you may need to disable this bit of magic if macresource.install isn't available. Please close the report when you're happy.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=688007&group_id=5470


From noreply@sourceforge.net  Tue Feb 18 23:44:26 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 18 Feb 2003 15:44:26 -0800
Subject: [Python-bugs-list] [ python-Bugs-688907 ] Applet support is broken
Message-ID: <E18lHPW-0005pf-00@sc8-sf-web2.sourceforge.net>

Bugs item #688907, was opened at 2003-02-18 19:40
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=688907&group_id=5470

Category: Macintosh
Group: None
Status: Open
Resolution: None
>Priority: 5
Submitted By: Jack Jansen (jackjansen)
Assigned to: Jack Jansen (jackjansen)
Summary: Applet support is broken

Initial Comment:
Applet support is broken in two ways:
- there is no sys.argv emulation anymore, and applets always see the -psnXXXX argument when started from the finder.
- when an applet is started from the commandline it is non-functional (no window manager connection).

Also, the building of the Idle applet doesn't work due to name conflicts.

----------------------------------------------------------------------

>Comment By: Jack Jansen (jackjansen)
Date: 2003-02-19 00:44

Message:
Logged In: YES 
user_id=45365

sys.argv emulation works again, and that's the main part of this bug. A note has been added to NEWS that starting from the finder is broken in 2.3a2.

Building of Idle I cannot test right now.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=688907&group_id=5470


From noreply@sourceforge.net  Tue Feb 18 23:48:02 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 18 Feb 2003 15:48:02 -0800
Subject: [Python-bugs-list] [ python-Bugs-688907 ] Applet support is broken
Message-ID: <E18lHT0-0000mg-00@sc8-sf-web4.sourceforge.net>

Bugs item #688907, was opened at 2003-02-18 19:40
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=688907&group_id=5470

Category: Macintosh
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Jack Jansen (jackjansen)
>Assigned to: Just van Rossum (jvr)
Summary: Applet support is broken

Initial Comment:
Applet support is broken in two ways:
- there is no sys.argv emulation anymore, and applets always see the -psnXXXX argument when started from the finder.
- when an applet is started from the commandline it is non-functional (no window manager connection).

Also, the building of the Idle applet doesn't work due to name conflicts.

----------------------------------------------------------------------

>Comment By: Jack Jansen (jackjansen)
Date: 2003-02-19 00:48

Message:
Logged In: YES 
user_id=45365

Just thinks he can fix the commandline problem with a Python script, so assigning to him. Will you please also update Misc/NEWS if you do?

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-02-19 00:44

Message:
Logged In: YES 
user_id=45365

sys.argv emulation works again, and that's the main part of this bug. A note has been added to NEWS that starting from the finder is broken in 2.3a2.

Building of Idle I cannot test right now.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=688907&group_id=5470


From noreply@sourceforge.net  Wed Feb 19 00:42:23 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 18 Feb 2003 16:42:23 -0800
Subject: [Python-bugs-list] [ python-Bugs-683658 ] PyErr_Warn may cause import deadlock
Message-ID: <E18lIJb-0002fW-00@sc8-sf-web1.sourceforge.net>

Bugs item #683658, was opened at 2003-02-10 11:26
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=683658&group_id=5470

Category: Python Interpreter Core
Group: Python 2.3
Status: Open
>Resolution: Fixed
Priority: 6
Submitted By: Mark Hammond (mhammond)
>Assigned to: Guido van Rossum (gvanrossum)
Summary: PyErr_Warn may cause import deadlock

Initial Comment:
PyErr_Warn() does an implicit import.  Thus, if
PyErr_Warn() is called on a thread while the main
thread holds the import lock, and the main thread then
subsequently waits for the child thread, Python will
deadlock.  The builtin 'apply' now calls PyErr_Warn(),
so simply calling 'apply' on another thread may cause
the deadlock.

Attaching a sample case.  Executing 'python -c "import
hang_apply"' will cause Python to deadlock.  Commenting
out the call to "apply" in that file will prevent the
deadlock (even though the apply() in question is
effectively a no-op)

The example is a little contrived, but is extracted
from real code of mine that saw this hang.  The code
path is:

* Main thread acquires import lock to import the module.
* Module import spawns a new thread.  This new thread
calls apply()
* builtin_apply() calls PyErr_Warn
* PyErr_Warn does an import of the warnings module, and
attempts to acquire the import lock.
* Main thread still waiting for child thread to
complete, but still holds import lock.

Note that removing the call to PyErr_Warn() in
builtin_apply also solves this particular hang -
however, it seems like this is a potential time bomb. 
A potential solution would be to prevent PyErr_Warn
from doing an import - this would mean importing
'warnings' at startup, and keeping a static reference
in errors.c.  Other thoughts welcome.


----------------------------------------------------------------------

>Comment By: Mark Hammond (mhammond)
Date: 2003-02-19 11:42

Message:
Logged In: YES 
user_id=14198

Checking in a patch that avoids the import, thereby avoiding
the hang.  As suggested, the "linecache" import in warnings
had to be moved.

Even though I checked it in, I attach the patch here, and
have assigned it to Guido for review.  If happy, just mark
it as closed.

/cvsroot/python/python/dist/src/Python/errors.c,v  <--  errors.c
new revision: 2.76; previous revision: 2.75
/cvsroot/python/python/dist/src/Python/pythonrun.c,v  <-- 
pythonrun.c
new revision: 2.178; previous revision: 2.177
/cvsroot/python/python/dist/src/Lib/warnings.py,v  <-- 
warnings.py
new revision: 1.19; previous revision: 1.18


----------------------------------------------------------------------

Comment By: Mark Hammond (mhammond)
Date: 2003-02-16 10:17

Message:
Logged In: YES 
user_id=14198

Yes, the linecache import was the problem.  I abandonded
that approach mainly to avoid yet another import at startup.
 Should 'warnings' get new features, all required imports in
the future will also need to be imported at process startup.
 It also struck me as a time bomb.

I will make an alternative patch that moves the imports.


----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-16 08:01

Message:
Logged In: YES 
user_id=6380

Oh, and the import of linecache by warnings can be put at
the top of warnings, if that's an issue.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-16 02:46

Message:
Logged In: YES 
user_id=6380

No time to review everything here, but maybe PyErr_Warn
should not do an import? The import could be done at startup
time (when site.py is also imported) and saved in the
interpreter state.

----------------------------------------------------------------------

Comment By: Mark Hammond (mhammond)
Date: 2003-02-15 16:02

Message:
Logged In: YES 
user_id=14198

The best I can come up with here is exposing the import lock
to the C API, with a "wait" flag.

This will allow a C function to reliably determine if an
"import" work block, or acquire the lock if not.  It can
then complete its import before releasing the lock.  If the
import would block, then it can take whatever action is
necessary - in the case of PyErr_Warn, it dumps the warning
to stdout.

Attaching a patch.  It exposes (to the core, but not in
headers) two functions:

extern int PyImport_LockImport(int wait);
extern void PyImport_UnlockImport(void);

PyErr_Warn then uses these.

If we do go this route, it makes sense to make these
functions truly public (ie, add them to the headers), and
cleanup import.c appropriately.  I didn't do this in the
interests of keeping the patch small so it can be easily
digested.

Comments?  Other ideas?

----------------------------------------------------------------------

Comment By: Mark Hammond (mhammond)
Date: 2003-02-14 16:16

Message:
Logged In: YES 
user_id=14198

This simple strategy doesn't work - avoiding the import of
'warnings' works fine, until 'warnings' imports 'linecache'!

I'll look at how we can simply throw the warning away if a
deadlock would occur.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-12 10:23

Message:
Logged In: YES 
user_id=33168

I can't think of any other/better solution.  Any idea if
there are there other ticking bombs like this?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=683658&group_id=5470


From noreply@sourceforge.net  Wed Feb 19 10:49:47 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Wed, 19 Feb 2003 02:49:47 -0800
Subject: [Python-bugs-list] [ python-Bugs-554676 ] unknown locale de_DE@euro on Suse 8.0 Linux
Message-ID: <E18lRnP-00010y-00@sc8-sf-web4.sourceforge.net>

Bugs item #554676, was opened at 2002-05-10 23:02
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=554676&group_id=5470

Category: Python Library
Group: Python 2.2
Status: Closed
Resolution: Fixed
Priority: 5
Submitted By: vincent wehren (vinweh)
Assigned to: Martin v. Löwis (loewis)
Summary: unknown locale de_DE@euro on Suse 8.0 Linux

Initial Comment:
Python 2.2 (#1, Mar 26 2002, 15:46:04)   
[GCC 2.95.3 20010315 (SuSE)] on linux2   
 
When calling the locale module's getdefaultlocale() 
method on SuSe 8.0 Linux you get:   
 
>>> locale.getdefaultlocale()   
Traceback (most recent call last):   
  File "<stdin>", line 1, in ?   
  File "/usr/lib/python2.2/locale.py", line 337, in   
getdefaultlocale   
    return _parse_localename(localename)   
  File "/usr/lib/python2.2/locale.py", line 271, in   
_parse_localename   
    raise ValueError, 'unknown locale: %s' %   
localename   
ValueError: unknown locale: de_DE@euro   
  
Evidently, Python2.2's locale module is unaware of 
the "somelang_SOMELANG@euro" nomenclature  
for euro-enabled locales on Linux.  
  
 

----------------------------------------------------------------------

Comment By: Florian Konnertz (groovehunter)
Date: 2003-02-19 11:49

Message:
Logged In: YES 
user_id=548376

I'm newbie to the Python bug tracker so maybe i missed
something, but...

...isn't this issue still open for 2.1.3. 

See  http://openspirit.homelinux.net/noowiki/python/Locale
for my error report.

----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2002-11-03 18:24

Message:
Logged In: YES 
user_id=21627

This is now fixed in

locale.py 1.22 and 1.19.16.1;
test_locale.py 1.5;
NEWS 1.506;
liblocale.tex 1.31;

The @euro modifier will now imply Latin-9; as indicated,
this might be bogus, but is better than the current behaviour.

----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2002-11-03 09:02

Message:
Logged In: YES 
user_id=21627

There is a shallow bug, namely that

locale._parse_localename("de_DE@euro")

crashes; it should return the same value that it does for
de_DE, or perhaps it should have hard-coded knowledge that
the codeset is iso-8859-15.

There is a deeper underlying bug that getdefaultlocale is a
hopeless case. I'll be adding a locale.getpreferredencoding
function to make getdefaultlocale unnecessary.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2002-11-03 04:29

Message:
Logged In: YES 
user_id=33168

Martin, should this be closed?  Is there a bug?

----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2002-07-08 10:55

Message:
Logged In: YES 
user_id=21627

I see. For that, you should not use getdefaultlocale. The
reason is that getdefaultlocale cannot possibly determine
the locale's encoding correctly. Instead, you should use
locale.nl_langinfo where available (invoking setlocale
beforehand).

The fix you are reporting as 'easy' is a hack rather than a
solution: there is no guarantee whatsoever that the encoding
in a @euro locale will be Latin-9; it could just as well be,
say, UTF-8. Likewise, there might be other locales with
implied encodings.

----------------------------------------------------------------------

Comment By: Chema Cortés (chemacortes)
Date: 2002-07-06 03:11

Message:
Logged In: YES 
user_id=78289

We, as non-english writers, need 'getdefaultlocale' to set the default encoding for 
unicode strings: 
 
lang,encoding=locale.getdefaultlocale() 
sys.setdefaultencoding(encoding) 
 
The problem can be fixed easyly by insert the new locales into the locale_alias of 
module locale: 
 
locale_alias={ 
... 
  "de_de@euro": "de_DE.iso8859_15", 
  "de_at@euro": "de_AT@iso8859_15", 
  "es_es@euro":"es_ES@iso8859_15", 
... 
} 
 
As a workarround, you can modify the locale_alias into the sitecustomize.py 
 
# adding euro locales 
import locale 
eurolocs=[ "ca_ES", "da_DK", "de_AT", "de_BE", "de_DE", "de_LU", "en_BE", 
           "en_IE", "es_ES", "eu_ES", "fi_FI", "fr_BE", "fr_FR", "fr_LU", 
           "ga_IE", "gl_ES", "it_IT", "nl_BE", "nl_NL", "pt_PT", "sv_FI" 
] 
 
for l in eurolocs: 
   key=l.lower()+"@euro"          # eg: "es_es@euro" 
   cod=l+".iso8859_15"            # eg: "es_ES.iso8859_15" 
   locale.locale_alias[key]=cod 
 
# Setting the unicode default encoding 
import sys 
if hasattr(sys,"setdefaultencoding"): 
  lang,encoding=locale.getdefaultlocale() 
  sys.setdefaultencoding(encoding) 
 

----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2002-06-09 11:10

Message:
Logged In: YES 
user_id=21627

Can you please explain what you need getdefaultlocale for?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=554676&group_id=5470


From noreply@sourceforge.net  Wed Feb 19 16:36:29 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Wed, 19 Feb 2003 08:36:29 -0800
Subject: [Python-bugs-list] [ python-Bugs-683658 ] PyErr_Warn may cause import deadlock
Message-ID: <E18lXCv-0003UK-00@sc8-sf-web3.sourceforge.net>

Bugs item #683658, was opened at 2003-02-09 19:26
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=683658&group_id=5470

Category: Python Interpreter Core
Group: Python 2.3
Status: Open
>Resolution: Remind
Priority: 6
Submitted By: Mark Hammond (mhammond)
Assigned to: Guido van Rossum (gvanrossum)
Summary: PyErr_Warn may cause import deadlock

Initial Comment:
PyErr_Warn() does an implicit import.  Thus, if
PyErr_Warn() is called on a thread while the main
thread holds the import lock, and the main thread then
subsequently waits for the child thread, Python will
deadlock.  The builtin 'apply' now calls PyErr_Warn(),
so simply calling 'apply' on another thread may cause
the deadlock.

Attaching a sample case.  Executing 'python -c "import
hang_apply"' will cause Python to deadlock.  Commenting
out the call to "apply" in that file will prevent the
deadlock (even though the apply() in question is
effectively a no-op)

The example is a little contrived, but is extracted
from real code of mine that saw this hang.  The code
path is:

* Main thread acquires import lock to import the module.
* Module import spawns a new thread.  This new thread
calls apply()
* builtin_apply() calls PyErr_Warn
* PyErr_Warn does an import of the warnings module, and
attempts to acquire the import lock.
* Main thread still waiting for child thread to
complete, but still holds import lock.

Note that removing the call to PyErr_Warn() in
builtin_apply also solves this particular hang -
however, it seems like this is a potential time bomb. 
A potential solution would be to prevent PyErr_Warn
from doing an import - this would mean importing
'warnings' at startup, and keeping a static reference
in errors.c.  Other thoughts welcome.


----------------------------------------------------------------------

>Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-19 11:36

Message:
Logged In: YES 
user_id=6380

I'm not happy with this solution, since it breaks for
multiple interpreters. So keeping this open. Will think
about what to do instead after 2.3a2 is released.

----------------------------------------------------------------------

Comment By: Mark Hammond (mhammond)
Date: 2003-02-18 19:42

Message:
Logged In: YES 
user_id=14198

Checking in a patch that avoids the import, thereby avoiding
the hang.  As suggested, the "linecache" import in warnings
had to be moved.

Even though I checked it in, I attach the patch here, and
have assigned it to Guido for review.  If happy, just mark
it as closed.

/cvsroot/python/python/dist/src/Python/errors.c,v  <--  errors.c
new revision: 2.76; previous revision: 2.75
/cvsroot/python/python/dist/src/Python/pythonrun.c,v  <-- 
pythonrun.c
new revision: 2.178; previous revision: 2.177
/cvsroot/python/python/dist/src/Lib/warnings.py,v  <-- 
warnings.py
new revision: 1.19; previous revision: 1.18


----------------------------------------------------------------------

Comment By: Mark Hammond (mhammond)
Date: 2003-02-15 18:17

Message:
Logged In: YES 
user_id=14198

Yes, the linecache import was the problem.  I abandonded
that approach mainly to avoid yet another import at startup.
 Should 'warnings' get new features, all required imports in
the future will also need to be imported at process startup.
 It also struck me as a time bomb.

I will make an alternative patch that moves the imports.


----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-15 16:01

Message:
Logged In: YES 
user_id=6380

Oh, and the import of linecache by warnings can be put at
the top of warnings, if that's an issue.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-15 10:46

Message:
Logged In: YES 
user_id=6380

No time to review everything here, but maybe PyErr_Warn
should not do an import? The import could be done at startup
time (when site.py is also imported) and saved in the
interpreter state.

----------------------------------------------------------------------

Comment By: Mark Hammond (mhammond)
Date: 2003-02-15 00:02

Message:
Logged In: YES 
user_id=14198

The best I can come up with here is exposing the import lock
to the C API, with a "wait" flag.

This will allow a C function to reliably determine if an
"import" work block, or acquire the lock if not.  It can
then complete its import before releasing the lock.  If the
import would block, then it can take whatever action is
necessary - in the case of PyErr_Warn, it dumps the warning
to stdout.

Attaching a patch.  It exposes (to the core, but not in
headers) two functions:

extern int PyImport_LockImport(int wait);
extern void PyImport_UnlockImport(void);

PyErr_Warn then uses these.

If we do go this route, it makes sense to make these
functions truly public (ie, add them to the headers), and
cleanup import.c appropriately.  I didn't do this in the
interests of keeping the patch small so it can be easily
digested.

Comments?  Other ideas?

----------------------------------------------------------------------

Comment By: Mark Hammond (mhammond)
Date: 2003-02-14 00:16

Message:
Logged In: YES 
user_id=14198

This simple strategy doesn't work - avoiding the import of
'warnings' works fine, until 'warnings' imports 'linecache'!

I'll look at how we can simply throw the warning away if a
deadlock would occur.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-11 18:23

Message:
Logged In: YES 
user_id=33168

I can't think of any other/better solution.  Any idea if
there are there other ticking bombs like this?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=683658&group_id=5470


From noreply@sourceforge.net  Wed Feb 19 18:03:37 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Wed, 19 Feb 2003 10:03:37 -0800
Subject: [Python-bugs-list] [ python-Bugs-688266 ] IDLE does not work on Mac OS X
Message-ID: <E18lYZF-00074R-00@sc8-sf-web1.sourceforge.net>

Bugs item #688266, was opened at 2003-02-17 21:47
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=688266&group_id=5470

Category: IDLE
Group: Platform-specific
Status: Open
Resolution: None
Priority: 5
Submitted By: Tony Lownds (tonylownds)
Assigned to: Nobody/Anonymous (nobody)
Summary: IDLE does not work on Mac OS X

Initial Comment:
When building a framework build of python, the IDLE application that is made does not work anymore.

Here is the command that is causing trouble:

/Library/Frameworks/../../Library/Frameworks/Python.framework/Versions/2.3/Resources/Python.app/Contents/MacOS/python ../python/dist/src/Mac/scripts/BuildApplet.py --output /Library/Frameworks/../../Applications/MacPython-2.3/IDLE.app --extra ../python/dist/src/Tools/idle ../python/dist/src/Tools/idle/idle

or essentially:

python Mac/scripts/BuildApplet.py --output /Applications/MacPython-2.3/IDLE.app --extra Tools/idle Tools/idle/idle

The semantics of BuildApplet.py (well, buildtools.py) changed slightly, in a way that upset IDLE's build - it used to copy its argument (the main program file) into Resources/ under a new name - __main__.py. This changed so that the main program file keeps the same file name.

For IDLE, the main program is Tools/idle/idle but the Tools/idle directory needs to be copied into the bundle as well; before the slight semantic change in buildtools.py this worked OK but now two things named "idle" are getting copied to Resources/ and its not working.

Its hard to argue that BuildApplet really should support an --extra argument with the same basename as the main program file. Perhaps the best fix is to integrate the latest idlefork with Python 2.3 - this would eliminate this problem because the main file for mac applets is not called idle in idlefork.

Another way to fix this would be to extend BuildApplet and buildtools and bundlebuilder so that it is possible to specify a new name for the --extra argument, something like:

python Mac/scripts/BuildApplet.py --output IDLE.app --extra Tools/idle --rename-extra idlelib Tools/idle/idle


But the first fix option seems way more palateable.

----------------------------------------------------------------------

>Comment By: Tony Lownds (tonylownds)
Date: 2003-02-19 18:03

Message:
Logged In: YES 
user_id=24100

Updated macosx_main.py - since Jack added an argvemulation wrapper, macosx_main.py needs no special code and is the same as Tools/idle/idle

----------------------------------------------------------------------

Comment By: Tony Lownds (tonylownds)
Date: 2003-02-17 22:33

Message:
Logged In: YES 
user_id=24100

I think I have a found a good solution. Instead of using Tools/idle/idle as the main program argument to BuildApplet.py, the patch to Mac/OSX/Makefile below uses Tools/idle/macosx_main.py, which is attached separately.

The macosx_main.py file I provide is a copy of Tools/idle/idle with some code to work around another recent problem with running IDLE on Mac OS X. The new code removes some funny arguments that Mac OS X adds when GUI applications are launched. This same code can be found in Lib/plat-mac/argvemulator.py. I would have used argvemulator.py directly but I don't know if that would have other consequences. 

The patch to Mac/OSX/Makefile will be necessary when idlefork is integrated back into python. That is the reason for naming the new file macosx_main.py

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=688266&group_id=5470


From noreply@sourceforge.net  Wed Feb 19 21:23:08 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Wed, 19 Feb 2003 13:23:08 -0800
Subject: [Python-bugs-list] [ python-Bugs-688581 ] sys.stdout.flush() in regrtest
Message-ID: <E18lbgK-0001EH-00@sc8-sf-web3.sourceforge.net>

Bugs item #688581, was opened at 2003-02-18 11:45
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=688581&group_id=5470

Category: Python Library
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Jack Jansen (jackjansen)
Assigned to: Guido van Rossum (gvanrossum)
Summary: sys.stdout.flush() in regrtest

Initial Comment:
regrtest.py got some sys.stdout.flush() calls in rev. 1.96 (and fred added more in 1.122). But according to the documentation of the sys module stdout can be "any object that has a write method" and nothing is said about flush.

This bug is in 2.2.2 also (but not in 2.2.1), so a fix will have to be backported.

----------------------------------------------------------------------

Comment By: Stefan Petrucha (pet314)
Date: 2003-02-19 22:23

Message:
Logged In: YES 
user_id=716082

That's me who found this bug.

I'm embedding Python into a DTP program (on Win + Mac
platforms). The program is windowing and so std* do not work
(I was even forced to patch of the core library on Mac as
the distributed version does not work at all). To get
diagnostic output I must redirect sys.stdout and sys.stderr.
I've done it according to the documentation, but it seem not
to work as expected with the newest releases.

Regression tests are quite important to me too - I want to
make as much available from the Lib as possible.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-18 14:00

Message:
Logged In: YES 
user_id=6380

What's your use case? Since regrtest mostly controls its own
stdout, I'm not sure that the documented requirement applies.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=688581&group_id=5470


From noreply@sourceforge.net  Wed Feb 19 21:29:19 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Wed, 19 Feb 2003 13:29:19 -0800
Subject: [Python-bugs-list] [ python-Bugs-688581 ] sys.stdout.flush() in regrtest
Message-ID: <E18lbmJ-0000SM-00@sc8-sf-web1.sourceforge.net>

Bugs item #688581, was opened at 2003-02-18 05:45
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=688581&group_id=5470

Category: Python Library
Group: None
>Status: Closed
>Resolution: Wont Fix
Priority: 5
Submitted By: Jack Jansen (jackjansen)
Assigned to: Guido van Rossum (gvanrossum)
Summary: sys.stdout.flush() in regrtest

Initial Comment:
regrtest.py got some sys.stdout.flush() calls in rev. 1.96 (and fred added more in 1.122). But according to the documentation of the sys module stdout can be "any object that has a write method" and nothing is said about flush.

This bug is in 2.2.2 also (but not in 2.2.1), so a fix will have to be backported.

----------------------------------------------------------------------

>Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-19 16:29

Message:
Logged In: YES 
user_id=6380

I suggest that you add a flush() method to your dummy stdout
object. At best this is a documentation deficiency, but I'm
not even sure about that.

----------------------------------------------------------------------

Comment By: Stefan Petrucha (pet314)
Date: 2003-02-19 16:23

Message:
Logged In: YES 
user_id=716082

That's me who found this bug.

I'm embedding Python into a DTP program (on Win + Mac
platforms). The program is windowing and so std* do not work
(I was even forced to patch of the core library on Mac as
the distributed version does not work at all). To get
diagnostic output I must redirect sys.stdout and sys.stderr.
I've done it according to the documentation, but it seem not
to work as expected with the newest releases.

Regression tests are quite important to me too - I want to
make as much available from the Lib as possible.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-18 08:00

Message:
Logged In: YES 
user_id=6380

What's your use case? Since regrtest mostly controls its own
stdout, I'm not sure that the documented requirement applies.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=688581&group_id=5470


From noreply@sourceforge.net  Wed Feb 19 22:38:09 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Wed, 19 Feb 2003 14:38:09 -0800
Subject: [Python-bugs-list] [ python-Bugs-614555 ] Rewrite _reduce and _reconstructor in C
Message-ID: <E18lcqv-00019U-00@sc8-sf-web2.sourceforge.net>

Bugs item #614555, was opened at 2002-09-25 14:15
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=614555&group_id=5470

Category: Python Interpreter Core
Group: Python 2.3
Status: Open
Resolution: None
>Priority: 5
Submitted By: Guido van Rossum (gvanrossum)
>Assigned to: Guido van Rossum (gvanrossum)
Summary: Rewrite _reduce and _reconstructor in C

Initial Comment:
The copy_reg defines two functions, _reduce() and
_reconstructor(), that are used for the pickling and
unpickling of new-style classes. These were originally
written in Python because the right implementation
wasn't entirely clear; _reduce() is actually called
from a C-level __reduce__ method defined in 'object',
and it references _reconstructor() in its return tuple.
It is now time to move both back into C for efficiency.


----------------------------------------------------------------------

>Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-19 17:38

Message:
Logged In: YES 
user_id=6380

This isn't going to be done for 2.3a2; we'll revisit who
gets to do it closer to 2.3b1.

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2003-02-13 21:27

Message:
Logged In: YES 
user_id=31435

Except that the new Python code _better_reduce() is used 
then.  As discussed, I intend to rewrite _reduce and 
_better_reduce in C, inside object's __reduce__ 
implementation, so I assigned this to me and boosted the 
priority.  I don't intend to recode _reconstructor in C, 
though.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-12 17:19

Message:
Logged In: YES 
user_id=6380

Lowering priority; when you use pickle protocol 2, none of
this Python code is used any more.

Jim's comment has been addressed by __newobj__ and
__getnewargs__ (see PEP 307).

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2002-11-14 11:46

Message:
Logged In: YES 
user_id=6380

I'd like to add some comments from Jim Fulton on this (or a
related) issue so they don't get lost.

"""
I still need to think of a good way to handle this. 
Currently, the pattern is a reduce return value of the form:

   reconstructor, (some_class, object, None), some_state

but there is a more general case where an uninitialized
object can be gotten by calling "type.__new__(some_class)",
or even: "some_class.__new__(some_class)".

There are really two issues:

- It's too hard to write __reduce__ functions for this case, and

- Too much data needs to be stored in pickles.

A new pickling code would handler the later, but I also want
to solve the former problem.

One thought is to return: (copy_reg.new, (some_class, ),
some_state) where copy_reg.new is:

   def new(class_): return class.__new__(class_)

The pickler could easily spot reduce returns with
copy_reg.new as the first value and generate a special
pickle code.
"""


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=614555&group_id=5470


From noreply@sourceforge.net  Wed Feb 19 23:17:40 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Wed, 19 Feb 2003 15:17:40 -0800
Subject: [Python-bugs-list] [ python-Bugs-689659 ] 64-bit int and long hash keys incompatible
Message-ID: <E18ldTA-0003VD-00@sc8-sf-web4.sourceforge.net>

Bugs item #689659, was opened at 2003-02-19 23:17
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=689659&group_id=5470

Category: Python Interpreter Core
Group: Platform-specific
Status: Open
Resolution: None
Priority: 5
Submitted By: Mark Dickinson (marketdickinson)
Assigned to: Nobody/Anonymous (nobody)
Summary: 64-bit int and long hash keys incompatible

Initial Comment:
For 64-bit platforms, it seems that ints and longs with the 
same value don't necessarily give the same hash key.  
An example is below; the phenomenon seems to occur 
for a range of numbers between 2**32 and 2**64. 
 
I assume this is unintentional?  It looks suspiciously as 
though the function long_hash in Objects/longobject.c 
has 32-bitness hard-coded. Lines 1288-1289 of that file 
in the Python 2.2.2 source read: 
 
      /* Force a 32-bit circular shift */ 
      x = ((x << SHIFT) & ~MASK) | ((x >> (32-SHIFT)) & 
MASK); 
 
And here's the Python example: 
 
Python 2.2.2 (#22, Dec 23 2002, 12:02:55) 
[GCC 3.0.3] on sunos5 
Type "help", "copyright", "credits" or "license" for more 
information. 
>>> x = 4503599627370496L 
>>> y = 4503599627370496 
>>> h = {x: 'anything', y: 'something else'} 
>>> h[x] 
'anything' 
>>> h[y] 
'something else' 
>>> x == y 
1 
 
All the best, 
 
Mark 
 

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=689659&group_id=5470


From noreply@sourceforge.net  Wed Feb 19 23:29:45 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Wed, 19 Feb 2003 15:29:45 -0800
Subject: [Python-bugs-list] [ python-Bugs-689659 ] 64-bit int and long hash keys incompatible
Message-ID: <E18lder-0003wh-00@sc8-sf-web4.sourceforge.net>

Bugs item #689659, was opened at 2003-02-19 18:17
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=689659&group_id=5470

Category: Python Interpreter Core
Group: Platform-specific
Status: Open
Resolution: None
>Priority: 6
Submitted By: Mark Dickinson (marketdickinson)
>Assigned to: Tim Peters (tim_one)
Summary: 64-bit int and long hash keys incompatible

Initial Comment:
For 64-bit platforms, it seems that ints and longs with the 
same value don't necessarily give the same hash key.  
An example is below; the phenomenon seems to occur 
for a range of numbers between 2**32 and 2**64. 
 
I assume this is unintentional?  It looks suspiciously as 
though the function long_hash in Objects/longobject.c 
has 32-bitness hard-coded. Lines 1288-1289 of that file 
in the Python 2.2.2 source read: 
 
      /* Force a 32-bit circular shift */ 
      x = ((x << SHIFT) & ~MASK) | ((x >> (32-SHIFT)) & 
MASK); 
 
And here's the Python example: 
 
Python 2.2.2 (#22, Dec 23 2002, 12:02:55) 
[GCC 3.0.3] on sunos5 
Type "help", "copyright", "credits" or "license" for more 
information. 
>>> x = 4503599627370496L 
>>> y = 4503599627370496 
>>> h = {x: 'anything', y: 'something else'} 
>>> h[x] 
'anything' 
>>> h[y] 
'something else' 
>>> x == y 
1 
 
All the best, 
 
Mark 
 

----------------------------------------------------------------------

>Comment By: Tim Peters (tim_one)
Date: 2003-02-19 18:29

Message:
Logged In: YES 
user_id=31435

Good catch!  Agreed on all counts, and boosted priority 
because Python is trying to blur the distinction between 
ints and longs, so that this is likely to bite harder in the 
future.  Assigned to me, but a patch would be appreciated 
(don't know when I can make time for this, and I don't have 
a 64-bit box for testing).

It should be enough to replace "32" with "8*sizeof(long)".

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=689659&group_id=5470


From noreply@sourceforge.net  Thu Feb 20 03:45:13 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Wed, 19 Feb 2003 19:45:13 -0800
Subject: [Python-bugs-list] [ python-Bugs-689659 ] 64-bit int and long hash keys incompatible
Message-ID: <E18lhe5-0003Xl-00@sc8-sf-web2.sourceforge.net>

Bugs item #689659, was opened at 2003-02-19 18:17
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=689659&group_id=5470

Category: Python Interpreter Core
Group: Platform-specific
Status: Open
Resolution: None
Priority: 6
Submitted By: Mark Dickinson (marketdickinson)
>Assigned to: Neal Norwitz (nnorwitz)
Summary: 64-bit int and long hash keys incompatible

Initial Comment:
For 64-bit platforms, it seems that ints and longs with the 
same value don't necessarily give the same hash key.  
An example is below; the phenomenon seems to occur 
for a range of numbers between 2**32 and 2**64. 
 
I assume this is unintentional?  It looks suspiciously as 
though the function long_hash in Objects/longobject.c 
has 32-bitness hard-coded. Lines 1288-1289 of that file 
in the Python 2.2.2 source read: 
 
      /* Force a 32-bit circular shift */ 
      x = ((x << SHIFT) & ~MASK) | ((x >> (32-SHIFT)) & 
MASK); 
 
And here's the Python example: 
 
Python 2.2.2 (#22, Dec 23 2002, 12:02:55) 
[GCC 3.0.3] on sunos5 
Type "help", "copyright", "credits" or "license" for more 
information. 
>>> x = 4503599627370496L 
>>> y = 4503599627370496 
>>> h = {x: 'anything', y: 'something else'} 
>>> h[x] 
'anything' 
>>> h[y] 
'something else' 
>>> x == y 
1 
 
All the best, 
 
Mark 
 

----------------------------------------------------------------------

>Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-19 22:45

Message:
Logged In: YES 
user_id=33168

Even though Tim gave me the rest of the year off, I'll take
a look. :-)  BTW, we all have access to a 64-bit box on the
SF compile farm. :-)  Mark, it would be great if you are
able to create a patch, but if not, I'll try to fix it.

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2003-02-19 18:29

Message:
Logged In: YES 
user_id=31435

Good catch!  Agreed on all counts, and boosted priority 
because Python is trying to blur the distinction between 
ints and longs, so that this is likely to bite harder in the 
future.  Assigned to me, but a patch would be appreciated 
(don't know when I can make time for this, and I don't have 
a 64-bit box for testing).

It should be enough to replace "32" with "8*sizeof(long)".

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=689659&group_id=5470


From noreply@sourceforge.net  Thu Feb 20 04:41:45 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Wed, 19 Feb 2003 20:41:45 -0800
Subject: [Python-bugs-list] [ python-Bugs-689826 ] Docs page has no PEPs link
Message-ID: <E18liWn-0002N8-00@sc8-sf-web3.sourceforge.net>

Bugs item #689826, was opened at 2003-02-20 04:41
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=689826&group_id=5470

Category: Documentation
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: paul rubin (phr)
Assigned to: Nobody/Anonymous (nobody)
Summary: Docs page has no PEPs link

Initial Comment:
The www.python.org/doc page should have a link to the
PEP collection.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=689826&group_id=5470


From noreply@sourceforge.net  Thu Feb 20 08:39:01 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 20 Feb 2003 00:39:01 -0800
Subject: [Python-bugs-list] [ python-Bugs-689895 ] Imports can deadlock
Message-ID: <E18lmEP-0003ys-00@sc8-sf-web4.sourceforge.net>

Bugs item #689895, was opened at 2003-02-20 09:39
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=689895&group_id=5470

Category: Python Interpreter Core
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: M.-A. Lemburg (lemburg)
Assigned to: Nobody/Anonymous (nobody)
Summary: Imports can deadlock

Initial Comment:
[Just copying some discussion details from python-dev
into this
bug report so that we can reopen it to Python 2.4]

[MAL]

>> I wonder whether a general lock such as the one used
in import
>> is such a good idea. Perhaps it should only lock the
importing
>> of a specific module, keeping the locks in a
dictionary indexed by
>> module name instead of a static C variable ?!


I've been thinking along the same lines.  We could make
the import
locking much finer-grained, and limit the blocking only
to threads
that are importing a module that is in the middle of
being loaded by
some other thread.

But this is hard work, and I suggest that we put this
off until Python
2.4 so we can do it right.


>> Then again it's hard to know the real name of the
module being
>> searched before finding it...


There could be a short-lived lock for that problem.


>> I see a more general problem here: the lock prevent
starting
>> up threaded applications which use client-server
logic between
>> the threads. If the application's main thread starts
a client
>> thread as a result of an import which then tries to
import
>> other Python modules, you have a deadlock. (At least
that's how
>> I understand the current implementation.)


Correct.


>> Don't know about others, but I frequently use the
idiom of
>> placing the server's main code in a separate module
and then
>> have small startup script importing this module.
This kind
>> of setup is also advertised for CGI programs, so it
may not
>> be uncommon out there.


We had this problem with Zope2 -- I don't know why the
import lock
didn't bite us before, but we decided to change the
Zope startup code
so that you have to import Zope first and then,
separately, make a
call to start it.  You could do the same for your
application, and I
recommend that Mark does the same for his.

--Guido van Rossum (home page:
http://www.python.org/~guido/)



----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=689895&group_id=5470


From noreply@sourceforge.net  Thu Feb 20 08:40:27 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 20 Feb 2003 00:40:27 -0800
Subject: [Python-bugs-list] [ python-Bugs-689895 ] Imports can deadlock
Message-ID: <E18lmFn-00042Y-00@sc8-sf-web4.sourceforge.net>

Bugs item #689895, was opened at 2003-02-20 09:39
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=689895&group_id=5470

Category: Python Interpreter Core
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: M.-A. Lemburg (lemburg)
Assigned to: Nobody/Anonymous (nobody)
Summary: Imports can deadlock

Initial Comment:
[Just copying some discussion details from python-dev
into this
bug report so that we can reopen it to Python 2.4]

[MAL]

>> I wonder whether a general lock such as the one used
in import
>> is such a good idea. Perhaps it should only lock the
importing
>> of a specific module, keeping the locks in a
dictionary indexed by
>> module name instead of a static C variable ?!


I've been thinking along the same lines.  We could make
the import
locking much finer-grained, and limit the blocking only
to threads
that are importing a module that is in the middle of
being loaded by
some other thread.

But this is hard work, and I suggest that we put this
off until Python
2.4 so we can do it right.


>> Then again it's hard to know the real name of the
module being
>> searched before finding it...


There could be a short-lived lock for that problem.


>> I see a more general problem here: the lock prevent
starting
>> up threaded applications which use client-server
logic between
>> the threads. If the application's main thread starts
a client
>> thread as a result of an import which then tries to
import
>> other Python modules, you have a deadlock. (At least
that's how
>> I understand the current implementation.)


Correct.


>> Don't know about others, but I frequently use the
idiom of
>> placing the server's main code in a separate module
and then
>> have small startup script importing this module.
This kind
>> of setup is also advertised for CGI programs, so it
may not
>> be uncommon out there.


We had this problem with Zope2 -- I don't know why the
import lock
didn't bite us before, but we decided to change the
Zope startup code
so that you have to import Zope first and then,
separately, make a
call to start it.  You could do the same for your
application, and I
recommend that Mark does the same for his.

--Guido van Rossum (home page:
http://www.python.org/~guido/)



----------------------------------------------------------------------

>Comment By: M.-A. Lemburg (lemburg)
Date: 2003-02-20 09:40

Message:
Logged In: YES 
user_id=38388

[eric@enthought.com]

>[MAL]
>
>>> > I wonder whether a general lock such as the one used
in import
>>> > is such a good idea. Perhaps it should only lock the
importing
>>> > of a specific module, keeping the locks in a
dictionary indexed by
>>> > module name instead of a static C variable ?!


I am interested in this also.


>> 
>> [Guido]
>> I've been thinking along the same lines.  We could make
the import
>> locking much finer-grained, and limit the blocking only
to threads
>> that are importing a module that is in the middle of
being loaded by
>> some other thread.
>> 
>> But this is hard work, and I suggest that we put this off
until Python
>> 2.4 so we can do it right.


I ran into this exact thing when trying to get wxPython
windows (data
plots) to co-exist peacefully on the screen in parallel with an
interactive shell.  The module is called gui_thread, and we
use it in
SciPy.

http://www.scipy.org/site_content/tutorials/gui_thread
  
The command line took the main thread, and a wxPython app
was started in
the background thread.  The first import of wxPython had to
occur in the
background thread for wxPython's to be happy.  Initially, I
put a lock
in the import of gui_thread which waited for the wxPython
import to
complete before allowing the gui_thread import to complete.
 If 'import
gui_thread' is the first statement executed, this guarantees
that
wxPython is always imported first in the background thread.
Unfortunately, it also causes deadlock because of the import
lock.

A discussion that occurred about this problem on the
thread-sig with is
summarized here:

http://www.scipy.org/site_content/tutorials/import_thread_lock_discussio
n

It includes an alternative version of several import.c routines
(compared to 1.5.2 I believe) that solved my problem as
proposed by MAL
above and passed all the regression tests at the time.  I
just plugged
the code into the 2.3CVS, and it mainly works.  But, there
are some
import related errors in the regression tests for: 

test_threaded_import
test_loggingllbacks


I could spend some time on these if there is interest in
getting this in
the current release.  Also, this code would need review by
someone that
is an expert on the subtleties of the import code to make
sure it is
sound.  Let me know, and I'll submit a patch with of the
current code.

eric


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=689895&group_id=5470


From noreply@sourceforge.net  Thu Feb 20 12:36:57 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 20 Feb 2003 04:36:57 -0800
Subject: [Python-bugs-list] [ python-Bugs-690012 ] 2.3a2 build fails under IRIX 6.5
Message-ID: <E18lpwf-0006Fo-00@sc8-sf-web2.sourceforge.net>

Bugs item #690012, was opened at 2003-02-20 04:36
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690012&group_id=5470

Category: Build
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Ralf W. Grosse-Kunstleve (rwgk)
Assigned to: Nobody/Anonymous (nobody)
Summary: 2.3a2 build fails under IRIX 6.5

Initial Comment:
Using the latest Python 2.3a2 release
./configure; make
fails on this platform:
IRIX rattler 6.5 10120734 IP32
See attached logs.

I've also tried configure --disable-ipv6, but the result is 
the same.

The previous 2.3a1 release works without a problem 
on the exact same machine.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690012&group_id=5470


From noreply@sourceforge.net  Thu Feb 20 15:06:31 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 20 Feb 2003 07:06:31 -0800
Subject: [Python-bugs-list] [ python-Bugs-690081 ] test_posix fails when run in non-interactive mode
Message-ID: <E18lsHP-00083m-00@sc8-sf-web1.sourceforge.net>

Bugs item #690081, was opened at 2003-02-20 16:06
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690081&group_id=5470

Category: Python Library
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: M.-A. Lemburg (lemburg)
Assigned to: Nobody/Anonymous (nobody)
Summary: test_posix fails when run in non-interactive mode

Initial Comment:
>> My nightly cronjob has issued two warnings for the
CVS checkout
>> at 02:00 CET today:
>>
>> test test_posix failed -- Traceback (most recent
call last):
>>   File
"/home/lemburg/projects/Python/Dev-Python/Lib/test/test_posix.py",
line 39, in testNoArgFunctions
>>     posix_func()
>> OSError: [Errno 25] Inappropriate ioctl for device
>>
>> The traceback is not very helpful I'm afraid and
when I run the
>> test by hand I can't reproduce it.
>
>
> Try running the test from a cron job without regrtest
(i.e. ./python
> Lib/test/test_posix.py), and putting some extra
prints in it.
>
> Some of the functions tested (e.g. ctermid, getlogin)
use stdin to
> determine who is logged in.  Maybe these should just
be ignored if
> they fail with this specific error (since the error
means they're
> working as advertised).


Ok, here's the output:

testNoArgFunctions (__main__.PosixTester) ... Running
function ctermid
Running function getcwd
Running function getcwdu
Running function uname
Running function times
Running function getlogin
   ERROR
Running function getloadavg
Running function tmpnam
Running function getegid
Running function geteuid
Running function getgid
Running function getgroups
Running function getpid
Running function getpgrp
Running function getppid
Running function getuid
ok

Looks like getlogin doesn't like being run in
non-interactive
mode (ie. without a terminal).



----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690081&group_id=5470


From noreply@sourceforge.net  Thu Feb 20 15:07:03 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 20 Feb 2003 07:07:03 -0800
Subject: [Python-bugs-list] [ python-Bugs-690083 ] test_random fails sometimes
Message-ID: <E18lsHv-0005Co-00@sc8-sf-web2.sourceforge.net>

Bugs item #690083, was opened at 2003-02-20 16:07
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690083&group_id=5470

Category: Python Library
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: M.-A. Lemburg (lemburg)
Assigned to: Nobody/Anonymous (nobody)
Summary: test_random fails sometimes

Initial Comment:
>> test test_random failed -- Traceback (most recent
call last):
>>   File
"/home/lemburg/projects/Python/Dev-Python/Lib/test/test_random.py",
line 25, in test_autoseed
>>     self.assertNotEqual(state1, state2)
>>   File
"/home/lemburg/projects/Python/Dev-Python/Lib/unittest.py",
line 300, in failIfEqual
>>     raise self.failureException, \
>> AssertionError: (2, (1045703145, 318847470,
620640488, 320379787, -1732263717, -1278593374, ...
>
>
>
> This one I don't understand.  Maybe the sleep(1) in
the test isn't
> long enough?


I could not reproduce this, but perhaps using
long(time.time()*256) isn't
safe enough to produce different states in the
WH-generator ?! I'd
add a counter or hash the previous stored state to add some
more noise.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690083&group_id=5470


From noreply@sourceforge.net  Thu Feb 20 15:47:00 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 20 Feb 2003 07:47:00 -0800
Subject: [Python-bugs-list] [ python-Bugs-690109 ] sys.last_type is missing
Message-ID: <E18lsua-0001by-00@sc8-sf-web3.sourceforge.net>

Bugs item #690109, was opened at 2003-02-20 16:46
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690109&group_id=5470

Category: Python Library
Group: Python 2.2.1
Status: Open
Resolution: None
Priority: 5
Submitted By: Viktor Ferenczi (complex)
Assigned to: Nobody/Anonymous (nobody)
Summary: sys.last_type is missing

Initial Comment:
ActivePython 2.2.1 Build 222 (ActiveState Corp.) based on
Python 2.2.1 (#34, Apr 15 2002, 09:51:39) [MSC 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from traceback import print_last
>>> try: dffddf
... except: print_last()
...
Traceback (most recent call last):
  File "<stdin>", line 2, in ?
  File "C:\Python\lib\traceback.py", line 219, in print_last
    print_exception(sys.last_type, sys.last_value, sys.last_traceback,
AttributeError: 'module' object has no attribute 'last_type'
>>>

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690109&group_id=5470


From noreply@sourceforge.net  Thu Feb 20 16:08:31 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 20 Feb 2003 08:08:31 -0800
Subject: [Python-bugs-list] [ python-Bugs-690083 ] test_random fails sometimes
Message-ID: <E18ltFP-0002lS-00@sc8-sf-web3.sourceforge.net>

Bugs item #690083, was opened at 2003-02-20 10:07
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690083&group_id=5470

Category: Python Library
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: M.-A. Lemburg (lemburg)
>Assigned to: Raymond Hettinger (rhettinger)
Summary: test_random fails sometimes

Initial Comment:
>> test test_random failed -- Traceback (most recent
call last):
>>   File
"/home/lemburg/projects/Python/Dev-Python/Lib/test/test_random.py",
line 25, in test_autoseed
>>     self.assertNotEqual(state1, state2)
>>   File
"/home/lemburg/projects/Python/Dev-Python/Lib/unittest.py",
line 300, in failIfEqual
>>     raise self.failureException, \
>> AssertionError: (2, (1045703145, 318847470,
620640488, 320379787, -1732263717, -1278593374, ...
>
>
>
> This one I don't understand.  Maybe the sleep(1) in
the test isn't
> long enough?


I could not reproduce this, but perhaps using
long(time.time()*256) isn't
safe enough to produce different states in the
WH-generator ?! I'd
add a counter or hash the previous stored state to add some
more noise.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690083&group_id=5470


From noreply@sourceforge.net  Thu Feb 20 16:38:16 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 20 Feb 2003 08:38:16 -0800
Subject: [Python-bugs-list] [ python-Bugs-690083 ] test_random fails sometimes
Message-ID: <E18ltiC-0008S6-00@sc8-sf-web4.sourceforge.net>

Bugs item #690083, was opened at 2003-02-20 10:07
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690083&group_id=5470

Category: Python Library
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: M.-A. Lemburg (lemburg)
Assigned to: Raymond Hettinger (rhettinger)
Summary: test_random fails sometimes

Initial Comment:
>> test test_random failed -- Traceback (most recent
call last):
>>   File
"/home/lemburg/projects/Python/Dev-Python/Lib/test/test_random.py",
line 25, in test_autoseed
>>     self.assertNotEqual(state1, state2)
>>   File
"/home/lemburg/projects/Python/Dev-Python/Lib/unittest.py",
line 300, in failIfEqual
>>     raise self.failureException, \
>> AssertionError: (2, (1045703145, 318847470,
620640488, 320379787, -1732263717, -1278593374, ...
>
>
>
> This one I don't understand.  Maybe the sleep(1) in
the test isn't
> long enough?


I could not reproduce this, but perhaps using
long(time.time()*256) isn't
safe enough to produce different states in the
WH-generator ?! I'd
add a counter or hash the previous stored state to add some
more noise.


----------------------------------------------------------------------

>Comment By: Tim Peters (tim_one)
Date: 2003-02-20 11:38

Message:
Logged In: YES 
user_id=31435

MAL, do you know that this failed during the WH part of the 
test?  The WH seed() guarantees to produce distinct states 
given distinct inputs, and time.time() all by itself should be 
enough so that it's significantly different between time.sleep
(1) calls.  Multiplying by 256 is just gravy, getting some 
benefit from the fractional seconds too.

The Twister's idea of time-based seeding uses the C time() 
function, though, and that's less robust because it doesn't 
produce "fractional seconds" (as time.time() produces, as 
well as it can).  But it should still be at least a second 
different, unless your system clock got fiddled between calls 
(so, e.g., do you use some web-based time synch service to 
keep your clock up to date?).

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690083&group_id=5470


From noreply@sourceforge.net  Thu Feb 20 17:22:50 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 20 Feb 2003 09:22:50 -0800
Subject: [Python-bugs-list] [ python-Bugs-690083 ] test_random fails sometimes
Message-ID: <E18luPK-00040d-00@sc8-sf-web2.sourceforge.net>

Bugs item #690083, was opened at 2003-02-20 16:07
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690083&group_id=5470

Category: Python Library
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: M.-A. Lemburg (lemburg)
Assigned to: Raymond Hettinger (rhettinger)
Summary: test_random fails sometimes

Initial Comment:
>> test test_random failed -- Traceback (most recent
call last):
>>   File
"/home/lemburg/projects/Python/Dev-Python/Lib/test/test_random.py",
line 25, in test_autoseed
>>     self.assertNotEqual(state1, state2)
>>   File
"/home/lemburg/projects/Python/Dev-Python/Lib/unittest.py",
line 300, in failIfEqual
>>     raise self.failureException, \
>> AssertionError: (2, (1045703145, 318847470,
620640488, 320379787, -1732263717, -1278593374, ...
>
>
>
> This one I don't understand.  Maybe the sleep(1) in
the test isn't
> long enough?


I could not reproduce this, but perhaps using
long(time.time()*256) isn't
safe enough to produce different states in the
WH-generator ?! I'd
add a counter or hash the previous stored state to add some
more noise.


----------------------------------------------------------------------

>Comment By: M.-A. Lemburg (lemburg)
Date: 2003-02-20 18:22

Message:
Logged In: YES 
user_id=38388

Tim, no I'm not sure that it's the WH seed method causing
this. I was basically just guessing here from looking at the
code in random.py.

You could be right in that the time() C function used in _random
is causing the problem.

I'll check with a time.sleep(0.01) call instead of
time.sleep(1).

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2003-02-20 17:38

Message:
Logged In: YES 
user_id=31435

MAL, do you know that this failed during the WH part of the 
test?  The WH seed() guarantees to produce distinct states 
given distinct inputs, and time.time() all by itself should be 
enough so that it's significantly different between time.sleep
(1) calls.  Multiplying by 256 is just gravy, getting some 
benefit from the fractional seconds too.

The Twister's idea of time-based seeding uses the C time() 
function, though, and that's less robust because it doesn't 
produce "fractional seconds" (as time.time() produces, as 
well as it can).  But it should still be at least a second 
different, unless your system clock got fiddled between calls 
(so, e.g., do you use some web-based time synch service to 
keep your clock up to date?).

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690083&group_id=5470


From noreply@sourceforge.net  Thu Feb 20 17:27:53 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 20 Feb 2003 09:27:53 -0800
Subject: [Python-bugs-list] [ python-Bugs-690083 ] test_random fails sometimes
Message-ID: <E18luUD-0001uI-00@sc8-sf-web4.sourceforge.net>

Bugs item #690083, was opened at 2003-02-20 16:07
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690083&group_id=5470

Category: Python Library
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: M.-A. Lemburg (lemburg)
Assigned to: Raymond Hettinger (rhettinger)
Summary: test_random fails sometimes

Initial Comment:
>> test test_random failed -- Traceback (most recent
call last):
>>   File
"/home/lemburg/projects/Python/Dev-Python/Lib/test/test_random.py",
line 25, in test_autoseed
>>     self.assertNotEqual(state1, state2)
>>   File
"/home/lemburg/projects/Python/Dev-Python/Lib/unittest.py",
line 300, in failIfEqual
>>     raise self.failureException, \
>> AssertionError: (2, (1045703145, 318847470,
620640488, 320379787, -1732263717, -1278593374, ...
>
>
>
> This one I don't understand.  Maybe the sleep(1) in
the test isn't
> long enough?


I could not reproduce this, but perhaps using
long(time.time()*256) isn't
safe enough to produce different states in the
WH-generator ?! I'd
add a counter or hash the previous stored state to add some
more noise.


----------------------------------------------------------------------

>Comment By: M.-A. Lemburg (lemburg)
Date: 2003-02-20 18:27

Message:
Logged In: YES 
user_id=38388

Using time.sleep(0.01) causes the test to always fail on my
machine
and instrumenting the code with a print reveals that it's indeed
the _random code failing:

test_autoseed (__main__.MersenneTwister_TestBasicOps) ...
Running class <class '__main__.MersenneTwister_TestBasicOps'>
FAIL


----------------------------------------------------------------------

Comment By: M.-A. Lemburg (lemburg)
Date: 2003-02-20 18:22

Message:
Logged In: YES 
user_id=38388

Tim, no I'm not sure that it's the WH seed method causing
this. I was basically just guessing here from looking at the
code in random.py.

You could be right in that the time() C function used in _random
is causing the problem.

I'll check with a time.sleep(0.01) call instead of
time.sleep(1).

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2003-02-20 17:38

Message:
Logged In: YES 
user_id=31435

MAL, do you know that this failed during the WH part of the 
test?  The WH seed() guarantees to produce distinct states 
given distinct inputs, and time.time() all by itself should be 
enough so that it's significantly different between time.sleep
(1) calls.  Multiplying by 256 is just gravy, getting some 
benefit from the fractional seconds too.

The Twister's idea of time-based seeding uses the C time() 
function, though, and that's less robust because it doesn't 
produce "fractional seconds" (as time.time() produces, as 
well as it can).  But it should still be at least a second 
different, unless your system clock got fiddled between calls 
(so, e.g., do you use some web-based time synch service to 
keep your clock up to date?).

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690083&group_id=5470


From noreply@sourceforge.net  Thu Feb 20 17:32:24 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 20 Feb 2003 09:32:24 -0800
Subject: [Python-bugs-list] [ python-Bugs-690083 ] test_random fails sometimes
Message-ID: <E18luYa-00073G-00@sc8-sf-web3.sourceforge.net>

Bugs item #690083, was opened at 2003-02-20 10:07
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690083&group_id=5470

Category: Python Library
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: M.-A. Lemburg (lemburg)
Assigned to: Raymond Hettinger (rhettinger)
Summary: test_random fails sometimes

Initial Comment:
>> test test_random failed -- Traceback (most recent
call last):
>>   File
"/home/lemburg/projects/Python/Dev-Python/Lib/test/test_random.py",
line 25, in test_autoseed
>>     self.assertNotEqual(state1, state2)
>>   File
"/home/lemburg/projects/Python/Dev-Python/Lib/unittest.py",
line 300, in failIfEqual
>>     raise self.failureException, \
>> AssertionError: (2, (1045703145, 318847470,
620640488, 320379787, -1732263717, -1278593374, ...
>
>
>
> This one I don't understand.  Maybe the sleep(1) in
the test isn't
> long enough?


I could not reproduce this, but perhaps using
long(time.time()*256) isn't
safe enough to produce different states in the
WH-generator ?! I'd
add a counter or hash the previous stored state to add some
more noise.


----------------------------------------------------------------------

>Comment By: Tim Peters (tim_one)
Date: 2003-02-20 12:32

Message:
Logged In: YES 
user_id=31435

I expect that time.sleep(0.01) would cause the Twister test to 
fail (on average) 99 times per 100.

For some reason we don't understand then, two calls to C 
time() returned the same value despite an intervening Python 
time.sleep(1) call.  Has there been a leap second lately 
<wink>?

----------------------------------------------------------------------

Comment By: M.-A. Lemburg (lemburg)
Date: 2003-02-20 12:27

Message:
Logged In: YES 
user_id=38388

Using time.sleep(0.01) causes the test to always fail on my
machine
and instrumenting the code with a print reveals that it's indeed
the _random code failing:

test_autoseed (__main__.MersenneTwister_TestBasicOps) ...
Running class <class '__main__.MersenneTwister_TestBasicOps'>
FAIL


----------------------------------------------------------------------

Comment By: M.-A. Lemburg (lemburg)
Date: 2003-02-20 12:22

Message:
Logged In: YES 
user_id=38388

Tim, no I'm not sure that it's the WH seed method causing
this. I was basically just guessing here from looking at the
code in random.py.

You could be right in that the time() C function used in _random
is causing the problem.

I'll check with a time.sleep(0.01) call instead of
time.sleep(1).

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2003-02-20 11:38

Message:
Logged In: YES 
user_id=31435

MAL, do you know that this failed during the WH part of the 
test?  The WH seed() guarantees to produce distinct states 
given distinct inputs, and time.time() all by itself should be 
enough so that it's significantly different between time.sleep
(1) calls.  Multiplying by 256 is just gravy, getting some 
benefit from the fractional seconds too.

The Twister's idea of time-based seeding uses the C time() 
function, though, and that's less robust because it doesn't 
produce "fractional seconds" (as time.time() produces, as 
well as it can).  But it should still be at least a second 
different, unless your system clock got fiddled between calls 
(so, e.g., do you use some web-based time synch service to 
keep your clock up to date?).

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690083&group_id=5470


From noreply@sourceforge.net  Thu Feb 20 17:58:07 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 20 Feb 2003 09:58:07 -0800
Subject: [Python-bugs-list] [ python-Bugs-690083 ] test_random fails sometimes
Message-ID: <E18luxT-0000GD-00@sc8-sf-web1.sourceforge.net>

Bugs item #690083, was opened at 2003-02-20 16:07
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690083&group_id=5470

Category: Python Library
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: M.-A. Lemburg (lemburg)
Assigned to: Raymond Hettinger (rhettinger)
Summary: test_random fails sometimes

Initial Comment:
>> test test_random failed -- Traceback (most recent
call last):
>>   File
"/home/lemburg/projects/Python/Dev-Python/Lib/test/test_random.py",
line 25, in test_autoseed
>>     self.assertNotEqual(state1, state2)
>>   File
"/home/lemburg/projects/Python/Dev-Python/Lib/unittest.py",
line 300, in failIfEqual
>>     raise self.failureException, \
>> AssertionError: (2, (1045703145, 318847470,
620640488, 320379787, -1732263717, -1278593374, ...
>
>
>
> This one I don't understand.  Maybe the sleep(1) in
the test isn't
> long enough?


I could not reproduce this, but perhaps using
long(time.time()*256) isn't
safe enough to produce different states in the
WH-generator ?! I'd
add a counter or hash the previous stored state to add some
more noise.


----------------------------------------------------------------------

>Comment By: M.-A. Lemburg (lemburg)
Date: 2003-02-20 18:58

Message:
Logged In: YES 
user_id=38388

Seconds are longer in Old Europe :-)

Seriously, I don't know how time.sleep() is implemented by the
C lib or the kernel. Could be that time.sleep(1) sometimes only
waits 0.99 seconds or that C libs time() uses some counter which
isn't always updated in exact one second intervals.

I'd suggest to use the same clock source used in the time module
for better accuracy rather than relying on _random's seed() to
provide it's own C lib time() based one.

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2003-02-20 18:32

Message:
Logged In: YES 
user_id=31435

I expect that time.sleep(0.01) would cause the Twister test to 
fail (on average) 99 times per 100.

For some reason we don't understand then, two calls to C 
time() returned the same value despite an intervening Python 
time.sleep(1) call.  Has there been a leap second lately 
<wink>?

----------------------------------------------------------------------

Comment By: M.-A. Lemburg (lemburg)
Date: 2003-02-20 18:27

Message:
Logged In: YES 
user_id=38388

Using time.sleep(0.01) causes the test to always fail on my
machine
and instrumenting the code with a print reveals that it's indeed
the _random code failing:

test_autoseed (__main__.MersenneTwister_TestBasicOps) ...
Running class <class '__main__.MersenneTwister_TestBasicOps'>
FAIL


----------------------------------------------------------------------

Comment By: M.-A. Lemburg (lemburg)
Date: 2003-02-20 18:22

Message:
Logged In: YES 
user_id=38388

Tim, no I'm not sure that it's the WH seed method causing
this. I was basically just guessing here from looking at the
code in random.py.

You could be right in that the time() C function used in _random
is causing the problem.

I'll check with a time.sleep(0.01) call instead of
time.sleep(1).

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2003-02-20 17:38

Message:
Logged In: YES 
user_id=31435

MAL, do you know that this failed during the WH part of the 
test?  The WH seed() guarantees to produce distinct states 
given distinct inputs, and time.time() all by itself should be 
enough so that it's significantly different between time.sleep
(1) calls.  Multiplying by 256 is just gravy, getting some 
benefit from the fractional seconds too.

The Twister's idea of time-based seeding uses the C time() 
function, though, and that's less robust because it doesn't 
produce "fractional seconds" (as time.time() produces, as 
well as it can).  But it should still be at least a second 
different, unless your system clock got fiddled between calls 
(so, e.g., do you use some web-based time synch service to 
keep your clock up to date?).

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690083&group_id=5470


From noreply@sourceforge.net  Thu Feb 20 18:50:42 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 20 Feb 2003 10:50:42 -0800
Subject: [Python-bugs-list] [ python-Bugs-690083 ] test_random fails sometimes
Message-ID: <E18lvmM-0005Ux-00@sc8-sf-web4.sourceforge.net>

Bugs item #690083, was opened at 2003-02-20 10:07
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690083&group_id=5470

Category: Python Library
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: M.-A. Lemburg (lemburg)
Assigned to: Raymond Hettinger (rhettinger)
Summary: test_random fails sometimes

Initial Comment:
>> test test_random failed -- Traceback (most recent
call last):
>>   File
"/home/lemburg/projects/Python/Dev-Python/Lib/test/test_random.py",
line 25, in test_autoseed
>>     self.assertNotEqual(state1, state2)
>>   File
"/home/lemburg/projects/Python/Dev-Python/Lib/unittest.py",
line 300, in failIfEqual
>>     raise self.failureException, \
>> AssertionError: (2, (1045703145, 318847470,
620640488, 320379787, -1732263717, -1278593374, ...
>
>
>
> This one I don't understand.  Maybe the sleep(1) in
the test isn't
> long enough?


I could not reproduce this, but perhaps using
long(time.time()*256) isn't
safe enough to produce different states in the
WH-generator ?! I'd
add a counter or hash the previous stored state to add some
more noise.


----------------------------------------------------------------------

>Comment By: Tim Peters (tim_one)
Date: 2003-02-20 13:50

Message:
Logged In: YES 
user_id=31435

datetimemodule.c contains a little routine time_time(), which 
imports the time module and calls time.time(), returning the 
result as a Python float.  As the comment above it says, 
maybe it would be better to make this part of Python's C API 
(time.time() plays a lot of platform-specific tricks to get better-
than-second resolution when possible).

OTOH, sticking time.sleep(1.1) in the test case would 
probably fix it too.

----------------------------------------------------------------------

Comment By: M.-A. Lemburg (lemburg)
Date: 2003-02-20 12:58

Message:
Logged In: YES 
user_id=38388

Seconds are longer in Old Europe :-)

Seriously, I don't know how time.sleep() is implemented by the
C lib or the kernel. Could be that time.sleep(1) sometimes only
waits 0.99 seconds or that C libs time() uses some counter which
isn't always updated in exact one second intervals.

I'd suggest to use the same clock source used in the time module
for better accuracy rather than relying on _random's seed() to
provide it's own C lib time() based one.

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2003-02-20 12:32

Message:
Logged In: YES 
user_id=31435

I expect that time.sleep(0.01) would cause the Twister test to 
fail (on average) 99 times per 100.

For some reason we don't understand then, two calls to C 
time() returned the same value despite an intervening Python 
time.sleep(1) call.  Has there been a leap second lately 
<wink>?

----------------------------------------------------------------------

Comment By: M.-A. Lemburg (lemburg)
Date: 2003-02-20 12:27

Message:
Logged In: YES 
user_id=38388

Using time.sleep(0.01) causes the test to always fail on my
machine
and instrumenting the code with a print reveals that it's indeed
the _random code failing:

test_autoseed (__main__.MersenneTwister_TestBasicOps) ...
Running class <class '__main__.MersenneTwister_TestBasicOps'>
FAIL


----------------------------------------------------------------------

Comment By: M.-A. Lemburg (lemburg)
Date: 2003-02-20 12:22

Message:
Logged In: YES 
user_id=38388

Tim, no I'm not sure that it's the WH seed method causing
this. I was basically just guessing here from looking at the
code in random.py.

You could be right in that the time() C function used in _random
is causing the problem.

I'll check with a time.sleep(0.01) call instead of
time.sleep(1).

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2003-02-20 11:38

Message:
Logged In: YES 
user_id=31435

MAL, do you know that this failed during the WH part of the 
test?  The WH seed() guarantees to produce distinct states 
given distinct inputs, and time.time() all by itself should be 
enough so that it's significantly different between time.sleep
(1) calls.  Multiplying by 256 is just gravy, getting some 
benefit from the fractional seconds too.

The Twister's idea of time-based seeding uses the C time() 
function, though, and that's less robust because it doesn't 
produce "fractional seconds" (as time.time() produces, as 
well as it can).  But it should still be at least a second 
different, unless your system clock got fiddled between calls 
(so, e.g., do you use some web-based time synch service to 
keep your clock up to date?).

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690083&group_id=5470


From noreply@sourceforge.net  Thu Feb 20 18:55:14 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 20 Feb 2003 10:55:14 -0800
Subject: [Python-bugs-list] [ python-Bugs-690214 ] robotparser only applies first applicable rule
Message-ID: <E18lvqk-0002mc-00@sc8-sf-web1.sourceforge.net>

Bugs item #690214, was opened at 2003-02-20 13:55
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690214&group_id=5470

Category: Python Library
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Mark Pilgrim (f8dy)
Assigned to: Nobody/Anonymous (nobody)
Summary: robotparser only applies first applicable rule

Initial Comment:
robotparser robotparser.py::RobotFileParser::can_fetch 
currently returns the result of the first applicable rule.  It 
should loop through all rules looking for anything that 
disallows access.  For example, if your first rule applies 
to 'wget' and 'python' and disallows access to /dir1/, and 
your second rule is a 'python' rule that disallows access 
to /dir2/, robotparser will falsely claim that python is 
allowed to access /dir2/.

Patch against current source attached.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690214&group_id=5470


From noreply@sourceforge.net  Thu Feb 20 19:33:29 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 20 Feb 2003 11:33:29 -0800
Subject: [Python-bugs-list] [ python-Bugs-667931 ] BoundaryError: multipart message with no defined boundary
Message-ID: <E18lwRl-0004hp-00@sc8-sf-web1.sourceforge.net>

Bugs item #667931, was opened at 2003-01-14 11:36
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=667931&group_id=5470

Category: Python Library
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Jason R. Mastaler (jasonrm)
Assigned to: Barry A. Warsaw (bwarsaw)
Summary: BoundaryError: multipart message with no defined boundary

Initial Comment:
More problems with the email package raising
exceptions when trying to parse non-compliant
messages.

Even when lax parsing is enabled, a BoundaryError
is raised when trying to parse the attached spam
message.

I'd like to see some sort of workaround to handle these
cases more gracefully when when lax parsing is
enabled.

This behavior seems like 'strict' parsing behavior to me.



----------------------------------------------------------------------

>Comment By: Jason R. Mastaler (jasonrm)
Date: 2003-02-20 12:33

Message:
Logged In: YES 
user_id=85984

As Python 2.3a2 was just released, I'm worried that
this one is going to fall through the cracks before
2.3-final is released which would be unfortunate because
it would mean I'd have to continue bundling my own
copy of email with my applications.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=667931&group_id=5470


From noreply@sourceforge.net  Thu Feb 20 19:33:55 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 20 Feb 2003 11:33:55 -0800
Subject: [Python-bugs-list] [ python-Bugs-633550 ] HeaderParseError: no header value
Message-ID: <E18lwSB-0004j5-00@sc8-sf-web1.sourceforge.net>

Bugs item #633550, was opened at 2002-11-04 16:55
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=633550&group_id=5470

Category: Python Library
Group: Python 2.2.2
Status: Open
Resolution: None
Priority: 5
Submitted By: Jason R. Mastaler (jasonrm)
Assigned to: Barry A. Warsaw (bwarsaw)
Summary: HeaderParseError: no header value

Initial Comment:
We all know according to rfc 2822, that a header
field should be composed of name: value, but
sometimes (as in the case of SPAM), this is not
the case.

In email 2.4.3, if a header field lacks a value portion
a HeaderParseError is raised.

For an example message and traceback, see
http://article.gmane.org/gmane.comp.python.mime.devel/325

I think when strict parsing is turned on, this 
behavior is fine, but not when it's turned off.

In lax parsing mode, I don't think this should
raise an exception. Perhaps just skipping the
bogus header field is acceptable.


----------------------------------------------------------------------

>Comment By: Jason R. Mastaler (jasonrm)
Date: 2003-02-20 12:33

Message:
Logged In: YES 
user_id=85984

As Python 2.3a2 was just released, I'm worried that
this one is going to fall through the cracks before
2.3-final is released which would be unfortunate because
it would mean I'd have to continue bundling my own
copy of email with my applications.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=633550&group_id=5470


From noreply@sourceforge.net  Thu Feb 20 20:02:11 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 20 Feb 2003 12:02:11 -0800
Subject: [Python-bugs-list] [ python-Bugs-690083 ] test_random fails sometimes
Message-ID: <E18lwtX-0004up-00@sc8-sf-web3.sourceforge.net>

Bugs item #690083, was opened at 2003-02-20 10:07
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690083&group_id=5470

Category: Python Library
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: M.-A. Lemburg (lemburg)
Assigned to: Raymond Hettinger (rhettinger)
Summary: test_random fails sometimes

Initial Comment:
>> test test_random failed -- Traceback (most recent
call last):
>>   File
"/home/lemburg/projects/Python/Dev-Python/Lib/test/test_random.py",
line 25, in test_autoseed
>>     self.assertNotEqual(state1, state2)
>>   File
"/home/lemburg/projects/Python/Dev-Python/Lib/unittest.py",
line 300, in failIfEqual
>>     raise self.failureException, \
>> AssertionError: (2, (1045703145, 318847470,
620640488, 320379787, -1732263717, -1278593374, ...
>
>
>
> This one I don't understand.  Maybe the sleep(1) in
the test isn't
> long enough?


I could not reproduce this, but perhaps using
long(time.time()*256) isn't
safe enough to produce different states in the
WH-generator ?! I'd
add a counter or hash the previous stored state to add some
more noise.


----------------------------------------------------------------------

>Comment By: Raymond Hettinger (rhettinger)
Date: 2003-02-20 15:02

Message:
Logged In: YES 
user_id=80475

The test verifies that time.time() is being called when no 
seed is provided.  We've shown that is true.  The issues of 
timer resolution appear to be just an interesting 
distraction, so Tim's simple time.sleep(1.1) approach 
looks best to me.

I'll use 1.11 just to make sure this bug stays dead.  




----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2003-02-20 13:50

Message:
Logged In: YES 
user_id=31435

datetimemodule.c contains a little routine time_time(), which 
imports the time module and calls time.time(), returning the 
result as a Python float.  As the comment above it says, 
maybe it would be better to make this part of Python's C API 
(time.time() plays a lot of platform-specific tricks to get better-
than-second resolution when possible).

OTOH, sticking time.sleep(1.1) in the test case would 
probably fix it too.

----------------------------------------------------------------------

Comment By: M.-A. Lemburg (lemburg)
Date: 2003-02-20 12:58

Message:
Logged In: YES 
user_id=38388

Seconds are longer in Old Europe :-)

Seriously, I don't know how time.sleep() is implemented by the
C lib or the kernel. Could be that time.sleep(1) sometimes only
waits 0.99 seconds or that C libs time() uses some counter which
isn't always updated in exact one second intervals.

I'd suggest to use the same clock source used in the time module
for better accuracy rather than relying on _random's seed() to
provide it's own C lib time() based one.

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2003-02-20 12:32

Message:
Logged In: YES 
user_id=31435

I expect that time.sleep(0.01) would cause the Twister test to 
fail (on average) 99 times per 100.

For some reason we don't understand then, two calls to C 
time() returned the same value despite an intervening Python 
time.sleep(1) call.  Has there been a leap second lately 
<wink>?

----------------------------------------------------------------------

Comment By: M.-A. Lemburg (lemburg)
Date: 2003-02-20 12:27

Message:
Logged In: YES 
user_id=38388

Using time.sleep(0.01) causes the test to always fail on my
machine
and instrumenting the code with a print reveals that it's indeed
the _random code failing:

test_autoseed (__main__.MersenneTwister_TestBasicOps) ...
Running class <class '__main__.MersenneTwister_TestBasicOps'>
FAIL


----------------------------------------------------------------------

Comment By: M.-A. Lemburg (lemburg)
Date: 2003-02-20 12:22

Message:
Logged In: YES 
user_id=38388

Tim, no I'm not sure that it's the WH seed method causing
this. I was basically just guessing here from looking at the
code in random.py.

You could be right in that the time() C function used in _random
is causing the problem.

I'll check with a time.sleep(0.01) call instead of
time.sleep(1).

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2003-02-20 11:38

Message:
Logged In: YES 
user_id=31435

MAL, do you know that this failed during the WH part of the 
test?  The WH seed() guarantees to produce distinct states 
given distinct inputs, and time.time() all by itself should be 
enough so that it's significantly different between time.sleep
(1) calls.  Multiplying by 256 is just gravy, getting some 
benefit from the fractional seconds too.

The Twister's idea of time-based seeding uses the C time() 
function, though, and that's less robust because it doesn't 
produce "fractional seconds" (as time.time() produces, as 
well as it can).  But it should still be at least a second 
different, unless your system clock got fiddled between calls 
(so, e.g., do you use some web-based time synch service to 
keep your clock up to date?).

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690083&group_id=5470


From noreply@sourceforge.net  Thu Feb 20 20:14:55 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 20 Feb 2003 12:14:55 -0800
Subject: [Python-bugs-list] [ python-Bugs-690253 ] Strange control flow in PyInt_AsLong
Message-ID: <E18lx5r-0006CC-00@sc8-sf-web1.sourceforge.net>

Bugs item #690253, was opened at 2003-02-20 21:14
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690253&group_id=5470

Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Thomas Heller (theller)
Assigned to: Nobody/Anonymous (nobody)
Summary: Strange control flow in PyInt_AsLong

Initial Comment:
When nb_int is called inside the PyInt_AsLong function,
and this returns a long, the value is first retrieved
with PyLong_AsLong, but afterwards overwritten by a
call to PyInt_AS_LONG.

Patch:

*** intobject.c	12 Feb 2003 20:43:33 -0000	2.102
--- intobject.c	20 Feb 2003 20:05:32 -0000
***************
*** 162,171 ****
  		if (PyLong_Check(io)) {
  			/* got a long? => retry int conversion */
  			val = PyLong_AsLong((PyObject *)io);
! 			if (PyErr_Occurred()) {
! 				Py_DECREF(io);
  				return -1;
! 			}
  		}
  		else
  		{
--- 162,171 ----
  		if (PyLong_Check(io)) {
  			/* got a long? => retry int conversion */
  			val = PyLong_AsLong((PyObject *)io);
! 			Py_DECREF(io);
! 			if (PyErr_Occurred())
  				return -1;
! 			return val;
  		}
  		else
  		{



----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690253&group_id=5470


From noreply@sourceforge.net  Thu Feb 20 20:34:01 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 20 Feb 2003 12:34:01 -0800
Subject: [Python-bugs-list] [ python-Bugs-690253 ] Strange control flow in PyInt_AsLong
Message-ID: <E18lxOL-0004By-00@sc8-sf-web2.sourceforge.net>

Bugs item #690253, was opened at 2003-02-20 15:14
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690253&group_id=5470

Category: None
Group: None
Status: Open
>Resolution: Accepted
Priority: 5
Submitted By: Thomas Heller (theller)
>Assigned to: Thomas Heller (theller)
Summary: Strange control flow in PyInt_AsLong

Initial Comment:
When nb_int is called inside the PyInt_AsLong function,
and this returns a long, the value is first retrieved
with PyLong_AsLong, but afterwards overwritten by a
call to PyInt_AS_LONG.

Patch:

*** intobject.c	12 Feb 2003 20:43:33 -0000	2.102
--- intobject.c	20 Feb 2003 20:05:32 -0000
***************
*** 162,171 ****
  		if (PyLong_Check(io)) {
  			/* got a long? => retry int conversion */
  			val = PyLong_AsLong((PyObject *)io);
! 			if (PyErr_Occurred()) {
! 				Py_DECREF(io);
  				return -1;
! 			}
  		}
  		else
  		{
--- 162,171 ----
  		if (PyLong_Check(io)) {
  			/* got a long? => retry int conversion */
  			val = PyLong_AsLong((PyObject *)io);
! 			Py_DECREF(io);
! 			if (PyErr_Occurred())
  				return -1;
! 			return val;
  		}
  		else
  		{



----------------------------------------------------------------------

>Comment By: Tim Peters (tim_one)
Date: 2003-02-20 15:34

Message:
Logged In: YES 
user_id=31435

Sounds good to me.  As long as you're at it <wink>, don't call 
PyErr_Occurred() unless val is -1 either.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690253&group_id=5470


From noreply@sourceforge.net  Thu Feb 20 20:45:26 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 20 Feb 2003 12:45:26 -0800
Subject: [Python-bugs-list] [ python-Bugs-595026 ] Support for masks in getargs.c
Message-ID: <E18lxZO-0002Ke-00@sc8-sf-web4.sourceforge.net>

Bugs item #595026, was opened at 2002-08-14 14:26
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=595026&group_id=5470

Category: Python Interpreter Core
Group: Python 2.3
Status: Open
Resolution: None
Priority: 6
Submitted By: Guido van Rossum (gvanrossum)
Assigned to: Guido van Rossum (gvanrossum)
Summary: Support for masks in getargs.c

Initial Comment:
We need this implemented:

> How about the following counterproposal.  This also
changes some of
> the other format codes to be a little more regular.
>
> Code    C type          	Range check
>
> b	unsigned char		0..UCHAR_MAX
> B	unsigned char		none **
> h	unsigned short		0..USHRT_MAX
> H	unsigned short		none **
> i	int
		INT_MIN..INT_MAX
> I *	unsigned int		0..UINT_MAX
> l	long
		LONG_MIN..LONG_MAX
> k *	unsigned long		none
> L	long long		LLONG_MIN..LLONG_MAX
> K *	unsigned long long	none
>
> Notes:
>
> * New format codes.
>
> ** Changed from previous "range-and-a-half" to
"none"; the
>    range-and-a-half checking wasn't particularly useful.

Plus a C API or two, e.g. PyInt_AsLongMask() ->
unsigned long and PyInt_AsLongLongMask() -> unsigned
long long (if that exists).

----------------------------------------------------------------------

>Comment By: Thomas Heller (theller)
Date: 2003-02-20 21:45

Message:
Logged In: YES 
user_id=11105

I've implemented the 'k' getargs code, and
PyInt_AsUnsignedLongMask and PyLong_AsUnsignedLongMask
functions.  Is there any facility which would help me to
test this new code?

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-02-18 23:08

Message:
Logged In: YES 
user_id=45365

Guido,
I would be happy with the no-rangecheck unsigned long formatcode. One request, though: if this is implemented can it be done ASAP after 2.3a2, so there's still some time to shake out the bugs before 2.3b1 comes out? Especially the hand-written code will have to be examined to see where l needs to be turned into k.

----------------------------------------------------------------------

Comment By: Thomas Heller (theller)
Date: 2003-02-18 21:59

Message:
Logged In: YES 
user_id=11105

Currently the h code means signed short
(SHRT_MIN..SHRT_MAX), do you really want to change that to
unsigned short (0..USHRT_MAX)?

----------------------------------------------------------------------

Comment By: Thomas Heller (theller)
Date: 2003-02-18 21:33

Message:
Logged In: YES 
user_id=11105

Ok, I'll use your codes.

Here is my suggestion for the C api functions:

int PyInt_AsLongMask(PyObject *v, unsigned long *pval);
int PyInt_AsLongLongMask(PyObject *v, unsigned LONG_LONG *pval);

return -1 and set exception on error, return 0 otherwise and
store the result in pval. This saves the PyErr_Occurred() in
case the value is -1.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-18 21:18

Message:
Logged In: YES 
user_id=6380

A few days is fine (this doesn't need to make it into 2.3a2).

But the proposal here is not backwards compatible, is it?
Those codes already mean something different now.

I think we'll need to invent new format codes, or a new
modifier.

----------------------------------------------------------------------

Comment By: Thomas Heller (theller)
Date: 2003-02-18 21:16

Message:
Logged In: YES 
user_id=11105

But it would probably take a few days.

What about these changes to the format codes, they are now
more in sync with the struct module (I hope the formatting
is kept):

Code    C type          	Range check

b	unsigned char		0..UCHAR_MAX
B	unsigned char		none **
h	unsigned short		0..USHRT_MAX
H	unsigned short		none **
i	int			INT_MIN..INT_MAX
I *	unsigned int		0..UINT_MAX
l	long			LONG_MIN..LONG_MAX
L *	unsigned long		none
q	long long		LLONG_MIN..LLONG_MAX
Q *	unsigned long long	none


----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-18 20:50

Message:
Logged In: YES 
user_id=6380

Thomas: that would be great! (As long it isn't killed by bw
compat. :-)


----------------------------------------------------------------------

Comment By: Thomas Heller (theller)
Date: 2003-02-18 20:18

Message:
Logged In: YES 
user_id=11105

If nobody else comes up, I can do this. I also had similar
checks for the struct module, but Tim killed it because of
backward compatibility.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=595026&group_id=5470


From noreply@sourceforge.net  Thu Feb 20 20:49:28 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 20 Feb 2003 12:49:28 -0800
Subject: [Python-bugs-list] [ python-Bugs-690253 ] Strange control flow in PyInt_AsLong
Message-ID: <E18lxdI-0002VV-00@sc8-sf-web4.sourceforge.net>

Bugs item #690253, was opened at 2003-02-20 21:14
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690253&group_id=5470

Category: None
Group: None
>Status: Closed
>Resolution: Fixed
Priority: 5
Submitted By: Thomas Heller (theller)
Assigned to: Thomas Heller (theller)
Summary: Strange control flow in PyInt_AsLong

Initial Comment:
When nb_int is called inside the PyInt_AsLong function,
and this returns a long, the value is first retrieved
with PyLong_AsLong, but afterwards overwritten by a
call to PyInt_AS_LONG.

Patch:

*** intobject.c	12 Feb 2003 20:43:33 -0000	2.102
--- intobject.c	20 Feb 2003 20:05:32 -0000
***************
*** 162,171 ****
  		if (PyLong_Check(io)) {
  			/* got a long? => retry int conversion */
  			val = PyLong_AsLong((PyObject *)io);
! 			if (PyErr_Occurred()) {
! 				Py_DECREF(io);
  				return -1;
! 			}
  		}
  		else
  		{
--- 162,171 ----
  		if (PyLong_Check(io)) {
  			/* got a long? => retry int conversion */
  			val = PyLong_AsLong((PyObject *)io);
! 			Py_DECREF(io);
! 			if (PyErr_Occurred())
  				return -1;
! 			return val;
  		}
  		else
  		{



----------------------------------------------------------------------

>Comment By: Thomas Heller (theller)
Date: 2003-02-20 21:49

Message:
Logged In: YES 
user_id=11105

Checked in, with your improvement. I don't have a 2.2-maint
branch handy at the moment, is this also an issue there?

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2003-02-20 21:34

Message:
Logged In: YES 
user_id=31435

Sounds good to me.  As long as you're at it <wink>, don't call 
PyErr_Occurred() unless val is -1 either.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690253&group_id=5470


From noreply@sourceforge.net  Thu Feb 20 21:12:19 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 20 Feb 2003 13:12:19 -0800
Subject: [Python-bugs-list] [ python-Bugs-690285 ] lines run together on input
Message-ID: <E18lxzP-00007s-00@sc8-sf-web3.sourceforge.net>

Bugs item #690285, was opened at 2003-02-20 16:12
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690285&group_id=5470

Category: Python Library
Group: Python 2.2.2
Status: Open
Resolution: None
Priority: 5
Submitted By: Thomas B Hickey (thomasbhickey)
Assigned to: Nobody/Anonymous (nobody)
Summary: lines run together on input

Initial Comment:

Using  'for line in file('xxxxx'):'  on a large (>4 Gbyte) text file 
3/4 of the way into the file occasionally two lines will be 
merged into one.  Happens consistently at the same lines.  
Counting lines this way always comes up short.  Changing 
the open to 'rb' fixes the problem.  These lines are 
terminated by '\n' even though we are running on a PC.

Haven't tried the universal newline mode in 2.3a, but would 
be willing to.

Running on Windows NT 2000, 2.2.2 (#37 Oct 14, 2002).  
Same problem seen in 2.2.1.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690285&group_id=5470


From noreply@sourceforge.net  Thu Feb 20 21:13:59 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 20 Feb 2003 13:13:59 -0800
Subject: [Python-bugs-list] [ python-Bugs-690253 ] Strange control flow in PyInt_AsLong
Message-ID: <E18ly11-0003bd-00@sc8-sf-web4.sourceforge.net>

Bugs item #690253, was opened at 2003-02-20 15:14
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690253&group_id=5470

Category: None
Group: None
Status: Closed
Resolution: Fixed
Priority: 5
Submitted By: Thomas Heller (theller)
Assigned to: Thomas Heller (theller)
Summary: Strange control flow in PyInt_AsLong

Initial Comment:
When nb_int is called inside the PyInt_AsLong function,
and this returns a long, the value is first retrieved
with PyLong_AsLong, but afterwards overwritten by a
call to PyInt_AS_LONG.

Patch:

*** intobject.c	12 Feb 2003 20:43:33 -0000	2.102
--- intobject.c	20 Feb 2003 20:05:32 -0000
***************
*** 162,171 ****
  		if (PyLong_Check(io)) {
  			/* got a long? => retry int conversion */
  			val = PyLong_AsLong((PyObject *)io);
! 			if (PyErr_Occurred()) {
! 				Py_DECREF(io);
  				return -1;
! 			}
  		}
  		else
  		{
--- 162,171 ----
  		if (PyLong_Check(io)) {
  			/* got a long? => retry int conversion */
  			val = PyLong_AsLong((PyObject *)io);
! 			Py_DECREF(io);
! 			if (PyErr_Occurred())
  				return -1;
! 			return val;
  		}
  		else
  		{



----------------------------------------------------------------------

>Comment By: Tim Peters (tim_one)
Date: 2003-02-20 16:13

Message:
Logged In: YES 
user_id=31435

Dunno --didn't pay enough attention.  Was it an actual bug, or 
just a lack of grace?

----------------------------------------------------------------------

Comment By: Thomas Heller (theller)
Date: 2003-02-20 15:49

Message:
Logged In: YES 
user_id=11105

Checked in, with your improvement. I don't have a 2.2-maint
branch handy at the moment, is this also an issue there?

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2003-02-20 15:34

Message:
Logged In: YES 
user_id=31435

Sounds good to me.  As long as you're at it <wink>, don't call 
PyErr_Occurred() unless val is -1 either.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690253&group_id=5470


From noreply@sourceforge.net  Thu Feb 20 21:16:26 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 20 Feb 2003 13:16:26 -0800
Subject: [Python-bugs-list] [ python-Bugs-690253 ] Strange control flow in PyInt_AsLong
Message-ID: <E18ly3O-00065i-00@sc8-sf-web2.sourceforge.net>

Bugs item #690253, was opened at 2003-02-20 21:14
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690253&group_id=5470

Category: None
Group: None
Status: Closed
Resolution: Fixed
Priority: 5
Submitted By: Thomas Heller (theller)
Assigned to: Thomas Heller (theller)
Summary: Strange control flow in PyInt_AsLong

Initial Comment:
When nb_int is called inside the PyInt_AsLong function,
and this returns a long, the value is first retrieved
with PyLong_AsLong, but afterwards overwritten by a
call to PyInt_AS_LONG.

Patch:

*** intobject.c	12 Feb 2003 20:43:33 -0000	2.102
--- intobject.c	20 Feb 2003 20:05:32 -0000
***************
*** 162,171 ****
  		if (PyLong_Check(io)) {
  			/* got a long? => retry int conversion */
  			val = PyLong_AsLong((PyObject *)io);
! 			if (PyErr_Occurred()) {
! 				Py_DECREF(io);
  				return -1;
! 			}
  		}
  		else
  		{
--- 162,171 ----
  		if (PyLong_Check(io)) {
  			/* got a long? => retry int conversion */
  			val = PyLong_AsLong((PyObject *)io);
! 			Py_DECREF(io);
! 			if (PyErr_Occurred())
  				return -1;
! 			return val;
  		}
  		else
  		{



----------------------------------------------------------------------

>Comment By: Thomas Heller (theller)
Date: 2003-02-20 22:16

Message:
Logged In: YES 
user_id=11105

Dunno either, I found it when reading the code, not when
hunting for a bug.

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2003-02-20 22:13

Message:
Logged In: YES 
user_id=31435

Dunno --didn't pay enough attention.  Was it an actual bug, or 
just a lack of grace?

----------------------------------------------------------------------

Comment By: Thomas Heller (theller)
Date: 2003-02-20 21:49

Message:
Logged In: YES 
user_id=11105

Checked in, with your improvement. I don't have a 2.2-maint
branch handy at the moment, is this also an issue there?

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2003-02-20 21:34

Message:
Logged In: YES 
user_id=31435

Sounds good to me.  As long as you're at it <wink>, don't call 
PyErr_Occurred() unless val is -1 either.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690253&group_id=5470


From noreply@sourceforge.net  Thu Feb 20 21:34:16 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 20 Feb 2003 13:34:16 -0800
Subject: [Python-bugs-list] [ python-Bugs-690253 ] Strange control flow in PyInt_AsLong
Message-ID: <E18lyKe-0001Oc-00@sc8-sf-web1.sourceforge.net>

Bugs item #690253, was opened at 2003-02-20 15:14
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690253&group_id=5470

Category: None
Group: None
Status: Closed
Resolution: Fixed
Priority: 5
Submitted By: Thomas Heller (theller)
Assigned to: Thomas Heller (theller)
Summary: Strange control flow in PyInt_AsLong

Initial Comment:
When nb_int is called inside the PyInt_AsLong function,
and this returns a long, the value is first retrieved
with PyLong_AsLong, but afterwards overwritten by a
call to PyInt_AS_LONG.

Patch:

*** intobject.c	12 Feb 2003 20:43:33 -0000	2.102
--- intobject.c	20 Feb 2003 20:05:32 -0000
***************
*** 162,171 ****
  		if (PyLong_Check(io)) {
  			/* got a long? => retry int conversion */
  			val = PyLong_AsLong((PyObject *)io);
! 			if (PyErr_Occurred()) {
! 				Py_DECREF(io);
  				return -1;
! 			}
  		}
  		else
  		{
--- 162,171 ----
  		if (PyLong_Check(io)) {
  			/* got a long? => retry int conversion */
  			val = PyLong_AsLong((PyObject *)io);
! 			Py_DECREF(io);
! 			if (PyErr_Occurred())
  				return -1;
! 			return val;
  		}
  		else
  		{



----------------------------------------------------------------------

>Comment By: Tim Peters (tim_one)
Date: 2003-02-20 16:34

Message:
Logged In: YES 
user_id=31435

Well, it's a small change, and clearly a good change -- it 
would go a lot faster to backport it than to think about it.

----------------------------------------------------------------------

Comment By: Thomas Heller (theller)
Date: 2003-02-20 16:16

Message:
Logged In: YES 
user_id=11105

Dunno either, I found it when reading the code, not when
hunting for a bug.

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2003-02-20 16:13

Message:
Logged In: YES 
user_id=31435

Dunno --didn't pay enough attention.  Was it an actual bug, or 
just a lack of grace?

----------------------------------------------------------------------

Comment By: Thomas Heller (theller)
Date: 2003-02-20 15:49

Message:
Logged In: YES 
user_id=11105

Checked in, with your improvement. I don't have a 2.2-maint
branch handy at the moment, is this also an issue there?

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2003-02-20 15:34

Message:
Logged In: YES 
user_id=31435

Sounds good to me.  As long as you're at it <wink>, don't call 
PyErr_Occurred() unless val is -1 either.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690253&group_id=5470


From noreply@sourceforge.net  Thu Feb 20 21:53:58 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 20 Feb 2003 13:53:58 -0800
Subject: [Python-bugs-list] [ python-Bugs-690309 ] 2.3a2 make fails at _iconv_codec.
Message-ID: <E18lydi-0005Qc-00@sc8-sf-web4.sourceforge.net>

Bugs item #690309, was opened at 2003-02-20 21:53
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690309&group_id=5470

Category: Build
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Inyeol Lee (inyeol)
Assigned to: Nobody/Anonymous (nobody)
Summary: 2.3a2 make fails at _iconv_codec.

Initial Comment:
Python 2.3a2 build process fails during building
_iconv_codec
module. The message is;

[...]
gcc -shared
build/temp.solaris-2.8-sun4u-2.3/_iconv_codec.o
-L/home/inyeol/util/python/lib -L/usr/local/lib -o
build/lib.solaris-2.8-sun4u-2.3/_iconv_codec.so
Traceback (most recent call last):

  [... long traceback here...]

  File "./setup.py", line 209, in build_extension
    imp.load_dynamic(ext.name, ext_filename)
RuntimeError: can't initialize the _iconv_codec module:
iconv_open() failed
*** Error code 1
make: Fatal error: Command failed for target
`sharedmods'

My build environment is gcc2.95.3 with Solaris 8
linker.
I've built 2.2, 2,2,1, 2,2,2, 2,3a1 with the same env
and there
was no (major) issues like this. I attached my
configure and
make log messages.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690309&group_id=5470


From noreply@sourceforge.net  Thu Feb 20 21:56:06 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 20 Feb 2003 13:56:06 -0800
Subject: [Python-bugs-list] [ python-Bugs-690309 ] 2.3a2 Sol8 make fails at _iconv_codec.
Message-ID: <E18lyfm-0001ju-00@sc8-sf-web3.sourceforge.net>

Bugs item #690309, was opened at 2003-02-20 21:53
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690309&group_id=5470

Category: Build
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Inyeol Lee (inyeol)
Assigned to: Nobody/Anonymous (nobody)
>Summary: 2.3a2 Sol8 make fails at _iconv_codec.

Initial Comment:
Python 2.3a2 build process fails during building
_iconv_codec
module. The message is;

[...]
gcc -shared
build/temp.solaris-2.8-sun4u-2.3/_iconv_codec.o
-L/home/inyeol/util/python/lib -L/usr/local/lib -o
build/lib.solaris-2.8-sun4u-2.3/_iconv_codec.so
Traceback (most recent call last):

  [... long traceback here...]

  File "./setup.py", line 209, in build_extension
    imp.load_dynamic(ext.name, ext_filename)
RuntimeError: can't initialize the _iconv_codec module:
iconv_open() failed
*** Error code 1
make: Fatal error: Command failed for target
`sharedmods'

My build environment is gcc2.95.3 with Solaris 8
linker.
I've built 2.2, 2,2,1, 2,2,2, 2,3a1 with the same env
and there
was no (major) issues like this. I attached my
configure and
make log messages.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690309&group_id=5470


From noreply@sourceforge.net  Thu Feb 20 22:06:14 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 20 Feb 2003 14:06:14 -0800
Subject: [Python-bugs-list] [ python-Bugs-690317 ] 2.3a2 build fails on Solaris: posixmodule
Message-ID: <E18lypa-00065f-00@sc8-sf-web4.sourceforge.net>

Bugs item #690317, was opened at 2003-02-20 16:06
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690317&group_id=5470

Category: Build
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Jeff Bauer (jeffbauer)
Assigned to: Nobody/Anonymous (nobody)
Summary: 2.3a2 build fails on Solaris: posixmodule

Initial Comment:
gcc 2.95.2
SunOS 5.6 sparc

gcc  -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -I.
-I./Include  -DPy_BUILD_CORE      -c
./Modules/posixmodule.c -o Modules/posixmodule.o

In file included from Include/stringobject.h:10,
                 from Include/Python.h:83,
                 from ./Modules/posixmodule.c:16:
/usr/local/lib/gcc-lib/sparc-sun-solaris2.6/2.95.2/include/stdarg.h:170:
warning                                                                                                          
: redefinition of `va_list'
/usr/include/stdio.h:90: warning: `va_list' previously
declared here
./Modules/posixmodule.c: In function `posix_access':
./Modules/posixmodule.c:1261: warning: implicit
declaration of function `access'
./Modules/posixmodule.c: In function `posix_ttyname':
./Modules/posixmodule.c:1302: warning: implicit
declaration of function
`ttyname                                                                                                          
'

----  <snip> <snip> <snip>  ---


                             f'
./Modules/posixmodule.c: In function `posix_confstr':
./Modules/posixmodule.c:6443: warning: implicit
declaration of function
`confstr                                                                                                          
'
./Modules/posixmodule.c: In function `posix_sysconf':
./Modules/posixmodule.c:6985: warning: implicit
declaration of function
`sysconf                                                                                                          
'
./Modules/posixmodule.c: In function
`setup_confname_table':
./Modules/posixmodule.c:7024: warning: implicit
declaration of function `qsort'
make: *** [Modules/posixmodule.o] Error 1



----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690317&group_id=5470


From noreply@sourceforge.net  Thu Feb 20 22:15:58 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 20 Feb 2003 14:15:58 -0800
Subject: [Python-bugs-list] [ python-Bugs-690309 ] 2.3a2 Sol8 make fails at _iconv_codec.
Message-ID: <E18lyz0-0002uf-00@sc8-sf-web1.sourceforge.net>

Bugs item #690309, was opened at 2003-02-20 16:53
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690309&group_id=5470

Category: Build
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Inyeol Lee (inyeol)
Assigned to: Nobody/Anonymous (nobody)
Summary: 2.3a2 Sol8 make fails at _iconv_codec.

Initial Comment:
Python 2.3a2 build process fails during building
_iconv_codec
module. The message is;

[...]
gcc -shared
build/temp.solaris-2.8-sun4u-2.3/_iconv_codec.o
-L/home/inyeol/util/python/lib -L/usr/local/lib -o
build/lib.solaris-2.8-sun4u-2.3/_iconv_codec.so
Traceback (most recent call last):

  [... long traceback here...]

  File "./setup.py", line 209, in build_extension
    imp.load_dynamic(ext.name, ext_filename)
RuntimeError: can't initialize the _iconv_codec module:
iconv_open() failed
*** Error code 1
make: Fatal error: Command failed for target
`sharedmods'

My build environment is gcc2.95.3 with Solaris 8
linker.
I've built 2.2, 2,2,1, 2,2,2, 2,3a1 with the same env
and there
was no (major) issues like this. I attached my
configure and
make log messages.


----------------------------------------------------------------------

>Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-20 17:15

Message:
Logged In: YES 
user_id=33168

In order to get the build to proceed, you can change line
210 of setup.py from:

        except ImportError, why:
to:
        except (ImportError, RuntimeError), why:

This is just before:
    self.announce('*** WARNING: renaming "%s" since
importing it' ...)

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690309&group_id=5470


From noreply@sourceforge.net  Thu Feb 20 22:21:49 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 20 Feb 2003 14:21:49 -0800
Subject: [Python-bugs-list] [ python-Bugs-690341 ] test_tarfile fails on MacOS9
Message-ID: <E18lz4f-0003Dn-00@sc8-sf-web1.sourceforge.net>

Bugs item #690341, was opened at 2003-02-20 23:21
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690341&group_id=5470

Category: Python Library
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Jack Jansen (jackjansen)
Assigned to: Neal Norwitz (nnorwitz)
Summary: test_tarfile fails on MacOS9

Initial Comment:
test_tarfile fails in MacPython-OS9. The problem seems to be pathname-related, and I'm not sure whether the problem is with the test or with the tarfile module itself.

Various of the tests (test_seek, test_readlines) expect to find a file called, for example, "0-REGTYPE-TEXT", while the archive holds a file "/0-REGTYPE-TEXT". Apparently the tarfile module has some support for os.sep being different than "/", but not enough.

An added problem (or maybe the only problem?) may be that on the mac there are no files at the "root", only directories (disks, actually). So while "/tmp" is representable as "tmp:" this doesn't really work for files as they would be interpreted as directories.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690341&group_id=5470


From noreply@sourceforge.net  Thu Feb 20 23:08:58 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 20 Feb 2003 15:08:58 -0800
Subject: [Python-bugs-list] [ python-Bugs-690317 ] 2.3a2 build fails on Solaris: posixmodule
Message-ID: <E18lzoI-0005Fh-00@sc8-sf-web1.sourceforge.net>

Bugs item #690317, was opened at 2003-02-20 17:06
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690317&group_id=5470

Category: Build
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Jeff Bauer (jeffbauer)
Assigned to: Nobody/Anonymous (nobody)
Summary: 2.3a2 build fails on Solaris: posixmodule

Initial Comment:
gcc 2.95.2
SunOS 5.6 sparc

gcc  -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -I.
-I./Include  -DPy_BUILD_CORE      -c
./Modules/posixmodule.c -o Modules/posixmodule.o

In file included from Include/stringobject.h:10,
                 from Include/Python.h:83,
                 from ./Modules/posixmodule.c:16:
/usr/local/lib/gcc-lib/sparc-sun-solaris2.6/2.95.2/include/stdarg.h:170:
warning                                                                                                          
: redefinition of `va_list'
/usr/include/stdio.h:90: warning: `va_list' previously
declared here
./Modules/posixmodule.c: In function `posix_access':
./Modules/posixmodule.c:1261: warning: implicit
declaration of function `access'
./Modules/posixmodule.c: In function `posix_ttyname':
./Modules/posixmodule.c:1302: warning: implicit
declaration of function
`ttyname                                                                                                          
'

----  <snip> <snip> <snip>  ---


                             f'
./Modules/posixmodule.c: In function `posix_confstr':
./Modules/posixmodule.c:6443: warning: implicit
declaration of function
`confstr                                                                                                          
'
./Modules/posixmodule.c: In function `posix_sysconf':
./Modules/posixmodule.c:6985: warning: implicit
declaration of function
`sysconf                                                                                                          
'
./Modules/posixmodule.c: In function
`setup_confname_table':
./Modules/posixmodule.c:7024: warning: implicit
declaration of function `qsort'
make: *** [Modules/posixmodule.o] Error 1



----------------------------------------------------------------------

>Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-20 18:08

Message:
Logged In: YES 
user_id=33168

Jeff, I don't know if anyone has access to Solaris 5.6.  Can
you provide a patch to get posixmodule to compile?  From the
bug report, I only see warnings, not an actual error which
is breaking the build.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690317&group_id=5470


From noreply@sourceforge.net  Thu Feb 20 23:14:29 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 20 Feb 2003 15:14:29 -0800
Subject: [Python-bugs-list] [ python-Bugs-690012 ] 2.3a2 build fails under IRIX 6.5
Message-ID: <E18lztd-0002dD-00@sc8-sf-web2.sourceforge.net>

Bugs item #690012, was opened at 2003-02-20 07:36
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690012&group_id=5470

Category: Build
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Ralf W. Grosse-Kunstleve (rwgk)
Assigned to: Nobody/Anonymous (nobody)
Summary: 2.3a2 build fails under IRIX 6.5

Initial Comment:
Using the latest Python 2.3a2 release
./configure; make
fails on this platform:
IRIX rattler 6.5 10120734 IP32
See attached logs.

I've also tried configure --disable-ipv6, but the result is 
the same.

The previous 2.3a1 release works without a problem 
on the exact same machine.


----------------------------------------------------------------------

>Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-20 18:14

Message:
Logged In: YES 
user_id=33168

In order to get the build to proceed, you can change line
210 of setup.py from:

        except ImportError, why:
to:
        except (ImportError, RuntimeError), why:

This is just before:
    self.announce('*** WARNING: renaming "%s" since
importing it' ...)



----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690012&group_id=5470


From noreply@sourceforge.net  Thu Feb 20 23:26:11 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 20 Feb 2003 15:26:11 -0800
Subject: [Python-bugs-list] [ python-Bugs-690377 ] Dud footnote reference in distutils docs
Message-ID: <E18m04x-00033M-00@sc8-sf-web2.sourceforge.net>

Bugs item #690377, was opened at 2003-02-21 10:26
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690377&group_id=5470

Category: Documentation
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Stuart Bishop (zenzen)
Assigned to: Nobody/Anonymous (nobody)
Summary: Dud footnote reference in distutils docs

Initial Comment:
In section 3.6, the wrong footnote is specified for
'trove_classifiers'.  Should be (3) instead of (2).

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690377&group_id=5470


From noreply@sourceforge.net  Thu Feb 20 23:45:11 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 20 Feb 2003 15:45:11 -0800
Subject: [Python-bugs-list] [ python-Bugs-690377 ] Dud footnote reference in distutils docs
Message-ID: <E18m0NL-0003ic-00@sc8-sf-web2.sourceforge.net>

Bugs item #690377, was opened at 2003-02-21 10:26
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690377&group_id=5470

Category: Documentation
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Stuart Bishop (zenzen)
Assigned to: Nobody/Anonymous (nobody)
Summary: Dud footnote reference in distutils docs

Initial Comment:
In section 3.6, the wrong footnote is specified for
'trove_classifiers'.  Should be (3) instead of (2).

----------------------------------------------------------------------

>Comment By: Stuart Bishop (zenzen)
Date: 2003-02-21 10:45

Message:
Logged In: YES 
user_id=46639

And while you are there, the required 'home_page' argument
to setup is incorrect - it should be 'url'. 

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690377&group_id=5470


From noreply@sourceforge.net  Thu Feb 20 23:52:51 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 20 Feb 2003 15:52:51 -0800
Subject: [Python-bugs-list] [ python-Bugs-690389 ] register command not listed in command line help
Message-ID: <E18m0Ul-00075b-00@sc8-sf-web1.sourceforge.net>

Bugs item #690389, was opened at 2003-02-21 10:52
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690389&group_id=5470

Category: Distutils
Group: Python 2.3
Status: Open
Resolution: None
>Priority: 2
Submitted By: Stuart Bishop (zenzen)
Assigned to: Nobody/Anonymous (nobody)
Summary: register command not listed in command line help

Initial Comment:
The register command is not listed when you run
python setup.py --help-commands

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690389&group_id=5470


From noreply@sourceforge.net  Thu Feb 20 23:52:08 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 20 Feb 2003 15:52:08 -0800
Subject: [Python-bugs-list] [ python-Bugs-690389 ] register command not listed in command line help
Message-ID: <E18m0U4-0003oo-00@sc8-sf-web2.sourceforge.net>

Bugs item #690389, was opened at 2003-02-21 10:52
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690389&group_id=5470

Category: Distutils
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Stuart Bishop (zenzen)
Assigned to: Nobody/Anonymous (nobody)
Summary: register command not listed in command line help

Initial Comment:
The register command is not listed when you run
python setup.py --help-commands

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690389&group_id=5470


From noreply@sourceforge.net  Fri Feb 21 01:12:03 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 20 Feb 2003 17:12:03 -0800
Subject: [Python-bugs-list] [ python-Bugs-690419 ] can't build bsddb for 2.3a2
Message-ID: <E18m1jP-0006PJ-00@sc8-sf-web2.sourceforge.net>

Bugs item #690419, was opened at 2003-02-21 01:12
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690419&group_id=5470

Category: Build
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Paul Jarc (prjsf)
Assigned to: Nobody/Anonymous (nobody)
Summary: can't build bsddb for 2.3a2

Initial Comment:
I have db-4.1.25 installed, and I'm trying to install
Python 2.3a2 with
bsddb.  The build fails:
gcc -fPIC  -DNDEBUG -g -O3 -Wall -Wstrict-prototypes
-I. -I./Include 
-I/package/misc/spf/python-2.3a2/spf/db/include -c
./Modules/bsddbmodule.c -o Modules/bsddbmodule.o
Modules/bsddbmodule.c: In function `newdbhashobject':
Modules/bsddbmodule.c:58: `HASHINFO' undeclared (first
use in this function)
Modules/bsddbmodule.c:58: (Each undeclared identifier
is reported only once
Modules/bsddbmodule.c:58: for each function it appears in.)
Modules/bsddbmodule.c:58: parse error before "info"
Modules/bsddbmodule.c:63: `info' undeclared (first use
in this function)
Modules/bsddbmodule.c:74: warning: implicit declaration
of function `dbopen'
Modules/bsddbmodule.c:74: warning: assignment makes
pointer from integer without a cast
Modules/bsddbmodule.c: In function `newdbbtobject':
Modules/bsddbmodule.c:106: `BTREEINFO' undeclared
(first use in this function)
Modules/bsddbmodule.c:106: parse error before "info"
Modules/bsddbmodule.c:111: `info' undeclared (first use
in this function)
Modules/bsddbmodule.c:124: warning: assignment makes
pointer from integer without a cast
Modules/bsddbmodule.c: In function `newdbrnobject':
Modules/bsddbmodule.c:156: `RECNOINFO' undeclared
(first use in this function)
Modules/bsddbmodule.c:156: parse error before "info"
Modules/bsddbmodule.c:162: `info' undeclared (first use
in this function)
Modules/bsddbmodule.c:182: warning: assignment makes
pointer from integer without a cast
Modules/bsddbmodule.c: In function `bsddb_dealloc':
Modules/bsddbmodule.c:223: too few arguments to function
Modules/bsddbmodule.c: In function `bsddb_length':
Modules/bsddbmodule.c:252: structure has no member
named `seq'
Modules/bsddbmodule.c:253: `R_FIRST' undeclared (first
use in this function)
Modules/bsddbmodule.c:255: structure has no member
named `seq'
Modules/bsddbmodule.c:256: `R_NEXT' undeclared (first
use in this function)
Modules/bsddbmodule.c: In function `bsddb_subscript':
Modules/bsddbmodule.c:276: `recno_t' undeclared (first
use in this function)
Modules/bsddbmodule.c:276: parse error before "recno"
Modules/bsddbmodule.c:279: `recno' undeclared (first
use in this function)
Modules/bsddbmodule.c:299: warning: passing arg 2 of
pointer to function from incompatible pointer type
Modules/bsddbmodule.c:299: too few arguments to function
Modules/bsddbmodule.c: In function `bsddb_ass_sub':
Modules/bsddbmodule.c:327: `recno_t' undeclared (first
use in this function)
Modules/bsddbmodule.c:327: parse error before "recno"
Modules/bsddbmodule.c:330: `recno' undeclared (first
use in this function)
Modules/bsddbmodule.c:351: warning: passing arg 2 of
pointer to function from incompatible pointer type
Modules/bsddbmodule.c:351: too few arguments to function
Modules/bsddbmodule.c:363: warning: passing arg 2 of
pointer to function from incompatible pointer type
Modules/bsddbmodule.c:363: too few arguments to function
Modules/bsddbmodule.c: In function `bsddb_close':
Modules/bsddbmodule.c:388: too few arguments to function
Modules/bsddbmodule.c: In function `bsddb_keys':
Modules/bsddbmodule.c:415: structure has no member
named `seq'
Modules/bsddbmodule.c:415: `R_FIRST' undeclared (first
use in this function)
Modules/bsddbmodule.c:441: structure has no member
named `seq'
Modules/bsddbmodule.c:442: `R_NEXT' undeclared (first
use in this function)
Modules/bsddbmodule.c: In function `bsddb_has_key':
Modules/bsddbmodule.c:470: `recno_t' undeclared (first
use in this function)
Modules/bsddbmodule.c:470: parse error before "recno"
Modules/bsddbmodule.c:474: `recno' undeclared (first
use in this function)
Modules/bsddbmodule.c:491: warning: passing arg 2 of
pointer to function from incompatible pointer type
Modules/bsddbmodule.c:491: too few arguments to function
Modules/bsddbmodule.c: In function `bsddb_set_location':
Modules/bsddbmodule.c:509: `recno_t' undeclared (first
use in this function)
Modules/bsddbmodule.c:509: parse error before "recno"
Modules/bsddbmodule.c:513: `recno' undeclared (first
use in this function)
Modules/bsddbmodule.c:530: structure has no member
named `seq'
Modules/bsddbmodule.c:530: `R_CURSOR' undeclared (first
use in this function)
Modules/bsddbmodule.c: In function `bsddb_seq':
Modules/bsddbmodule.c:570: structure has no member
named `seq'
Modules/bsddbmodule.c: In function `bsddb_next':
Modules/bsddbmodule.c:608: `R_NEXT' undeclared (first
use in this function)
Modules/bsddbmodule.c: In function `bsddb_previous':
Modules/bsddbmodule.c:613: `R_PREV' undeclared (first
use in this function)
Modules/bsddbmodule.c: In function `bsddb_first':
Modules/bsddbmodule.c:618: `R_FIRST' undeclared (first
use in this function)
Modules/bsddbmodule.c: In function `bsddb_last':
Modules/bsddbmodule.c:623: `R_LAST' undeclared (first
use in this function)
make: *** [Modules/bsddbmodule.o] Error 1


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690419&group_id=5470


From noreply@sourceforge.net  Fri Feb 21 01:51:14 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 20 Feb 2003 17:51:14 -0800
Subject: [Python-bugs-list] [ python-Bugs-690083 ] test_random fails sometimes
Message-ID: <E18m2LK-0001gz-00@sc8-sf-web3.sourceforge.net>

Bugs item #690083, was opened at 2003-02-20 10:07
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690083&group_id=5470

Category: Python Library
Group: Python 2.3
>Status: Closed
>Resolution: Fixed
Priority: 5
Submitted By: M.-A. Lemburg (lemburg)
Assigned to: Raymond Hettinger (rhettinger)
Summary: test_random fails sometimes

Initial Comment:
>> test test_random failed -- Traceback (most recent
call last):
>>   File
"/home/lemburg/projects/Python/Dev-Python/Lib/test/test_random.py",
line 25, in test_autoseed
>>     self.assertNotEqual(state1, state2)
>>   File
"/home/lemburg/projects/Python/Dev-Python/Lib/unittest.py",
line 300, in failIfEqual
>>     raise self.failureException, \
>> AssertionError: (2, (1045703145, 318847470,
620640488, 320379787, -1732263717, -1278593374, ...
>
>
>
> This one I don't understand.  Maybe the sleep(1) in
the test isn't
> long enough?


I could not reproduce this, but perhaps using
long(time.time()*256) isn't
safe enough to produce different states in the
WH-generator ?! I'd
add a counter or hash the previous stored state to add some
more noise.


----------------------------------------------------------------------

>Comment By: Raymond Hettinger (rhettinger)
Date: 2003-02-20 20:51

Message:
Logged In: YES 
user_id=80475

Checked in a Lib/test/test_random.py 1.9.

----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2003-02-20 15:02

Message:
Logged In: YES 
user_id=80475

The test verifies that time.time() is being called when no 
seed is provided.  We've shown that is true.  The issues of 
timer resolution appear to be just an interesting 
distraction, so Tim's simple time.sleep(1.1) approach 
looks best to me.

I'll use 1.11 just to make sure this bug stays dead.  




----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2003-02-20 13:50

Message:
Logged In: YES 
user_id=31435

datetimemodule.c contains a little routine time_time(), which 
imports the time module and calls time.time(), returning the 
result as a Python float.  As the comment above it says, 
maybe it would be better to make this part of Python's C API 
(time.time() plays a lot of platform-specific tricks to get better-
than-second resolution when possible).

OTOH, sticking time.sleep(1.1) in the test case would 
probably fix it too.

----------------------------------------------------------------------

Comment By: M.-A. Lemburg (lemburg)
Date: 2003-02-20 12:58

Message:
Logged In: YES 
user_id=38388

Seconds are longer in Old Europe :-)

Seriously, I don't know how time.sleep() is implemented by the
C lib or the kernel. Could be that time.sleep(1) sometimes only
waits 0.99 seconds or that C libs time() uses some counter which
isn't always updated in exact one second intervals.

I'd suggest to use the same clock source used in the time module
for better accuracy rather than relying on _random's seed() to
provide it's own C lib time() based one.

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2003-02-20 12:32

Message:
Logged In: YES 
user_id=31435

I expect that time.sleep(0.01) would cause the Twister test to 
fail (on average) 99 times per 100.

For some reason we don't understand then, two calls to C 
time() returned the same value despite an intervening Python 
time.sleep(1) call.  Has there been a leap second lately 
<wink>?

----------------------------------------------------------------------

Comment By: M.-A. Lemburg (lemburg)
Date: 2003-02-20 12:27

Message:
Logged In: YES 
user_id=38388

Using time.sleep(0.01) causes the test to always fail on my
machine
and instrumenting the code with a print reveals that it's indeed
the _random code failing:

test_autoseed (__main__.MersenneTwister_TestBasicOps) ...
Running class <class '__main__.MersenneTwister_TestBasicOps'>
FAIL


----------------------------------------------------------------------

Comment By: M.-A. Lemburg (lemburg)
Date: 2003-02-20 12:22

Message:
Logged In: YES 
user_id=38388

Tim, no I'm not sure that it's the WH seed method causing
this. I was basically just guessing here from looking at the
code in random.py.

You could be right in that the time() C function used in _random
is causing the problem.

I'll check with a time.sleep(0.01) call instead of
time.sleep(1).

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2003-02-20 11:38

Message:
Logged In: YES 
user_id=31435

MAL, do you know that this failed during the WH part of the 
test?  The WH seed() guarantees to produce distinct states 
given distinct inputs, and time.time() all by itself should be 
enough so that it's significantly different between time.sleep
(1) calls.  Multiplying by 256 is just gravy, getting some 
benefit from the fractional seconds too.

The Twister's idea of time-based seeding uses the C time() 
function, though, and that's less robust because it doesn't 
produce "fractional seconds" (as time.time() produces, as 
well as it can).  But it should still be at least a second 
different, unless your system clock got fiddled between calls 
(so, e.g., do you use some web-based time synch service to 
keep your clock up to date?).

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690083&group_id=5470


From noreply@sourceforge.net  Fri Feb 21 01:54:57 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 20 Feb 2003 17:54:57 -0800
Subject: [Python-bugs-list] [ python-Bugs-690377 ] Dud footnote reference in distutils docs
Message-ID: <E18m2Ov-0007oG-00@sc8-sf-web2.sourceforge.net>

Bugs item #690377, was opened at 2003-02-20 18:26
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690377&group_id=5470

Category: Documentation
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Stuart Bishop (zenzen)
>Assigned to: Raymond Hettinger (rhettinger)
Summary: Dud footnote reference in distutils docs

Initial Comment:
In section 3.6, the wrong footnote is specified for
'trove_classifiers'.  Should be (3) instead of (2).

----------------------------------------------------------------------

Comment By: Stuart Bishop (zenzen)
Date: 2003-02-20 18:45

Message:
Logged In: YES 
user_id=46639

And while you are there, the required 'home_page' argument
to setup is incorrect - it should be 'url'. 

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690377&group_id=5470


From noreply@sourceforge.net  Fri Feb 21 02:08:27 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 20 Feb 2003 18:08:27 -0800
Subject: [Python-bugs-list] [ python-Bugs-690435 ] apply fails to check if warning raises exception
Message-ID: <E18m2bz-0002Dh-00@sc8-sf-web3.sourceforge.net>

Bugs item #690435, was opened at 2003-02-20 17:08
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690435&group_id=5470

Category: Python Interpreter Core
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Greg Chapman (glchapman)
Assigned to: Nobody/Anonymous (nobody)
Summary: apply fails to check if warning raises exception

Initial Comment:
The apply builtin function now generates a 
PendingDeprecationWarning.  However, it fails to check 
if the warning has raised an exception.  Attached is a 
small patch which adds the check to apply.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690435&group_id=5470


From noreply@sourceforge.net  Fri Feb 21 02:56:14 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 20 Feb 2003 18:56:14 -0800
Subject: [Python-bugs-list] [ python-Bugs-690377 ] Dud footnote reference in distutils docs
Message-ID: <E18m3ME-00058E-00@sc8-sf-web1.sourceforge.net>

Bugs item #690377, was opened at 2003-02-20 18:26
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690377&group_id=5470

Category: Documentation
Group: Python 2.3
>Status: Closed
>Resolution: Fixed
Priority: 5
Submitted By: Stuart Bishop (zenzen)
Assigned to: Raymond Hettinger (rhettinger)
Summary: Dud footnote reference in distutils docs

Initial Comment:
In section 3.6, the wrong footnote is specified for
'trove_classifiers'.  Should be (3) instead of (2).

----------------------------------------------------------------------

>Comment By: Raymond Hettinger (rhettinger)
Date: 2003-02-20 21:56

Message:
Logged In: YES 
user_id=80475

Doc/dist/dist.tex 1.49


----------------------------------------------------------------------

Comment By: Stuart Bishop (zenzen)
Date: 2003-02-20 18:45

Message:
Logged In: YES 
user_id=46639

And while you are there, the required 'home_page' argument
to setup is incorrect - it should be 'url'. 

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690377&group_id=5470


From noreply@sourceforge.net  Fri Feb 21 03:23:21 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 20 Feb 2003 19:23:21 -0800
Subject: [Python-bugs-list] [ python-Bugs-685775 ] turtle circle() documentation error
Message-ID: <E18m3mT-0005uP-00@sc8-sf-web1.sourceforge.net>

Bugs item #685775, was opened at 2003-02-13 02:10
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=685775&group_id=5470

Category: Documentation
Group: Python 2.3
>Status: Closed
>Resolution: Fixed
Priority: 5
Submitted By: Christopher Smith (smichr)
Assigned to: Raymond Hettinger (rhettinger)
Summary: turtle circle() documentation error

Initial Comment:
The following refer to doc page 
http://www.python.org/doc/current/lib/module-turtle.html

1) The "circle()" function of the turtle module does not behave as 
indicated in the documentation.  The doc's say that this command 
should...

"Draw a circle with radius radius whose center-point is where the 
pen would be if a forward(radius) were called."

In fact, it draws a circle centered on the point that is radius units to 
the left of the turtle.

2) Further information is given in the second paragraph that starts:

"If extent is not a full circle, one endpoint of the arc is the current 
pen position."

It might be good to add "...and the angle of the turtle is changed by 
extent."  i.e., if it started at 0 degrees and you drew 90 degrees of 
a circle, the turtle is now facing at 90 degrees.

/c


----------------------------------------------------------------------

>Comment By: Raymond Hettinger (rhettinger)
Date: 2003-02-20 22:23

Message:
Logged In: YES 
user_id=80475

Fixed.
Doc/lib/libturtle.tex 1.5

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=685775&group_id=5470


From noreply@sourceforge.net  Fri Feb 21 07:42:15 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 20 Feb 2003 23:42:15 -0800
Subject: [Python-bugs-list] [ python-Bugs-676342 ] after using pdb readline does not work correctly
Message-ID: <E18m7p1-0001BL-00@sc8-sf-web2.sourceforge.net>

Bugs item #676342, was opened at 2003-01-28 15:53
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=676342&group_id=5470

Category: Python Library
Group: Python 2.2.2
>Status: Open
Resolution: Accepted
Priority: 5
Submitted By: Michael Stone (mbrierst)
>Assigned to: Guido van Rossum (gvanrossum)
Summary: after using pdb readline does not work correctly

Initial Comment:

After I use pdb in the interpreter my readline settings are messed up.  It only knows about the pdb readline stuff forever afterward.  In fact, this happens as soon as a Pdb object is instantiated, regardless of whether or not you use it.

This is a result of some aspects of the Cmd object in Lib/cmd.py.

Currently Cmd registers a new readline completer as soon as a Cmd object is instantiated.  This is probably incorrect.  I believe the correct place to register the new handler is in the preloop hook.  That way the new readline completer is only used while the Cmd object is actually responsible for processing user input.

Next, the old readline completer should probably be re-registered in the postloop hook.  In order to do this, a new call must be added to the readline module to return the current completer so it can be saved and later restored.

The files included patch readline.c (version 2.41.64) and cmd.py (version 1.26.16.2) to make the changes described above.  After this patch readline works as expected in the interpreter.

Let me know what you think of it.

----------------------------------------------------------------------

>Comment By: Raymond Hettinger (rhettinger)
Date: 2003-02-21 02:42

Message:
Logged In: YES 
user_id=80475

Backport?

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2003-02-03 06:11

Message:
Logged In: YES 
user_id=6656

Checked in as 

Lib/cmd.py revision 1.34

Thanks!  You probably didn't need to attach the patch three
times, though :-)

----------------------------------------------------------------------

Comment By: Michael Stone (mbrierst)
Date: 2003-02-01 01:41

Message:
Logged In: YES 
user_id=670441

I started using the latest cvs version today.
I have attached a diff to the latest cvs as patchcmd2 if that's
more convenient for you to apply.

----------------------------------------------------------------------

Comment By: Michael Stone (mbrierst)
Date: 2003-02-01 01:37

Message:
Logged In: YES 
user_id=670441

I started using the latest cvs version today.
I have attached a diff to the latest cvs as patchcmd2 if that's
more convenient for you to apply.

----------------------------------------------------------------------

Comment By: Michael Stone (mbrierst)
Date: 2003-01-31 16:25

Message:
Logged In: YES 
user_id=670441

I started using the latest cvs version today.
I have attached a diff to the latest cvs as patchcmd2 if that's
more convenient for you to apply.

----------------------------------------------------------------------

Comment By: Michael Stone (mbrierst)
Date: 2003-01-30 11:59

Message:
Logged In: YES 
user_id=670441

That sounds good to me.  Go ahead and check in patchcmd.

Thanks for your suggestions, I will follow them in future patches.  In this one, I was trying to follow the conventions I saw being used in the module, but they were outdated or incorrect (and I didn't always follow them correctly anyway!).  Okay that's definitely enough commentary for a few line patch.

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2003-01-30 05:22

Message:
Logged In: YES 
user_id=6656

Thanks for the new patch, it looks OK.  There are a couple
of things that spring to mind for future patches: 

- the function should be METH_NOARGS (or you should check
the args tuple)

- there should be a blank line between the argspec line in
the docstring and the description

- you should use the PyDoc_STRVAR macro (but this was only
used in the CVS trunk's version of readline).

I made these changes and checked in revision 2.59 of
Modules/readline.c.

Wrt the cmd.py changes, I think I'll probably let that
sleeping dog lie.  I agree with you in general: cmd.py
probably shouldn't be rebinding any keys, but

1) the interface is "specify the completion key to get
completion",   so backwards compatibility raises its head

2) as you say, most people do probably rebind tab anyway

so I propose to check in your patchcmd, close the bug and
wait for someone to submit a bug report about pdb messing up
their bindings.

----------------------------------------------------------------------

Comment By: Michael Stone (mbrierst)
Date: 2003-01-29 13:19

Message:
Logged In: YES 
user_id=670441

Actually, the readline library does have the functionality to get the current binding of the tab key, so it could be saved and restored.  However, to do this in python some stuff would have to be added to the readline.c module, to create some python layer that deals with the way readline stores macros, functions, and keymaps.  It's probably not worth the effort unless someone wants to do major changes to the readline module to add the full functionality of the underlying library to the python interface. 

Probably cmd.py shouldn't be rebinding the tab key to complete at all.  If the user wants that, won't they have already done it somewhere else, like in their pythonrc or inputrc files?  And if they don't it might be impolite to change their bindings.

----------------------------------------------------------------------

Comment By: Michael Stone (mbrierst)
Date: 2003-01-29 10:27

Message:
Logged In: YES 
user_id=670441

I attached a changed readline.c patch to hopefully get rid of any "sickness".
(There was a not-checking-for-null problem too... ouch)

I think it's good now.  But note, there may not be a way to do this sort of thing perfectly.  If you mess around enough with starting debuggers within debuggers (or in general anything that uses a Cmd object) it might be possible to end up with an unexpected completer function in the end.

And it looks like under the current system there's no way to get back the completer key that was initially used.  But this isn't SUCH a big deal since I think everyone uses tab, right?

At any rate, the change is definitely an improvement for me.

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2003-01-29 05:36

Message:
Logged In: YES 
user_id=6656

I approve of this patch in principle.  I hadn't done enough
investigation to see that it was so easy to get the current
completer function!

OTOH:

return Py_BuildValue("O", completer);

is sick.  Change that to

Py_INCREF(completer);
return completer;

and if noone else howls, I'll check it in.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=676342&group_id=5470


From noreply@sourceforge.net  Fri Feb 21 07:43:34 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 20 Feb 2003 23:43:34 -0800
Subject: [Python-bugs-list] [ python-Bugs-501622 ] Missing trailing newline should not raise SyntaxError
Message-ID: <E18m7qI-0001CD-00@sc8-sf-web2.sourceforge.net>

Bugs item #501622, was opened at 2002-01-09 23:01
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=501622&group_id=5470

Category: Parser/Compiler
Group: None
>Status: Open
Resolution: Fixed
Priority: 5
Submitted By: David Bolen (db3l)
Assigned to: Guido van Rossum (gvanrossum)
Summary: Missing trailing newline should not raise SyntaxError

Initial Comment:
If you have a module that you wish to compile using 
the builtin compile() function (in 'exec' mode), it 
will fail with a SyntaxError if that module does not 
have a newline as its final token.

The same module can be executed directly by the 
interpreter, or imported by another module, and Python 
can properly compile and save a pyc for the module.

I believe the difference is rooted in the fact that 
the tokenizer (tokenizer.c, in tok_nextc()) 
will "fake" a newline at the end of a file if it 
doesn't find one, but it will not do so when 
tokenizing a string buffer.

What I'm not certain of is whether faking such a token 
for strings as well won't break something else (such 
as when parsing a string for an expression rather than 
a full module).  But without such a change, you have a 
state where a module that works (and compiles) in 
other circumstances cannot be read into memory and 
compiled with the compile() builtin.

This came up while tracking down a problem with 
failures using Gordan McMillan's Installer package 
which compiles modules using compile() before 
including them in the archive.

I believe this is true for all releases since at least 
1.5.2.

-- David

----------------------------------------------------------------------

>Comment By: Raymond Hettinger (rhettinger)
Date: 2003-02-21 02:43

Message:
Logged In: YES 
user_id=80475

Backport?

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-13 17:21

Message:
Logged In: YES 
user_id=6380

Fixed. This was simpler than I thought (only 4 lines in
parsetok.c) but also harder (codeop.py depend on the broken
behavior!) Most of the changes checked in had to do with
adding a flag to enable the old behavior. :-(

I note that Tim's second comment is visionary in this respect.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2002-03-22 19:35

Message:
Logged In: YES 
user_id=6380

Hm, adding it to builtin_compile isn't enough. We'd have to
add it to exec as well.

I think the lexer and/or parser should take care of this --
just as it should take care of accepting \r as well as \n as
well as \r\n.

Yes, it's hard to find. But there's got to be a way.

----------------------------------------------------------------------

Comment By: Neil Schemenauer (nascheme)
Date: 2002-03-22 18:14

Message:
Logged In: YES 
user_id=35752

I'm +1 on builtin_compile adding the newline.  It's the
lazy way out and it's better than every person hacking
with the parser stumbling into it and coming up with their
own work around.

Guido?

----------------------------------------------------------------------

Comment By: David Bolen (db3l)
Date: 2002-03-22 18:06

Message:
Logged In: YES 
user_id=53196

If compile() is being used in exec mode with a non-
terminated multi-line string, it's not going to work unless 
the application generates that copy itself in any event.  
So without an interpreter fix, I'd think the string copy is 
inevitable, and it might simplify things to have the 
builtin function take care of it.  It's something easy to 
overlook at the application level and could thus be fixed 
in one place rather than at each point of use.

On the other hand, I also noticed something I overlooked 
when first encountering the problem - the 2.2 docs added 
some text to compile() talking about this need for 
termination.  So it could be argued that it's now a 
documented restriction, and should the newline append (with 
any requisite string duplication) be needed, it leaves it 
to the individual applications rather than forcing it in 
the builtin.

Not to mention a documentation solution could thus be 
declared already done.


----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2002-03-22 18:01

Message:
Logged In: YES 
user_id=31435

Well, the user can't append an '\n' inplace either.  The 
question is whether we do that for them, or let it blow 
up.  OTOH, codeop.py has a lot of fun <wink> now trying to 
compile as-is, tben with one '\n' tacked on, then with two 
of 'em.  It would take me a long time to figure out exactly 
why it's doing all that, and to guess exactly how it would 
break.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2002-03-22 17:46

Message:
Logged In: YES 
user_id=6380

Probably, unless the start symbol is "expr" (which doesn't
need a newline).

But it would mean copying a potentially huge string -- we
can't append the \n in place.

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2002-03-22 17:41

Message:
Logged In: YES 
user_id=31435

Would it make sense for builtin_compile() to append a 
newline itself (say, if one weren't already present)?

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2002-03-22 17:20

Message:
Logged In: YES 
user_id=6380

> the tok_nextc code is hairy and whatever
> I tried broke something else.

That's exactly what happened to me when I tried to fix this
myself long ago. :-(

The workaround is simple enough: whoever calls compile()
should append a newline to the string just to be sure.

----------------------------------------------------------------------

Comment By: Neil Schemenauer (nascheme)
Date: 2002-03-22 17:07

Message:
Logged In: YES 
user_id=35752

I ran into this bug myself when writing the PTL compiler.
Here's a test case:

code = "def foo():\n  pass"
open("bug.py", "w").write(code)
import bug # works
compile(code, "<string>", "exec") # doesn't work

I traced this bug to tok_nextc.  If the input is coming from
a file and the last bit of input doesn't end with a newline
then one is faked.  This doesn't happen if the input is
coming from a string.  I spent time trying to figure out
how to fix it but the tok_nextc code is hairy and whatever
I tried broke something else.

----------------------------------------------------------------------

Comment By: Neil Schemenauer (nascheme)
Date: 2002-03-22 17:07

Message:
Logged In: YES 
user_id=35752

I ran into this bug myself when writing the PTL compiler.
Here's a test case:

code = "def foo():\n  pass"
open("bug.py", "w").write(code)
import bug # works
compile(code, "<string>", "exec") # doesn't work

I traced this bug to tok_nextc.  If the input is coming from
a file and the last bit of input doesn't end with a newline
then one is faked.  This doesn't happen if the input is
coming from a string.  I spent time trying to figure out
how to fix it but the tok_nextc code is hairy and whatever
I tried broke something else.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=501622&group_id=5470


From noreply@sourceforge.net  Fri Feb 21 07:44:19 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 20 Feb 2003 23:44:19 -0800
Subject: [Python-bugs-list] [ python-Bugs-690253 ] Strange control flow in PyInt_AsLong
Message-ID: <E18m7r1-0002GY-00@sc8-sf-web3.sourceforge.net>

Bugs item #690253, was opened at 2003-02-20 21:14
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690253&group_id=5470

Category: None
Group: None
Status: Closed
Resolution: Fixed
Priority: 5
Submitted By: Thomas Heller (theller)
Assigned to: Thomas Heller (theller)
Summary: Strange control flow in PyInt_AsLong

Initial Comment:
When nb_int is called inside the PyInt_AsLong function,
and this returns a long, the value is first retrieved
with PyLong_AsLong, but afterwards overwritten by a
call to PyInt_AS_LONG.

Patch:

*** intobject.c	12 Feb 2003 20:43:33 -0000	2.102
--- intobject.c	20 Feb 2003 20:05:32 -0000
***************
*** 162,171 ****
  		if (PyLong_Check(io)) {
  			/* got a long? => retry int conversion */
  			val = PyLong_AsLong((PyObject *)io);
! 			if (PyErr_Occurred()) {
! 				Py_DECREF(io);
  				return -1;
! 			}
  		}
  		else
  		{
--- 162,171 ----
  		if (PyLong_Check(io)) {
  			/* got a long? => retry int conversion */
  			val = PyLong_AsLong((PyObject *)io);
! 			Py_DECREF(io);
! 			if (PyErr_Occurred())
  				return -1;
! 			return val;
  		}
  		else
  		{



----------------------------------------------------------------------

>Comment By: Thomas Heller (theller)
Date: 2003-02-21 08:44

Message:
Logged In: YES 
user_id=11105

This change doesn't apply to the release22-maint branch,
because PyInt_AsLong doesn't accept Python longs anywhere.
Thanks.

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2003-02-20 22:34

Message:
Logged In: YES 
user_id=31435

Well, it's a small change, and clearly a good change -- it 
would go a lot faster to backport it than to think about it.

----------------------------------------------------------------------

Comment By: Thomas Heller (theller)
Date: 2003-02-20 22:16

Message:
Logged In: YES 
user_id=11105

Dunno either, I found it when reading the code, not when
hunting for a bug.

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2003-02-20 22:13

Message:
Logged In: YES 
user_id=31435

Dunno --didn't pay enough attention.  Was it an actual bug, or 
just a lack of grace?

----------------------------------------------------------------------

Comment By: Thomas Heller (theller)
Date: 2003-02-20 21:49

Message:
Logged In: YES 
user_id=11105

Checked in, with your improvement. I don't have a 2.2-maint
branch handy at the moment, is this also an issue there?

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2003-02-20 21:34

Message:
Logged In: YES 
user_id=31435

Sounds good to me.  As long as you're at it <wink>, don't call 
PyErr_Occurred() unless val is -1 either.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690253&group_id=5470


From noreply@sourceforge.net  Fri Feb 21 11:34:39 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Fri, 21 Feb 2003 03:34:39 -0800
Subject: [Python-bugs-list] [ python-Bugs-690611 ] String formatting error - From global scope
Message-ID: <E18mBRv-00068N-00@sc8-sf-web1.sourceforge.net>

Bugs item #690611, was opened at 2003-02-21 02:34
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690611&group_id=5470

Category: Parser/Compiler
Group: Python 2.2.2
Status: Open
Resolution: None
Priority: 5
Submitted By: Lars Petersen (daath)
Assigned to: Nobody/Anonymous (nobody)
Summary: String formatting error - From global scope

Initial Comment:
This small script fails:

s = 'hejsa'

def main():
    print '\n%(s)s' % vars()

main()

The error is KeyError: s - it works if you move s into the 
function scope...
This does not fail however:

s = 'hejsa'

def main():
    print '\n%s' % (s)

main()


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690611&group_id=5470


From noreply@sourceforge.net  Fri Feb 21 12:15:42 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Fri, 21 Feb 2003 04:15:42 -0800
Subject: [Python-bugs-list] [ python-Bugs-690622 ] test_cpickle overflows stack on MacOS9
Message-ID: <E18mC5e-0007rP-00@sc8-sf-web1.sourceforge.net>

Bugs item #690622, was opened at 2003-02-21 13:15
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690622&group_id=5470

Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Jack Jansen (jackjansen)
Assigned to: Nobody/Anonymous (nobody)
Summary: test_cpickle overflows stack on MacOS9

Initial Comment:
Test_cpickle overflows the stack and hangs the interpreter on MacOS9. This happens in test_nonrecursive_deep(), in the dumps() call.

I think there's a PyOS_CheckStack() call missing somewhere in cPickle.

Also, it's probably a good idea to lower the recursion to 50 (the test passes as long as the recusrion is less than 65 deep), at least on MacOS9. 

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690622&group_id=5470


From noreply@sourceforge.net  Fri Feb 21 12:42:40 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Fri, 21 Feb 2003 04:42:40 -0800
Subject: [Python-bugs-list] [ python-Bugs-676342 ] after using pdb readline does not work correctly
Message-ID: <E18mCVk-0000ME-00@sc8-sf-web1.sourceforge.net>

Bugs item #676342, was opened at 2003-01-28 15:53
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=676342&group_id=5470

Category: Python Library
Group: Python 2.2.2
Status: Open
Resolution: Accepted
Priority: 5
Submitted By: Michael Stone (mbrierst)
>Assigned to: Raymond Hettinger (rhettinger)
Summary: after using pdb readline does not work correctly

Initial Comment:

After I use pdb in the interpreter my readline settings are messed up.  It only knows about the pdb readline stuff forever afterward.  In fact, this happens as soon as a Pdb object is instantiated, regardless of whether or not you use it.

This is a result of some aspects of the Cmd object in Lib/cmd.py.

Currently Cmd registers a new readline completer as soon as a Cmd object is instantiated.  This is probably incorrect.  I believe the correct place to register the new handler is in the preloop hook.  That way the new readline completer is only used while the Cmd object is actually responsible for processing user input.

Next, the old readline completer should probably be re-registered in the postloop hook.  In order to do this, a new call must be added to the readline module to return the current completer so it can be saved and later restored.

The files included patch readline.c (version 2.41.64) and cmd.py (version 1.26.16.2) to make the changes described above.  After this patch readline works as expected in the interpreter.

Let me know what you think of it.

----------------------------------------------------------------------

>Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-21 07:42

Message:
Logged In: YES 
user_id=6380

Good idea.

----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2003-02-21 02:42

Message:
Logged In: YES 
user_id=80475

Backport?

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2003-02-03 06:11

Message:
Logged In: YES 
user_id=6656

Checked in as 

Lib/cmd.py revision 1.34

Thanks!  You probably didn't need to attach the patch three
times, though :-)

----------------------------------------------------------------------

Comment By: Michael Stone (mbrierst)
Date: 2003-02-01 01:41

Message:
Logged In: YES 
user_id=670441

I started using the latest cvs version today.
I have attached a diff to the latest cvs as patchcmd2 if that's
more convenient for you to apply.

----------------------------------------------------------------------

Comment By: Michael Stone (mbrierst)
Date: 2003-02-01 01:37

Message:
Logged In: YES 
user_id=670441

I started using the latest cvs version today.
I have attached a diff to the latest cvs as patchcmd2 if that's
more convenient for you to apply.

----------------------------------------------------------------------

Comment By: Michael Stone (mbrierst)
Date: 2003-01-31 16:25

Message:
Logged In: YES 
user_id=670441

I started using the latest cvs version today.
I have attached a diff to the latest cvs as patchcmd2 if that's
more convenient for you to apply.

----------------------------------------------------------------------

Comment By: Michael Stone (mbrierst)
Date: 2003-01-30 11:59

Message:
Logged In: YES 
user_id=670441

That sounds good to me.  Go ahead and check in patchcmd.

Thanks for your suggestions, I will follow them in future patches.  In this one, I was trying to follow the conventions I saw being used in the module, but they were outdated or incorrect (and I didn't always follow them correctly anyway!).  Okay that's definitely enough commentary for a few line patch.

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2003-01-30 05:22

Message:
Logged In: YES 
user_id=6656

Thanks for the new patch, it looks OK.  There are a couple
of things that spring to mind for future patches: 

- the function should be METH_NOARGS (or you should check
the args tuple)

- there should be a blank line between the argspec line in
the docstring and the description

- you should use the PyDoc_STRVAR macro (but this was only
used in the CVS trunk's version of readline).

I made these changes and checked in revision 2.59 of
Modules/readline.c.

Wrt the cmd.py changes, I think I'll probably let that
sleeping dog lie.  I agree with you in general: cmd.py
probably shouldn't be rebinding any keys, but

1) the interface is "specify the completion key to get
completion",   so backwards compatibility raises its head

2) as you say, most people do probably rebind tab anyway

so I propose to check in your patchcmd, close the bug and
wait for someone to submit a bug report about pdb messing up
their bindings.

----------------------------------------------------------------------

Comment By: Michael Stone (mbrierst)
Date: 2003-01-29 13:19

Message:
Logged In: YES 
user_id=670441

Actually, the readline library does have the functionality to get the current binding of the tab key, so it could be saved and restored.  However, to do this in python some stuff would have to be added to the readline.c module, to create some python layer that deals with the way readline stores macros, functions, and keymaps.  It's probably not worth the effort unless someone wants to do major changes to the readline module to add the full functionality of the underlying library to the python interface. 

Probably cmd.py shouldn't be rebinding the tab key to complete at all.  If the user wants that, won't they have already done it somewhere else, like in their pythonrc or inputrc files?  And if they don't it might be impolite to change their bindings.

----------------------------------------------------------------------

Comment By: Michael Stone (mbrierst)
Date: 2003-01-29 10:27

Message:
Logged In: YES 
user_id=670441

I attached a changed readline.c patch to hopefully get rid of any "sickness".
(There was a not-checking-for-null problem too... ouch)

I think it's good now.  But note, there may not be a way to do this sort of thing perfectly.  If you mess around enough with starting debuggers within debuggers (or in general anything that uses a Cmd object) it might be possible to end up with an unexpected completer function in the end.

And it looks like under the current system there's no way to get back the completer key that was initially used.  But this isn't SUCH a big deal since I think everyone uses tab, right?

At any rate, the change is definitely an improvement for me.

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2003-01-29 05:36

Message:
Logged In: YES 
user_id=6656

I approve of this patch in principle.  I hadn't done enough
investigation to see that it was so easy to get the current
completer function!

OTOH:

return Py_BuildValue("O", completer);

is sick.  Change that to

Py_INCREF(completer);
return completer;

and if noone else howls, I'll check it in.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=676342&group_id=5470


From noreply@sourceforge.net  Fri Feb 21 12:43:24 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Fri, 21 Feb 2003 04:43:24 -0800
Subject: [Python-bugs-list] [ python-Bugs-501622 ] Missing trailing newline should not raise SyntaxError
Message-ID: <E18mCWS-0000PY-00@sc8-sf-web1.sourceforge.net>

Bugs item #501622, was opened at 2002-01-09 23:01
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=501622&group_id=5470

Category: Parser/Compiler
Group: None
>Status: Closed
Resolution: Fixed
Priority: 5
Submitted By: David Bolen (db3l)
Assigned to: Guido van Rossum (gvanrossum)
Summary: Missing trailing newline should not raise SyntaxError

Initial Comment:
If you have a module that you wish to compile using 
the builtin compile() function (in 'exec' mode), it 
will fail with a SyntaxError if that module does not 
have a newline as its final token.

The same module can be executed directly by the 
interpreter, or imported by another module, and Python 
can properly compile and save a pyc for the module.

I believe the difference is rooted in the fact that 
the tokenizer (tokenizer.c, in tok_nextc()) 
will "fake" a newline at the end of a file if it 
doesn't find one, but it will not do so when 
tokenizing a string buffer.

What I'm not certain of is whether faking such a token 
for strings as well won't break something else (such 
as when parsing a string for an expression rather than 
a full module).  But without such a change, you have a 
state where a module that works (and compiles) in 
other circumstances cannot be read into memory and 
compiled with the compile() builtin.

This came up while tracking down a problem with 
failures using Gordan McMillan's Installer package 
which compiles modules using compile() before 
including them in the archive.

I believe this is true for all releases since at least 
1.5.2.

-- David

----------------------------------------------------------------------

>Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-21 07:43

Message:
Logged In: YES 
user_id=6380

No, too much code (have you seen the number of associated
checkins?)

----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2003-02-21 02:43

Message:
Logged In: YES 
user_id=80475

Backport?

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-13 17:21

Message:
Logged In: YES 
user_id=6380

Fixed. This was simpler than I thought (only 4 lines in
parsetok.c) but also harder (codeop.py depend on the broken
behavior!) Most of the changes checked in had to do with
adding a flag to enable the old behavior. :-(

I note that Tim's second comment is visionary in this respect.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2002-03-22 19:35

Message:
Logged In: YES 
user_id=6380

Hm, adding it to builtin_compile isn't enough. We'd have to
add it to exec as well.

I think the lexer and/or parser should take care of this --
just as it should take care of accepting \r as well as \n as
well as \r\n.

Yes, it's hard to find. But there's got to be a way.

----------------------------------------------------------------------

Comment By: Neil Schemenauer (nascheme)
Date: 2002-03-22 18:14

Message:
Logged In: YES 
user_id=35752

I'm +1 on builtin_compile adding the newline.  It's the
lazy way out and it's better than every person hacking
with the parser stumbling into it and coming up with their
own work around.

Guido?

----------------------------------------------------------------------

Comment By: David Bolen (db3l)
Date: 2002-03-22 18:06

Message:
Logged In: YES 
user_id=53196

If compile() is being used in exec mode with a non-
terminated multi-line string, it's not going to work unless 
the application generates that copy itself in any event.  
So without an interpreter fix, I'd think the string copy is 
inevitable, and it might simplify things to have the 
builtin function take care of it.  It's something easy to 
overlook at the application level and could thus be fixed 
in one place rather than at each point of use.

On the other hand, I also noticed something I overlooked 
when first encountering the problem - the 2.2 docs added 
some text to compile() talking about this need for 
termination.  So it could be argued that it's now a 
documented restriction, and should the newline append (with 
any requisite string duplication) be needed, it leaves it 
to the individual applications rather than forcing it in 
the builtin.

Not to mention a documentation solution could thus be 
declared already done.


----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2002-03-22 18:01

Message:
Logged In: YES 
user_id=31435

Well, the user can't append an '\n' inplace either.  The 
question is whether we do that for them, or let it blow 
up.  OTOH, codeop.py has a lot of fun <wink> now trying to 
compile as-is, tben with one '\n' tacked on, then with two 
of 'em.  It would take me a long time to figure out exactly 
why it's doing all that, and to guess exactly how it would 
break.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2002-03-22 17:46

Message:
Logged In: YES 
user_id=6380

Probably, unless the start symbol is "expr" (which doesn't
need a newline).

But it would mean copying a potentially huge string -- we
can't append the \n in place.

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2002-03-22 17:41

Message:
Logged In: YES 
user_id=31435

Would it make sense for builtin_compile() to append a 
newline itself (say, if one weren't already present)?

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2002-03-22 17:20

Message:
Logged In: YES 
user_id=6380

> the tok_nextc code is hairy and whatever
> I tried broke something else.

That's exactly what happened to me when I tried to fix this
myself long ago. :-(

The workaround is simple enough: whoever calls compile()
should append a newline to the string just to be sure.

----------------------------------------------------------------------

Comment By: Neil Schemenauer (nascheme)
Date: 2002-03-22 17:07

Message:
Logged In: YES 
user_id=35752

I ran into this bug myself when writing the PTL compiler.
Here's a test case:

code = "def foo():\n  pass"
open("bug.py", "w").write(code)
import bug # works
compile(code, "<string>", "exec") # doesn't work

I traced this bug to tok_nextc.  If the input is coming from
a file and the last bit of input doesn't end with a newline
then one is faked.  This doesn't happen if the input is
coming from a string.  I spent time trying to figure out
how to fix it but the tok_nextc code is hairy and whatever
I tried broke something else.

----------------------------------------------------------------------

Comment By: Neil Schemenauer (nascheme)
Date: 2002-03-22 17:07

Message:
Logged In: YES 
user_id=35752

I ran into this bug myself when writing the PTL compiler.
Here's a test case:

code = "def foo():\n  pass"
open("bug.py", "w").write(code)
import bug # works
compile(code, "<string>", "exec") # doesn't work

I traced this bug to tok_nextc.  If the input is coming from
a file and the last bit of input doesn't end with a newline
then one is faked.  This doesn't happen if the input is
coming from a string.  I spent time trying to figure out
how to fix it but the tok_nextc code is hairy and whatever
I tried broke something else.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=501622&group_id=5470


From noreply@sourceforge.net  Fri Feb 21 12:55:04 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Fri, 21 Feb 2003 04:55:04 -0800
Subject: [Python-bugs-list] [ python-Bugs-521782 ] unreliable file.read() error handling
Message-ID: <E18mChk-00011N-00@sc8-sf-web1.sourceforge.net>

Bugs item #521782, was opened at 2002-02-23 08:44
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=521782&group_id=5470

Category: Python Library
>Group: Python 2.2.2
>Status: Open
Resolution: Accepted
>Priority: 5
Submitted By: Marius Gedminas (mgedmin)
Assigned to: Gustavo Niemeyer (niemeyer)
Summary: unreliable file.read() error handling

Initial Comment:
fread(3) manual page states
       fread  and  fwrite return the number of items
successfully
       read or written (i.e., not the number of
characters).   If
       an error occurs, or the end-of-file is reached,
the return
       value is a short item count (or zero).

Python only checks ferror status when the return value
is zero (Objects/fileobject.c line 550 from
Python-2.1.2 sources).  I agree that it is a good idea
to delay exception throwing until after the user has
processed the partial chunk of data returned by fread,
but there are two problems with the current
implementation: loss of errno and occasional loss of data.

Both problems are illustrated with this scenario taken
from real life:

  suppose the file descriptor refers to a pipe, and we
set O_NONBLOCK mode with fcntl (the application was
reading from multiple pipes in a select() loop and
couldn't afford to block)
  fread(4096) returns an incomplete block and sets
errno to EAGAIN
  chunksize != 0 so we do not check ferror() and return
successfully
  the next time file.read() is called we reset errno
and do fread(4096) again.  It returns a full block
(i.e. bytesread == buffersize on line 559), so we
repeat the loop and call fread(0).  It returns 0, of
course.  Now we check ferror() and find it was set
(by a previous fread(4096) called maybe a century ago).
The errno information is already lost, so we throw
an IOError with errno=0.  And also lose that 4K chunk
of valuable user data.

Regarding solutions, I can see two alternatives:
- call clearerr(f->f_fp) just before fread(), where
Python currently sets errno = 0;  This makes sure that
we do not have stale ferror() flag and errno is valid,
but we might not notice some errors.  That doesn't
matter for EAGAIN, and for errors that occur reliably
if we repeat fread() on the same stream.  We might
still lose data if an exception is thrown on the second
or later loop iteration.
- always check for ferror() immediatelly after fread().
- regarding data loss, maybe it is possible to store
the errno somewhere inside the file object and delay
exception throwing if we have successfully read some
data (i.e. bytesread > 0).  The exception could be
thrown on the next call to file.read() before
performing anything else.


----------------------------------------------------------------------

>Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-21 07:55

Message:
Logged In: YES 
user_id=6380

Yes, please backport.

----------------------------------------------------------------------

Comment By: Gustavo Niemeyer (niemeyer)
Date: 2002-12-16 13:24

Message:
Logged In: YES 
user_id=7887

Applied as

Objects/fileobject.c: 2.172
Doc/lib/libstdtypes.tex: 1.114
Misc/NEWS: 1.563

Thank you!


----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2002-12-01 08:23

Message:
Logged In: YES 
user_id=21627

The patch looks good. Please apply it.

If we can both agree that this is a bug fix (as it arranges to 
return data which were previously silently lost), I suggest to 
backport the patch also to the 2.2 tree (no further review 
needed).

----------------------------------------------------------------------

Comment By: Gustavo Niemeyer (niemeyer)
Date: 2002-11-30 20:55

Message:
Logged In: YES 
user_id=7887

Thanks for noticing that Martin. Here is a new revision,
with suggested documentation changes.


----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2002-11-28 06:00

Message:
Logged In: YES 
user_id=21627

I'm still missing proposed documentation changes.

----------------------------------------------------------------------

Comment By: Gustavo Niemeyer (niemeyer)
Date: 2002-11-27 10:58

Message:
Logged In: YES 
user_id=7887

I'm attaching a new revised patch. I'll also increase the
priority, to reflect the serious problems this bug could cause.

----------------------------------------------------------------------

Comment By: Gustavo Niemeyer (niemeyer)
Date: 2002-11-18 11:18

Message:
Logged In: YES 
user_id=7887

You're right about EAGAIN. The behavior I mentioned seems to
be valid only for old Unixes, as presented here:

http://www.gnu.org/manual/glibc/html_mono/libc.html

So indeed we should check for both. OTOH, the above URL also
mentions that EAGAIN can happen in blocking situations as well:

"On some systems, reading a large amount of data from a
character special file can also fail with EAGAIN if the
kernel cannot find enough physical memory to lock down the
user's pages."

Also, the statement in the documentation you mention isn't
true even without my patch. We currently check for
"readbytes < requestedbytes" and break the loop if it
happens. Indeed, that's one of the reasons that
created the second problem I've mentioned. Enforcing this
behavior, I could see that mentioned in the URL above:

"Any condition that could result in EAGAIN can instead
result in a successful read which returns fewer bytes than
requested. Calling read again immediately would result in
EAGAIN."

Thusly, applying this patch we wouldn't be introducing a new
behavior, but just enforcing a single behavior in every case.


----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2002-11-18 09:18

Message:
Logged In: YES 
user_id=21627

Depends on what system you have. On Linux-i386, we have

#define EWOULDBLOCK     EAGAIN

so it is necessarily the same thing. Can you report on what
system EAGAIN happens even if the file descriptor is
non-blocking?

As for documentation changes: The documentation currently says

   Read at most size bytes from the file (less if the read
hits EOF before obtaining size bytes)

I believe this statement is no longer true: with your
changes, it will return less data even without hitting EOF.

----------------------------------------------------------------------

Comment By: Gustavo Niemeyer (niemeyer)
Date: 2002-11-18 08:50

Message:
Logged In: YES 
user_id=7887

Martin, EAGAIN is not the same as EWOULDBLOCK. While
EWOULDBLOCK will only be issued in a situation where the
filedescriptor is marked as non-blocking, EAGAIN can happen
at any time, including when requestsize == -1 (read
everything from the fd).

If we're going to handle EAGAIN, we must ensure that the
loop is not broken when it happens, instead of using the
same solution proposed for EWOULDBLOCK.

Do you think we should do that as well?

You're right about EWOULDBLOCK not being available. I'll
include the required #ifdef, and also the suggested comment.

Could you please clarify what you mean in this sentence: "In
any case, please update the documentation to describe the
special cases that you add.". IMO, the proposed patch is not
adding special cases. It's just fixing the algorithm to
behave the same way in all situations. In that case, what
doucmentation should be updated, in your opinion? Or perhaps
you're talking about the proposed e.data solution, which
would be used in other cases like EINTR similars?

Thank you!

----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2002-11-14 03:29

Message:
Logged In: YES 
user_id=21627

I think the processing of the error condition is wrong, in a
number of ways:
- Posix allows for both EAGAIN and EWOULDBLOCK to be
signalled in this case.
- Neither EAGAIN nor EWOULDBLOCK are guaranteed to exist on
all systems.
- Please add a comment to the break; to indicate what the
purpose of this code is.

Also, I think there are a number of other cases where
bytesread might be non-zero, e.g. if EINTR was signalled.
It's not clear what the best solution would be, I propose
that the exception carries an attribute "data" to denote the
short data. Of course, in the EINTR case, the original
exception is lost and a KeyboardInterrupt is raised instead,
so putting the data into the exception might be pointless...

In any case, please update the documentation to describe the
special cases that you add.

----------------------------------------------------------------------

Comment By: Gustavo Niemeyer (niemeyer)
Date: 2002-11-12 11:43

Message:
Logged In: YES 
user_id=7887

Martin, could you please review that?

----------------------------------------------------------------------

Comment By: Gustavo Niemeyer (niemeyer)
Date: 2002-11-12 10:58

Message:
Logged In: YES 
user_id=7887

Great catch Marius! Thank you very much!

I could identify the following problems from your report:

- If (requestsize == -1) and after a few iterations, fread()
  signals EWOULDBLOCK with (chunksize == 0), all read
  data is lost because an exception is raised.

- When (chunksize != 0) and EWOULDBLOCK is signaled,
  ferror() will be flagged. Then the next iteration will
fail if:
  1) read data has length 0 because EWOULDBLOCK is
  signaled again but errno is still set to 0 (I don't know why
  errno is not reset in that case); 2) read data has length 0
  because a block with the exact requested size is read.

The attached tests show these problems. The first problem
is a little bit harder to trigger. I had to preset SMALLCHUNK
to 4096 in fileobject.c, because my Linux refused to return
a chunk greater than that from fread(). The second problem
should be visible without further tweakings.

I have also attached a solution to the problem. This solution
works because:

- If (chunksize == 0), (errno == EWOULDBLOCK) and
  (readbytes > 0), there's no point in raising an exception, as
  we got some data for the user, just like when
  (bytesread < buffersize).

- When (chunksize != 0) and EWOULDBLOCK is signaled, it's
  not considered an error. OTOH, ferror() is still being set.
  clearerr() should be called in the stream before breaking the
  loop.

- If (bytesread == buffersize) and (bytesrequest >= 0), it means
  that we already got all the requested data, and thus
there's no
  point in trying to read more data from the file and forcing
  (chunksize == 0) without ferror() being set.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=521782&group_id=5470


From noreply@sourceforge.net  Fri Feb 21 12:58:27 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Fri, 21 Feb 2003 04:58:27 -0800
Subject: [Python-bugs-list] [ python-Bugs-595026 ] Support for masks in getargs.c
Message-ID: <E18mCl1-0003yX-00@sc8-sf-web2.sourceforge.net>

Bugs item #595026, was opened at 2002-08-14 08:26
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=595026&group_id=5470

Category: Python Interpreter Core
Group: Python 2.3
Status: Open
Resolution: None
>Priority: 7
Submitted By: Guido van Rossum (gvanrossum)
>Assigned to: Thomas Heller (theller)
Summary: Support for masks in getargs.c

Initial Comment:
We need this implemented:

> How about the following counterproposal.  This also
changes some of
> the other format codes to be a little more regular.
>
> Code    C type          	Range check
>
> b	unsigned char		0..UCHAR_MAX
> B	unsigned char		none **
> h	unsigned short		0..USHRT_MAX
> H	unsigned short		none **
> i	int
		INT_MIN..INT_MAX
> I *	unsigned int		0..UINT_MAX
> l	long
		LONG_MIN..LONG_MAX
> k *	unsigned long		none
> L	long long		LLONG_MIN..LLONG_MAX
> K *	unsigned long long	none
>
> Notes:
>
> * New format codes.
>
> ** Changed from previous "range-and-a-half" to
"none"; the
>    range-and-a-half checking wasn't particularly useful.

Plus a C API or two, e.g. PyInt_AsLongMask() ->
unsigned long and PyInt_AsLongLongMask() -> unsigned
long long (if that exists).

----------------------------------------------------------------------

>Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-21 07:58

Message:
Logged In: YES 
user_id=6380

I suggest writing on python-dev or python-list.


----------------------------------------------------------------------

Comment By: Thomas Heller (theller)
Date: 2003-02-20 15:45

Message:
Logged In: YES 
user_id=11105

I've implemented the 'k' getargs code, and
PyInt_AsUnsignedLongMask and PyLong_AsUnsignedLongMask
functions.  Is there any facility which would help me to
test this new code?

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-02-18 17:08

Message:
Logged In: YES 
user_id=45365

Guido,
I would be happy with the no-rangecheck unsigned long formatcode. One request, though: if this is implemented can it be done ASAP after 2.3a2, so there's still some time to shake out the bugs before 2.3b1 comes out? Especially the hand-written code will have to be examined to see where l needs to be turned into k.

----------------------------------------------------------------------

Comment By: Thomas Heller (theller)
Date: 2003-02-18 15:59

Message:
Logged In: YES 
user_id=11105

Currently the h code means signed short
(SHRT_MIN..SHRT_MAX), do you really want to change that to
unsigned short (0..USHRT_MAX)?

----------------------------------------------------------------------

Comment By: Thomas Heller (theller)
Date: 2003-02-18 15:33

Message:
Logged In: YES 
user_id=11105

Ok, I'll use your codes.

Here is my suggestion for the C api functions:

int PyInt_AsLongMask(PyObject *v, unsigned long *pval);
int PyInt_AsLongLongMask(PyObject *v, unsigned LONG_LONG *pval);

return -1 and set exception on error, return 0 otherwise and
store the result in pval. This saves the PyErr_Occurred() in
case the value is -1.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-18 15:18

Message:
Logged In: YES 
user_id=6380

A few days is fine (this doesn't need to make it into 2.3a2).

But the proposal here is not backwards compatible, is it?
Those codes already mean something different now.

I think we'll need to invent new format codes, or a new
modifier.

----------------------------------------------------------------------

Comment By: Thomas Heller (theller)
Date: 2003-02-18 15:16

Message:
Logged In: YES 
user_id=11105

But it would probably take a few days.

What about these changes to the format codes, they are now
more in sync with the struct module (I hope the formatting
is kept):

Code    C type          	Range check

b	unsigned char		0..UCHAR_MAX
B	unsigned char		none **
h	unsigned short		0..USHRT_MAX
H	unsigned short		none **
i	int			INT_MIN..INT_MAX
I *	unsigned int		0..UINT_MAX
l	long			LONG_MIN..LONG_MAX
L *	unsigned long		none
q	long long		LLONG_MIN..LLONG_MAX
Q *	unsigned long long	none


----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-18 14:50

Message:
Logged In: YES 
user_id=6380

Thomas: that would be great! (As long it isn't killed by bw
compat. :-)


----------------------------------------------------------------------

Comment By: Thomas Heller (theller)
Date: 2003-02-18 14:18

Message:
Logged In: YES 
user_id=11105

If nobody else comes up, I can do this. I also had similar
checks for the struct module, but Tim killed it because of
backward compatibility.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=595026&group_id=5470


From noreply@sourceforge.net  Fri Feb 21 15:16:13 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Fri, 21 Feb 2003 07:16:13 -0800
Subject: [Python-bugs-list] [ python-Bugs-458874 ] Long module names fail on the Mac
Message-ID: <E18mEuL-0004pB-00@sc8-sf-web3.sourceforge.net>

Bugs item #458874, was opened at 2001-09-05 22:20
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=458874&group_id=5470

Category: Macintosh
Group: None
>Status: Closed
>Resolution: Wont Fix
Priority: 2
Submitted By: Jack Jansen (jackjansen)
Assigned to: Jack Jansen (jackjansen)
Summary: Long module names fail on the Mac

Initial Comment:
Long module names fail on the Mac. Or, more precisely, test_repr fails (if you take out
the "if" that disables the testing of long module names on the Mac).

It needs to be investigated whether it is the mkdir() calls and such in test_repr which
are the cause of the problem (in which case the problem is really with the test: methinks
platforms are free to raise EINVAL for "illegal" filenames in I/O calls).

The other question is what the Python specs say about the number of significant characters
in a module name, and how to handle modules that have names longer than the underlying
storage system supports. The language reference simply says that identifiers are unlimited in
length, but on Unix "unlimited" for modulenames means "255" (and probably something
similar on Windows).

This last paragraph is the reason I'm assigning this to Guido, I can't start on a fix until I
know what I should aim for. Feel free to assign it back (unless you want to fix it yourself:-).


----------------------------------------------------------------------

>Comment By: Jack Jansen (jackjansen)
Date: 2003-02-21 16:16

Message:
Logged In: YES 
user_id=45365

Let's be frank about it: this isn't going to be fixed.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2001-09-05 22:32

Message:
Logged In: YES 
user_id=6380

Hm...  I think that an import of a really long module name
might truncate the name that it looks for on the filesystem;
that's what Windows used to do in 8+3 mode AFAIK. But this
is really up to the platform; I haven't really thought that
much about the mapping of long module names.

But I think it's OK for a mkdir() or open() to fail if a
component of the name is too long -- filename syntax is up
to the OS.

For a fix of the test, I would simply change the really long
package name to a slightly shorter one (say, 28 chars long
to allow for the ".pyc" suffix).


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=458874&group_id=5470


From noreply@sourceforge.net  Fri Feb 21 17:52:59 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Fri, 21 Feb 2003 09:52:59 -0800
Subject: [Python-bugs-list] [ python-Bugs-676342 ] after using pdb readline does not work correctly
Message-ID: <E18mHM3-00011L-00@sc8-sf-web4.sourceforge.net>

Bugs item #676342, was opened at 2003-01-28 20:53
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=676342&group_id=5470

Category: Python Library
Group: Python 2.2.2
Status: Open
Resolution: Accepted
Priority: 5
Submitted By: Michael Stone (mbrierst)
Assigned to: Raymond Hettinger (rhettinger)
Summary: after using pdb readline does not work correctly

Initial Comment:

After I use pdb in the interpreter my readline settings are messed up.  It only knows about the pdb readline stuff forever afterward.  In fact, this happens as soon as a Pdb object is instantiated, regardless of whether or not you use it.

This is a result of some aspects of the Cmd object in Lib/cmd.py.

Currently Cmd registers a new readline completer as soon as a Cmd object is instantiated.  This is probably incorrect.  I believe the correct place to register the new handler is in the preloop hook.  That way the new readline completer is only used while the Cmd object is actually responsible for processing user input.

Next, the old readline completer should probably be re-registered in the postloop hook.  In order to do this, a new call must be added to the readline module to return the current completer so it can be saved and later restored.

The files included patch readline.c (version 2.41.64) and cmd.py (version 1.26.16.2) to make the changes described above.  After this patch readline works as expected in the interpreter.

Let me know what you think of it.

----------------------------------------------------------------------

>Comment By: Michael Stone (mbrierst)
Date: 2003-02-21 17:52

Message:
Logged In: YES 
user_id=670441

I'm attaching backported versions against
cmd.py 1.26.16.3
readline.c 2.41.6.4
if you want them.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-21 12:42

Message:
Logged In: YES 
user_id=6380

Good idea.

----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2003-02-21 07:42

Message:
Logged In: YES 
user_id=80475

Backport?

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2003-02-03 11:11

Message:
Logged In: YES 
user_id=6656

Checked in as 

Lib/cmd.py revision 1.34

Thanks!  You probably didn't need to attach the patch three
times, though :-)

----------------------------------------------------------------------

Comment By: Michael Stone (mbrierst)
Date: 2003-02-01 06:41

Message:
Logged In: YES 
user_id=670441

I started using the latest cvs version today.
I have attached a diff to the latest cvs as patchcmd2 if that's
more convenient for you to apply.

----------------------------------------------------------------------

Comment By: Michael Stone (mbrierst)
Date: 2003-02-01 06:37

Message:
Logged In: YES 
user_id=670441

I started using the latest cvs version today.
I have attached a diff to the latest cvs as patchcmd2 if that's
more convenient for you to apply.

----------------------------------------------------------------------

Comment By: Michael Stone (mbrierst)
Date: 2003-01-31 21:25

Message:
Logged In: YES 
user_id=670441

I started using the latest cvs version today.
I have attached a diff to the latest cvs as patchcmd2 if that's
more convenient for you to apply.

----------------------------------------------------------------------

Comment By: Michael Stone (mbrierst)
Date: 2003-01-30 16:59

Message:
Logged In: YES 
user_id=670441

That sounds good to me.  Go ahead and check in patchcmd.

Thanks for your suggestions, I will follow them in future patches.  In this one, I was trying to follow the conventions I saw being used in the module, but they were outdated or incorrect (and I didn't always follow them correctly anyway!).  Okay that's definitely enough commentary for a few line patch.

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2003-01-30 10:22

Message:
Logged In: YES 
user_id=6656

Thanks for the new patch, it looks OK.  There are a couple
of things that spring to mind for future patches: 

- the function should be METH_NOARGS (or you should check
the args tuple)

- there should be a blank line between the argspec line in
the docstring and the description

- you should use the PyDoc_STRVAR macro (but this was only
used in the CVS trunk's version of readline).

I made these changes and checked in revision 2.59 of
Modules/readline.c.

Wrt the cmd.py changes, I think I'll probably let that
sleeping dog lie.  I agree with you in general: cmd.py
probably shouldn't be rebinding any keys, but

1) the interface is "specify the completion key to get
completion",   so backwards compatibility raises its head

2) as you say, most people do probably rebind tab anyway

so I propose to check in your patchcmd, close the bug and
wait for someone to submit a bug report about pdb messing up
their bindings.

----------------------------------------------------------------------

Comment By: Michael Stone (mbrierst)
Date: 2003-01-29 18:19

Message:
Logged In: YES 
user_id=670441

Actually, the readline library does have the functionality to get the current binding of the tab key, so it could be saved and restored.  However, to do this in python some stuff would have to be added to the readline.c module, to create some python layer that deals with the way readline stores macros, functions, and keymaps.  It's probably not worth the effort unless someone wants to do major changes to the readline module to add the full functionality of the underlying library to the python interface. 

Probably cmd.py shouldn't be rebinding the tab key to complete at all.  If the user wants that, won't they have already done it somewhere else, like in their pythonrc or inputrc files?  And if they don't it might be impolite to change their bindings.

----------------------------------------------------------------------

Comment By: Michael Stone (mbrierst)
Date: 2003-01-29 15:27

Message:
Logged In: YES 
user_id=670441

I attached a changed readline.c patch to hopefully get rid of any "sickness".
(There was a not-checking-for-null problem too... ouch)

I think it's good now.  But note, there may not be a way to do this sort of thing perfectly.  If you mess around enough with starting debuggers within debuggers (or in general anything that uses a Cmd object) it might be possible to end up with an unexpected completer function in the end.

And it looks like under the current system there's no way to get back the completer key that was initially used.  But this isn't SUCH a big deal since I think everyone uses tab, right?

At any rate, the change is definitely an improvement for me.

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2003-01-29 10:36

Message:
Logged In: YES 
user_id=6656

I approve of this patch in principle.  I hadn't done enough
investigation to see that it was so easy to get the current
completer function!

OTOH:

return Py_BuildValue("O", completer);

is sick.  Change that to

Py_INCREF(completer);
return completer;

and if noone else howls, I'll check it in.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=676342&group_id=5470


From noreply@sourceforge.net  Fri Feb 21 18:29:39 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Fri, 21 Feb 2003 10:29:39 -0800
Subject: [Python-bugs-list] [ python-Bugs-690309 ] 2.3a2 Sol8 make fails at _iconv_codec.
Message-ID: <E18mHvX-0004BG-00@sc8-sf-web2.sourceforge.net>

Bugs item #690309, was opened at 2003-02-20 22:53
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690309&group_id=5470

Category: Build
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Inyeol Lee (inyeol)
Assigned to: Nobody/Anonymous (nobody)
Summary: 2.3a2 Sol8 make fails at _iconv_codec.

Initial Comment:
Python 2.3a2 build process fails during building
_iconv_codec
module. The message is;

[...]
gcc -shared
build/temp.solaris-2.8-sun4u-2.3/_iconv_codec.o
-L/home/inyeol/util/python/lib -L/usr/local/lib -o
build/lib.solaris-2.8-sun4u-2.3/_iconv_codec.so
Traceback (most recent call last):

  [... long traceback here...]

  File "./setup.py", line 209, in build_extension
    imp.load_dynamic(ext.name, ext_filename)
RuntimeError: can't initialize the _iconv_codec module:
iconv_open() failed
*** Error code 1
make: Fatal error: Command failed for target
`sharedmods'

My build environment is gcc2.95.3 with Solaris 8
linker.
I've built 2.2, 2,2,1, 2,2,2, 2,3a1 with the same env
and there
was no (major) issues like this. I attached my
configure and
make log messages.


----------------------------------------------------------------------

>Comment By: Walter Dörwald (doerwalter)
Date: 2003-02-21 19:29

Message:
Logged In: YES 
user_id=89016

I changed the init function to use 'ISO8859-1' instead of
'ASCII' as the test encoding. Can someone test whether this
fixes anything?

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-20 23:15

Message:
Logged In: YES 
user_id=33168

In order to get the build to proceed, you can change line
210 of setup.py from:

        except ImportError, why:
to:
        except (ImportError, RuntimeError), why:

This is just before:
    self.announce('*** WARNING: renaming "%s" since
importing it' ...)

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690309&group_id=5470


From noreply@sourceforge.net  Fri Feb 21 18:36:15 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Fri, 21 Feb 2003 10:36:15 -0800
Subject: [Python-bugs-list] [ python-Bugs-595026 ] Support for masks in getargs.c
Message-ID: <E18mI1v-0002k1-00@sc8-sf-web4.sourceforge.net>

Bugs item #595026, was opened at 2002-08-14 14:26
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=595026&group_id=5470

Category: Python Interpreter Core
Group: Python 2.3
Status: Open
Resolution: None
Priority: 7
Submitted By: Guido van Rossum (gvanrossum)
Assigned to: Thomas Heller (theller)
Summary: Support for masks in getargs.c

Initial Comment:
We need this implemented:

> How about the following counterproposal.  This also
changes some of
> the other format codes to be a little more regular.
>
> Code    C type          	Range check
>
> b	unsigned char		0..UCHAR_MAX
> B	unsigned char		none **
> h	unsigned short		0..USHRT_MAX
> H	unsigned short		none **
> i	int
		INT_MIN..INT_MAX
> I *	unsigned int		0..UINT_MAX
> l	long
		LONG_MIN..LONG_MAX
> k *	unsigned long		none
> L	long long		LLONG_MIN..LLONG_MAX
> K *	unsigned long long	none
>
> Notes:
>
> * New format codes.
>
> ** Changed from previous "range-and-a-half" to
"none"; the
>    range-and-a-half checking wasn't particularly useful.

Plus a C API or two, e.g. PyInt_AsLongMask() ->
unsigned long and PyInt_AsLongLongMask() -> unsigned
long long (if that exists).

----------------------------------------------------------------------

>Comment By: Thomas Heller (theller)
Date: 2003-02-21 19:36

Message:
Logged In: YES 
user_id=11105

Uploading patch whcih implements the 'k' format code.  Any
comments? Is this what is needed?
I know, docs are missing, and 'K' is missing.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-21 13:58

Message:
Logged In: YES 
user_id=6380

I suggest writing on python-dev or python-list.


----------------------------------------------------------------------

Comment By: Thomas Heller (theller)
Date: 2003-02-20 21:45

Message:
Logged In: YES 
user_id=11105

I've implemented the 'k' getargs code, and
PyInt_AsUnsignedLongMask and PyLong_AsUnsignedLongMask
functions.  Is there any facility which would help me to
test this new code?

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-02-18 23:08

Message:
Logged In: YES 
user_id=45365

Guido,
I would be happy with the no-rangecheck unsigned long formatcode. One request, though: if this is implemented can it be done ASAP after 2.3a2, so there's still some time to shake out the bugs before 2.3b1 comes out? Especially the hand-written code will have to be examined to see where l needs to be turned into k.

----------------------------------------------------------------------

Comment By: Thomas Heller (theller)
Date: 2003-02-18 21:59

Message:
Logged In: YES 
user_id=11105

Currently the h code means signed short
(SHRT_MIN..SHRT_MAX), do you really want to change that to
unsigned short (0..USHRT_MAX)?

----------------------------------------------------------------------

Comment By: Thomas Heller (theller)
Date: 2003-02-18 21:33

Message:
Logged In: YES 
user_id=11105

Ok, I'll use your codes.

Here is my suggestion for the C api functions:

int PyInt_AsLongMask(PyObject *v, unsigned long *pval);
int PyInt_AsLongLongMask(PyObject *v, unsigned LONG_LONG *pval);

return -1 and set exception on error, return 0 otherwise and
store the result in pval. This saves the PyErr_Occurred() in
case the value is -1.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-18 21:18

Message:
Logged In: YES 
user_id=6380

A few days is fine (this doesn't need to make it into 2.3a2).

But the proposal here is not backwards compatible, is it?
Those codes already mean something different now.

I think we'll need to invent new format codes, or a new
modifier.

----------------------------------------------------------------------

Comment By: Thomas Heller (theller)
Date: 2003-02-18 21:16

Message:
Logged In: YES 
user_id=11105

But it would probably take a few days.

What about these changes to the format codes, they are now
more in sync with the struct module (I hope the formatting
is kept):

Code    C type          	Range check

b	unsigned char		0..UCHAR_MAX
B	unsigned char		none **
h	unsigned short		0..USHRT_MAX
H	unsigned short		none **
i	int			INT_MIN..INT_MAX
I *	unsigned int		0..UINT_MAX
l	long			LONG_MIN..LONG_MAX
L *	unsigned long		none
q	long long		LLONG_MIN..LLONG_MAX
Q *	unsigned long long	none


----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-18 20:50

Message:
Logged In: YES 
user_id=6380

Thomas: that would be great! (As long it isn't killed by bw
compat. :-)


----------------------------------------------------------------------

Comment By: Thomas Heller (theller)
Date: 2003-02-18 20:18

Message:
Logged In: YES 
user_id=11105

If nobody else comes up, I can do this. I also had similar
checks for the struct module, but Tim killed it because of
backward compatibility.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=595026&group_id=5470


From noreply@sourceforge.net  Fri Feb 21 19:55:24 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Fri, 21 Feb 2003 11:55:24 -0800
Subject: [Python-bugs-list] [ python-Bugs-690622 ] test_cpickle overflows stack on MacOS9
Message-ID: <E18mJGW-0000Sz-00@sc8-sf-web2.sourceforge.net>

Bugs item #690622, was opened at 2003-02-21 07:15
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690622&group_id=5470

Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Jack Jansen (jackjansen)
>Assigned to: Guido van Rossum (gvanrossum)
Summary: test_cpickle overflows stack on MacOS9

Initial Comment:
Test_cpickle overflows the stack and hangs the interpreter on MacOS9. This happens in test_nonrecursive_deep(), in the dumps() call.

I think there's a PyOS_CheckStack() call missing somewhere in cPickle.

Also, it's probably a good idea to lower the recursion to 50 (the test passes as long as the recusrion is less than 65 deep), at least on MacOS9. 

----------------------------------------------------------------------

>Comment By: Tim Peters (tim_one)
Date: 2003-02-21 14:55

Message:
Logged In: YES 
user_id=31435

There are no PyOS_CheckStack()  calls in cPickle, and I'm 
not sure anyone really wants to bother adding them (cPickle 
is meant to be high-performance, not safe <wink>).

That test is consuming more stack frames than it used to, 
because each level of list pickling now bites another stack 
frame to call the new "batch list elements" routine.

Reducing to 50 doesn't fly because cPickle's internal 
PY_CPICKLE_FAST_LIMIT #define happens to be 50 now, 
and the purpose of test_nonrecursive_deep is to be sure that 
non-cyclic containers get pickled OK in a "fast" pickler even if 
they're nested as deeply as PY_CPICKLE_FAST_LIMIT.

Assigning to Guido for further cogitation on his day off <wink>.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690622&group_id=5470


From noreply@sourceforge.net  Fri Feb 21 20:01:41 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Fri, 21 Feb 2003 12:01:41 -0800
Subject: [Python-bugs-list] [ python-Bugs-690622 ] test_cpickle overflows stack on MacOS9
Message-ID: <E18mJMb-0000ld-00@sc8-sf-web2.sourceforge.net>

Bugs item #690622, was opened at 2003-02-21 07:15
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690622&group_id=5470

Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Jack Jansen (jackjansen)
>Assigned to: Jack Jansen (jackjansen)
Summary: test_cpickle overflows stack on MacOS9

Initial Comment:
Test_cpickle overflows the stack and hangs the interpreter on MacOS9. This happens in test_nonrecursive_deep(), in the dumps() call.

I think there's a PyOS_CheckStack() call missing somewhere in cPickle.

Also, it's probably a good idea to lower the recursion to 50 (the test passes as long as the recusrion is less than 65 deep), at least on MacOS9. 

----------------------------------------------------------------------

>Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-21 15:01

Message:
Logged In: YES 
user_id=6380

I agree that I'd rather not see CheckStack calls added to
cPickle. But using 60 instead of 100 should work for both of
you, right?

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2003-02-21 14:55

Message:
Logged In: YES 
user_id=31435

There are no PyOS_CheckStack()  calls in cPickle, and I'm 
not sure anyone really wants to bother adding them (cPickle 
is meant to be high-performance, not safe <wink>).

That test is consuming more stack frames than it used to, 
because each level of list pickling now bites another stack 
frame to call the new "batch list elements" routine.

Reducing to 50 doesn't fly because cPickle's internal 
PY_CPICKLE_FAST_LIMIT #define happens to be 50 now, 
and the purpose of test_nonrecursive_deep is to be sure that 
non-cyclic containers get pickled OK in a "fast" pickler even if 
they're nested as deeply as PY_CPICKLE_FAST_LIMIT.

Assigning to Guido for further cogitation on his day off <wink>.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690622&group_id=5470


From noreply@sourceforge.net  Fri Feb 21 20:04:17 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Fri, 21 Feb 2003 12:04:17 -0800
Subject: [Python-bugs-list] [ python-Bugs-690622 ] test_cpickle overflows stack on MacOS9
Message-ID: <E18mJP7-0004F0-00@sc8-sf-web1.sourceforge.net>

Bugs item #690622, was opened at 2003-02-21 07:15
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690622&group_id=5470

Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Jack Jansen (jackjansen)
Assigned to: Jack Jansen (jackjansen)
Summary: test_cpickle overflows stack on MacOS9

Initial Comment:
Test_cpickle overflows the stack and hangs the interpreter on MacOS9. This happens in test_nonrecursive_deep(), in the dumps() call.

I think there's a PyOS_CheckStack() call missing somewhere in cPickle.

Also, it's probably a good idea to lower the recursion to 50 (the test passes as long as the recusrion is less than 65 deep), at least on MacOS9. 

----------------------------------------------------------------------

>Comment By: Tim Peters (tim_one)
Date: 2003-02-21 15:04

Message:
Logged In: YES 
user_id=31435

Right, AFAICT 60 is enough to test what this is trying to test 
(although I'm not the "fast pickler" expert -- but I think it's 
clear enough).  Jack, please confirm that 60 is OK for you?

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-21 15:01

Message:
Logged In: YES 
user_id=6380

I agree that I'd rather not see CheckStack calls added to
cPickle. But using 60 instead of 100 should work for both of
you, right?

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2003-02-21 14:55

Message:
Logged In: YES 
user_id=31435

There are no PyOS_CheckStack()  calls in cPickle, and I'm 
not sure anyone really wants to bother adding them (cPickle 
is meant to be high-performance, not safe <wink>).

That test is consuming more stack frames than it used to, 
because each level of list pickling now bites another stack 
frame to call the new "batch list elements" routine.

Reducing to 50 doesn't fly because cPickle's internal 
PY_CPICKLE_FAST_LIMIT #define happens to be 50 now, 
and the purpose of test_nonrecursive_deep is to be sure that 
non-cyclic containers get pickled OK in a "fast" pickler even if 
they're nested as deeply as PY_CPICKLE_FAST_LIMIT.

Assigning to Guido for further cogitation on his day off <wink>.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690622&group_id=5470


From noreply@sourceforge.net  Fri Feb 21 20:24:14 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Fri, 21 Feb 2003 12:24:14 -0800
Subject: [Python-bugs-list] [ python-Bugs-690622 ] test_cpickle overflows stack on MacOS9
Message-ID: <E18mJiQ-00086f-00@sc8-sf-web4.sourceforge.net>

Bugs item #690622, was opened at 2003-02-21 07:15
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690622&group_id=5470

Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Jack Jansen (jackjansen)
Assigned to: Jack Jansen (jackjansen)
Summary: test_cpickle overflows stack on MacOS9

Initial Comment:
Test_cpickle overflows the stack and hangs the interpreter on MacOS9. This happens in test_nonrecursive_deep(), in the dumps() call.

I think there's a PyOS_CheckStack() call missing somewhere in cPickle.

Also, it's probably a good idea to lower the recursion to 50 (the test passes as long as the recusrion is less than 65 deep), at least on MacOS9. 

----------------------------------------------------------------------

>Comment By: Tim Peters (tim_one)
Date: 2003-02-21 15:24

Message:
Logged In: YES 
user_id=31435

I reduced the nesting depth to 60, in Lib/test/test_cpickle.py, 
rev 1.14.  Jack, please close this bug as Fixed if your 
problem goes away now.

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2003-02-21 15:04

Message:
Logged In: YES 
user_id=31435

Right, AFAICT 60 is enough to test what this is trying to test 
(although I'm not the "fast pickler" expert -- but I think it's 
clear enough).  Jack, please confirm that 60 is OK for you?

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-21 15:01

Message:
Logged In: YES 
user_id=6380

I agree that I'd rather not see CheckStack calls added to
cPickle. But using 60 instead of 100 should work for both of
you, right?

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2003-02-21 14:55

Message:
Logged In: YES 
user_id=31435

There are no PyOS_CheckStack()  calls in cPickle, and I'm 
not sure anyone really wants to bother adding them (cPickle 
is meant to be high-performance, not safe <wink>).

That test is consuming more stack frames than it used to, 
because each level of list pickling now bites another stack 
frame to call the new "batch list elements" routine.

Reducing to 50 doesn't fly because cPickle's internal 
PY_CPICKLE_FAST_LIMIT #define happens to be 50 now, 
and the purpose of test_nonrecursive_deep is to be sure that 
non-cyclic containers get pickled OK in a "fast" pickler even if 
they're nested as deeply as PY_CPICKLE_FAST_LIMIT.

Assigning to Guido for further cogitation on his day off <wink>.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690622&group_id=5470


From noreply@sourceforge.net  Sat Feb 22 00:06:40 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Fri, 21 Feb 2003 16:06:40 -0800
Subject: [Python-bugs-list] [ python-Bugs-690974 ] re.LOCALE, umlaut and \w
Message-ID: <E18mNBg-0005bA-00@sc8-sf-web1.sourceforge.net>

Bugs item #690974, was opened at 2003-02-22 00:06
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690974&group_id=5470

Category: Regular Expressions
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: peter nordlund (peterno)
Assigned to: Fredrik Lundh (effbot)
Summary: re.LOCALE, umlaut and \w

Initial Comment:
I submit this problem although I am not sure it is
a real bug. It could be that I don't know how this
locale stuff works.

Anyway, I have been browsing around quite some time on
the net to find some
good examples of code demonstating how to use regexp in
python to get hold
of åäö when using \w, but I have not found any complete
examples.

If the code below behaves correctly, I suggest that the
regexp documentation
is improved by adding a complete example that shows how
to use re.LOCALE.
(The code behaves in the same way with python 2.2.2.)

#----------------------------------------
import locale
locale.setlocale(locale.LC_ALL,'swedish')
import re
reguml=re.compile(r"[a-zä]", re.LOCALE) # I expect
reguml and regw to give the same result.
regw=re.compile(r"\w", re.LOCALE)
reguml2=re.compile(r"[a-zä]+", re.LOCALE) # I expect
reguml2 and regw2 to give the same result.
regw2=re.compile(r"[\w]+", re.LOCALE)
str="abcä d\344e ä f ";

print reguml.findall(str) # Behaves as I expect.
print regw.findall(str) # Here I expect same result as
above, but I don't get it.
print reguml2.findall(str) # Behaves as I expect.
print regw2.findall(str) # Behaves as I expect.
#----------------------------------------



>>> import locale
>>> locale.setlocale(locale.LC_ALL,'swedish')
'swedish'
>>> import re
>>> reguml=re.compile(r"[a-zä]", re.LOCALE) # I expect
reguml and regw to give the same result.
>>> regw=re.compile(r"\w", re.LOCALE)
>>> reguml2=re.compile(r"[a-zä]+", re.LOCALE) # I
expect reguml2 and regw2 to give the same result.
>>> regw2=re.compile(r"[\w]+", re.LOCALE)
>>> str="abcä d\344e ä f ";
>>>
>>> print reguml.findall(str) # Behaves as I expect.
['a', 'b', 'c', '\xe4', 'd', '\xe4', 'e', '\xe4', 'f']
>>> print regw.findall(str) # Here I expect same result
as above, but I don't get it.
['a', 'b', 'c', 'd', 'e', 'f']
>>> print reguml2.findall(str) # Behaves as I expect.
['abc\xe4', 'd\xe4e', '\xe4', 'f']
>>> print regw2.findall(str) # Behaves as I expect.
['abc\xe4', 'd\xe4e', '\xe4', 'f']
---------------------------------------------------------
peternl:Python-2.3a2>>
/work1/pkg/dev-tools/python/2.3a2/bin/python -V
Python 2.3a2
peternl:Python-2.3a2>>uname -a
Linux peternl.computervision.se 2.4.18-6mdk-petern #2
Thu May 23 06:40:30 CEST 2002 i686 unknown


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690974&group_id=5470


From noreply@sourceforge.net  Sat Feb 22 00:43:44 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Fri, 21 Feb 2003 16:43:44 -0800
Subject: [Python-bugs-list] [ python-Bugs-690611 ] String formatting error - From global scope
Message-ID: <E18mNlY-0003eb-00@sc8-sf-web3.sourceforge.net>

Bugs item #690611, was opened at 2003-02-21 05:34
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690611&group_id=5470

Category: Parser/Compiler
Group: Python 2.2.2
Status: Open
Resolution: None
Priority: 5
Submitted By: Lars Petersen (daath)
Assigned to: Nobody/Anonymous (nobody)
Summary: String formatting error - From global scope

Initial Comment:
This small script fails:

s = 'hejsa'

def main():
    print '\n%(s)s' % vars()

main()

The error is KeyError: s - it works if you move s into the 
function scope...
This does not fail however:

s = 'hejsa'

def main():
    print '\n%s' % (s)

main()


----------------------------------------------------------------------

Comment By: logistix (logistix)
Date: 2003-02-21 18:43

Message:
Logged In: YES 
user_id=699438

This isn't a bug.  vars() only returns local variables.  If you did:

def main():
    print '\n%(s)s' % globals()

if would work.

>>> help(vars)
Help on built-in function vars:

vars(...)
    vars([object]) -> dictionary
    
    Without arguments, equivalent to locals().
    With an argument, equivalent to object.__dict__.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690611&group_id=5470


From noreply@sourceforge.net  Sat Feb 22 02:24:58 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Fri, 21 Feb 2003 18:24:58 -0800
Subject: [Python-bugs-list] [ python-Bugs-691005 ] _POSIX_C_SOURCE redefined
Message-ID: <E18mPLW-0004He-00@sc8-sf-web4.sourceforge.net>

Bugs item #691005, was opened at 2003-02-21 19:24
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=691005&group_id=5470

Category: Python Library
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Mike Romberg (romberg)
Assigned to: Nobody/Anonymous (nobody)
Summary: _POSIX_C_SOURCE redefined

Initial Comment:
  Not really a bug.  Just a warning which might be
easy to cleanup with an ifndef.  If the following simple
file is compiled with gcc-3.2.2 on redhat-8.0 (i686) using
the c++ compiler, you get the following warning:

--------------------- test.C -------------------
#include <iostream>
#include <Python.h>
-----------------------------------------------

In file included from
/usr/local/python-2.3a2/include/python2.3/Python.h:8,
                 from test.C:2:
/usr/local/python-2.3a2/include/python2.3/pyconfig.h:819:1:
warning: "_POSIX_C_SOURCE" redefined
In file included from
/usr/local/gcc-3.2.2/include/c++/3.2.2/i686-pc-linux-gnu/bits/os_defines.h:39,
                 from
/usr/local/gcc-3.2.2/include/c++/3.2.2/i686-pc-linux-gnu/bits/c++config.h:34,
                 from
/usr/local/gcc-3.2.2/include/c++/3.2.2/iostream:44,
                 from test.C:1:
/usr/include/features.h:131:1: warning: this is the
location of the previous definition




----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=691005&group_id=5470


From noreply@sourceforge.net  Sat Feb 22 03:51:32 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Fri, 21 Feb 2003 19:51:32 -0800
Subject: [Python-bugs-list] [ python-Bugs-690611 ] String formatting error - From global scope
Message-ID: <E18mQhI-0000uj-00@sc8-sf-web2.sourceforge.net>

Bugs item #690611, was opened at 2003-02-21 06:34
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690611&group_id=5470

Category: Parser/Compiler
>Group: Not a Bug
>Status: Closed
>Resolution: Invalid
Priority: 5
Submitted By: Lars Petersen (daath)
Assigned to: Nobody/Anonymous (nobody)
Summary: String formatting error - From global scope

Initial Comment:
This small script fails:

s = 'hejsa'

def main():
    print '\n%(s)s' % vars()

main()

The error is KeyError: s - it works if you move s into the 
function scope...
This does not fail however:

s = 'hejsa'

def main():
    print '\n%s' % (s)

main()


----------------------------------------------------------------------

>Comment By: Tim Peters (tim_one)
Date: 2003-02-21 22:51

Message:
Logged In: YES 
user_id=31435

logistix's explanation is on target, so closing this as 
NotABug.  Python should probably grow another function 
representing the entire visible namespace via a dict-like 
object, but that would be a new feature request (and not a 
bug).

----------------------------------------------------------------------

Comment By: logistix (logistix)
Date: 2003-02-21 19:43

Message:
Logged In: YES 
user_id=699438

This isn't a bug.  vars() only returns local variables.  If you did:

def main():
    print '\n%(s)s' % globals()

if would work.

>>> help(vars)
Help on built-in function vars:

vars(...)
    vars([object]) -> dictionary
    
    Without arguments, equivalent to locals().
    With an argument, equivalent to object.__dict__.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690611&group_id=5470


From noreply@sourceforge.net  Sat Feb 22 10:19:48 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sat, 22 Feb 2003 02:19:48 -0800
Subject: [Python-bugs-list] [ python-Bugs-690611 ] String formatting error - From global scope
Message-ID: <E18mWl2-0006UF-00@sc8-sf-web4.sourceforge.net>

Bugs item #690611, was opened at 2003-02-21 02:34
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690611&group_id=5470

Category: Parser/Compiler
Group: Not a Bug
Status: Closed
Resolution: Invalid
Priority: 5
Submitted By: Lars Petersen (daath)
Assigned to: Nobody/Anonymous (nobody)
Summary: String formatting error - From global scope

Initial Comment:
This small script fails:

s = 'hejsa'

def main():
    print '\n%(s)s' % vars()

main()

The error is KeyError: s - it works if you move s into the 
function scope...
This does not fail however:

s = 'hejsa'

def main():
    print '\n%s' % (s)

main()


----------------------------------------------------------------------

>Comment By: Lars Petersen (daath)
Date: 2003-02-22 01:19

Message:
Logged In: YES 
user_id=133121

Aaaah! Thank you for your swift responses :)
Python is really nice :)

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2003-02-21 18:51

Message:
Logged In: YES 
user_id=31435

logistix's explanation is on target, so closing this as 
NotABug.  Python should probably grow another function 
representing the entire visible namespace via a dict-like 
object, but that would be a new feature request (and not a 
bug).

----------------------------------------------------------------------

Comment By: logistix (logistix)
Date: 2003-02-21 15:43

Message:
Logged In: YES 
user_id=699438

This isn't a bug.  vars() only returns local variables.  If you did:

def main():
    print '\n%(s)s' % globals()

if would work.

>>> help(vars)
Help on built-in function vars:

vars(...)
    vars([object]) -> dictionary
    
    Without arguments, equivalent to locals().
    With an argument, equivalent to object.__dict__.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690611&group_id=5470


From noreply@sourceforge.net  Sat Feb 22 10:43:08 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sat, 22 Feb 2003 02:43:08 -0800
Subject: [Python-bugs-list] [ python-Bugs-691005 ] _POSIX_C_SOURCE redefined
Message-ID: <E18mX7c-0005p5-00@sc8-sf-web1.sourceforge.net>

Bugs item #691005, was opened at 2003-02-22 02:24
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=691005&group_id=5470

Category: Python Library
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Mike Romberg (romberg)
Assigned to: Nobody/Anonymous (nobody)
Summary: _POSIX_C_SOURCE redefined

Initial Comment:
  Not really a bug.  Just a warning which might be
easy to cleanup with an ifndef.  If the following simple
file is compiled with gcc-3.2.2 on redhat-8.0 (i686) using
the c++ compiler, you get the following warning:

--------------------- test.C -------------------
#include <iostream>
#include <Python.h>
-----------------------------------------------

In file included from
/usr/local/python-2.3a2/include/python2.3/Python.h:8,
                 from test.C:2:
/usr/local/python-2.3a2/include/python2.3/pyconfig.h:819:1:
warning: "_POSIX_C_SOURCE" redefined
In file included from
/usr/local/gcc-3.2.2/include/c++/3.2.2/i686-pc-linux-gnu/bits/os_defines.h:39,
                 from
/usr/local/gcc-3.2.2/include/c++/3.2.2/i686-pc-linux-gnu/bits/c++config.h:34,
                 from
/usr/local/gcc-3.2.2/include/c++/3.2.2/iostream:44,
                 from test.C:1:
/usr/include/features.h:131:1: warning: this is the
location of the previous definition




----------------------------------------------------------------------

>Comment By: Michael Hudson (mwh)
Date: 2003-02-22 10:43

Message:
Logged In: YES 
user_id=6656

It's a really bad idea to include system headers *before*
including Python.h.

Do you still get the errors if you reverse the order?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=691005&group_id=5470


From noreply@sourceforge.net  Sat Feb 22 11:21:12 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sat, 22 Feb 2003 03:21:12 -0800
Subject: [Python-bugs-list] [ python-Bugs-589427 ] standard include paths on command line
Message-ID: <E18mXiS-00073j-00@sc8-sf-web1.sourceforge.net>

Bugs item #589427, was opened at 2002-08-01 00:13
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=589427&group_id=5470

Category: Build
Group: Python 2.3
Status: Open
Resolution: Remind
Priority: 5
Submitted By: Matthias Klose (doko)
Assigned to: Skip Montanaro (montanaro)
Summary: standard include paths on command line

Initial Comment:
when compiling shared modules (using gcc-3.1.1), 
standard include paths are included (by setup.py?) in 
the include path, which can change the search order. 
gcc-3.1 warns about:

building 'xreadlines' extension
gcc-3.1 -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -
fPIC -I. -I/home/packages/pyt
hon2.3/python2.3-2.2.90/./Include -I/usr/include -
I/usr/local/include -I/home/pa
ckages/python2.3/python2.3-2.2.90/Include -
I/home/packages/python2.3/python2.3-2
.2.90/build-static -
c /home/packages/python2.3/python2.3-
2.2.90/Modules/xreadlin
esmodule.c -o build/temp.linux-i686-
2.3/xreadlinesmodule.o
gcc -shared build/temp.linux-i686-
2.3/xreadlinesmodule.o -L/usr/lib -L/usr/local
/lib -o build/lib.linux-i686-2.3/xreadlines.so
building 'array' extension
gcc-3.1 -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -
fPIC -I. -I/hocc1: warning: c
hanging search order for system 
directory "/usr/local/include"
cc1: warning:   as it has already been specified as a non-
system directory
cc1: warning: changing search order for system 
directory "/usr/include"
cc1: warning:   as it has already been specified as a non-
system directory
cc1: warning: changing search order for system 
directory "/usr/local/include"
cc1: warning:   as it has already been specified as a non-
system directory
cc1: warning: changing search order for system 
directory "/usr/include"
cc1: warning:   as it has already been specified as a non-
system directory
cc1: warning: changing search order for system 
directory "/usr/local/include"
cc1: warning:   as it has already been specified as a non-
system directory

In this case, /usr/include and /usr/local/include should 
not be included on the command line.


----------------------------------------------------------------------

>Comment By: Matthias Klose (doko)
Date: 2003-02-22 11:21

Message:
Logged In: YES 
user_id=60903

Neil,

the /usr/include problems are solved now (in 2.3a2). I
didn't check for the /usr/local/include problems again,
because for the Debian build, I know that /usr/local/include
is the locale include dir anyway and commented out the two
lines in setup.py.

I think, the report can be closed.

     Matthias


----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-07 22:18

Message:
Logged In: YES 
user_id=33168

Matthias, I seem to recall that the warnings were due to a
change in gcc or a bug.  I think I heard that it would be
fixed in gcc 3.2.2.  Do you still have any problems?

----------------------------------------------------------------------

Comment By: Matthias Klose (doko)
Date: 2002-08-20 21:32

Message:
Logged In: YES 
user_id=60903

Two more problems:

On Debian, the db headers are installed in /usr/include.

- This directory is not searched and bsddb not built.

- Adding '/usr/include' to setup.py in the db setup,
   the header is found, but /usr/include added to
   the command line.

One unrelated thing is that the bsddb setup adds
-R/usr/lib, which is not support on Linux by gcc-3.2.


----------------------------------------------------------------------

Comment By: Skip Montanaro (montanaro)
Date: 2002-08-14 02:29

Message:
Logged In: YES 
user_id=44345

Doko, are you still having problems?  I really have nothing to go on here.  I 
don't see /usr/include turning up on the command line and don't have a 
Debian system to try things out on. (Does the SF compile farm?)

Skip


----------------------------------------------------------------------

Comment By: Skip Montanaro (montanaro)
Date: 2002-08-05 15:25

Message:
Logged In: YES 
user_id=44345

Sorry guys, I'm coming in on this late.  

I'm done with the bsddb trickery.  In any case /usr/include is not added 
as a -I argument under any circumstances in the bsddb build section of 
setup.py.  Various Linux vendors allow you to install multiple versions of 
Berkeley DB simultaneously.  The only way you can guarantee that a 
particular version is compiled against and linked with is to add the 
appropriate -I and -L flags to the command line.  On RedHat-derived 
systems the include files are in /usr/include/db{1,2,3,4} if installed as part 
of a system install, while if you install from Sleepycat source I believe it's 
/usr/local/BerkeleyDB<version>/include.   I can't really get rid of the 
RH-related include directories. 

Wait a minute.  I take some of that back.  *If* you are foolish enough to 
uncomment the section of code which enables compilation and linkage 
with Berkeley DB 1.85, you might wind up with -I/usr/include on the 
command line to build the bsddb module.  I will correct that (make it so 
/usr/include isn't added to the -I flag) and also reorder the directory 
search so system include directories are scanned after non-system 
directories.

How's this going to help the OP?  I thought the problem he saw was 
more pervasive (xreadlines, _ssl, etc).



----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2002-08-05 14:51

Message:
Logged In: YES 
user_id=6656

Skip, are you still working on the bsddb related bits of
setup.py?  Can you look into how /usr/include is getting
onto the commandline?


----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2002-08-04 12:47

Message:
Logged In: YES 
user_id=21627

I can't reproduce this. With the current CVS, on Debian,
with --prefix=/usr, I get

gcc -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -fPIC
-DWITH_APPINIT=1 -I/usr/include/tcl8.3 -I/usr/X11R6/include
-I. -I/vol/marvin-vol8/loewis/python/dist/src/./Include
-I/usr/local/include
-I/vol/marvin-vol8/loewis/python/dist/src/Include
-I/vol/marvin-vol8/loewis/python/dist/src -c
/vol/marvin-vol8/loewis/python/dist/src/Modules/_tkinter.c
-o build/temp.linux-i686-2.3/_tkinter.o

So it won't add -I/usr/include to the command line options.

----------------------------------------------------------------------

Comment By: Matthias Klose (doko)
Date: 2002-08-03 16:30

Message:
Logged In: YES 
user_id=60903

No. It's:

 - bsddb
 - dbm
 - _tkinter

all modules, which are installed in /usr at least on Debian
GNU/Linux (but maybe on other distros as well).

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2002-08-03 16:19

Message:
Logged In: YES 
user_id=6656

It's only the _ssl module, right?  I've fixed that.

The -R/usr/lib and -L/usr/lib's come from various database
modules, the configuration of which is under review anyway.
 I hope those doing the reviewing have seen this bug traffic...

----------------------------------------------------------------------

Comment By: Matthias Klose (doko)
Date: 2002-08-03 15:56

Message:
Logged In: YES 
user_id=60903

build log attached. please tell me if you need an account on
a Debian machine (doko   debian org).
configure \
        --with-gcc=gcc-3.1 \
        --with-cxx=g++-3.1 \
        --prefix=/usr \
        --with-fpectl \
        --enable-ipv6


----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2002-08-03 15:12

Message:
Logged In: YES 
user_id=6656

Really?  Nuts.  They're not on my machine.

What arguments are you passing to configure?

----------------------------------------------------------------------

Comment By: Matthias Klose (doko)
Date: 2002-08-02 21:48

Message:
Logged In: YES 
user_id=60903

Thanks for the fix, however when building shared modules,
-I/usr/include and -L/usr/lib are still passed to the compiler.


----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2002-08-02 14:05

Message:
Logged In: YES 
user_id=21627

And indeed, we should not remove /usr/local/include, since
the system compiler may not be gcc.

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2002-08-02 13:56

Message:
Logged In: YES 
user_id=6656

OK, done, in

setup.py revision 1.100

I don't see us removing /usr/local/include from the command
line.

----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2002-08-02 13:49

Message:
Logged In: YES 
user_id=21627

The patch looks fine to me, please apply it.

I still think that the bogus warning should be removed from
gcc entirely.

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2002-08-02 12:54

Message:
Logged In: YES 
user_id=6656

OK, can you try the attached patch?

Martin, does this look OK to you?

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2002-08-02 08:53

Message:
Logged In: YES 
user_id=6656

Ah, sorry!

Hmm.  Now I don't see the problem, i.e. for me
-I/usr/include is not part of the command line passed to the
compiler.

Are you building from up-to-date CVS?  Though I don't recall
any recent changes in this area.  Ahh, you're running
".../configure --prefix=/usr", right?

I'll dig.

----------------------------------------------------------------------

Comment By: Matthias Klose (doko)
Date: 2002-08-01 20:23

Message:
Logged In: YES 
user_id=60903

I disagree. The discussion is on /usr/local/include. We are 
talking about /usr/include, which is a system include dir on 
almost all systems.

See the thread "gcc 3.2's cpp breaks configure scripts" at 
http://gcc.gnu.org/ml/gcc/2002-07/, i.e.:

http://gcc.gnu.org/ml/gcc/2002-07/msg01527.html

Inclusion of /usr/include can lead to errors, as explained in:
http://gcc.gnu.org/ml/gcc/2002-07/msg01541.html

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2002-08-01 09:08

Message:
Logged In: YES 
user_id=6656

This has been deemed a bug in gcc, see this thread:

http://mail.python.org/pipermail/python-dev/2002-June/025990.html

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=589427&group_id=5470


From noreply@sourceforge.net  Sat Feb 22 15:41:30 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sat, 22 Feb 2003 07:41:30 -0800
Subject: [Python-bugs-list] [ python-Bugs-589427 ] standard include paths on command line
Message-ID: <E18mbmM-0003lB-00@sc8-sf-web3.sourceforge.net>

Bugs item #589427, was opened at 2002-07-31 20:13
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=589427&group_id=5470

Category: Build
Group: Python 2.3
>Status: Closed
>Resolution: Fixed
Priority: 5
Submitted By: Matthias Klose (doko)
Assigned to: Skip Montanaro (montanaro)
Summary: standard include paths on command line

Initial Comment:
when compiling shared modules (using gcc-3.1.1), 
standard include paths are included (by setup.py?) in 
the include path, which can change the search order. 
gcc-3.1 warns about:

building 'xreadlines' extension
gcc-3.1 -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -
fPIC -I. -I/home/packages/pyt
hon2.3/python2.3-2.2.90/./Include -I/usr/include -
I/usr/local/include -I/home/pa
ckages/python2.3/python2.3-2.2.90/Include -
I/home/packages/python2.3/python2.3-2
.2.90/build-static -
c /home/packages/python2.3/python2.3-
2.2.90/Modules/xreadlin
esmodule.c -o build/temp.linux-i686-
2.3/xreadlinesmodule.o
gcc -shared build/temp.linux-i686-
2.3/xreadlinesmodule.o -L/usr/lib -L/usr/local
/lib -o build/lib.linux-i686-2.3/xreadlines.so
building 'array' extension
gcc-3.1 -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -
fPIC -I. -I/hocc1: warning: c
hanging search order for system 
directory "/usr/local/include"
cc1: warning:   as it has already been specified as a non-
system directory
cc1: warning: changing search order for system 
directory "/usr/include"
cc1: warning:   as it has already been specified as a non-
system directory
cc1: warning: changing search order for system 
directory "/usr/local/include"
cc1: warning:   as it has already been specified as a non-
system directory
cc1: warning: changing search order for system 
directory "/usr/include"
cc1: warning:   as it has already been specified as a non-
system directory
cc1: warning: changing search order for system 
directory "/usr/local/include"
cc1: warning:   as it has already been specified as a non-
system directory

In this case, /usr/include and /usr/local/include should 
not be included on the command line.


----------------------------------------------------------------------

>Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-22 10:41

Message:
Logged In: YES 
user_id=33168

Thanks, Matthias.  Closing.


----------------------------------------------------------------------

Comment By: Matthias Klose (doko)
Date: 2003-02-22 06:21

Message:
Logged In: YES 
user_id=60903

Neil,

the /usr/include problems are solved now (in 2.3a2). I
didn't check for the /usr/local/include problems again,
because for the Debian build, I know that /usr/local/include
is the locale include dir anyway and commented out the two
lines in setup.py.

I think, the report can be closed.

     Matthias


----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-07 17:18

Message:
Logged In: YES 
user_id=33168

Matthias, I seem to recall that the warnings were due to a
change in gcc or a bug.  I think I heard that it would be
fixed in gcc 3.2.2.  Do you still have any problems?

----------------------------------------------------------------------

Comment By: Matthias Klose (doko)
Date: 2002-08-20 17:32

Message:
Logged In: YES 
user_id=60903

Two more problems:

On Debian, the db headers are installed in /usr/include.

- This directory is not searched and bsddb not built.

- Adding '/usr/include' to setup.py in the db setup,
   the header is found, but /usr/include added to
   the command line.

One unrelated thing is that the bsddb setup adds
-R/usr/lib, which is not support on Linux by gcc-3.2.


----------------------------------------------------------------------

Comment By: Skip Montanaro (montanaro)
Date: 2002-08-13 22:29

Message:
Logged In: YES 
user_id=44345

Doko, are you still having problems?  I really have nothing to go on here.  I 
don't see /usr/include turning up on the command line and don't have a 
Debian system to try things out on. (Does the SF compile farm?)

Skip


----------------------------------------------------------------------

Comment By: Skip Montanaro (montanaro)
Date: 2002-08-05 11:25

Message:
Logged In: YES 
user_id=44345

Sorry guys, I'm coming in on this late.  

I'm done with the bsddb trickery.  In any case /usr/include is not added 
as a -I argument under any circumstances in the bsddb build section of 
setup.py.  Various Linux vendors allow you to install multiple versions of 
Berkeley DB simultaneously.  The only way you can guarantee that a 
particular version is compiled against and linked with is to add the 
appropriate -I and -L flags to the command line.  On RedHat-derived 
systems the include files are in /usr/include/db{1,2,3,4} if installed as part 
of a system install, while if you install from Sleepycat source I believe it's 
/usr/local/BerkeleyDB<version>/include.   I can't really get rid of the 
RH-related include directories. 

Wait a minute.  I take some of that back.  *If* you are foolish enough to 
uncomment the section of code which enables compilation and linkage 
with Berkeley DB 1.85, you might wind up with -I/usr/include on the 
command line to build the bsddb module.  I will correct that (make it so 
/usr/include isn't added to the -I flag) and also reorder the directory 
search so system include directories are scanned after non-system 
directories.

How's this going to help the OP?  I thought the problem he saw was 
more pervasive (xreadlines, _ssl, etc).



----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2002-08-05 10:51

Message:
Logged In: YES 
user_id=6656

Skip, are you still working on the bsddb related bits of
setup.py?  Can you look into how /usr/include is getting
onto the commandline?


----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2002-08-04 08:47

Message:
Logged In: YES 
user_id=21627

I can't reproduce this. With the current CVS, on Debian,
with --prefix=/usr, I get

gcc -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -fPIC
-DWITH_APPINIT=1 -I/usr/include/tcl8.3 -I/usr/X11R6/include
-I. -I/vol/marvin-vol8/loewis/python/dist/src/./Include
-I/usr/local/include
-I/vol/marvin-vol8/loewis/python/dist/src/Include
-I/vol/marvin-vol8/loewis/python/dist/src -c
/vol/marvin-vol8/loewis/python/dist/src/Modules/_tkinter.c
-o build/temp.linux-i686-2.3/_tkinter.o

So it won't add -I/usr/include to the command line options.

----------------------------------------------------------------------

Comment By: Matthias Klose (doko)
Date: 2002-08-03 12:30

Message:
Logged In: YES 
user_id=60903

No. It's:

 - bsddb
 - dbm
 - _tkinter

all modules, which are installed in /usr at least on Debian
GNU/Linux (but maybe on other distros as well).

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2002-08-03 12:19

Message:
Logged In: YES 
user_id=6656

It's only the _ssl module, right?  I've fixed that.

The -R/usr/lib and -L/usr/lib's come from various database
modules, the configuration of which is under review anyway.
 I hope those doing the reviewing have seen this bug traffic...

----------------------------------------------------------------------

Comment By: Matthias Klose (doko)
Date: 2002-08-03 11:56

Message:
Logged In: YES 
user_id=60903

build log attached. please tell me if you need an account on
a Debian machine (doko   debian org).
configure \
        --with-gcc=gcc-3.1 \
        --with-cxx=g++-3.1 \
        --prefix=/usr \
        --with-fpectl \
        --enable-ipv6


----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2002-08-03 11:12

Message:
Logged In: YES 
user_id=6656

Really?  Nuts.  They're not on my machine.

What arguments are you passing to configure?

----------------------------------------------------------------------

Comment By: Matthias Klose (doko)
Date: 2002-08-02 17:48

Message:
Logged In: YES 
user_id=60903

Thanks for the fix, however when building shared modules,
-I/usr/include and -L/usr/lib are still passed to the compiler.


----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2002-08-02 10:05

Message:
Logged In: YES 
user_id=21627

And indeed, we should not remove /usr/local/include, since
the system compiler may not be gcc.

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2002-08-02 09:56

Message:
Logged In: YES 
user_id=6656

OK, done, in

setup.py revision 1.100

I don't see us removing /usr/local/include from the command
line.

----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2002-08-02 09:49

Message:
Logged In: YES 
user_id=21627

The patch looks fine to me, please apply it.

I still think that the bogus warning should be removed from
gcc entirely.

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2002-08-02 08:54

Message:
Logged In: YES 
user_id=6656

OK, can you try the attached patch?

Martin, does this look OK to you?

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2002-08-02 04:53

Message:
Logged In: YES 
user_id=6656

Ah, sorry!

Hmm.  Now I don't see the problem, i.e. for me
-I/usr/include is not part of the command line passed to the
compiler.

Are you building from up-to-date CVS?  Though I don't recall
any recent changes in this area.  Ahh, you're running
".../configure --prefix=/usr", right?

I'll dig.

----------------------------------------------------------------------

Comment By: Matthias Klose (doko)
Date: 2002-08-01 16:23

Message:
Logged In: YES 
user_id=60903

I disagree. The discussion is on /usr/local/include. We are 
talking about /usr/include, which is a system include dir on 
almost all systems.

See the thread "gcc 3.2's cpp breaks configure scripts" at 
http://gcc.gnu.org/ml/gcc/2002-07/, i.e.:

http://gcc.gnu.org/ml/gcc/2002-07/msg01527.html

Inclusion of /usr/include can lead to errors, as explained in:
http://gcc.gnu.org/ml/gcc/2002-07/msg01541.html

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2002-08-01 05:08

Message:
Logged In: YES 
user_id=6656

This has been deemed a bug in gcc, see this thread:

http://mail.python.org/pipermail/python-dev/2002-June/025990.html

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=589427&group_id=5470


From noreply@sourceforge.net  Sat Feb 22 17:15:07 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sat, 22 Feb 2003 09:15:07 -0800
Subject: [Python-bugs-list] [ python-Bugs-690974 ] re.LOCALE, umlaut and \w
Message-ID: <E18mdEx-0002SZ-00@sc8-sf-web1.sourceforge.net>

Bugs item #690974, was opened at 2003-02-21 15:06
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690974&group_id=5470

Category: Regular Expressions
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: peter nordlund (peterno)
Assigned to: Fredrik Lundh (effbot)
Summary: re.LOCALE, umlaut and \w

Initial Comment:
I submit this problem although I am not sure it is
a real bug. It could be that I don't know how this
locale stuff works.

Anyway, I have been browsing around quite some time on
the net to find some
good examples of code demonstating how to use regexp in
python to get hold
of åäö when using \w, but I have not found any complete
examples.

If the code below behaves correctly, I suggest that the
regexp documentation
is improved by adding a complete example that shows how
to use re.LOCALE.
(The code behaves in the same way with python 2.2.2.)

#----------------------------------------
import locale
locale.setlocale(locale.LC_ALL,'swedish')
import re
reguml=re.compile(r"[a-zä]", re.LOCALE) # I expect
reguml and regw to give the same result.
regw=re.compile(r"\w", re.LOCALE)
reguml2=re.compile(r"[a-zä]+", re.LOCALE) # I expect
reguml2 and regw2 to give the same result.
regw2=re.compile(r"[\w]+", re.LOCALE)
str="abcä d\344e ä f ";

print reguml.findall(str) # Behaves as I expect.
print regw.findall(str) # Here I expect same result as
above, but I don't get it.
print reguml2.findall(str) # Behaves as I expect.
print regw2.findall(str) # Behaves as I expect.
#----------------------------------------



>>> import locale
>>> locale.setlocale(locale.LC_ALL,'swedish')
'swedish'
>>> import re
>>> reguml=re.compile(r"[a-zä]", re.LOCALE) # I expect
reguml and regw to give the same result.
>>> regw=re.compile(r"\w", re.LOCALE)
>>> reguml2=re.compile(r"[a-zä]+", re.LOCALE) # I
expect reguml2 and regw2 to give the same result.
>>> regw2=re.compile(r"[\w]+", re.LOCALE)
>>> str="abcä d\344e ä f ";
>>>
>>> print reguml.findall(str) # Behaves as I expect.
['a', 'b', 'c', '\xe4', 'd', '\xe4', 'e', '\xe4', 'f']
>>> print regw.findall(str) # Here I expect same result
as above, but I don't get it.
['a', 'b', 'c', 'd', 'e', 'f']
>>> print reguml2.findall(str) # Behaves as I expect.
['abc\xe4', 'd\xe4e', '\xe4', 'f']
>>> print regw2.findall(str) # Behaves as I expect.
['abc\xe4', 'd\xe4e', '\xe4', 'f']
---------------------------------------------------------
peternl:Python-2.3a2>>
/work1/pkg/dev-tools/python/2.3a2/bin/python -V
Python 2.3a2
peternl:Python-2.3a2>>uname -a
Linux peternl.computervision.se 2.4.18-6mdk-petern #2
Thu May 23 06:40:30 CEST 2002 i686 unknown


----------------------------------------------------------------------

Comment By: Greg Chapman (glchapman)
Date: 2003-02-22 08:15

Message:
Logged In: YES 
user_id=86307

I believe this is fixed by this patch:

   http://www.python.org/sf/633359

At any rate, using a patched 2.22, regw behaves identically to reguml. 


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690974&group_id=5470


From noreply@sourceforge.net  Sat Feb 22 18:41:15 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sat, 22 Feb 2003 10:41:15 -0800
Subject: [Python-bugs-list] [ python-Bugs-691276 ] shutil.copytree documentation bug
Message-ID: <E18meaJ-0001Zw-00@sc8-sf-web3.sourceforge.net>

Bugs item #691276, was opened at 2003-02-22 18:41
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=691276&group_id=5470

Category: Documentation
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Jason Orendorff (jorend)
Assigned to: Nobody/Anonymous (nobody)
Summary: shutil.copytree documentation bug

Initial Comment:
The docstring for shutil.copytree (and corresponding
Python module docs) says:

    Error are reported to standard output.

which is no longer the case.  The function now throws
shutil.Error if an error occurs.

Also, the documentation uses the source code of
shutil.copytree as an example.  That example is now out
of date.  That is, the copytree function has changed,
but the example hasn't.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=691276&group_id=5470


From noreply@sourceforge.net  Sat Feb 22 18:41:12 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sat, 22 Feb 2003 10:41:12 -0800
Subject: [Python-bugs-list] [ python-Bugs-691276 ] shutil.copytree documentation bug
Message-ID: <E18meaG-0000Mu-00@sc8-sf-web2.sourceforge.net>

Bugs item #691276, was opened at 2003-02-22 18:41
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=691276&group_id=5470

Category: Documentation
Group: Python 2.3
Status: Open
Resolution: None
>Priority: 3
Submitted By: Jason Orendorff (jorend)
Assigned to: Nobody/Anonymous (nobody)
Summary: shutil.copytree documentation bug

Initial Comment:
The docstring for shutil.copytree (and corresponding
Python module docs) says:

    Error are reported to standard output.

which is no longer the case.  The function now throws
shutil.Error if an error occurs.

Also, the documentation uses the source code of
shutil.copytree as an example.  That example is now out
of date.  That is, the copytree function has changed,
but the example hasn't.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=691276&group_id=5470


From noreply@sourceforge.net  Sat Feb 22 19:21:01 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sat, 22 Feb 2003 11:21:01 -0800
Subject: [Python-bugs-list] [ python-Bugs-691291 ] codecs.open(filename, 'U', 'UTF-16') corrupts text
Message-ID: <E18mfCn-0000dK-00@sc8-sf-web4.sourceforge.net>

Bugs item #691291, was opened at 2003-02-22 19:21
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=691291&group_id=5470

Category: Unicode
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Jason Orendorff (jorend)
Assigned to: M.-A. Lemburg (lemburg)
Summary: codecs.open(filename, 'U', 'UTF-16') corrupts text

Initial Comment:
Tested in Python 2.3a1.

If I write u'Hello\r\nworld\r\n' to a file, then read
it back in 'U' mode, I should get u'Hello\nworld\n'.

However, if I do this using codecs.open() and the
UTF-16 encoding, I get u'Hello\n\nworld\n\n'.

codecs.open() is not 'U'-mode-aware.  The underlying
file is opened in universal newline mode, so the byte
'\x0d' is erroneously translated to '\x0a' before the
UTF-16 codec has a chance to decode it.

The attached unit test should show specifically what it
is that I wish would work.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=691291&group_id=5470


From noreply@sourceforge.net  Sat Feb 22 21:17:31 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sat, 22 Feb 2003 13:17:31 -0800
Subject: [Python-bugs-list] [ python-Bugs-691291 ] codecs.open(filename, 'U', 'UTF-16') corrupts text
Message-ID: <E18mh1X-0004jt-00@sc8-sf-web4.sourceforge.net>

Bugs item #691291, was opened at 2003-02-22 19:21
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=691291&group_id=5470

Category: Unicode
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Jason Orendorff (jorend)
Assigned to: M.-A. Lemburg (lemburg)
Summary: codecs.open(filename, 'U', 'UTF-16') corrupts text

Initial Comment:
Tested in Python 2.3a1.

If I write u'Hello\r\nworld\r\n' to a file, then read
it back in 'U' mode, I should get u'Hello\nworld\n'.

However, if I do this using codecs.open() and the
UTF-16 encoding, I get u'Hello\n\nworld\n\n'.

codecs.open() is not 'U'-mode-aware.  The underlying
file is opened in universal newline mode, so the byte
'\x0d' is erroneously translated to '\x0a' before the
UTF-16 codec has a chance to decode it.

The attached unit test should show specifically what it
is that I wish would work.


----------------------------------------------------------------------

>Comment By: Jason Orendorff (jorend)
Date: 2003-02-22 21:17

Message:
Logged In: YES 
user_id=18139

Tested in Python 2.3a2 as well (the bug is still there).

Note that this isn't limited to UTF-16.  It will affect any
encoding that uses the byte '\x0d' to mean anything other
than u'\r'.  The most common American/European encodings are
safe (ASCII, Latin-1 and friends, and UTF-8).


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=691291&group_id=5470


From noreply@sourceforge.net  Sat Feb 22 23:25:23 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sat, 22 Feb 2003 15:25:23 -0800
Subject: [Python-bugs-list] [ python-Bugs-690285 ] lines run together on input
Message-ID: <E18mj1H-0006vj-00@sc8-sf-web1.sourceforge.net>

Bugs item #690285, was opened at 2003-02-20 16:12
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690285&group_id=5470

Category: Python Library
Group: Python 2.2.2
Status: Open
Resolution: None
Priority: 5
Submitted By: Thomas B Hickey (thomasbhickey)
Assigned to: Nobody/Anonymous (nobody)
Summary: lines run together on input

Initial Comment:

Using  'for line in file('xxxxx'):'  on a large (>4 Gbyte) text file 
3/4 of the way into the file occasionally two lines will be 
merged into one.  Happens consistently at the same lines.  
Counting lines this way always comes up short.  Changing 
the open to 'rb' fixes the problem.  These lines are 
terminated by '\n' even though we are running on a PC.

Haven't tried the universal newline mode in 2.3a, but would 
be willing to.

Running on Windows NT 2000, 2.2.2 (#37 Oct 14, 2002).  
Same problem seen in 2.2.1.

----------------------------------------------------------------------

>Comment By: Thomas B Hickey (thomasbhickey)
Date: 2003-02-22 18:25

Message:
Logged In: YES 
user_id=274109

We have repeated this same problem with C, so the bug does 
not seem to be peculiar to python.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690285&group_id=5470


From noreply@sourceforge.net  Sun Feb 23 12:25:29 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sun, 23 Feb 2003 04:25:29 -0800
Subject: [Python-bugs-list] [ python-Bugs-690309 ] 2.3a2 Sol8 make fails at _iconv_codec.
Message-ID: <E18mvCD-0000MG-00@sc8-sf-web3.sourceforge.net>

Bugs item #690309, was opened at 2003-02-20 16:53
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690309&group_id=5470

Category: Build
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Inyeol Lee (inyeol)
Assigned to: Nobody/Anonymous (nobody)
Summary: 2.3a2 Sol8 make fails at _iconv_codec.

Initial Comment:
Python 2.3a2 build process fails during building
_iconv_codec
module. The message is;

[...]
gcc -shared
build/temp.solaris-2.8-sun4u-2.3/_iconv_codec.o
-L/home/inyeol/util/python/lib -L/usr/local/lib -o
build/lib.solaris-2.8-sun4u-2.3/_iconv_codec.so
Traceback (most recent call last):

  [... long traceback here...]

  File "./setup.py", line 209, in build_extension
    imp.load_dynamic(ext.name, ext_filename)
RuntimeError: can't initialize the _iconv_codec module:
iconv_open() failed
*** Error code 1
make: Fatal error: Command failed for target
`sharedmods'

My build environment is gcc2.95.3 with Solaris 8
linker.
I've built 2.2, 2,2,1, 2,2,2, 2,3a1 with the same env
and there
was no (major) issues like this. I attached my
configure and
make log messages.


----------------------------------------------------------------------

Comment By: Alejandro Lopez-Valencia (dradul)
Date: 2003-02-23 07:25

Message:
Logged In: YES 
user_id=659006

The fix applied to _iconv_codec.c (in version 1.12) *breaks* the 
extension unnder cygwin and possibly other systems that use 
GNU iconv instead of a native library (namely, Solaris and BSD 
derivatives). The relevant errors:

building '_iconv_codec' extension
gcc -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -I. -
I/cvsroot/python/dist/src/./Include -I/usr/local/include -
I/tmp/Include -I/tmp/python -c /cvsroot/python/dist
/src/Modules/_iconv_codec.c -o build/temp.cygwin-1.3.20-i686-
2.3/_iconv_codec.o
cc1: warning: changing search order for system 
directory "/usr/local/include"
cc1: warning:   as it has already been specified as a non-system 
directory
/cvsroot/python/dist/src/Modules/_iconv_codec.c: In function 
`iconvcodec_encode'
:
/cvsroot/python/dist/src/Modules/_iconv_codec.c:147: warning: 
passing arg 2 of `
libiconv' from incompatible pointer type
/cvsroot/python/dist/src/Modules/_iconv_codec.c: In function 
`iconvcodec_decode'
:
/cvsroot/python/dist/src/Modules/_iconv_codec.c:357: warning: 
passing arg 2 of `
libiconv' from incompatible pointer type
/cvsroot/python/dist/src/Modules/_iconv_codec.c: In function 
`init_iconv_codec':

/cvsroot/python/dist/src/Modules/_iconv_codec.c:682: warning: 
passing arg 2 of `
libiconv' from incompatible pointer type
gcc -shared -Wl,--enable-auto-image-base build/temp.cygwin-
1.3.20-i686-2.3/_icon
v_codec.o -L/usr/local/lib -L. -liconv -lpython2.3 -o 
build/lib.cygwin-1.3.20-i6
86-2.3/_iconv_codec.dll
running build_scripts

----------------------------------------------------------------------

Comment By: Walter Dörwald (doerwalter)
Date: 2003-02-21 13:29

Message:
Logged In: YES 
user_id=89016

I changed the init function to use 'ISO8859-1' instead of
'ASCII' as the test encoding. Can someone test whether this
fixes anything?

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-20 17:15

Message:
Logged In: YES 
user_id=33168

In order to get the build to proceed, you can change line
210 of setup.py from:

        except ImportError, why:
to:
        except (ImportError, RuntimeError), why:

This is just before:
    self.announce('*** WARNING: renaming "%s" since
importing it' ...)

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690309&group_id=5470


From noreply@sourceforge.net  Sun Feb 23 13:48:51 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sun, 23 Feb 2003 05:48:51 -0800
Subject: [Python-bugs-list] [ python-Bugs-687648 ] classic division in demos/ directory
Message-ID: <E18mwUt-0002rk-00@sc8-sf-web3.sourceforge.net>

Bugs item #687648, was opened at 2003-02-16 17:27
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=687648&group_id=5470

Category: Demos and Tools
Group: Not a Bug
Status: Open
Resolution: None
Priority: 5
Submitted By: Jp Calderone (kuran)
Assigned to: Nobody/Anonymous (nobody)
Summary: classic division in demos/ directory

Initial Comment:
PEP 238 states:

     - The standard library will use the future
division statement and
       the // operator when appropriate, so as to
completely avoid
       classic division.

  While the demos/ directory is not technically part of
the standard library, it does contain code that should
work, and that newbies may examine in the course of
learning Python.  Python source there should follow the
same rules as anyplace else.

  I'll volunteer to make the changes and submit a
patch, if it is agreed that the changes should indeed
be made.


----------------------------------------------------------------------

>Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-23 08:48

Message:
Logged In: YES 
user_id=6380

Excellent.  I hope you'll try to also make a judgement (at
least in some cases) whether a particular piece of demo code
is still relevant -- there is unfortunately a lot of stale
demo code.

Also see if there are other kinds of modernization that
could be done (see PEP 290).

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=687648&group_id=5470


From noreply@sourceforge.net  Sun Feb 23 13:52:01 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sun, 23 Feb 2003 05:52:01 -0800
Subject: [Python-bugs-list] [ python-Bugs-687654 ] Future division breaks mpz
Message-ID: <E18mwXx-0002wz-00@sc8-sf-web3.sourceforge.net>

Bugs item #687654, was opened at 2003-02-16 17:41
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=687654&group_id=5470

Category: None
Group: Python 2.2.2
>Status: Closed
>Resolution: Wont Fix
>Priority: 1
Submitted By: Jp Calderone (kuran)
Assigned to: Nobody/Anonymous (nobody)
Summary: Future division breaks mpz

Initial Comment:
When future division is enabled in Python 2.2.2,
division of mpz objects no longer functions.

>>> from mpz import mpz
>>> mpz(12) / mpz(4)
mpz(3)
>>> from __future__ import division
>>> mpz(12) / mpz(4)
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
TypeError: unsupported operand type(s) for /: 'mpz.mpz'
and 'mpz.mpz'
>>> mpz(12) // mpz(4)
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
TypeError: unsupported operand type(s) for //:
'mpz.mpz' and 'mpz.mpz'


----------------------------------------------------------------------

>Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-23 08:52

Message:
Logged In: YES 
user_id=6380

I don't recall why mpz is being deprecated (it's not listed
in PEP 4, nor has it actually been removed from 2.3 yet),
but I bet it's out of lack of maintainers, so I'll close
this as a WON'T FIX. If a volunteer to maintain mpz shows up
we could reverse the deprecation, but it looks like there
are already better solutions as 3rd party modules (gmpy and
mxNumber).

----------------------------------------------------------------------

Comment By: A.M. Kuchling (akuchling)
Date: 2003-02-17 19:59

Message:
Logged In: YES 
user_id=11375

According to the docs for Python 2.2, the mpz module is deprecated and will be removed in Python 2.3, so it doesn't seem worth fixing this for a 
2.2.3 release.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=687654&group_id=5470


From noreply@sourceforge.net  Sun Feb 23 13:53:20 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sun, 23 Feb 2003 05:53:20 -0800
Subject: [Python-bugs-list] [ python-Bugs-687747 ] _iscommand() in webbrowser module
Message-ID: <E18mwZE-00011M-00@sc8-sf-web2.sourceforge.net>

Bugs item #687747, was opened at 2003-02-16 23:17
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=687747&group_id=5470

Category: Python Library
Group: Python 2.2.2
Status: Open
Resolution: None
Priority: 5
Submitted By: Matthew Cowles (mdcowles)
Assigned to: Nobody/Anonymous (nobody)
Summary: _iscommand() in webbrowser module

Initial Comment:
[From a post to python-help]

Under KDE under Mandrake Linux the BROWSER environment
variable is set to

kfmclient openProfile webbrowsing

The problem is that _iscommand() in the webbrowser
module doesn't realize that only the first thing there
is the name of the executable. It looks for an
executable with that whole thing as its name and
doesn't find one. Since the module doesn't use any
browser it has found if BROWSER is set, it raises an
error rather than opening the page.

The possible fixes that are obvious to me all have
potential disadvantages:

One solution would be to assume that the name of the
executable ran only up to the first space. But
executables can have spaces in their names, especially
on a Macintosh, I think.

Another possibility would be not to call _iscommand()
on anything in BROWSER. That would have the additional
advantage of not requiring that the executable
specified there be in a directory that's in the user's
PATH. The problem with doing things this way is that it
would be impossible to tell which of the browsers
specified in BROWSER should be used until the user
called open().

I'm prepared to work on a fix if given some guidance
about the best way to go.

----------------------------------------------------------------------

>Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-23 08:53

Message:
Logged In: YES 
user_id=6380

Please ask around on c.l.py if the macintosh problem
actually exists; if it doesn't, stopping at the first space
sounds like the right idea.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=687747&group_id=5470


From noreply@sourceforge.net  Sun Feb 23 13:58:49 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sun, 23 Feb 2003 05:58:49 -0800
Subject: [Python-bugs-list] [ python-Bugs-689826 ] Docs page has no PEPs link
Message-ID: <E18mweX-0000f3-00@sc8-sf-web4.sourceforge.net>

Bugs item #689826, was opened at 2003-02-19 23:41
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=689826&group_id=5470

Category: Documentation
Group: None
>Status: Closed
>Resolution: Fixed
Priority: 5
Submitted By: paul rubin (phr)
Assigned to: Nobody/Anonymous (nobody)
Summary: Docs page has no PEPs link

Initial Comment:
The www.python.org/doc page should have a link to the
PEP collection.

----------------------------------------------------------------------

>Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-23 08:58

Message:
Logged In: YES 
user_id=6380

Added.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=689826&group_id=5470


From noreply@sourceforge.net  Sun Feb 23 14:08:04 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sun, 23 Feb 2003 06:08:04 -0800
Subject: [Python-bugs-list] [ python-Bugs-690109 ] sys.last_type is missing
Message-ID: <E18mwnU-0000vs-00@sc8-sf-web4.sourceforge.net>

Bugs item #690109, was opened at 2003-02-20 10:46
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690109&group_id=5470

Category: Python Library
Group: Python 2.2.1
>Status: Closed
>Resolution: Invalid
Priority: 5
Submitted By: Viktor Ferenczi (complex)
Assigned to: Nobody/Anonymous (nobody)
Summary: sys.last_type is missing

Initial Comment:
ActivePython 2.2.1 Build 222 (ActiveState Corp.) based on
Python 2.2.1 (#34, Apr 15 2002, 09:51:39) [MSC 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from traceback import print_last
>>> try: dffddf
... except: print_last()
...
Traceback (most recent call last):
  File "<stdin>", line 2, in ?
  File "C:\Python\lib\traceback.py", line 219, in print_last
    print_exception(sys.last_type, sys.last_value, sys.last_traceback,
AttributeError: 'module' object has no attribute 'last_type'
>>>

----------------------------------------------------------------------

>Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-23 09:08

Message:
Logged In: YES 
user_id=6380

You're looking for exc_type, not last_type. last_type is
only set when an exception is *not* caught.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690109&group_id=5470


From noreply@sourceforge.net  Sun Feb 23 14:11:23 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sun, 23 Feb 2003 06:11:23 -0800
Subject: [Python-bugs-list] [ python-Bugs-690285 ] lines run together on input
Message-ID: <E18mwqh-0003Yp-00@sc8-sf-web3.sourceforge.net>

Bugs item #690285, was opened at 2003-02-20 16:12
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690285&group_id=5470

Category: Python Library
Group: Python 2.2.2
Status: Open
Resolution: None
>Priority: 1
Submitted By: Thomas B Hickey (thomasbhickey)
Assigned to: Nobody/Anonymous (nobody)
Summary: lines run together on input

Initial Comment:

Using  'for line in file('xxxxx'):'  on a large (>4 Gbyte) text file 
3/4 of the way into the file occasionally two lines will be 
merged into one.  Happens consistently at the same lines.  
Counting lines this way always comes up short.  Changing 
the open to 'rb' fixes the problem.  These lines are 
terminated by '\n' even though we are running on a PC.

Haven't tried the universal newline mode in 2.3a, but would 
be willing to.

Running on Windows NT 2000, 2.2.2 (#37 Oct 14, 2002).  
Same problem seen in 2.2.1.

----------------------------------------------------------------------

>Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-23 09:11

Message:
Logged In: YES 
user_id=6380

If as you suggest it's a bug in the C library, we can't do
anything about this. Are you sure it's not a problem with
your data?

----------------------------------------------------------------------

Comment By: Thomas B Hickey (thomasbhickey)
Date: 2003-02-22 18:25

Message:
Logged In: YES 
user_id=274109

We have repeated this same problem with C, so the bug does 
not seem to be peculiar to python.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690285&group_id=5470


From noreply@sourceforge.net  Sun Feb 23 14:12:14 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sun, 23 Feb 2003 06:12:14 -0800
Subject: [Python-bugs-list] [ python-Bugs-690419 ] can't build bsddb for 2.3a2
Message-ID: <E18mwrW-0003bg-00@sc8-sf-web3.sourceforge.net>

Bugs item #690419, was opened at 2003-02-20 20:12
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690419&group_id=5470

Category: Build
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Paul Jarc (prjsf)
Assigned to: Nobody/Anonymous (nobody)
Summary: can't build bsddb for 2.3a2

Initial Comment:
I have db-4.1.25 installed, and I'm trying to install
Python 2.3a2 with
bsddb.  The build fails:
gcc -fPIC  -DNDEBUG -g -O3 -Wall -Wstrict-prototypes
-I. -I./Include 
-I/package/misc/spf/python-2.3a2/spf/db/include -c
./Modules/bsddbmodule.c -o Modules/bsddbmodule.o
Modules/bsddbmodule.c: In function `newdbhashobject':
Modules/bsddbmodule.c:58: `HASHINFO' undeclared (first
use in this function)
Modules/bsddbmodule.c:58: (Each undeclared identifier
is reported only once
Modules/bsddbmodule.c:58: for each function it appears in.)
Modules/bsddbmodule.c:58: parse error before "info"
Modules/bsddbmodule.c:63: `info' undeclared (first use
in this function)
Modules/bsddbmodule.c:74: warning: implicit declaration
of function `dbopen'
Modules/bsddbmodule.c:74: warning: assignment makes
pointer from integer without a cast
Modules/bsddbmodule.c: In function `newdbbtobject':
Modules/bsddbmodule.c:106: `BTREEINFO' undeclared
(first use in this function)
Modules/bsddbmodule.c:106: parse error before "info"
Modules/bsddbmodule.c:111: `info' undeclared (first use
in this function)
Modules/bsddbmodule.c:124: warning: assignment makes
pointer from integer without a cast
Modules/bsddbmodule.c: In function `newdbrnobject':
Modules/bsddbmodule.c:156: `RECNOINFO' undeclared
(first use in this function)
Modules/bsddbmodule.c:156: parse error before "info"
Modules/bsddbmodule.c:162: `info' undeclared (first use
in this function)
Modules/bsddbmodule.c:182: warning: assignment makes
pointer from integer without a cast
Modules/bsddbmodule.c: In function `bsddb_dealloc':
Modules/bsddbmodule.c:223: too few arguments to function
Modules/bsddbmodule.c: In function `bsddb_length':
Modules/bsddbmodule.c:252: structure has no member
named `seq'
Modules/bsddbmodule.c:253: `R_FIRST' undeclared (first
use in this function)
Modules/bsddbmodule.c:255: structure has no member
named `seq'
Modules/bsddbmodule.c:256: `R_NEXT' undeclared (first
use in this function)
Modules/bsddbmodule.c: In function `bsddb_subscript':
Modules/bsddbmodule.c:276: `recno_t' undeclared (first
use in this function)
Modules/bsddbmodule.c:276: parse error before "recno"
Modules/bsddbmodule.c:279: `recno' undeclared (first
use in this function)
Modules/bsddbmodule.c:299: warning: passing arg 2 of
pointer to function from incompatible pointer type
Modules/bsddbmodule.c:299: too few arguments to function
Modules/bsddbmodule.c: In function `bsddb_ass_sub':
Modules/bsddbmodule.c:327: `recno_t' undeclared (first
use in this function)
Modules/bsddbmodule.c:327: parse error before "recno"
Modules/bsddbmodule.c:330: `recno' undeclared (first
use in this function)
Modules/bsddbmodule.c:351: warning: passing arg 2 of
pointer to function from incompatible pointer type
Modules/bsddbmodule.c:351: too few arguments to function
Modules/bsddbmodule.c:363: warning: passing arg 2 of
pointer to function from incompatible pointer type
Modules/bsddbmodule.c:363: too few arguments to function
Modules/bsddbmodule.c: In function `bsddb_close':
Modules/bsddbmodule.c:388: too few arguments to function
Modules/bsddbmodule.c: In function `bsddb_keys':
Modules/bsddbmodule.c:415: structure has no member
named `seq'
Modules/bsddbmodule.c:415: `R_FIRST' undeclared (first
use in this function)
Modules/bsddbmodule.c:441: structure has no member
named `seq'
Modules/bsddbmodule.c:442: `R_NEXT' undeclared (first
use in this function)
Modules/bsddbmodule.c: In function `bsddb_has_key':
Modules/bsddbmodule.c:470: `recno_t' undeclared (first
use in this function)
Modules/bsddbmodule.c:470: parse error before "recno"
Modules/bsddbmodule.c:474: `recno' undeclared (first
use in this function)
Modules/bsddbmodule.c:491: warning: passing arg 2 of
pointer to function from incompatible pointer type
Modules/bsddbmodule.c:491: too few arguments to function
Modules/bsddbmodule.c: In function `bsddb_set_location':
Modules/bsddbmodule.c:509: `recno_t' undeclared (first
use in this function)
Modules/bsddbmodule.c:509: parse error before "recno"
Modules/bsddbmodule.c:513: `recno' undeclared (first
use in this function)
Modules/bsddbmodule.c:530: structure has no member
named `seq'
Modules/bsddbmodule.c:530: `R_CURSOR' undeclared (first
use in this function)
Modules/bsddbmodule.c: In function `bsddb_seq':
Modules/bsddbmodule.c:570: structure has no member
named `seq'
Modules/bsddbmodule.c: In function `bsddb_next':
Modules/bsddbmodule.c:608: `R_NEXT' undeclared (first
use in this function)
Modules/bsddbmodule.c: In function `bsddb_previous':
Modules/bsddbmodule.c:613: `R_PREV' undeclared (first
use in this function)
Modules/bsddbmodule.c: In function `bsddb_first':
Modules/bsddbmodule.c:618: `R_FIRST' undeclared (first
use in this function)
Modules/bsddbmodule.c: In function `bsddb_last':
Modules/bsddbmodule.c:623: `R_LAST' undeclared (first
use in this function)
make: *** [Modules/bsddbmodule.o] Error 1


----------------------------------------------------------------------

>Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-23 09:12

Message:
Logged In: YES 
user_id=6380

Please don't include long listings in the bug report. Attach
then separately.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690419&group_id=5470


From noreply@sourceforge.net  Sun Feb 23 14:13:11 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sun, 23 Feb 2003 06:13:11 -0800
Subject: [Python-bugs-list] [ python-Bugs-690435 ] apply fails to check if warning raises exception
Message-ID: <E18mwsR-00013l-00@sc8-sf-web4.sourceforge.net>

Bugs item #690435, was opened at 2003-02-20 21:08
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690435&group_id=5470

Category: Python Interpreter Core
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Greg Chapman (glchapman)
>Assigned to: Neal Norwitz (nnorwitz)
Summary: apply fails to check if warning raises exception

Initial Comment:
The apply builtin function now generates a 
PendingDeprecationWarning.  However, it fails to check 
if the warning has raised an exception.  Attached is a 
small patch which adds the check to apply.


----------------------------------------------------------------------

>Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-23 09:13

Message:
Logged In: YES 
user_id=6380

I think Neal can fix this.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690435&group_id=5470


From noreply@sourceforge.net  Sun Feb 23 14:15:34 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sun, 23 Feb 2003 06:15:34 -0800
Subject: [Python-bugs-list] [ python-Bugs-691005 ] _POSIX_C_SOURCE redefined
Message-ID: <E18mwuk-00016i-00@sc8-sf-web4.sourceforge.net>

Bugs item #691005, was opened at 2003-02-21 21:24
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=691005&group_id=5470

Category: Python Library
Group: Python 2.3
Status: Open
Resolution: None
>Priority: 1
Submitted By: Mike Romberg (romberg)
Assigned to: Nobody/Anonymous (nobody)
Summary: _POSIX_C_SOURCE redefined

Initial Comment:
  Not really a bug.  Just a warning which might be
easy to cleanup with an ifndef.  If the following simple
file is compiled with gcc-3.2.2 on redhat-8.0 (i686) using
the c++ compiler, you get the following warning:

--------------------- test.C -------------------
#include <iostream>
#include <Python.h>
-----------------------------------------------

In file included from
/usr/local/python-2.3a2/include/python2.3/Python.h:8,
                 from test.C:2:
/usr/local/python-2.3a2/include/python2.3/pyconfig.h:819:1:
warning: "_POSIX_C_SOURCE" redefined
In file included from
/usr/local/gcc-3.2.2/include/c++/3.2.2/i686-pc-linux-gnu/bits/os_defines.h:39,
                 from
/usr/local/gcc-3.2.2/include/c++/3.2.2/i686-pc-linux-gnu/bits/c++config.h:34,
                 from
/usr/local/gcc-3.2.2/include/c++/3.2.2/iostream:44,
                 from test.C:1:
/usr/include/features.h:131:1: warning: this is the
location of the previous definition




----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2003-02-22 05:43

Message:
Logged In: YES 
user_id=6656

It's a really bad idea to include system headers *before*
including Python.h.

Do you still get the errors if you reverse the order?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=691005&group_id=5470


From noreply@sourceforge.net  Sun Feb 23 17:08:04 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sun, 23 Feb 2003 09:08:04 -0800
Subject: [Python-bugs-list] [ python-Bugs-691733 ] Let assign to as raise SyntaxWarning as well
Message-ID: <E18mzbg-00016c-00@sc8-sf-web3.sourceforge.net>

Bugs item #691733, was opened at 2003-02-23 17:08
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=691733&group_id=5470

Category: Parser/Compiler
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Gerrit Holl (gerrit)
Assigned to: Nobody/Anonymous (nobody)
Summary: Let assign to as raise SyntaxWarning as well

Initial Comment:
according to the Python Language Reference Manual:
> In some future version of Python, the identifiers
> as and None will both become keywords.

Hence, it seems natural to me to raise a SyntaxWarning
when assigning to
either of these. However, the current Python
implementation doesn't:

103 >>> None="foo"
<stdin>:1: SyntaxWarning: assignment to None
104 >>> as="foo"
105 >>>

For consistency and cleanliness, assignment to 'as'
should raise a SyntaxWarning as well. Currently, it's
possible to not know it'll be a keyword and use it as a
variable; people shouldn't, so a SyntaxWarning would be
good.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=691733&group_id=5470


From noreply@sourceforge.net  Sun Feb 23 17:21:56 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sun, 23 Feb 2003 09:21:56 -0800
Subject: [Python-bugs-list] [ python-Bugs-691733 ] Let assign to as raise SyntaxWarning as well
Message-ID: <E18mzp6-00083z-00@sc8-sf-web2.sourceforge.net>

Bugs item #691733, was opened at 2003-02-23 17:08
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=691733&group_id=5470

Category: Parser/Compiler
Group: Python 2.3
Status: Open
Resolution: None
>Priority: 2
Submitted By: Gerrit Holl (gerrit)
Assigned to: Nobody/Anonymous (nobody)
Summary: Let assign to as raise SyntaxWarning as well

Initial Comment:
according to the Python Language Reference Manual:
> In some future version of Python, the identifiers
> as and None will both become keywords.

Hence, it seems natural to me to raise a SyntaxWarning
when assigning to
either of these. However, the current Python
implementation doesn't:

103 >>> None="foo"
<stdin>:1: SyntaxWarning: assignment to None
104 >>> as="foo"
105 >>>

For consistency and cleanliness, assignment to 'as'
should raise a SyntaxWarning as well. Currently, it's
possible to not know it'll be a keyword and use it as a
variable; people shouldn't, so a SyntaxWarning would be
good.

----------------------------------------------------------------------

>Comment By: Gerrit Holl (gerrit)
Date: 2003-02-23 17:21

Message:
Logged In: YES 
user_id=13298

I'm not sure whether this should be considered as a feature.
I'd call it a minor wart... I'm also not sure about the
category, is this 'Python interpreter core' or am I right
with 'parser/compiler'?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=691733&group_id=5470


From noreply@sourceforge.net  Sun Feb 23 18:46:57 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sun, 23 Feb 2003 10:46:57 -0800
Subject: [Python-bugs-list] [ python-Bugs-691793 ] Python 2.3a2 build fails on Tru64
Message-ID: <E18n19N-0002MO-00@sc8-sf-web2.sourceforge.net>

Bugs item #691793, was opened at 2003-02-23 11:46
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=691793&group_id=5470

Category: Build
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Stephan A. Terre (sfiedler)
Assigned to: Nobody/Anonymous (nobody)
Summary: Python 2.3a2 build fails on Tru64

Initial Comment:
There is one persistent warning, and then a
build-stopping error, on Tru64. 

% uname -a
OSF1 ruby V4.0 1229 alpha
Ran configure to use the vendor compiler as:

env CC=cc CXX=cxx ./configure
--prefix=/var/tmp/sat/python_test


The configuration appeared to go fine. Upon building,
this warning was repeated numerous times:

cc -c -Olimit 1500 -DNDEBUG -O -I. -I./Include 
-DPy_BUILD_CORE -o Objects/descrobject.o
Objects/descrobject.c
cc: Info: ./Include/objimpl.h, line 252: In this
declaration, type long double has the same
representation as type double on this platform and is
treated as a synonym for type double in this
compilation mode. (longdoubleny1)
        long double dummy;  /* force worst-case
alignment */
--------^


And then these errors appeared:

cc -c -Olimit 1500 -DNDEBUG -O -I. -I./Include 
-DPy_BUILD_CORE -o Python/thread.o Python/thread.c
cc: Error: /usr/include/sys/psx4_nspace_ts.h, line 147:
Missing type specifier or type qualifier. (missingtype)
       ushort           entry_cnt;      /* Number of
key entries total in the
-------^
cc: Error: /usr/include/sys/psx4_nspace_ts.h, line 149:
Missing type specifier or type qualifier. (missingtype)
       ushort           entry_size;     /* size of key
entry */
-------^


I am happy to try candidate fixes from CVS if that
would be useful.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=691793&group_id=5470


From noreply@sourceforge.net  Sun Feb 23 21:32:35 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sun, 23 Feb 2003 13:32:35 -0800
Subject: [Python-bugs-list] [ python-Bugs-691793 ] Python 2.3a2 build fails on Tru64
Message-ID: <E18n3jf-0000eD-00@sc8-sf-web4.sourceforge.net>

Bugs item #691793, was opened at 2003-02-23 13:46
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=691793&group_id=5470

Category: Build
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Stephan A. Terre (sfiedler)
Assigned to: Nobody/Anonymous (nobody)
Summary: Python 2.3a2 build fails on Tru64

Initial Comment:
There is one persistent warning, and then a
build-stopping error, on Tru64. 

% uname -a
OSF1 ruby V4.0 1229 alpha
Ran configure to use the vendor compiler as:

env CC=cc CXX=cxx ./configure
--prefix=/var/tmp/sat/python_test


The configuration appeared to go fine. Upon building,
this warning was repeated numerous times:

cc -c -Olimit 1500 -DNDEBUG -O -I. -I./Include 
-DPy_BUILD_CORE -o Objects/descrobject.o
Objects/descrobject.c
cc: Info: ./Include/objimpl.h, line 252: In this
declaration, type long double has the same
representation as type double on this platform and is
treated as a synonym for type double in this
compilation mode. (longdoubleny1)
        long double dummy;  /* force worst-case
alignment */
--------^


And then these errors appeared:

cc -c -Olimit 1500 -DNDEBUG -O -I. -I./Include 
-DPy_BUILD_CORE -o Python/thread.o Python/thread.c
cc: Error: /usr/include/sys/psx4_nspace_ts.h, line 147:
Missing type specifier or type qualifier. (missingtype)
       ushort           entry_cnt;      /* Number of
key entries total in the
-------^
cc: Error: /usr/include/sys/psx4_nspace_ts.h, line 149:
Missing type specifier or type qualifier. (missingtype)
       ushort           entry_size;     /* size of key
entry */
-------^


I am happy to try candidate fixes from CVS if that
would be useful.

----------------------------------------------------------------------

>Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-23 16:32

Message:
Logged In: YES 
user_id=33168

I don't know of any developer that has access to OSF1 v4.0.
 Patches would certainly be welcome. :-)  The first message
appears to be a warning and seems like it can be ignored. 
The two errors from psx4_nspace_ts.h appear to be a problem
with a header file that needs to be included to get the
definition for ushort.  If you can attach a patch to this
bug report, I can check on other architectures.  Thanks.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=691793&group_id=5470


From noreply@sourceforge.net  Sun Feb 23 21:46:08 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sun, 23 Feb 2003 13:46:08 -0800
Subject: [Python-bugs-list] [ python-Bugs-691276 ] shutil.copytree documentation bug
Message-ID: <E18n3wm-0002XA-00@sc8-sf-web3.sourceforge.net>

Bugs item #691276, was opened at 2003-02-22 13:41
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=691276&group_id=5470

Category: Documentation
Group: Python 2.3
>Status: Closed
>Resolution: Fixed
Priority: 3
Submitted By: Jason Orendorff (jorend)
>Assigned to: Neal Norwitz (nnorwitz)
Summary: shutil.copytree documentation bug

Initial Comment:
The docstring for shutil.copytree (and corresponding
Python module docs) says:

    Error are reported to standard output.

which is no longer the case.  The function now throws
shutil.Error if an error occurs.

Also, the documentation uses the source code of
shutil.copytree as an example.  That example is now out
of date.  That is, the copytree function has changed,
but the example hasn't.


----------------------------------------------------------------------

>Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-23 16:46

Message:
Logged In: YES 
user_id=33168

Checked in doc fixes as:
 * Lib/shutil.py 1.28
 * Doc/lib/libshutil.tex 1.13

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=691276&group_id=5470


From noreply@sourceforge.net  Sun Feb 23 21:54:22 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sun, 23 Feb 2003 13:54:22 -0800
Subject: [Python-bugs-list] [ python-Bugs-690435 ] apply fails to check if warning raises exception
Message-ID: <E18n44k-0000qg-00@sc8-sf-web2.sourceforge.net>

Bugs item #690435, was opened at 2003-02-20 21:08
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690435&group_id=5470

Category: Python Interpreter Core
Group: Python 2.3
>Status: Closed
>Resolution: Accepted
Priority: 5
Submitted By: Greg Chapman (glchapman)
Assigned to: Neal Norwitz (nnorwitz)
Summary: apply fails to check if warning raises exception

Initial Comment:
The apply builtin function now generates a 
PendingDeprecationWarning.  However, it fails to check 
if the warning has raised an exception.  Attached is a 
small patch which adds the check to apply.


----------------------------------------------------------------------

>Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-23 16:54

Message:
Logged In: YES 
user_id=33168

Checked in as:
 * Python/bltinmodule.c 2.282

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-23 09:13

Message:
Logged In: YES 
user_id=6380

I think Neal can fix this.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690435&group_id=5470


From noreply@sourceforge.net  Sun Feb 23 21:59:32 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sun, 23 Feb 2003 13:59:32 -0800
Subject: [Python-bugs-list] [ python-Bugs-690012 ] 2.3a2 build fails under IRIX 6.5
Message-ID: <E18n49k-00016N-00@sc8-sf-web2.sourceforge.net>

Bugs item #690012, was opened at 2003-02-20 07:36
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690012&group_id=5470

Category: Build
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Ralf W. Grosse-Kunstleve (rwgk)
>Assigned to: Guido van Rossum (gvanrossum)
Summary: 2.3a2 build fails under IRIX 6.5

Initial Comment:
Using the latest Python 2.3a2 release
./configure; make
fails on this platform:
IRIX rattler 6.5 10120734 IP32
See attached logs.

I've also tried configure --disable-ipv6, but the result is 
the same.

The previous 2.3a1 release works without a problem 
on the exact same machine.


----------------------------------------------------------------------

>Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-23 16:59

Message:
Logged In: YES 
user_id=33168

Guido, I agree with your comment on python-dev that the fix
below is not best.  Attached is a patch which catches all
exceptions, and changes the warning message appropriately. 
There are 2 other issues.
 1) I don't know why the if 1: is there, should that be removed?
 2) During a normal build these warning messages aren't
printed, shouldn't they be?

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-20 18:14

Message:
Logged In: YES 
user_id=33168

In order to get the build to proceed, you can change line
210 of setup.py from:

        except ImportError, why:
to:
        except (ImportError, RuntimeError), why:

This is just before:
    self.announce('*** WARNING: renaming "%s" since
importing it' ...)



----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690012&group_id=5470


From noreply@sourceforge.net  Sun Feb 23 22:05:23 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sun, 23 Feb 2003 14:05:23 -0800
Subject: [Python-bugs-list] [ python-Bugs-690341 ] test_tarfile fails on MacOS9
Message-ID: <E18n4FP-0003Nc-00@sc8-sf-web3.sourceforge.net>

Bugs item #690341, was opened at 2003-02-20 17:21
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690341&group_id=5470

Category: Python Library
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Jack Jansen (jackjansen)
>Assigned to: Jack Jansen (jackjansen)
Summary: test_tarfile fails on MacOS9

Initial Comment:
test_tarfile fails in MacPython-OS9. The problem seems to be pathname-related, and I'm not sure whether the problem is with the test or with the tarfile module itself.

Various of the tests (test_seek, test_readlines) expect to find a file called, for example, "0-REGTYPE-TEXT", while the archive holds a file "/0-REGTYPE-TEXT". Apparently the tarfile module has some support for os.sep being different than "/", but not enough.

An added problem (or maybe the only problem?) may be that on the mac there are no files at the "root", only directories (disks, actually). So while "/tmp" is representable as "tmp:" this doesn't really work for files as they would be interpreted as directories.

----------------------------------------------------------------------

>Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-23 17:05

Message:
Logged In: YES 
user_id=33168

Jack, I don't really know how to fix these.  My guess is
that doing something like os.path.normpath or abspath on the
names would help.  The code in test_seek and test_readlines
does this:

  file(os.path.join(dirname(), filename), "r")

Could that be changed to:

  file(os.path.normpath(os.path.join(dirname(), filename)), "r")

Perhaps, there should be a utility function to get the
normalized name?  I can try to make the changes, but can't
build or test on MacOS 9.  If you can suggest some changes,
I can make a patch for you to try.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690341&group_id=5470


From noreply@sourceforge.net  Sun Feb 23 22:21:24 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sun, 23 Feb 2003 14:21:24 -0800
Subject: [Python-bugs-list] [ python-Bugs-690081 ] test_posix fails when run in non-interactive mode
Message-ID: <E18n4Uu-0006X1-00@sc8-sf-web1.sourceforge.net>

Bugs item #690081, was opened at 2003-02-20 10:06
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690081&group_id=5470

Category: Python Library
Group: Python 2.3
>Status: Closed
>Resolution: Fixed
Priority: 5
Submitted By: M.-A. Lemburg (lemburg)
>Assigned to: Neal Norwitz (nnorwitz)
Summary: test_posix fails when run in non-interactive mode

Initial Comment:
>> My nightly cronjob has issued two warnings for the
CVS checkout
>> at 02:00 CET today:
>>
>> test test_posix failed -- Traceback (most recent
call last):
>>   File
"/home/lemburg/projects/Python/Dev-Python/Lib/test/test_posix.py",
line 39, in testNoArgFunctions
>>     posix_func()
>> OSError: [Errno 25] Inappropriate ioctl for device
>>
>> The traceback is not very helpful I'm afraid and
when I run the
>> test by hand I can't reproduce it.
>
>
> Try running the test from a cron job without regrtest
(i.e. ./python
> Lib/test/test_posix.py), and putting some extra
prints in it.
>
> Some of the functions tested (e.g. ctermid, getlogin)
use stdin to
> determine who is logged in.  Maybe these should just
be ignored if
> they fail with this specific error (since the error
means they're
> working as advertised).


Ok, here's the output:

testNoArgFunctions (__main__.PosixTester) ... Running
function ctermid
Running function getcwd
Running function getcwdu
Running function uname
Running function times
Running function getlogin
   ERROR
Running function getloadavg
Running function tmpnam
Running function getegid
Running function geteuid
Running function getgid
Running function getgroups
Running function getpid
Running function getpgrp
Running function getppid
Running function getuid
ok

Looks like getlogin doesn't like being run in
non-interactive
mode (ie. without a terminal).



----------------------------------------------------------------------

>Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-23 17:21

Message:
Logged In: YES 
user_id=33168

I changed the test to not try os.getlogin() if sys.stdin is
a tty.

Checked in as:
 * Lib/test/test_posix.py 1.4

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690081&group_id=5470


From noreply@sourceforge.net  Sun Feb 23 22:27:27 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sun, 23 Feb 2003 14:27:27 -0800
Subject: [Python-bugs-list] [ python-Feature Requests-691889 ] Add MacOS.framework attribute
Message-ID: <E18n4al-0004DQ-00@sc8-sf-web3.sourceforge.net>

Feature Requests item #691889, was opened at 2003-02-23 22:27
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=691889&group_id=5470

Category: Macintosh
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: John Ehresman (jpe)
Assigned to: Nobody/Anonymous (nobody)
Summary: Add MacOS.framework attribute

Initial Comment:
On MacOS X, python may be compiled with a named
framework and extension modules must be linked at build
time against the framework with the same name.  This
new attribute would make the name of the framework
available at runtime.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=691889&group_id=5470


From noreply@sourceforge.net  Sun Feb 23 22:36:50 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sun, 23 Feb 2003 14:36:50 -0800
Subject: [Python-bugs-list] [ python-Bugs-691793 ] Python 2.3a2 build fails on Tru64
Message-ID: <E18n4jq-0004Wh-00@sc8-sf-web3.sourceforge.net>

Bugs item #691793, was opened at 2003-02-23 11:46
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=691793&group_id=5470

Category: Build
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Stephan A. Terre (sfiedler)
Assigned to: Nobody/Anonymous (nobody)
Summary: Python 2.3a2 build fails on Tru64

Initial Comment:
There is one persistent warning, and then a
build-stopping error, on Tru64. 

% uname -a
OSF1 ruby V4.0 1229 alpha
Ran configure to use the vendor compiler as:

env CC=cc CXX=cxx ./configure
--prefix=/var/tmp/sat/python_test


The configuration appeared to go fine. Upon building,
this warning was repeated numerous times:

cc -c -Olimit 1500 -DNDEBUG -O -I. -I./Include 
-DPy_BUILD_CORE -o Objects/descrobject.o
Objects/descrobject.c
cc: Info: ./Include/objimpl.h, line 252: In this
declaration, type long double has the same
representation as type double on this platform and is
treated as a synonym for type double in this
compilation mode. (longdoubleny1)
        long double dummy;  /* force worst-case
alignment */
--------^


And then these errors appeared:

cc -c -Olimit 1500 -DNDEBUG -O -I. -I./Include 
-DPy_BUILD_CORE -o Python/thread.o Python/thread.c
cc: Error: /usr/include/sys/psx4_nspace_ts.h, line 147:
Missing type specifier or type qualifier. (missingtype)
       ushort           entry_cnt;      /* Number of
key entries total in the
-------^
cc: Error: /usr/include/sys/psx4_nspace_ts.h, line 149:
Missing type specifier or type qualifier. (missingtype)
       ushort           entry_size;     /* size of key
entry */
-------^


I am happy to try candidate fixes from CVS if that
would be useful.

----------------------------------------------------------------------

>Comment By: Stephan A. Terre (sfiedler)
Date: 2003-02-23 15:36

Message:
Logged In: YES 
user_id=246063

Well, this is rather interesting. The file psx4_nspace_ts.h
is included by <semaphore.h>, which is included by
Python/thread_pthread.h . The type ushort is defined in
<sys/types.h> if the preprocessor symbol _OSF_SOURCE is
defined (which it is not in this case). Here's where it gets
interesting.

It appears that there's an error in <sys/types.h>, because
there's a mismatched #ifdef check for _OSF_SOURCE. I include
the relevant portion below in this message. So it looks like
the intent might actually be to define (via typedef) ushort
merely if  _XOPEN_SOURCE_EXTENDED is defined (which it is).
I don't know what the Python policy is for broken system
include files. It may be safe to #define _OSF_SOURCE based
on a configure-time check; however, it appears that
configure.in is presently broken in the way it tries to do
this. Here's the excerpt from config.log that makes me think so:

int
main ()
{
 makedev(0, 0)
  ;
  return 0;
}
configure:4807: cc -o conftest -g   conftest.c  >&5
cc: Warning: configure, line 4787: # not in column 1 is
ignored, skipping to end of line. (ignoretokens)
    #define _OSF_SOURCE 1
----^
cc: Warning: configure, line 4788: # not in column 1 is
ignored, skipping to end of line. (ignoretokens)
    #include <sys/types.h>
----^
ld:
Unresolved:
makedev
configure:4810: $? = 1
configure: failed program was:
#line 4785 "configure"
#include "confdefs.h"

    #define _OSF_SOURCE 1
    #include <sys/types.h>

#ifdef F77_DUMMY_MAIN
#  ifdef __cplusplus
     extern "C"
#  endif
   int F77_DUMMY_MAIN() { return 1; }
#endif
int
main ()
{
 makedev(0, 0)
  ;
  return 0;
}
configure:4833: result: no


As you can see, the Compaq/HP preprocessor does not seem to
accept preprocessor directives unless the # is in the first
column. The result is that the attempt to determine whether
_OSF_SOURCE should be defined for purposes of "makedev"
itself fails. I don't know enough about the configuration
here to know whether fixing this problem would allow the
_OSF_SOURCE symbol to be present. I am going to try changing
pyconfig.h to explicitly define _OSF_SOURCE and see if
everything else still works. I'll post here after I have
done this experiment. 

So I guess what I'd like to know from you is whether it's
better to pursue the configure.in problem or to try to add a
specific preprocessor directive somewhere in
thread_pthread.h or thereabouts in order to fix this. Thanks!

Here's the relevant portion of <sys/types.h>. If you try to
match up the #ifdef and the #endif statements, you'll see
that there is apparently a missing #endif somewhere after
the definition of P_MYID. But without knowing the
programmer's intent, it's not clear whether the "corrected"
include file would define ushort in any case.

#if defined(_OSF_SOURCE) || defined(_XOPEN_SOURCE_EXTENDED)
#ifdef _OSF_SOURCE

#ifndef NULL
#define
NULL
0L
#endif

#ifndef TRUE
#define TRUE	1
#endif

#ifndef FALSE
#define FALSE	0
#endif

#ifndef _KERNEL
typedef void (*sig_t) __((int,void (*)(int)));
#else
typedef void (*sig_t) __((int));
#endif

#endif /* _OSF_SOURCE */
/*
 * The following type is for various kinds of identifiers.  The
 * actual type must be the same for all since some system calls
 * (such as sigsend) take arguments that may be any of these
 * types.  The enumeration type idtype_t defined in
sys/procset.h
 * is used to indicate what type of id is being specified.
 */
typedef pid_t		id_t;		/* A process id,	*/
					/* process group id,	*/
					/* session id, 		*/
					/* scheduling class id,	*/
					/* user id, or group id.*/
typedef unsigned int	useconds_t;	/* time in microseconds */

#ifdef _OSF_SOURCE
#define
P_MYID
(-1)
/* a usually illegal value for IDs, but specifying
			   whatever the value is for my process */

/*
 * Types used by dev_t annotation macros (see below)
 */
typedef uint_t	major_t;      /* major device number   */
typedef uint_t	minor_t;      /* minor device number   */
typedef uint_t	devs_t;       /* device-specific info  */
typedef uint_t	unit_t;       /* unit number of device */

/*
 * Basic system types and major/minor device
constructing/busting macros.
 */
#define major(x)	((major_t)  (((dev_t)(x)>>20)&07777))
#define minor(x)	((minor_t)  ((dev_t)(x)&03777777))
#define makedev(x,y)	((dev_t)    (((major_t)(x)<<20) | (minor_t)(y)))

/*
 * Disk/Tape (SCSI/CAM - DSA) specific dev_t annotations macros.
 */
#define MAKEMINOR(u,d)  ((minor_t)  (((unit_t)(u)<<6)
|(devs_t)(d)))
#define GETUNIT(dev)	((unit_t)   (minor(dev)>>6)&037777)
#define GETDEVS(dev)	((devs_t)   (minor(dev))&077)
#define MAKECAMMINOR(u,d) ((minor_t)
MAKEMINOR((((u&0770)<<5)|((u&07)<<4)),d))
#define GETCAMUNIT(x)   ((unit_t)
(((GETUNIT(x))>>5)&0770)|((GETUNIT(x)>>4)&07))
#define GETCAMTARG(x)   ((unit_t) ((x >> 3)&07))
#define GETREUNIT(x)	((unit_t) ((x)&07))

#include <mach/machine/vm_types.h>

/*
 * shorthand type definitions for unsigned storage classes
 */
typedef
uchar_t
	uchar;
typedef
ushort_t
ushort;
typedef
uint_t
	uint;
typedef ulong_t		ulong;

typedef
physadr_t
physadr;

#ifdef _OSF_SOURCE
/* typedefs for BSD unsigned things */
typedef
uchar_t
	u_char;
typedef
ushort_t 	u_short;
typedef
uint_t
	u_int;
typedef
ulong_t
	u_long;
typedef
vuchar_t
vu_char;
typedef
vushort_t 	vu_short;
typedef
vuint_t
	vu_int;
typedef
vulong_t
vu_long;
#endif

#ifdef  _KERNEL
typedef struct  _quad { u_int val[2]; } quad;
#else   /* _KERNEL */
typedef struct  _quad { int val[2]; } quad;
#endif  /* _KERNEL */

typedef
long
swblk_t;
typedef u_long	fixpt_t;

#include <sys/select.h>

#endif   /* _OSF_SOURCE */




----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-23 14:32

Message:
Logged In: YES 
user_id=33168

I don't know of any developer that has access to OSF1 v4.0.
 Patches would certainly be welcome. :-)  The first message
appears to be a warning and seems like it can be ignored. 
The two errors from psx4_nspace_ts.h appear to be a problem
with a header file that needs to be included to get the
definition for ushort.  If you can attach a patch to this
bug report, I can check on other architectures.  Thanks.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=691793&group_id=5470


From noreply@sourceforge.net  Sun Feb 23 23:01:07 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sun, 23 Feb 2003 15:01:07 -0800
Subject: [Python-bugs-list] [ python-Bugs-691793 ] Python 2.3a2 build fails on Tru64
Message-ID: <E18n57L-0002yZ-00@sc8-sf-web2.sourceforge.net>

Bugs item #691793, was opened at 2003-02-23 11:46
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=691793&group_id=5470

Category: Build
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Stephan A. Terre (sfiedler)
Assigned to: Nobody/Anonymous (nobody)
Summary: Python 2.3a2 build fails on Tru64

Initial Comment:
There is one persistent warning, and then a
build-stopping error, on Tru64. 

% uname -a
OSF1 ruby V4.0 1229 alpha
Ran configure to use the vendor compiler as:

env CC=cc CXX=cxx ./configure
--prefix=/var/tmp/sat/python_test


The configuration appeared to go fine. Upon building,
this warning was repeated numerous times:

cc -c -Olimit 1500 -DNDEBUG -O -I. -I./Include 
-DPy_BUILD_CORE -o Objects/descrobject.o
Objects/descrobject.c
cc: Info: ./Include/objimpl.h, line 252: In this
declaration, type long double has the same
representation as type double on this platform and is
treated as a synonym for type double in this
compilation mode. (longdoubleny1)
        long double dummy;  /* force worst-case
alignment */
--------^


And then these errors appeared:

cc -c -Olimit 1500 -DNDEBUG -O -I. -I./Include 
-DPy_BUILD_CORE -o Python/thread.o Python/thread.c
cc: Error: /usr/include/sys/psx4_nspace_ts.h, line 147:
Missing type specifier or type qualifier. (missingtype)
       ushort           entry_cnt;      /* Number of
key entries total in the
-------^
cc: Error: /usr/include/sys/psx4_nspace_ts.h, line 149:
Missing type specifier or type qualifier. (missingtype)
       ushort           entry_size;     /* size of key
entry */
-------^


I am happy to try candidate fixes from CVS if that
would be useful.

----------------------------------------------------------------------

>Comment By: Stephan A. Terre (sfiedler)
Date: 2003-02-23 16:01

Message:
Logged In: YES 
user_id=246063

I have verified that everything seems to basically build and
run if I alter the test in configure to put the # at the
first column for preprocessor directives. Attached is a
somewhat more ambitious patch than that: I went through
config.log and found all the places where this seemed to be
happening, and fixed it based on my Very Limited
Understanding of autoconf. I don't have autoconf here to try
this with. So if this is a bad idea, we should solve it some
other way. But hopefully this will work everywhere and will
avoid some other minor problems in the future.

----------------------------------------------------------------------

Comment By: Stephan A. Terre (sfiedler)
Date: 2003-02-23 15:36

Message:
Logged In: YES 
user_id=246063

Well, this is rather interesting. The file psx4_nspace_ts.h
is included by <semaphore.h>, which is included by
Python/thread_pthread.h . The type ushort is defined in
<sys/types.h> if the preprocessor symbol _OSF_SOURCE is
defined (which it is not in this case). Here's where it gets
interesting.

It appears that there's an error in <sys/types.h>, because
there's a mismatched #ifdef check for _OSF_SOURCE. I include
the relevant portion below in this message. So it looks like
the intent might actually be to define (via typedef) ushort
merely if  _XOPEN_SOURCE_EXTENDED is defined (which it is).
I don't know what the Python policy is for broken system
include files. It may be safe to #define _OSF_SOURCE based
on a configure-time check; however, it appears that
configure.in is presently broken in the way it tries to do
this. Here's the excerpt from config.log that makes me think so:

int
main ()
{
 makedev(0, 0)
  ;
  return 0;
}
configure:4807: cc -o conftest -g   conftest.c  >&5
cc: Warning: configure, line 4787: # not in column 1 is
ignored, skipping to end of line. (ignoretokens)
    #define _OSF_SOURCE 1
----^
cc: Warning: configure, line 4788: # not in column 1 is
ignored, skipping to end of line. (ignoretokens)
    #include <sys/types.h>
----^
ld:
Unresolved:
makedev
configure:4810: $? = 1
configure: failed program was:
#line 4785 "configure"
#include "confdefs.h"

    #define _OSF_SOURCE 1
    #include <sys/types.h>

#ifdef F77_DUMMY_MAIN
#  ifdef __cplusplus
     extern "C"
#  endif
   int F77_DUMMY_MAIN() { return 1; }
#endif
int
main ()
{
 makedev(0, 0)
  ;
  return 0;
}
configure:4833: result: no


As you can see, the Compaq/HP preprocessor does not seem to
accept preprocessor directives unless the # is in the first
column. The result is that the attempt to determine whether
_OSF_SOURCE should be defined for purposes of "makedev"
itself fails. I don't know enough about the configuration
here to know whether fixing this problem would allow the
_OSF_SOURCE symbol to be present. I am going to try changing
pyconfig.h to explicitly define _OSF_SOURCE and see if
everything else still works. I'll post here after I have
done this experiment. 

So I guess what I'd like to know from you is whether it's
better to pursue the configure.in problem or to try to add a
specific preprocessor directive somewhere in
thread_pthread.h or thereabouts in order to fix this. Thanks!

Here's the relevant portion of <sys/types.h>. If you try to
match up the #ifdef and the #endif statements, you'll see
that there is apparently a missing #endif somewhere after
the definition of P_MYID. But without knowing the
programmer's intent, it's not clear whether the "corrected"
include file would define ushort in any case.

#if defined(_OSF_SOURCE) || defined(_XOPEN_SOURCE_EXTENDED)
#ifdef _OSF_SOURCE

#ifndef NULL
#define
NULL
0L
#endif

#ifndef TRUE
#define TRUE	1
#endif

#ifndef FALSE
#define FALSE	0
#endif

#ifndef _KERNEL
typedef void (*sig_t) __((int,void (*)(int)));
#else
typedef void (*sig_t) __((int));
#endif

#endif /* _OSF_SOURCE */
/*
 * The following type is for various kinds of identifiers.  The
 * actual type must be the same for all since some system calls
 * (such as sigsend) take arguments that may be any of these
 * types.  The enumeration type idtype_t defined in
sys/procset.h
 * is used to indicate what type of id is being specified.
 */
typedef pid_t		id_t;		/* A process id,	*/
					/* process group id,	*/
					/* session id, 		*/
					/* scheduling class id,	*/
					/* user id, or group id.*/
typedef unsigned int	useconds_t;	/* time in microseconds */

#ifdef _OSF_SOURCE
#define
P_MYID
(-1)
/* a usually illegal value for IDs, but specifying
			   whatever the value is for my process */

/*
 * Types used by dev_t annotation macros (see below)
 */
typedef uint_t	major_t;      /* major device number   */
typedef uint_t	minor_t;      /* minor device number   */
typedef uint_t	devs_t;       /* device-specific info  */
typedef uint_t	unit_t;       /* unit number of device */

/*
 * Basic system types and major/minor device
constructing/busting macros.
 */
#define major(x)	((major_t)  (((dev_t)(x)>>20)&07777))
#define minor(x)	((minor_t)  ((dev_t)(x)&03777777))
#define makedev(x,y)	((dev_t)    (((major_t)(x)<<20) | (minor_t)(y)))

/*
 * Disk/Tape (SCSI/CAM - DSA) specific dev_t annotations macros.
 */
#define MAKEMINOR(u,d)  ((minor_t)  (((unit_t)(u)<<6)
|(devs_t)(d)))
#define GETUNIT(dev)	((unit_t)   (minor(dev)>>6)&037777)
#define GETDEVS(dev)	((devs_t)   (minor(dev))&077)
#define MAKECAMMINOR(u,d) ((minor_t)
MAKEMINOR((((u&0770)<<5)|((u&07)<<4)),d))
#define GETCAMUNIT(x)   ((unit_t)
(((GETUNIT(x))>>5)&0770)|((GETUNIT(x)>>4)&07))
#define GETCAMTARG(x)   ((unit_t) ((x >> 3)&07))
#define GETREUNIT(x)	((unit_t) ((x)&07))

#include <mach/machine/vm_types.h>

/*
 * shorthand type definitions for unsigned storage classes
 */
typedef
uchar_t
	uchar;
typedef
ushort_t
ushort;
typedef
uint_t
	uint;
typedef ulong_t		ulong;

typedef
physadr_t
physadr;

#ifdef _OSF_SOURCE
/* typedefs for BSD unsigned things */
typedef
uchar_t
	u_char;
typedef
ushort_t 	u_short;
typedef
uint_t
	u_int;
typedef
ulong_t
	u_long;
typedef
vuchar_t
vu_char;
typedef
vushort_t 	vu_short;
typedef
vuint_t
	vu_int;
typedef
vulong_t
vu_long;
#endif

#ifdef  _KERNEL
typedef struct  _quad { u_int val[2]; } quad;
#else   /* _KERNEL */
typedef struct  _quad { int val[2]; } quad;
#endif  /* _KERNEL */

typedef
long
swblk_t;
typedef u_long	fixpt_t;

#include <sys/select.h>

#endif   /* _OSF_SOURCE */




----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-23 14:32

Message:
Logged In: YES 
user_id=33168

I don't know of any developer that has access to OSF1 v4.0.
 Patches would certainly be welcome. :-)  The first message
appears to be a warning and seems like it can be ignored. 
The two errors from psx4_nspace_ts.h appear to be a problem
with a header file that needs to be included to get the
definition for ushort.  If you can attach a patch to this
bug report, I can check on other architectures.  Thanks.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=691793&group_id=5470


From noreply@sourceforge.net  Sun Feb 23 23:06:39 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sun, 23 Feb 2003 15:06:39 -0800
Subject: [Python-bugs-list] [ python-Bugs-691793 ] Python 2.3a2 build fails on Tru64
Message-ID: <E18n5Ch-0008Iz-00@sc8-sf-web1.sourceforge.net>

Bugs item #691793, was opened at 2003-02-23 11:46
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=691793&group_id=5470

Category: Build
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Stephan A. Terre (sfiedler)
Assigned to: Nobody/Anonymous (nobody)
Summary: Python 2.3a2 build fails on Tru64

Initial Comment:
There is one persistent warning, and then a
build-stopping error, on Tru64. 

% uname -a
OSF1 ruby V4.0 1229 alpha
Ran configure to use the vendor compiler as:

env CC=cc CXX=cxx ./configure
--prefix=/var/tmp/sat/python_test


The configuration appeared to go fine. Upon building,
this warning was repeated numerous times:

cc -c -Olimit 1500 -DNDEBUG -O -I. -I./Include 
-DPy_BUILD_CORE -o Objects/descrobject.o
Objects/descrobject.c
cc: Info: ./Include/objimpl.h, line 252: In this
declaration, type long double has the same
representation as type double on this platform and is
treated as a synonym for type double in this
compilation mode. (longdoubleny1)
        long double dummy;  /* force worst-case
alignment */
--------^


And then these errors appeared:

cc -c -Olimit 1500 -DNDEBUG -O -I. -I./Include 
-DPy_BUILD_CORE -o Python/thread.o Python/thread.c
cc: Error: /usr/include/sys/psx4_nspace_ts.h, line 147:
Missing type specifier or type qualifier. (missingtype)
       ushort           entry_cnt;      /* Number of
key entries total in the
-------^
cc: Error: /usr/include/sys/psx4_nspace_ts.h, line 149:
Missing type specifier or type qualifier. (missingtype)
       ushort           entry_size;     /* size of key
entry */
-------^


I am happy to try candidate fixes from CVS if that
would be useful.

----------------------------------------------------------------------

>Comment By: Stephan A. Terre (sfiedler)
Date: 2003-02-23 16:06

Message:
Logged In: YES 
user_id=246063

Just realized I was unclear in my previous comment. To test
the fix for _OSF_SOURCE as relates to the test for
"makedev", I altered the file "configure", and rebuilt.
Everything was happy.

The file attached to this bug is the result of changing
"configure.in" along the same lines. However, I do not have
autoconf and so did not regenerate "configure" from
"configure.in". I have *not* tested the patch I'm
submitting, just something which I think is a close
approximation. Although the file is named
"configure_patch.txt", it is really a patch to "configure.in".

I can try to get autoconf working on this system and verify
the patch if you would prefer.

----------------------------------------------------------------------

Comment By: Stephan A. Terre (sfiedler)
Date: 2003-02-23 16:01

Message:
Logged In: YES 
user_id=246063

I have verified that everything seems to basically build and
run if I alter the test in configure to put the # at the
first column for preprocessor directives. Attached is a
somewhat more ambitious patch than that: I went through
config.log and found all the places where this seemed to be
happening, and fixed it based on my Very Limited
Understanding of autoconf. I don't have autoconf here to try
this with. So if this is a bad idea, we should solve it some
other way. But hopefully this will work everywhere and will
avoid some other minor problems in the future.

----------------------------------------------------------------------

Comment By: Stephan A. Terre (sfiedler)
Date: 2003-02-23 15:36

Message:
Logged In: YES 
user_id=246063

Well, this is rather interesting. The file psx4_nspace_ts.h
is included by <semaphore.h>, which is included by
Python/thread_pthread.h . The type ushort is defined in
<sys/types.h> if the preprocessor symbol _OSF_SOURCE is
defined (which it is not in this case). Here's where it gets
interesting.

It appears that there's an error in <sys/types.h>, because
there's a mismatched #ifdef check for _OSF_SOURCE. I include
the relevant portion below in this message. So it looks like
the intent might actually be to define (via typedef) ushort
merely if  _XOPEN_SOURCE_EXTENDED is defined (which it is).
I don't know what the Python policy is for broken system
include files. It may be safe to #define _OSF_SOURCE based
on a configure-time check; however, it appears that
configure.in is presently broken in the way it tries to do
this. Here's the excerpt from config.log that makes me think so:

int
main ()
{
 makedev(0, 0)
  ;
  return 0;
}
configure:4807: cc -o conftest -g   conftest.c  >&5
cc: Warning: configure, line 4787: # not in column 1 is
ignored, skipping to end of line. (ignoretokens)
    #define _OSF_SOURCE 1
----^
cc: Warning: configure, line 4788: # not in column 1 is
ignored, skipping to end of line. (ignoretokens)
    #include <sys/types.h>
----^
ld:
Unresolved:
makedev
configure:4810: $? = 1
configure: failed program was:
#line 4785 "configure"
#include "confdefs.h"

    #define _OSF_SOURCE 1
    #include <sys/types.h>

#ifdef F77_DUMMY_MAIN
#  ifdef __cplusplus
     extern "C"
#  endif
   int F77_DUMMY_MAIN() { return 1; }
#endif
int
main ()
{
 makedev(0, 0)
  ;
  return 0;
}
configure:4833: result: no


As you can see, the Compaq/HP preprocessor does not seem to
accept preprocessor directives unless the # is in the first
column. The result is that the attempt to determine whether
_OSF_SOURCE should be defined for purposes of "makedev"
itself fails. I don't know enough about the configuration
here to know whether fixing this problem would allow the
_OSF_SOURCE symbol to be present. I am going to try changing
pyconfig.h to explicitly define _OSF_SOURCE and see if
everything else still works. I'll post here after I have
done this experiment. 

So I guess what I'd like to know from you is whether it's
better to pursue the configure.in problem or to try to add a
specific preprocessor directive somewhere in
thread_pthread.h or thereabouts in order to fix this. Thanks!

Here's the relevant portion of <sys/types.h>. If you try to
match up the #ifdef and the #endif statements, you'll see
that there is apparently a missing #endif somewhere after
the definition of P_MYID. But without knowing the
programmer's intent, it's not clear whether the "corrected"
include file would define ushort in any case.

#if defined(_OSF_SOURCE) || defined(_XOPEN_SOURCE_EXTENDED)
#ifdef _OSF_SOURCE

#ifndef NULL
#define
NULL
0L
#endif

#ifndef TRUE
#define TRUE	1
#endif

#ifndef FALSE
#define FALSE	0
#endif

#ifndef _KERNEL
typedef void (*sig_t) __((int,void (*)(int)));
#else
typedef void (*sig_t) __((int));
#endif

#endif /* _OSF_SOURCE */
/*
 * The following type is for various kinds of identifiers.  The
 * actual type must be the same for all since some system calls
 * (such as sigsend) take arguments that may be any of these
 * types.  The enumeration type idtype_t defined in
sys/procset.h
 * is used to indicate what type of id is being specified.
 */
typedef pid_t		id_t;		/* A process id,	*/
					/* process group id,	*/
					/* session id, 		*/
					/* scheduling class id,	*/
					/* user id, or group id.*/
typedef unsigned int	useconds_t;	/* time in microseconds */

#ifdef _OSF_SOURCE
#define
P_MYID
(-1)
/* a usually illegal value for IDs, but specifying
			   whatever the value is for my process */

/*
 * Types used by dev_t annotation macros (see below)
 */
typedef uint_t	major_t;      /* major device number   */
typedef uint_t	minor_t;      /* minor device number   */
typedef uint_t	devs_t;       /* device-specific info  */
typedef uint_t	unit_t;       /* unit number of device */

/*
 * Basic system types and major/minor device
constructing/busting macros.
 */
#define major(x)	((major_t)  (((dev_t)(x)>>20)&07777))
#define minor(x)	((minor_t)  ((dev_t)(x)&03777777))
#define makedev(x,y)	((dev_t)    (((major_t)(x)<<20) | (minor_t)(y)))

/*
 * Disk/Tape (SCSI/CAM - DSA) specific dev_t annotations macros.
 */
#define MAKEMINOR(u,d)  ((minor_t)  (((unit_t)(u)<<6)
|(devs_t)(d)))
#define GETUNIT(dev)	((unit_t)   (minor(dev)>>6)&037777)
#define GETDEVS(dev)	((devs_t)   (minor(dev))&077)
#define MAKECAMMINOR(u,d) ((minor_t)
MAKEMINOR((((u&0770)<<5)|((u&07)<<4)),d))
#define GETCAMUNIT(x)   ((unit_t)
(((GETUNIT(x))>>5)&0770)|((GETUNIT(x)>>4)&07))
#define GETCAMTARG(x)   ((unit_t) ((x >> 3)&07))
#define GETREUNIT(x)	((unit_t) ((x)&07))

#include <mach/machine/vm_types.h>

/*
 * shorthand type definitions for unsigned storage classes
 */
typedef
uchar_t
	uchar;
typedef
ushort_t
ushort;
typedef
uint_t
	uint;
typedef ulong_t		ulong;

typedef
physadr_t
physadr;

#ifdef _OSF_SOURCE
/* typedefs for BSD unsigned things */
typedef
uchar_t
	u_char;
typedef
ushort_t 	u_short;
typedef
uint_t
	u_int;
typedef
ulong_t
	u_long;
typedef
vuchar_t
vu_char;
typedef
vushort_t 	vu_short;
typedef
vuint_t
	vu_int;
typedef
vulong_t
vu_long;
#endif

#ifdef  _KERNEL
typedef struct  _quad { u_int val[2]; } quad;
#else   /* _KERNEL */
typedef struct  _quad { int val[2]; } quad;
#endif  /* _KERNEL */

typedef
long
swblk_t;
typedef u_long	fixpt_t;

#include <sys/select.h>

#endif   /* _OSF_SOURCE */




----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-23 14:32

Message:
Logged In: YES 
user_id=33168

I don't know of any developer that has access to OSF1 v4.0.
 Patches would certainly be welcome. :-)  The first message
appears to be a warning and seems like it can be ignored. 
The two errors from psx4_nspace_ts.h appear to be a problem
with a header file that needs to be included to get the
definition for ushort.  If you can attach a patch to this
bug report, I can check on other architectures.  Thanks.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=691793&group_id=5470


From noreply@sourceforge.net  Sun Feb 23 23:24:47 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sun, 23 Feb 2003 15:24:47 -0800
Subject: [Python-bugs-list] [ python-Bugs-689659 ] 64-bit int and long hash keys incompatible
Message-ID: <E18n5UF-0003c1-00@sc8-sf-web2.sourceforge.net>

Bugs item #689659, was opened at 2003-02-19 18:17
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=689659&group_id=5470

Category: Python Interpreter Core
Group: Platform-specific
>Status: Closed
>Resolution: Fixed
Priority: 6
Submitted By: Mark Dickinson (marketdickinson)
Assigned to: Neal Norwitz (nnorwitz)
Summary: 64-bit int and long hash keys incompatible

Initial Comment:
For 64-bit platforms, it seems that ints and longs with the 
same value don't necessarily give the same hash key.  
An example is below; the phenomenon seems to occur 
for a range of numbers between 2**32 and 2**64. 
 
I assume this is unintentional?  It looks suspiciously as 
though the function long_hash in Objects/longobject.c 
has 32-bitness hard-coded. Lines 1288-1289 of that file 
in the Python 2.2.2 source read: 
 
      /* Force a 32-bit circular shift */ 
      x = ((x << SHIFT) & ~MASK) | ((x >> (32-SHIFT)) & 
MASK); 
 
And here's the Python example: 
 
Python 2.2.2 (#22, Dec 23 2002, 12:02:55) 
[GCC 3.0.3] on sunos5 
Type "help", "copyright", "credits" or "license" for more 
information. 
>>> x = 4503599627370496L 
>>> y = 4503599627370496 
>>> h = {x: 'anything', y: 'something else'} 
>>> h[x] 
'anything' 
>>> h[y] 
'something else' 
>>> x == y 
1 
 
All the best, 
 
Mark 
 

----------------------------------------------------------------------

>Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-23 18:24

Message:
Logged In: YES 
user_id=33168

Thanks!  Checked in as:
 * Objects/longobject.c 1.156
 * Lib/test/test_types.py 1.46
 * Misc/NEWS 1.675

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-19 22:45

Message:
Logged In: YES 
user_id=33168

Even though Tim gave me the rest of the year off, I'll take
a look. :-)  BTW, we all have access to a 64-bit box on the
SF compile farm. :-)  Mark, it would be great if you are
able to create a patch, but if not, I'll try to fix it.

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2003-02-19 18:29

Message:
Logged In: YES 
user_id=31435

Good catch!  Agreed on all counts, and boosted priority 
because Python is trying to blur the distinction between 
ints and longs, so that this is likely to bite harder in the 
future.  Assigned to me, but a patch would be appreciated 
(don't know when I can make time for this, and I don't have 
a 64-bit box for testing).

It should be enough to replace "32" with "8*sizeof(long)".

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=689659&group_id=5470


From noreply@sourceforge.net  Sun Feb 23 23:35:03 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sun, 23 Feb 2003 15:35:03 -0800
Subject: [Python-bugs-list] [ python-Feature Requests-691889 ] Add MacOS.framework attribute
Message-ID: <E18n5eB-0006pc-00@sc8-sf-web3.sourceforge.net>

Feature Requests item #691889, was opened at 2003-02-23 23:27
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=691889&group_id=5470

Category: Macintosh
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: John Ehresman (jpe)
Assigned to: Nobody/Anonymous (nobody)
Summary: Add MacOS.framework attribute

Initial Comment:
On MacOS X, python may be compiled with a named
framework and extension modules must be linked at build
time against the framework with the same name.  This
new attribute would make the name of the framework
available at runtime.

----------------------------------------------------------------------

>Comment By: Jack Jansen (jackjansen)
Date: 2003-02-24 00:35

Message:
Logged In: YES 
user_id=45365

Implemented in macosmodule.c rev. 1.65 in a slightly different way. There's now an attribute MacOS.linkmodel which can have one of four values:
'static', 'framework', 'shared' (not used currently, but is there for compleness if I ever get around to fixing non-framework shared-library builds) and 'cfm' (for MacPython-OS9).

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=691889&group_id=5470


From noreply@sourceforge.net  Sun Feb 23 23:36:27 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sun, 23 Feb 2003 15:36:27 -0800
Subject: [Python-bugs-list] [ python-Feature Requests-691889 ] Add MacOS.framework attribute
Message-ID: <E18n5fX-0006tu-00@sc8-sf-web3.sourceforge.net>

Feature Requests item #691889, was opened at 2003-02-23 23:27
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=691889&group_id=5470

Category: Macintosh
Group: None
>Status: Closed
>Resolution: Accepted
Priority: 5
Submitted By: John Ehresman (jpe)
>Assigned to: Jack Jansen (jackjansen)
Summary: Add MacOS.framework attribute

Initial Comment:
On MacOS X, python may be compiled with a named
framework and extension modules must be linked at build
time against the framework with the same name.  This
new attribute would make the name of the framework
available at runtime.

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-02-24 00:35

Message:
Logged In: YES 
user_id=45365

Implemented in macosmodule.c rev. 1.65 in a slightly different way. There's now an attribute MacOS.linkmodel which can have one of four values:
'static', 'framework', 'shared' (not used currently, but is there for compleness if I ever get around to fixing non-framework shared-library builds) and 'cfm' (for MacPython-OS9).

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=691889&group_id=5470


From noreply@sourceforge.net  Sun Feb 23 23:38:56 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sun, 23 Feb 2003 15:38:56 -0800
Subject: [Python-bugs-list] [ python-Bugs-691793 ] Python 2.3a2 build fails on Tru64
Message-ID: <E18n5hw-0000yU-00@sc8-sf-web1.sourceforge.net>

Bugs item #691793, was opened at 2003-02-23 13:46
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=691793&group_id=5470

Category: Build
Group: Python 2.3
Status: Open
>Resolution: Fixed
Priority: 5
Submitted By: Stephan A. Terre (sfiedler)
>Assigned to: Neal Norwitz (nnorwitz)
Summary: Python 2.3a2 build fails on Tru64

Initial Comment:
There is one persistent warning, and then a
build-stopping error, on Tru64. 

% uname -a
OSF1 ruby V4.0 1229 alpha
Ran configure to use the vendor compiler as:

env CC=cc CXX=cxx ./configure
--prefix=/var/tmp/sat/python_test


The configuration appeared to go fine. Upon building,
this warning was repeated numerous times:

cc -c -Olimit 1500 -DNDEBUG -O -I. -I./Include 
-DPy_BUILD_CORE -o Objects/descrobject.o
Objects/descrobject.c
cc: Info: ./Include/objimpl.h, line 252: In this
declaration, type long double has the same
representation as type double on this platform and is
treated as a synonym for type double in this
compilation mode. (longdoubleny1)
        long double dummy;  /* force worst-case
alignment */
--------^


And then these errors appeared:

cc -c -Olimit 1500 -DNDEBUG -O -I. -I./Include 
-DPy_BUILD_CORE -o Python/thread.o Python/thread.c
cc: Error: /usr/include/sys/psx4_nspace_ts.h, line 147:
Missing type specifier or type qualifier. (missingtype)
       ushort           entry_cnt;      /* Number of
key entries total in the
-------^
cc: Error: /usr/include/sys/psx4_nspace_ts.h, line 149:
Missing type specifier or type qualifier. (missingtype)
       ushort           entry_size;     /* size of key
entry */
-------^


I am happy to try candidate fixes from CVS if that
would be useful.

----------------------------------------------------------------------

>Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-23 18:38

Message:
Logged In: YES 
user_id=33168

Stephan, thanks for your help.  Your fixes did seem correct,
although I had already done them, so the fix I checked in
was slightly different.  Could you test out the version in
CVS and verify if it corrects your problem?  Also, I recall
a bug about OSF1 v4.0 for 2.2.2.  These fixes should be
backported, are you able to test the 2.2.2+ version from
CVS?  There is a planned release of 2.2.3 soon.

----------------------------------------------------------------------

Comment By: Stephan A. Terre (sfiedler)
Date: 2003-02-23 18:06

Message:
Logged In: YES 
user_id=246063

Just realized I was unclear in my previous comment. To test
the fix for _OSF_SOURCE as relates to the test for
"makedev", I altered the file "configure", and rebuilt.
Everything was happy.

The file attached to this bug is the result of changing
"configure.in" along the same lines. However, I do not have
autoconf and so did not regenerate "configure" from
"configure.in". I have *not* tested the patch I'm
submitting, just something which I think is a close
approximation. Although the file is named
"configure_patch.txt", it is really a patch to "configure.in".

I can try to get autoconf working on this system and verify
the patch if you would prefer.

----------------------------------------------------------------------

Comment By: Stephan A. Terre (sfiedler)
Date: 2003-02-23 18:01

Message:
Logged In: YES 
user_id=246063

I have verified that everything seems to basically build and
run if I alter the test in configure to put the # at the
first column for preprocessor directives. Attached is a
somewhat more ambitious patch than that: I went through
config.log and found all the places where this seemed to be
happening, and fixed it based on my Very Limited
Understanding of autoconf. I don't have autoconf here to try
this with. So if this is a bad idea, we should solve it some
other way. But hopefully this will work everywhere and will
avoid some other minor problems in the future.

----------------------------------------------------------------------

Comment By: Stephan A. Terre (sfiedler)
Date: 2003-02-23 17:36

Message:
Logged In: YES 
user_id=246063

Well, this is rather interesting. The file psx4_nspace_ts.h
is included by <semaphore.h>, which is included by
Python/thread_pthread.h . The type ushort is defined in
<sys/types.h> if the preprocessor symbol _OSF_SOURCE is
defined (which it is not in this case). Here's where it gets
interesting.

It appears that there's an error in <sys/types.h>, because
there's a mismatched #ifdef check for _OSF_SOURCE. I include
the relevant portion below in this message. So it looks like
the intent might actually be to define (via typedef) ushort
merely if  _XOPEN_SOURCE_EXTENDED is defined (which it is).
I don't know what the Python policy is for broken system
include files. It may be safe to #define _OSF_SOURCE based
on a configure-time check; however, it appears that
configure.in is presently broken in the way it tries to do
this. Here's the excerpt from config.log that makes me think so:

int
main ()
{
 makedev(0, 0)
  ;
  return 0;
}
configure:4807: cc -o conftest -g   conftest.c  >&5
cc: Warning: configure, line 4787: # not in column 1 is
ignored, skipping to end of line. (ignoretokens)
    #define _OSF_SOURCE 1
----^
cc: Warning: configure, line 4788: # not in column 1 is
ignored, skipping to end of line. (ignoretokens)
    #include <sys/types.h>
----^
ld:
Unresolved:
makedev
configure:4810: $? = 1
configure: failed program was:
#line 4785 "configure"
#include "confdefs.h"

    #define _OSF_SOURCE 1
    #include <sys/types.h>

#ifdef F77_DUMMY_MAIN
#  ifdef __cplusplus
     extern "C"
#  endif
   int F77_DUMMY_MAIN() { return 1; }
#endif
int
main ()
{
 makedev(0, 0)
  ;
  return 0;
}
configure:4833: result: no


As you can see, the Compaq/HP preprocessor does not seem to
accept preprocessor directives unless the # is in the first
column. The result is that the attempt to determine whether
_OSF_SOURCE should be defined for purposes of "makedev"
itself fails. I don't know enough about the configuration
here to know whether fixing this problem would allow the
_OSF_SOURCE symbol to be present. I am going to try changing
pyconfig.h to explicitly define _OSF_SOURCE and see if
everything else still works. I'll post here after I have
done this experiment. 

So I guess what I'd like to know from you is whether it's
better to pursue the configure.in problem or to try to add a
specific preprocessor directive somewhere in
thread_pthread.h or thereabouts in order to fix this. Thanks!

Here's the relevant portion of <sys/types.h>. If you try to
match up the #ifdef and the #endif statements, you'll see
that there is apparently a missing #endif somewhere after
the definition of P_MYID. But without knowing the
programmer's intent, it's not clear whether the "corrected"
include file would define ushort in any case.

#if defined(_OSF_SOURCE) || defined(_XOPEN_SOURCE_EXTENDED)
#ifdef _OSF_SOURCE

#ifndef NULL
#define
NULL
0L
#endif

#ifndef TRUE
#define TRUE	1
#endif

#ifndef FALSE
#define FALSE	0
#endif

#ifndef _KERNEL
typedef void (*sig_t) __((int,void (*)(int)));
#else
typedef void (*sig_t) __((int));
#endif

#endif /* _OSF_SOURCE */
/*
 * The following type is for various kinds of identifiers.  The
 * actual type must be the same for all since some system calls
 * (such as sigsend) take arguments that may be any of these
 * types.  The enumeration type idtype_t defined in
sys/procset.h
 * is used to indicate what type of id is being specified.
 */
typedef pid_t		id_t;		/* A process id,	*/
					/* process group id,	*/
					/* session id, 		*/
					/* scheduling class id,	*/
					/* user id, or group id.*/
typedef unsigned int	useconds_t;	/* time in microseconds */

#ifdef _OSF_SOURCE
#define
P_MYID
(-1)
/* a usually illegal value for IDs, but specifying
			   whatever the value is for my process */

/*
 * Types used by dev_t annotation macros (see below)
 */
typedef uint_t	major_t;      /* major device number   */
typedef uint_t	minor_t;      /* minor device number   */
typedef uint_t	devs_t;       /* device-specific info  */
typedef uint_t	unit_t;       /* unit number of device */

/*
 * Basic system types and major/minor device
constructing/busting macros.
 */
#define major(x)	((major_t)  (((dev_t)(x)>>20)&07777))
#define minor(x)	((minor_t)  ((dev_t)(x)&03777777))
#define makedev(x,y)	((dev_t)    (((major_t)(x)<<20) | (minor_t)(y)))

/*
 * Disk/Tape (SCSI/CAM - DSA) specific dev_t annotations macros.
 */
#define MAKEMINOR(u,d)  ((minor_t)  (((unit_t)(u)<<6)
|(devs_t)(d)))
#define GETUNIT(dev)	((unit_t)   (minor(dev)>>6)&037777)
#define GETDEVS(dev)	((devs_t)   (minor(dev))&077)
#define MAKECAMMINOR(u,d) ((minor_t)
MAKEMINOR((((u&0770)<<5)|((u&07)<<4)),d))
#define GETCAMUNIT(x)   ((unit_t)
(((GETUNIT(x))>>5)&0770)|((GETUNIT(x)>>4)&07))
#define GETCAMTARG(x)   ((unit_t) ((x >> 3)&07))
#define GETREUNIT(x)	((unit_t) ((x)&07))

#include <mach/machine/vm_types.h>

/*
 * shorthand type definitions for unsigned storage classes
 */
typedef
uchar_t
	uchar;
typedef
ushort_t
ushort;
typedef
uint_t
	uint;
typedef ulong_t		ulong;

typedef
physadr_t
physadr;

#ifdef _OSF_SOURCE
/* typedefs for BSD unsigned things */
typedef
uchar_t
	u_char;
typedef
ushort_t 	u_short;
typedef
uint_t
	u_int;
typedef
ulong_t
	u_long;
typedef
vuchar_t
vu_char;
typedef
vushort_t 	vu_short;
typedef
vuint_t
	vu_int;
typedef
vulong_t
vu_long;
#endif

#ifdef  _KERNEL
typedef struct  _quad { u_int val[2]; } quad;
#else   /* _KERNEL */
typedef struct  _quad { int val[2]; } quad;
#endif  /* _KERNEL */

typedef
long
swblk_t;
typedef u_long	fixpt_t;

#include <sys/select.h>

#endif   /* _OSF_SOURCE */




----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-23 16:32

Message:
Logged In: YES 
user_id=33168

I don't know of any developer that has access to OSF1 v4.0.
 Patches would certainly be welcome. :-)  The first message
appears to be a warning and seems like it can be ignored. 
The two errors from psx4_nspace_ts.h appear to be a problem
with a header file that needs to be included to get the
definition for ushort.  If you can attach a patch to this
bug report, I can check on other architectures.  Thanks.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=691793&group_id=5470


From noreply@sourceforge.net  Sun Feb 23 23:52:44 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sun, 23 Feb 2003 15:52:44 -0800
Subject: [Python-bugs-list] [ python-Bugs-690012 ] 2.3a2 build fails under IRIX 6.5
Message-ID: <E18n5vI-0004eV-00@sc8-sf-web2.sourceforge.net>

Bugs item #690012, was opened at 2003-02-20 07:36
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690012&group_id=5470

Category: Build
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Ralf W. Grosse-Kunstleve (rwgk)
Assigned to: Guido van Rossum (gvanrossum)
Summary: 2.3a2 build fails under IRIX 6.5

Initial Comment:
Using the latest Python 2.3a2 release
./configure; make
fails on this platform:
IRIX rattler 6.5 10120734 IP32
See attached logs.

I've also tried configure --disable-ipv6, but the result is 
the same.

The previous 2.3a1 release works without a problem 
on the exact same machine.


----------------------------------------------------------------------

>Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-23 18:52

Message:
Logged In: YES 
user_id=6380

- I think the renaming shouldn't happen at all unless the
exception is ImportError.

- The if 1 can then be replaced by if isinstance(why,
ImportError).

- The message in the else branch should be changed to
include the exception name.

- I thought I fixed the logging level of these messages, by
adding "level=3", in one of the first checkins after
releasing 2.3a2. But possibly I may have botched that. Can
you check that they really aren't printed?

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-23 16:59

Message:
Logged In: YES 
user_id=33168

Guido, I agree with your comment on python-dev that the fix
below is not best.  Attached is a patch which catches all
exceptions, and changes the warning message appropriately. 
There are 2 other issues.
 1) I don't know why the if 1: is there, should that be removed?
 2) During a normal build these warning messages aren't
printed, shouldn't they be?

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-20 18:14

Message:
Logged In: YES 
user_id=33168

In order to get the build to proceed, you can change line
210 of setup.py from:

        except ImportError, why:
to:
        except (ImportError, RuntimeError), why:

This is just before:
    self.announce('*** WARNING: renaming "%s" since
importing it' ...)



----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690012&group_id=5470


From noreply@sourceforge.net  Sun Feb 23 23:54:11 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sun, 23 Feb 2003 15:54:11 -0800
Subject: [Python-bugs-list] [ python-Feature Requests-691889 ] Add MacOS.framework attribute
Message-ID: <E18n5wh-0004kY-00@sc8-sf-web2.sourceforge.net>

Feature Requests item #691889, was opened at 2003-02-23 22:27
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=691889&group_id=5470

Category: Macintosh
Group: None
Status: Closed
Resolution: Accepted
Priority: 5
Submitted By: John Ehresman (jpe)
Assigned to: Jack Jansen (jackjansen)
Summary: Add MacOS.framework attribute

Initial Comment:
On MacOS X, python may be compiled with a named
framework and extension modules must be linked at build
time against the framework with the same name.  This
new attribute would make the name of the framework
available at runtime.

----------------------------------------------------------------------

>Comment By: John Ehresman (jpe)
Date: 2003-02-23 23:54

Message:
Logged In: YES 
user_id=22785

The revised scheme is fine with me if the framework name is
always going to be Python.  If it is, you might want to
remove the option to set it at configure time.  If there's a
reason for someone to compile with a different framework
name, I think the name should be available at runtime.

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-02-23 23:35

Message:
Logged In: YES 
user_id=45365

Implemented in macosmodule.c rev. 1.65 in a slightly different way. There's now an attribute MacOS.linkmodel which can have one of four values:
'static', 'framework', 'shared' (not used currently, but is there for compleness if I ever get around to fixing non-framework shared-library builds) and 'cfm' (for MacPython-OS9).

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=691889&group_id=5470


From noreply@sourceforge.net  Mon Feb 24 00:07:26 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sun, 23 Feb 2003 16:07:26 -0800
Subject: [Python-bugs-list] [ python-Bugs-691793 ] Python 2.3a2 build fails on Tru64
Message-ID: <E18n69W-0006MQ-00@sc8-sf-web4.sourceforge.net>

Bugs item #691793, was opened at 2003-02-23 11:46
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=691793&group_id=5470

Category: Build
Group: Python 2.3
Status: Open
Resolution: Fixed
Priority: 5
Submitted By: Stephan A. Terre (sfiedler)
Assigned to: Neal Norwitz (nnorwitz)
Summary: Python 2.3a2 build fails on Tru64

Initial Comment:
There is one persistent warning, and then a
build-stopping error, on Tru64. 

% uname -a
OSF1 ruby V4.0 1229 alpha
Ran configure to use the vendor compiler as:

env CC=cc CXX=cxx ./configure
--prefix=/var/tmp/sat/python_test


The configuration appeared to go fine. Upon building,
this warning was repeated numerous times:

cc -c -Olimit 1500 -DNDEBUG -O -I. -I./Include 
-DPy_BUILD_CORE -o Objects/descrobject.o
Objects/descrobject.c
cc: Info: ./Include/objimpl.h, line 252: In this
declaration, type long double has the same
representation as type double on this platform and is
treated as a synonym for type double in this
compilation mode. (longdoubleny1)
        long double dummy;  /* force worst-case
alignment */
--------^


And then these errors appeared:

cc -c -Olimit 1500 -DNDEBUG -O -I. -I./Include 
-DPy_BUILD_CORE -o Python/thread.o Python/thread.c
cc: Error: /usr/include/sys/psx4_nspace_ts.h, line 147:
Missing type specifier or type qualifier. (missingtype)
       ushort           entry_cnt;      /* Number of
key entries total in the
-------^
cc: Error: /usr/include/sys/psx4_nspace_ts.h, line 149:
Missing type specifier or type qualifier. (missingtype)
       ushort           entry_size;     /* size of key
entry */
-------^


I am happy to try candidate fixes from CVS if that
would be useful.

----------------------------------------------------------------------

>Comment By: Stephan A. Terre (sfiedler)
Date: 2003-02-23 17:07

Message:
Logged In: YES 
user_id=246063

I checked the current Python out of CVS. 'configure' worked
fine. Searching config.log for warnings, I find only the
ones about uninitialized variables in test for library
functions, which I believe are fine.

The build succeeded. There was that warning about "long
double" on pretty much every file, and I think perhaps some
other warnings buried in there. I can get a log if you like.

Running the new 'python' produced this warning:

/home/sat/lib/python/startup.py:19: DeprecationWarning:
Non-ASCII character '\xf8' in file
/usr/var/tmp/sat/python/dist/src/Lib/pydoc.py on line 38,
but no encoding declared; see
http://www.python.org/peps/pep-0263.html for details
  from pydoc import help


Running "make test" showed a few errors, but I don't know if
these are specific to Tru64 or just part of this release
being "in process". I'm happy to run any of them verbose,
once I figure out how :)

209 tests OK.
5 tests failed:
    test_cpickle test_datetime test_format test_pep263
test_xpickle
31 tests skipped:
    test_aepack test_al test_audioop test_bsddb test_bsddb3
test_bz2
    test_cd test_cl test_curses test_dl test_email_codecs
test_gdbm
    test_gl test_iconv_codecs test_imageop test_imgfile
    test_linuxaudiodev test_locale test_macfs
test_macostools test_mpz
    test_normalization test_ossaudiodev test_pep277 test_rgbimg
    test_socket_ssl test_socketserver test_sunaudiodev
    test_unicode_file test_winreg test_winsound
Ask someone to teach regrtest.py about which tests are
expected to get skipped on osf1V4.

Note that there were also a couple of similar warnings
during the regression test:

/usr/var/tmp/sat/python/dist/src/Lib/test/test_tarfile.py:6:
DeprecationWarning: Non-ASCII character '\xe4' in file
/usr/var/tmp/sat/python/dist/src/Lib/tarfile.py on line 6,
but no encoding declared; see
http://www.python.org/peps/pep-0263.html for details
  import tarfile

The test suite runs twice for some reason, but I don't know
if that's also expected.

I can definitely try the 2.2.2+ version. Just let me know
the tag or how to find the tag for myself. We've been using
2.2.2 since it came out on a variety of Tru64 4.0 and 5.1
machines, without any problems that seem particular to this
platform. So if I haven't seen the bug you're thinking
about, I might not be able to provide good information as to
whether it's fixed. But I can definitely tell you whether
2.2.2+ works for me.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-23 16:38

Message:
Logged In: YES 
user_id=33168

Stephan, thanks for your help.  Your fixes did seem correct,
although I had already done them, so the fix I checked in
was slightly different.  Could you test out the version in
CVS and verify if it corrects your problem?  Also, I recall
a bug about OSF1 v4.0 for 2.2.2.  These fixes should be
backported, are you able to test the 2.2.2+ version from
CVS?  There is a planned release of 2.2.3 soon.

----------------------------------------------------------------------

Comment By: Stephan A. Terre (sfiedler)
Date: 2003-02-23 16:06

Message:
Logged In: YES 
user_id=246063

Just realized I was unclear in my previous comment. To test
the fix for _OSF_SOURCE as relates to the test for
"makedev", I altered the file "configure", and rebuilt.
Everything was happy.

The file attached to this bug is the result of changing
"configure.in" along the same lines. However, I do not have
autoconf and so did not regenerate "configure" from
"configure.in". I have *not* tested the patch I'm
submitting, just something which I think is a close
approximation. Although the file is named
"configure_patch.txt", it is really a patch to "configure.in".

I can try to get autoconf working on this system and verify
the patch if you would prefer.

----------------------------------------------------------------------

Comment By: Stephan A. Terre (sfiedler)
Date: 2003-02-23 16:01

Message:
Logged In: YES 
user_id=246063

I have verified that everything seems to basically build and
run if I alter the test in configure to put the # at the
first column for preprocessor directives. Attached is a
somewhat more ambitious patch than that: I went through
config.log and found all the places where this seemed to be
happening, and fixed it based on my Very Limited
Understanding of autoconf. I don't have autoconf here to try
this with. So if this is a bad idea, we should solve it some
other way. But hopefully this will work everywhere and will
avoid some other minor problems in the future.

----------------------------------------------------------------------

Comment By: Stephan A. Terre (sfiedler)
Date: 2003-02-23 15:36

Message:
Logged In: YES 
user_id=246063

Well, this is rather interesting. The file psx4_nspace_ts.h
is included by <semaphore.h>, which is included by
Python/thread_pthread.h . The type ushort is defined in
<sys/types.h> if the preprocessor symbol _OSF_SOURCE is
defined (which it is not in this case). Here's where it gets
interesting.

It appears that there's an error in <sys/types.h>, because
there's a mismatched #ifdef check for _OSF_SOURCE. I include
the relevant portion below in this message. So it looks like
the intent might actually be to define (via typedef) ushort
merely if  _XOPEN_SOURCE_EXTENDED is defined (which it is).
I don't know what the Python policy is for broken system
include files. It may be safe to #define _OSF_SOURCE based
on a configure-time check; however, it appears that
configure.in is presently broken in the way it tries to do
this. Here's the excerpt from config.log that makes me think so:

int
main ()
{
 makedev(0, 0)
  ;
  return 0;
}
configure:4807: cc -o conftest -g   conftest.c  >&5
cc: Warning: configure, line 4787: # not in column 1 is
ignored, skipping to end of line. (ignoretokens)
    #define _OSF_SOURCE 1
----^
cc: Warning: configure, line 4788: # not in column 1 is
ignored, skipping to end of line. (ignoretokens)
    #include <sys/types.h>
----^
ld:
Unresolved:
makedev
configure:4810: $? = 1
configure: failed program was:
#line 4785 "configure"
#include "confdefs.h"

    #define _OSF_SOURCE 1
    #include <sys/types.h>

#ifdef F77_DUMMY_MAIN
#  ifdef __cplusplus
     extern "C"
#  endif
   int F77_DUMMY_MAIN() { return 1; }
#endif
int
main ()
{
 makedev(0, 0)
  ;
  return 0;
}
configure:4833: result: no


As you can see, the Compaq/HP preprocessor does not seem to
accept preprocessor directives unless the # is in the first
column. The result is that the attempt to determine whether
_OSF_SOURCE should be defined for purposes of "makedev"
itself fails. I don't know enough about the configuration
here to know whether fixing this problem would allow the
_OSF_SOURCE symbol to be present. I am going to try changing
pyconfig.h to explicitly define _OSF_SOURCE and see if
everything else still works. I'll post here after I have
done this experiment. 

So I guess what I'd like to know from you is whether it's
better to pursue the configure.in problem or to try to add a
specific preprocessor directive somewhere in
thread_pthread.h or thereabouts in order to fix this. Thanks!

Here's the relevant portion of <sys/types.h>. If you try to
match up the #ifdef and the #endif statements, you'll see
that there is apparently a missing #endif somewhere after
the definition of P_MYID. But without knowing the
programmer's intent, it's not clear whether the "corrected"
include file would define ushort in any case.

#if defined(_OSF_SOURCE) || defined(_XOPEN_SOURCE_EXTENDED)
#ifdef _OSF_SOURCE

#ifndef NULL
#define
NULL
0L
#endif

#ifndef TRUE
#define TRUE	1
#endif

#ifndef FALSE
#define FALSE	0
#endif

#ifndef _KERNEL
typedef void (*sig_t) __((int,void (*)(int)));
#else
typedef void (*sig_t) __((int));
#endif

#endif /* _OSF_SOURCE */
/*
 * The following type is for various kinds of identifiers.  The
 * actual type must be the same for all since some system calls
 * (such as sigsend) take arguments that may be any of these
 * types.  The enumeration type idtype_t defined in
sys/procset.h
 * is used to indicate what type of id is being specified.
 */
typedef pid_t		id_t;		/* A process id,	*/
					/* process group id,	*/
					/* session id, 		*/
					/* scheduling class id,	*/
					/* user id, or group id.*/
typedef unsigned int	useconds_t;	/* time in microseconds */

#ifdef _OSF_SOURCE
#define
P_MYID
(-1)
/* a usually illegal value for IDs, but specifying
			   whatever the value is for my process */

/*
 * Types used by dev_t annotation macros (see below)
 */
typedef uint_t	major_t;      /* major device number   */
typedef uint_t	minor_t;      /* minor device number   */
typedef uint_t	devs_t;       /* device-specific info  */
typedef uint_t	unit_t;       /* unit number of device */

/*
 * Basic system types and major/minor device
constructing/busting macros.
 */
#define major(x)	((major_t)  (((dev_t)(x)>>20)&07777))
#define minor(x)	((minor_t)  ((dev_t)(x)&03777777))
#define makedev(x,y)	((dev_t)    (((major_t)(x)<<20) | (minor_t)(y)))

/*
 * Disk/Tape (SCSI/CAM - DSA) specific dev_t annotations macros.
 */
#define MAKEMINOR(u,d)  ((minor_t)  (((unit_t)(u)<<6)
|(devs_t)(d)))
#define GETUNIT(dev)	((unit_t)   (minor(dev)>>6)&037777)
#define GETDEVS(dev)	((devs_t)   (minor(dev))&077)
#define MAKECAMMINOR(u,d) ((minor_t)
MAKEMINOR((((u&0770)<<5)|((u&07)<<4)),d))
#define GETCAMUNIT(x)   ((unit_t)
(((GETUNIT(x))>>5)&0770)|((GETUNIT(x)>>4)&07))
#define GETCAMTARG(x)   ((unit_t) ((x >> 3)&07))
#define GETREUNIT(x)	((unit_t) ((x)&07))

#include <mach/machine/vm_types.h>

/*
 * shorthand type definitions for unsigned storage classes
 */
typedef
uchar_t
	uchar;
typedef
ushort_t
ushort;
typedef
uint_t
	uint;
typedef ulong_t		ulong;

typedef
physadr_t
physadr;

#ifdef _OSF_SOURCE
/* typedefs for BSD unsigned things */
typedef
uchar_t
	u_char;
typedef
ushort_t 	u_short;
typedef
uint_t
	u_int;
typedef
ulong_t
	u_long;
typedef
vuchar_t
vu_char;
typedef
vushort_t 	vu_short;
typedef
vuint_t
	vu_int;
typedef
vulong_t
vu_long;
#endif

#ifdef  _KERNEL
typedef struct  _quad { u_int val[2]; } quad;
#else   /* _KERNEL */
typedef struct  _quad { int val[2]; } quad;
#endif  /* _KERNEL */

typedef
long
swblk_t;
typedef u_long	fixpt_t;

#include <sys/select.h>

#endif   /* _OSF_SOURCE */




----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-23 14:32

Message:
Logged In: YES 
user_id=33168

I don't know of any developer that has access to OSF1 v4.0.
 Patches would certainly be welcome. :-)  The first message
appears to be a warning and seems like it can be ignored. 
The two errors from psx4_nspace_ts.h appear to be a problem
with a header file that needs to be included to get the
definition for ushort.  If you can attach a patch to this
bug report, I can check on other architectures.  Thanks.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=691793&group_id=5470


From noreply@sourceforge.net  Mon Feb 24 00:52:36 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sun, 23 Feb 2003 16:52:36 -0800
Subject: [Python-bugs-list] [ python-Bugs-691793 ] Python 2.3a2 build fails on Tru64
Message-ID: <E18n6rE-00038I-00@sc8-sf-web1.sourceforge.net>

Bugs item #691793, was opened at 2003-02-23 13:46
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=691793&group_id=5470

Category: Build
Group: Python 2.3
>Status: Closed
Resolution: Fixed
Priority: 5
Submitted By: Stephan A. Terre (sfiedler)
Assigned to: Neal Norwitz (nnorwitz)
Summary: Python 2.3a2 build fails on Tru64

Initial Comment:
There is one persistent warning, and then a
build-stopping error, on Tru64. 

% uname -a
OSF1 ruby V4.0 1229 alpha
Ran configure to use the vendor compiler as:

env CC=cc CXX=cxx ./configure
--prefix=/var/tmp/sat/python_test


The configuration appeared to go fine. Upon building,
this warning was repeated numerous times:

cc -c -Olimit 1500 -DNDEBUG -O -I. -I./Include 
-DPy_BUILD_CORE -o Objects/descrobject.o
Objects/descrobject.c
cc: Info: ./Include/objimpl.h, line 252: In this
declaration, type long double has the same
representation as type double on this platform and is
treated as a synonym for type double in this
compilation mode. (longdoubleny1)
        long double dummy;  /* force worst-case
alignment */
--------^


And then these errors appeared:

cc -c -Olimit 1500 -DNDEBUG -O -I. -I./Include 
-DPy_BUILD_CORE -o Python/thread.o Python/thread.c
cc: Error: /usr/include/sys/psx4_nspace_ts.h, line 147:
Missing type specifier or type qualifier. (missingtype)
       ushort           entry_cnt;      /* Number of
key entries total in the
-------^
cc: Error: /usr/include/sys/psx4_nspace_ts.h, line 149:
Missing type specifier or type qualifier. (missingtype)
       ushort           entry_size;     /* size of key
entry */
-------^


I am happy to try candidate fixes from CVS if that
would be useful.

----------------------------------------------------------------------

>Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-23 19:52

Message:
Logged In: YES 
user_id=33168

This problem is fixed, so closing bug.  Responded to other
issues via email.

----------------------------------------------------------------------

Comment By: Stephan A. Terre (sfiedler)
Date: 2003-02-23 19:07

Message:
Logged In: YES 
user_id=246063

I checked the current Python out of CVS. 'configure' worked
fine. Searching config.log for warnings, I find only the
ones about uninitialized variables in test for library
functions, which I believe are fine.

The build succeeded. There was that warning about "long
double" on pretty much every file, and I think perhaps some
other warnings buried in there. I can get a log if you like.

Running the new 'python' produced this warning:

/home/sat/lib/python/startup.py:19: DeprecationWarning:
Non-ASCII character '\xf8' in file
/usr/var/tmp/sat/python/dist/src/Lib/pydoc.py on line 38,
but no encoding declared; see
http://www.python.org/peps/pep-0263.html for details
  from pydoc import help


Running "make test" showed a few errors, but I don't know if
these are specific to Tru64 or just part of this release
being "in process". I'm happy to run any of them verbose,
once I figure out how :)

209 tests OK.
5 tests failed:
    test_cpickle test_datetime test_format test_pep263
test_xpickle
31 tests skipped:
    test_aepack test_al test_audioop test_bsddb test_bsddb3
test_bz2
    test_cd test_cl test_curses test_dl test_email_codecs
test_gdbm
    test_gl test_iconv_codecs test_imageop test_imgfile
    test_linuxaudiodev test_locale test_macfs
test_macostools test_mpz
    test_normalization test_ossaudiodev test_pep277 test_rgbimg
    test_socket_ssl test_socketserver test_sunaudiodev
    test_unicode_file test_winreg test_winsound
Ask someone to teach regrtest.py about which tests are
expected to get skipped on osf1V4.

Note that there were also a couple of similar warnings
during the regression test:

/usr/var/tmp/sat/python/dist/src/Lib/test/test_tarfile.py:6:
DeprecationWarning: Non-ASCII character '\xe4' in file
/usr/var/tmp/sat/python/dist/src/Lib/tarfile.py on line 6,
but no encoding declared; see
http://www.python.org/peps/pep-0263.html for details
  import tarfile

The test suite runs twice for some reason, but I don't know
if that's also expected.

I can definitely try the 2.2.2+ version. Just let me know
the tag or how to find the tag for myself. We've been using
2.2.2 since it came out on a variety of Tru64 4.0 and 5.1
machines, without any problems that seem particular to this
platform. So if I haven't seen the bug you're thinking
about, I might not be able to provide good information as to
whether it's fixed. But I can definitely tell you whether
2.2.2+ works for me.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-23 18:38

Message:
Logged In: YES 
user_id=33168

Stephan, thanks for your help.  Your fixes did seem correct,
although I had already done them, so the fix I checked in
was slightly different.  Could you test out the version in
CVS and verify if it corrects your problem?  Also, I recall
a bug about OSF1 v4.0 for 2.2.2.  These fixes should be
backported, are you able to test the 2.2.2+ version from
CVS?  There is a planned release of 2.2.3 soon.

----------------------------------------------------------------------

Comment By: Stephan A. Terre (sfiedler)
Date: 2003-02-23 18:06

Message:
Logged In: YES 
user_id=246063

Just realized I was unclear in my previous comment. To test
the fix for _OSF_SOURCE as relates to the test for
"makedev", I altered the file "configure", and rebuilt.
Everything was happy.

The file attached to this bug is the result of changing
"configure.in" along the same lines. However, I do not have
autoconf and so did not regenerate "configure" from
"configure.in". I have *not* tested the patch I'm
submitting, just something which I think is a close
approximation. Although the file is named
"configure_patch.txt", it is really a patch to "configure.in".

I can try to get autoconf working on this system and verify
the patch if you would prefer.

----------------------------------------------------------------------

Comment By: Stephan A. Terre (sfiedler)
Date: 2003-02-23 18:01

Message:
Logged In: YES 
user_id=246063

I have verified that everything seems to basically build and
run if I alter the test in configure to put the # at the
first column for preprocessor directives. Attached is a
somewhat more ambitious patch than that: I went through
config.log and found all the places where this seemed to be
happening, and fixed it based on my Very Limited
Understanding of autoconf. I don't have autoconf here to try
this with. So if this is a bad idea, we should solve it some
other way. But hopefully this will work everywhere and will
avoid some other minor problems in the future.

----------------------------------------------------------------------

Comment By: Stephan A. Terre (sfiedler)
Date: 2003-02-23 17:36

Message:
Logged In: YES 
user_id=246063

Well, this is rather interesting. The file psx4_nspace_ts.h
is included by <semaphore.h>, which is included by
Python/thread_pthread.h . The type ushort is defined in
<sys/types.h> if the preprocessor symbol _OSF_SOURCE is
defined (which it is not in this case). Here's where it gets
interesting.

It appears that there's an error in <sys/types.h>, because
there's a mismatched #ifdef check for _OSF_SOURCE. I include
the relevant portion below in this message. So it looks like
the intent might actually be to define (via typedef) ushort
merely if  _XOPEN_SOURCE_EXTENDED is defined (which it is).
I don't know what the Python policy is for broken system
include files. It may be safe to #define _OSF_SOURCE based
on a configure-time check; however, it appears that
configure.in is presently broken in the way it tries to do
this. Here's the excerpt from config.log that makes me think so:

int
main ()
{
 makedev(0, 0)
  ;
  return 0;
}
configure:4807: cc -o conftest -g   conftest.c  >&5
cc: Warning: configure, line 4787: # not in column 1 is
ignored, skipping to end of line. (ignoretokens)
    #define _OSF_SOURCE 1
----^
cc: Warning: configure, line 4788: # not in column 1 is
ignored, skipping to end of line. (ignoretokens)
    #include <sys/types.h>
----^
ld:
Unresolved:
makedev
configure:4810: $? = 1
configure: failed program was:
#line 4785 "configure"
#include "confdefs.h"

    #define _OSF_SOURCE 1
    #include <sys/types.h>

#ifdef F77_DUMMY_MAIN
#  ifdef __cplusplus
     extern "C"
#  endif
   int F77_DUMMY_MAIN() { return 1; }
#endif
int
main ()
{
 makedev(0, 0)
  ;
  return 0;
}
configure:4833: result: no


As you can see, the Compaq/HP preprocessor does not seem to
accept preprocessor directives unless the # is in the first
column. The result is that the attempt to determine whether
_OSF_SOURCE should be defined for purposes of "makedev"
itself fails. I don't know enough about the configuration
here to know whether fixing this problem would allow the
_OSF_SOURCE symbol to be present. I am going to try changing
pyconfig.h to explicitly define _OSF_SOURCE and see if
everything else still works. I'll post here after I have
done this experiment. 

So I guess what I'd like to know from you is whether it's
better to pursue the configure.in problem or to try to add a
specific preprocessor directive somewhere in
thread_pthread.h or thereabouts in order to fix this. Thanks!

Here's the relevant portion of <sys/types.h>. If you try to
match up the #ifdef and the #endif statements, you'll see
that there is apparently a missing #endif somewhere after
the definition of P_MYID. But without knowing the
programmer's intent, it's not clear whether the "corrected"
include file would define ushort in any case.

#if defined(_OSF_SOURCE) || defined(_XOPEN_SOURCE_EXTENDED)
#ifdef _OSF_SOURCE

#ifndef NULL
#define
NULL
0L
#endif

#ifndef TRUE
#define TRUE	1
#endif

#ifndef FALSE
#define FALSE	0
#endif

#ifndef _KERNEL
typedef void (*sig_t) __((int,void (*)(int)));
#else
typedef void (*sig_t) __((int));
#endif

#endif /* _OSF_SOURCE */
/*
 * The following type is for various kinds of identifiers.  The
 * actual type must be the same for all since some system calls
 * (such as sigsend) take arguments that may be any of these
 * types.  The enumeration type idtype_t defined in
sys/procset.h
 * is used to indicate what type of id is being specified.
 */
typedef pid_t		id_t;		/* A process id,	*/
					/* process group id,	*/
					/* session id, 		*/
					/* scheduling class id,	*/
					/* user id, or group id.*/
typedef unsigned int	useconds_t;	/* time in microseconds */

#ifdef _OSF_SOURCE
#define
P_MYID
(-1)
/* a usually illegal value for IDs, but specifying
			   whatever the value is for my process */

/*
 * Types used by dev_t annotation macros (see below)
 */
typedef uint_t	major_t;      /* major device number   */
typedef uint_t	minor_t;      /* minor device number   */
typedef uint_t	devs_t;       /* device-specific info  */
typedef uint_t	unit_t;       /* unit number of device */

/*
 * Basic system types and major/minor device
constructing/busting macros.
 */
#define major(x)	((major_t)  (((dev_t)(x)>>20)&07777))
#define minor(x)	((minor_t)  ((dev_t)(x)&03777777))
#define makedev(x,y)	((dev_t)    (((major_t)(x)<<20) | (minor_t)(y)))

/*
 * Disk/Tape (SCSI/CAM - DSA) specific dev_t annotations macros.
 */
#define MAKEMINOR(u,d)  ((minor_t)  (((unit_t)(u)<<6)
|(devs_t)(d)))
#define GETUNIT(dev)	((unit_t)   (minor(dev)>>6)&037777)
#define GETDEVS(dev)	((devs_t)   (minor(dev))&077)
#define MAKECAMMINOR(u,d) ((minor_t)
MAKEMINOR((((u&0770)<<5)|((u&07)<<4)),d))
#define GETCAMUNIT(x)   ((unit_t)
(((GETUNIT(x))>>5)&0770)|((GETUNIT(x)>>4)&07))
#define GETCAMTARG(x)   ((unit_t) ((x >> 3)&07))
#define GETREUNIT(x)	((unit_t) ((x)&07))

#include <mach/machine/vm_types.h>

/*
 * shorthand type definitions for unsigned storage classes
 */
typedef
uchar_t
	uchar;
typedef
ushort_t
ushort;
typedef
uint_t
	uint;
typedef ulong_t		ulong;

typedef
physadr_t
physadr;

#ifdef _OSF_SOURCE
/* typedefs for BSD unsigned things */
typedef
uchar_t
	u_char;
typedef
ushort_t 	u_short;
typedef
uint_t
	u_int;
typedef
ulong_t
	u_long;
typedef
vuchar_t
vu_char;
typedef
vushort_t 	vu_short;
typedef
vuint_t
	vu_int;
typedef
vulong_t
vu_long;
#endif

#ifdef  _KERNEL
typedef struct  _quad { u_int val[2]; } quad;
#else   /* _KERNEL */
typedef struct  _quad { int val[2]; } quad;
#endif  /* _KERNEL */

typedef
long
swblk_t;
typedef u_long	fixpt_t;

#include <sys/select.h>

#endif   /* _OSF_SOURCE */




----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-23 16:32

Message:
Logged In: YES 
user_id=33168

I don't know of any developer that has access to OSF1 v4.0.
 Patches would certainly be welcome. :-)  The first message
appears to be a warning and seems like it can be ignored. 
The two errors from psx4_nspace_ts.h appear to be a problem
with a header file that needs to be included to get the
definition for ushort.  If you can attach a patch to this
bug report, I can check on other architectures.  Thanks.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=691793&group_id=5470


From noreply@sourceforge.net  Mon Feb 24 01:29:20 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sun, 23 Feb 2003 17:29:20 -0800
Subject: [Python-bugs-list] [ python-Bugs-610299 ] unicode alphanumeric regexp bug
Message-ID: <E18n7Qm-00022s-00@sc8-sf-web3.sourceforge.net>

Bugs item #610299, was opened at 2002-09-16 21:18
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=610299&group_id=5470

Category: Regular Expressions
Group: Python 2.3
>Status: Closed
>Resolution: Fixed
Priority: 5
Submitted By: Florent Guillaume (efge)
>Assigned to: Guido van Rossum (gvanrossum)
Summary: unicode alphanumeric regexp bug

Initial Comment:
I've got the following problem, in python 2.1, 2.2 and
2.3a0 (Debian):

>>> import re
>>> re.compile(r'\w+', re.U).sub('X', u'hello caf\xe9')
u'X X'
>>> re.compile(r'\w{1}', re.U).sub('X', u'hello caf\xe9')
u'XXXXX XXXX'
>>> re.compile(r'\w', re.U).sub('X', u'hello caf\xe9')
u'XXXXX XXX\xe9'

The first two results are ok, but the third is not.


----------------------------------------------------------------------

>Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-23 20:29

Message:
Logged In: YES 
user_id=6380

Fixed in 2.3 CVS using Greg's patch. Will backport to 2.2 as
well.

----------------------------------------------------------------------

Comment By: Greg Chapman (glchapman)
Date: 2002-11-04 11:51

Message:
Logged In: YES 
user_id=86307

I just posted a small patch to sre_compile.py which should fix this:

http://sourceforge.net/tracker/?
func=detail&aid=633359&group_id=5470&atid=305470

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=610299&group_id=5470


From noreply@sourceforge.net  Mon Feb 24 01:30:59 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sun, 23 Feb 2003 17:30:59 -0800
Subject: [Python-bugs-list] [ python-Bugs-690012 ] 2.3a2 build fails under IRIX 6.5
Message-ID: <E18n7SN-0007qo-00@sc8-sf-web2.sourceforge.net>

Bugs item #690012, was opened at 2003-02-20 07:36
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690012&group_id=5470

Category: Build
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Ralf W. Grosse-Kunstleve (rwgk)
Assigned to: Guido van Rossum (gvanrossum)
Summary: 2.3a2 build fails under IRIX 6.5

Initial Comment:
Using the latest Python 2.3a2 release
./configure; make
fails on this platform:
IRIX rattler 6.5 10120734 IP32
See attached logs.

I've also tried configure --disable-ipv6, but the result is 
the same.

The previous 2.3a1 release works without a problem 
on the exact same machine.


----------------------------------------------------------------------

>Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-23 20:30

Message:
Logged In: YES 
user_id=6380

(Note, there are other compilation errors in the attached logs.)

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-23 18:52

Message:
Logged In: YES 
user_id=6380

- I think the renaming shouldn't happen at all unless the
exception is ImportError.

- The if 1 can then be replaced by if isinstance(why,
ImportError).

- The message in the else branch should be changed to
include the exception name.

- I thought I fixed the logging level of these messages, by
adding "level=3", in one of the first checkins after
releasing 2.3a2. But possibly I may have botched that. Can
you check that they really aren't printed?

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-23 16:59

Message:
Logged In: YES 
user_id=33168

Guido, I agree with your comment on python-dev that the fix
below is not best.  Attached is a patch which catches all
exceptions, and changes the warning message appropriately. 
There are 2 other issues.
 1) I don't know why the if 1: is there, should that be removed?
 2) During a normal build these warning messages aren't
printed, shouldn't they be?

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-20 18:14

Message:
Logged In: YES 
user_id=33168

In order to get the build to proceed, you can change line
210 of setup.py from:

        except ImportError, why:
to:
        except (ImportError, RuntimeError), why:

This is just before:
    self.announce('*** WARNING: renaming "%s" since
importing it' ...)



----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690012&group_id=5470


From noreply@sourceforge.net  Mon Feb 24 04:30:37 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sun, 23 Feb 2003 20:30:37 -0800
Subject: [Python-bugs-list] [ python-Bugs-692016 ] LibRef 4.2.1: {m,n} description update
Message-ID: <E18nAGD-0001Lj-00@sc8-sf-web1.sourceforge.net>

Bugs item #692016, was opened at 2003-02-23 23:30
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=692016&group_id=5470

Category: Documentation
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Terry J. Reedy (tjreedy)
Assigned to: Nobody/Anonymous (nobody)
Summary: LibRef 4.2.1: {m,n} description update

Initial Comment:
LibRef 4.2.1 Regular Expression Syntax 

{m,n} "...you can't omit m...."

Actually, you can: for sre, it defaults to 0

>>> a4=re.compile('a{,4}')
>>> a4.match('').group(0)
''
>>> a4.match('aaaaa').group(0)
'aaaa'

This is predictable from sre_parse.py code
            elif this == "{":
                min, max = 0, MAXREPEAT
                /* lo set to digits between { and ,.*/
                if lo: 
                    min = atoi(lo)
 
Result for pre seems buggy: compiles but does not 
match.  (so 'can't' is sort of correct, but not in way 
expected -- by raising exception):

>>> import pre
>>> pa4=pre.compile('a{,4}')
>>> pa4.match('') # None response
>>> pa4.match('aaaaa').group(0)
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
AttributeError: 'NoneType' object has no 
attribute 'group'

So, suggested replacement:
"For sre, m defaults to 0; for older pre, missing m 
compiles but does not match."

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=692016&group_id=5470


From noreply@sourceforge.net  Mon Feb 24 07:13:18 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Sun, 23 Feb 2003 23:13:18 -0800
Subject: [Python-bugs-list] [ python-Bugs-216289 ] Programs using Tkinter sometimes can't shut down (Windows)
Message-ID: <E18nCne-0002hB-00@sc8-sf-web3.sourceforge.net>

Bugs item #216289, was opened at 2000-10-07 04:25
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=216289&group_id=5470

Category: Windows
Group: Python 2.3
Status: Open
Resolution: Later
Priority: 3
Submitted By: Tim Peters (tim_one)
Assigned to: Fredrik Lundh (effbot)
Summary: Programs using Tkinter sometimes can't shut down (Windows)

Initial Comment:
The following msg from the Tutor list is about 1.6, but I noticed the same thing several times today using 2.0b2+CVS.  In my case, I was running IDLE via

python ../tool/idle/idle.pyw

from a DOS box in my PCbuild directory.  Win98SE.  *Most* of the time, shutting down IDLE via Ctrl+Q left the DOS box hanging.  As with the poster, the only way to regain control was to use the Task Manager to kill off Winoldap.

-----Original Message-----
From: Joseph Stubenrauch <nothingisgoingtochangemyworld@yahoo.com>
Sent: Friday, October 06, 2000 9:23 PM
To: tutor@python.org
Subject: Re: [Tutor] Python 1.6 BUG


Strange, I have been experiencing the same bug myself.
Here's the low down for me:

Python 1.6 with win95
I am running a little Tkinter program
The command line I use is simply: "python foo.py"
About 25-35% of the time, when I close the Tkinter
window, DOS seems to "freeze" and never returns to the
c:\ command prompt.  I have to ctrl-alt-delete
repeatedly and shut down "winoldapp" to get rid of the
window and then shell back into DOS and keep working.
It's a bit of a pain, since I have the habit of
testing EVERYTHING in tiny little stages, so I change
one little thing, test it ... freeze ... ARGH!  Change
one more tiny thing, test it ... freeze ... ARGH! 
However, sometimes it seems to behave and doesn't
bother me for an entire several hour session of python
work.

That's my report on the problem.

Cheers,

Joe


----------------------------------------------------------------------

>Comment By: Fredrik Lundh (effbot)
Date: 2003-02-24 08:13

Message:
Logged In: YES 
user_id=38376

(a bugfix a day keeps etc)

Tim, can we close this one?  I don't have access to any-
thing even remotely similar to a Win95-box any longer...

----------------------------------------------------------------------

Comment By: Jeffrey Hobbs (hobbs)
Date: 2002-12-10 05:16

Message:
Logged In: YES 
user_id=72656

Interesting last couple of comments ...

Anyway, I think the cause of this problem may have finally 
been identified by Tcl bug 651139.  I have attached a patch 
here for it against 8.3.5.  Both the 8.3.5 branch and 8.4.1+ 
head have this fix.  Amazingly it didn't effect Tcl, but could 
likely have caught Tkinter (only on Windows with non-
threaded builds).

Oh look, not allowed to attach a patch, so I'll attach it to Tcl 
bug 651139.

----------------------------------------------------------------------

Comment By: christian soldan (elacademico)
Date: 2002-12-01 03:07

Message:
Logged In: YES 
user_id=659858

how do i hack my fuckers friends

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2002-10-01 01:23

Message:
Logged In: NO 

i would like to know some moves

----------------------------------------------------------------------

Comment By: John Popplewell (johnnypops)
Date: 2002-08-22 02:08

Message:
Logged In: YES 
user_id=143340

Hi,

tried wrapping the WaitForSingleObject() with if (consolePtr-
>toWrite) { ... as suggested. Solves the problem for me in 
debug and release builds (without thread support).

What is "the tcl83.dll from www.tcl.tk", what changes does 
it include?

cheers,
John.


----------------------------------------------------------------------

Comment By: Doug Henderson (djhender)
Date: 2002-08-17 01:03

Message:
Logged In: YES 
user_id=148444

I downloaded and tested the tcl83.dll from www.tcl.tk with 
Python 2.2.1 on Win98SE using my BugDemo.py script 
from a related bug report [231207]. Several test runs 
showed that a problem still exists. The app hangs until 
Winopldapp is killed, however this leaves tcl83.dll in use. 
The problem does not occur on every execution. Some 
types of activity seem to provoke the hang, for instance, 
minimizing and maximizing the window seems to increase 
the change that it will hang on exit.

----------------------------------------------------------------------

Comment By: David Gravereaux (davygrvy)
Date: 2002-08-16 23:53

Message:
Logged In: YES 
user_id=7549

Whoop, error in the patch..  consoleMutex might not need to 
be locked.  It could cause a new dead-lock.

----------------------------------------------------------------------

Comment By: David Gravereaux (davygrvy)
Date: 2002-08-16 23:41

Message:
Logged In: YES 
user_id=7549

>It is hanging up in ConsoleCloseProc() in 'tclWinConsole.c'
>at about line 527:
>WaitForSingleObject(consolePtr->writeThread, INFINITE);

Jeff, this might need the same attention the pipe reader 
thread got for that [exec] memory leak bug.  I think 
ConsoleWriterThread() should use WaitForMultipleObjects 
instead of WaitForSingleObject with a second event handle 
used to single a fall-out like the pipe reader thread does.  It 
appears that ConsoleCloseProc() has no way to signal the 
writer thread to close at all.

This doesn't address the blocking WriteFile() in 
ConsoleWriterThread(), though.  Is it blocking there, too?
Sorry for the large post.  I'm given the option to attach a file.

Index: tclWinConsole.c
=============================================
======================
RCS file: /cvsroot/tcl/tcl/win/tclWinConsole.c,v
retrieving revision 1.7
diff -c -r1.7 tclWinConsole.c
*** tclWinConsole.c	15 Jan 2002 17:55:30 -0000	1.7
--- tclWinConsole.c	16 Aug 2002 21:31:56 -0000
***************
*** 79,84 ****
--- 79,87 ----
      HANDLE startWriter;		/* Auto-reset 
event used by the main thread to
  				 * signal when 
the writer thread should attempt
  				 * to write to the 
console. */
+     HANDLE stopWriter;		/* Auto-reset 
event used by the main thread to
+ 				 * signal when 
the writer thread should
+ 				 * terminate. */
      HANDLE startReader;		/* Auto-reset 
event used by the main thread to
  				 * signal when 
the reader thread should attempt
  				 * to read from 
the console. */
***************
*** 516,522 ****
  	 */
  
  	Tcl_MutexLock(&consoleMutex);
! 	TerminateThread(consolePtr->writeThread, 0);
  
  	/*
  	 * Wait for the thread to terminate.  This ensures 
that we are
--- 519,525 ----
  	 */
  
  	Tcl_MutexLock(&consoleMutex);
! 	SetEvent(consolePtr->stopWriter);
  
  	/*
  	 * Wait for the thread to terminate.  This ensures 
that we are
***************
*** 1134,1149 ****
  {
  
      ConsoleInfo *infoPtr = (ConsoleInfo *)arg;
      HANDLE *handle = infoPtr->handle;
      DWORD count, toWrite;
      char *buf;
  
      for (;;) {
  	/*
  	 * Wait for the main thread to signal before 
attempting to write.
  	 */
  
! 	WaitForSingleObject(infoPtr->startWriter, 
INFINITE);
  
  	buf = infoPtr->writeBuf;
  	toWrite = infoPtr->toWrite;
--- 1137,1163 ----
  {
  
      ConsoleInfo *infoPtr = (ConsoleInfo *)arg;
+     HANDLE events[2];
      HANDLE *handle = infoPtr->handle;
      DWORD count, toWrite;
      char *buf;
  
+     events[0] = infoPtr->stopWriter;
+     events[1] = infoPtr->startWriter;
+ 
      for (;;) {
+ 	DWORD dwWait;
+ 
  	/*
  	 * Wait for the main thread to signal before 
attempting to write.
  	 */
  
! 	dwWait = WaitForMultipleObjects(events, 2, 
FALSE, INFINITE);
! 
! 	if (dwWait != (WAIT_OBJECT_0 + 1)) {
! 	    /* either the stop event or some other error, so 
exit */
! 	    return 0;
! 	}
  
  	buf = infoPtr->writeBuf;
  	toWrite = infoPtr->toWrite;
***************
*** 1251,1256 ****
--- 1265,1271 ----
      if (permissions & TCL_WRITABLE) {
  	infoPtr->writable = CreateEvent(NULL, TRUE, 
TRUE, NULL);
  	infoPtr->startWriter = CreateEvent(NULL, FALSE, 
FALSE, NULL);
+ 	infoPtr->stopWriter = CreateEvent(NULL, FALSE, 
FALSE, NULL);
  	infoPtr->writeThread = CreateThread(NULL, 8000, 
ConsoleWriterThread,
  	        infoPtr, 0, &id);
      }


----------------------------------------------------------------------

Comment By: Jeffrey Hobbs (hobbs)
Date: 2002-08-16 20:56

Message:
Logged In: YES 
user_id=72656

I'm finding it a bit hard to reproduce, and now not at all.  
Could people please try replacing their current tcl83.dll with 
the one at http://www.tcl.tk/tcl83.dll.  The sig on it is:
634880 Aug 16 10:53 tcl83.dll
(md5) b35628568ddc4afed4f675d2d9a50faf  tcl83.dll

I can't hang anymore with python 2.2.1 at the Win98 
command prompt with:
   python \Python22\Tools\idle\idle.pyw

----------------------------------------------------------------------

Comment By: Jeffrey Hobbs (hobbs)
Date: 2002-08-16 19:50

Message:
Logged In: YES 
user_id=72656

johnny - That actually helps a lot and fairly clearly indicates 
the source of this hanging problem.  To give more context to 
that area, this is in the final shutdown code where it is 
(obviously) shutting down the console I/O.  The reader is 
shutdown without a check for more input (why, we are 
exiting?), so there is no issue there.

The writer however wants to make sure that the console 
output channel is drained before exit.  A blocking channel 
that is not being flushed will cause a hang.

So the question is, what is the stdio setup for Tcl in 
Py/Tkinter?  That should be examined to see whether stdio is 
clear to flush on exit.

Perhaps you can also help with more debugging.  Could you 
wrap the problematic WaitForSingleObject call with
if (consolePtr->toWrite) { ...
and perhaps also print out what to write.  It may be that 
blocking output isn't the problem at all, but rather that we are 
never receiving the signal on consolePtr->writable (usually 
done in the writer thread).  The consolePtr->toWrite check 
will at least ensure flushed channels do not enter this 
potential hanging state.

----------------------------------------------------------------------

Comment By: John Popplewell (johnnypops)
Date: 2002-08-16 16:16

Message:
Logged In: YES 
user_id=143340

I have dug out the setup I used previously and, apart from 
using Python 2.2 instead of 2.0.1, have had a go at 
reproducing my previous results.

With a debug build of Tcl/Tk and threads disabled I have 
managed to get this stack trace when it locks on exit (after 
doing a break):

ConsoleCloseProc(int * 0x006a0ae4, Tcl_Interp * 
0x00000000) line 527 + 17 bytes
TclFinalizeIOSubsystem() line 241 + 20 bytes
Tcl_FinalizeThread() line 911
Tcl_Finalize() line 812
DllMain(HINSTANCE__ * 0x00970000, unsigned long 
0x00000000, void * 0x00000001) line 197
TCL83! _DllMainCRTStartup@12 + 80 bytes
KERNEL32! bff7ddd6()
KERNEL32! bff8d123()
8176b5fc()

It is hanging up in ConsoleCloseProc() in 'tclWinConsole.c'
at about line 527:
WaitForSingleObject(consolePtr->writeThread, INFINITE);

I tried changing the timeout from INFINITE to 100ms and 
testing the return value for WAIT_TIMEOUT. Sure enough it 
occasionaly printed a message (and didn't lock-up).

Don't know if this helps. Will try to reproduce the double 
free problem in the threaded build later,

cheers,
John.


----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2002-08-15 16:19

Message:
Logged In: YES 
user_id=6380

Just a note so I won't forget this: Jeff Hobbs once said the
code to look at is

tk/generic/tkConsole.c:Tk_InitConsoleChannels

----------------------------------------------------------------------

Comment By: David Gravereaux (davygrvy)
Date: 2002-08-15 15:23

Message:
Logged In: YES 
user_id=7549

Hi John,

I have seen Tcl extension DLLs that set exit handlers get 
called by Tcl after they were torn-down.  NT and Win9x seem 
to be a bit different in the order of what DLLs get torn-down.  
here's a note from some old code of mine:

#ifdef __WIN32__
    // It can happen that the HEAP could have already been 
unloaded
    // from an awkward teardown caused by a Ctrl+C or other.  
Win32
    // seems to do a teardown in reverse order and by the time 
Tcl
    // knows what's going on and Tcl_Finalize calls the exit
    // handlers, this extension's data (heap?) has already been
    // unloaded by the OS.  Do a quick legal check on the 
pointer
    // first.
    //
    if (IsBadReadPtr(adapt, sizeof(T *))) return;
#endif

what's the back/stacktrace?  who is calling 
Tcl_FinalizeNotifier?  And had anyone called it before during 
the shutdown phase? I can't say with much confidence that 
this is happening here, too.  Maybe..  Could Tk be telling Tcl 
shutdown?  That's sort of backwards, if so.

I run w2k here, so I can't help in debugging from a test case.  
tclsh with a threaded build doesn't even call Tcl_Finalize to 
avoid all this!

----------------------------------------------------------------------

Comment By: Donal K. Fellows (dkf)
Date: 2002-08-15 15:09

Message:
Logged In: YES 
user_id=79902

Firstly, on the matter of threads.  Tcl/Tk on Windows uses
threads internally even in non-threaded builds.  That's
Windows for you (in particular, you can get calls parachuted
in from completely separate threads with relatively little
warning.  Also, you need threaded helpers to handle
non-blocking waiting on some kinds of objects.)

It's a bit hard to work out if the problem's been fixed
without a stack-trace (preferably with debugging symbols
enabled so that we can see what the function names and
arguments to those functions are.)  I have no  chance of
replicating this bug here (I'm currently running IRIX, but
also have some access to Solaris and Linux...)

----------------------------------------------------------------------

Comment By: John Popplewell (johnnypops)
Date: 2002-08-15 14:15

Message:
Logged In: YES 
user_id=143340

Hi to davygrvy,

my fix isn't for a thread related (deadlock) type problem - it 
is a fix for a double free type problem which is presumably 
screwing up the heap and causing the C runtime to hang.

It *is* with a threaded build (THREADDEFINES = -
DTCL_THREADS=1) which I think I enabled because I was 
using  'after' and it didn't work unless threading was 
enabled - but I may be mis-remembering here!

Also, I ended up doing all this with the debug build and 
found the -GZ flag useful - in fact it pointed out the fact that 
there was a problem with the heap.

It is worth noting that I have only seen this on Win98 and 
that using 'pythonw.exe' did *not* prevent this from 
happening,

regards John.


----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2002-08-15 03:00

Message:
Logged In: YES 
user_id=6380

AFAIK we never distributed a threaded build.

----------------------------------------------------------------------

Comment By: David Gravereaux (davygrvy)
Date: 2002-08-15 02:01

Message:
Logged In: YES 
user_id=7549

To johnnypops,

Where in Tcl_FinalizeNotifier() is the dead-lock occuring and 
is this with a threaded build of Tcl?


----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2002-07-02 21:39

Message:
Logged In: YES 
user_id=31435

Jeff, a long time ago Fredrik Lundh said he was able to 
reproduce this with a short wish script (no Python, no Tkinter, 
just Tcl/Tk + wish).  I'm reassigning this to /F in the hopes 
he'll remember how.

----------------------------------------------------------------------

Comment By: Jeffrey Hobbs (hobbs)
Date: 2002-07-02 20:55

Message:
Logged In: YES 
user_id=72656

I appreciate that many others see this is a problem in 
embedding Tcl, but without any clear way to reproduce this in 
Tcl/Tk itself, it's very hard for me to fix it.  I don't think that 
there is anything done in 8.4 to fix this that I recall, but if 
someone finds that the problem "goes away" with 8.4, please 
report that.

For those of course that experience this with python, the 
simple and "correct" work-around is to launch with 
pythonw.exe instead of python.exe.

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2002-06-26 19:40

Message:
Logged In: NO 

According to http://tcl.activestate.com ActiveTcl 8.4.0 is now 
available for download.  It is new as of June 24.  I don't know 
whether this will solve our problems or not, but it's worth 
looking into.

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2002-06-19 15:36

Message:
Logged In: NO 

I just downloaded Ruby 1.6.6 for Windows, which also uses 
Tcl/Tk 8.3 and I experience exactly the same problem there. 
So this is definitely a Tcl problem (but we know that already).

----------------------------------------------------------------------

Comment By: John Popplewell (johnnypops)
Date: 2002-06-07 21:01

Message:
Logged In: YES 
user_id=143340

Hmmm. 
I run a Python Tcl/Tk application several times every day - 
the very application that prompted my bug-hunt.

I do use "pythonw.exe" to launch it, but it used to hang 
almost as badly as "python.exe".

Whilst developing this app I used "python.exe" and 
experienced no hang-ups.

The Tcl guys point out that this bug is only one of several 
possible causes of the hang-up, maybe you are just out of 
luck.

The patch has fixed the problem on three Win98 machines, 
all of which exhibited the problem before updating the DLLs.

One thought - are you sure there weren't multiple copies of 
the tcl83.dll and tk83.dll files? My (only) copies were 
in "D:\python20\DLLs"

good luck,
John.


----------------------------------------------------------------------

Comment By: Jeffrey Hobbs (hobbs)
Date: 2002-06-03 20:55

Message:
Logged In: YES 
user_id=72656

providing Tim with a fixed DLL exactly according to John 
Popplewell's patch did not fix the problem for him.  That 
makes it seem fishy that John really didn't see the problem 
any more.

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2002-06-03 18:59

Message:
Logged In: YES 
user_id=31435

I tried replacing Python 2.2.1's tk83.dll and tcl83.dll (from 
Tcl/Tk 8.3.2) with newer matched sets from

A) PythonWare's py21 distribution (Tcl/Tk 8.3.3).

and again from

B) ActiveState's Tcl/Tk 8.3.4.

All the symptoms originally reported here persisted on 
Win98SE despite that.

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2002-05-22 15:57

Message:
Logged In: NO 

I am new to Python and encountered this bug on my very first 
Tkinter application.  It persists in Python 2.2.1 under 
Windows Me. Needless to say, it makes a bad first 
impression.  Basically, this thread says that Tkinter has been 
broken for a year and a half on a popular platform at a time 
when Python is trying to expand its user base.  Hopefully, this 
bug will be corrected soon.

If Tcl/Tk 8.4 is not released shortly you might try repackaging 
Tkinter for Win32 with an earlier version of Tcl/Tk, such as 
8.0, that doesn't encounter this problem.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2002-03-23 04:36

Message:
Logged In: YES 
user_id=6380

The proper action is to wait until Tcl 8.3.5 is released,
and then shaming someone else in providing you with a set of
Windowsbinaries for Tcl. You may also ask Hobbs if there's a
Windows project file to build Tcl/Tk for Windows.

----------------------------------------------------------------------

Comment By: Jeffrey Hobbs (hobbs)
Date: 2002-03-12 03:31

Message:
Logged In: YES 
user_id=72656

I did end up back-porting this to the 8.3.4+ Tcl CVS on 
2002-02-25, which means it will be in an eventual 8.3.5.  
It is already in the release 8.4 alpha series.

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2002-03-10 02:50

Message:
Logged In: YES 
user_id=31435

Back to Guido:  Do you think you know what the proper 
action is?  (You said that's why you reopened it, and 
there's been no new info here since then.)

----------------------------------------------------------------------

Comment By: John Popplewell (johnnypops)
Date: 2002-02-26 01:55

Message:
Logged In: YES 
user_id=143340

I knew I wasn't getting to the heart of it ....

Almost a one-liner!

It has been seriously spoiling my (otherwise crash free) 
Python experience on Win9x - hope it gets fixed soon.

cheers,
John.


----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2002-02-26 00:39

Message:
Logged In: YES 
user_id=6380

Reopened until we know what the proper action is.

----------------------------------------------------------------------

Comment By: Jeffrey Hobbs (hobbs)
Date: 2002-02-26 00:32

Message:
Logged In: YES 
user_id=72656

This is mostly correct, and is fixed in the 8.4 Tcl sources 
already (I guess we can backport this).  This was mentioned 
in SF Tcl bug (account for chopped URL):

https://sourceforge.net/tracker/?
func=detail&aid=217982&group_id=10894&atid=110894

and the code comment is:

    /*
     * Only finalize the notifier if a notifier was 
installed in the
     * current thread; there is a route in which this is not
     * guaranteed to be true (when tclWin32Dll.c:DllMain() 
is called
     * with the flag DLL_PROCESS_DETACH by the OS, which 
could be
     * doing so from a thread that's never previously been 
involved
     * with Tcl, e.g. the task manager) so this check is 
important.
     *
     * Fixes Bug #217982 reported by Hugh Vu and Gene 
Leache.
     */
    if (tsdPtr == NULL) {
	return;
    }


----------------------------------------------------------------------

Comment By: John Popplewell (johnnypops)
Date: 2002-02-25 23:41

Message:
Logged In: YES 
user_id=143340

This one has been torturing me for a while.

Managed to track it down to a problem inside Tcl.
For the Tcl8.3.4 source distribution the problem is in
the file win/tclWinNotify.c 

void Tcl_FinalizeNotifier(ClientData clientData)
{
    ThreadSpecificData *tsdPtr = 
                         (ThreadSpecificData *) clientData;

    /* sometimes called with tsdPtr == NULL */
    if ( tsdPtr != NULL )
    {
        DeleteCriticalSection(&tsdPtr->crit);
        CloseHandle(tsdPtr->event);

        /*
         * Clean up the timer and messaging 
         * window for this thread.
         */

        if (tsdPtr->hwnd) {
	    KillTimer(tsdPtr->hwnd, INTERVAL_TIMER);
            DestroyWindow(tsdPtr->hwnd);
        }
    }

    /*
     * If this is the last thread to use the notifier, 
     * unregister the notifier window class.
     */

    Tcl_MutexLock(&notifierMutex);
    if ( notifierCount && !--notifierCount ) {
	UnregisterClassA( "TclNotifier", 
                          TclWinGetTclInstance());
    }
    Tcl_MutexUnlock(&notifierMutex);
}

This bodge doesn't address the underlying problem but has 
stopped me from tearing all my hair out,

cheers,
John Popplewell.


----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2001-10-25 00:27

Message:
Logged In: YES 
user_id=31435

FYI, you don't need an IDE to do this -- in Win9x, hit 
Ctrl+Alt+Del and kill the process directly.

A saner <wink> solution is to develop under Win2K, which 
doesn't appear to suffer this problem (the only reports 
I've seen, and experienced myself, came from Win9x boxes).

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2001-10-24 10:52

Message:
Logged In: NO 

For those who are still trapped in this bug's hell, I will 
gladly share the one thing that saved my sanity:  
WingIDE's 'Kill' command will shut down the program with 
apparent 100% certainty and no fear of lockups.  WingIDE 
has its own issues, so its not a perfect solution, but if 
you are like me and Joe (above) who test in small 
iterations, then using 'Kill' to quit out of your app while 
testing is a workaround that may preserve your sanity.

Perhaps the python gods and the Wing guys can get together 
and tell us how to replicate 'kill' into our code.  For 
now, I'll use WingIDE to edit, and pythonw.exe for my final 
client's delivery.

----------------------------------------------------------------------

Comment By: Howard Lightstone (hlightstone)
Date: 2001-09-05 19:43

Message:
Logged In: YES 
user_id=66570

I sometimes get bunches of these....

A tool I use (Taskinfo2000) reports that (after killing 
winoldap):

python.exe is blocked on a mutex named OLESCELOCKMUTEX.
The reported state is "Console Terminating".  There appears 
to be only one (os) thread running.


----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2001-04-02 22:06

Message:
Logged In: YES 
user_id=31435

No sign of progress on this presumed Tk/Tcl Windows bug in 
over 3 months, so closing it for lack of hope.

----------------------------------------------------------------------

Comment By: Doug Henderson (djhender)
Date: 2001-02-06 06:13

Message:
This was a symptom I saw while tracking down the essence of the problem reported in #131207.  Using Win98SE, I would get an error dialog (GPF?) in the Kernel, and sometimes the dos prompt would not come back.

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2000-12-13 03:00

Message:
Just reproduced w/ current CVS, but didn't hang until the 8th try.

http://dev.scriptics.com/software/tcltk/ says 8.3 is still the latest released version; don't know whether that URL still makes sense, though.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2000-12-12 21:58

Message:
Tim, can you still reproduce this with the current CVS version?

There's been one critical patch to _tkinter since the 2.0 release.

An alternative would be to try with a newer version of Tcl (isn't 8.4 out already?).


----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2000-10-15 18:47

Message:
Same as I've reported earlier; it hangs in the call to
Tcl_Finalize (which is called by the DLL finalization
code).  It's less likely to hang if I call Tcl_Finalize
from the _tkinter DLL (from user code).

Note that the problem isn't really Python-related -- I
have stand-alone samples (based on wish) that hangs in
the same way.

More later.

</F>

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2000-10-13 16:40

Message:
Back to Tim since I have no clue what to do here.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2000-10-12 19:25

Message:
The recent fix to _tkinter (Tcl_GetStringResult(interp) instead of interp->result) didn't fix this either.

As Tim has remarked in private but not yet recorded here, a workaround is to use pythonw instead of python, so I'm lowering thepriority again.

Also note that the hanging process that Tim writes about apparently prevents Win98 from shutting down properly.

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2000-10-07 09:37

Message:
More info (none good, but some worse so boosted priority):

+ Happens under release and debug builds.

+ Have not been able to provoke when starting in the debugger.

+ Ctrl+Alt+Del and killing Winoldap is not enough to clean everything up.  There's still a Python (or Python_d) process hanging around that Ctrl+Alt+Del doesn't show.

+ This process makes it impossible to delete the associated Python .dll, and in particular makes it impossible to rebuild Python successfully without a reboot.

+ These processes cannot be killed!  Wintop and Process Viewer both fail to get the job done.  PrcView (a freeware process viewer) itself locks up if I try to kill them using it.  Process Viewer freezes for several seconds before giving up.

+ Attempting to attach to the process with the MSVC debugger (in order to find out what the heck it's doing) finds the process OK, but then yields the cryptic and undocumented error msg "Cannot execute program".

+ The processes are not accumulating cycles.

+ Smells like deadlock.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=216289&group_id=5470


From noreply@sourceforge.net  Mon Feb 24 09:59:15 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Mon, 24 Feb 2003 01:59:15 -0800
Subject: [Python-bugs-list] [ python-Feature Requests-691889 ] Add MacOS.framework attribute
Message-ID: <E18nFOF-0000Gp-00@sc8-sf-web3.sourceforge.net>

Feature Requests item #691889, was opened at 2003-02-23 23:27
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=691889&group_id=5470

Category: Macintosh
Group: None
Status: Closed
Resolution: Accepted
Priority: 5
Submitted By: John Ehresman (jpe)
Assigned to: Jack Jansen (jackjansen)
Summary: Add MacOS.framework attribute

Initial Comment:
On MacOS X, python may be compiled with a named
framework and extension modules must be linked at build
time against the framework with the same name.  This
new attribute would make the name of the framework
available at runtime.

----------------------------------------------------------------------

>Comment By: Jack Jansen (jackjansen)
Date: 2003-02-24 10:59

Message:
Logged In: YES 
user_id=45365

You can change the location where the framework is installed, by specifying something like --enable-framework=$HOME/Library/Frameworks, but you cannot change the name of the framework, that will always be Python.framework. You can of course manually edit the Makefile, but that isn't supported behaviour.

Or am I missing something?

----------------------------------------------------------------------

Comment By: John Ehresman (jpe)
Date: 2003-02-24 00:54

Message:
Logged In: YES 
user_id=22785

The revised scheme is fine with me if the framework name is
always going to be Python.  If it is, you might want to
remove the option to set it at configure time.  If there's a
reason for someone to compile with a different framework
name, I think the name should be available at runtime.

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-02-24 00:35

Message:
Logged In: YES 
user_id=45365

Implemented in macosmodule.c rev. 1.65 in a slightly different way. There's now an attribute MacOS.linkmodel which can have one of four values:
'static', 'framework', 'shared' (not used currently, but is there for compleness if I ever get around to fixing non-framework shared-library builds) and 'cfm' (for MacPython-OS9).

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=691889&group_id=5470


From noreply@sourceforge.net  Mon Feb 24 12:59:01 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Mon, 24 Feb 2003 04:59:01 -0800
Subject: [Python-bugs-list] [ python-Bugs-216289 ] Programs using Tkinter sometimes can't shut down (Windows)
Message-ID: <E18nICD-0008KS-00@sc8-sf-web3.sourceforge.net>

Bugs item #216289, was opened at 2000-10-06 22:25
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=216289&group_id=5470

Category: Windows
Group: Python 2.3
Status: Open
Resolution: Later
Priority: 3
Submitted By: Tim Peters (tim_one)
>Assigned to: Tim Peters (tim_one)
Summary: Programs using Tkinter sometimes can't shut down (Windows)

Initial Comment:
The following msg from the Tutor list is about 1.6, but I noticed the same thing several times today using 2.0b2+CVS.  In my case, I was running IDLE via

python ../tool/idle/idle.pyw

from a DOS box in my PCbuild directory.  Win98SE.  *Most* of the time, shutting down IDLE via Ctrl+Q left the DOS box hanging.  As with the poster, the only way to regain control was to use the Task Manager to kill off Winoldap.

-----Original Message-----
From: Joseph Stubenrauch <nothingisgoingtochangemyworld@yahoo.com>
Sent: Friday, October 06, 2000 9:23 PM
To: tutor@python.org
Subject: Re: [Tutor] Python 1.6 BUG


Strange, I have been experiencing the same bug myself.
Here's the low down for me:

Python 1.6 with win95
I am running a little Tkinter program
The command line I use is simply: "python foo.py"
About 25-35% of the time, when I close the Tkinter
window, DOS seems to "freeze" and never returns to the
c:\ command prompt.  I have to ctrl-alt-delete
repeatedly and shut down "winoldapp" to get rid of the
window and then shell back into DOS and keep working.
It's a bit of a pain, since I have the habit of
testing EVERYTHING in tiny little stages, so I change
one little thing, test it ... freeze ... ARGH!  Change
one more tiny thing, test it ... freeze ... ARGH! 
However, sometimes it seems to behave and doesn't
bother me for an entire several hour session of python
work.

That's my report on the problem.

Cheers,

Joe


----------------------------------------------------------------------

>Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-24 07:59

Message:
Logged In: YES 
user_id=6380

Um, I think it's *not* closed, but I'll assign it back to
Tim. I believe Tim said about this somewhere "the Tcl folks
have often thought they'd nailed it, but they still
haven't". Unless that was an entirely different problem. But
it used to happen for me on Win98 too (haven't tried in a
while).

----------------------------------------------------------------------

Comment By: Fredrik Lundh (effbot)
Date: 2003-02-24 02:13

Message:
Logged In: YES 
user_id=38376

(a bugfix a day keeps etc)

Tim, can we close this one?  I don't have access to any-
thing even remotely similar to a Win95-box any longer...

----------------------------------------------------------------------

Comment By: Jeffrey Hobbs (hobbs)
Date: 2002-12-09 23:16

Message:
Logged In: YES 
user_id=72656

Interesting last couple of comments ...

Anyway, I think the cause of this problem may have finally 
been identified by Tcl bug 651139.  I have attached a patch 
here for it against 8.3.5.  Both the 8.3.5 branch and 8.4.1+ 
head have this fix.  Amazingly it didn't effect Tcl, but could 
likely have caught Tkinter (only on Windows with non-
threaded builds).

Oh look, not allowed to attach a patch, so I'll attach it to Tcl 
bug 651139.

----------------------------------------------------------------------

Comment By: christian soldan (elacademico)
Date: 2002-11-30 21:07

Message:
Logged In: YES 
user_id=659858

how do i hack my fuckers friends

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2002-09-30 19:23

Message:
Logged In: NO 

i would like to know some moves

----------------------------------------------------------------------

Comment By: John Popplewell (johnnypops)
Date: 2002-08-21 20:08

Message:
Logged In: YES 
user_id=143340

Hi,

tried wrapping the WaitForSingleObject() with if (consolePtr-
>toWrite) { ... as suggested. Solves the problem for me in 
debug and release builds (without thread support).

What is "the tcl83.dll from www.tcl.tk", what changes does 
it include?

cheers,
John.


----------------------------------------------------------------------

Comment By: Doug Henderson (djhender)
Date: 2002-08-16 19:03

Message:
Logged In: YES 
user_id=148444

I downloaded and tested the tcl83.dll from www.tcl.tk with 
Python 2.2.1 on Win98SE using my BugDemo.py script 
from a related bug report [231207]. Several test runs 
showed that a problem still exists. The app hangs until 
Winopldapp is killed, however this leaves tcl83.dll in use. 
The problem does not occur on every execution. Some 
types of activity seem to provoke the hang, for instance, 
minimizing and maximizing the window seems to increase 
the change that it will hang on exit.

----------------------------------------------------------------------

Comment By: David Gravereaux (davygrvy)
Date: 2002-08-16 17:53

Message:
Logged In: YES 
user_id=7549

Whoop, error in the patch..  consoleMutex might not need to 
be locked.  It could cause a new dead-lock.

----------------------------------------------------------------------

Comment By: David Gravereaux (davygrvy)
Date: 2002-08-16 17:41

Message:
Logged In: YES 
user_id=7549

>It is hanging up in ConsoleCloseProc() in 'tclWinConsole.c'
>at about line 527:
>WaitForSingleObject(consolePtr->writeThread, INFINITE);

Jeff, this might need the same attention the pipe reader 
thread got for that [exec] memory leak bug.  I think 
ConsoleWriterThread() should use WaitForMultipleObjects 
instead of WaitForSingleObject with a second event handle 
used to single a fall-out like the pipe reader thread does.  It 
appears that ConsoleCloseProc() has no way to signal the 
writer thread to close at all.

This doesn't address the blocking WriteFile() in 
ConsoleWriterThread(), though.  Is it blocking there, too?
Sorry for the large post.  I'm given the option to attach a file.

Index: tclWinConsole.c
=============================================
======================
RCS file: /cvsroot/tcl/tcl/win/tclWinConsole.c,v
retrieving revision 1.7
diff -c -r1.7 tclWinConsole.c
*** tclWinConsole.c	15 Jan 2002 17:55:30 -0000	1.7
--- tclWinConsole.c	16 Aug 2002 21:31:56 -0000
***************
*** 79,84 ****
--- 79,87 ----
      HANDLE startWriter;		/* Auto-reset 
event used by the main thread to
  				 * signal when 
the writer thread should attempt
  				 * to write to the 
console. */
+     HANDLE stopWriter;		/* Auto-reset 
event used by the main thread to
+ 				 * signal when 
the writer thread should
+ 				 * terminate. */
      HANDLE startReader;		/* Auto-reset 
event used by the main thread to
  				 * signal when 
the reader thread should attempt
  				 * to read from 
the console. */
***************
*** 516,522 ****
  	 */
  
  	Tcl_MutexLock(&consoleMutex);
! 	TerminateThread(consolePtr->writeThread, 0);
  
  	/*
  	 * Wait for the thread to terminate.  This ensures 
that we are
--- 519,525 ----
  	 */
  
  	Tcl_MutexLock(&consoleMutex);
! 	SetEvent(consolePtr->stopWriter);
  
  	/*
  	 * Wait for the thread to terminate.  This ensures 
that we are
***************
*** 1134,1149 ****
  {
  
      ConsoleInfo *infoPtr = (ConsoleInfo *)arg;
      HANDLE *handle = infoPtr->handle;
      DWORD count, toWrite;
      char *buf;
  
      for (;;) {
  	/*
  	 * Wait for the main thread to signal before 
attempting to write.
  	 */
  
! 	WaitForSingleObject(infoPtr->startWriter, 
INFINITE);
  
  	buf = infoPtr->writeBuf;
  	toWrite = infoPtr->toWrite;
--- 1137,1163 ----
  {
  
      ConsoleInfo *infoPtr = (ConsoleInfo *)arg;
+     HANDLE events[2];
      HANDLE *handle = infoPtr->handle;
      DWORD count, toWrite;
      char *buf;
  
+     events[0] = infoPtr->stopWriter;
+     events[1] = infoPtr->startWriter;
+ 
      for (;;) {
+ 	DWORD dwWait;
+ 
  	/*
  	 * Wait for the main thread to signal before 
attempting to write.
  	 */
  
! 	dwWait = WaitForMultipleObjects(events, 2, 
FALSE, INFINITE);
! 
! 	if (dwWait != (WAIT_OBJECT_0 + 1)) {
! 	    /* either the stop event or some other error, so 
exit */
! 	    return 0;
! 	}
  
  	buf = infoPtr->writeBuf;
  	toWrite = infoPtr->toWrite;
***************
*** 1251,1256 ****
--- 1265,1271 ----
      if (permissions & TCL_WRITABLE) {
  	infoPtr->writable = CreateEvent(NULL, TRUE, 
TRUE, NULL);
  	infoPtr->startWriter = CreateEvent(NULL, FALSE, 
FALSE, NULL);
+ 	infoPtr->stopWriter = CreateEvent(NULL, FALSE, 
FALSE, NULL);
  	infoPtr->writeThread = CreateThread(NULL, 8000, 
ConsoleWriterThread,
  	        infoPtr, 0, &id);
      }


----------------------------------------------------------------------

Comment By: Jeffrey Hobbs (hobbs)
Date: 2002-08-16 14:56

Message:
Logged In: YES 
user_id=72656

I'm finding it a bit hard to reproduce, and now not at all.  
Could people please try replacing their current tcl83.dll with 
the one at http://www.tcl.tk/tcl83.dll.  The sig on it is:
634880 Aug 16 10:53 tcl83.dll
(md5) b35628568ddc4afed4f675d2d9a50faf  tcl83.dll

I can't hang anymore with python 2.2.1 at the Win98 
command prompt with:
   python \Python22\Tools\idle\idle.pyw

----------------------------------------------------------------------

Comment By: Jeffrey Hobbs (hobbs)
Date: 2002-08-16 13:50

Message:
Logged In: YES 
user_id=72656

johnny - That actually helps a lot and fairly clearly indicates 
the source of this hanging problem.  To give more context to 
that area, this is in the final shutdown code where it is 
(obviously) shutting down the console I/O.  The reader is 
shutdown without a check for more input (why, we are 
exiting?), so there is no issue there.

The writer however wants to make sure that the console 
output channel is drained before exit.  A blocking channel 
that is not being flushed will cause a hang.

So the question is, what is the stdio setup for Tcl in 
Py/Tkinter?  That should be examined to see whether stdio is 
clear to flush on exit.

Perhaps you can also help with more debugging.  Could you 
wrap the problematic WaitForSingleObject call with
if (consolePtr->toWrite) { ...
and perhaps also print out what to write.  It may be that 
blocking output isn't the problem at all, but rather that we are 
never receiving the signal on consolePtr->writable (usually 
done in the writer thread).  The consolePtr->toWrite check 
will at least ensure flushed channels do not enter this 
potential hanging state.

----------------------------------------------------------------------

Comment By: John Popplewell (johnnypops)
Date: 2002-08-16 10:16

Message:
Logged In: YES 
user_id=143340

I have dug out the setup I used previously and, apart from 
using Python 2.2 instead of 2.0.1, have had a go at 
reproducing my previous results.

With a debug build of Tcl/Tk and threads disabled I have 
managed to get this stack trace when it locks on exit (after 
doing a break):

ConsoleCloseProc(int * 0x006a0ae4, Tcl_Interp * 
0x00000000) line 527 + 17 bytes
TclFinalizeIOSubsystem() line 241 + 20 bytes
Tcl_FinalizeThread() line 911
Tcl_Finalize() line 812
DllMain(HINSTANCE__ * 0x00970000, unsigned long 
0x00000000, void * 0x00000001) line 197
TCL83! _DllMainCRTStartup@12 + 80 bytes
KERNEL32! bff7ddd6()
KERNEL32! bff8d123()
8176b5fc()

It is hanging up in ConsoleCloseProc() in 'tclWinConsole.c'
at about line 527:
WaitForSingleObject(consolePtr->writeThread, INFINITE);

I tried changing the timeout from INFINITE to 100ms and 
testing the return value for WAIT_TIMEOUT. Sure enough it 
occasionaly printed a message (and didn't lock-up).

Don't know if this helps. Will try to reproduce the double 
free problem in the threaded build later,

cheers,
John.


----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2002-08-15 10:19

Message:
Logged In: YES 
user_id=6380

Just a note so I won't forget this: Jeff Hobbs once said the
code to look at is

tk/generic/tkConsole.c:Tk_InitConsoleChannels

----------------------------------------------------------------------

Comment By: David Gravereaux (davygrvy)
Date: 2002-08-15 09:23

Message:
Logged In: YES 
user_id=7549

Hi John,

I have seen Tcl extension DLLs that set exit handlers get 
called by Tcl after they were torn-down.  NT and Win9x seem 
to be a bit different in the order of what DLLs get torn-down.  
here's a note from some old code of mine:

#ifdef __WIN32__
    // It can happen that the HEAP could have already been 
unloaded
    // from an awkward teardown caused by a Ctrl+C or other.  
Win32
    // seems to do a teardown in reverse order and by the time 
Tcl
    // knows what's going on and Tcl_Finalize calls the exit
    // handlers, this extension's data (heap?) has already been
    // unloaded by the OS.  Do a quick legal check on the 
pointer
    // first.
    //
    if (IsBadReadPtr(adapt, sizeof(T *))) return;
#endif

what's the back/stacktrace?  who is calling 
Tcl_FinalizeNotifier?  And had anyone called it before during 
the shutdown phase? I can't say with much confidence that 
this is happening here, too.  Maybe..  Could Tk be telling Tcl 
shutdown?  That's sort of backwards, if so.

I run w2k here, so I can't help in debugging from a test case.  
tclsh with a threaded build doesn't even call Tcl_Finalize to 
avoid all this!

----------------------------------------------------------------------

Comment By: Donal K. Fellows (dkf)
Date: 2002-08-15 09:09

Message:
Logged In: YES 
user_id=79902

Firstly, on the matter of threads.  Tcl/Tk on Windows uses
threads internally even in non-threaded builds.  That's
Windows for you (in particular, you can get calls parachuted
in from completely separate threads with relatively little
warning.  Also, you need threaded helpers to handle
non-blocking waiting on some kinds of objects.)

It's a bit hard to work out if the problem's been fixed
without a stack-trace (preferably with debugging symbols
enabled so that we can see what the function names and
arguments to those functions are.)  I have no  chance of
replicating this bug here (I'm currently running IRIX, but
also have some access to Solaris and Linux...)

----------------------------------------------------------------------

Comment By: John Popplewell (johnnypops)
Date: 2002-08-15 08:15

Message:
Logged In: YES 
user_id=143340

Hi to davygrvy,

my fix isn't for a thread related (deadlock) type problem - it 
is a fix for a double free type problem which is presumably 
screwing up the heap and causing the C runtime to hang.

It *is* with a threaded build (THREADDEFINES = -
DTCL_THREADS=1) which I think I enabled because I was 
using  'after' and it didn't work unless threading was 
enabled - but I may be mis-remembering here!

Also, I ended up doing all this with the debug build and 
found the -GZ flag useful - in fact it pointed out the fact that 
there was a problem with the heap.

It is worth noting that I have only seen this on Win98 and 
that using 'pythonw.exe' did *not* prevent this from 
happening,

regards John.


----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2002-08-14 21:00

Message:
Logged In: YES 
user_id=6380

AFAIK we never distributed a threaded build.

----------------------------------------------------------------------

Comment By: David Gravereaux (davygrvy)
Date: 2002-08-14 20:01

Message:
Logged In: YES 
user_id=7549

To johnnypops,

Where in Tcl_FinalizeNotifier() is the dead-lock occuring and 
is this with a threaded build of Tcl?


----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2002-07-02 15:39

Message:
Logged In: YES 
user_id=31435

Jeff, a long time ago Fredrik Lundh said he was able to 
reproduce this with a short wish script (no Python, no Tkinter, 
just Tcl/Tk + wish).  I'm reassigning this to /F in the hopes 
he'll remember how.

----------------------------------------------------------------------

Comment By: Jeffrey Hobbs (hobbs)
Date: 2002-07-02 14:55

Message:
Logged In: YES 
user_id=72656

I appreciate that many others see this is a problem in 
embedding Tcl, but without any clear way to reproduce this in 
Tcl/Tk itself, it's very hard for me to fix it.  I don't think that 
there is anything done in 8.4 to fix this that I recall, but if 
someone finds that the problem "goes away" with 8.4, please 
report that.

For those of course that experience this with python, the 
simple and "correct" work-around is to launch with 
pythonw.exe instead of python.exe.

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2002-06-26 13:40

Message:
Logged In: NO 

According to http://tcl.activestate.com ActiveTcl 8.4.0 is now 
available for download.  It is new as of June 24.  I don't know 
whether this will solve our problems or not, but it's worth 
looking into.

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2002-06-19 09:36

Message:
Logged In: NO 

I just downloaded Ruby 1.6.6 for Windows, which also uses 
Tcl/Tk 8.3 and I experience exactly the same problem there. 
So this is definitely a Tcl problem (but we know that already).

----------------------------------------------------------------------

Comment By: John Popplewell (johnnypops)
Date: 2002-06-07 15:01

Message:
Logged In: YES 
user_id=143340

Hmmm. 
I run a Python Tcl/Tk application several times every day - 
the very application that prompted my bug-hunt.

I do use "pythonw.exe" to launch it, but it used to hang 
almost as badly as "python.exe".

Whilst developing this app I used "python.exe" and 
experienced no hang-ups.

The Tcl guys point out that this bug is only one of several 
possible causes of the hang-up, maybe you are just out of 
luck.

The patch has fixed the problem on three Win98 machines, 
all of which exhibited the problem before updating the DLLs.

One thought - are you sure there weren't multiple copies of 
the tcl83.dll and tk83.dll files? My (only) copies were 
in "D:\python20\DLLs"

good luck,
John.


----------------------------------------------------------------------

Comment By: Jeffrey Hobbs (hobbs)
Date: 2002-06-03 14:55

Message:
Logged In: YES 
user_id=72656

providing Tim with a fixed DLL exactly according to John 
Popplewell's patch did not fix the problem for him.  That 
makes it seem fishy that John really didn't see the problem 
any more.

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2002-06-03 12:59

Message:
Logged In: YES 
user_id=31435

I tried replacing Python 2.2.1's tk83.dll and tcl83.dll (from 
Tcl/Tk 8.3.2) with newer matched sets from

A) PythonWare's py21 distribution (Tcl/Tk 8.3.3).

and again from

B) ActiveState's Tcl/Tk 8.3.4.

All the symptoms originally reported here persisted on 
Win98SE despite that.

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2002-05-22 09:57

Message:
Logged In: NO 

I am new to Python and encountered this bug on my very first 
Tkinter application.  It persists in Python 2.2.1 under 
Windows Me. Needless to say, it makes a bad first 
impression.  Basically, this thread says that Tkinter has been 
broken for a year and a half on a popular platform at a time 
when Python is trying to expand its user base.  Hopefully, this 
bug will be corrected soon.

If Tcl/Tk 8.4 is not released shortly you might try repackaging 
Tkinter for Win32 with an earlier version of Tcl/Tk, such as 
8.0, that doesn't encounter this problem.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2002-03-22 22:36

Message:
Logged In: YES 
user_id=6380

The proper action is to wait until Tcl 8.3.5 is released,
and then shaming someone else in providing you with a set of
Windowsbinaries for Tcl. You may also ask Hobbs if there's a
Windows project file to build Tcl/Tk for Windows.

----------------------------------------------------------------------

Comment By: Jeffrey Hobbs (hobbs)
Date: 2002-03-11 21:31

Message:
Logged In: YES 
user_id=72656

I did end up back-porting this to the 8.3.4+ Tcl CVS on 
2002-02-25, which means it will be in an eventual 8.3.5.  
It is already in the release 8.4 alpha series.

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2002-03-09 20:50

Message:
Logged In: YES 
user_id=31435

Back to Guido:  Do you think you know what the proper 
action is?  (You said that's why you reopened it, and 
there's been no new info here since then.)

----------------------------------------------------------------------

Comment By: John Popplewell (johnnypops)
Date: 2002-02-25 19:55

Message:
Logged In: YES 
user_id=143340

I knew I wasn't getting to the heart of it ....

Almost a one-liner!

It has been seriously spoiling my (otherwise crash free) 
Python experience on Win9x - hope it gets fixed soon.

cheers,
John.


----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2002-02-25 18:39

Message:
Logged In: YES 
user_id=6380

Reopened until we know what the proper action is.

----------------------------------------------------------------------

Comment By: Jeffrey Hobbs (hobbs)
Date: 2002-02-25 18:32

Message:
Logged In: YES 
user_id=72656

This is mostly correct, and is fixed in the 8.4 Tcl sources 
already (I guess we can backport this).  This was mentioned 
in SF Tcl bug (account for chopped URL):

https://sourceforge.net/tracker/?
func=detail&aid=217982&group_id=10894&atid=110894

and the code comment is:

    /*
     * Only finalize the notifier if a notifier was 
installed in the
     * current thread; there is a route in which this is not
     * guaranteed to be true (when tclWin32Dll.c:DllMain() 
is called
     * with the flag DLL_PROCESS_DETACH by the OS, which 
could be
     * doing so from a thread that's never previously been 
involved
     * with Tcl, e.g. the task manager) so this check is 
important.
     *
     * Fixes Bug #217982 reported by Hugh Vu and Gene 
Leache.
     */
    if (tsdPtr == NULL) {
	return;
    }


----------------------------------------------------------------------

Comment By: John Popplewell (johnnypops)
Date: 2002-02-25 17:41

Message:
Logged In: YES 
user_id=143340

This one has been torturing me for a while.

Managed to track it down to a problem inside Tcl.
For the Tcl8.3.4 source distribution the problem is in
the file win/tclWinNotify.c 

void Tcl_FinalizeNotifier(ClientData clientData)
{
    ThreadSpecificData *tsdPtr = 
                         (ThreadSpecificData *) clientData;

    /* sometimes called with tsdPtr == NULL */
    if ( tsdPtr != NULL )
    {
        DeleteCriticalSection(&tsdPtr->crit);
        CloseHandle(tsdPtr->event);

        /*
         * Clean up the timer and messaging 
         * window for this thread.
         */

        if (tsdPtr->hwnd) {
	    KillTimer(tsdPtr->hwnd, INTERVAL_TIMER);
            DestroyWindow(tsdPtr->hwnd);
        }
    }

    /*
     * If this is the last thread to use the notifier, 
     * unregister the notifier window class.
     */

    Tcl_MutexLock(&notifierMutex);
    if ( notifierCount && !--notifierCount ) {
	UnregisterClassA( "TclNotifier", 
                          TclWinGetTclInstance());
    }
    Tcl_MutexUnlock(&notifierMutex);
}

This bodge doesn't address the underlying problem but has 
stopped me from tearing all my hair out,

cheers,
John Popplewell.


----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2001-10-24 18:27

Message:
Logged In: YES 
user_id=31435

FYI, you don't need an IDE to do this -- in Win9x, hit 
Ctrl+Alt+Del and kill the process directly.

A saner <wink> solution is to develop under Win2K, which 
doesn't appear to suffer this problem (the only reports 
I've seen, and experienced myself, came from Win9x boxes).

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2001-10-24 04:52

Message:
Logged In: NO 

For those who are still trapped in this bug's hell, I will 
gladly share the one thing that saved my sanity:  
WingIDE's 'Kill' command will shut down the program with 
apparent 100% certainty and no fear of lockups.  WingIDE 
has its own issues, so its not a perfect solution, but if 
you are like me and Joe (above) who test in small 
iterations, then using 'Kill' to quit out of your app while 
testing is a workaround that may preserve your sanity.

Perhaps the python gods and the Wing guys can get together 
and tell us how to replicate 'kill' into our code.  For 
now, I'll use WingIDE to edit, and pythonw.exe for my final 
client's delivery.

----------------------------------------------------------------------

Comment By: Howard Lightstone (hlightstone)
Date: 2001-09-05 13:43

Message:
Logged In: YES 
user_id=66570

I sometimes get bunches of these....

A tool I use (Taskinfo2000) reports that (after killing 
winoldap):

python.exe is blocked on a mutex named OLESCELOCKMUTEX.
The reported state is "Console Terminating".  There appears 
to be only one (os) thread running.


----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2001-04-02 16:06

Message:
Logged In: YES 
user_id=31435

No sign of progress on this presumed Tk/Tcl Windows bug in 
over 3 months, so closing it for lack of hope.

----------------------------------------------------------------------

Comment By: Doug Henderson (djhender)
Date: 2001-02-06 00:13

Message:
This was a symptom I saw while tracking down the essence of the problem reported in #131207.  Using Win98SE, I would get an error dialog (GPF?) in the Kernel, and sometimes the dos prompt would not come back.

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2000-12-12 21:00

Message:
Just reproduced w/ current CVS, but didn't hang until the 8th try.

http://dev.scriptics.com/software/tcltk/ says 8.3 is still the latest released version; don't know whether that URL still makes sense, though.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2000-12-12 15:58

Message:
Tim, can you still reproduce this with the current CVS version?

There's been one critical patch to _tkinter since the 2.0 release.

An alternative would be to try with a newer version of Tcl (isn't 8.4 out already?).


----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2000-10-15 12:47

Message:
Same as I've reported earlier; it hangs in the call to
Tcl_Finalize (which is called by the DLL finalization
code).  It's less likely to hang if I call Tcl_Finalize
from the _tkinter DLL (from user code).

Note that the problem isn't really Python-related -- I
have stand-alone samples (based on wish) that hangs in
the same way.

More later.

</F>

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2000-10-13 10:40

Message:
Back to Tim since I have no clue what to do here.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2000-10-12 13:25

Message:
The recent fix to _tkinter (Tcl_GetStringResult(interp) instead of interp->result) didn't fix this either.

As Tim has remarked in private but not yet recorded here, a workaround is to use pythonw instead of python, so I'm lowering thepriority again.

Also note that the hanging process that Tim writes about apparently prevents Win98 from shutting down properly.

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2000-10-07 03:37

Message:
More info (none good, but some worse so boosted priority):

+ Happens under release and debug builds.

+ Have not been able to provoke when starting in the debugger.

+ Ctrl+Alt+Del and killing Winoldap is not enough to clean everything up.  There's still a Python (or Python_d) process hanging around that Ctrl+Alt+Del doesn't show.

+ This process makes it impossible to delete the associated Python .dll, and in particular makes it impossible to rebuild Python successfully without a reboot.

+ These processes cannot be killed!  Wintop and Process Viewer both fail to get the job done.  PrcView (a freeware process viewer) itself locks up if I try to kill them using it.  Process Viewer freezes for several seconds before giving up.

+ Attempting to attach to the process with the MSVC debugger (in order to find out what the heck it's doing) finds the process OK, but then yields the cryptic and undocumented error msg "Cannot execute program".

+ The processes are not accumulating cycles.

+ Smells like deadlock.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=216289&group_id=5470


From noreply@sourceforge.net  Mon Feb 24 13:20:06 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Mon, 24 Feb 2003 05:20:06 -0800
Subject: [Python-bugs-list] [ python-Bugs-692222 ] On MacOS X, make test fails on some tests/spaces in path
Message-ID: <E18nIWc-0000sI-00@sc8-sf-web3.sourceforge.net>

Bugs item #692222, was opened at 2003-02-24 14:20
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=692222&group_id=5470

Category: Macintosh
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Pascal Oberndoerfer (pascal_efi)
Assigned to: Jack Jansen (jackjansen)
Summary: On MacOS X, make test fails on some tests/spaces in path

Initial Comment:
On MacOS X, make test fails on some tests (aetext and
popen) if there are spaces in the path to your build
directory (e.g.: ~/Desktop/MacPython 2.3
alpha/Python-2.3a2).

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=692222&group_id=5470


From noreply@sourceforge.net  Mon Feb 24 13:23:58 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Mon, 24 Feb 2003 05:23:58 -0800
Subject: [Python-bugs-list] [ python-Bugs-692222 ] On MacOS X, make test fails on some tests/spaces in path
Message-ID: <E18nIaM-0004eH-00@sc8-sf-web1.sourceforge.net>

Bugs item #692222, was opened at 2003-02-24 14:20
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=692222&group_id=5470

Category: Macintosh
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Pascal Oberndoerfer (pascal_efi)
Assigned to: Jack Jansen (jackjansen)
Summary: On MacOS X, make test fails on some tests/spaces in path

Initial Comment:
On MacOS X, make test fails on some tests (aetext and
popen) if there are spaces in the path to your build
directory (e.g.: ~/Desktop/MacPython 2.3
alpha/Python-2.3a2).

----------------------------------------------------------------------

>Comment By: Pascal Oberndoerfer (pascal_efi)
Date: 2003-02-24 14:23

Message:
Logged In: YES 
user_id=688343

That is 'test_atexit', of course...

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=692222&group_id=5470


From noreply@sourceforge.net  Mon Feb 24 13:33:59 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Mon, 24 Feb 2003 05:33:59 -0800
Subject: [Python-bugs-list] [ python-Bugs-216289 ] Programs using Tkinter sometimes can't shut down (Windows)
Message-ID: <E18nIk3-00053G-00@sc8-sf-web1.sourceforge.net>

Bugs item #216289, was opened at 2000-10-07 03:25
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=216289&group_id=5470

Category: Windows
Group: Python 2.3
Status: Open
Resolution: Later
Priority: 3
Submitted By: Tim Peters (tim_one)
Assigned to: Tim Peters (tim_one)
Summary: Programs using Tkinter sometimes can't shut down (Windows)

Initial Comment:
The following msg from the Tutor list is about 1.6, but I noticed the same thing several times today using 2.0b2+CVS.  In my case, I was running IDLE via

python ../tool/idle/idle.pyw

from a DOS box in my PCbuild directory.  Win98SE.  *Most* of the time, shutting down IDLE via Ctrl+Q left the DOS box hanging.  As with the poster, the only way to regain control was to use the Task Manager to kill off Winoldap.

-----Original Message-----
From: Joseph Stubenrauch <nothingisgoingtochangemyworld@yahoo.com>
Sent: Friday, October 06, 2000 9:23 PM
To: tutor@python.org
Subject: Re: [Tutor] Python 1.6 BUG


Strange, I have been experiencing the same bug myself.
Here's the low down for me:

Python 1.6 with win95
I am running a little Tkinter program
The command line I use is simply: "python foo.py"
About 25-35% of the time, when I close the Tkinter
window, DOS seems to "freeze" and never returns to the
c:\ command prompt.  I have to ctrl-alt-delete
repeatedly and shut down "winoldapp" to get rid of the
window and then shell back into DOS and keep working.
It's a bit of a pain, since I have the habit of
testing EVERYTHING in tiny little stages, so I change
one little thing, test it ... freeze ... ARGH!  Change
one more tiny thing, test it ... freeze ... ARGH! 
However, sometimes it seems to behave and doesn't
bother me for an entire several hour session of python
work.

That's my report on the problem.

Cheers,

Joe


----------------------------------------------------------------------

Comment By: Donal K. Fellows (dkf)
Date: 2003-02-24 13:33

Message:
Logged In: YES 
user_id=79902

Exactly which version (including patchlevel) of tcl.dll and
tk.dll is this bug reported against?  (Well, which is the
most recent version...)  Is it one which the Tcl Maintainers
believe to have all identified relevant bugs in the area
resolved, or does it predate that?

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-24 12:59

Message:
Logged In: YES 
user_id=6380

Um, I think it's *not* closed, but I'll assign it back to
Tim. I believe Tim said about this somewhere "the Tcl folks
have often thought they'd nailed it, but they still
haven't". Unless that was an entirely different problem. But
it used to happen for me on Win98 too (haven't tried in a
while).

----------------------------------------------------------------------

Comment By: Fredrik Lundh (effbot)
Date: 2003-02-24 07:13

Message:
Logged In: YES 
user_id=38376

(a bugfix a day keeps etc)

Tim, can we close this one?  I don't have access to any-
thing even remotely similar to a Win95-box any longer...

----------------------------------------------------------------------

Comment By: Jeffrey Hobbs (hobbs)
Date: 2002-12-10 04:16

Message:
Logged In: YES 
user_id=72656

Interesting last couple of comments ...

Anyway, I think the cause of this problem may have finally 
been identified by Tcl bug 651139.  I have attached a patch 
here for it against 8.3.5.  Both the 8.3.5 branch and 8.4.1+ 
head have this fix.  Amazingly it didn't effect Tcl, but could 
likely have caught Tkinter (only on Windows with non-
threaded builds).

Oh look, not allowed to attach a patch, so I'll attach it to Tcl 
bug 651139.

----------------------------------------------------------------------

Comment By: christian soldan (elacademico)
Date: 2002-12-01 02:07

Message:
Logged In: YES 
user_id=659858

how do i hack my fuckers friends

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2002-10-01 00:23

Message:
Logged In: NO 

i would like to know some moves

----------------------------------------------------------------------

Comment By: John Popplewell (johnnypops)
Date: 2002-08-22 01:08

Message:
Logged In: YES 
user_id=143340

Hi,

tried wrapping the WaitForSingleObject() with if (consolePtr-
>toWrite) { ... as suggested. Solves the problem for me in 
debug and release builds (without thread support).

What is "the tcl83.dll from www.tcl.tk", what changes does 
it include?

cheers,
John.


----------------------------------------------------------------------

Comment By: Doug Henderson (djhender)
Date: 2002-08-17 00:03

Message:
Logged In: YES 
user_id=148444

I downloaded and tested the tcl83.dll from www.tcl.tk with 
Python 2.2.1 on Win98SE using my BugDemo.py script 
from a related bug report [231207]. Several test runs 
showed that a problem still exists. The app hangs until 
Winopldapp is killed, however this leaves tcl83.dll in use. 
The problem does not occur on every execution. Some 
types of activity seem to provoke the hang, for instance, 
minimizing and maximizing the window seems to increase 
the change that it will hang on exit.

----------------------------------------------------------------------

Comment By: David Gravereaux (davygrvy)
Date: 2002-08-16 22:53

Message:
Logged In: YES 
user_id=7549

Whoop, error in the patch..  consoleMutex might not need to 
be locked.  It could cause a new dead-lock.

----------------------------------------------------------------------

Comment By: David Gravereaux (davygrvy)
Date: 2002-08-16 22:41

Message:
Logged In: YES 
user_id=7549

>It is hanging up in ConsoleCloseProc() in 'tclWinConsole.c'
>at about line 527:
>WaitForSingleObject(consolePtr->writeThread, INFINITE);

Jeff, this might need the same attention the pipe reader 
thread got for that [exec] memory leak bug.  I think 
ConsoleWriterThread() should use WaitForMultipleObjects 
instead of WaitForSingleObject with a second event handle 
used to single a fall-out like the pipe reader thread does.  It 
appears that ConsoleCloseProc() has no way to signal the 
writer thread to close at all.

This doesn't address the blocking WriteFile() in 
ConsoleWriterThread(), though.  Is it blocking there, too?
Sorry for the large post.  I'm given the option to attach a file.

Index: tclWinConsole.c
=============================================
======================
RCS file: /cvsroot/tcl/tcl/win/tclWinConsole.c,v
retrieving revision 1.7
diff -c -r1.7 tclWinConsole.c
*** tclWinConsole.c	15 Jan 2002 17:55:30 -0000	1.7
--- tclWinConsole.c	16 Aug 2002 21:31:56 -0000
***************
*** 79,84 ****
--- 79,87 ----
      HANDLE startWriter;		/* Auto-reset 
event used by the main thread to
  				 * signal when 
the writer thread should attempt
  				 * to write to the 
console. */
+     HANDLE stopWriter;		/* Auto-reset 
event used by the main thread to
+ 				 * signal when 
the writer thread should
+ 				 * terminate. */
      HANDLE startReader;		/* Auto-reset 
event used by the main thread to
  				 * signal when 
the reader thread should attempt
  				 * to read from 
the console. */
***************
*** 516,522 ****
  	 */
  
  	Tcl_MutexLock(&consoleMutex);
! 	TerminateThread(consolePtr->writeThread, 0);
  
  	/*
  	 * Wait for the thread to terminate.  This ensures 
that we are
--- 519,525 ----
  	 */
  
  	Tcl_MutexLock(&consoleMutex);
! 	SetEvent(consolePtr->stopWriter);
  
  	/*
  	 * Wait for the thread to terminate.  This ensures 
that we are
***************
*** 1134,1149 ****
  {
  
      ConsoleInfo *infoPtr = (ConsoleInfo *)arg;
      HANDLE *handle = infoPtr->handle;
      DWORD count, toWrite;
      char *buf;
  
      for (;;) {
  	/*
  	 * Wait for the main thread to signal before 
attempting to write.
  	 */
  
! 	WaitForSingleObject(infoPtr->startWriter, 
INFINITE);
  
  	buf = infoPtr->writeBuf;
  	toWrite = infoPtr->toWrite;
--- 1137,1163 ----
  {
  
      ConsoleInfo *infoPtr = (ConsoleInfo *)arg;
+     HANDLE events[2];
      HANDLE *handle = infoPtr->handle;
      DWORD count, toWrite;
      char *buf;
  
+     events[0] = infoPtr->stopWriter;
+     events[1] = infoPtr->startWriter;
+ 
      for (;;) {
+ 	DWORD dwWait;
+ 
  	/*
  	 * Wait for the main thread to signal before 
attempting to write.
  	 */
  
! 	dwWait = WaitForMultipleObjects(events, 2, 
FALSE, INFINITE);
! 
! 	if (dwWait != (WAIT_OBJECT_0 + 1)) {
! 	    /* either the stop event or some other error, so 
exit */
! 	    return 0;
! 	}
  
  	buf = infoPtr->writeBuf;
  	toWrite = infoPtr->toWrite;
***************
*** 1251,1256 ****
--- 1265,1271 ----
      if (permissions & TCL_WRITABLE) {
  	infoPtr->writable = CreateEvent(NULL, TRUE, 
TRUE, NULL);
  	infoPtr->startWriter = CreateEvent(NULL, FALSE, 
FALSE, NULL);
+ 	infoPtr->stopWriter = CreateEvent(NULL, FALSE, 
FALSE, NULL);
  	infoPtr->writeThread = CreateThread(NULL, 8000, 
ConsoleWriterThread,
  	        infoPtr, 0, &id);
      }


----------------------------------------------------------------------

Comment By: Jeffrey Hobbs (hobbs)
Date: 2002-08-16 19:56

Message:
Logged In: YES 
user_id=72656

I'm finding it a bit hard to reproduce, and now not at all.  
Could people please try replacing their current tcl83.dll with 
the one at http://www.tcl.tk/tcl83.dll.  The sig on it is:
634880 Aug 16 10:53 tcl83.dll
(md5) b35628568ddc4afed4f675d2d9a50faf  tcl83.dll

I can't hang anymore with python 2.2.1 at the Win98 
command prompt with:
   python \Python22\Tools\idle\idle.pyw

----------------------------------------------------------------------

Comment By: Jeffrey Hobbs (hobbs)
Date: 2002-08-16 18:50

Message:
Logged In: YES 
user_id=72656

johnny - That actually helps a lot and fairly clearly indicates 
the source of this hanging problem.  To give more context to 
that area, this is in the final shutdown code where it is 
(obviously) shutting down the console I/O.  The reader is 
shutdown without a check for more input (why, we are 
exiting?), so there is no issue there.

The writer however wants to make sure that the console 
output channel is drained before exit.  A blocking channel 
that is not being flushed will cause a hang.

So the question is, what is the stdio setup for Tcl in 
Py/Tkinter?  That should be examined to see whether stdio is 
clear to flush on exit.

Perhaps you can also help with more debugging.  Could you 
wrap the problematic WaitForSingleObject call with
if (consolePtr->toWrite) { ...
and perhaps also print out what to write.  It may be that 
blocking output isn't the problem at all, but rather that we are 
never receiving the signal on consolePtr->writable (usually 
done in the writer thread).  The consolePtr->toWrite check 
will at least ensure flushed channels do not enter this 
potential hanging state.

----------------------------------------------------------------------

Comment By: John Popplewell (johnnypops)
Date: 2002-08-16 15:16

Message:
Logged In: YES 
user_id=143340

I have dug out the setup I used previously and, apart from 
using Python 2.2 instead of 2.0.1, have had a go at 
reproducing my previous results.

With a debug build of Tcl/Tk and threads disabled I have 
managed to get this stack trace when it locks on exit (after 
doing a break):

ConsoleCloseProc(int * 0x006a0ae4, Tcl_Interp * 
0x00000000) line 527 + 17 bytes
TclFinalizeIOSubsystem() line 241 + 20 bytes
Tcl_FinalizeThread() line 911
Tcl_Finalize() line 812
DllMain(HINSTANCE__ * 0x00970000, unsigned long 
0x00000000, void * 0x00000001) line 197
TCL83! _DllMainCRTStartup@12 + 80 bytes
KERNEL32! bff7ddd6()
KERNEL32! bff8d123()
8176b5fc()

It is hanging up in ConsoleCloseProc() in 'tclWinConsole.c'
at about line 527:
WaitForSingleObject(consolePtr->writeThread, INFINITE);

I tried changing the timeout from INFINITE to 100ms and 
testing the return value for WAIT_TIMEOUT. Sure enough it 
occasionaly printed a message (and didn't lock-up).

Don't know if this helps. Will try to reproduce the double 
free problem in the threaded build later,

cheers,
John.


----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2002-08-15 15:19

Message:
Logged In: YES 
user_id=6380

Just a note so I won't forget this: Jeff Hobbs once said the
code to look at is

tk/generic/tkConsole.c:Tk_InitConsoleChannels

----------------------------------------------------------------------

Comment By: David Gravereaux (davygrvy)
Date: 2002-08-15 14:23

Message:
Logged In: YES 
user_id=7549

Hi John,

I have seen Tcl extension DLLs that set exit handlers get 
called by Tcl after they were torn-down.  NT and Win9x seem 
to be a bit different in the order of what DLLs get torn-down.  
here's a note from some old code of mine:

#ifdef __WIN32__
    // It can happen that the HEAP could have already been 
unloaded
    // from an awkward teardown caused by a Ctrl+C or other.  
Win32
    // seems to do a teardown in reverse order and by the time 
Tcl
    // knows what's going on and Tcl_Finalize calls the exit
    // handlers, this extension's data (heap?) has already been
    // unloaded by the OS.  Do a quick legal check on the 
pointer
    // first.
    //
    if (IsBadReadPtr(adapt, sizeof(T *))) return;
#endif

what's the back/stacktrace?  who is calling 
Tcl_FinalizeNotifier?  And had anyone called it before during 
the shutdown phase? I can't say with much confidence that 
this is happening here, too.  Maybe..  Could Tk be telling Tcl 
shutdown?  That's sort of backwards, if so.

I run w2k here, so I can't help in debugging from a test case.  
tclsh with a threaded build doesn't even call Tcl_Finalize to 
avoid all this!

----------------------------------------------------------------------

Comment By: Donal K. Fellows (dkf)
Date: 2002-08-15 14:09

Message:
Logged In: YES 
user_id=79902

Firstly, on the matter of threads.  Tcl/Tk on Windows uses
threads internally even in non-threaded builds.  That's
Windows for you (in particular, you can get calls parachuted
in from completely separate threads with relatively little
warning.  Also, you need threaded helpers to handle
non-blocking waiting on some kinds of objects.)

It's a bit hard to work out if the problem's been fixed
without a stack-trace (preferably with debugging symbols
enabled so that we can see what the function names and
arguments to those functions are.)  I have no  chance of
replicating this bug here (I'm currently running IRIX, but
also have some access to Solaris and Linux...)

----------------------------------------------------------------------

Comment By: John Popplewell (johnnypops)
Date: 2002-08-15 13:15

Message:
Logged In: YES 
user_id=143340

Hi to davygrvy,

my fix isn't for a thread related (deadlock) type problem - it 
is a fix for a double free type problem which is presumably 
screwing up the heap and causing the C runtime to hang.

It *is* with a threaded build (THREADDEFINES = -
DTCL_THREADS=1) which I think I enabled because I was 
using  'after' and it didn't work unless threading was 
enabled - but I may be mis-remembering here!

Also, I ended up doing all this with the debug build and 
found the -GZ flag useful - in fact it pointed out the fact that 
there was a problem with the heap.

It is worth noting that I have only seen this on Win98 and 
that using 'pythonw.exe' did *not* prevent this from 
happening,

regards John.


----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2002-08-15 02:00

Message:
Logged In: YES 
user_id=6380

AFAIK we never distributed a threaded build.

----------------------------------------------------------------------

Comment By: David Gravereaux (davygrvy)
Date: 2002-08-15 01:01

Message:
Logged In: YES 
user_id=7549

To johnnypops,

Where in Tcl_FinalizeNotifier() is the dead-lock occuring and 
is this with a threaded build of Tcl?


----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2002-07-02 20:39

Message:
Logged In: YES 
user_id=31435

Jeff, a long time ago Fredrik Lundh said he was able to 
reproduce this with a short wish script (no Python, no Tkinter, 
just Tcl/Tk + wish).  I'm reassigning this to /F in the hopes 
he'll remember how.

----------------------------------------------------------------------

Comment By: Jeffrey Hobbs (hobbs)
Date: 2002-07-02 19:55

Message:
Logged In: YES 
user_id=72656

I appreciate that many others see this is a problem in 
embedding Tcl, but without any clear way to reproduce this in 
Tcl/Tk itself, it's very hard for me to fix it.  I don't think that 
there is anything done in 8.4 to fix this that I recall, but if 
someone finds that the problem "goes away" with 8.4, please 
report that.

For those of course that experience this with python, the 
simple and "correct" work-around is to launch with 
pythonw.exe instead of python.exe.

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2002-06-26 18:40

Message:
Logged In: NO 

According to http://tcl.activestate.com ActiveTcl 8.4.0 is now 
available for download.  It is new as of June 24.  I don't know 
whether this will solve our problems or not, but it's worth 
looking into.

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2002-06-19 14:36

Message:
Logged In: NO 

I just downloaded Ruby 1.6.6 for Windows, which also uses 
Tcl/Tk 8.3 and I experience exactly the same problem there. 
So this is definitely a Tcl problem (but we know that already).

----------------------------------------------------------------------

Comment By: John Popplewell (johnnypops)
Date: 2002-06-07 20:01

Message:
Logged In: YES 
user_id=143340

Hmmm. 
I run a Python Tcl/Tk application several times every day - 
the very application that prompted my bug-hunt.

I do use "pythonw.exe" to launch it, but it used to hang 
almost as badly as "python.exe".

Whilst developing this app I used "python.exe" and 
experienced no hang-ups.

The Tcl guys point out that this bug is only one of several 
possible causes of the hang-up, maybe you are just out of 
luck.

The patch has fixed the problem on three Win98 machines, 
all of which exhibited the problem before updating the DLLs.

One thought - are you sure there weren't multiple copies of 
the tcl83.dll and tk83.dll files? My (only) copies were 
in "D:\python20\DLLs"

good luck,
John.


----------------------------------------------------------------------

Comment By: Jeffrey Hobbs (hobbs)
Date: 2002-06-03 19:55

Message:
Logged In: YES 
user_id=72656

providing Tim with a fixed DLL exactly according to John 
Popplewell's patch did not fix the problem for him.  That 
makes it seem fishy that John really didn't see the problem 
any more.

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2002-06-03 17:59

Message:
Logged In: YES 
user_id=31435

I tried replacing Python 2.2.1's tk83.dll and tcl83.dll (from 
Tcl/Tk 8.3.2) with newer matched sets from

A) PythonWare's py21 distribution (Tcl/Tk 8.3.3).

and again from

B) ActiveState's Tcl/Tk 8.3.4.

All the symptoms originally reported here persisted on 
Win98SE despite that.

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2002-05-22 14:57

Message:
Logged In: NO 

I am new to Python and encountered this bug on my very first 
Tkinter application.  It persists in Python 2.2.1 under 
Windows Me. Needless to say, it makes a bad first 
impression.  Basically, this thread says that Tkinter has been 
broken for a year and a half on a popular platform at a time 
when Python is trying to expand its user base.  Hopefully, this 
bug will be corrected soon.

If Tcl/Tk 8.4 is not released shortly you might try repackaging 
Tkinter for Win32 with an earlier version of Tcl/Tk, such as 
8.0, that doesn't encounter this problem.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2002-03-23 03:36

Message:
Logged In: YES 
user_id=6380

The proper action is to wait until Tcl 8.3.5 is released,
and then shaming someone else in providing you with a set of
Windowsbinaries for Tcl. You may also ask Hobbs if there's a
Windows project file to build Tcl/Tk for Windows.

----------------------------------------------------------------------

Comment By: Jeffrey Hobbs (hobbs)
Date: 2002-03-12 02:31

Message:
Logged In: YES 
user_id=72656

I did end up back-porting this to the 8.3.4+ Tcl CVS on 
2002-02-25, which means it will be in an eventual 8.3.5.  
It is already in the release 8.4 alpha series.

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2002-03-10 01:50

Message:
Logged In: YES 
user_id=31435

Back to Guido:  Do you think you know what the proper 
action is?  (You said that's why you reopened it, and 
there's been no new info here since then.)

----------------------------------------------------------------------

Comment By: John Popplewell (johnnypops)
Date: 2002-02-26 00:55

Message:
Logged In: YES 
user_id=143340

I knew I wasn't getting to the heart of it ....

Almost a one-liner!

It has been seriously spoiling my (otherwise crash free) 
Python experience on Win9x - hope it gets fixed soon.

cheers,
John.


----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2002-02-25 23:39

Message:
Logged In: YES 
user_id=6380

Reopened until we know what the proper action is.

----------------------------------------------------------------------

Comment By: Jeffrey Hobbs (hobbs)
Date: 2002-02-25 23:32

Message:
Logged In: YES 
user_id=72656

This is mostly correct, and is fixed in the 8.4 Tcl sources 
already (I guess we can backport this).  This was mentioned 
in SF Tcl bug (account for chopped URL):

https://sourceforge.net/tracker/?
func=detail&aid=217982&group_id=10894&atid=110894

and the code comment is:

    /*
     * Only finalize the notifier if a notifier was 
installed in the
     * current thread; there is a route in which this is not
     * guaranteed to be true (when tclWin32Dll.c:DllMain() 
is called
     * with the flag DLL_PROCESS_DETACH by the OS, which 
could be
     * doing so from a thread that's never previously been 
involved
     * with Tcl, e.g. the task manager) so this check is 
important.
     *
     * Fixes Bug #217982 reported by Hugh Vu and Gene 
Leache.
     */
    if (tsdPtr == NULL) {
	return;
    }


----------------------------------------------------------------------

Comment By: John Popplewell (johnnypops)
Date: 2002-02-25 22:41

Message:
Logged In: YES 
user_id=143340

This one has been torturing me for a while.

Managed to track it down to a problem inside Tcl.
For the Tcl8.3.4 source distribution the problem is in
the file win/tclWinNotify.c 

void Tcl_FinalizeNotifier(ClientData clientData)
{
    ThreadSpecificData *tsdPtr = 
                         (ThreadSpecificData *) clientData;

    /* sometimes called with tsdPtr == NULL */
    if ( tsdPtr != NULL )
    {
        DeleteCriticalSection(&tsdPtr->crit);
        CloseHandle(tsdPtr->event);

        /*
         * Clean up the timer and messaging 
         * window for this thread.
         */

        if (tsdPtr->hwnd) {
	    KillTimer(tsdPtr->hwnd, INTERVAL_TIMER);
            DestroyWindow(tsdPtr->hwnd);
        }
    }

    /*
     * If this is the last thread to use the notifier, 
     * unregister the notifier window class.
     */

    Tcl_MutexLock(&notifierMutex);
    if ( notifierCount && !--notifierCount ) {
	UnregisterClassA( "TclNotifier", 
                          TclWinGetTclInstance());
    }
    Tcl_MutexUnlock(&notifierMutex);
}

This bodge doesn't address the underlying problem but has 
stopped me from tearing all my hair out,

cheers,
John Popplewell.


----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2001-10-24 23:27

Message:
Logged In: YES 
user_id=31435

FYI, you don't need an IDE to do this -- in Win9x, hit 
Ctrl+Alt+Del and kill the process directly.

A saner <wink> solution is to develop under Win2K, which 
doesn't appear to suffer this problem (the only reports 
I've seen, and experienced myself, came from Win9x boxes).

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2001-10-24 09:52

Message:
Logged In: NO 

For those who are still trapped in this bug's hell, I will 
gladly share the one thing that saved my sanity:  
WingIDE's 'Kill' command will shut down the program with 
apparent 100% certainty and no fear of lockups.  WingIDE 
has its own issues, so its not a perfect solution, but if 
you are like me and Joe (above) who test in small 
iterations, then using 'Kill' to quit out of your app while 
testing is a workaround that may preserve your sanity.

Perhaps the python gods and the Wing guys can get together 
and tell us how to replicate 'kill' into our code.  For 
now, I'll use WingIDE to edit, and pythonw.exe for my final 
client's delivery.

----------------------------------------------------------------------

Comment By: Howard Lightstone (hlightstone)
Date: 2001-09-05 18:43

Message:
Logged In: YES 
user_id=66570

I sometimes get bunches of these....

A tool I use (Taskinfo2000) reports that (after killing 
winoldap):

python.exe is blocked on a mutex named OLESCELOCKMUTEX.
The reported state is "Console Terminating".  There appears 
to be only one (os) thread running.


----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2001-04-02 21:06

Message:
Logged In: YES 
user_id=31435

No sign of progress on this presumed Tk/Tcl Windows bug in 
over 3 months, so closing it for lack of hope.

----------------------------------------------------------------------

Comment By: Doug Henderson (djhender)
Date: 2001-02-06 05:13

Message:
This was a symptom I saw while tracking down the essence of the problem reported in #131207.  Using Win98SE, I would get an error dialog (GPF?) in the Kernel, and sometimes the dos prompt would not come back.

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2000-12-13 02:00

Message:
Just reproduced w/ current CVS, but didn't hang until the 8th try.

http://dev.scriptics.com/software/tcltk/ says 8.3 is still the latest released version; don't know whether that URL still makes sense, though.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2000-12-12 20:58

Message:
Tim, can you still reproduce this with the current CVS version?

There's been one critical patch to _tkinter since the 2.0 release.

An alternative would be to try with a newer version of Tcl (isn't 8.4 out already?).


----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2000-10-15 17:47

Message:
Same as I've reported earlier; it hangs in the call to
Tcl_Finalize (which is called by the DLL finalization
code).  It's less likely to hang if I call Tcl_Finalize
from the _tkinter DLL (from user code).

Note that the problem isn't really Python-related -- I
have stand-alone samples (based on wish) that hangs in
the same way.

More later.

</F>

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2000-10-13 15:40

Message:
Back to Tim since I have no clue what to do here.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2000-10-12 18:25

Message:
The recent fix to _tkinter (Tcl_GetStringResult(interp) instead of interp->result) didn't fix this either.

As Tim has remarked in private but not yet recorded here, a workaround is to use pythonw instead of python, so I'm lowering thepriority again.

Also note that the hanging process that Tim writes about apparently prevents Win98 from shutting down properly.

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2000-10-07 08:37

Message:
More info (none good, but some worse so boosted priority):

+ Happens under release and debug builds.

+ Have not been able to provoke when starting in the debugger.

+ Ctrl+Alt+Del and killing Winoldap is not enough to clean everything up.  There's still a Python (or Python_d) process hanging around that Ctrl+Alt+Del doesn't show.

+ This process makes it impossible to delete the associated Python .dll, and in particular makes it impossible to rebuild Python successfully without a reboot.

+ These processes cannot be killed!  Wintop and Process Viewer both fail to get the job done.  PrcView (a freeware process viewer) itself locks up if I try to kill them using it.  Process Viewer freezes for several seconds before giving up.

+ Attempting to attach to the process with the MSVC debugger (in order to find out what the heck it's doing) finds the process OK, but then yields the cryptic and undocumented error msg "Cannot execute program".

+ The processes are not accumulating cycles.

+ Smells like deadlock.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=216289&group_id=5470


From noreply@sourceforge.net  Mon Feb 24 14:30:04 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Mon, 24 Feb 2003 06:30:04 -0800
Subject: [Python-bugs-list] [ python-Bugs-690285 ] lines run together on input
Message-ID: <E18nJcK-0001gm-00@sc8-sf-web2.sourceforge.net>

Bugs item #690285, was opened at 2003-02-20 16:12
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690285&group_id=5470

Category: Python Library
Group: Python 2.2.2
>Status: Closed
Resolution: None
Priority: 1
Submitted By: Thomas B Hickey (thomasbhickey)
Assigned to: Nobody/Anonymous (nobody)
Summary: lines run together on input

Initial Comment:

Using  'for line in file('xxxxx'):'  on a large (>4 Gbyte) text file 
3/4 of the way into the file occasionally two lines will be 
merged into one.  Happens consistently at the same lines.  
Counting lines this way always comes up short.  Changing 
the open to 'rb' fixes the problem.  These lines are 
terminated by '\n' even though we are running on a PC.

Haven't tried the universal newline mode in 2.3a, but would 
be willing to.

Running on Windows NT 2000, 2.2.2 (#37 Oct 14, 2002).  
Same problem seen in 2.2.1.

----------------------------------------------------------------------

>Comment By: Thomas B Hickey (thomasbhickey)
Date: 2003-02-24 09:30

Message:
Logged In: YES 
user_id=274109

I agree this isn't a Python problem.  Does not seem to be a 
problem with the data.  We've seen it now in two text files of over 
4 gig, and doing a count of /n & /r gives the expected correct 
results.  Wouldn't have reported it if I'd seen the C problem 
earlier.

Thanks for the attention.  We've been processing 40gig+ files 
with Python regularly (in binary mode!)  with great success.

--Th

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-23 09:11

Message:
Logged In: YES 
user_id=6380

If as you suggest it's a bug in the C library, we can't do
anything about this. Are you sure it's not a problem with
your data?

----------------------------------------------------------------------

Comment By: Thomas B Hickey (thomasbhickey)
Date: 2003-02-22 18:25

Message:
Logged In: YES 
user_id=274109

We have repeated this same problem with C, so the bug does 
not seem to be peculiar to python.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690285&group_id=5470


From noreply@sourceforge.net  Mon Feb 24 15:10:17 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Mon, 24 Feb 2003 07:10:17 -0800
Subject: [Python-bugs-list] [ python-Bugs-690309 ] 2.3a2 Sol8 make fails at _iconv_codec.
Message-ID: <E18nKFF-0008Vc-00@sc8-sf-web1.sourceforge.net>

Bugs item #690309, was opened at 2003-02-20 22:53
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690309&group_id=5470

Category: Build
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Inyeol Lee (inyeol)
Assigned to: Nobody/Anonymous (nobody)
Summary: 2.3a2 Sol8 make fails at _iconv_codec.

Initial Comment:
Python 2.3a2 build process fails during building
_iconv_codec
module. The message is;

[...]
gcc -shared
build/temp.solaris-2.8-sun4u-2.3/_iconv_codec.o
-L/home/inyeol/util/python/lib -L/usr/local/lib -o
build/lib.solaris-2.8-sun4u-2.3/_iconv_codec.so
Traceback (most recent call last):

  [... long traceback here...]

  File "./setup.py", line 209, in build_extension
    imp.load_dynamic(ext.name, ext_filename)
RuntimeError: can't initialize the _iconv_codec module:
iconv_open() failed
*** Error code 1
make: Fatal error: Command failed for target
`sharedmods'

My build environment is gcc2.95.3 with Solaris 8
linker.
I've built 2.2, 2,2,1, 2,2,2, 2,3a1 with the same env
and there
was no (major) issues like this. I attached my
configure and
make log messages.


----------------------------------------------------------------------

>Comment By: Walter Dörwald (doerwalter)
Date: 2003-02-24 16:10

Message:
Logged In: YES 
user_id=89016

These are just warnings, I see no error here. BTW the GNU
implementation does supports both ASCII and ISO8859-1 as
encodings, so I don't see why 1.12 should break anything

----------------------------------------------------------------------

Comment By: Alejandro Lopez-Valencia (dradul)
Date: 2003-02-23 13:25

Message:
Logged In: YES 
user_id=659006

The fix applied to _iconv_codec.c (in version 1.12) *breaks* the 
extension unnder cygwin and possibly other systems that use 
GNU iconv instead of a native library (namely, Solaris and BSD 
derivatives). The relevant errors:

building '_iconv_codec' extension
gcc -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -I. -
I/cvsroot/python/dist/src/./Include -I/usr/local/include -
I/tmp/Include -I/tmp/python -c /cvsroot/python/dist
/src/Modules/_iconv_codec.c -o build/temp.cygwin-1.3.20-i686-
2.3/_iconv_codec.o
cc1: warning: changing search order for system 
directory "/usr/local/include"
cc1: warning:   as it has already been specified as a non-system 
directory
/cvsroot/python/dist/src/Modules/_iconv_codec.c: In function 
`iconvcodec_encode'
:
/cvsroot/python/dist/src/Modules/_iconv_codec.c:147: warning: 
passing arg 2 of `
libiconv' from incompatible pointer type
/cvsroot/python/dist/src/Modules/_iconv_codec.c: In function 
`iconvcodec_decode'
:
/cvsroot/python/dist/src/Modules/_iconv_codec.c:357: warning: 
passing arg 2 of `
libiconv' from incompatible pointer type
/cvsroot/python/dist/src/Modules/_iconv_codec.c: In function 
`init_iconv_codec':

/cvsroot/python/dist/src/Modules/_iconv_codec.c:682: warning: 
passing arg 2 of `
libiconv' from incompatible pointer type
gcc -shared -Wl,--enable-auto-image-base build/temp.cygwin-
1.3.20-i686-2.3/_icon
v_codec.o -L/usr/local/lib -L. -liconv -lpython2.3 -o 
build/lib.cygwin-1.3.20-i6
86-2.3/_iconv_codec.dll
running build_scripts

----------------------------------------------------------------------

Comment By: Walter Dörwald (doerwalter)
Date: 2003-02-21 19:29

Message:
Logged In: YES 
user_id=89016

I changed the init function to use 'ISO8859-1' instead of
'ASCII' as the test encoding. Can someone test whether this
fixes anything?

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-20 23:15

Message:
Logged In: YES 
user_id=33168

In order to get the build to proceed, you can change line
210 of setup.py from:

        except ImportError, why:
to:
        except (ImportError, RuntimeError), why:

This is just before:
    self.announce('*** WARNING: renaming "%s" since
importing it' ...)

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690309&group_id=5470


From noreply@sourceforge.net  Mon Feb 24 15:35:51 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Mon, 24 Feb 2003 07:35:51 -0800
Subject: [Python-bugs-list] [ python-Bugs-692222 ] On MacOS X, make test fails on some tests/spaces in path
Message-ID: <E18nKdz-0007XI-00@sc8-sf-web4.sourceforge.net>

Bugs item #692222, was opened at 2003-02-24 14:20
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=692222&group_id=5470

Category: Macintosh
Group: Python 2.3
>Status: Closed
>Resolution: Fixed
Priority: 5
Submitted By: Pascal Oberndoerfer (pascal_efi)
Assigned to: Jack Jansen (jackjansen)
Summary: On MacOS X, make test fails on some tests/spaces in path

Initial Comment:
On MacOS X, make test fails on some tests (aetext and
popen) if there are spaces in the path to your build
directory (e.g.: ~/Desktop/MacPython 2.3
alpha/Python-2.3a2).

----------------------------------------------------------------------

>Comment By: Jack Jansen (jackjansen)
Date: 2003-02-24 16:35

Message:
Logged In: YES 
user_id=45365

Fixed in test_atexit.py rev. 1.10, test_popen.py rev. 1.4.

----------------------------------------------------------------------

Comment By: Pascal Oberndoerfer (pascal_efi)
Date: 2003-02-24 14:23

Message:
Logged In: YES 
user_id=688343

That is 'test_atexit', of course...

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=692222&group_id=5470


From noreply@sourceforge.net  Mon Feb 24 15:41:45 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Mon, 24 Feb 2003 07:41:45 -0800
Subject: [Python-bugs-list] [ python-Bugs-488184 ] import with undefineds can crash python
Message-ID: <E18nKjh-0007sj-00@sc8-sf-web4.sourceforge.net>

Bugs item #488184, was opened at 2001-12-02 23:19
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=488184&group_id=5470

Category: Macintosh
Group: None
>Status: Closed
>Resolution: Fixed
Priority: 1
Submitted By: Jack Jansen (jackjansen)
Assigned to: Jack Jansen (jackjansen)
Summary: import with undefineds can crash python

Initial Comment:
Importing a module which references undefined externals, or which references libraries that
fail initialization, can crash the interpreter.


----------------------------------------------------------------------

>Comment By: Jack Jansen (jackjansen)
Date: 2003-02-24 16:41

Message:
Logged In: YES 
user_id=45365

This has been fixed with the introduction of MacOS.WMAvailable().

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2002-10-19 22:12

Message:
Logged In: YES 
user_id=45365

This seems to have been fixed on OSX 10.2. The import of, say, Carbon.Win will now work fine. But: python will still exit (not crash, a clean exit) when you actually try to access the window manager if you don't have the right so I am keeping this bug open. I will try and find out if there's a way to detect this situation. 

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2001-12-11 11:01

Message:
Logged In: YES 
user_id=45365

Apple was already aware of this bug, their bug number for it
is #2806458. There is no workaround known, so we'll have to
wait until it is fixed.

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2001-12-06 23:56

Message:
Logged In: YES 
user_id=45365

Unfortunately the fix I had in mind, adding the "return on error" flag to NSLinkModule(), is not good enough. Python crashes while the initialization routine for the dynamic library is running, and NSLinkModule also never gets control back, it seems. I now think this is definitely an Apple bug. Lowering the priority because of that, not because it's not important.

I did find an easy way to reproduce the bug, though: "su" to someone else (not root, someone who can't access your window server) and import _Win or something similar.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=488184&group_id=5470


From noreply@sourceforge.net  Mon Feb 24 16:40:08 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Mon, 24 Feb 2003 08:40:08 -0800
Subject: [Python-bugs-list] [ python-Bugs-688907 ] Applet support is broken
Message-ID: <E18nLeC-0003Jb-00@sc8-sf-web4.sourceforge.net>

Bugs item #688907, was opened at 2003-02-18 19:40
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=688907&group_id=5470

Category: Macintosh
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Jack Jansen (jackjansen)
Assigned to: Just van Rossum (jvr)
Summary: Applet support is broken

Initial Comment:
Applet support is broken in two ways:
- there is no sys.argv emulation anymore, and applets always see the -psnXXXX argument when started from the finder.
- when an applet is started from the commandline it is non-functional (no window manager connection).

Also, the building of the Idle applet doesn't work due to name conflicts.

----------------------------------------------------------------------

>Comment By: Jack Jansen (jackjansen)
Date: 2003-02-24 17:40

Message:
Logged In: YES 
user_id=45365

Building the Idle applet has been fixed (Mac/OSX/Makefile rev 1.36).

So now the python-based applet main program is the only thing that needs to be done.

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-02-19 00:48

Message:
Logged In: YES 
user_id=45365

Just thinks he can fix the commandline problem with a Python script, so assigning to him. Will you please also update Misc/NEWS if you do?

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-02-19 00:44

Message:
Logged In: YES 
user_id=45365

sys.argv emulation works again, and that's the main part of this bug. A note has been added to NEWS that starting from the finder is broken in 2.3a2.

Building of Idle I cannot test right now.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=688907&group_id=5470


From noreply@sourceforge.net  Mon Feb 24 16:41:46 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Mon, 24 Feb 2003 08:41:46 -0800
Subject: [Python-bugs-list] [ python-Bugs-688007 ] Bundlebuilder needs to pre-convert resource files
Message-ID: <E18nLfm-0003Qf-00@sc8-sf-web4.sourceforge.net>

Bugs item #688007, was opened at 2003-02-17 15:21
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=688007&group_id=5470

Category: Macintosh
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Jack Jansen (jackjansen)
>Assigned to: Jack Jansen (jackjansen)
Summary: Bundlebuilder needs to pre-convert resource files

Initial Comment:
If bundlebuilder encounters an applesingle-encoded resource file it should pre-convert this to a datafork-based resource file.

At the moment the file is copied to the Resources directory verbatim, and this can cause problems if you use "sudo make frameworkinstall" to install Python: the resource folder will be mode 755 owned by root, and any normal user trying to run the applet will get a crash when the applet tries to unpack the applesingle .rsrc file into the datafork resource file .rsrc.df.rsrc.

----------------------------------------------------------------------

>Comment By: Jack Jansen (jackjansen)
Date: 2003-02-24 17:41

Message:
Logged In: YES 
user_id=45365

I'll pick this up again. Now I know about the (src, dst) possibility of bundlebuilder it should be easy to do the conversion in buildtools.

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-02-19 00:42

Message:
Logged In: YES 
user_id=45365

With some hacking, yes. buildtools would have to create the datafork-based resource file, but it would have to keep its original name, so it should be copied to a temp directory somewhere....

Can this wait until after 2.3a2?

----------------------------------------------------------------------

Comment By: Just van Rossum (jvr)
Date: 2003-02-18 20:17

Message:
Logged In: YES 
user_id=92689

Ok, next try: can't BuildApplet and/or buildtools.py do the conversion?

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-02-18 16:50

Message:
Logged In: YES 
user_id=45365

The FSRef APIs for data fork stuff is available under OS9, but (a) this won't work for applets, (b) ResEdit and Resorcerer don't understand this and (c) I don't think CodeWarrior handles it. So until OS9 support is deprecated wer're stuck with resource forks:-(

----------------------------------------------------------------------

Comment By: Just van Rossum (jvr)
Date: 2003-02-18 16:08

Message:
Logged In: YES 
user_id=92689

Aren't datafork res files supported in OS9? If so, this might be an opportunity to get rid of dataforks (as well as macresource.py) altogether...

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-02-18 15:04

Message:
Logged In: YES 
user_id=45365

I agree on a theoretical level, but not on a practical one. The .rsrc files in the CVS repository are AppleSingle encoded, and I can't think of another way to keep them useable on both MacOS9 and MacOSX.

Any suggestions?

----------------------------------------------------------------------

Comment By: Just van Rossum (jvr)
Date: 2003-02-18 13:32

Message:
Logged In: YES 
user_id=92689

Yes, it must be compatible with 2.2 (it also ships with PyObjC). bundlebuilder.py was deliberately written to have very few dependencies: plistlib.py is the only dependency outside the std lib.

I'm not sure I like this patch to begin with: for one it seems it's doing this resource processing at the wrong abstraction level, it adds a special case for something that I don't like being treated specially within bundlebuilder. It would be much better if this preprocessing was done elsewhere.

In other words: I think the bug is in attempting to write the converted file to begin with, not with bundlebuilder not doing the conversion.

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-02-17 17:56

Message:
Logged In: YES 
user_id=45365

Fixed in bundlebuilder.py rev 1.8, macresource rev. 1.3.

Just, I'm assigning this to you because I'm not sure whether you still keep bundlebuilder 2.2-compatible. If so you may need to disable this bit of magic if macresource.install isn't available. Please close the report when you're happy.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=688007&group_id=5470


From noreply@sourceforge.net  Mon Feb 24 16:42:51 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Mon, 24 Feb 2003 08:42:51 -0800
Subject: [Python-bugs-list] [ python-Bugs-492465 ] mkcwproject: custom __initialize routine
Message-ID: <E18nLgp-0003Vi-00@sc8-sf-web4.sourceforge.net>

Bugs item #492465, was opened at 2001-12-13 15:23
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=492465&group_id=5470

Category: Macintosh
Group: Feature Request
Status: Open
Resolution: None
Priority: 1
Submitted By: Just van Rossum (jvr)
Assigned to: Just van Rossum (jvr)
Summary: mkcwproject: custom __initialize routine

Initial Comment:
Both for _Tkinter and CoreGraphics it is neccesary to
specify a custom fragment initializer. It would be nice
if this were doable with mkcwproject, as it would reduce
the number of CW project file under CVS by 4...

----------------------------------------------------------------------

>Comment By: Jack Jansen (jackjansen)
Date: 2003-02-24 17:42

Message:
Logged In: YES 
user_id=45365

Just, what do you think about closing this with a "won't fix"?

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2002-05-24 00:37

Message:
Logged In: YES 
user_id=45365

genpluginprojects has been modified to allow an optional initialize=xxx keyword argument (and its mkcwproject foundation too). The only thing I've tested is that it doesn't have any adverse effects on the existing projects that are built. Assigning it back to you so you can try to generate the CG project with it and remove the hand-built CG project file. If that all works assign it back to me and I'll do the same for Tkinter.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=492465&group_id=5470


From noreply@sourceforge.net  Mon Feb 24 16:45:07 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Mon, 24 Feb 2003 08:45:07 -0800
Subject: [Python-bugs-list] [ python-Bugs-539990 ] Framework.mainloop() - multiple threads
Message-ID: <E18nLj1-0003fF-00@sc8-sf-web4.sourceforge.net>

Bugs item #539990, was opened at 2002-04-05 23:29
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=539990&group_id=5470

Category: Macintosh
Group: Platform-specific
>Status: Closed
>Resolution: Wont Fix
Priority: 7
Submitted By: Pieter Claerhout (pclaerhout)
Assigned to: Jack Jansen (jackjansen)
Summary: Framework.mainloop() - multiple threads

Initial Comment:
I tried to use the threading module to start the XML 
RPC server in a different thread (xmlrpcServer itself 
also is a threaded piece of code), but that seems to 
conflict with the mainloop. If you start this 
application, it starts the thread, starts the mainloop 
and stays in there, once you finish the mainloop, the 
thread continues.

You've stumbled on a bug in Framework.mainloop(): it 
doesn't know anything about multiple threads. It gives 
time to other applications (by calling WaitNextEvent) 
but not to other threads within Python.

Also see
http://mail.python.org/pipermail/pythonmac-sig/2002-
April/005416.html

and 

http://mail.python.org/pipermail/pythonmac-sig/2002-
April/005428.html

----------------------------------------------------------------------

>Comment By: Jack Jansen (jackjansen)
Date: 2003-02-24 17:45

Message:
Logged In: YES 
user_id=45365

Let's admit this isn't going to be fixed unless someone wants it real bad.

Pieter: if you want it real bad then reopen the bug and I'll find time for it.

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2002-04-22 16:03

Message:
Logged In: YES 
user_id=45365

After some discussion on the SIG it turns out this is a rather complicated problem, because WNE() may cause callbacks to be called (for Carbon Event handlers, for instance), and releasing the GIL would mean those callbacks would be hosed.

A solution will have to have all callback wrappers acquire the GIL, and it will either have to make very sure we never enter WNE() or friends with the GIL held, or the callback code will need to be able to handle both the case of being called with the GIL held and the case of being called without it.

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2002-04-07 23:26

Message:
Logged In: YES 
user_id=45365

Adding an optional time.sleep(0) to the mainloop should do the trick, but unfortunately this won't work with the current MacPython, as time.sleep() is implemented with select(). And while GUSI sleep(0) does the threadswitch sleect(..., 0) doesn't. So this needs to be fixed in timemodule, and then the thread switch can be forced in Framework.Application.mainloop().

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=539990&group_id=5470


From noreply@sourceforge.net  Mon Feb 24 16:47:42 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Mon, 24 Feb 2003 08:47:42 -0800
Subject: [Python-bugs-list] [ python-Bugs-585923 ] macfs.FSSpec and Carbon.File.FSSpec fail for "new" files
Message-ID: <E18nLlW-0003mg-00@sc8-sf-web4.sourceforge.net>

Bugs item #585923, was opened at 2002-07-24 15:35
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=585923&group_id=5470

Category: Macintosh
Group: Python 2.3
>Status: Closed
>Resolution: Wont Fix
Priority: 4
Submitted By: Just van Rossum (jvr)
Assigned to: Jack Jansen (jackjansen)
>Summary: macfs.FSSpec and Carbon.File.FSSpec fail for "new" files

Initial Comment:
In the mach-o version of Python on MacOSX, the macfs.FSSpec() 
library function fails for files that don't yet exist.

The problem is caused by (internally) using an FSRef to construct 
the FSSpec: FSRefs can't hold references to non-existent files, 
whereas FSSpecs can.

----------------------------------------------------------------------

>Comment By: Jack Jansen (jackjansen)
Date: 2003-02-24 17:47

Message:
Logged In: YES 
user_id=45365

Macfs is no longer used in the core. Anyone running into this problem in user code will have to create a workaround based on constructing the FSSpec by hand (i.e. with the (vrefnum, dirid, filename) interface).

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-02-09 00:18

Message:
Logged In: YES 
user_id=45365

No, the problem still exists, also in Carbon.File, and probably is solvable. It has become less urgent, though, as FSSpecs are phased out in the core.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-07 23:20

Message:
Logged In: YES 
user_id=33168

Can this be closed since macfs has been removed?

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2002-08-05 23:54

Message:
Logged In: YES 
user_id=45365

macostools.copy() has been fixed in rev. 1.16. mkalias()
still has the problem, though.

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2002-08-03 23:01

Message:
Logged In: YES 
user_id=45365

As the workaround is going to be both difficult to implement and incomplete I'm tempted to punt on this one. Difficult because we have to split the path in C, and we have to do /: substitutions on the last component; incomplete because we cannot cater for utf8/macroman differences and filenames > 32 characters.

If we punt on this we should make sure we phase out using fsspec's in the standard library in favor of the fsref calls.

Do you think this is an acceptable solution? How much work would it be?

----------------------------------------------------------------------

Comment By: Just van Rossum (jvr)
Date: 2002-07-29 23:56

Message:
Logged In: YES 
user_id=92689

This bug also cause macostools.copy() to fail, making the breakage 
larger than I thought :-(

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=585923&group_id=5470


From noreply@sourceforge.net  Mon Feb 24 16:50:00 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Mon, 24 Feb 2003 08:50:00 -0800
Subject: [Python-bugs-list] [ python-Bugs-607821 ] IDE output window
Message-ID: <E18nLnk-0003ts-00@sc8-sf-web4.sourceforge.net>

Bugs item #607821, was opened at 2002-09-11 15:30
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=607821&group_id=5470

Category: Macintosh
Group: None
>Status: Closed
>Resolution: Rejected
Priority: 5
Submitted By: Jack Jansen (jackjansen)
Assigned to: Just van Rossum (jvr)
Summary: IDE output window

Initial Comment:
I think the IDE output window should pop to the front if new output appears there. Especially on OSX, where it may be obscured not only by other IDE windows but also by windows from other apps, you often don't realise that the script has produced output.

----------------------------------------------------------------------

>Comment By: Jack Jansen (jackjansen)
Date: 2003-02-24 17:49

Message:
Logged In: YES 
user_id=45365

As nothing happened (and Just didn't agree in the first place so there's a good chance that nothing is going to happen either:-) I'm retracting this bug.

----------------------------------------------------------------------

Comment By: Samuel Bronson (sbronson)
Date: 2002-09-28 06:43

Message:
Logged In: YES 
user_id=237753

it can be obscured by other windows in just about any window
system, including X. So why especially on OSX?

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2002-09-11 17:07

Message:
Logged In: YES 
user_id=45365

Hmm, true. Maybe make it an easy to set option?

----------------------------------------------------------------------

Comment By: Just van Rossum (jvr)
Date: 2002-09-11 15:37

Message:
Logged In: YES 
user_id=92689

I often play with a script interactively, with the output window and the 
script window side-by-side. For such use it would be awful to have to 
output window come on top every time. I don't know...

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=607821&group_id=5470


From noreply@sourceforge.net  Mon Feb 24 17:04:02 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Mon, 24 Feb 2003 09:04:02 -0800
Subject: [Python-bugs-list] [ python-Bugs-216289 ] Programs using Tkinter sometimes can't shut down (Windows)
Message-ID: <E18nM1K-0004hb-00@sc8-sf-web3.sourceforge.net>

Bugs item #216289, was opened at 2000-10-06 19:25
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=216289&group_id=5470

Category: Windows
Group: Python 2.3
Status: Open
Resolution: Later
Priority: 3
Submitted By: Tim Peters (tim_one)
Assigned to: Tim Peters (tim_one)
Summary: Programs using Tkinter sometimes can't shut down (Windows)

Initial Comment:
The following msg from the Tutor list is about 1.6, but I noticed the same thing several times today using 2.0b2+CVS.  In my case, I was running IDLE via

python ../tool/idle/idle.pyw

from a DOS box in my PCbuild directory.  Win98SE.  *Most* of the time, shutting down IDLE via Ctrl+Q left the DOS box hanging.  As with the poster, the only way to regain control was to use the Task Manager to kill off Winoldap.

-----Original Message-----
From: Joseph Stubenrauch <nothingisgoingtochangemyworld@yahoo.com>
Sent: Friday, October 06, 2000 9:23 PM
To: tutor@python.org
Subject: Re: [Tutor] Python 1.6 BUG


Strange, I have been experiencing the same bug myself.
Here's the low down for me:

Python 1.6 with win95
I am running a little Tkinter program
The command line I use is simply: "python foo.py"
About 25-35% of the time, when I close the Tkinter
window, DOS seems to "freeze" and never returns to the
c:\ command prompt.  I have to ctrl-alt-delete
repeatedly and shut down "winoldapp" to get rid of the
window and then shell back into DOS and keep working.
It's a bit of a pain, since I have the habit of
testing EVERYTHING in tiny little stages, so I change
one little thing, test it ... freeze ... ARGH!  Change
one more tiny thing, test it ... freeze ... ARGH! 
However, sometimes it seems to behave and doesn't
bother me for an entire several hour session of python
work.

That's my report on the problem.

Cheers,

Joe


----------------------------------------------------------------------

Comment By: Jeffrey Hobbs (hobbs)
Date: 2003-02-24 09:04

Message:
Logged In: YES 
user_id=72656

This was against an older version (8.3.2 or 8.3.3).  Make sure 
any verification occurs against the HEAD or 8.3.5.

Also, Martin provided a threaded Tkinter implmentation, IIUC, 
so that makes this doubly out-of-date.

----------------------------------------------------------------------

Comment By: Donal K. Fellows (dkf)
Date: 2003-02-24 05:33

Message:
Logged In: YES 
user_id=79902

Exactly which version (including patchlevel) of tcl.dll and
tk.dll is this bug reported against?  (Well, which is the
most recent version...)  Is it one which the Tcl Maintainers
believe to have all identified relevant bugs in the area
resolved, or does it predate that?

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-24 04:59

Message:
Logged In: YES 
user_id=6380

Um, I think it's *not* closed, but I'll assign it back to
Tim. I believe Tim said about this somewhere "the Tcl folks
have often thought they'd nailed it, but they still
haven't". Unless that was an entirely different problem. But
it used to happen for me on Win98 too (haven't tried in a
while).

----------------------------------------------------------------------

Comment By: Fredrik Lundh (effbot)
Date: 2003-02-23 23:13

Message:
Logged In: YES 
user_id=38376

(a bugfix a day keeps etc)

Tim, can we close this one?  I don't have access to any-
thing even remotely similar to a Win95-box any longer...

----------------------------------------------------------------------

Comment By: Jeffrey Hobbs (hobbs)
Date: 2002-12-09 20:16

Message:
Logged In: YES 
user_id=72656

Interesting last couple of comments ...

Anyway, I think the cause of this problem may have finally 
been identified by Tcl bug 651139.  I have attached a patch 
here for it against 8.3.5.  Both the 8.3.5 branch and 8.4.1+ 
head have this fix.  Amazingly it didn't effect Tcl, but could 
likely have caught Tkinter (only on Windows with non-
threaded builds).

Oh look, not allowed to attach a patch, so I'll attach it to Tcl 
bug 651139.

----------------------------------------------------------------------

Comment By: christian soldan (elacademico)
Date: 2002-11-30 18:07

Message:
Logged In: YES 
user_id=659858

how do i hack my fuckers friends

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2002-09-30 16:23

Message:
Logged In: NO 

i would like to know some moves

----------------------------------------------------------------------

Comment By: John Popplewell (johnnypops)
Date: 2002-08-21 17:08

Message:
Logged In: YES 
user_id=143340

Hi,

tried wrapping the WaitForSingleObject() with if (consolePtr-
>toWrite) { ... as suggested. Solves the problem for me in 
debug and release builds (without thread support).

What is "the tcl83.dll from www.tcl.tk", what changes does 
it include?

cheers,
John.


----------------------------------------------------------------------

Comment By: Doug Henderson (djhender)
Date: 2002-08-16 16:03

Message:
Logged In: YES 
user_id=148444

I downloaded and tested the tcl83.dll from www.tcl.tk with 
Python 2.2.1 on Win98SE using my BugDemo.py script 
from a related bug report [231207]. Several test runs 
showed that a problem still exists. The app hangs until 
Winopldapp is killed, however this leaves tcl83.dll in use. 
The problem does not occur on every execution. Some 
types of activity seem to provoke the hang, for instance, 
minimizing and maximizing the window seems to increase 
the change that it will hang on exit.

----------------------------------------------------------------------

Comment By: David Gravereaux (davygrvy)
Date: 2002-08-16 14:53

Message:
Logged In: YES 
user_id=7549

Whoop, error in the patch..  consoleMutex might not need to 
be locked.  It could cause a new dead-lock.

----------------------------------------------------------------------

Comment By: David Gravereaux (davygrvy)
Date: 2002-08-16 14:41

Message:
Logged In: YES 
user_id=7549

>It is hanging up in ConsoleCloseProc() in 'tclWinConsole.c'
>at about line 527:
>WaitForSingleObject(consolePtr->writeThread, INFINITE);

Jeff, this might need the same attention the pipe reader 
thread got for that [exec] memory leak bug.  I think 
ConsoleWriterThread() should use WaitForMultipleObjects 
instead of WaitForSingleObject with a second event handle 
used to single a fall-out like the pipe reader thread does.  It 
appears that ConsoleCloseProc() has no way to signal the 
writer thread to close at all.

This doesn't address the blocking WriteFile() in 
ConsoleWriterThread(), though.  Is it blocking there, too?
Sorry for the large post.  I'm given the option to attach a file.

Index: tclWinConsole.c
=============================================
======================
RCS file: /cvsroot/tcl/tcl/win/tclWinConsole.c,v
retrieving revision 1.7
diff -c -r1.7 tclWinConsole.c
*** tclWinConsole.c	15 Jan 2002 17:55:30 -0000	1.7
--- tclWinConsole.c	16 Aug 2002 21:31:56 -0000
***************
*** 79,84 ****
--- 79,87 ----
      HANDLE startWriter;		/* Auto-reset 
event used by the main thread to
  				 * signal when 
the writer thread should attempt
  				 * to write to the 
console. */
+     HANDLE stopWriter;		/* Auto-reset 
event used by the main thread to
+ 				 * signal when 
the writer thread should
+ 				 * terminate. */
      HANDLE startReader;		/* Auto-reset 
event used by the main thread to
  				 * signal when 
the reader thread should attempt
  				 * to read from 
the console. */
***************
*** 516,522 ****
  	 */
  
  	Tcl_MutexLock(&consoleMutex);
! 	TerminateThread(consolePtr->writeThread, 0);
  
  	/*
  	 * Wait for the thread to terminate.  This ensures 
that we are
--- 519,525 ----
  	 */
  
  	Tcl_MutexLock(&consoleMutex);
! 	SetEvent(consolePtr->stopWriter);
  
  	/*
  	 * Wait for the thread to terminate.  This ensures 
that we are
***************
*** 1134,1149 ****
  {
  
      ConsoleInfo *infoPtr = (ConsoleInfo *)arg;
      HANDLE *handle = infoPtr->handle;
      DWORD count, toWrite;
      char *buf;
  
      for (;;) {
  	/*
  	 * Wait for the main thread to signal before 
attempting to write.
  	 */
  
! 	WaitForSingleObject(infoPtr->startWriter, 
INFINITE);
  
  	buf = infoPtr->writeBuf;
  	toWrite = infoPtr->toWrite;
--- 1137,1163 ----
  {
  
      ConsoleInfo *infoPtr = (ConsoleInfo *)arg;
+     HANDLE events[2];
      HANDLE *handle = infoPtr->handle;
      DWORD count, toWrite;
      char *buf;
  
+     events[0] = infoPtr->stopWriter;
+     events[1] = infoPtr->startWriter;
+ 
      for (;;) {
+ 	DWORD dwWait;
+ 
  	/*
  	 * Wait for the main thread to signal before 
attempting to write.
  	 */
  
! 	dwWait = WaitForMultipleObjects(events, 2, 
FALSE, INFINITE);
! 
! 	if (dwWait != (WAIT_OBJECT_0 + 1)) {
! 	    /* either the stop event or some other error, so 
exit */
! 	    return 0;
! 	}
  
  	buf = infoPtr->writeBuf;
  	toWrite = infoPtr->toWrite;
***************
*** 1251,1256 ****
--- 1265,1271 ----
      if (permissions & TCL_WRITABLE) {
  	infoPtr->writable = CreateEvent(NULL, TRUE, 
TRUE, NULL);
  	infoPtr->startWriter = CreateEvent(NULL, FALSE, 
FALSE, NULL);
+ 	infoPtr->stopWriter = CreateEvent(NULL, FALSE, 
FALSE, NULL);
  	infoPtr->writeThread = CreateThread(NULL, 8000, 
ConsoleWriterThread,
  	        infoPtr, 0, &id);
      }


----------------------------------------------------------------------

Comment By: Jeffrey Hobbs (hobbs)
Date: 2002-08-16 11:56

Message:
Logged In: YES 
user_id=72656

I'm finding it a bit hard to reproduce, and now not at all.  
Could people please try replacing their current tcl83.dll with 
the one at http://www.tcl.tk/tcl83.dll.  The sig on it is:
634880 Aug 16 10:53 tcl83.dll
(md5) b35628568ddc4afed4f675d2d9a50faf  tcl83.dll

I can't hang anymore with python 2.2.1 at the Win98 
command prompt with:
   python \Python22\Tools\idle\idle.pyw

----------------------------------------------------------------------

Comment By: Jeffrey Hobbs (hobbs)
Date: 2002-08-16 10:50

Message:
Logged In: YES 
user_id=72656

johnny - That actually helps a lot and fairly clearly indicates 
the source of this hanging problem.  To give more context to 
that area, this is in the final shutdown code where it is 
(obviously) shutting down the console I/O.  The reader is 
shutdown without a check for more input (why, we are 
exiting?), so there is no issue there.

The writer however wants to make sure that the console 
output channel is drained before exit.  A blocking channel 
that is not being flushed will cause a hang.

So the question is, what is the stdio setup for Tcl in 
Py/Tkinter?  That should be examined to see whether stdio is 
clear to flush on exit.

Perhaps you can also help with more debugging.  Could you 
wrap the problematic WaitForSingleObject call with
if (consolePtr->toWrite) { ...
and perhaps also print out what to write.  It may be that 
blocking output isn't the problem at all, but rather that we are 
never receiving the signal on consolePtr->writable (usually 
done in the writer thread).  The consolePtr->toWrite check 
will at least ensure flushed channels do not enter this 
potential hanging state.

----------------------------------------------------------------------

Comment By: John Popplewell (johnnypops)
Date: 2002-08-16 07:16

Message:
Logged In: YES 
user_id=143340

I have dug out the setup I used previously and, apart from 
using Python 2.2 instead of 2.0.1, have had a go at 
reproducing my previous results.

With a debug build of Tcl/Tk and threads disabled I have 
managed to get this stack trace when it locks on exit (after 
doing a break):

ConsoleCloseProc(int * 0x006a0ae4, Tcl_Interp * 
0x00000000) line 527 + 17 bytes
TclFinalizeIOSubsystem() line 241 + 20 bytes
Tcl_FinalizeThread() line 911
Tcl_Finalize() line 812
DllMain(HINSTANCE__ * 0x00970000, unsigned long 
0x00000000, void * 0x00000001) line 197
TCL83! _DllMainCRTStartup@12 + 80 bytes
KERNEL32! bff7ddd6()
KERNEL32! bff8d123()
8176b5fc()

It is hanging up in ConsoleCloseProc() in 'tclWinConsole.c'
at about line 527:
WaitForSingleObject(consolePtr->writeThread, INFINITE);

I tried changing the timeout from INFINITE to 100ms and 
testing the return value for WAIT_TIMEOUT. Sure enough it 
occasionaly printed a message (and didn't lock-up).

Don't know if this helps. Will try to reproduce the double 
free problem in the threaded build later,

cheers,
John.


----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2002-08-15 07:19

Message:
Logged In: YES 
user_id=6380

Just a note so I won't forget this: Jeff Hobbs once said the
code to look at is

tk/generic/tkConsole.c:Tk_InitConsoleChannels

----------------------------------------------------------------------

Comment By: David Gravereaux (davygrvy)
Date: 2002-08-15 06:23

Message:
Logged In: YES 
user_id=7549

Hi John,

I have seen Tcl extension DLLs that set exit handlers get 
called by Tcl after they were torn-down.  NT and Win9x seem 
to be a bit different in the order of what DLLs get torn-down.  
here's a note from some old code of mine:

#ifdef __WIN32__
    // It can happen that the HEAP could have already been 
unloaded
    // from an awkward teardown caused by a Ctrl+C or other.  
Win32
    // seems to do a teardown in reverse order and by the time 
Tcl
    // knows what's going on and Tcl_Finalize calls the exit
    // handlers, this extension's data (heap?) has already been
    // unloaded by the OS.  Do a quick legal check on the 
pointer
    // first.
    //
    if (IsBadReadPtr(adapt, sizeof(T *))) return;
#endif

what's the back/stacktrace?  who is calling 
Tcl_FinalizeNotifier?  And had anyone called it before during 
the shutdown phase? I can't say with much confidence that 
this is happening here, too.  Maybe..  Could Tk be telling Tcl 
shutdown?  That's sort of backwards, if so.

I run w2k here, so I can't help in debugging from a test case.  
tclsh with a threaded build doesn't even call Tcl_Finalize to 
avoid all this!

----------------------------------------------------------------------

Comment By: Donal K. Fellows (dkf)
Date: 2002-08-15 06:09

Message:
Logged In: YES 
user_id=79902

Firstly, on the matter of threads.  Tcl/Tk on Windows uses
threads internally even in non-threaded builds.  That's
Windows for you (in particular, you can get calls parachuted
in from completely separate threads with relatively little
warning.  Also, you need threaded helpers to handle
non-blocking waiting on some kinds of objects.)

It's a bit hard to work out if the problem's been fixed
without a stack-trace (preferably with debugging symbols
enabled so that we can see what the function names and
arguments to those functions are.)  I have no  chance of
replicating this bug here (I'm currently running IRIX, but
also have some access to Solaris and Linux...)

----------------------------------------------------------------------

Comment By: John Popplewell (johnnypops)
Date: 2002-08-15 05:15

Message:
Logged In: YES 
user_id=143340

Hi to davygrvy,

my fix isn't for a thread related (deadlock) type problem - it 
is a fix for a double free type problem which is presumably 
screwing up the heap and causing the C runtime to hang.

It *is* with a threaded build (THREADDEFINES = -
DTCL_THREADS=1) which I think I enabled because I was 
using  'after' and it didn't work unless threading was 
enabled - but I may be mis-remembering here!

Also, I ended up doing all this with the debug build and 
found the -GZ flag useful - in fact it pointed out the fact that 
there was a problem with the heap.

It is worth noting that I have only seen this on Win98 and 
that using 'pythonw.exe' did *not* prevent this from 
happening,

regards John.


----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2002-08-14 18:00

Message:
Logged In: YES 
user_id=6380

AFAIK we never distributed a threaded build.

----------------------------------------------------------------------

Comment By: David Gravereaux (davygrvy)
Date: 2002-08-14 17:01

Message:
Logged In: YES 
user_id=7549

To johnnypops,

Where in Tcl_FinalizeNotifier() is the dead-lock occuring and 
is this with a threaded build of Tcl?


----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2002-07-02 12:39

Message:
Logged In: YES 
user_id=31435

Jeff, a long time ago Fredrik Lundh said he was able to 
reproduce this with a short wish script (no Python, no Tkinter, 
just Tcl/Tk + wish).  I'm reassigning this to /F in the hopes 
he'll remember how.

----------------------------------------------------------------------

Comment By: Jeffrey Hobbs (hobbs)
Date: 2002-07-02 11:55

Message:
Logged In: YES 
user_id=72656

I appreciate that many others see this is a problem in 
embedding Tcl, but without any clear way to reproduce this in 
Tcl/Tk itself, it's very hard for me to fix it.  I don't think that 
there is anything done in 8.4 to fix this that I recall, but if 
someone finds that the problem "goes away" with 8.4, please 
report that.

For those of course that experience this with python, the 
simple and "correct" work-around is to launch with 
pythonw.exe instead of python.exe.

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2002-06-26 10:40

Message:
Logged In: NO 

According to http://tcl.activestate.com ActiveTcl 8.4.0 is now 
available for download.  It is new as of June 24.  I don't know 
whether this will solve our problems or not, but it's worth 
looking into.

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2002-06-19 06:36

Message:
Logged In: NO 

I just downloaded Ruby 1.6.6 for Windows, which also uses 
Tcl/Tk 8.3 and I experience exactly the same problem there. 
So this is definitely a Tcl problem (but we know that already).

----------------------------------------------------------------------

Comment By: John Popplewell (johnnypops)
Date: 2002-06-07 12:01

Message:
Logged In: YES 
user_id=143340

Hmmm. 
I run a Python Tcl/Tk application several times every day - 
the very application that prompted my bug-hunt.

I do use "pythonw.exe" to launch it, but it used to hang 
almost as badly as "python.exe".

Whilst developing this app I used "python.exe" and 
experienced no hang-ups.

The Tcl guys point out that this bug is only one of several 
possible causes of the hang-up, maybe you are just out of 
luck.

The patch has fixed the problem on three Win98 machines, 
all of which exhibited the problem before updating the DLLs.

One thought - are you sure there weren't multiple copies of 
the tcl83.dll and tk83.dll files? My (only) copies were 
in "D:\python20\DLLs"

good luck,
John.


----------------------------------------------------------------------

Comment By: Jeffrey Hobbs (hobbs)
Date: 2002-06-03 11:55

Message:
Logged In: YES 
user_id=72656

providing Tim with a fixed DLL exactly according to John 
Popplewell's patch did not fix the problem for him.  That 
makes it seem fishy that John really didn't see the problem 
any more.

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2002-06-03 09:59

Message:
Logged In: YES 
user_id=31435

I tried replacing Python 2.2.1's tk83.dll and tcl83.dll (from 
Tcl/Tk 8.3.2) with newer matched sets from

A) PythonWare's py21 distribution (Tcl/Tk 8.3.3).

and again from

B) ActiveState's Tcl/Tk 8.3.4.

All the symptoms originally reported here persisted on 
Win98SE despite that.

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2002-05-22 06:57

Message:
Logged In: NO 

I am new to Python and encountered this bug on my very first 
Tkinter application.  It persists in Python 2.2.1 under 
Windows Me. Needless to say, it makes a bad first 
impression.  Basically, this thread says that Tkinter has been 
broken for a year and a half on a popular platform at a time 
when Python is trying to expand its user base.  Hopefully, this 
bug will be corrected soon.

If Tcl/Tk 8.4 is not released shortly you might try repackaging 
Tkinter for Win32 with an earlier version of Tcl/Tk, such as 
8.0, that doesn't encounter this problem.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2002-03-22 19:36

Message:
Logged In: YES 
user_id=6380

The proper action is to wait until Tcl 8.3.5 is released,
and then shaming someone else in providing you with a set of
Windowsbinaries for Tcl. You may also ask Hobbs if there's a
Windows project file to build Tcl/Tk for Windows.

----------------------------------------------------------------------

Comment By: Jeffrey Hobbs (hobbs)
Date: 2002-03-11 18:31

Message:
Logged In: YES 
user_id=72656

I did end up back-porting this to the 8.3.4+ Tcl CVS on 
2002-02-25, which means it will be in an eventual 8.3.5.  
It is already in the release 8.4 alpha series.

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2002-03-09 17:50

Message:
Logged In: YES 
user_id=31435

Back to Guido:  Do you think you know what the proper 
action is?  (You said that's why you reopened it, and 
there's been no new info here since then.)

----------------------------------------------------------------------

Comment By: John Popplewell (johnnypops)
Date: 2002-02-25 16:55

Message:
Logged In: YES 
user_id=143340

I knew I wasn't getting to the heart of it ....

Almost a one-liner!

It has been seriously spoiling my (otherwise crash free) 
Python experience on Win9x - hope it gets fixed soon.

cheers,
John.


----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2002-02-25 15:39

Message:
Logged In: YES 
user_id=6380

Reopened until we know what the proper action is.

----------------------------------------------------------------------

Comment By: Jeffrey Hobbs (hobbs)
Date: 2002-02-25 15:32

Message:
Logged In: YES 
user_id=72656

This is mostly correct, and is fixed in the 8.4 Tcl sources 
already (I guess we can backport this).  This was mentioned 
in SF Tcl bug (account for chopped URL):

https://sourceforge.net/tracker/?
func=detail&aid=217982&group_id=10894&atid=110894

and the code comment is:

    /*
     * Only finalize the notifier if a notifier was 
installed in the
     * current thread; there is a route in which this is not
     * guaranteed to be true (when tclWin32Dll.c:DllMain() 
is called
     * with the flag DLL_PROCESS_DETACH by the OS, which 
could be
     * doing so from a thread that's never previously been 
involved
     * with Tcl, e.g. the task manager) so this check is 
important.
     *
     * Fixes Bug #217982 reported by Hugh Vu and Gene 
Leache.
     */
    if (tsdPtr == NULL) {
	return;
    }


----------------------------------------------------------------------

Comment By: John Popplewell (johnnypops)
Date: 2002-02-25 14:41

Message:
Logged In: YES 
user_id=143340

This one has been torturing me for a while.

Managed to track it down to a problem inside Tcl.
For the Tcl8.3.4 source distribution the problem is in
the file win/tclWinNotify.c 

void Tcl_FinalizeNotifier(ClientData clientData)
{
    ThreadSpecificData *tsdPtr = 
                         (ThreadSpecificData *) clientData;

    /* sometimes called with tsdPtr == NULL */
    if ( tsdPtr != NULL )
    {
        DeleteCriticalSection(&tsdPtr->crit);
        CloseHandle(tsdPtr->event);

        /*
         * Clean up the timer and messaging 
         * window for this thread.
         */

        if (tsdPtr->hwnd) {
	    KillTimer(tsdPtr->hwnd, INTERVAL_TIMER);
            DestroyWindow(tsdPtr->hwnd);
        }
    }

    /*
     * If this is the last thread to use the notifier, 
     * unregister the notifier window class.
     */

    Tcl_MutexLock(&notifierMutex);
    if ( notifierCount && !--notifierCount ) {
	UnregisterClassA( "TclNotifier", 
                          TclWinGetTclInstance());
    }
    Tcl_MutexUnlock(&notifierMutex);
}

This bodge doesn't address the underlying problem but has 
stopped me from tearing all my hair out,

cheers,
John Popplewell.


----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2001-10-24 15:27

Message:
Logged In: YES 
user_id=31435

FYI, you don't need an IDE to do this -- in Win9x, hit 
Ctrl+Alt+Del and kill the process directly.

A saner <wink> solution is to develop under Win2K, which 
doesn't appear to suffer this problem (the only reports 
I've seen, and experienced myself, came from Win9x boxes).

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2001-10-24 01:52

Message:
Logged In: NO 

For those who are still trapped in this bug's hell, I will 
gladly share the one thing that saved my sanity:  
WingIDE's 'Kill' command will shut down the program with 
apparent 100% certainty and no fear of lockups.  WingIDE 
has its own issues, so its not a perfect solution, but if 
you are like me and Joe (above) who test in small 
iterations, then using 'Kill' to quit out of your app while 
testing is a workaround that may preserve your sanity.

Perhaps the python gods and the Wing guys can get together 
and tell us how to replicate 'kill' into our code.  For 
now, I'll use WingIDE to edit, and pythonw.exe for my final 
client's delivery.

----------------------------------------------------------------------

Comment By: Howard Lightstone (hlightstone)
Date: 2001-09-05 10:43

Message:
Logged In: YES 
user_id=66570

I sometimes get bunches of these....

A tool I use (Taskinfo2000) reports that (after killing 
winoldap):

python.exe is blocked on a mutex named OLESCELOCKMUTEX.
The reported state is "Console Terminating".  There appears 
to be only one (os) thread running.


----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2001-04-02 13:06

Message:
Logged In: YES 
user_id=31435

No sign of progress on this presumed Tk/Tcl Windows bug in 
over 3 months, so closing it for lack of hope.

----------------------------------------------------------------------

Comment By: Doug Henderson (djhender)
Date: 2001-02-05 21:13

Message:
This was a symptom I saw while tracking down the essence of the problem reported in #131207.  Using Win98SE, I would get an error dialog (GPF?) in the Kernel, and sometimes the dos prompt would not come back.

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2000-12-12 18:00

Message:
Just reproduced w/ current CVS, but didn't hang until the 8th try.

http://dev.scriptics.com/software/tcltk/ says 8.3 is still the latest released version; don't know whether that URL still makes sense, though.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2000-12-12 12:58

Message:
Tim, can you still reproduce this with the current CVS version?

There's been one critical patch to _tkinter since the 2.0 release.

An alternative would be to try with a newer version of Tcl (isn't 8.4 out already?).


----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2000-10-15 09:47

Message:
Same as I've reported earlier; it hangs in the call to
Tcl_Finalize (which is called by the DLL finalization
code).  It's less likely to hang if I call Tcl_Finalize
from the _tkinter DLL (from user code).

Note that the problem isn't really Python-related -- I
have stand-alone samples (based on wish) that hangs in
the same way.

More later.

</F>

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2000-10-13 07:40

Message:
Back to Tim since I have no clue what to do here.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2000-10-12 10:25

Message:
The recent fix to _tkinter (Tcl_GetStringResult(interp) instead of interp->result) didn't fix this either.

As Tim has remarked in private but not yet recorded here, a workaround is to use pythonw instead of python, so I'm lowering thepriority again.

Also note that the hanging process that Tim writes about apparently prevents Win98 from shutting down properly.

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2000-10-07 00:37

Message:
More info (none good, but some worse so boosted priority):

+ Happens under release and debug builds.

+ Have not been able to provoke when starting in the debugger.

+ Ctrl+Alt+Del and killing Winoldap is not enough to clean everything up.  There's still a Python (or Python_d) process hanging around that Ctrl+Alt+Del doesn't show.

+ This process makes it impossible to delete the associated Python .dll, and in particular makes it impossible to rebuild Python successfully without a reboot.

+ These processes cannot be killed!  Wintop and Process Viewer both fail to get the job done.  PrcView (a freeware process viewer) itself locks up if I try to kill them using it.  Process Viewer freezes for several seconds before giving up.

+ Attempting to attach to the process with the MSVC debugger (in order to find out what the heck it's doing) finds the process OK, but then yields the cryptic and undocumented error msg "Cannot execute program".

+ The processes are not accumulating cycles.

+ Smells like deadlock.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=216289&group_id=5470


From noreply@sourceforge.net  Mon Feb 24 17:22:40 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Mon, 24 Feb 2003 09:22:40 -0800
Subject: [Python-bugs-list] [ python-Bugs-690419 ] can't build bsddb for 2.3a2
Message-ID: <E18nMJM-0005t1-00@sc8-sf-web4.sourceforge.net>

Bugs item #690419, was opened at 2003-02-21 01:12
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690419&group_id=5470

Category: Build
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Paul Jarc (prjsf)
Assigned to: Nobody/Anonymous (nobody)
Summary: can't build bsddb for 2.3a2

Initial Comment:
I have db-4.1.25 installed, and I'm trying to install
Python 2.3a2 with
bsddb.  The build fails:
gcc -fPIC  -DNDEBUG -g -O3 -Wall -Wstrict-prototypes
-I. -I./Include 
-I/package/misc/spf/python-2.3a2/spf/db/include -c
./Modules/bsddbmodule.c -o Modules/bsddbmodule.o
Modules/bsddbmodule.c: In function `newdbhashobject':
Modules/bsddbmodule.c:58: `HASHINFO' undeclared (first
use in this function)
Modules/bsddbmodule.c:58: (Each undeclared identifier
is reported only once
Modules/bsddbmodule.c:58: for each function it appears in.)
Modules/bsddbmodule.c:58: parse error before "info"
Modules/bsddbmodule.c:63: `info' undeclared (first use
in this function)
Modules/bsddbmodule.c:74: warning: implicit declaration
of function `dbopen'
Modules/bsddbmodule.c:74: warning: assignment makes
pointer from integer without a cast
Modules/bsddbmodule.c: In function `newdbbtobject':
Modules/bsddbmodule.c:106: `BTREEINFO' undeclared
(first use in this function)
Modules/bsddbmodule.c:106: parse error before "info"
Modules/bsddbmodule.c:111: `info' undeclared (first use
in this function)
Modules/bsddbmodule.c:124: warning: assignment makes
pointer from integer without a cast
Modules/bsddbmodule.c: In function `newdbrnobject':
Modules/bsddbmodule.c:156: `RECNOINFO' undeclared
(first use in this function)
Modules/bsddbmodule.c:156: parse error before "info"
Modules/bsddbmodule.c:162: `info' undeclared (first use
in this function)
Modules/bsddbmodule.c:182: warning: assignment makes
pointer from integer without a cast
Modules/bsddbmodule.c: In function `bsddb_dealloc':
Modules/bsddbmodule.c:223: too few arguments to function
Modules/bsddbmodule.c: In function `bsddb_length':
Modules/bsddbmodule.c:252: structure has no member
named `seq'
Modules/bsddbmodule.c:253: `R_FIRST' undeclared (first
use in this function)
Modules/bsddbmodule.c:255: structure has no member
named `seq'
Modules/bsddbmodule.c:256: `R_NEXT' undeclared (first
use in this function)
Modules/bsddbmodule.c: In function `bsddb_subscript':
Modules/bsddbmodule.c:276: `recno_t' undeclared (first
use in this function)
Modules/bsddbmodule.c:276: parse error before "recno"
Modules/bsddbmodule.c:279: `recno' undeclared (first
use in this function)
Modules/bsddbmodule.c:299: warning: passing arg 2 of
pointer to function from incompatible pointer type
Modules/bsddbmodule.c:299: too few arguments to function
Modules/bsddbmodule.c: In function `bsddb_ass_sub':
Modules/bsddbmodule.c:327: `recno_t' undeclared (first
use in this function)
Modules/bsddbmodule.c:327: parse error before "recno"
Modules/bsddbmodule.c:330: `recno' undeclared (first
use in this function)
Modules/bsddbmodule.c:351: warning: passing arg 2 of
pointer to function from incompatible pointer type
Modules/bsddbmodule.c:351: too few arguments to function
Modules/bsddbmodule.c:363: warning: passing arg 2 of
pointer to function from incompatible pointer type
Modules/bsddbmodule.c:363: too few arguments to function
Modules/bsddbmodule.c: In function `bsddb_close':
Modules/bsddbmodule.c:388: too few arguments to function
Modules/bsddbmodule.c: In function `bsddb_keys':
Modules/bsddbmodule.c:415: structure has no member
named `seq'
Modules/bsddbmodule.c:415: `R_FIRST' undeclared (first
use in this function)
Modules/bsddbmodule.c:441: structure has no member
named `seq'
Modules/bsddbmodule.c:442: `R_NEXT' undeclared (first
use in this function)
Modules/bsddbmodule.c: In function `bsddb_has_key':
Modules/bsddbmodule.c:470: `recno_t' undeclared (first
use in this function)
Modules/bsddbmodule.c:470: parse error before "recno"
Modules/bsddbmodule.c:474: `recno' undeclared (first
use in this function)
Modules/bsddbmodule.c:491: warning: passing arg 2 of
pointer to function from incompatible pointer type
Modules/bsddbmodule.c:491: too few arguments to function
Modules/bsddbmodule.c: In function `bsddb_set_location':
Modules/bsddbmodule.c:509: `recno_t' undeclared (first
use in this function)
Modules/bsddbmodule.c:509: parse error before "recno"
Modules/bsddbmodule.c:513: `recno' undeclared (first
use in this function)
Modules/bsddbmodule.c:530: structure has no member
named `seq'
Modules/bsddbmodule.c:530: `R_CURSOR' undeclared (first
use in this function)
Modules/bsddbmodule.c: In function `bsddb_seq':
Modules/bsddbmodule.c:570: structure has no member
named `seq'
Modules/bsddbmodule.c: In function `bsddb_next':
Modules/bsddbmodule.c:608: `R_NEXT' undeclared (first
use in this function)
Modules/bsddbmodule.c: In function `bsddb_previous':
Modules/bsddbmodule.c:613: `R_PREV' undeclared (first
use in this function)
Modules/bsddbmodule.c: In function `bsddb_first':
Modules/bsddbmodule.c:618: `R_FIRST' undeclared (first
use in this function)
Modules/bsddbmodule.c: In function `bsddb_last':
Modules/bsddbmodule.c:623: `R_LAST' undeclared (first
use in this function)
make: *** [Modules/bsddbmodule.o] Error 1


----------------------------------------------------------------------

>Comment By: Paul Jarc (prjsf)
Date: 2003-02-24 17:22

Message:
Logged In: YES 
user_id=412110

Ok, here it is as an attachment.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-23 14:12

Message:
Logged In: YES 
user_id=6380

Please don't include long listings in the bug report. Attach
then separately.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690419&group_id=5470


From noreply@sourceforge.net  Mon Feb 24 18:20:43 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Mon, 24 Feb 2003 10:20:43 -0800
Subject: [Python-bugs-list] [ python-Bugs-691005 ] _POSIX_C_SOURCE redefined
Message-ID: <E18nNDX-0000HX-00@sc8-sf-web3.sourceforge.net>

Bugs item #691005, was opened at 2003-02-21 19:24
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=691005&group_id=5470

Category: Python Library
Group: Python 2.3
>Status: Closed
>Resolution: Invalid
Priority: 1
Submitted By: Mike Romberg (romberg)
Assigned to: Nobody/Anonymous (nobody)
Summary: _POSIX_C_SOURCE redefined

Initial Comment:
  Not really a bug.  Just a warning which might be
easy to cleanup with an ifndef.  If the following simple
file is compiled with gcc-3.2.2 on redhat-8.0 (i686) using
the c++ compiler, you get the following warning:

--------------------- test.C -------------------
#include <iostream>
#include <Python.h>
-----------------------------------------------

In file included from
/usr/local/python-2.3a2/include/python2.3/Python.h:8,
                 from test.C:2:
/usr/local/python-2.3a2/include/python2.3/pyconfig.h:819:1:
warning: "_POSIX_C_SOURCE" redefined
In file included from
/usr/local/gcc-3.2.2/include/c++/3.2.2/i686-pc-linux-gnu/bits/os_defines.h:39,
                 from
/usr/local/gcc-3.2.2/include/c++/3.2.2/i686-pc-linux-gnu/bits/c++config.h:34,
                 from
/usr/local/gcc-3.2.2/include/c++/3.2.2/iostream:44,
                 from test.C:1:
/usr/include/features.h:131:1: warning: this is the
location of the previous definition




----------------------------------------------------------------------

>Comment By: Mike Romberg (romberg)
Date: 2003-02-24 11:20

Message:
Logged In: YES 
user_id=61373

  Doh!  This is even documented on the python
C API reference.  Sorry for wasting your time.


----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2003-02-22 03:43

Message:
Logged In: YES 
user_id=6656

It's a really bad idea to include system headers *before*
including Python.h.

Do you still get the errors if you reverse the order?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=691005&group_id=5470


From noreply@sourceforge.net  Mon Feb 24 18:44:05 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Mon, 24 Feb 2003 10:44:05 -0800
Subject: [Python-bugs-list] [ python-Bugs-492465 ] mkcwproject: custom __initialize routine
Message-ID: <E18nNa9-0002AA-00@sc8-sf-web1.sourceforge.net>

Bugs item #492465, was opened at 2001-12-13 15:23
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=492465&group_id=5470

Category: Macintosh
Group: Feature Request
>Status: Closed
>Resolution: Wont Fix
Priority: 1
Submitted By: Just van Rossum (jvr)
Assigned to: Just van Rossum (jvr)
Summary: mkcwproject: custom __initialize routine

Initial Comment:
Both for _Tkinter and CoreGraphics it is neccesary to
specify a custom fragment initializer. It would be nice
if this were doable with mkcwproject, as it would reduce
the number of CW project file under CVS by 4...

----------------------------------------------------------------------

>Comment By: Just van Rossum (jvr)
Date: 2003-02-24 19:44

Message:
Logged In: YES 
user_id=92689

Good idea, it's not like CW+CFM has a great future for Python anyway...

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-02-24 17:42

Message:
Logged In: YES 
user_id=45365

Just, what do you think about closing this with a "won't fix"?

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2002-05-24 00:37

Message:
Logged In: YES 
user_id=45365

genpluginprojects has been modified to allow an optional initialize=xxx keyword argument (and its mkcwproject foundation too). The only thing I've tested is that it doesn't have any adverse effects on the existing projects that are built. Assigning it back to you so you can try to generate the CG project with it and remove the hand-built CG project file. If that all works assign it back to me and I'll do the same for Tkinter.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=492465&group_id=5470


From noreply@sourceforge.net  Mon Feb 24 18:56:29 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Mon, 24 Feb 2003 10:56:29 -0800
Subject: [Python-bugs-list] [ python-Bugs-690309 ] 2.3a2 Sol8 make fails at _iconv_codec.
Message-ID: <E18nNm9-00010J-00@sc8-sf-web4.sourceforge.net>

Bugs item #690309, was opened at 2003-02-20 16:53
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690309&group_id=5470

Category: Build
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Inyeol Lee (inyeol)
Assigned to: Nobody/Anonymous (nobody)
Summary: 2.3a2 Sol8 make fails at _iconv_codec.

Initial Comment:
Python 2.3a2 build process fails during building
_iconv_codec
module. The message is;

[...]
gcc -shared
build/temp.solaris-2.8-sun4u-2.3/_iconv_codec.o
-L/home/inyeol/util/python/lib -L/usr/local/lib -o
build/lib.solaris-2.8-sun4u-2.3/_iconv_codec.so
Traceback (most recent call last):

  [... long traceback here...]

  File "./setup.py", line 209, in build_extension
    imp.load_dynamic(ext.name, ext_filename)
RuntimeError: can't initialize the _iconv_codec module:
iconv_open() failed
*** Error code 1
make: Fatal error: Command failed for target
`sharedmods'

My build environment is gcc2.95.3 with Solaris 8
linker.
I've built 2.2, 2,2,1, 2,2,2, 2,3a1 with the same env
and there
was no (major) issues like this. I attached my
configure and
make log messages.


----------------------------------------------------------------------

Comment By: Alejandro Lopez-Valencia (dradul)
Date: 2003-02-24 13:56

Message:
Logged In: YES 
user_id=659006

Ahh! I skimmed too fast through the code and didn't see that one! 
You see, GNU iconv is, errr, anal in the matter of ecoding 
names. And the author/maintainer is adamant on using stricly 
and only the names as specified in RFC1345, RFC1468, 
RFC1557 and related (I have a document on the subject I can 
mail you if you want). 

And... ISO8859-1 is not a valid name as per iconv_open(3) for 
GNU iconv's (any linux system has that man page). On the other 
hand, ISO-8859-1 is valid. That shoud fix the problem.


----------------------------------------------------------------------

Comment By: Walter Dörwald (doerwalter)
Date: 2003-02-24 10:10

Message:
Logged In: YES 
user_id=89016

These are just warnings, I see no error here. BTW the GNU
implementation does supports both ASCII and ISO8859-1 as
encodings, so I don't see why 1.12 should break anything

----------------------------------------------------------------------

Comment By: Alejandro Lopez-Valencia (dradul)
Date: 2003-02-23 07:25

Message:
Logged In: YES 
user_id=659006

The fix applied to _iconv_codec.c (in version 1.12) *breaks* the 
extension unnder cygwin and possibly other systems that use 
GNU iconv instead of a native library (namely, Solaris and BSD 
derivatives). The relevant errors:

building '_iconv_codec' extension
gcc -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -I. -
I/cvsroot/python/dist/src/./Include -I/usr/local/include -
I/tmp/Include -I/tmp/python -c /cvsroot/python/dist
/src/Modules/_iconv_codec.c -o build/temp.cygwin-1.3.20-i686-
2.3/_iconv_codec.o
cc1: warning: changing search order for system 
directory "/usr/local/include"
cc1: warning:   as it has already been specified as a non-system 
directory
/cvsroot/python/dist/src/Modules/_iconv_codec.c: In function 
`iconvcodec_encode'
:
/cvsroot/python/dist/src/Modules/_iconv_codec.c:147: warning: 
passing arg 2 of `
libiconv' from incompatible pointer type
/cvsroot/python/dist/src/Modules/_iconv_codec.c: In function 
`iconvcodec_decode'
:
/cvsroot/python/dist/src/Modules/_iconv_codec.c:357: warning: 
passing arg 2 of `
libiconv' from incompatible pointer type
/cvsroot/python/dist/src/Modules/_iconv_codec.c: In function 
`init_iconv_codec':

/cvsroot/python/dist/src/Modules/_iconv_codec.c:682: warning: 
passing arg 2 of `
libiconv' from incompatible pointer type
gcc -shared -Wl,--enable-auto-image-base build/temp.cygwin-
1.3.20-i686-2.3/_icon
v_codec.o -L/usr/local/lib -L. -liconv -lpython2.3 -o 
build/lib.cygwin-1.3.20-i6
86-2.3/_iconv_codec.dll
running build_scripts

----------------------------------------------------------------------

Comment By: Walter Dörwald (doerwalter)
Date: 2003-02-21 13:29

Message:
Logged In: YES 
user_id=89016

I changed the init function to use 'ISO8859-1' instead of
'ASCII' as the test encoding. Can someone test whether this
fixes anything?

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-20 17:15

Message:
Logged In: YES 
user_id=33168

In order to get the build to proceed, you can change line
210 of setup.py from:

        except ImportError, why:
to:
        except (ImportError, RuntimeError), why:

This is just before:
    self.announce('*** WARNING: renaming "%s" since
importing it' ...)

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690309&group_id=5470


From noreply@sourceforge.net  Mon Feb 24 18:54:11 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Mon, 24 Feb 2003 10:54:11 -0800
Subject: [Python-bugs-list] [ python-Bugs-692416 ] tkinter.createfilehandler dumps core
Message-ID: <E18nNjv-0001xh-00@sc8-sf-web3.sourceforge.net>

Bugs item #692416, was opened at 2003-02-24 11:54
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=692416&group_id=5470

Category: Tkinter
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Mike Romberg (romberg)
Assigned to: Nobody/Anonymous (nobody)
Summary: tkinter.createfilehandler dumps core

Initial Comment:
  This small example probably uses an invalid
mask and the filedescriptor for a dummy file.  But
I'm seeing this core dump even when using
a correct mask and sockets.  This is with 2.3a2.

>>> import Tkinter
>>> tk = Tkinter.Tk()
>>> fp = open('foo', 'w')
>>> def foo():
...     pass
... 
>>> Tkinter._tkinter.createfilehandler(fp.fileno(), 0, foo)
Segmentation fault (core dumped)


  Here is the traceback:

#0  Tkapp_CreateFileHandler (self=0x0, args=0x1) at
Modules/_tkinter.c:2205
#1  0x080f279a in PyCFunction_Call (func=0x0,
arg=0x40730a68, kw=0x0)
    at Objects/methodobject.c:108
#2  0x080a73cd in call_function (pp_stack=0xbffff2ec,
oparg=1)
    at Python/ceval.c:3285
#3  0x080a58ed in eval_frame (f=0x815700c) at
Python/ceval.c:2041
#4  0x080a630c in PyEval_EvalCodeEx (co=0x40569160,
globals=0x0, locals=0x1, 
    args=0x815700c, argcount=0, kws=0x0, kwcount=0,
defs=0x0, defcount=0, 
    closure=0x0) at Python/ceval.c:2588
#5  0x080a8af7 in PyEval_EvalCode (co=0x1, globals=0x1,
locals=0x1)
    at Python/ceval.c:535
#6  0x080cf25b in run_node (n=0x4015f4e8, 
    filename=0x1 <Address 0x1 out of bounds>,
globals=0x1, locals=0x1, 
    flags=0x1) at Python/pythonrun.c:1105
#7  0x080ce7ae in PyRun_InteractiveOneFlags (fp=0x1, 
    filename=0x80f530a "<stdin>", flags=0xbffff4d8) at
Python/pythonrun.c:609
#8  0x080ce5a3 in PyRun_InteractiveLoopFlags
(fp=0x421271c0, 
    filename=0x80f530a "<stdin>", flags=0x40569160) at
Python/pythonrun.c:542
#9  0x080cfae2 in PyRun_AnyFileExFlags (fp=0x421271c0, 
    filename=0x80f530a "<stdin>", closeit=0,
flags=0xbffff4d8)
    at Python/pythonrun.c:505
#10 0x08054909 in Py_Main (argc=0, argv=0xbffff544) at
Modules/main.c:446
#11 0x0805448b in main (argc=1, argv=0x1) at
Modules/python.c:23
#12 0x420158d4 in __libc_start_main () from
/lib/i686/libc.so.6


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=692416&group_id=5470


From noreply@sourceforge.net  Mon Feb 24 18:51:10 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Mon, 24 Feb 2003 10:51:10 -0800
Subject: [Python-bugs-list] [ python-Bugs-539990 ] Framework.mainloop() - multiple threads
Message-ID: <E18nNh0-0002S0-00@sc8-sf-web1.sourceforge.net>

Bugs item #539990, was opened at 2002-04-05 23:29
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=539990&group_id=5470

Category: Macintosh
Group: Platform-specific
Status: Closed
Resolution: Wont Fix
Priority: 7
Submitted By: Pieter Claerhout (pclaerhout)
Assigned to: Jack Jansen (jackjansen)
Summary: Framework.mainloop() - multiple threads

Initial Comment:
I tried to use the threading module to start the XML 
RPC server in a different thread (xmlrpcServer itself 
also is a threaded piece of code), but that seems to 
conflict with the mainloop. If you start this 
application, it starts the thread, starts the mainloop 
and stays in there, once you finish the mainloop, the 
thread continues.

You've stumbled on a bug in Framework.mainloop(): it 
doesn't know anything about multiple threads. It gives 
time to other applications (by calling WaitNextEvent) 
but not to other threads within Python.

Also see
http://mail.python.org/pipermail/pythonmac-sig/2002-
April/005416.html

and 

http://mail.python.org/pipermail/pythonmac-sig/2002-
April/005428.html

----------------------------------------------------------------------

>Comment By: Just van Rossum (jvr)
Date: 2003-02-24 19:51

Message:
Logged In: YES 
user_id=92689

Jack, have you forgotten about autoGIL? It's quite likely that the autoGIL module (as part of the PyObjC project) would fix this issue out of the box on OSX. Do "import autoGIL; autoGIL.installAutoGIL()" before you enter the event loop and the second thread gets time to run.

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-02-24 17:45

Message:
Logged In: YES 
user_id=45365

Let's admit this isn't going to be fixed unless someone wants it real bad.

Pieter: if you want it real bad then reopen the bug and I'll find time for it.

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2002-04-22 16:03

Message:
Logged In: YES 
user_id=45365

After some discussion on the SIG it turns out this is a rather complicated problem, because WNE() may cause callbacks to be called (for Carbon Event handlers, for instance), and releasing the GIL would mean those callbacks would be hosed.

A solution will have to have all callback wrappers acquire the GIL, and it will either have to make very sure we never enter WNE() or friends with the GIL held, or the callback code will need to be able to handle both the case of being called with the GIL held and the case of being called without it.

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2002-04-07 23:26

Message:
Logged In: YES 
user_id=45365

Adding an optional time.sleep(0) to the mainloop should do the trick, but unfortunately this won't work with the current MacPython, as time.sleep() is implemented with select(). And while GUSI sleep(0) does the threadswitch sleect(..., 0) doesn't. So this needs to be fixed in timemodule, and then the thread switch can be forced in Framework.Application.mainloop().

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=539990&group_id=5470


From noreply@sourceforge.net  Mon Feb 24 20:29:29 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Mon, 24 Feb 2003 12:29:29 -0800
Subject: [Python-bugs-list] [ python-Bugs-690309 ] 2.3a2 Sol8 make fails at _iconv_codec.
Message-ID: <E18nPE9-00054r-00@sc8-sf-web4.sourceforge.net>

Bugs item #690309, was opened at 2003-02-20 22:53
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690309&group_id=5470

Category: Build
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Inyeol Lee (inyeol)
Assigned to: Nobody/Anonymous (nobody)
Summary: 2.3a2 Sol8 make fails at _iconv_codec.

Initial Comment:
Python 2.3a2 build process fails during building
_iconv_codec
module. The message is;

[...]
gcc -shared
build/temp.solaris-2.8-sun4u-2.3/_iconv_codec.o
-L/home/inyeol/util/python/lib -L/usr/local/lib -o
build/lib.solaris-2.8-sun4u-2.3/_iconv_codec.so
Traceback (most recent call last):

  [... long traceback here...]

  File "./setup.py", line 209, in build_extension
    imp.load_dynamic(ext.name, ext_filename)
RuntimeError: can't initialize the _iconv_codec module:
iconv_open() failed
*** Error code 1
make: Fatal error: Command failed for target
`sharedmods'

My build environment is gcc2.95.3 with Solaris 8
linker.
I've built 2.2, 2,2,1, 2,2,2, 2,3a1 with the same env
and there
was no (major) issues like this. I attached my
configure and
make log messages.


----------------------------------------------------------------------

>Comment By: Walter Dörwald (doerwalter)
Date: 2003-02-24 21:29

Message:
Logged In: YES 
user_id=89016

OK, I changed the test encoding from "ISO8859-1" to
"ISO-8859-1". But I tested this under Linux, and neither
"ASCII" nor "ISO8859-1" produced any problems for me.

----------------------------------------------------------------------

Comment By: Alejandro Lopez-Valencia (dradul)
Date: 2003-02-24 19:56

Message:
Logged In: YES 
user_id=659006

Ahh! I skimmed too fast through the code and didn't see that one! 
You see, GNU iconv is, errr, anal in the matter of ecoding 
names. And the author/maintainer is adamant on using stricly 
and only the names as specified in RFC1345, RFC1468, 
RFC1557 and related (I have a document on the subject I can 
mail you if you want). 

And... ISO8859-1 is not a valid name as per iconv_open(3) for 
GNU iconv's (any linux system has that man page). On the other 
hand, ISO-8859-1 is valid. That shoud fix the problem.


----------------------------------------------------------------------

Comment By: Walter Dörwald (doerwalter)
Date: 2003-02-24 16:10

Message:
Logged In: YES 
user_id=89016

These are just warnings, I see no error here. BTW the GNU
implementation does supports both ASCII and ISO8859-1 as
encodings, so I don't see why 1.12 should break anything

----------------------------------------------------------------------

Comment By: Alejandro Lopez-Valencia (dradul)
Date: 2003-02-23 13:25

Message:
Logged In: YES 
user_id=659006

The fix applied to _iconv_codec.c (in version 1.12) *breaks* the 
extension unnder cygwin and possibly other systems that use 
GNU iconv instead of a native library (namely, Solaris and BSD 
derivatives). The relevant errors:

building '_iconv_codec' extension
gcc -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -I. -
I/cvsroot/python/dist/src/./Include -I/usr/local/include -
I/tmp/Include -I/tmp/python -c /cvsroot/python/dist
/src/Modules/_iconv_codec.c -o build/temp.cygwin-1.3.20-i686-
2.3/_iconv_codec.o
cc1: warning: changing search order for system 
directory "/usr/local/include"
cc1: warning:   as it has already been specified as a non-system 
directory
/cvsroot/python/dist/src/Modules/_iconv_codec.c: In function 
`iconvcodec_encode'
:
/cvsroot/python/dist/src/Modules/_iconv_codec.c:147: warning: 
passing arg 2 of `
libiconv' from incompatible pointer type
/cvsroot/python/dist/src/Modules/_iconv_codec.c: In function 
`iconvcodec_decode'
:
/cvsroot/python/dist/src/Modules/_iconv_codec.c:357: warning: 
passing arg 2 of `
libiconv' from incompatible pointer type
/cvsroot/python/dist/src/Modules/_iconv_codec.c: In function 
`init_iconv_codec':

/cvsroot/python/dist/src/Modules/_iconv_codec.c:682: warning: 
passing arg 2 of `
libiconv' from incompatible pointer type
gcc -shared -Wl,--enable-auto-image-base build/temp.cygwin-
1.3.20-i686-2.3/_icon
v_codec.o -L/usr/local/lib -L. -liconv -lpython2.3 -o 
build/lib.cygwin-1.3.20-i6
86-2.3/_iconv_codec.dll
running build_scripts

----------------------------------------------------------------------

Comment By: Walter Dörwald (doerwalter)
Date: 2003-02-21 19:29

Message:
Logged In: YES 
user_id=89016

I changed the init function to use 'ISO8859-1' instead of
'ASCII' as the test encoding. Can someone test whether this
fixes anything?

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-20 23:15

Message:
Logged In: YES 
user_id=33168

In order to get the build to proceed, you can change line
210 of setup.py from:

        except ImportError, why:
to:
        except (ImportError, RuntimeError), why:

This is just before:
    self.announce('*** WARNING: renaming "%s" since
importing it' ...)

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690309&group_id=5470


From noreply@sourceforge.net  Mon Feb 24 21:01:13 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Mon, 24 Feb 2003 13:01:13 -0800
Subject: [Python-bugs-list] [ python-Bugs-532007 ] urllib ftp broken if Win32 proxy
Message-ID: <E18nPir-0004GJ-00@sc8-sf-web2.sourceforge.net>

Bugs item #532007, was opened at 2002-03-19 20:36
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=532007&group_id=5470

Category: Python Library
Group: Python 2.2
Status: Open
Resolution: None
Priority: 5
Submitted By: David Medberry (medberry)
Assigned to: Nobody/Anonymous (nobody)
Summary: urllib ftp broken if Win32 proxy

Initial Comment:
If Win32 has a proxy enabled, ftp urls will NOT work in
urllib.

Here is a trace:
Traceback (most recent call last):
  File
"C:\Python22\lib\site-packages\Pythonwin\pywin\framework\scriptutils.py",
line 301, in RunScript
    exec codeObject in __main__.__dict__
  File "C:\Python22\Lib\urllib.py", line 1457, in ?
    main()
  File "C:\Python22\Lib\urllib.py", line 1448, in main
    test(args)
  File "C:\Python22\Lib\urllib.py", line 1410, in test
    fn, h = urlretrieve(url, None, reporthook)
  File "C:\Python22\Lib\urllib.py", line 80, in urlretrieve
    return _urlopener.retrieve(url, filename,
reporthook, data)
  File "C:\Python22\Lib\urllib.py", line 210, in retrieve
    fp = self.open(url, data)
  File "C:\Python22\Lib\urllib.py", line 178, in open
    return getattr(self, name)(url)
  File "C:\Python22\Lib\urllib.py", line 438, in open_ftp
    host, path = splithost(url)
  File "C:\Python22\Lib\urllib.py", line 944, in splithost
    match = _hostprog.match(url)
TypeError: expected string or buffer

If the Win32 proxy is removed, it resumes working. In
urllib splithost, url is None or an empty tuple
(instead of the actual url.)

To reproduce, just enable a proxy on Win32 and execute
the "urllib.py -t" selftest. Disable the proxy and it
will be restored.

I haven't got the setup to check if web_proxy or
ftp_proxy on Posix machines causes the same problem.



----------------------------------------------------------------------

Comment By: Bram Moolenaar (vimboss)
Date: 2003-02-24 22:01

Message:
Logged In: YES 
user_id=57665

Problem is that open_ftp() does not check if the "url"
argument is a string or a tuple.  When ftp_proxy is set it
is a tuple, thus always causes the open() call to fail. 
This is not specific for Win32.

open_http() starts with this check:
       if type(url) is types.StringType:
open_ftp() should do something similar.

This is unrelated to bug 503031.


----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2002-04-01 01:42

Message:
Logged In: YES 
user_id=6380

Medberry, can you see if the patch from SF bug 503031
(http://www.python.org/sf/503031) solves your problem?

----------------------------------------------------------------------

Comment By: David Medberry (medberry)
Date: 2002-03-19 23:46

Message:
Logged In: YES 
user_id=95087

<bold>Is this bold?</bold>

----------------------------------------------------------------------

Comment By: David Medberry (medberry)
Date: 2002-03-19 23:44

Message:
Logged In: YES 
user_id=95087

This is _NOT_ a bug under Linux Python 2.2--it works just
fine there. (Makes me curious why it would behave
differently under Windows, but I'm not sure where to look yet.)

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=532007&group_id=5470


From noreply@sourceforge.net  Mon Feb 24 21:13:29 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Mon, 24 Feb 2003 13:13:29 -0800
Subject: [Python-bugs-list] [ python-Bugs-690309 ] 2.3a2 Sol8 make fails at _iconv_codec.
Message-ID: <E18nPuj-0001ag-00@sc8-sf-web1.sourceforge.net>

Bugs item #690309, was opened at 2003-02-20 16:53
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690309&group_id=5470

Category: Build
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Inyeol Lee (inyeol)
Assigned to: Nobody/Anonymous (nobody)
Summary: 2.3a2 Sol8 make fails at _iconv_codec.

Initial Comment:
Python 2.3a2 build process fails during building
_iconv_codec
module. The message is;

[...]
gcc -shared
build/temp.solaris-2.8-sun4u-2.3/_iconv_codec.o
-L/home/inyeol/util/python/lib -L/usr/local/lib -o
build/lib.solaris-2.8-sun4u-2.3/_iconv_codec.so
Traceback (most recent call last):

  [... long traceback here...]

  File "./setup.py", line 209, in build_extension
    imp.load_dynamic(ext.name, ext_filename)
RuntimeError: can't initialize the _iconv_codec module:
iconv_open() failed
*** Error code 1
make: Fatal error: Command failed for target
`sharedmods'

My build environment is gcc2.95.3 with Solaris 8
linker.
I've built 2.2, 2,2,1, 2,2,2, 2,3a1 with the same env
and there
was no (major) issues like this. I attached my
configure and
make log messages.


----------------------------------------------------------------------

Comment By: Alejandro Lopez-Valencia (dradul)
Date: 2003-02-24 16:13

Message:
Logged In: YES 
user_id=659006

Hmmm.... Perhaps the glibc maintainers have relaxed the naming 
rules when merging GNU iconv into the library.

I also changed ISO8859-1 to ISO-8859-1 in my local CVS copy 
and now _iconv_codec works fine under cygwin, soo I think this 
solved. 

Cheers

Al


----------------------------------------------------------------------

Comment By: Walter Dörwald (doerwalter)
Date: 2003-02-24 15:29

Message:
Logged In: YES 
user_id=89016

OK, I changed the test encoding from "ISO8859-1" to
"ISO-8859-1". But I tested this under Linux, and neither
"ASCII" nor "ISO8859-1" produced any problems for me.

----------------------------------------------------------------------

Comment By: Alejandro Lopez-Valencia (dradul)
Date: 2003-02-24 13:56

Message:
Logged In: YES 
user_id=659006

Ahh! I skimmed too fast through the code and didn't see that one! 
You see, GNU iconv is, errr, anal in the matter of ecoding 
names. And the author/maintainer is adamant on using stricly 
and only the names as specified in RFC1345, RFC1468, 
RFC1557 and related (I have a document on the subject I can 
mail you if you want). 

And... ISO8859-1 is not a valid name as per iconv_open(3) for 
GNU iconv's (any linux system has that man page). On the other 
hand, ISO-8859-1 is valid. That shoud fix the problem.


----------------------------------------------------------------------

Comment By: Walter Dörwald (doerwalter)
Date: 2003-02-24 10:10

Message:
Logged In: YES 
user_id=89016

These are just warnings, I see no error here. BTW the GNU
implementation does supports both ASCII and ISO8859-1 as
encodings, so I don't see why 1.12 should break anything

----------------------------------------------------------------------

Comment By: Alejandro Lopez-Valencia (dradul)
Date: 2003-02-23 07:25

Message:
Logged In: YES 
user_id=659006

The fix applied to _iconv_codec.c (in version 1.12) *breaks* the 
extension unnder cygwin and possibly other systems that use 
GNU iconv instead of a native library (namely, Solaris and BSD 
derivatives). The relevant errors:

building '_iconv_codec' extension
gcc -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -I. -
I/cvsroot/python/dist/src/./Include -I/usr/local/include -
I/tmp/Include -I/tmp/python -c /cvsroot/python/dist
/src/Modules/_iconv_codec.c -o build/temp.cygwin-1.3.20-i686-
2.3/_iconv_codec.o
cc1: warning: changing search order for system 
directory "/usr/local/include"
cc1: warning:   as it has already been specified as a non-system 
directory
/cvsroot/python/dist/src/Modules/_iconv_codec.c: In function 
`iconvcodec_encode'
:
/cvsroot/python/dist/src/Modules/_iconv_codec.c:147: warning: 
passing arg 2 of `
libiconv' from incompatible pointer type
/cvsroot/python/dist/src/Modules/_iconv_codec.c: In function 
`iconvcodec_decode'
:
/cvsroot/python/dist/src/Modules/_iconv_codec.c:357: warning: 
passing arg 2 of `
libiconv' from incompatible pointer type
/cvsroot/python/dist/src/Modules/_iconv_codec.c: In function 
`init_iconv_codec':

/cvsroot/python/dist/src/Modules/_iconv_codec.c:682: warning: 
passing arg 2 of `
libiconv' from incompatible pointer type
gcc -shared -Wl,--enable-auto-image-base build/temp.cygwin-
1.3.20-i686-2.3/_icon
v_codec.o -L/usr/local/lib -L. -liconv -lpython2.3 -o 
build/lib.cygwin-1.3.20-i6
86-2.3/_iconv_codec.dll
running build_scripts

----------------------------------------------------------------------

Comment By: Walter Dörwald (doerwalter)
Date: 2003-02-21 13:29

Message:
Logged In: YES 
user_id=89016

I changed the init function to use 'ISO8859-1' instead of
'ASCII' as the test encoding. Can someone test whether this
fixes anything?

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-20 17:15

Message:
Logged In: YES 
user_id=33168

In order to get the build to proceed, you can change line
210 of setup.py from:

        except ImportError, why:
to:
        except (ImportError, RuntimeError), why:

This is just before:
    self.announce('*** WARNING: renaming "%s" since
importing it' ...)

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690309&group_id=5470


From noreply@sourceforge.net  Mon Feb 24 22:30:35 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Mon, 24 Feb 2003 14:30:35 -0800
Subject: [Python-bugs-list] [ python-Bugs-676342 ] after using pdb readline does not work correctly
Message-ID: <E18nR7L-0004d3-00@sc8-sf-web3.sourceforge.net>

Bugs item #676342, was opened at 2003-01-28 15:53
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=676342&group_id=5470

Category: Python Library
Group: Python 2.2.2
Status: Open
Resolution: Accepted
Priority: 5
Submitted By: Michael Stone (mbrierst)
>Assigned to: Neal Norwitz (nnorwitz)
Summary: after using pdb readline does not work correctly

Initial Comment:

After I use pdb in the interpreter my readline settings are messed up.  It only knows about the pdb readline stuff forever afterward.  In fact, this happens as soon as a Pdb object is instantiated, regardless of whether or not you use it.

This is a result of some aspects of the Cmd object in Lib/cmd.py.

Currently Cmd registers a new readline completer as soon as a Cmd object is instantiated.  This is probably incorrect.  I believe the correct place to register the new handler is in the preloop hook.  That way the new readline completer is only used while the Cmd object is actually responsible for processing user input.

Next, the old readline completer should probably be re-registered in the postloop hook.  In order to do this, a new call must be added to the readline module to return the current completer so it can be saved and later restored.

The files included patch readline.c (version 2.41.64) and cmd.py (version 1.26.16.2) to make the changes described above.  After this patch readline works as expected in the interpreter.

Let me know what you think of it.

----------------------------------------------------------------------

>Comment By: Raymond Hettinger (rhettinger)
Date: 2003-02-24 17:30

Message:
Logged In: YES 
user_id=80475

Assigning to Neal because I can't test this on my Windows 
machine.

----------------------------------------------------------------------

Comment By: Michael Stone (mbrierst)
Date: 2003-02-21 12:52

Message:
Logged In: YES 
user_id=670441

I'm attaching backported versions against
cmd.py 1.26.16.3
readline.c 2.41.6.4
if you want them.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-21 07:42

Message:
Logged In: YES 
user_id=6380

Good idea.

----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2003-02-21 02:42

Message:
Logged In: YES 
user_id=80475

Backport?

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2003-02-03 06:11

Message:
Logged In: YES 
user_id=6656

Checked in as 

Lib/cmd.py revision 1.34

Thanks!  You probably didn't need to attach the patch three
times, though :-)

----------------------------------------------------------------------

Comment By: Michael Stone (mbrierst)
Date: 2003-02-01 01:41

Message:
Logged In: YES 
user_id=670441

I started using the latest cvs version today.
I have attached a diff to the latest cvs as patchcmd2 if that's
more convenient for you to apply.

----------------------------------------------------------------------

Comment By: Michael Stone (mbrierst)
Date: 2003-02-01 01:37

Message:
Logged In: YES 
user_id=670441

I started using the latest cvs version today.
I have attached a diff to the latest cvs as patchcmd2 if that's
more convenient for you to apply.

----------------------------------------------------------------------

Comment By: Michael Stone (mbrierst)
Date: 2003-01-31 16:25

Message:
Logged In: YES 
user_id=670441

I started using the latest cvs version today.
I have attached a diff to the latest cvs as patchcmd2 if that's
more convenient for you to apply.

----------------------------------------------------------------------

Comment By: Michael Stone (mbrierst)
Date: 2003-01-30 11:59

Message:
Logged In: YES 
user_id=670441

That sounds good to me.  Go ahead and check in patchcmd.

Thanks for your suggestions, I will follow them in future patches.  In this one, I was trying to follow the conventions I saw being used in the module, but they were outdated or incorrect (and I didn't always follow them correctly anyway!).  Okay that's definitely enough commentary for a few line patch.

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2003-01-30 05:22

Message:
Logged In: YES 
user_id=6656

Thanks for the new patch, it looks OK.  There are a couple
of things that spring to mind for future patches: 

- the function should be METH_NOARGS (or you should check
the args tuple)

- there should be a blank line between the argspec line in
the docstring and the description

- you should use the PyDoc_STRVAR macro (but this was only
used in the CVS trunk's version of readline).

I made these changes and checked in revision 2.59 of
Modules/readline.c.

Wrt the cmd.py changes, I think I'll probably let that
sleeping dog lie.  I agree with you in general: cmd.py
probably shouldn't be rebinding any keys, but

1) the interface is "specify the completion key to get
completion",   so backwards compatibility raises its head

2) as you say, most people do probably rebind tab anyway

so I propose to check in your patchcmd, close the bug and
wait for someone to submit a bug report about pdb messing up
their bindings.

----------------------------------------------------------------------

Comment By: Michael Stone (mbrierst)
Date: 2003-01-29 13:19

Message:
Logged In: YES 
user_id=670441

Actually, the readline library does have the functionality to get the current binding of the tab key, so it could be saved and restored.  However, to do this in python some stuff would have to be added to the readline.c module, to create some python layer that deals with the way readline stores macros, functions, and keymaps.  It's probably not worth the effort unless someone wants to do major changes to the readline module to add the full functionality of the underlying library to the python interface. 

Probably cmd.py shouldn't be rebinding the tab key to complete at all.  If the user wants that, won't they have already done it somewhere else, like in their pythonrc or inputrc files?  And if they don't it might be impolite to change their bindings.

----------------------------------------------------------------------

Comment By: Michael Stone (mbrierst)
Date: 2003-01-29 10:27

Message:
Logged In: YES 
user_id=670441

I attached a changed readline.c patch to hopefully get rid of any "sickness".
(There was a not-checking-for-null problem too... ouch)

I think it's good now.  But note, there may not be a way to do this sort of thing perfectly.  If you mess around enough with starting debuggers within debuggers (or in general anything that uses a Cmd object) it might be possible to end up with an unexpected completer function in the end.

And it looks like under the current system there's no way to get back the completer key that was initially used.  But this isn't SUCH a big deal since I think everyone uses tab, right?

At any rate, the change is definitely an improvement for me.

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2003-01-29 05:36

Message:
Logged In: YES 
user_id=6656

I approve of this patch in principle.  I hadn't done enough
investigation to see that it was so easy to get the current
completer function!

OTOH:

return Py_BuildValue("O", completer);

is sick.  Change that to

Py_INCREF(completer);
return completer;

and if noone else howls, I'll check it in.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=676342&group_id=5470


From noreply@sourceforge.net  Tue Feb 25 03:35:59 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Mon, 24 Feb 2003 19:35:59 -0800
Subject: [Python-bugs-list] [ python-Bugs-216289 ] Programs using Tkinter sometimes can't shut down (Windows)
Message-ID: <E18nVst-0006Gc-00@sc8-sf-web4.sourceforge.net>

Bugs item #216289, was opened at 2000-10-06 22:25
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=216289&group_id=5470

Category: Windows
Group: Python 2.3
Status: Open
Resolution: Later
Priority: 3
Submitted By: Tim Peters (tim_one)
Assigned to: Tim Peters (tim_one)
Summary: Programs using Tkinter sometimes can't shut down (Windows)

Initial Comment:
The following msg from the Tutor list is about 1.6, but I noticed the same thing several times today using 2.0b2+CVS.  In my case, I was running IDLE via

python ../tool/idle/idle.pyw

from a DOS box in my PCbuild directory.  Win98SE.  *Most* of the time, shutting down IDLE via Ctrl+Q left the DOS box hanging.  As with the poster, the only way to regain control was to use the Task Manager to kill off Winoldap.

-----Original Message-----
From: Joseph Stubenrauch <nothingisgoingtochangemyworld@yahoo.com>
Sent: Friday, October 06, 2000 9:23 PM
To: tutor@python.org
Subject: Re: [Tutor] Python 1.6 BUG


Strange, I have been experiencing the same bug myself.
Here's the low down for me:

Python 1.6 with win95
I am running a little Tkinter program
The command line I use is simply: "python foo.py"
About 25-35% of the time, when I close the Tkinter
window, DOS seems to "freeze" and never returns to the
c:\ command prompt.  I have to ctrl-alt-delete
repeatedly and shut down "winoldapp" to get rid of the
window and then shell back into DOS and keep working.
It's a bit of a pain, since I have the habit of
testing EVERYTHING in tiny little stages, so I change
one little thing, test it ... freeze ... ARGH!  Change
one more tiny thing, test it ... freeze ... ARGH! 
However, sometimes it seems to behave and doesn't
bother me for an entire several hour session of python
work.

That's my report on the problem.

Cheers,

Joe


----------------------------------------------------------------------

>Comment By: Tim Peters (tim_one)
Date: 2003-02-24 22:35

Message:
Logged In: YES 
user_id=31435

I just tried, and saw the original symptom on the 3rd try, 
with Python 2.3a2 on Win98SE.  Then reproduced it twice 
after two reboots.  That installs Tcl/Tk 8.4.1.  They were 
not built with OPTS=threads.  I don't know whether 
OPTS=threads would cure it.

----------------------------------------------------------------------

Comment By: Jeffrey Hobbs (hobbs)
Date: 2003-02-24 12:04

Message:
Logged In: YES 
user_id=72656

This was against an older version (8.3.2 or 8.3.3).  Make sure 
any verification occurs against the HEAD or 8.3.5.

Also, Martin provided a threaded Tkinter implmentation, IIUC, 
so that makes this doubly out-of-date.

----------------------------------------------------------------------

Comment By: Donal K. Fellows (dkf)
Date: 2003-02-24 08:33

Message:
Logged In: YES 
user_id=79902

Exactly which version (including patchlevel) of tcl.dll and
tk.dll is this bug reported against?  (Well, which is the
most recent version...)  Is it one which the Tcl Maintainers
believe to have all identified relevant bugs in the area
resolved, or does it predate that?

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-24 07:59

Message:
Logged In: YES 
user_id=6380

Um, I think it's *not* closed, but I'll assign it back to
Tim. I believe Tim said about this somewhere "the Tcl folks
have often thought they'd nailed it, but they still
haven't". Unless that was an entirely different problem. But
it used to happen for me on Win98 too (haven't tried in a
while).

----------------------------------------------------------------------

Comment By: Fredrik Lundh (effbot)
Date: 2003-02-24 02:13

Message:
Logged In: YES 
user_id=38376

(a bugfix a day keeps etc)

Tim, can we close this one?  I don't have access to any-
thing even remotely similar to a Win95-box any longer...

----------------------------------------------------------------------

Comment By: Jeffrey Hobbs (hobbs)
Date: 2002-12-09 23:16

Message:
Logged In: YES 
user_id=72656

Interesting last couple of comments ...

Anyway, I think the cause of this problem may have finally 
been identified by Tcl bug 651139.  I have attached a patch 
here for it against 8.3.5.  Both the 8.3.5 branch and 8.4.1+ 
head have this fix.  Amazingly it didn't effect Tcl, but could 
likely have caught Tkinter (only on Windows with non-
threaded builds).

Oh look, not allowed to attach a patch, so I'll attach it to Tcl 
bug 651139.

----------------------------------------------------------------------

Comment By: christian soldan (elacademico)
Date: 2002-11-30 21:07

Message:
Logged In: YES 
user_id=659858

how do i hack my fuckers friends

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2002-09-30 19:23

Message:
Logged In: NO 

i would like to know some moves

----------------------------------------------------------------------

Comment By: John Popplewell (johnnypops)
Date: 2002-08-21 20:08

Message:
Logged In: YES 
user_id=143340

Hi,

tried wrapping the WaitForSingleObject() with if (consolePtr-
>toWrite) { ... as suggested. Solves the problem for me in 
debug and release builds (without thread support).

What is "the tcl83.dll from www.tcl.tk", what changes does 
it include?

cheers,
John.


----------------------------------------------------------------------

Comment By: Doug Henderson (djhender)
Date: 2002-08-16 19:03

Message:
Logged In: YES 
user_id=148444

I downloaded and tested the tcl83.dll from www.tcl.tk with 
Python 2.2.1 on Win98SE using my BugDemo.py script 
from a related bug report [231207]. Several test runs 
showed that a problem still exists. The app hangs until 
Winopldapp is killed, however this leaves tcl83.dll in use. 
The problem does not occur on every execution. Some 
types of activity seem to provoke the hang, for instance, 
minimizing and maximizing the window seems to increase 
the change that it will hang on exit.

----------------------------------------------------------------------

Comment By: David Gravereaux (davygrvy)
Date: 2002-08-16 17:53

Message:
Logged In: YES 
user_id=7549

Whoop, error in the patch..  consoleMutex might not need to 
be locked.  It could cause a new dead-lock.

----------------------------------------------------------------------

Comment By: David Gravereaux (davygrvy)
Date: 2002-08-16 17:41

Message:
Logged In: YES 
user_id=7549

>It is hanging up in ConsoleCloseProc() in 'tclWinConsole.c'
>at about line 527:
>WaitForSingleObject(consolePtr->writeThread, INFINITE);

Jeff, this might need the same attention the pipe reader 
thread got for that [exec] memory leak bug.  I think 
ConsoleWriterThread() should use WaitForMultipleObjects 
instead of WaitForSingleObject with a second event handle 
used to single a fall-out like the pipe reader thread does.  It 
appears that ConsoleCloseProc() has no way to signal the 
writer thread to close at all.

This doesn't address the blocking WriteFile() in 
ConsoleWriterThread(), though.  Is it blocking there, too?
Sorry for the large post.  I'm given the option to attach a file.

Index: tclWinConsole.c
=============================================
======================
RCS file: /cvsroot/tcl/tcl/win/tclWinConsole.c,v
retrieving revision 1.7
diff -c -r1.7 tclWinConsole.c
*** tclWinConsole.c	15 Jan 2002 17:55:30 -0000	1.7
--- tclWinConsole.c	16 Aug 2002 21:31:56 -0000
***************
*** 79,84 ****
--- 79,87 ----
      HANDLE startWriter;		/* Auto-reset 
event used by the main thread to
  				 * signal when 
the writer thread should attempt
  				 * to write to the 
console. */
+     HANDLE stopWriter;		/* Auto-reset 
event used by the main thread to
+ 				 * signal when 
the writer thread should
+ 				 * terminate. */
      HANDLE startReader;		/* Auto-reset 
event used by the main thread to
  				 * signal when 
the reader thread should attempt
  				 * to read from 
the console. */
***************
*** 516,522 ****
  	 */
  
  	Tcl_MutexLock(&consoleMutex);
! 	TerminateThread(consolePtr->writeThread, 0);
  
  	/*
  	 * Wait for the thread to terminate.  This ensures 
that we are
--- 519,525 ----
  	 */
  
  	Tcl_MutexLock(&consoleMutex);
! 	SetEvent(consolePtr->stopWriter);
  
  	/*
  	 * Wait for the thread to terminate.  This ensures 
that we are
***************
*** 1134,1149 ****
  {
  
      ConsoleInfo *infoPtr = (ConsoleInfo *)arg;
      HANDLE *handle = infoPtr->handle;
      DWORD count, toWrite;
      char *buf;
  
      for (;;) {
  	/*
  	 * Wait for the main thread to signal before 
attempting to write.
  	 */
  
! 	WaitForSingleObject(infoPtr->startWriter, 
INFINITE);
  
  	buf = infoPtr->writeBuf;
  	toWrite = infoPtr->toWrite;
--- 1137,1163 ----
  {
  
      ConsoleInfo *infoPtr = (ConsoleInfo *)arg;
+     HANDLE events[2];
      HANDLE *handle = infoPtr->handle;
      DWORD count, toWrite;
      char *buf;
  
+     events[0] = infoPtr->stopWriter;
+     events[1] = infoPtr->startWriter;
+ 
      for (;;) {
+ 	DWORD dwWait;
+ 
  	/*
  	 * Wait for the main thread to signal before 
attempting to write.
  	 */
  
! 	dwWait = WaitForMultipleObjects(events, 2, 
FALSE, INFINITE);
! 
! 	if (dwWait != (WAIT_OBJECT_0 + 1)) {
! 	    /* either the stop event or some other error, so 
exit */
! 	    return 0;
! 	}
  
  	buf = infoPtr->writeBuf;
  	toWrite = infoPtr->toWrite;
***************
*** 1251,1256 ****
--- 1265,1271 ----
      if (permissions & TCL_WRITABLE) {
  	infoPtr->writable = CreateEvent(NULL, TRUE, 
TRUE, NULL);
  	infoPtr->startWriter = CreateEvent(NULL, FALSE, 
FALSE, NULL);
+ 	infoPtr->stopWriter = CreateEvent(NULL, FALSE, 
FALSE, NULL);
  	infoPtr->writeThread = CreateThread(NULL, 8000, 
ConsoleWriterThread,
  	        infoPtr, 0, &id);
      }


----------------------------------------------------------------------

Comment By: Jeffrey Hobbs (hobbs)
Date: 2002-08-16 14:56

Message:
Logged In: YES 
user_id=72656

I'm finding it a bit hard to reproduce, and now not at all.  
Could people please try replacing their current tcl83.dll with 
the one at http://www.tcl.tk/tcl83.dll.  The sig on it is:
634880 Aug 16 10:53 tcl83.dll
(md5) b35628568ddc4afed4f675d2d9a50faf  tcl83.dll

I can't hang anymore with python 2.2.1 at the Win98 
command prompt with:
   python \Python22\Tools\idle\idle.pyw

----------------------------------------------------------------------

Comment By: Jeffrey Hobbs (hobbs)
Date: 2002-08-16 13:50

Message:
Logged In: YES 
user_id=72656

johnny - That actually helps a lot and fairly clearly indicates 
the source of this hanging problem.  To give more context to 
that area, this is in the final shutdown code where it is 
(obviously) shutting down the console I/O.  The reader is 
shutdown without a check for more input (why, we are 
exiting?), so there is no issue there.

The writer however wants to make sure that the console 
output channel is drained before exit.  A blocking channel 
that is not being flushed will cause a hang.

So the question is, what is the stdio setup for Tcl in 
Py/Tkinter?  That should be examined to see whether stdio is 
clear to flush on exit.

Perhaps you can also help with more debugging.  Could you 
wrap the problematic WaitForSingleObject call with
if (consolePtr->toWrite) { ...
and perhaps also print out what to write.  It may be that 
blocking output isn't the problem at all, but rather that we are 
never receiving the signal on consolePtr->writable (usually 
done in the writer thread).  The consolePtr->toWrite check 
will at least ensure flushed channels do not enter this 
potential hanging state.

----------------------------------------------------------------------

Comment By: John Popplewell (johnnypops)
Date: 2002-08-16 10:16

Message:
Logged In: YES 
user_id=143340

I have dug out the setup I used previously and, apart from 
using Python 2.2 instead of 2.0.1, have had a go at 
reproducing my previous results.

With a debug build of Tcl/Tk and threads disabled I have 
managed to get this stack trace when it locks on exit (after 
doing a break):

ConsoleCloseProc(int * 0x006a0ae4, Tcl_Interp * 
0x00000000) line 527 + 17 bytes
TclFinalizeIOSubsystem() line 241 + 20 bytes
Tcl_FinalizeThread() line 911
Tcl_Finalize() line 812
DllMain(HINSTANCE__ * 0x00970000, unsigned long 
0x00000000, void * 0x00000001) line 197
TCL83! _DllMainCRTStartup@12 + 80 bytes
KERNEL32! bff7ddd6()
KERNEL32! bff8d123()
8176b5fc()

It is hanging up in ConsoleCloseProc() in 'tclWinConsole.c'
at about line 527:
WaitForSingleObject(consolePtr->writeThread, INFINITE);

I tried changing the timeout from INFINITE to 100ms and 
testing the return value for WAIT_TIMEOUT. Sure enough it 
occasionaly printed a message (and didn't lock-up).

Don't know if this helps. Will try to reproduce the double 
free problem in the threaded build later,

cheers,
John.


----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2002-08-15 10:19

Message:
Logged In: YES 
user_id=6380

Just a note so I won't forget this: Jeff Hobbs once said the
code to look at is

tk/generic/tkConsole.c:Tk_InitConsoleChannels

----------------------------------------------------------------------

Comment By: David Gravereaux (davygrvy)
Date: 2002-08-15 09:23

Message:
Logged In: YES 
user_id=7549

Hi John,

I have seen Tcl extension DLLs that set exit handlers get 
called by Tcl after they were torn-down.  NT and Win9x seem 
to be a bit different in the order of what DLLs get torn-down.  
here's a note from some old code of mine:

#ifdef __WIN32__
    // It can happen that the HEAP could have already been 
unloaded
    // from an awkward teardown caused by a Ctrl+C or other.  
Win32
    // seems to do a teardown in reverse order and by the time 
Tcl
    // knows what's going on and Tcl_Finalize calls the exit
    // handlers, this extension's data (heap?) has already been
    // unloaded by the OS.  Do a quick legal check on the 
pointer
    // first.
    //
    if (IsBadReadPtr(adapt, sizeof(T *))) return;
#endif

what's the back/stacktrace?  who is calling 
Tcl_FinalizeNotifier?  And had anyone called it before during 
the shutdown phase? I can't say with much confidence that 
this is happening here, too.  Maybe..  Could Tk be telling Tcl 
shutdown?  That's sort of backwards, if so.

I run w2k here, so I can't help in debugging from a test case.  
tclsh with a threaded build doesn't even call Tcl_Finalize to 
avoid all this!

----------------------------------------------------------------------

Comment By: Donal K. Fellows (dkf)
Date: 2002-08-15 09:09

Message:
Logged In: YES 
user_id=79902

Firstly, on the matter of threads.  Tcl/Tk on Windows uses
threads internally even in non-threaded builds.  That's
Windows for you (in particular, you can get calls parachuted
in from completely separate threads with relatively little
warning.  Also, you need threaded helpers to handle
non-blocking waiting on some kinds of objects.)

It's a bit hard to work out if the problem's been fixed
without a stack-trace (preferably with debugging symbols
enabled so that we can see what the function names and
arguments to those functions are.)  I have no  chance of
replicating this bug here (I'm currently running IRIX, but
also have some access to Solaris and Linux...)

----------------------------------------------------------------------

Comment By: John Popplewell (johnnypops)
Date: 2002-08-15 08:15

Message:
Logged In: YES 
user_id=143340

Hi to davygrvy,

my fix isn't for a thread related (deadlock) type problem - it 
is a fix for a double free type problem which is presumably 
screwing up the heap and causing the C runtime to hang.

It *is* with a threaded build (THREADDEFINES = -
DTCL_THREADS=1) which I think I enabled because I was 
using  'after' and it didn't work unless threading was 
enabled - but I may be mis-remembering here!

Also, I ended up doing all this with the debug build and 
found the -GZ flag useful - in fact it pointed out the fact that 
there was a problem with the heap.

It is worth noting that I have only seen this on Win98 and 
that using 'pythonw.exe' did *not* prevent this from 
happening,

regards John.


----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2002-08-14 21:00

Message:
Logged In: YES 
user_id=6380

AFAIK we never distributed a threaded build.

----------------------------------------------------------------------

Comment By: David Gravereaux (davygrvy)
Date: 2002-08-14 20:01

Message:
Logged In: YES 
user_id=7549

To johnnypops,

Where in Tcl_FinalizeNotifier() is the dead-lock occuring and 
is this with a threaded build of Tcl?


----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2002-07-02 15:39

Message:
Logged In: YES 
user_id=31435

Jeff, a long time ago Fredrik Lundh said he was able to 
reproduce this with a short wish script (no Python, no Tkinter, 
just Tcl/Tk + wish).  I'm reassigning this to /F in the hopes 
he'll remember how.

----------------------------------------------------------------------

Comment By: Jeffrey Hobbs (hobbs)
Date: 2002-07-02 14:55

Message:
Logged In: YES 
user_id=72656

I appreciate that many others see this is a problem in 
embedding Tcl, but without any clear way to reproduce this in 
Tcl/Tk itself, it's very hard for me to fix it.  I don't think that 
there is anything done in 8.4 to fix this that I recall, but if 
someone finds that the problem "goes away" with 8.4, please 
report that.

For those of course that experience this with python, the 
simple and "correct" work-around is to launch with 
pythonw.exe instead of python.exe.

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2002-06-26 13:40

Message:
Logged In: NO 

According to http://tcl.activestate.com ActiveTcl 8.4.0 is now 
available for download.  It is new as of June 24.  I don't know 
whether this will solve our problems or not, but it's worth 
looking into.

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2002-06-19 09:36

Message:
Logged In: NO 

I just downloaded Ruby 1.6.6 for Windows, which also uses 
Tcl/Tk 8.3 and I experience exactly the same problem there. 
So this is definitely a Tcl problem (but we know that already).

----------------------------------------------------------------------

Comment By: John Popplewell (johnnypops)
Date: 2002-06-07 15:01

Message:
Logged In: YES 
user_id=143340

Hmmm. 
I run a Python Tcl/Tk application several times every day - 
the very application that prompted my bug-hunt.

I do use "pythonw.exe" to launch it, but it used to hang 
almost as badly as "python.exe".

Whilst developing this app I used "python.exe" and 
experienced no hang-ups.

The Tcl guys point out that this bug is only one of several 
possible causes of the hang-up, maybe you are just out of 
luck.

The patch has fixed the problem on three Win98 machines, 
all of which exhibited the problem before updating the DLLs.

One thought - are you sure there weren't multiple copies of 
the tcl83.dll and tk83.dll files? My (only) copies were 
in "D:\python20\DLLs"

good luck,
John.


----------------------------------------------------------------------

Comment By: Jeffrey Hobbs (hobbs)
Date: 2002-06-03 14:55

Message:
Logged In: YES 
user_id=72656

providing Tim with a fixed DLL exactly according to John 
Popplewell's patch did not fix the problem for him.  That 
makes it seem fishy that John really didn't see the problem 
any more.

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2002-06-03 12:59

Message:
Logged In: YES 
user_id=31435

I tried replacing Python 2.2.1's tk83.dll and tcl83.dll (from 
Tcl/Tk 8.3.2) with newer matched sets from

A) PythonWare's py21 distribution (Tcl/Tk 8.3.3).

and again from

B) ActiveState's Tcl/Tk 8.3.4.

All the symptoms originally reported here persisted on 
Win98SE despite that.

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2002-05-22 09:57

Message:
Logged In: NO 

I am new to Python and encountered this bug on my very first 
Tkinter application.  It persists in Python 2.2.1 under 
Windows Me. Needless to say, it makes a bad first 
impression.  Basically, this thread says that Tkinter has been 
broken for a year and a half on a popular platform at a time 
when Python is trying to expand its user base.  Hopefully, this 
bug will be corrected soon.

If Tcl/Tk 8.4 is not released shortly you might try repackaging 
Tkinter for Win32 with an earlier version of Tcl/Tk, such as 
8.0, that doesn't encounter this problem.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2002-03-22 22:36

Message:
Logged In: YES 
user_id=6380

The proper action is to wait until Tcl 8.3.5 is released,
and then shaming someone else in providing you with a set of
Windowsbinaries for Tcl. You may also ask Hobbs if there's a
Windows project file to build Tcl/Tk for Windows.

----------------------------------------------------------------------

Comment By: Jeffrey Hobbs (hobbs)
Date: 2002-03-11 21:31

Message:
Logged In: YES 
user_id=72656

I did end up back-porting this to the 8.3.4+ Tcl CVS on 
2002-02-25, which means it will be in an eventual 8.3.5.  
It is already in the release 8.4 alpha series.

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2002-03-09 20:50

Message:
Logged In: YES 
user_id=31435

Back to Guido:  Do you think you know what the proper 
action is?  (You said that's why you reopened it, and 
there's been no new info here since then.)

----------------------------------------------------------------------

Comment By: John Popplewell (johnnypops)
Date: 2002-02-25 19:55

Message:
Logged In: YES 
user_id=143340

I knew I wasn't getting to the heart of it ....

Almost a one-liner!

It has been seriously spoiling my (otherwise crash free) 
Python experience on Win9x - hope it gets fixed soon.

cheers,
John.


----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2002-02-25 18:39

Message:
Logged In: YES 
user_id=6380

Reopened until we know what the proper action is.

----------------------------------------------------------------------

Comment By: Jeffrey Hobbs (hobbs)
Date: 2002-02-25 18:32

Message:
Logged In: YES 
user_id=72656

This is mostly correct, and is fixed in the 8.4 Tcl sources 
already (I guess we can backport this).  This was mentioned 
in SF Tcl bug (account for chopped URL):

https://sourceforge.net/tracker/?
func=detail&aid=217982&group_id=10894&atid=110894

and the code comment is:

    /*
     * Only finalize the notifier if a notifier was 
installed in the
     * current thread; there is a route in which this is not
     * guaranteed to be true (when tclWin32Dll.c:DllMain() 
is called
     * with the flag DLL_PROCESS_DETACH by the OS, which 
could be
     * doing so from a thread that's never previously been 
involved
     * with Tcl, e.g. the task manager) so this check is 
important.
     *
     * Fixes Bug #217982 reported by Hugh Vu and Gene 
Leache.
     */
    if (tsdPtr == NULL) {
	return;
    }


----------------------------------------------------------------------

Comment By: John Popplewell (johnnypops)
Date: 2002-02-25 17:41

Message:
Logged In: YES 
user_id=143340

This one has been torturing me for a while.

Managed to track it down to a problem inside Tcl.
For the Tcl8.3.4 source distribution the problem is in
the file win/tclWinNotify.c 

void Tcl_FinalizeNotifier(ClientData clientData)
{
    ThreadSpecificData *tsdPtr = 
                         (ThreadSpecificData *) clientData;

    /* sometimes called with tsdPtr == NULL */
    if ( tsdPtr != NULL )
    {
        DeleteCriticalSection(&tsdPtr->crit);
        CloseHandle(tsdPtr->event);

        /*
         * Clean up the timer and messaging 
         * window for this thread.
         */

        if (tsdPtr->hwnd) {
	    KillTimer(tsdPtr->hwnd, INTERVAL_TIMER);
            DestroyWindow(tsdPtr->hwnd);
        }
    }

    /*
     * If this is the last thread to use the notifier, 
     * unregister the notifier window class.
     */

    Tcl_MutexLock(&notifierMutex);
    if ( notifierCount && !--notifierCount ) {
	UnregisterClassA( "TclNotifier", 
                          TclWinGetTclInstance());
    }
    Tcl_MutexUnlock(&notifierMutex);
}

This bodge doesn't address the underlying problem but has 
stopped me from tearing all my hair out,

cheers,
John Popplewell.


----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2001-10-24 18:27

Message:
Logged In: YES 
user_id=31435

FYI, you don't need an IDE to do this -- in Win9x, hit 
Ctrl+Alt+Del and kill the process directly.

A saner <wink> solution is to develop under Win2K, which 
doesn't appear to suffer this problem (the only reports 
I've seen, and experienced myself, came from Win9x boxes).

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2001-10-24 04:52

Message:
Logged In: NO 

For those who are still trapped in this bug's hell, I will 
gladly share the one thing that saved my sanity:  
WingIDE's 'Kill' command will shut down the program with 
apparent 100% certainty and no fear of lockups.  WingIDE 
has its own issues, so its not a perfect solution, but if 
you are like me and Joe (above) who test in small 
iterations, then using 'Kill' to quit out of your app while 
testing is a workaround that may preserve your sanity.

Perhaps the python gods and the Wing guys can get together 
and tell us how to replicate 'kill' into our code.  For 
now, I'll use WingIDE to edit, and pythonw.exe for my final 
client's delivery.

----------------------------------------------------------------------

Comment By: Howard Lightstone (hlightstone)
Date: 2001-09-05 13:43

Message:
Logged In: YES 
user_id=66570

I sometimes get bunches of these....

A tool I use (Taskinfo2000) reports that (after killing 
winoldap):

python.exe is blocked on a mutex named OLESCELOCKMUTEX.
The reported state is "Console Terminating".  There appears 
to be only one (os) thread running.


----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2001-04-02 16:06

Message:
Logged In: YES 
user_id=31435

No sign of progress on this presumed Tk/Tcl Windows bug in 
over 3 months, so closing it for lack of hope.

----------------------------------------------------------------------

Comment By: Doug Henderson (djhender)
Date: 2001-02-06 00:13

Message:
This was a symptom I saw while tracking down the essence of the problem reported in #131207.  Using Win98SE, I would get an error dialog (GPF?) in the Kernel, and sometimes the dos prompt would not come back.

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2000-12-12 21:00

Message:
Just reproduced w/ current CVS, but didn't hang until the 8th try.

http://dev.scriptics.com/software/tcltk/ says 8.3 is still the latest released version; don't know whether that URL still makes sense, though.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2000-12-12 15:58

Message:
Tim, can you still reproduce this with the current CVS version?

There's been one critical patch to _tkinter since the 2.0 release.

An alternative would be to try with a newer version of Tcl (isn't 8.4 out already?).


----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2000-10-15 12:47

Message:
Same as I've reported earlier; it hangs in the call to
Tcl_Finalize (which is called by the DLL finalization
code).  It's less likely to hang if I call Tcl_Finalize
from the _tkinter DLL (from user code).

Note that the problem isn't really Python-related -- I
have stand-alone samples (based on wish) that hangs in
the same way.

More later.

</F>

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2000-10-13 10:40

Message:
Back to Tim since I have no clue what to do here.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2000-10-12 13:25

Message:
The recent fix to _tkinter (Tcl_GetStringResult(interp) instead of interp->result) didn't fix this either.

As Tim has remarked in private but not yet recorded here, a workaround is to use pythonw instead of python, so I'm lowering thepriority again.

Also note that the hanging process that Tim writes about apparently prevents Win98 from shutting down properly.

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2000-10-07 03:37

Message:
More info (none good, but some worse so boosted priority):

+ Happens under release and debug builds.

+ Have not been able to provoke when starting in the debugger.

+ Ctrl+Alt+Del and killing Winoldap is not enough to clean everything up.  There's still a Python (or Python_d) process hanging around that Ctrl+Alt+Del doesn't show.

+ This process makes it impossible to delete the associated Python .dll, and in particular makes it impossible to rebuild Python successfully without a reboot.

+ These processes cannot be killed!  Wintop and Process Viewer both fail to get the job done.  PrcView (a freeware process viewer) itself locks up if I try to kill them using it.  Process Viewer freezes for several seconds before giving up.

+ Attempting to attach to the process with the MSVC debugger (in order to find out what the heck it's doing) finds the process OK, but then yields the cryptic and undocumented error msg "Cannot execute program".

+ The processes are not accumulating cycles.

+ Smells like deadlock.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=216289&group_id=5470


From noreply@sourceforge.net  Tue Feb 25 03:49:31 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Mon, 24 Feb 2003 19:49:31 -0800
Subject: [Python-bugs-list] [ python-Bugs-216289 ] Programs using Tkinter sometimes can't shut down (Windows)
Message-ID: <E18nW5z-00081n-00@sc8-sf-web1.sourceforge.net>

Bugs item #216289, was opened at 2000-10-06 19:25
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=216289&group_id=5470

Category: Windows
Group: Python 2.3
Status: Open
Resolution: Later
Priority: 3
Submitted By: Tim Peters (tim_one)
Assigned to: Tim Peters (tim_one)
Summary: Programs using Tkinter sometimes can't shut down (Windows)

Initial Comment:
The following msg from the Tutor list is about 1.6, but I noticed the same thing several times today using 2.0b2+CVS.  In my case, I was running IDLE via

python ../tool/idle/idle.pyw

from a DOS box in my PCbuild directory.  Win98SE.  *Most* of the time, shutting down IDLE via Ctrl+Q left the DOS box hanging.  As with the poster, the only way to regain control was to use the Task Manager to kill off Winoldap.

-----Original Message-----
From: Joseph Stubenrauch <nothingisgoingtochangemyworld@yahoo.com>
Sent: Friday, October 06, 2000 9:23 PM
To: tutor@python.org
Subject: Re: [Tutor] Python 1.6 BUG


Strange, I have been experiencing the same bug myself.
Here's the low down for me:

Python 1.6 with win95
I am running a little Tkinter program
The command line I use is simply: "python foo.py"
About 25-35% of the time, when I close the Tkinter
window, DOS seems to "freeze" and never returns to the
c:\ command prompt.  I have to ctrl-alt-delete
repeatedly and shut down "winoldapp" to get rid of the
window and then shell back into DOS and keep working.
It's a bit of a pain, since I have the habit of
testing EVERYTHING in tiny little stages, so I change
one little thing, test it ... freeze ... ARGH!  Change
one more tiny thing, test it ... freeze ... ARGH! 
However, sometimes it seems to behave and doesn't
bother me for an entire several hour session of python
work.

That's my report on the problem.

Cheers,

Joe


----------------------------------------------------------------------

Comment By: Jeffrey Hobbs (hobbs)
Date: 2003-02-24 19:49

Message:
Logged In: YES 
user_id=72656

Note that I really meant 8.4.1+, as in post-8.4.1.  Just 
grabbing 8.4.1 final will not suffice.  8.4.2 is about to come 
out for a checkpoint release with the fix.

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2003-02-24 19:35

Message:
Logged In: YES 
user_id=31435

I just tried, and saw the original symptom on the 3rd try, 
with Python 2.3a2 on Win98SE.  Then reproduced it twice 
after two reboots.  That installs Tcl/Tk 8.4.1.  They were 
not built with OPTS=threads.  I don't know whether 
OPTS=threads would cure it.

----------------------------------------------------------------------

Comment By: Jeffrey Hobbs (hobbs)
Date: 2003-02-24 09:04

Message:
Logged In: YES 
user_id=72656

This was against an older version (8.3.2 or 8.3.3).  Make sure 
any verification occurs against the HEAD or 8.3.5.

Also, Martin provided a threaded Tkinter implmentation, IIUC, 
so that makes this doubly out-of-date.

----------------------------------------------------------------------

Comment By: Donal K. Fellows (dkf)
Date: 2003-02-24 05:33

Message:
Logged In: YES 
user_id=79902

Exactly which version (including patchlevel) of tcl.dll and
tk.dll is this bug reported against?  (Well, which is the
most recent version...)  Is it one which the Tcl Maintainers
believe to have all identified relevant bugs in the area
resolved, or does it predate that?

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-24 04:59

Message:
Logged In: YES 
user_id=6380

Um, I think it's *not* closed, but I'll assign it back to
Tim. I believe Tim said about this somewhere "the Tcl folks
have often thought they'd nailed it, but they still
haven't". Unless that was an entirely different problem. But
it used to happen for me on Win98 too (haven't tried in a
while).

----------------------------------------------------------------------

Comment By: Fredrik Lundh (effbot)
Date: 2003-02-23 23:13

Message:
Logged In: YES 
user_id=38376

(a bugfix a day keeps etc)

Tim, can we close this one?  I don't have access to any-
thing even remotely similar to a Win95-box any longer...

----------------------------------------------------------------------

Comment By: Jeffrey Hobbs (hobbs)
Date: 2002-12-09 20:16

Message:
Logged In: YES 
user_id=72656

Interesting last couple of comments ...

Anyway, I think the cause of this problem may have finally 
been identified by Tcl bug 651139.  I have attached a patch 
here for it against 8.3.5.  Both the 8.3.5 branch and 8.4.1+ 
head have this fix.  Amazingly it didn't effect Tcl, but could 
likely have caught Tkinter (only on Windows with non-
threaded builds).

Oh look, not allowed to attach a patch, so I'll attach it to Tcl 
bug 651139.

----------------------------------------------------------------------

Comment By: christian soldan (elacademico)
Date: 2002-11-30 18:07

Message:
Logged In: YES 
user_id=659858

how do i hack my fuckers friends

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2002-09-30 16:23

Message:
Logged In: NO 

i would like to know some moves

----------------------------------------------------------------------

Comment By: John Popplewell (johnnypops)
Date: 2002-08-21 17:08

Message:
Logged In: YES 
user_id=143340

Hi,

tried wrapping the WaitForSingleObject() with if (consolePtr-
>toWrite) { ... as suggested. Solves the problem for me in 
debug and release builds (without thread support).

What is "the tcl83.dll from www.tcl.tk", what changes does 
it include?

cheers,
John.


----------------------------------------------------------------------

Comment By: Doug Henderson (djhender)
Date: 2002-08-16 16:03

Message:
Logged In: YES 
user_id=148444

I downloaded and tested the tcl83.dll from www.tcl.tk with 
Python 2.2.1 on Win98SE using my BugDemo.py script 
from a related bug report [231207]. Several test runs 
showed that a problem still exists. The app hangs until 
Winopldapp is killed, however this leaves tcl83.dll in use. 
The problem does not occur on every execution. Some 
types of activity seem to provoke the hang, for instance, 
minimizing and maximizing the window seems to increase 
the change that it will hang on exit.

----------------------------------------------------------------------

Comment By: David Gravereaux (davygrvy)
Date: 2002-08-16 14:53

Message:
Logged In: YES 
user_id=7549

Whoop, error in the patch..  consoleMutex might not need to 
be locked.  It could cause a new dead-lock.

----------------------------------------------------------------------

Comment By: David Gravereaux (davygrvy)
Date: 2002-08-16 14:41

Message:
Logged In: YES 
user_id=7549

>It is hanging up in ConsoleCloseProc() in 'tclWinConsole.c'
>at about line 527:
>WaitForSingleObject(consolePtr->writeThread, INFINITE);

Jeff, this might need the same attention the pipe reader 
thread got for that [exec] memory leak bug.  I think 
ConsoleWriterThread() should use WaitForMultipleObjects 
instead of WaitForSingleObject with a second event handle 
used to single a fall-out like the pipe reader thread does.  It 
appears that ConsoleCloseProc() has no way to signal the 
writer thread to close at all.

This doesn't address the blocking WriteFile() in 
ConsoleWriterThread(), though.  Is it blocking there, too?
Sorry for the large post.  I'm given the option to attach a file.

Index: tclWinConsole.c
=============================================
======================
RCS file: /cvsroot/tcl/tcl/win/tclWinConsole.c,v
retrieving revision 1.7
diff -c -r1.7 tclWinConsole.c
*** tclWinConsole.c	15 Jan 2002 17:55:30 -0000	1.7
--- tclWinConsole.c	16 Aug 2002 21:31:56 -0000
***************
*** 79,84 ****
--- 79,87 ----
      HANDLE startWriter;		/* Auto-reset 
event used by the main thread to
  				 * signal when 
the writer thread should attempt
  				 * to write to the 
console. */
+     HANDLE stopWriter;		/* Auto-reset 
event used by the main thread to
+ 				 * signal when 
the writer thread should
+ 				 * terminate. */
      HANDLE startReader;		/* Auto-reset 
event used by the main thread to
  				 * signal when 
the reader thread should attempt
  				 * to read from 
the console. */
***************
*** 516,522 ****
  	 */
  
  	Tcl_MutexLock(&consoleMutex);
! 	TerminateThread(consolePtr->writeThread, 0);
  
  	/*
  	 * Wait for the thread to terminate.  This ensures 
that we are
--- 519,525 ----
  	 */
  
  	Tcl_MutexLock(&consoleMutex);
! 	SetEvent(consolePtr->stopWriter);
  
  	/*
  	 * Wait for the thread to terminate.  This ensures 
that we are
***************
*** 1134,1149 ****
  {
  
      ConsoleInfo *infoPtr = (ConsoleInfo *)arg;
      HANDLE *handle = infoPtr->handle;
      DWORD count, toWrite;
      char *buf;
  
      for (;;) {
  	/*
  	 * Wait for the main thread to signal before 
attempting to write.
  	 */
  
! 	WaitForSingleObject(infoPtr->startWriter, 
INFINITE);
  
  	buf = infoPtr->writeBuf;
  	toWrite = infoPtr->toWrite;
--- 1137,1163 ----
  {
  
      ConsoleInfo *infoPtr = (ConsoleInfo *)arg;
+     HANDLE events[2];
      HANDLE *handle = infoPtr->handle;
      DWORD count, toWrite;
      char *buf;
  
+     events[0] = infoPtr->stopWriter;
+     events[1] = infoPtr->startWriter;
+ 
      for (;;) {
+ 	DWORD dwWait;
+ 
  	/*
  	 * Wait for the main thread to signal before 
attempting to write.
  	 */
  
! 	dwWait = WaitForMultipleObjects(events, 2, 
FALSE, INFINITE);
! 
! 	if (dwWait != (WAIT_OBJECT_0 + 1)) {
! 	    /* either the stop event or some other error, so 
exit */
! 	    return 0;
! 	}
  
  	buf = infoPtr->writeBuf;
  	toWrite = infoPtr->toWrite;
***************
*** 1251,1256 ****
--- 1265,1271 ----
      if (permissions & TCL_WRITABLE) {
  	infoPtr->writable = CreateEvent(NULL, TRUE, 
TRUE, NULL);
  	infoPtr->startWriter = CreateEvent(NULL, FALSE, 
FALSE, NULL);
+ 	infoPtr->stopWriter = CreateEvent(NULL, FALSE, 
FALSE, NULL);
  	infoPtr->writeThread = CreateThread(NULL, 8000, 
ConsoleWriterThread,
  	        infoPtr, 0, &id);
      }


----------------------------------------------------------------------

Comment By: Jeffrey Hobbs (hobbs)
Date: 2002-08-16 11:56

Message:
Logged In: YES 
user_id=72656

I'm finding it a bit hard to reproduce, and now not at all.  
Could people please try replacing their current tcl83.dll with 
the one at http://www.tcl.tk/tcl83.dll.  The sig on it is:
634880 Aug 16 10:53 tcl83.dll
(md5) b35628568ddc4afed4f675d2d9a50faf  tcl83.dll

I can't hang anymore with python 2.2.1 at the Win98 
command prompt with:
   python \Python22\Tools\idle\idle.pyw

----------------------------------------------------------------------

Comment By: Jeffrey Hobbs (hobbs)
Date: 2002-08-16 10:50

Message:
Logged In: YES 
user_id=72656

johnny - That actually helps a lot and fairly clearly indicates 
the source of this hanging problem.  To give more context to 
that area, this is in the final shutdown code where it is 
(obviously) shutting down the console I/O.  The reader is 
shutdown without a check for more input (why, we are 
exiting?), so there is no issue there.

The writer however wants to make sure that the console 
output channel is drained before exit.  A blocking channel 
that is not being flushed will cause a hang.

So the question is, what is the stdio setup for Tcl in 
Py/Tkinter?  That should be examined to see whether stdio is 
clear to flush on exit.

Perhaps you can also help with more debugging.  Could you 
wrap the problematic WaitForSingleObject call with
if (consolePtr->toWrite) { ...
and perhaps also print out what to write.  It may be that 
blocking output isn't the problem at all, but rather that we are 
never receiving the signal on consolePtr->writable (usually 
done in the writer thread).  The consolePtr->toWrite check 
will at least ensure flushed channels do not enter this 
potential hanging state.

----------------------------------------------------------------------

Comment By: John Popplewell (johnnypops)
Date: 2002-08-16 07:16

Message:
Logged In: YES 
user_id=143340

I have dug out the setup I used previously and, apart from 
using Python 2.2 instead of 2.0.1, have had a go at 
reproducing my previous results.

With a debug build of Tcl/Tk and threads disabled I have 
managed to get this stack trace when it locks on exit (after 
doing a break):

ConsoleCloseProc(int * 0x006a0ae4, Tcl_Interp * 
0x00000000) line 527 + 17 bytes
TclFinalizeIOSubsystem() line 241 + 20 bytes
Tcl_FinalizeThread() line 911
Tcl_Finalize() line 812
DllMain(HINSTANCE__ * 0x00970000, unsigned long 
0x00000000, void * 0x00000001) line 197
TCL83! _DllMainCRTStartup@12 + 80 bytes
KERNEL32! bff7ddd6()
KERNEL32! bff8d123()
8176b5fc()

It is hanging up in ConsoleCloseProc() in 'tclWinConsole.c'
at about line 527:
WaitForSingleObject(consolePtr->writeThread, INFINITE);

I tried changing the timeout from INFINITE to 100ms and 
testing the return value for WAIT_TIMEOUT. Sure enough it 
occasionaly printed a message (and didn't lock-up).

Don't know if this helps. Will try to reproduce the double 
free problem in the threaded build later,

cheers,
John.


----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2002-08-15 07:19

Message:
Logged In: YES 
user_id=6380

Just a note so I won't forget this: Jeff Hobbs once said the
code to look at is

tk/generic/tkConsole.c:Tk_InitConsoleChannels

----------------------------------------------------------------------

Comment By: David Gravereaux (davygrvy)
Date: 2002-08-15 06:23

Message:
Logged In: YES 
user_id=7549

Hi John,

I have seen Tcl extension DLLs that set exit handlers get 
called by Tcl after they were torn-down.  NT and Win9x seem 
to be a bit different in the order of what DLLs get torn-down.  
here's a note from some old code of mine:

#ifdef __WIN32__
    // It can happen that the HEAP could have already been 
unloaded
    // from an awkward teardown caused by a Ctrl+C or other.  
Win32
    // seems to do a teardown in reverse order and by the time 
Tcl
    // knows what's going on and Tcl_Finalize calls the exit
    // handlers, this extension's data (heap?) has already been
    // unloaded by the OS.  Do a quick legal check on the 
pointer
    // first.
    //
    if (IsBadReadPtr(adapt, sizeof(T *))) return;
#endif

what's the back/stacktrace?  who is calling 
Tcl_FinalizeNotifier?  And had anyone called it before during 
the shutdown phase? I can't say with much confidence that 
this is happening here, too.  Maybe..  Could Tk be telling Tcl 
shutdown?  That's sort of backwards, if so.

I run w2k here, so I can't help in debugging from a test case.  
tclsh with a threaded build doesn't even call Tcl_Finalize to 
avoid all this!

----------------------------------------------------------------------

Comment By: Donal K. Fellows (dkf)
Date: 2002-08-15 06:09

Message:
Logged In: YES 
user_id=79902

Firstly, on the matter of threads.  Tcl/Tk on Windows uses
threads internally even in non-threaded builds.  That's
Windows for you (in particular, you can get calls parachuted
in from completely separate threads with relatively little
warning.  Also, you need threaded helpers to handle
non-blocking waiting on some kinds of objects.)

It's a bit hard to work out if the problem's been fixed
without a stack-trace (preferably with debugging symbols
enabled so that we can see what the function names and
arguments to those functions are.)  I have no  chance of
replicating this bug here (I'm currently running IRIX, but
also have some access to Solaris and Linux...)

----------------------------------------------------------------------

Comment By: John Popplewell (johnnypops)
Date: 2002-08-15 05:15

Message:
Logged In: YES 
user_id=143340

Hi to davygrvy,

my fix isn't for a thread related (deadlock) type problem - it 
is a fix for a double free type problem which is presumably 
screwing up the heap and causing the C runtime to hang.

It *is* with a threaded build (THREADDEFINES = -
DTCL_THREADS=1) which I think I enabled because I was 
using  'after' and it didn't work unless threading was 
enabled - but I may be mis-remembering here!

Also, I ended up doing all this with the debug build and 
found the -GZ flag useful - in fact it pointed out the fact that 
there was a problem with the heap.

It is worth noting that I have only seen this on Win98 and 
that using 'pythonw.exe' did *not* prevent this from 
happening,

regards John.


----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2002-08-14 18:00

Message:
Logged In: YES 
user_id=6380

AFAIK we never distributed a threaded build.

----------------------------------------------------------------------

Comment By: David Gravereaux (davygrvy)
Date: 2002-08-14 17:01

Message:
Logged In: YES 
user_id=7549

To johnnypops,

Where in Tcl_FinalizeNotifier() is the dead-lock occuring and 
is this with a threaded build of Tcl?


----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2002-07-02 12:39

Message:
Logged In: YES 
user_id=31435

Jeff, a long time ago Fredrik Lundh said he was able to 
reproduce this with a short wish script (no Python, no Tkinter, 
just Tcl/Tk + wish).  I'm reassigning this to /F in the hopes 
he'll remember how.

----------------------------------------------------------------------

Comment By: Jeffrey Hobbs (hobbs)
Date: 2002-07-02 11:55

Message:
Logged In: YES 
user_id=72656

I appreciate that many others see this is a problem in 
embedding Tcl, but without any clear way to reproduce this in 
Tcl/Tk itself, it's very hard for me to fix it.  I don't think that 
there is anything done in 8.4 to fix this that I recall, but if 
someone finds that the problem "goes away" with 8.4, please 
report that.

For those of course that experience this with python, the 
simple and "correct" work-around is to launch with 
pythonw.exe instead of python.exe.

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2002-06-26 10:40

Message:
Logged In: NO 

According to http://tcl.activestate.com ActiveTcl 8.4.0 is now 
available for download.  It is new as of June 24.  I don't know 
whether this will solve our problems or not, but it's worth 
looking into.

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2002-06-19 06:36

Message:
Logged In: NO 

I just downloaded Ruby 1.6.6 for Windows, which also uses 
Tcl/Tk 8.3 and I experience exactly the same problem there. 
So this is definitely a Tcl problem (but we know that already).

----------------------------------------------------------------------

Comment By: John Popplewell (johnnypops)
Date: 2002-06-07 12:01

Message:
Logged In: YES 
user_id=143340

Hmmm. 
I run a Python Tcl/Tk application several times every day - 
the very application that prompted my bug-hunt.

I do use "pythonw.exe" to launch it, but it used to hang 
almost as badly as "python.exe".

Whilst developing this app I used "python.exe" and 
experienced no hang-ups.

The Tcl guys point out that this bug is only one of several 
possible causes of the hang-up, maybe you are just out of 
luck.

The patch has fixed the problem on three Win98 machines, 
all of which exhibited the problem before updating the DLLs.

One thought - are you sure there weren't multiple copies of 
the tcl83.dll and tk83.dll files? My (only) copies were 
in "D:\python20\DLLs"

good luck,
John.


----------------------------------------------------------------------

Comment By: Jeffrey Hobbs (hobbs)
Date: 2002-06-03 11:55

Message:
Logged In: YES 
user_id=72656

providing Tim with a fixed DLL exactly according to John 
Popplewell's patch did not fix the problem for him.  That 
makes it seem fishy that John really didn't see the problem 
any more.

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2002-06-03 09:59

Message:
Logged In: YES 
user_id=31435

I tried replacing Python 2.2.1's tk83.dll and tcl83.dll (from 
Tcl/Tk 8.3.2) with newer matched sets from

A) PythonWare's py21 distribution (Tcl/Tk 8.3.3).

and again from

B) ActiveState's Tcl/Tk 8.3.4.

All the symptoms originally reported here persisted on 
Win98SE despite that.

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2002-05-22 06:57

Message:
Logged In: NO 

I am new to Python and encountered this bug on my very first 
Tkinter application.  It persists in Python 2.2.1 under 
Windows Me. Needless to say, it makes a bad first 
impression.  Basically, this thread says that Tkinter has been 
broken for a year and a half on a popular platform at a time 
when Python is trying to expand its user base.  Hopefully, this 
bug will be corrected soon.

If Tcl/Tk 8.4 is not released shortly you might try repackaging 
Tkinter for Win32 with an earlier version of Tcl/Tk, such as 
8.0, that doesn't encounter this problem.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2002-03-22 19:36

Message:
Logged In: YES 
user_id=6380

The proper action is to wait until Tcl 8.3.5 is released,
and then shaming someone else in providing you with a set of
Windowsbinaries for Tcl. You may also ask Hobbs if there's a
Windows project file to build Tcl/Tk for Windows.

----------------------------------------------------------------------

Comment By: Jeffrey Hobbs (hobbs)
Date: 2002-03-11 18:31

Message:
Logged In: YES 
user_id=72656

I did end up back-porting this to the 8.3.4+ Tcl CVS on 
2002-02-25, which means it will be in an eventual 8.3.5.  
It is already in the release 8.4 alpha series.

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2002-03-09 17:50

Message:
Logged In: YES 
user_id=31435

Back to Guido:  Do you think you know what the proper 
action is?  (You said that's why you reopened it, and 
there's been no new info here since then.)

----------------------------------------------------------------------

Comment By: John Popplewell (johnnypops)
Date: 2002-02-25 16:55

Message:
Logged In: YES 
user_id=143340

I knew I wasn't getting to the heart of it ....

Almost a one-liner!

It has been seriously spoiling my (otherwise crash free) 
Python experience on Win9x - hope it gets fixed soon.

cheers,
John.


----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2002-02-25 15:39

Message:
Logged In: YES 
user_id=6380

Reopened until we know what the proper action is.

----------------------------------------------------------------------

Comment By: Jeffrey Hobbs (hobbs)
Date: 2002-02-25 15:32

Message:
Logged In: YES 
user_id=72656

This is mostly correct, and is fixed in the 8.4 Tcl sources 
already (I guess we can backport this).  This was mentioned 
in SF Tcl bug (account for chopped URL):

https://sourceforge.net/tracker/?
func=detail&aid=217982&group_id=10894&atid=110894

and the code comment is:

    /*
     * Only finalize the notifier if a notifier was 
installed in the
     * current thread; there is a route in which this is not
     * guaranteed to be true (when tclWin32Dll.c:DllMain() 
is called
     * with the flag DLL_PROCESS_DETACH by the OS, which 
could be
     * doing so from a thread that's never previously been 
involved
     * with Tcl, e.g. the task manager) so this check is 
important.
     *
     * Fixes Bug #217982 reported by Hugh Vu and Gene 
Leache.
     */
    if (tsdPtr == NULL) {
	return;
    }


----------------------------------------------------------------------

Comment By: John Popplewell (johnnypops)
Date: 2002-02-25 14:41

Message:
Logged In: YES 
user_id=143340

This one has been torturing me for a while.

Managed to track it down to a problem inside Tcl.
For the Tcl8.3.4 source distribution the problem is in
the file win/tclWinNotify.c 

void Tcl_FinalizeNotifier(ClientData clientData)
{
    ThreadSpecificData *tsdPtr = 
                         (ThreadSpecificData *) clientData;

    /* sometimes called with tsdPtr == NULL */
    if ( tsdPtr != NULL )
    {
        DeleteCriticalSection(&tsdPtr->crit);
        CloseHandle(tsdPtr->event);

        /*
         * Clean up the timer and messaging 
         * window for this thread.
         */

        if (tsdPtr->hwnd) {
	    KillTimer(tsdPtr->hwnd, INTERVAL_TIMER);
            DestroyWindow(tsdPtr->hwnd);
        }
    }

    /*
     * If this is the last thread to use the notifier, 
     * unregister the notifier window class.
     */

    Tcl_MutexLock(&notifierMutex);
    if ( notifierCount && !--notifierCount ) {
	UnregisterClassA( "TclNotifier", 
                          TclWinGetTclInstance());
    }
    Tcl_MutexUnlock(&notifierMutex);
}

This bodge doesn't address the underlying problem but has 
stopped me from tearing all my hair out,

cheers,
John Popplewell.


----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2001-10-24 15:27

Message:
Logged In: YES 
user_id=31435

FYI, you don't need an IDE to do this -- in Win9x, hit 
Ctrl+Alt+Del and kill the process directly.

A saner <wink> solution is to develop under Win2K, which 
doesn't appear to suffer this problem (the only reports 
I've seen, and experienced myself, came from Win9x boxes).

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2001-10-24 01:52

Message:
Logged In: NO 

For those who are still trapped in this bug's hell, I will 
gladly share the one thing that saved my sanity:  
WingIDE's 'Kill' command will shut down the program with 
apparent 100% certainty and no fear of lockups.  WingIDE 
has its own issues, so its not a perfect solution, but if 
you are like me and Joe (above) who test in small 
iterations, then using 'Kill' to quit out of your app while 
testing is a workaround that may preserve your sanity.

Perhaps the python gods and the Wing guys can get together 
and tell us how to replicate 'kill' into our code.  For 
now, I'll use WingIDE to edit, and pythonw.exe for my final 
client's delivery.

----------------------------------------------------------------------

Comment By: Howard Lightstone (hlightstone)
Date: 2001-09-05 10:43

Message:
Logged In: YES 
user_id=66570

I sometimes get bunches of these....

A tool I use (Taskinfo2000) reports that (after killing 
winoldap):

python.exe is blocked on a mutex named OLESCELOCKMUTEX.
The reported state is "Console Terminating".  There appears 
to be only one (os) thread running.


----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2001-04-02 13:06

Message:
Logged In: YES 
user_id=31435

No sign of progress on this presumed Tk/Tcl Windows bug in 
over 3 months, so closing it for lack of hope.

----------------------------------------------------------------------

Comment By: Doug Henderson (djhender)
Date: 2001-02-05 21:13

Message:
This was a symptom I saw while tracking down the essence of the problem reported in #131207.  Using Win98SE, I would get an error dialog (GPF?) in the Kernel, and sometimes the dos prompt would not come back.

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2000-12-12 18:00

Message:
Just reproduced w/ current CVS, but didn't hang until the 8th try.

http://dev.scriptics.com/software/tcltk/ says 8.3 is still the latest released version; don't know whether that URL still makes sense, though.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2000-12-12 12:58

Message:
Tim, can you still reproduce this with the current CVS version?

There's been one critical patch to _tkinter since the 2.0 release.

An alternative would be to try with a newer version of Tcl (isn't 8.4 out already?).


----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2000-10-15 09:47

Message:
Same as I've reported earlier; it hangs in the call to
Tcl_Finalize (which is called by the DLL finalization
code).  It's less likely to hang if I call Tcl_Finalize
from the _tkinter DLL (from user code).

Note that the problem isn't really Python-related -- I
have stand-alone samples (based on wish) that hangs in
the same way.

More later.

</F>

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2000-10-13 07:40

Message:
Back to Tim since I have no clue what to do here.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2000-10-12 10:25

Message:
The recent fix to _tkinter (Tcl_GetStringResult(interp) instead of interp->result) didn't fix this either.

As Tim has remarked in private but not yet recorded here, a workaround is to use pythonw instead of python, so I'm lowering thepriority again.

Also note that the hanging process that Tim writes about apparently prevents Win98 from shutting down properly.

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2000-10-07 00:37

Message:
More info (none good, but some worse so boosted priority):

+ Happens under release and debug builds.

+ Have not been able to provoke when starting in the debugger.

+ Ctrl+Alt+Del and killing Winoldap is not enough to clean everything up.  There's still a Python (or Python_d) process hanging around that Ctrl+Alt+Del doesn't show.

+ This process makes it impossible to delete the associated Python .dll, and in particular makes it impossible to rebuild Python successfully without a reboot.

+ These processes cannot be killed!  Wintop and Process Viewer both fail to get the job done.  PrcView (a freeware process viewer) itself locks up if I try to kill them using it.  Process Viewer freezes for several seconds before giving up.

+ Attempting to attach to the process with the MSVC debugger (in order to find out what the heck it's doing) finds the process OK, but then yields the cryptic and undocumented error msg "Cannot execute program".

+ The processes are not accumulating cycles.

+ Smells like deadlock.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=216289&group_id=5470


From noreply@sourceforge.net  Tue Feb 25 03:55:56 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Mon, 24 Feb 2003 19:55:56 -0800
Subject: [Python-bugs-list] [ python-Bugs-216289 ] Programs using Tkinter sometimes can't shut down (Windows)
Message-ID: <E18nWCC-0006x6-00@sc8-sf-web4.sourceforge.net>

Bugs item #216289, was opened at 2000-10-06 22:25
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=216289&group_id=5470

Category: Windows
Group: Python 2.3
Status: Open
Resolution: Later
Priority: 3
Submitted By: Tim Peters (tim_one)
Assigned to: Tim Peters (tim_one)
Summary: Programs using Tkinter sometimes can't shut down (Windows)

Initial Comment:
The following msg from the Tutor list is about 1.6, but I noticed the same thing several times today using 2.0b2+CVS.  In my case, I was running IDLE via

python ../tool/idle/idle.pyw

from a DOS box in my PCbuild directory.  Win98SE.  *Most* of the time, shutting down IDLE via Ctrl+Q left the DOS box hanging.  As with the poster, the only way to regain control was to use the Task Manager to kill off Winoldap.

-----Original Message-----
From: Joseph Stubenrauch <nothingisgoingtochangemyworld@yahoo.com>
Sent: Friday, October 06, 2000 9:23 PM
To: tutor@python.org
Subject: Re: [Tutor] Python 1.6 BUG


Strange, I have been experiencing the same bug myself.
Here's the low down for me:

Python 1.6 with win95
I am running a little Tkinter program
The command line I use is simply: "python foo.py"
About 25-35% of the time, when I close the Tkinter
window, DOS seems to "freeze" and never returns to the
c:\ command prompt.  I have to ctrl-alt-delete
repeatedly and shut down "winoldapp" to get rid of the
window and then shell back into DOS and keep working.
It's a bit of a pain, since I have the habit of
testing EVERYTHING in tiny little stages, so I change
one little thing, test it ... freeze ... ARGH!  Change
one more tiny thing, test it ... freeze ... ARGH! 
However, sometimes it seems to behave and doesn't
bother me for an entire several hour session of python
work.

That's my report on the problem.

Cheers,

Joe


----------------------------------------------------------------------

>Comment By: Tim Peters (tim_one)
Date: 2003-02-24 22:55

Message:
Logged In: YES 
user_id=31435

That's cool -- Python 2.3 is on its second alpha release, so 
there's no time crunch yet.  I'll grab 8.4.2 when it's ready 
and try it again.  Thanks for paying attention to this 
miserable report for so many years <wink>!

----------------------------------------------------------------------

Comment By: Jeffrey Hobbs (hobbs)
Date: 2003-02-24 22:49

Message:
Logged In: YES 
user_id=72656

Note that I really meant 8.4.1+, as in post-8.4.1.  Just 
grabbing 8.4.1 final will not suffice.  8.4.2 is about to come 
out for a checkpoint release with the fix.

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2003-02-24 22:35

Message:
Logged In: YES 
user_id=31435

I just tried, and saw the original symptom on the 3rd try, 
with Python 2.3a2 on Win98SE.  Then reproduced it twice 
after two reboots.  That installs Tcl/Tk 8.4.1.  They were 
not built with OPTS=threads.  I don't know whether 
OPTS=threads would cure it.

----------------------------------------------------------------------

Comment By: Jeffrey Hobbs (hobbs)
Date: 2003-02-24 12:04

Message:
Logged In: YES 
user_id=72656

This was against an older version (8.3.2 or 8.3.3).  Make sure 
any verification occurs against the HEAD or 8.3.5.

Also, Martin provided a threaded Tkinter implmentation, IIUC, 
so that makes this doubly out-of-date.

----------------------------------------------------------------------

Comment By: Donal K. Fellows (dkf)
Date: 2003-02-24 08:33

Message:
Logged In: YES 
user_id=79902

Exactly which version (including patchlevel) of tcl.dll and
tk.dll is this bug reported against?  (Well, which is the
most recent version...)  Is it one which the Tcl Maintainers
believe to have all identified relevant bugs in the area
resolved, or does it predate that?

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-24 07:59

Message:
Logged In: YES 
user_id=6380

Um, I think it's *not* closed, but I'll assign it back to
Tim. I believe Tim said about this somewhere "the Tcl folks
have often thought they'd nailed it, but they still
haven't". Unless that was an entirely different problem. But
it used to happen for me on Win98 too (haven't tried in a
while).

----------------------------------------------------------------------

Comment By: Fredrik Lundh (effbot)
Date: 2003-02-24 02:13

Message:
Logged In: YES 
user_id=38376

(a bugfix a day keeps etc)

Tim, can we close this one?  I don't have access to any-
thing even remotely similar to a Win95-box any longer...

----------------------------------------------------------------------

Comment By: Jeffrey Hobbs (hobbs)
Date: 2002-12-09 23:16

Message:
Logged In: YES 
user_id=72656

Interesting last couple of comments ...

Anyway, I think the cause of this problem may have finally 
been identified by Tcl bug 651139.  I have attached a patch 
here for it against 8.3.5.  Both the 8.3.5 branch and 8.4.1+ 
head have this fix.  Amazingly it didn't effect Tcl, but could 
likely have caught Tkinter (only on Windows with non-
threaded builds).

Oh look, not allowed to attach a patch, so I'll attach it to Tcl 
bug 651139.

----------------------------------------------------------------------

Comment By: christian soldan (elacademico)
Date: 2002-11-30 21:07

Message:
Logged In: YES 
user_id=659858

how do i hack my fuckers friends

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2002-09-30 19:23

Message:
Logged In: NO 

i would like to know some moves

----------------------------------------------------------------------

Comment By: John Popplewell (johnnypops)
Date: 2002-08-21 20:08

Message:
Logged In: YES 
user_id=143340

Hi,

tried wrapping the WaitForSingleObject() with if (consolePtr-
>toWrite) { ... as suggested. Solves the problem for me in 
debug and release builds (without thread support).

What is "the tcl83.dll from www.tcl.tk", what changes does 
it include?

cheers,
John.


----------------------------------------------------------------------

Comment By: Doug Henderson (djhender)
Date: 2002-08-16 19:03

Message:
Logged In: YES 
user_id=148444

I downloaded and tested the tcl83.dll from www.tcl.tk with 
Python 2.2.1 on Win98SE using my BugDemo.py script 
from a related bug report [231207]. Several test runs 
showed that a problem still exists. The app hangs until 
Winopldapp is killed, however this leaves tcl83.dll in use. 
The problem does not occur on every execution. Some 
types of activity seem to provoke the hang, for instance, 
minimizing and maximizing the window seems to increase 
the change that it will hang on exit.

----------------------------------------------------------------------

Comment By: David Gravereaux (davygrvy)
Date: 2002-08-16 17:53

Message:
Logged In: YES 
user_id=7549

Whoop, error in the patch..  consoleMutex might not need to 
be locked.  It could cause a new dead-lock.

----------------------------------------------------------------------

Comment By: David Gravereaux (davygrvy)
Date: 2002-08-16 17:41

Message:
Logged In: YES 
user_id=7549

>It is hanging up in ConsoleCloseProc() in 'tclWinConsole.c'
>at about line 527:
>WaitForSingleObject(consolePtr->writeThread, INFINITE);

Jeff, this might need the same attention the pipe reader 
thread got for that [exec] memory leak bug.  I think 
ConsoleWriterThread() should use WaitForMultipleObjects 
instead of WaitForSingleObject with a second event handle 
used to single a fall-out like the pipe reader thread does.  It 
appears that ConsoleCloseProc() has no way to signal the 
writer thread to close at all.

This doesn't address the blocking WriteFile() in 
ConsoleWriterThread(), though.  Is it blocking there, too?
Sorry for the large post.  I'm given the option to attach a file.

Index: tclWinConsole.c
=============================================
======================
RCS file: /cvsroot/tcl/tcl/win/tclWinConsole.c,v
retrieving revision 1.7
diff -c -r1.7 tclWinConsole.c
*** tclWinConsole.c	15 Jan 2002 17:55:30 -0000	1.7
--- tclWinConsole.c	16 Aug 2002 21:31:56 -0000
***************
*** 79,84 ****
--- 79,87 ----
      HANDLE startWriter;		/* Auto-reset 
event used by the main thread to
  				 * signal when 
the writer thread should attempt
  				 * to write to the 
console. */
+     HANDLE stopWriter;		/* Auto-reset 
event used by the main thread to
+ 				 * signal when 
the writer thread should
+ 				 * terminate. */
      HANDLE startReader;		/* Auto-reset 
event used by the main thread to
  				 * signal when 
the reader thread should attempt
  				 * to read from 
the console. */
***************
*** 516,522 ****
  	 */
  
  	Tcl_MutexLock(&consoleMutex);
! 	TerminateThread(consolePtr->writeThread, 0);
  
  	/*
  	 * Wait for the thread to terminate.  This ensures 
that we are
--- 519,525 ----
  	 */
  
  	Tcl_MutexLock(&consoleMutex);
! 	SetEvent(consolePtr->stopWriter);
  
  	/*
  	 * Wait for the thread to terminate.  This ensures 
that we are
***************
*** 1134,1149 ****
  {
  
      ConsoleInfo *infoPtr = (ConsoleInfo *)arg;
      HANDLE *handle = infoPtr->handle;
      DWORD count, toWrite;
      char *buf;
  
      for (;;) {
  	/*
  	 * Wait for the main thread to signal before 
attempting to write.
  	 */
  
! 	WaitForSingleObject(infoPtr->startWriter, 
INFINITE);
  
  	buf = infoPtr->writeBuf;
  	toWrite = infoPtr->toWrite;
--- 1137,1163 ----
  {
  
      ConsoleInfo *infoPtr = (ConsoleInfo *)arg;
+     HANDLE events[2];
      HANDLE *handle = infoPtr->handle;
      DWORD count, toWrite;
      char *buf;
  
+     events[0] = infoPtr->stopWriter;
+     events[1] = infoPtr->startWriter;
+ 
      for (;;) {
+ 	DWORD dwWait;
+ 
  	/*
  	 * Wait for the main thread to signal before 
attempting to write.
  	 */
  
! 	dwWait = WaitForMultipleObjects(events, 2, 
FALSE, INFINITE);
! 
! 	if (dwWait != (WAIT_OBJECT_0 + 1)) {
! 	    /* either the stop event or some other error, so 
exit */
! 	    return 0;
! 	}
  
  	buf = infoPtr->writeBuf;
  	toWrite = infoPtr->toWrite;
***************
*** 1251,1256 ****
--- 1265,1271 ----
      if (permissions & TCL_WRITABLE) {
  	infoPtr->writable = CreateEvent(NULL, TRUE, 
TRUE, NULL);
  	infoPtr->startWriter = CreateEvent(NULL, FALSE, 
FALSE, NULL);
+ 	infoPtr->stopWriter = CreateEvent(NULL, FALSE, 
FALSE, NULL);
  	infoPtr->writeThread = CreateThread(NULL, 8000, 
ConsoleWriterThread,
  	        infoPtr, 0, &id);
      }


----------------------------------------------------------------------

Comment By: Jeffrey Hobbs (hobbs)
Date: 2002-08-16 14:56

Message:
Logged In: YES 
user_id=72656

I'm finding it a bit hard to reproduce, and now not at all.  
Could people please try replacing their current tcl83.dll with 
the one at http://www.tcl.tk/tcl83.dll.  The sig on it is:
634880 Aug 16 10:53 tcl83.dll
(md5) b35628568ddc4afed4f675d2d9a50faf  tcl83.dll

I can't hang anymore with python 2.2.1 at the Win98 
command prompt with:
   python \Python22\Tools\idle\idle.pyw

----------------------------------------------------------------------

Comment By: Jeffrey Hobbs (hobbs)
Date: 2002-08-16 13:50

Message:
Logged In: YES 
user_id=72656

johnny - That actually helps a lot and fairly clearly indicates 
the source of this hanging problem.  To give more context to 
that area, this is in the final shutdown code where it is 
(obviously) shutting down the console I/O.  The reader is 
shutdown without a check for more input (why, we are 
exiting?), so there is no issue there.

The writer however wants to make sure that the console 
output channel is drained before exit.  A blocking channel 
that is not being flushed will cause a hang.

So the question is, what is the stdio setup for Tcl in 
Py/Tkinter?  That should be examined to see whether stdio is 
clear to flush on exit.

Perhaps you can also help with more debugging.  Could you 
wrap the problematic WaitForSingleObject call with
if (consolePtr->toWrite) { ...
and perhaps also print out what to write.  It may be that 
blocking output isn't the problem at all, but rather that we are 
never receiving the signal on consolePtr->writable (usually 
done in the writer thread).  The consolePtr->toWrite check 
will at least ensure flushed channels do not enter this 
potential hanging state.

----------------------------------------------------------------------

Comment By: John Popplewell (johnnypops)
Date: 2002-08-16 10:16

Message:
Logged In: YES 
user_id=143340

I have dug out the setup I used previously and, apart from 
using Python 2.2 instead of 2.0.1, have had a go at 
reproducing my previous results.

With a debug build of Tcl/Tk and threads disabled I have 
managed to get this stack trace when it locks on exit (after 
doing a break):

ConsoleCloseProc(int * 0x006a0ae4, Tcl_Interp * 
0x00000000) line 527 + 17 bytes
TclFinalizeIOSubsystem() line 241 + 20 bytes
Tcl_FinalizeThread() line 911
Tcl_Finalize() line 812
DllMain(HINSTANCE__ * 0x00970000, unsigned long 
0x00000000, void * 0x00000001) line 197
TCL83! _DllMainCRTStartup@12 + 80 bytes
KERNEL32! bff7ddd6()
KERNEL32! bff8d123()
8176b5fc()

It is hanging up in ConsoleCloseProc() in 'tclWinConsole.c'
at about line 527:
WaitForSingleObject(consolePtr->writeThread, INFINITE);

I tried changing the timeout from INFINITE to 100ms and 
testing the return value for WAIT_TIMEOUT. Sure enough it 
occasionaly printed a message (and didn't lock-up).

Don't know if this helps. Will try to reproduce the double 
free problem in the threaded build later,

cheers,
John.


----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2002-08-15 10:19

Message:
Logged In: YES 
user_id=6380

Just a note so I won't forget this: Jeff Hobbs once said the
code to look at is

tk/generic/tkConsole.c:Tk_InitConsoleChannels

----------------------------------------------------------------------

Comment By: David Gravereaux (davygrvy)
Date: 2002-08-15 09:23

Message:
Logged In: YES 
user_id=7549

Hi John,

I have seen Tcl extension DLLs that set exit handlers get 
called by Tcl after they were torn-down.  NT and Win9x seem 
to be a bit different in the order of what DLLs get torn-down.  
here's a note from some old code of mine:

#ifdef __WIN32__
    // It can happen that the HEAP could have already been 
unloaded
    // from an awkward teardown caused by a Ctrl+C or other.  
Win32
    // seems to do a teardown in reverse order and by the time 
Tcl
    // knows what's going on and Tcl_Finalize calls the exit
    // handlers, this extension's data (heap?) has already been
    // unloaded by the OS.  Do a quick legal check on the 
pointer
    // first.
    //
    if (IsBadReadPtr(adapt, sizeof(T *))) return;
#endif

what's the back/stacktrace?  who is calling 
Tcl_FinalizeNotifier?  And had anyone called it before during 
the shutdown phase? I can't say with much confidence that 
this is happening here, too.  Maybe..  Could Tk be telling Tcl 
shutdown?  That's sort of backwards, if so.

I run w2k here, so I can't help in debugging from a test case.  
tclsh with a threaded build doesn't even call Tcl_Finalize to 
avoid all this!

----------------------------------------------------------------------

Comment By: Donal K. Fellows (dkf)
Date: 2002-08-15 09:09

Message:
Logged In: YES 
user_id=79902

Firstly, on the matter of threads.  Tcl/Tk on Windows uses
threads internally even in non-threaded builds.  That's
Windows for you (in particular, you can get calls parachuted
in from completely separate threads with relatively little
warning.  Also, you need threaded helpers to handle
non-blocking waiting on some kinds of objects.)

It's a bit hard to work out if the problem's been fixed
without a stack-trace (preferably with debugging symbols
enabled so that we can see what the function names and
arguments to those functions are.)  I have no  chance of
replicating this bug here (I'm currently running IRIX, but
also have some access to Solaris and Linux...)

----------------------------------------------------------------------

Comment By: John Popplewell (johnnypops)
Date: 2002-08-15 08:15

Message:
Logged In: YES 
user_id=143340

Hi to davygrvy,

my fix isn't for a thread related (deadlock) type problem - it 
is a fix for a double free type problem which is presumably 
screwing up the heap and causing the C runtime to hang.

It *is* with a threaded build (THREADDEFINES = -
DTCL_THREADS=1) which I think I enabled because I was 
using  'after' and it didn't work unless threading was 
enabled - but I may be mis-remembering here!

Also, I ended up doing all this with the debug build and 
found the -GZ flag useful - in fact it pointed out the fact that 
there was a problem with the heap.

It is worth noting that I have only seen this on Win98 and 
that using 'pythonw.exe' did *not* prevent this from 
happening,

regards John.


----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2002-08-14 21:00

Message:
Logged In: YES 
user_id=6380

AFAIK we never distributed a threaded build.

----------------------------------------------------------------------

Comment By: David Gravereaux (davygrvy)
Date: 2002-08-14 20:01

Message:
Logged In: YES 
user_id=7549

To johnnypops,

Where in Tcl_FinalizeNotifier() is the dead-lock occuring and 
is this with a threaded build of Tcl?


----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2002-07-02 15:39

Message:
Logged In: YES 
user_id=31435

Jeff, a long time ago Fredrik Lundh said he was able to 
reproduce this with a short wish script (no Python, no Tkinter, 
just Tcl/Tk + wish).  I'm reassigning this to /F in the hopes 
he'll remember how.

----------------------------------------------------------------------

Comment By: Jeffrey Hobbs (hobbs)
Date: 2002-07-02 14:55

Message:
Logged In: YES 
user_id=72656

I appreciate that many others see this is a problem in 
embedding Tcl, but without any clear way to reproduce this in 
Tcl/Tk itself, it's very hard for me to fix it.  I don't think that 
there is anything done in 8.4 to fix this that I recall, but if 
someone finds that the problem "goes away" with 8.4, please 
report that.

For those of course that experience this with python, the 
simple and "correct" work-around is to launch with 
pythonw.exe instead of python.exe.

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2002-06-26 13:40

Message:
Logged In: NO 

According to http://tcl.activestate.com ActiveTcl 8.4.0 is now 
available for download.  It is new as of June 24.  I don't know 
whether this will solve our problems or not, but it's worth 
looking into.

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2002-06-19 09:36

Message:
Logged In: NO 

I just downloaded Ruby 1.6.6 for Windows, which also uses 
Tcl/Tk 8.3 and I experience exactly the same problem there. 
So this is definitely a Tcl problem (but we know that already).

----------------------------------------------------------------------

Comment By: John Popplewell (johnnypops)
Date: 2002-06-07 15:01

Message:
Logged In: YES 
user_id=143340

Hmmm. 
I run a Python Tcl/Tk application several times every day - 
the very application that prompted my bug-hunt.

I do use "pythonw.exe" to launch it, but it used to hang 
almost as badly as "python.exe".

Whilst developing this app I used "python.exe" and 
experienced no hang-ups.

The Tcl guys point out that this bug is only one of several 
possible causes of the hang-up, maybe you are just out of 
luck.

The patch has fixed the problem on three Win98 machines, 
all of which exhibited the problem before updating the DLLs.

One thought - are you sure there weren't multiple copies of 
the tcl83.dll and tk83.dll files? My (only) copies were 
in "D:\python20\DLLs"

good luck,
John.


----------------------------------------------------------------------

Comment By: Jeffrey Hobbs (hobbs)
Date: 2002-06-03 14:55

Message:
Logged In: YES 
user_id=72656

providing Tim with a fixed DLL exactly according to John 
Popplewell's patch did not fix the problem for him.  That 
makes it seem fishy that John really didn't see the problem 
any more.

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2002-06-03 12:59

Message:
Logged In: YES 
user_id=31435

I tried replacing Python 2.2.1's tk83.dll and tcl83.dll (from 
Tcl/Tk 8.3.2) with newer matched sets from

A) PythonWare's py21 distribution (Tcl/Tk 8.3.3).

and again from

B) ActiveState's Tcl/Tk 8.3.4.

All the symptoms originally reported here persisted on 
Win98SE despite that.

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2002-05-22 09:57

Message:
Logged In: NO 

I am new to Python and encountered this bug on my very first 
Tkinter application.  It persists in Python 2.2.1 under 
Windows Me. Needless to say, it makes a bad first 
impression.  Basically, this thread says that Tkinter has been 
broken for a year and a half on a popular platform at a time 
when Python is trying to expand its user base.  Hopefully, this 
bug will be corrected soon.

If Tcl/Tk 8.4 is not released shortly you might try repackaging 
Tkinter for Win32 with an earlier version of Tcl/Tk, such as 
8.0, that doesn't encounter this problem.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2002-03-22 22:36

Message:
Logged In: YES 
user_id=6380

The proper action is to wait until Tcl 8.3.5 is released,
and then shaming someone else in providing you with a set of
Windowsbinaries for Tcl. You may also ask Hobbs if there's a
Windows project file to build Tcl/Tk for Windows.

----------------------------------------------------------------------

Comment By: Jeffrey Hobbs (hobbs)
Date: 2002-03-11 21:31

Message:
Logged In: YES 
user_id=72656

I did end up back-porting this to the 8.3.4+ Tcl CVS on 
2002-02-25, which means it will be in an eventual 8.3.5.  
It is already in the release 8.4 alpha series.

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2002-03-09 20:50

Message:
Logged In: YES 
user_id=31435

Back to Guido:  Do you think you know what the proper 
action is?  (You said that's why you reopened it, and 
there's been no new info here since then.)

----------------------------------------------------------------------

Comment By: John Popplewell (johnnypops)
Date: 2002-02-25 19:55

Message:
Logged In: YES 
user_id=143340

I knew I wasn't getting to the heart of it ....

Almost a one-liner!

It has been seriously spoiling my (otherwise crash free) 
Python experience on Win9x - hope it gets fixed soon.

cheers,
John.


----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2002-02-25 18:39

Message:
Logged In: YES 
user_id=6380

Reopened until we know what the proper action is.

----------------------------------------------------------------------

Comment By: Jeffrey Hobbs (hobbs)
Date: 2002-02-25 18:32

Message:
Logged In: YES 
user_id=72656

This is mostly correct, and is fixed in the 8.4 Tcl sources 
already (I guess we can backport this).  This was mentioned 
in SF Tcl bug (account for chopped URL):

https://sourceforge.net/tracker/?
func=detail&aid=217982&group_id=10894&atid=110894

and the code comment is:

    /*
     * Only finalize the notifier if a notifier was 
installed in the
     * current thread; there is a route in which this is not
     * guaranteed to be true (when tclWin32Dll.c:DllMain() 
is called
     * with the flag DLL_PROCESS_DETACH by the OS, which 
could be
     * doing so from a thread that's never previously been 
involved
     * with Tcl, e.g. the task manager) so this check is 
important.
     *
     * Fixes Bug #217982 reported by Hugh Vu and Gene 
Leache.
     */
    if (tsdPtr == NULL) {
	return;
    }


----------------------------------------------------------------------

Comment By: John Popplewell (johnnypops)
Date: 2002-02-25 17:41

Message:
Logged In: YES 
user_id=143340

This one has been torturing me for a while.

Managed to track it down to a problem inside Tcl.
For the Tcl8.3.4 source distribution the problem is in
the file win/tclWinNotify.c 

void Tcl_FinalizeNotifier(ClientData clientData)
{
    ThreadSpecificData *tsdPtr = 
                         (ThreadSpecificData *) clientData;

    /* sometimes called with tsdPtr == NULL */
    if ( tsdPtr != NULL )
    {
        DeleteCriticalSection(&tsdPtr->crit);
        CloseHandle(tsdPtr->event);

        /*
         * Clean up the timer and messaging 
         * window for this thread.
         */

        if (tsdPtr->hwnd) {
	    KillTimer(tsdPtr->hwnd, INTERVAL_TIMER);
            DestroyWindow(tsdPtr->hwnd);
        }
    }

    /*
     * If this is the last thread to use the notifier, 
     * unregister the notifier window class.
     */

    Tcl_MutexLock(&notifierMutex);
    if ( notifierCount && !--notifierCount ) {
	UnregisterClassA( "TclNotifier", 
                          TclWinGetTclInstance());
    }
    Tcl_MutexUnlock(&notifierMutex);
}

This bodge doesn't address the underlying problem but has 
stopped me from tearing all my hair out,

cheers,
John Popplewell.


----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2001-10-24 18:27

Message:
Logged In: YES 
user_id=31435

FYI, you don't need an IDE to do this -- in Win9x, hit 
Ctrl+Alt+Del and kill the process directly.

A saner <wink> solution is to develop under Win2K, which 
doesn't appear to suffer this problem (the only reports 
I've seen, and experienced myself, came from Win9x boxes).

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2001-10-24 04:52

Message:
Logged In: NO 

For those who are still trapped in this bug's hell, I will 
gladly share the one thing that saved my sanity:  
WingIDE's 'Kill' command will shut down the program with 
apparent 100% certainty and no fear of lockups.  WingIDE 
has its own issues, so its not a perfect solution, but if 
you are like me and Joe (above) who test in small 
iterations, then using 'Kill' to quit out of your app while 
testing is a workaround that may preserve your sanity.

Perhaps the python gods and the Wing guys can get together 
and tell us how to replicate 'kill' into our code.  For 
now, I'll use WingIDE to edit, and pythonw.exe for my final 
client's delivery.

----------------------------------------------------------------------

Comment By: Howard Lightstone (hlightstone)
Date: 2001-09-05 13:43

Message:
Logged In: YES 
user_id=66570

I sometimes get bunches of these....

A tool I use (Taskinfo2000) reports that (after killing 
winoldap):

python.exe is blocked on a mutex named OLESCELOCKMUTEX.
The reported state is "Console Terminating".  There appears 
to be only one (os) thread running.


----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2001-04-02 16:06

Message:
Logged In: YES 
user_id=31435

No sign of progress on this presumed Tk/Tcl Windows bug in 
over 3 months, so closing it for lack of hope.

----------------------------------------------------------------------

Comment By: Doug Henderson (djhender)
Date: 2001-02-06 00:13

Message:
This was a symptom I saw while tracking down the essence of the problem reported in #131207.  Using Win98SE, I would get an error dialog (GPF?) in the Kernel, and sometimes the dos prompt would not come back.

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2000-12-12 21:00

Message:
Just reproduced w/ current CVS, but didn't hang until the 8th try.

http://dev.scriptics.com/software/tcltk/ says 8.3 is still the latest released version; don't know whether that URL still makes sense, though.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2000-12-12 15:58

Message:
Tim, can you still reproduce this with the current CVS version?

There's been one critical patch to _tkinter since the 2.0 release.

An alternative would be to try with a newer version of Tcl (isn't 8.4 out already?).


----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2000-10-15 12:47

Message:
Same as I've reported earlier; it hangs in the call to
Tcl_Finalize (which is called by the DLL finalization
code).  It's less likely to hang if I call Tcl_Finalize
from the _tkinter DLL (from user code).

Note that the problem isn't really Python-related -- I
have stand-alone samples (based on wish) that hangs in
the same way.

More later.

</F>

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2000-10-13 10:40

Message:
Back to Tim since I have no clue what to do here.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2000-10-12 13:25

Message:
The recent fix to _tkinter (Tcl_GetStringResult(interp) instead of interp->result) didn't fix this either.

As Tim has remarked in private but not yet recorded here, a workaround is to use pythonw instead of python, so I'm lowering thepriority again.

Also note that the hanging process that Tim writes about apparently prevents Win98 from shutting down properly.

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2000-10-07 03:37

Message:
More info (none good, but some worse so boosted priority):

+ Happens under release and debug builds.

+ Have not been able to provoke when starting in the debugger.

+ Ctrl+Alt+Del and killing Winoldap is not enough to clean everything up.  There's still a Python (or Python_d) process hanging around that Ctrl+Alt+Del doesn't show.

+ This process makes it impossible to delete the associated Python .dll, and in particular makes it impossible to rebuild Python successfully without a reboot.

+ These processes cannot be killed!  Wintop and Process Viewer both fail to get the job done.  PrcView (a freeware process viewer) itself locks up if I try to kill them using it.  Process Viewer freezes for several seconds before giving up.

+ Attempting to attach to the process with the MSVC debugger (in order to find out what the heck it's doing) finds the process OK, but then yields the cryptic and undocumented error msg "Cannot execute program".

+ The processes are not accumulating cycles.

+ Smells like deadlock.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=216289&group_id=5470


From noreply@sourceforge.net  Tue Feb 25 04:05:15 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Mon, 24 Feb 2003 20:05:15 -0800
Subject: [Python-bugs-list] [ python-Feature Requests-691889 ] Add MacOS.framework attribute
Message-ID: <E18nWLD-0008RF-00@sc8-sf-web1.sourceforge.net>

Feature Requests item #691889, was opened at 2003-02-23 22:27
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=691889&group_id=5470

Category: Macintosh
Group: None
Status: Closed
Resolution: Accepted
Priority: 5
Submitted By: John Ehresman (jpe)
Assigned to: Jack Jansen (jackjansen)
Summary: Add MacOS.framework attribute

Initial Comment:
On MacOS X, python may be compiled with a named
framework and extension modules must be linked at build
time against the framework with the same name.  This
new attribute would make the name of the framework
available at runtime.

----------------------------------------------------------------------

>Comment By: John Ehresman (jpe)
Date: 2003-02-25 04:05

Message:
Logged In: YES 
user_id=22785

You're right.  I was confused about what the configure
option did.  There's no need to have the name of the
framework at runtime since it will almost always be Python.

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-02-24 09:59

Message:
Logged In: YES 
user_id=45365

You can change the location where the framework is installed, by specifying something like --enable-framework=$HOME/Library/Frameworks, but you cannot change the name of the framework, that will always be Python.framework. You can of course manually edit the Makefile, but that isn't supported behaviour.

Or am I missing something?

----------------------------------------------------------------------

Comment By: John Ehresman (jpe)
Date: 2003-02-23 23:54

Message:
Logged In: YES 
user_id=22785

The revised scheme is fine with me if the framework name is
always going to be Python.  If it is, you might want to
remove the option to set it at configure time.  If there's a
reason for someone to compile with a different framework
name, I think the name should be available at runtime.

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-02-23 23:35

Message:
Logged In: YES 
user_id=45365

Implemented in macosmodule.c rev. 1.65 in a slightly different way. There's now an attribute MacOS.linkmodel which can have one of four values:
'static', 'framework', 'shared' (not used currently, but is there for compleness if I ever get around to fixing non-framework shared-library builds) and 'cfm' (for MacPython-OS9).

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=691889&group_id=5470


From noreply@sourceforge.net  Tue Feb 25 08:04:23 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 25 Feb 2003 00:04:23 -0800
Subject: [Python-bugs-list] [ python-Bugs-692776 ] new.function() leads to segfault
Message-ID: <E18na4d-00080o-00@sc8-sf-web3.sourceforge.net>

Bugs item #692776, was opened at 2003-02-25 03:04
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=692776&group_id=5470

Category: Python Interpreter Core
Group: Python 2.2.2
Status: Open
Resolution: None
Priority: 5
Submitted By: Jp Calderone (kuran)
Assigned to: Nobody/Anonymous (nobody)
Summary: new.function() leads to segfault

Initial Comment:
If a code object which requires a closure is passed to
new.function(), a segfault (Python/ceval.c:2578).  This
seems to be fixed in 2.3.  Attached is a short code
example that demonstrates the behavior.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=692776&group_id=5470


From noreply@sourceforge.net  Tue Feb 25 08:54:15 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 25 Feb 2003 00:54:15 -0800
Subject: [Python-bugs-list] [ python-Bugs-688907 ] Applet support is broken
Message-ID: <E18naqt-0000M1-00@sc8-sf-web1.sourceforge.net>

Bugs item #688907, was opened at 2003-02-18 19:40
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=688907&group_id=5470

Category: Macintosh
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Jack Jansen (jackjansen)
Assigned to: Just van Rossum (jvr)
Summary: Applet support is broken

Initial Comment:
Applet support is broken in two ways:
- there is no sys.argv emulation anymore, and applets always see the -psnXXXX argument when started from the finder.
- when an applet is started from the commandline it is non-functional (no window manager connection).

Also, the building of the Idle applet doesn't work due to name conflicts.

----------------------------------------------------------------------

>Comment By: Just van Rossum (jvr)
Date: 2003-02-25 09:54

Message:
Logged In: YES 
user_id=92689

I have a fix ready, but as I wrote in private, this will cause sys.executable to be different (ie. it will be the same as CFBundleExecutable). This apparently breaks your stuff, as well as bundlebuilder itself if it is run from a bundle it created itself...

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-02-24 17:40

Message:
Logged In: YES 
user_id=45365

Building the Idle applet has been fixed (Mac/OSX/Makefile rev 1.36).

So now the python-based applet main program is the only thing that needs to be done.

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-02-19 00:48

Message:
Logged In: YES 
user_id=45365

Just thinks he can fix the commandline problem with a Python script, so assigning to him. Will you please also update Misc/NEWS if you do?

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-02-19 00:44

Message:
Logged In: YES 
user_id=45365

sys.argv emulation works again, and that's the main part of this bug. A note has been added to NEWS that starting from the finder is broken in 2.3a2.

Building of Idle I cannot test right now.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=688907&group_id=5470


From noreply@sourceforge.net  Tue Feb 25 09:31:10 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 25 Feb 2003 01:31:10 -0800
Subject: [Python-bugs-list] [ python-Bugs-692776 ] new.function() leads to segfault
Message-ID: <E18nbQc-0001hr-00@sc8-sf-web4.sourceforge.net>

Bugs item #692776, was opened at 2003-02-25 08:04
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=692776&group_id=5470

Category: Python Interpreter Core
Group: Python 2.2.2
Status: Open
Resolution: None
Priority: 5
Submitted By: Jp Calderone (kuran)
>Assigned to: Michael Hudson (mwh)
Summary: new.function() leads to segfault

Initial Comment:
If a code object which requires a closure is passed to
new.function(), a segfault (Python/ceval.c:2578).  This
seems to be fixed in 2.3.  Attached is a short code
example that demonstrates the behavior.


----------------------------------------------------------------------

>Comment By: Michael Hudson (mwh)
Date: 2003-02-25 09:31

Message:
Logged In: YES 
user_id=6656

You're right.

I guess the fix is to backport the extra 'closure' argument
from 2.3.  I'll look into this.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=692776&group_id=5470


From noreply@sourceforge.net  Tue Feb 25 10:43:35 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 25 Feb 2003 02:43:35 -0800
Subject: [Python-bugs-list] [ python-Bugs-539990 ] Framework.mainloop() - multiple threads
Message-ID: <E18ncYh-0002c9-00@sc8-sf-web2.sourceforge.net>

Bugs item #539990, was opened at 2002-04-05 23:29
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=539990&group_id=5470

Category: Macintosh
Group: Platform-specific
>Status: Open
Resolution: Wont Fix
Priority: 7
Submitted By: Pieter Claerhout (pclaerhout)
>Assigned to: Just van Rossum (jvr)
Summary: Framework.mainloop() - multiple threads

Initial Comment:
I tried to use the threading module to start the XML 
RPC server in a different thread (xmlrpcServer itself 
also is a threaded piece of code), but that seems to 
conflict with the mainloop. If you start this 
application, it starts the thread, starts the mainloop 
and stays in there, once you finish the mainloop, the 
thread continues.

You've stumbled on a bug in Framework.mainloop(): it 
doesn't know anything about multiple threads. It gives 
time to other applications (by calling WaitNextEvent) 
but not to other threads within Python.

Also see
http://mail.python.org/pipermail/pythonmac-sig/2002-
April/005416.html

and 

http://mail.python.org/pipermail/pythonmac-sig/2002-
April/005428.html

----------------------------------------------------------------------

>Comment By: Jack Jansen (jackjansen)
Date: 2003-02-25 11:43

Message:
Logged In: YES 
user_id=45365

If that is true I would be very happy. Could you please investigate this and put it in if it works? Before 2.3b1 please: I don't want to mess with the core of the MacOS support modules after that...

----------------------------------------------------------------------

Comment By: Just van Rossum (jvr)
Date: 2003-02-24 19:51

Message:
Logged In: YES 
user_id=92689

Jack, have you forgotten about autoGIL? It's quite likely that the autoGIL module (as part of the PyObjC project) would fix this issue out of the box on OSX. Do "import autoGIL; autoGIL.installAutoGIL()" before you enter the event loop and the second thread gets time to run.

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-02-24 17:45

Message:
Logged In: YES 
user_id=45365

Let's admit this isn't going to be fixed unless someone wants it real bad.

Pieter: if you want it real bad then reopen the bug and I'll find time for it.

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2002-04-22 16:03

Message:
Logged In: YES 
user_id=45365

After some discussion on the SIG it turns out this is a rather complicated problem, because WNE() may cause callbacks to be called (for Carbon Event handlers, for instance), and releasing the GIL would mean those callbacks would be hosed.

A solution will have to have all callback wrappers acquire the GIL, and it will either have to make very sure we never enter WNE() or friends with the GIL held, or the callback code will need to be able to handle both the case of being called with the GIL held and the case of being called without it.

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2002-04-07 23:26

Message:
Logged In: YES 
user_id=45365

Adding an optional time.sleep(0) to the mainloop should do the trick, but unfortunately this won't work with the current MacPython, as time.sleep() is implemented with select(). And while GUSI sleep(0) does the threadswitch sleect(..., 0) doesn't. So this needs to be fixed in timemodule, and then the thread switch can be forced in Framework.Application.mainloop().

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=539990&group_id=5470


From noreply@sourceforge.net  Tue Feb 25 10:48:55 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 25 Feb 2003 02:48:55 -0800
Subject: [Python-bugs-list] [ python-Bugs-688907 ] Applet support is broken
Message-ID: <E18ncdr-0005QF-00@sc8-sf-web1.sourceforge.net>

Bugs item #688907, was opened at 2003-02-18 19:40
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=688907&group_id=5470

Category: Macintosh
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Jack Jansen (jackjansen)
Assigned to: Just van Rossum (jvr)
Summary: Applet support is broken

Initial Comment:
Applet support is broken in two ways:
- there is no sys.argv emulation anymore, and applets always see the -psnXXXX argument when started from the finder.
- when an applet is started from the commandline it is non-functional (no window manager connection).

Also, the building of the Idle applet doesn't work due to name conflicts.

----------------------------------------------------------------------

>Comment By: Jack Jansen (jackjansen)
Date: 2003-02-25 11:48

Message:
Logged In: YES 
user_id=45365

Idea: we solve this for 50% and document it for 50%.

The "solve" bit involves applets that have the sys.argv support. This code is generated anyway, so we generate one extra line that sets sys.executable. It can do so, because we know what we want to put in there (bundlebuilder has just created the symlink anyway).

The document bit means that we have to state somewhere that if you create an applet without sys.argv emulation you are responsible yourself for setting it correctly. We do this in the IDE, so if we clearly flag the code that does this we've alreadly done quite a bit of "documenting".

----------------------------------------------------------------------

Comment By: Just van Rossum (jvr)
Date: 2003-02-25 09:54

Message:
Logged In: YES 
user_id=92689

I have a fix ready, but as I wrote in private, this will cause sys.executable to be different (ie. it will be the same as CFBundleExecutable). This apparently breaks your stuff, as well as bundlebuilder itself if it is run from a bundle it created itself...

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-02-24 17:40

Message:
Logged In: YES 
user_id=45365

Building the Idle applet has been fixed (Mac/OSX/Makefile rev 1.36).

So now the python-based applet main program is the only thing that needs to be done.

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-02-19 00:48

Message:
Logged In: YES 
user_id=45365

Just thinks he can fix the commandline problem with a Python script, so assigning to him. Will you please also update Misc/NEWS if you do?

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-02-19 00:44

Message:
Logged In: YES 
user_id=45365

sys.argv emulation works again, and that's the main part of this bug. A note has been added to NEWS that starting from the finder is broken in 2.3a2.

Building of Idle I cannot test right now.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=688907&group_id=5470


From noreply@sourceforge.net  Tue Feb 25 12:48:11 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 25 Feb 2003 04:48:11 -0800
Subject: [Python-bugs-list] [ python-Bugs-688266 ] IDLE does not work on Mac OS X
Message-ID: <E18neVH-0001eo-00@sc8-sf-web1.sourceforge.net>

Bugs item #688266, was opened at 2003-02-17 22:47
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=688266&group_id=5470

Category: IDLE
Group: Platform-specific
>Status: Closed
>Resolution: Fixed
Priority: 5
Submitted By: Tony Lownds (tonylownds)
Assigned to: Nobody/Anonymous (nobody)
Summary: IDLE does not work on Mac OS X

Initial Comment:
When building a framework build of python, the IDLE application that is made does not work anymore.

Here is the command that is causing trouble:

/Library/Frameworks/../../Library/Frameworks/Python.framework/Versions/2.3/Resources/Python.app/Contents/MacOS/python ../python/dist/src/Mac/scripts/BuildApplet.py --output /Library/Frameworks/../../Applications/MacPython-2.3/IDLE.app --extra ../python/dist/src/Tools/idle ../python/dist/src/Tools/idle/idle

or essentially:

python Mac/scripts/BuildApplet.py --output /Applications/MacPython-2.3/IDLE.app --extra Tools/idle Tools/idle/idle

The semantics of BuildApplet.py (well, buildtools.py) changed slightly, in a way that upset IDLE's build - it used to copy its argument (the main program file) into Resources/ under a new name - __main__.py. This changed so that the main program file keeps the same file name.

For IDLE, the main program is Tools/idle/idle but the Tools/idle directory needs to be copied into the bundle as well; before the slight semantic change in buildtools.py this worked OK but now two things named "idle" are getting copied to Resources/ and its not working.

Its hard to argue that BuildApplet really should support an --extra argument with the same basename as the main program file. Perhaps the best fix is to integrate the latest idlefork with Python 2.3 - this would eliminate this problem because the main file for mac applets is not called idle in idlefork.

Another way to fix this would be to extend BuildApplet and buildtools and bundlebuilder so that it is possible to specify a new name for the --extra argument, something like:

python Mac/scripts/BuildApplet.py --output IDLE.app --extra Tools/idle --rename-extra idlelib Tools/idle/idle


But the first fix option seems way more palateable.

----------------------------------------------------------------------

>Comment By: Jack Jansen (jackjansen)
Date: 2003-02-25 13:48

Message:
Logged In: YES 
user_id=45365

Fixed in Mac/OSX/Makefile rev. 1.36. In a slightly different way than suggested, though: BuildApplet can now have different names for source and destination for files to include in the bundle, and this is used to copy ../Tools/idle to Contents/Resources/idlelib in the bundle. This makes the original main program (idle) work again.

----------------------------------------------------------------------

Comment By: Tony Lownds (tonylownds)
Date: 2003-02-19 19:03

Message:
Logged In: YES 
user_id=24100

Updated macosx_main.py - since Jack added an argvemulation wrapper, macosx_main.py needs no special code and is the same as Tools/idle/idle

----------------------------------------------------------------------

Comment By: Tony Lownds (tonylownds)
Date: 2003-02-17 23:33

Message:
Logged In: YES 
user_id=24100

I think I have a found a good solution. Instead of using Tools/idle/idle as the main program argument to BuildApplet.py, the patch to Mac/OSX/Makefile below uses Tools/idle/macosx_main.py, which is attached separately.

The macosx_main.py file I provide is a copy of Tools/idle/idle with some code to work around another recent problem with running IDLE on Mac OS X. The new code removes some funny arguments that Mac OS X adds when GUI applications are launched. This same code can be found in Lib/plat-mac/argvemulator.py. I would have used argvemulator.py directly but I don't know if that would have other consequences. 

The patch to Mac/OSX/Makefile will be necessary when idlefork is integrated back into python. That is the reason for naming the new file macosx_main.py

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=688266&group_id=5470


From noreply@sourceforge.net  Tue Feb 25 12:50:52 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 25 Feb 2003 04:50:52 -0800
Subject: [Python-bugs-list] [ python-Bugs-677753 ] python.exe expected in extension builds
Message-ID: <E18neXs-0001kd-00@sc8-sf-web1.sourceforge.net>

Bugs item #677753, was opened at 2003-01-30 22:00
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=677753&group_id=5470

Category: Macintosh
Group: Python 2.3
>Status: Closed
>Resolution: Fixed
Priority: 5
Submitted By: Robin Dunn (robind)
Assigned to: Jack Jansen (jackjansen)
Summary: python.exe expected in extension builds

Initial Comment:
When building on OS X configure senses whether the
build filesystem is case sensitive or not and if not
builds "python.exe" instead of "python" to avoid
conflicting with the "Python" dir.

There are some things in doing the framework build that
expect to have "python.exe" so i fyou ar building on a
case-sensitive filesystem and only have "python" then
they fail.  Perhaps configure should make a python.exe
--> python link.

----------------------------------------------------------------------

>Comment By: Jack Jansen (jackjansen)
Date: 2003-02-25 13:50

Message:
Logged In: YES 
user_id=45365

Fixed in Makefile.pre.in 1.114, Mac/OSX/Makefile 1.37.

----------------------------------------------------------------------

Comment By: Robin Dunn (robind)
Date: 2003-01-31 17:22

Message:
Logged In: YES 
user_id=53955

Sorry, I meant to do that but must have been sidetracked...
 Here is the first one:

./python.exe /projects/Python-2.3a1/Mac/scripts/cachersrc.py
-v
/projects/_py/install/Library/Frameworks/Python.framework/Versions/2.3/lib/python2.3/plat-mac
/projects/_py/install/Library/Frameworks/Python.framework/Versions/2.3/Mac/Tools
make[1]: ./python.exe: Command not found
make[1]: *** [installmacsubtree] Error 127
make: *** [frameworkinstallmaclib] Error 2


ISTR that after making a link and running make again that
there were a few more "python.exe"'s that floated by after
this one too.


----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-01-31 10:04

Message:
Logged In: YES 
user_id=45365

Robin, could you give an example of something that fails?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=677753&group_id=5470


From noreply@sourceforge.net  Tue Feb 25 13:09:35 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 25 Feb 2003 05:09:35 -0800
Subject: [Python-bugs-list] [ python-Bugs-682317 ] plistlib.py selftest fails
Message-ID: <E18nepz-0003ek-00@sc8-sf-web3.sourceforge.net>

Bugs item #682317, was opened at 2003-02-07 14:46
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=682317&group_id=5470

Category: Macintosh
Group: None
>Status: Closed
>Resolution: Fixed
Priority: 5
Submitted By: Jack Jansen (jackjansen)
Assigned to: Just van Rossum (jvr)
Summary: plistlib.py selftest fails

Initial Comment:
Running the plistlib selftest fails for me: it uses xml.utils.iso8601, which apparently isn't part of the standard library.

----------------------------------------------------------------------

>Comment By: Jack Jansen (jackjansen)
Date: 2003-02-25 14:09

Message:
Logged In: YES 
user_id=45365

I've added a test_plistlib test, which optionally does the Date() test, it skips it if it can't import xml.utils.iso8601.

----------------------------------------------------------------------

Comment By: Just van Rossum (jvr)
Date: 2003-02-07 17:04

Message:
Logged In: YES 
user_id=92689

I'd like to give this a go at _some_ point, but not now ;-)

Btw. I see xmlrpclib.py does writing of iso8601 dates by itself, but doesn't read them. It's been a while since I read xml.utils.iso8601.py, but if it's not all that much code, perhaps we should just duplicate it in plistlib.py. I have no great use for it, though, plistlib.py is/was mostly targeted to deal with Info.plist files, in which dates are fairly rare...

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-02-07 16:55

Message:
Logged In: YES 
user_id=45365

Ah, I see.

I'd like to turn it into a standard test (to go into test.test_plistlib). Then it can also easily test Date() only if the prerequisites are available.

Are you willing to give this a go? If not reassign to me and I'll get to it eventually.

----------------------------------------------------------------------

Comment By: Just van Rossum (jvr)
Date: 2003-02-07 16:04

Message:
Logged In: YES 
user_id=92689

It's part of PyXML. The doc string explains that Date support depends on PyXML. The test is only an onformal test, would you like to formalize it? Or make Date testing optional?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=682317&group_id=5470


From noreply@sourceforge.net  Tue Feb 25 13:19:46 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 25 Feb 2003 05:19:46 -0800
Subject: [Python-bugs-list] [ python-Bugs-677293 ] Can't build C ext on OS X with 'altinstall' python
Message-ID: <E18nezq-0002n5-00@sc8-sf-web1.sourceforge.net>

Bugs item #677293, was opened at 2003-01-30 07:15
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=677293&group_id=5470

Category: Distutils
Group: Platform-specific
Status: Open
Resolution: None
Priority: 5
Submitted By: Stuart Bishop (zenzen)
>Assigned to: Jack Jansen (jackjansen)
Summary: Can't build C ext on OS X with 'altinstall' python

Initial Comment:
I think this is OS X specific, but I may be wrong. This also may be a fault of the build/installation scripts and not distutils.

After building python 2.3, and installing with 'make altinstall', I am unable to install packages with 'distutils'. gcc is being told to get stuff out of 'python', which doesn't exist (ie. the final component of the python interpreter's path is wrong).

gcc -bundle -bundle_loader /usr/local/stow/python-2.3/bin/python build/temp.darwin-6.3-Power_Macintosh-2.3/_PosixTimeZone.o -o build/lib.darwin-6.3-Power_Macintosh-2.3/_PosixTimeZone.so
ld: can't open: /usr/local/stow/python-2.3/bin/python (No such file or directory, errno = 2)
error: command 'gcc' failed with exit status 1
make: *** [build] Error 1

----------------------------------------------------------------------

>Comment By: Jack Jansen (jackjansen)
Date: 2003-02-25 14:19

Message:
Logged In: YES 
user_id=45365

The problem is that the Makefile LDFLAGS specify the bundle loader (the hosting main program) as $(bindir)/$(PYTHON), which doesn't exist after an altinstall. I'll fix this to refer to the altbininstall filename, which should always exist.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=677293&group_id=5470


From noreply@sourceforge.net  Tue Feb 25 13:24:22 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 25 Feb 2003 05:24:22 -0800
Subject: [Python-bugs-list] [ python-Bugs-677293 ] Can't build C ext on OS X with 'altinstall' python
Message-ID: <E18nf4I-0002rN-00@sc8-sf-web1.sourceforge.net>

Bugs item #677293, was opened at 2003-01-30 07:15
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=677293&group_id=5470

Category: Distutils
Group: Platform-specific
>Status: Closed
>Resolution: Fixed
Priority: 5
Submitted By: Stuart Bishop (zenzen)
Assigned to: Jack Jansen (jackjansen)
Summary: Can't build C ext on OS X with 'altinstall' python

Initial Comment:
I think this is OS X specific, but I may be wrong. This also may be a fault of the build/installation scripts and not distutils.

After building python 2.3, and installing with 'make altinstall', I am unable to install packages with 'distutils'. gcc is being told to get stuff out of 'python', which doesn't exist (ie. the final component of the python interpreter's path is wrong).

gcc -bundle -bundle_loader /usr/local/stow/python-2.3/bin/python build/temp.darwin-6.3-Power_Macintosh-2.3/_PosixTimeZone.o -o build/lib.darwin-6.3-Power_Macintosh-2.3/_PosixTimeZone.so
ld: can't open: /usr/local/stow/python-2.3/bin/python (No such file or directory, errno = 2)
error: command 'gcc' failed with exit status 1
make: *** [build] Error 1

----------------------------------------------------------------------

>Comment By: Jack Jansen (jackjansen)
Date: 2003-02-25 14:24

Message:
Logged In: YES 
user_id=45365

Fixed in configure.in rev 1.392, configure rev. 1.381.

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-02-25 14:19

Message:
Logged In: YES 
user_id=45365

The problem is that the Makefile LDFLAGS specify the bundle loader (the hosting main program) as $(bindir)/$(PYTHON), which doesn't exist after an altinstall. I'll fix this to refer to the altbininstall filename, which should always exist.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=677293&group_id=5470


From noreply@sourceforge.net  Tue Feb 25 13:28:45 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 25 Feb 2003 05:28:45 -0800
Subject: [Python-bugs-list] [ python-Bugs-586998 ] OSA Python integration
Message-ID: <E18nf8X-0002v0-00@sc8-sf-web1.sourceforge.net>

Bugs item #586998, was opened at 2002-07-26 15:20
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=586998&group_id=5470

Category: Macintosh
Group: Feature Request
>Status: Closed
>Resolution: Postponed
Priority: 2
Submitted By: Jack Jansen (jackjansen)
Assigned to: Jack Jansen (jackjansen)
Summary: OSA Python integration

Initial Comment:
Bill Fancher has a wonderful thingy for MacOSX: an OSA component that allows you to use Python as the scripting language in stead of AppleScript. It is available via http://home.pacbell.net/fancher/OSA_Python/OSA_Python_Notes_1_0a5.html, but currently only in binary form (because he isn't satisfied with it yet).

At some point it should be made easy for Mac{ho}Python users to download and install this and/or it should be incorporated in the standard distribution.

----------------------------------------------------------------------

>Comment By: Jack Jansen (jackjansen)
Date: 2003-02-25 14:28

Message:
Logged In: YES 
user_id=45365

Unfortunately Bill is busy with more important things right now, and he hasn't managed to pass on his code to me. So I'm closing this because any progress has to wait for his return.

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2002-07-29 11:16

Message:
Logged In: YES 
user_id=6656

Yes please!

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=586998&group_id=5470


From noreply@sourceforge.net  Tue Feb 25 13:34:20 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 25 Feb 2003 05:34:20 -0800
Subject: [Python-bugs-list] [ python-Bugs-625725 ] Reorganize MacPython resources on OSX
Message-ID: <E18nfDw-0002aE-00@sc8-sf-web2.sourceforge.net>

Bugs item #625725, was opened at 2002-10-19 23:13
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=625725&group_id=5470

Category: Macintosh
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Jack Jansen (jackjansen)
Assigned to: Just van Rossum (jvr)
Summary: Reorganize MacPython resources on OSX

Initial Comment:
The resource file for Python on OSX should be reorganized. Some of the resources are only used by specific modules such as EasyDialogs, these should go into private resource files. Some are used by the interpreter itself (The "Estr" resource that maps error numbers to strings comes to mind). The most obvious place for these would be the resource file of the framework, but these may also be useful in non-framework builds (where you now get the uninformative 'MacOS.Error: Error -1234').

----------------------------------------------------------------------

>Comment By: Jack Jansen (jackjansen)
Date: 2003-02-25 14:34

Message:
Logged In: YES 
user_id=45365

Just, this error report has drifted quite a bit from the original intention: it was about splitting the .rsrc files into separate files living with the modules they belong to, but now it is about phasing out .rsrc files.

The former has been finished a while ago, how about closing this bug and opening a new one with a better title and description?

----------------------------------------------------------------------

Comment By: Just van Rossum (jvr)
Date: 2003-01-06 23:22

Message:
Logged In: YES 
user_id=92689

Ok, I've played around a bit and it's certainly doable: I
have a simple dialog running from a nib. It would still be a
fairly major rewrite of EasyDialogs as DialogRef's are not
supported in IBCarbon: must use straight windows.

One thing that's missing is the ability to load nibs from a
path, as opposed to finding them by name from an app bundle:
for this we need to create
Carbon.IBCarbon.CreateNibReferenceWithCFBundle(), and for
that to work we need CFBundle support in Carbon.CF.

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-01-06 13:04

Message:
Logged In: YES 
user_id=45365

Yes, if you want to put some effort into this: please do!

----------------------------------------------------------------------

Comment By: Just van Rossum (jvr)
Date: 2003-01-06 12:33

Message:
Logged In: YES 
user_id=92689

Wait, there already is Carbon.IBCarbon, cool! Shall I try to
find out what it takes to convert EasyDialogs to use it?

----------------------------------------------------------------------

Comment By: Just van Rossum (jvr)
Date: 2003-01-06 12:24

Message:
Logged In: YES 
user_id=92689

Ok, I checked in a change to macresource.py to this effect
(it simply adds sys.path to searchdirs, after
os.path.dirname(__file__).

Apart from Estr, how about trying to use nibs instead of res
files? I think this is meant to work for OS9 as well, but
would require interfacing some *FromNib API's in Carbon.Win
and Carbon.Dlg. Just a thought.

Either way, we should use a strict naming convention, eg.
EasyDialogs.py's resources should be in a file called
EasyDialogs.rsrc (and/or EasyDialogs.nib if we take the
above route).

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-01-06 12:02

Message:
Logged In: YES 
user_id=45365

Okay with me, as long as you look on sys.path *after* checking the __file__. So the order would be
1) Check wheter it happens to be available already
2) Check whether __file__ gives a clue
3) check sys.path

An alternative solution would be that bundlebuilder takes the .rsrc file for each module automatically and combines them all (including errors.rsrc)into the resource file that is automatically opened when the .app starts (I think it's <executablename>.rsrc, check the apple docs).

A Python module in stead of the Estr resource I'm not too thrilled about: there are a lot of error messages, and if it was a Python module then they would all be in-core once you import the module.

I'm assigning this over to you, close it when you're done (the rest of the reorganization is finished).

----------------------------------------------------------------------

Comment By: Just van Rossum (jvr)
Date: 2003-01-05 23:08

Message:
Logged In: YES 
user_id=92689

The Estr resource: is this resource generated, and if so
how? Would it be  an idea to generate a Python module instead?

The macresource module also has issues: eg. for EasyDialogs
it looks for "dialogs.rsrc" in
os.path.dirname(EasyDialogs.__file__), which doesn't work
when EasyDialogs is loaded from a zip file. If it would
(additionally) look on sys.path it could work for standalone
apps created by bundlebuilder: it should simply be included
in <appname>.app/Contents/Resources/ then.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=625725&group_id=5470


From noreply@sourceforge.net  Tue Feb 25 13:35:16 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 25 Feb 2003 05:35:16 -0800
Subject: [Python-bugs-list] [ python-Bugs-608085 ] IDE - Breakpoints don't stick to lines
Message-ID: <E18nfEq-0002e7-00@sc8-sf-web2.sourceforge.net>

Bugs item #608085, was opened at 2002-09-12 00:04
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=608085&group_id=5470

Category: Macintosh
Group: None
>Status: Closed
>Resolution: Wont Fix
Priority: 5
Submitted By: Jack Jansen (jackjansen)
Assigned to: Just van Rossum (jvr)
Summary: IDE - Breakpoints don't stick to lines

Initial Comment:
Breakpoints don't always seem to stick to the line they are set on in case of edits.

Here's a scenario.

In an edit window, enable debugging. Set a breakpoint on, say, line 10. Disable debugging. Delete line 5. Enable debugging again. See how the breakpoint is still on line 10 in stead of having moved to line 9.

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2002-09-12 21:36

Message:
Logged In: YES 
user_id=45365

Hmm, yes, with Waste handling our text editing it is indeed non-trivial, I hadn't thought of that.

How about sticking a custom waste object into the text stream you feed to waste? Although that would complicate other matters and cause ideosyncracies when copy/pasting to other apps....

Ok, let's document it and call it a feature:-)

----------------------------------------------------------------------

Comment By: Just van Rossum (jvr)
Date: 2002-09-12 09:05

Message:
Logged In: YES 
user_id=92689

Would be nice. If you have any idea how to implement it, I'm all ears...

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=608085&group_id=5470


From noreply@sourceforge.net  Tue Feb 25 13:37:33 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 25 Feb 2003 05:37:33 -0800
Subject: [Python-bugs-list] [ python-Feature Requests-692881 ] import files from any directory
Message-ID: <E18nfH3-0002nG-00@sc8-sf-web2.sourceforge.net>

Feature Requests item #692881, was opened at 2003-02-25 15:37
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=692881&group_id=5470

Category: Python Interpreter Core
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Noam Raphael (noamr)
Assigned to: Nobody/Anonymous (nobody)
Summary: import files from any directory

Initial Comment:
Many times I have a .py file with some functions, which
I want to import. However, its functionality is very
specific, so I don't want to copy it to a directory
where it will always be available as a module. If I
start Python from the directory of the file, I can
import it, but this is not always the case. I can use
execfile(), but import is better since it runs the file
in its own namespace.

What I suggest is to extend the import statement
syntax, to allow  "module" to be a stringliteral as
well as (identifier ".")* identifier. When it is a
string literal, the string will be interpreted as the
file name to import. The default name of the module
will be the name which would import the file if its
directory were in PYTHONPATH. If there is no such name,
an exception will be raised if "as" is not used.

For example, say the file with my functions is called
"specific_functions.py" and sits in the directory
"/dir". When I open the Python shell from the directory
/dir, I can now call "import specific_functions". I
suggest that this would do the same thing: "import
'specific_functions.py' ". When I open the Python shell
from another directory, I currently have no simple way
to import the file, but I suggest that I could use the
command "import '/dir/specific_functions.py' ". The
command "from '/dir/specific_functions.py' import *"
should also work, of course.

This has no backward compatibility issues as far as I
can see, and is quite helpful, I think.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=692881&group_id=5470


From noreply@sourceforge.net  Tue Feb 25 13:43:12 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 25 Feb 2003 05:43:12 -0800
Subject: [Python-bugs-list] [ python-Bugs-692884 ] python always searches python23.zip
Message-ID: <E18nfMW-0003NE-00@sc8-sf-web1.sourceforge.net>

Bugs item #692884, was opened at 2003-02-25 07:43
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=692884&group_id=5470

Category: Python Interpreter Core
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Skip Montanaro (montanaro)
Assigned to: Just van Rossum (jvr)
Summary: python always searches python23.zip

Initial Comment:
In trying to figure out why startup of the
spambayes hammiefilter seems so slow, I ran it
under the control of ktrace (also known as strace
or truss on other systems).  I noticed that even
though it's not in sys.path, the interpreter is
always searching for modules in
$prefix/python23.zip.  Why's that?  Here's a
little ktrace/kdump output:

  2361 python   CALL  stat(0xbfffe1d0,0xbfffe5e0)
  2361 python   NAMI  "/Users/skip/local/lib/python23.zip"
  2361 python   RET   stat -1 errno 2 No such file or directory
  2361 python   CALL  stat(0xbfffe1d0,0xbfffe5e0)
  2361 python   NAMI  "/Users/skip/local/lib"
  2361 python   RET   stat 0
  2361 python   CALL  stat(0xbfffece0,0xbfffe7d0)
  2361 python   NAMI  "/Users/skip/local/lib/python23.zip/warnings"
  2361 python   RET   stat -1 errno 2 No such file or directory
  2361 python   CALL  open(0xbfffece0,0,0x1b6)
  2361 python   NAMI  "/Users/skip/local/lib/python23.zip/warnings.py"
  2361 python   RET   open -1 errno 2 No such file or directory
  2361 python   CALL  open(0xbfffece0,0,0x1b6)
  2361 python   NAMI  "/Users/skip/local/lib/python23.zip/warnings.pyc"
  2361 python   RET   open -1 errno 2 No such file or directory
  2361 python   CALL  open(0xbfffece0,0,0x1b6)
  2361 python   NAMI  "/Users/skip/local/lib/python23.zip/warnings.so"
  2361 python   RET   open -1 errno 2 No such file or directory
  2361 python   CALL  open(0xbfffece0,0,0x1b6)
  2361 python   NAMI  "/Users/skip/local/lib/python23.zip/warningsmodule.so"
  2361 python   RET   open -1 errno 2 No such file or directory

What's particularly odd is that even though the
search for python23.zip fails, it continues to
march along looking for files within the
nonexistent zip file.

Searching for modules in something not in sys.path
seems like a bug to me.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=692884&group_id=5470


From noreply@sourceforge.net  Tue Feb 25 13:44:15 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 25 Feb 2003 05:44:15 -0800
Subject: [Python-bugs-list] [ python-Bugs-644243 ] ic module "path too long" error
Message-ID: <E18nfNX-0003FY-00@sc8-sf-web2.sourceforge.net>

Bugs item #644243, was opened at 2002-11-26 18:49
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=644243&group_id=5470

Category: Macintosh
Group: Python 2.2.2
>Status: Closed
>Resolution: Fixed
Priority: 5
Submitted By: Russell Owen (reowen)
Assigned to: Jack Jansen (jackjansen)
>Summary: ic module "path too long" error

Initial Comment:
The IC module returns a path too long error for file:/localhost type URLs. Here is an example:

# this works:
>>> webbrowser.open("file:///Users/rowen/PythonRO/TUI/TUIHelp/Telescope.html#CoordSys")

# this fails, though URLs of this form are accepted on unix:
>>> webbrowser.open("file:/Users/rowen/PythonRO/TUI/TUIHelp/Telescope.html#CoordSys")
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "/Library/Frameworks/Python.framework/Versions/2.2/lib/python2.2/webbrowser.py", line 43, in open
    get().open(url, new, autoraise)
  File "/Library/Frameworks/Python.framework/Versions/2.2/lib/python2.2/webbrowser.py", line 275, in open
    ic.launchurl(url)
  File "/Library/Frameworks/Python.framework/Versions/2.2/Mac/Lib/ic.py", line 231, in launchurl
    return _dft_ic.launchurl(url, hint)
  File "/Library/Frameworks/Python.framework/Versions/2.2/Mac/Lib/ic.py", line 194, in launchurl
    self.ic.ICLaunchURL(hint, url, 0, len(url))
MacOS.Error: (-2110, 'pathTooLongErr')


----------------------------------------------------------------------

>Comment By: Jack Jansen (jackjansen)
Date: 2003-02-25 14:44

Message:
Logged In: YES 
user_id=45365

I think this is actually an Apple bug, but I've implemented a workaround in ic.py rev. 1.4: URLs of the second form are converted to the first form before being passed to ICLaunchURL.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=644243&group_id=5470


From noreply@sourceforge.net  Tue Feb 25 14:28:44 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 25 Feb 2003 06:28:44 -0800
Subject: [Python-bugs-list] [ python-Bugs-635570 ] remove debug prints from macmain.c
Message-ID: <E18ng4a-00061c-00@sc8-sf-web1.sourceforge.net>

Bugs item #635570, was opened at 2002-11-08 17:24
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=635570&group_id=5470

Category: Macintosh
Group: Python 2.2.2
>Status: Closed
>Resolution: Fixed
Priority: 5
Submitted By: Russell Owen (reowen)
Assigned to: Jack Jansen (jackjansen)
Summary: remove debug prints from macmain.c

Initial Comment:
I posted to the MacPython mailing list:

pythonw does the following when run with a source file as its argument:
% pythonw TUI.py
original argc=2
original argv[0] = "/Applications/Python.app/Contents/MacOS/python"
original argv[1] = "TUI.py"
modified argc=2
modified argv[0] = "/Applications/Python.app/Contents/MacOS/python"
modified argv[1] = "TUI.py"

Details:
- Everything is built as per the instructions I posted at http://www.astro.washington.edu/owen in particular:
- MacOS X 10.2
- Python 2.2.2 framework built from source, with setup.py modified to be similar to or identical to the 2.3a0 setup.py, so it can find aqua Tk 8.4.1
- The file pythonw is:
#!/bin/sh
exec /Applications/Python.app/Contents/MacOS/python $@

Jack Jansen kindly suggested that I could fix this by removing the debug prints from macmain.c and rebuilding. He also asked me to file this bug report.

----------------------------------------------------------------------

>Comment By: Jack Jansen (jackjansen)
Date: 2003-02-25 15:28

Message:
Logged In: YES 
user_id=45365

Fixed in Mac/Python/macmain.c rev. 1.82.8.3.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=635570&group_id=5470


From noreply@sourceforge.net  Tue Feb 25 14:30:18 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 25 Feb 2003 06:30:18 -0800
Subject: [Python-bugs-list] [ python-Bugs-652590 ] dynload_next needs better errors
Message-ID: <E18ng66-00067H-00@sc8-sf-web1.sourceforge.net>

Bugs item #652590, was opened at 2002-12-12 11:38
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=652590&group_id=5470

Category: Macintosh
Group: Python 2.2.2
>Status: Closed
>Resolution: Fixed
Priority: 5
Submitted By: Jack Jansen (jackjansen)
Assigned to: Jack Jansen (jackjansen)
Summary: dynload_next needs better errors

Initial Comment:
Dynload_next needs to give better error messages. The code in rev. 2.11 should give an idea.

----------------------------------------------------------------------

>Comment By: Jack Jansen (jackjansen)
Date: 2003-02-25 15:30

Message:
Logged In: YES 
user_id=45365

Fixed in rev 2.10.6.1.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=652590&group_id=5470


From noreply@sourceforge.net  Tue Feb 25 15:07:23 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 25 Feb 2003 07:07:23 -0800
Subject: [Python-bugs-list] [ python-Bugs-692884 ] python always searches python23.zip
Message-ID: <E18ngfz-0008FD-00@sc8-sf-web4.sourceforge.net>

Bugs item #692884, was opened at 2003-02-25 14:43
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=692884&group_id=5470

Category: Python Interpreter Core
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Skip Montanaro (montanaro)
Assigned to: Just van Rossum (jvr)
Summary: python always searches python23.zip

Initial Comment:
In trying to figure out why startup of the
spambayes hammiefilter seems so slow, I ran it
under the control of ktrace (also known as strace
or truss on other systems).  I noticed that even
though it's not in sys.path, the interpreter is
always searching for modules in
$prefix/python23.zip.  Why's that?  Here's a
little ktrace/kdump output:

  2361 python   CALL  stat(0xbfffe1d0,0xbfffe5e0)
  2361 python   NAMI  "/Users/skip/local/lib/python23.zip"
  2361 python   RET   stat -1 errno 2 No such file or directory
  2361 python   CALL  stat(0xbfffe1d0,0xbfffe5e0)
  2361 python   NAMI  "/Users/skip/local/lib"
  2361 python   RET   stat 0
  2361 python   CALL  stat(0xbfffece0,0xbfffe7d0)
  2361 python   NAMI  "/Users/skip/local/lib/python23.zip/warnings"
  2361 python   RET   stat -1 errno 2 No such file or directory
  2361 python   CALL  open(0xbfffece0,0,0x1b6)
  2361 python   NAMI  "/Users/skip/local/lib/python23.zip/warnings.py"
  2361 python   RET   open -1 errno 2 No such file or directory
  2361 python   CALL  open(0xbfffece0,0,0x1b6)
  2361 python   NAMI  "/Users/skip/local/lib/python23.zip/warnings.pyc"
  2361 python   RET   open -1 errno 2 No such file or directory
  2361 python   CALL  open(0xbfffece0,0,0x1b6)
  2361 python   NAMI  "/Users/skip/local/lib/python23.zip/warnings.so"
  2361 python   RET   open -1 errno 2 No such file or directory
  2361 python   CALL  open(0xbfffece0,0,0x1b6)
  2361 python   NAMI  "/Users/skip/local/lib/python23.zip/warningsmodule.so"
  2361 python   RET   open -1 errno 2 No such file or directory

What's particularly odd is that even though the
search for python23.zip fails, it continues to
march along looking for files within the
nonexistent zip file.

Searching for modules in something not in sys.path
seems like a bug to me.

----------------------------------------------------------------------

>Comment By: Just van Rossum (jvr)
Date: 2003-02-25 16:07

Message:
Logged In: YES 
user_id=92689

(You probably could have found this with python -vv ;-)

Since Python 2.3a1, site.py no longer removes sys.path items that don't exist on the file system, because it may be (eg.) a URL, handled by a custom importer. PEP 273 explains why a standard zip entry must be on sys.path.

The problem is that the PEP 302 mechanism doesn't make a difference between "no importer found for this sys.path entry" and "use the builtin path importer". This will be fixed in "phase 2" of PEP 302, which consits of adding an explicit importer object on sys.meta_path representing the builtin path importer. (I should really do this before 2.3b1, this report should serve as a reminder for me.)

That said, Jack _yesterday_ fixed a problem on OSX which caused importing Mac modules to be extremely slow (a delay of *seconds*). If spambayes happens to import a Mac module on OSX this would have affected you. Rebuilding from current CVS would help then.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=692884&group_id=5470


From noreply@sourceforge.net  Tue Feb 25 15:16:26 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 25 Feb 2003 07:16:26 -0800
Subject: [Python-bugs-list] [ python-Bugs-692951 ] Python 2.3a2 make test fails timeout
Message-ID: <E18ngok-0008Oq-00@sc8-sf-web2.sourceforge.net>

Bugs item #692951, was opened at 2003-02-25 15:16
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=692951&group_id=5470

Category: Build
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Charles Hixson (quixo)
Assigned to: Nobody/Anonymous (nobody)
Summary: Python 2.3a2 make test fails timeout

Initial Comment:
after:
./config
make
I decided to do:
make test
This mainly worked, but ended up like this:
221 tests OK.
1 test failed:
    test_timeout
22 tests skipped:
    test_aepack test_al test_bsddb3 test_bz2 test_cd
test_cl
    test_curses test_email_codecs test_gl test_imgfile
    test_linuxaudiodev test_macfs test_macostools test_nis
    test_normalization test_ossaudiodev test_pep277
test_socket_ssl
    test_socketserver test_sunaudiodev test_winreg
test_winsound
1 skip unexpected on linux2:
    test_bz2
make: *** [test] Error 1

See attached file for a more complete list of the run.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=692951&group_id=5470


From noreply@sourceforge.net  Tue Feb 25 15:15:36 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 25 Feb 2003 07:15:36 -0800
Subject: [Python-bugs-list] [ python-Bugs-692884 ] python always searches python23.zip
Message-ID: <E18ngnw-0008Lz-00@sc8-sf-web2.sourceforge.net>

Bugs item #692884, was opened at 2003-02-25 07:43
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=692884&group_id=5470

Category: Python Interpreter Core
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Skip Montanaro (montanaro)
Assigned to: Just van Rossum (jvr)
Summary: python always searches python23.zip

Initial Comment:
In trying to figure out why startup of the
spambayes hammiefilter seems so slow, I ran it
under the control of ktrace (also known as strace
or truss on other systems).  I noticed that even
though it's not in sys.path, the interpreter is
always searching for modules in
$prefix/python23.zip.  Why's that?  Here's a
little ktrace/kdump output:

  2361 python   CALL  stat(0xbfffe1d0,0xbfffe5e0)
  2361 python   NAMI  "/Users/skip/local/lib/python23.zip"
  2361 python   RET   stat -1 errno 2 No such file or directory
  2361 python   CALL  stat(0xbfffe1d0,0xbfffe5e0)
  2361 python   NAMI  "/Users/skip/local/lib"
  2361 python   RET   stat 0
  2361 python   CALL  stat(0xbfffece0,0xbfffe7d0)
  2361 python   NAMI  "/Users/skip/local/lib/python23.zip/warnings"
  2361 python   RET   stat -1 errno 2 No such file or directory
  2361 python   CALL  open(0xbfffece0,0,0x1b6)
  2361 python   NAMI  "/Users/skip/local/lib/python23.zip/warnings.py"
  2361 python   RET   open -1 errno 2 No such file or directory
  2361 python   CALL  open(0xbfffece0,0,0x1b6)
  2361 python   NAMI  "/Users/skip/local/lib/python23.zip/warnings.pyc"
  2361 python   RET   open -1 errno 2 No such file or directory
  2361 python   CALL  open(0xbfffece0,0,0x1b6)
  2361 python   NAMI  "/Users/skip/local/lib/python23.zip/warnings.so"
  2361 python   RET   open -1 errno 2 No such file or directory
  2361 python   CALL  open(0xbfffece0,0,0x1b6)
  2361 python   NAMI  "/Users/skip/local/lib/python23.zip/warningsmodule.so"
  2361 python   RET   open -1 errno 2 No such file or directory

What's particularly odd is that even though the
search for python23.zip fails, it continues to
march along looking for files within the
nonexistent zip file.

Searching for modules in something not in sys.path
seems like a bug to me.

----------------------------------------------------------------------

>Comment By: Skip Montanaro (montanaro)
Date: 2003-02-25 09:15

Message:
Logged In: YES 
user_id=44345

If $prefix/python23.zip must always be present and is always going to be
searched (and searched ahead of $prefix/python2.3, why not have the
installation process create it with the .pyo contents of $prefix/python2.3?

(Hammiefilter doesn't import any Mac modules, so the problem Jack
fixed shouldn't affect this case.)

----------------------------------------------------------------------

Comment By: Just van Rossum (jvr)
Date: 2003-02-25 09:07

Message:
Logged In: YES 
user_id=92689

(You probably could have found this with python -vv ;-)

Since Python 2.3a1, site.py no longer removes sys.path items that don't exist on the file system, because it may be (eg.) a URL, handled by a custom importer. PEP 273 explains why a standard zip entry must be on sys.path.

The problem is that the PEP 302 mechanism doesn't make a difference between "no importer found for this sys.path entry" and "use the builtin path importer". This will be fixed in "phase 2" of PEP 302, which consits of adding an explicit importer object on sys.meta_path representing the builtin path importer. (I should really do this before 2.3b1, this report should serve as a reminder for me.)

That said, Jack _yesterday_ fixed a problem on OSX which caused importing Mac modules to be extremely slow (a delay of *seconds*). If spambayes happens to import a Mac module on OSX this would have affected you. Rebuilding from current CVS would help then.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=692884&group_id=5470


From noreply@sourceforge.net  Tue Feb 25 15:16:34 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 25 Feb 2003 07:16:34 -0800
Subject: [Python-bugs-list] [ python-Bugs-571343 ] Mixing framework and static Pythons
Message-ID: <E18ngor-0008Bw-00@sc8-sf-web1.sourceforge.net>

Bugs item #571343, was opened at 2002-06-19 23:49
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=571343&group_id=5470

Category: Macintosh
>Group: Python 2.2.2
Status: Open
Resolution: None
Priority: 5
Submitted By: Jack Jansen (jackjansen)
Assigned to: Jack Jansen (jackjansen)
Summary: Mixing framework and static Pythons

Initial Comment:
We need to check that (on OS X) the error message you get when you attempt to load an extension module in the "wrong" Python (i.e. an extension module built for a static Python into a framework python or vice versa) is of a form understandable to naive users, and not of the "undefined external: foobar" variety. As the situation of having both Pythons on your system may well become commonplace this needs to be addressed in 2.2.X too.


----------------------------------------------------------------------

>Comment By: Jack Jansen (jackjansen)
Date: 2003-02-25 16:16

Message:
Logged In: YES 
user_id=45365

In Python 2.3 this doesn't seem to be a problem, you always get an error message on import if there is a problem.

In 2.2 it is a problem, which can be repeated as follows:
- Build and install a framework 2.2.X
- Add /Library/Framework/Python.framework/..../lib/python2.2 and .../lib-dynload to sys.path (at the beginning)
- run /usr/bin/python (version 2.2)
- run import test.regrtest; test.regrtest.main()
- witness weird errors culminating in a crash.

The reverse (loading non-framework modules into a framework 2.2 python)
seems to work okay.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=571343&group_id=5470


From noreply@sourceforge.net  Tue Feb 25 15:17:42 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 25 Feb 2003 07:17:42 -0800
Subject: [Python-bugs-list] [ python-Bugs-607810 ] IDE should have "open recent" menu
Message-ID: <E18ngpy-0008HB-00@sc8-sf-web1.sourceforge.net>

Bugs item #607810, was opened at 2002-09-11 15:19
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=607810&group_id=5470

Category: Macintosh
Group: None
>Status: Closed
>Resolution: Fixed
Priority: 5
Submitted By: Jack Jansen (jackjansen)
Assigned to: Just van Rossum (jvr)
>Summary: IDE should have "open recent" menu

Initial Comment:
The IDE needs an "open recent" menu, like most Mac programs. 

----------------------------------------------------------------------

>Comment By: Jack Jansen (jackjansen)
Date: 2003-02-25 16:17

Message:
Logged In: YES 
user_id=45365

Fixed in CVS.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=607810&group_id=5470


From noreply@sourceforge.net  Tue Feb 25 15:21:15 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 25 Feb 2003 07:21:15 -0800
Subject: [Python-bugs-list] [ python-Bugs-688907 ] Applet support is broken
Message-ID: <E18ngtP-0008UP-00@sc8-sf-web1.sourceforge.net>

Bugs item #688907, was opened at 2003-02-18 19:40
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=688907&group_id=5470

Category: Macintosh
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Jack Jansen (jackjansen)
Assigned to: Just van Rossum (jvr)
Summary: Applet support is broken

Initial Comment:
Applet support is broken in two ways:
- there is no sys.argv emulation anymore, and applets always see the -psnXXXX argument when started from the finder.
- when an applet is started from the commandline it is non-functional (no window manager connection).

Also, the building of the Idle applet doesn't work due to name conflicts.

----------------------------------------------------------------------

>Comment By: Just van Rossum (jvr)
Date: 2003-02-25 16:21

Message:
Logged In: YES 
user_id=92689

Here's a further idea: let the bootstrap script instead set an environment variable called PYTHONEXECUTABLE. The argv support code could use this to properly set sys.executable, and apps without the need for argv emulation (ie. any not-100% braindead GUI app) would also be able to get at the right value. Perhaps it's even better to _not_ set sys.executable in wrapper code, but change the client code to:

if "PYTHONEXECUTABLE" in os.environ:
    executable = os.environ["PYTHONEXECUTABLE"]
else:
    executable = sys.executable

That way the client code is guaranteed to work correctly independent of whether the argv wrapper was used.

Perhaps a less generic name should be chosen, eg. PYTHONBUNDLEEXECUTABLE.

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-02-25 11:48

Message:
Logged In: YES 
user_id=45365

Idea: we solve this for 50% and document it for 50%.

The "solve" bit involves applets that have the sys.argv support. This code is generated anyway, so we generate one extra line that sets sys.executable. It can do so, because we know what we want to put in there (bundlebuilder has just created the symlink anyway).

The document bit means that we have to state somewhere that if you create an applet without sys.argv emulation you are responsible yourself for setting it correctly. We do this in the IDE, so if we clearly flag the code that does this we've alreadly done quite a bit of "documenting".

----------------------------------------------------------------------

Comment By: Just van Rossum (jvr)
Date: 2003-02-25 09:54

Message:
Logged In: YES 
user_id=92689

I have a fix ready, but as I wrote in private, this will cause sys.executable to be different (ie. it will be the same as CFBundleExecutable). This apparently breaks your stuff, as well as bundlebuilder itself if it is run from a bundle it created itself...

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-02-24 17:40

Message:
Logged In: YES 
user_id=45365

Building the Idle applet has been fixed (Mac/OSX/Makefile rev 1.36).

So now the python-based applet main program is the only thing that needs to be done.

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-02-19 00:48

Message:
Logged In: YES 
user_id=45365

Just thinks he can fix the commandline problem with a Python script, so assigning to him. Will you please also update Misc/NEWS if you do?

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-02-19 00:44

Message:
Logged In: YES 
user_id=45365

sys.argv emulation works again, and that's the main part of this bug. A note has been added to NEWS that starting from the finder is broken in 2.3a2.

Building of Idle I cannot test right now.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=688907&group_id=5470


From noreply@sourceforge.net  Tue Feb 25 15:13:16 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 25 Feb 2003 07:13:16 -0800
Subject: [Python-bugs-list] [ python-Bugs-625725 ] Reorganize MacPython resources on OSX
Message-ID: <E18nglg-0007yP-00@sc8-sf-web1.sourceforge.net>

Bugs item #625725, was opened at 2002-10-19 23:13
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=625725&group_id=5470

Category: Macintosh
Group: None
>Status: Closed
>Resolution: Fixed
Priority: 5
Submitted By: Jack Jansen (jackjansen)
Assigned to: Just van Rossum (jvr)
Summary: Reorganize MacPython resources on OSX

Initial Comment:
The resource file for Python on OSX should be reorganized. Some of the resources are only used by specific modules such as EasyDialogs, these should go into private resource files. Some are used by the interpreter itself (The "Estr" resource that maps error numbers to strings comes to mind). The most obvious place for these would be the resource file of the framework, but these may also be useful in non-framework builds (where you now get the uninformative 'MacOS.Error: Error -1234').

----------------------------------------------------------------------

>Comment By: Just van Rossum (jvr)
Date: 2003-02-25 16:13

Message:
Logged In: YES 
user_id=92689

Sure. But I must warn you: my incentive to rewrite EasyDialogs using Carbon nibs has dropped to pretty much below zero...

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-02-25 14:34

Message:
Logged In: YES 
user_id=45365

Just, this error report has drifted quite a bit from the original intention: it was about splitting the .rsrc files into separate files living with the modules they belong to, but now it is about phasing out .rsrc files.

The former has been finished a while ago, how about closing this bug and opening a new one with a better title and description?

----------------------------------------------------------------------

Comment By: Just van Rossum (jvr)
Date: 2003-01-06 23:22

Message:
Logged In: YES 
user_id=92689

Ok, I've played around a bit and it's certainly doable: I
have a simple dialog running from a nib. It would still be a
fairly major rewrite of EasyDialogs as DialogRef's are not
supported in IBCarbon: must use straight windows.

One thing that's missing is the ability to load nibs from a
path, as opposed to finding them by name from an app bundle:
for this we need to create
Carbon.IBCarbon.CreateNibReferenceWithCFBundle(), and for
that to work we need CFBundle support in Carbon.CF.

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-01-06 13:04

Message:
Logged In: YES 
user_id=45365

Yes, if you want to put some effort into this: please do!

----------------------------------------------------------------------

Comment By: Just van Rossum (jvr)
Date: 2003-01-06 12:33

Message:
Logged In: YES 
user_id=92689

Wait, there already is Carbon.IBCarbon, cool! Shall I try to
find out what it takes to convert EasyDialogs to use it?

----------------------------------------------------------------------

Comment By: Just van Rossum (jvr)
Date: 2003-01-06 12:24

Message:
Logged In: YES 
user_id=92689

Ok, I checked in a change to macresource.py to this effect
(it simply adds sys.path to searchdirs, after
os.path.dirname(__file__).

Apart from Estr, how about trying to use nibs instead of res
files? I think this is meant to work for OS9 as well, but
would require interfacing some *FromNib API's in Carbon.Win
and Carbon.Dlg. Just a thought.

Either way, we should use a strict naming convention, eg.
EasyDialogs.py's resources should be in a file called
EasyDialogs.rsrc (and/or EasyDialogs.nib if we take the
above route).

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-01-06 12:02

Message:
Logged In: YES 
user_id=45365

Okay with me, as long as you look on sys.path *after* checking the __file__. So the order would be
1) Check wheter it happens to be available already
2) Check whether __file__ gives a clue
3) check sys.path

An alternative solution would be that bundlebuilder takes the .rsrc file for each module automatically and combines them all (including errors.rsrc)into the resource file that is automatically opened when the .app starts (I think it's <executablename>.rsrc, check the apple docs).

A Python module in stead of the Estr resource I'm not too thrilled about: there are a lot of error messages, and if it was a Python module then they would all be in-core once you import the module.

I'm assigning this over to you, close it when you're done (the rest of the reorganization is finished).

----------------------------------------------------------------------

Comment By: Just van Rossum (jvr)
Date: 2003-01-05 23:08

Message:
Logged In: YES 
user_id=92689

The Estr resource: is this resource generated, and if so
how? Would it be  an idea to generate a Python module instead?

The macresource module also has issues: eg. for EasyDialogs
it looks for "dialogs.rsrc" in
os.path.dirname(EasyDialogs.__file__), which doesn't work
when EasyDialogs is loaded from a zip file. If it would
(additionally) look on sys.path it could work for standalone
apps created by bundlebuilder: it should simply be included
in <appname>.app/Contents/Resources/ then.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=625725&group_id=5470


From noreply@sourceforge.net  Tue Feb 25 15:24:41 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 25 Feb 2003 07:24:41 -0800
Subject: [Python-bugs-list] [ python-Bugs-692959 ] new.function ignores keyword arguments
Message-ID: <E18ngwj-0000uB-00@sc8-sf-web4.sourceforge.net>

Bugs item #692959, was opened at 2003-02-25 15:24
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=692959&group_id=5470

Category: Python Interpreter Core
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Michael Hudson (mwh)
Assigned to: Nobody/Anonymous (nobody)
Summary: new.function ignores keyword arguments

Initial Comment:
it's no biggie, but:

>>> import new
>>> new.function(a=1)
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
TypeError: function() takes at least 2 arguments (0 given)

is kind of sucky.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=692959&group_id=5470


From noreply@sourceforge.net  Tue Feb 25 15:32:27 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 25 Feb 2003 07:32:27 -0800
Subject: [Python-bugs-list] [ python-Bugs-692884 ] python always searches python23.zip
Message-ID: <E18nh4F-0001MS-00@sc8-sf-web4.sourceforge.net>

Bugs item #692884, was opened at 2003-02-25 14:43
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=692884&group_id=5470

Category: Python Interpreter Core
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Skip Montanaro (montanaro)
Assigned to: Just van Rossum (jvr)
Summary: python always searches python23.zip

Initial Comment:
In trying to figure out why startup of the
spambayes hammiefilter seems so slow, I ran it
under the control of ktrace (also known as strace
or truss on other systems).  I noticed that even
though it's not in sys.path, the interpreter is
always searching for modules in
$prefix/python23.zip.  Why's that?  Here's a
little ktrace/kdump output:

  2361 python   CALL  stat(0xbfffe1d0,0xbfffe5e0)
  2361 python   NAMI  "/Users/skip/local/lib/python23.zip"
  2361 python   RET   stat -1 errno 2 No such file or directory
  2361 python   CALL  stat(0xbfffe1d0,0xbfffe5e0)
  2361 python   NAMI  "/Users/skip/local/lib"
  2361 python   RET   stat 0
  2361 python   CALL  stat(0xbfffece0,0xbfffe7d0)
  2361 python   NAMI  "/Users/skip/local/lib/python23.zip/warnings"
  2361 python   RET   stat -1 errno 2 No such file or directory
  2361 python   CALL  open(0xbfffece0,0,0x1b6)
  2361 python   NAMI  "/Users/skip/local/lib/python23.zip/warnings.py"
  2361 python   RET   open -1 errno 2 No such file or directory
  2361 python   CALL  open(0xbfffece0,0,0x1b6)
  2361 python   NAMI  "/Users/skip/local/lib/python23.zip/warnings.pyc"
  2361 python   RET   open -1 errno 2 No such file or directory
  2361 python   CALL  open(0xbfffece0,0,0x1b6)
  2361 python   NAMI  "/Users/skip/local/lib/python23.zip/warnings.so"
  2361 python   RET   open -1 errno 2 No such file or directory
  2361 python   CALL  open(0xbfffece0,0,0x1b6)
  2361 python   NAMI  "/Users/skip/local/lib/python23.zip/warningsmodule.so"
  2361 python   RET   open -1 errno 2 No such file or directory

What's particularly odd is that even though the
search for python23.zip fails, it continues to
march along looking for files within the
nonexistent zip file.

Searching for modules in something not in sys.path
seems like a bug to me.

----------------------------------------------------------------------

>Comment By: Just van Rossum (jvr)
Date: 2003-02-25 16:32

Message:
Logged In: YES 
user_id=92689

I think it would cause more confusion to have the .pyo's in an earlier location than the .pyc's and .py's. Also, this would cause the .pyo's to always be used, even when not using -O.

The solution is what I said: phase two of PEP 302.

----------------------------------------------------------------------

Comment By: Skip Montanaro (montanaro)
Date: 2003-02-25 16:15

Message:
Logged In: YES 
user_id=44345

If $prefix/python23.zip must always be present and is always going to be
searched (and searched ahead of $prefix/python2.3, why not have the
installation process create it with the .pyo contents of $prefix/python2.3?

(Hammiefilter doesn't import any Mac modules, so the problem Jack
fixed shouldn't affect this case.)

----------------------------------------------------------------------

Comment By: Just van Rossum (jvr)
Date: 2003-02-25 16:07

Message:
Logged In: YES 
user_id=92689

(You probably could have found this with python -vv ;-)

Since Python 2.3a1, site.py no longer removes sys.path items that don't exist on the file system, because it may be (eg.) a URL, handled by a custom importer. PEP 273 explains why a standard zip entry must be on sys.path.

The problem is that the PEP 302 mechanism doesn't make a difference between "no importer found for this sys.path entry" and "use the builtin path importer". This will be fixed in "phase 2" of PEP 302, which consits of adding an explicit importer object on sys.meta_path representing the builtin path importer. (I should really do this before 2.3b1, this report should serve as a reminder for me.)

That said, Jack _yesterday_ fixed a problem on OSX which caused importing Mac modules to be extremely slow (a delay of *seconds*). If spambayes happens to import a Mac module on OSX this would have affected you. Rebuilding from current CVS would help then.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=692884&group_id=5470


From noreply@sourceforge.net  Tue Feb 25 15:35:42 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 25 Feb 2003 07:35:42 -0800
Subject: [Python-bugs-list] [ python-Bugs-688907 ] Applet support is broken
Message-ID: <E18nh7O-0000mu-00@sc8-sf-web1.sourceforge.net>

Bugs item #688907, was opened at 2003-02-18 19:40
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=688907&group_id=5470

Category: Macintosh
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Jack Jansen (jackjansen)
Assigned to: Just van Rossum (jvr)
Summary: Applet support is broken

Initial Comment:
Applet support is broken in two ways:
- there is no sys.argv emulation anymore, and applets always see the -psnXXXX argument when started from the finder.
- when an applet is started from the commandline it is non-functional (no window manager connection).

Also, the building of the Idle applet doesn't work due to name conflicts.

----------------------------------------------------------------------

>Comment By: Jack Jansen (jackjansen)
Date: 2003-02-25 16:35

Message:
Logged In: YES 
user_id=45365

Good idea, but I would suggest that we *do* set sys.executable. It may be used deep down in other modules. And as per Guido's dictum sys.executable is expected to point to a python interpreter.

As to the name of the environment variable, why not simply PYTHON?

----------------------------------------------------------------------

Comment By: Just van Rossum (jvr)
Date: 2003-02-25 16:21

Message:
Logged In: YES 
user_id=92689

Here's a further idea: let the bootstrap script instead set an environment variable called PYTHONEXECUTABLE. The argv support code could use this to properly set sys.executable, and apps without the need for argv emulation (ie. any not-100% braindead GUI app) would also be able to get at the right value. Perhaps it's even better to _not_ set sys.executable in wrapper code, but change the client code to:

if "PYTHONEXECUTABLE" in os.environ:
    executable = os.environ["PYTHONEXECUTABLE"]
else:
    executable = sys.executable

That way the client code is guaranteed to work correctly independent of whether the argv wrapper was used.

Perhaps a less generic name should be chosen, eg. PYTHONBUNDLEEXECUTABLE.

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-02-25 11:48

Message:
Logged In: YES 
user_id=45365

Idea: we solve this for 50% and document it for 50%.

The "solve" bit involves applets that have the sys.argv support. This code is generated anyway, so we generate one extra line that sets sys.executable. It can do so, because we know what we want to put in there (bundlebuilder has just created the symlink anyway).

The document bit means that we have to state somewhere that if you create an applet without sys.argv emulation you are responsible yourself for setting it correctly. We do this in the IDE, so if we clearly flag the code that does this we've alreadly done quite a bit of "documenting".

----------------------------------------------------------------------

Comment By: Just van Rossum (jvr)
Date: 2003-02-25 09:54

Message:
Logged In: YES 
user_id=92689

I have a fix ready, but as I wrote in private, this will cause sys.executable to be different (ie. it will be the same as CFBundleExecutable). This apparently breaks your stuff, as well as bundlebuilder itself if it is run from a bundle it created itself...

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-02-24 17:40

Message:
Logged In: YES 
user_id=45365

Building the Idle applet has been fixed (Mac/OSX/Makefile rev 1.36).

So now the python-based applet main program is the only thing that needs to be done.

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-02-19 00:48

Message:
Logged In: YES 
user_id=45365

Just thinks he can fix the commandline problem with a Python script, so assigning to him. Will you please also update Misc/NEWS if you do?

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-02-19 00:44

Message:
Logged In: YES 
user_id=45365

sys.argv emulation works again, and that's the main part of this bug. A note has been added to NEWS that starting from the finder is broken in 2.3a2.

Building of Idle I cannot test right now.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=688907&group_id=5470


From noreply@sourceforge.net  Tue Feb 25 15:43:33 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 25 Feb 2003 07:43:33 -0800
Subject: [Python-bugs-list] [ python-Bugs-688907 ] Applet support is broken
Message-ID: <E18nhEz-0001u1-00@sc8-sf-web4.sourceforge.net>

Bugs item #688907, was opened at 2003-02-18 19:40
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=688907&group_id=5470

Category: Macintosh
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Jack Jansen (jackjansen)
Assigned to: Just van Rossum (jvr)
Summary: Applet support is broken

Initial Comment:
Applet support is broken in two ways:
- there is no sys.argv emulation anymore, and applets always see the -psnXXXX argument when started from the finder.
- when an applet is started from the commandline it is non-functional (no window manager connection).

Also, the building of the Idle applet doesn't work due to name conflicts.

----------------------------------------------------------------------

>Comment By: Just van Rossum (jvr)
Date: 2003-02-25 16:43

Message:
Logged In: YES 
user_id=92689

But we can only set it if there's a wrapper, and that's not always the case. Unless we add code to the Python _core_ that sets sys.executable if the environment variable is set. I doubt that would be accepted.

Or would there at the C level be a different way to distinguish between argv[0] as set by execve() and the actual executable?

Name: I would like to avoid accidental breakage, so longer is better...

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-02-25 16:35

Message:
Logged In: YES 
user_id=45365

Good idea, but I would suggest that we *do* set sys.executable. It may be used deep down in other modules. And as per Guido's dictum sys.executable is expected to point to a python interpreter.

As to the name of the environment variable, why not simply PYTHON?

----------------------------------------------------------------------

Comment By: Just van Rossum (jvr)
Date: 2003-02-25 16:21

Message:
Logged In: YES 
user_id=92689

Here's a further idea: let the bootstrap script instead set an environment variable called PYTHONEXECUTABLE. The argv support code could use this to properly set sys.executable, and apps without the need for argv emulation (ie. any not-100% braindead GUI app) would also be able to get at the right value. Perhaps it's even better to _not_ set sys.executable in wrapper code, but change the client code to:

if "PYTHONEXECUTABLE" in os.environ:
    executable = os.environ["PYTHONEXECUTABLE"]
else:
    executable = sys.executable

That way the client code is guaranteed to work correctly independent of whether the argv wrapper was used.

Perhaps a less generic name should be chosen, eg. PYTHONBUNDLEEXECUTABLE.

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-02-25 11:48

Message:
Logged In: YES 
user_id=45365

Idea: we solve this for 50% and document it for 50%.

The "solve" bit involves applets that have the sys.argv support. This code is generated anyway, so we generate one extra line that sets sys.executable. It can do so, because we know what we want to put in there (bundlebuilder has just created the symlink anyway).

The document bit means that we have to state somewhere that if you create an applet without sys.argv emulation you are responsible yourself for setting it correctly. We do this in the IDE, so if we clearly flag the code that does this we've alreadly done quite a bit of "documenting".

----------------------------------------------------------------------

Comment By: Just van Rossum (jvr)
Date: 2003-02-25 09:54

Message:
Logged In: YES 
user_id=92689

I have a fix ready, but as I wrote in private, this will cause sys.executable to be different (ie. it will be the same as CFBundleExecutable). This apparently breaks your stuff, as well as bundlebuilder itself if it is run from a bundle it created itself...

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-02-24 17:40

Message:
Logged In: YES 
user_id=45365

Building the Idle applet has been fixed (Mac/OSX/Makefile rev 1.36).

So now the python-based applet main program is the only thing that needs to be done.

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-02-19 00:48

Message:
Logged In: YES 
user_id=45365

Just thinks he can fix the commandline problem with a Python script, so assigning to him. Will you please also update Misc/NEWS if you do?

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-02-19 00:44

Message:
Logged In: YES 
user_id=45365

sys.argv emulation works again, and that's the main part of this bug. A note has been added to NEWS that starting from the finder is broken in 2.3a2.

Building of Idle I cannot test right now.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=688907&group_id=5470


From noreply@sourceforge.net  Tue Feb 25 16:08:11 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 25 Feb 2003 08:08:11 -0800
Subject: [Python-bugs-list] [ python-Bugs-692988 ] test_timeout.py needs 'network' resource
Message-ID: <E18nhcp-0002uo-00@sc8-sf-web1.sourceforge.net>

Bugs item #692988, was opened at 2003-02-25 13:01
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=692988&group_id=5470

Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Cesar Eduardo Barros (cesarb)
Assigned to: Nobody/Anonymous (nobody)
Summary: test_timeout.py needs 'network' resource

Initial Comment:
test_timeout.py fails in two tests if the network is
down. This is because it tries to connect to
www.google.com:80. It should be skipped unless the
'network' resource is enabled.

Also, the failure mode of the testConnectTimeout is
weird; it took 10 seconds for it to timeout. Maybe
that's a real bug somewhere.

The version I used is test_timeout.py revision 1.11

----------------------------------------------------------------------

>Comment By: Cesar Eduardo Barros (cesarb)
Date: 2003-02-25 13:08

Message:
Logged In: YES 
user_id=57799

Weird, the attachment didn't work. Trying again to attach
the file.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=692988&group_id=5470


From noreply@sourceforge.net  Tue Feb 25 16:01:23 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 25 Feb 2003 08:01:23 -0800
Subject: [Python-bugs-list] [ python-Bugs-692988 ] test_timeout.py needs 'network' resource
Message-ID: <E18nhWF-0004SC-00@sc8-sf-web3.sourceforge.net>

Bugs item #692988, was opened at 2003-02-25 13:01
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=692988&group_id=5470

Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Cesar Eduardo Barros (cesarb)
Assigned to: Nobody/Anonymous (nobody)
Summary: test_timeout.py needs 'network' resource

Initial Comment:
test_timeout.py fails in two tests if the network is
down. This is because it tries to connect to
www.google.com:80. It should be skipped unless the
'network' resource is enabled.

Also, the failure mode of the testConnectTimeout is
weird; it took 10 seconds for it to timeout. Maybe
that's a real bug somewhere.

The version I used is test_timeout.py revision 1.11

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=692988&group_id=5470


From noreply@sourceforge.net  Tue Feb 25 17:20:18 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 25 Feb 2003 09:20:18 -0800
Subject: [Python-bugs-list] [ python-Bugs-683658 ] PyErr_Warn may cause import deadlock
Message-ID: <E18nikc-0006a8-00@sc8-sf-web2.sourceforge.net>

Bugs item #683658, was opened at 2003-02-10 01:26
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=683658&group_id=5470

Category: Python Interpreter Core
Group: Python 2.3
Status: Open
Resolution: Remind
Priority: 6
Submitted By: Mark Hammond (mhammond)
Assigned to: Guido van Rossum (gvanrossum)
Summary: PyErr_Warn may cause import deadlock

Initial Comment:
PyErr_Warn() does an implicit import.  Thus, if
PyErr_Warn() is called on a thread while the main
thread holds the import lock, and the main thread then
subsequently waits for the child thread, Python will
deadlock.  The builtin 'apply' now calls PyErr_Warn(),
so simply calling 'apply' on another thread may cause
the deadlock.

Attaching a sample case.  Executing 'python -c "import
hang_apply"' will cause Python to deadlock.  Commenting
out the call to "apply" in that file will prevent the
deadlock (even though the apply() in question is
effectively a no-op)

The example is a little contrived, but is extracted
from real code of mine that saw this hang.  The code
path is:

* Main thread acquires import lock to import the module.
* Module import spawns a new thread.  This new thread
calls apply()
* builtin_apply() calls PyErr_Warn
* PyErr_Warn does an import of the warnings module, and
attempts to acquire the import lock.
* Main thread still waiting for child thread to
complete, but still holds import lock.

Note that removing the call to PyErr_Warn() in
builtin_apply also solves this particular hang -
however, it seems like this is a potential time bomb. 
A potential solution would be to prevent PyErr_Warn
from doing an import - this would mean importing
'warnings' at startup, and keeping a static reference
in errors.c.  Other thoughts welcome.


----------------------------------------------------------------------

>Comment By: Just van Rossum (jvr)
Date: 2003-02-25 18:20

Message:
Logged In: YES 
user_id=92689

Here's an issue with the patch I just ran into: it seems this patch causes the first Python module to be imported to be warnings.py instead of site.py. This means site.py doesn't have full control of the import mechanism anymore, as all modules loaded by warnings.py will already be loaded. Can't the PyModule_WarningsModule global be set _after_ site.py has run?

Use case: I'm building standalone applications in which the import bootstrapping happens in a custom site.py, assuming it will be the first Python module that will be imported. It sets up sys.path appropriately. Now it turns out that when my site.py gets run (eg.) os.py is already loaded from my system-wide installation, instead of from the zip file embedded in the application. This means I can't reliably test whether my built app really works when there's no Python installation available.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-19 17:36

Message:
Logged In: YES 
user_id=6380

I'm not happy with this solution, since it breaks for
multiple interpreters. So keeping this open. Will think
about what to do instead after 2.3a2 is released.

----------------------------------------------------------------------

Comment By: Mark Hammond (mhammond)
Date: 2003-02-19 01:42

Message:
Logged In: YES 
user_id=14198

Checking in a patch that avoids the import, thereby avoiding
the hang.  As suggested, the "linecache" import in warnings
had to be moved.

Even though I checked it in, I attach the patch here, and
have assigned it to Guido for review.  If happy, just mark
it as closed.

/cvsroot/python/python/dist/src/Python/errors.c,v  <--  errors.c
new revision: 2.76; previous revision: 2.75
/cvsroot/python/python/dist/src/Python/pythonrun.c,v  <-- 
pythonrun.c
new revision: 2.178; previous revision: 2.177
/cvsroot/python/python/dist/src/Lib/warnings.py,v  <-- 
warnings.py
new revision: 1.19; previous revision: 1.18


----------------------------------------------------------------------

Comment By: Mark Hammond (mhammond)
Date: 2003-02-16 00:17

Message:
Logged In: YES 
user_id=14198

Yes, the linecache import was the problem.  I abandonded
that approach mainly to avoid yet another import at startup.
 Should 'warnings' get new features, all required imports in
the future will also need to be imported at process startup.
 It also struck me as a time bomb.

I will make an alternative patch that moves the imports.


----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-15 22:01

Message:
Logged In: YES 
user_id=6380

Oh, and the import of linecache by warnings can be put at
the top of warnings, if that's an issue.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-15 16:46

Message:
Logged In: YES 
user_id=6380

No time to review everything here, but maybe PyErr_Warn
should not do an import? The import could be done at startup
time (when site.py is also imported) and saved in the
interpreter state.

----------------------------------------------------------------------

Comment By: Mark Hammond (mhammond)
Date: 2003-02-15 06:02

Message:
Logged In: YES 
user_id=14198

The best I can come up with here is exposing the import lock
to the C API, with a "wait" flag.

This will allow a C function to reliably determine if an
"import" work block, or acquire the lock if not.  It can
then complete its import before releasing the lock.  If the
import would block, then it can take whatever action is
necessary - in the case of PyErr_Warn, it dumps the warning
to stdout.

Attaching a patch.  It exposes (to the core, but not in
headers) two functions:

extern int PyImport_LockImport(int wait);
extern void PyImport_UnlockImport(void);

PyErr_Warn then uses these.

If we do go this route, it makes sense to make these
functions truly public (ie, add them to the headers), and
cleanup import.c appropriately.  I didn't do this in the
interests of keeping the patch small so it can be easily
digested.

Comments?  Other ideas?

----------------------------------------------------------------------

Comment By: Mark Hammond (mhammond)
Date: 2003-02-14 06:16

Message:
Logged In: YES 
user_id=14198

This simple strategy doesn't work - avoiding the import of
'warnings' works fine, until 'warnings' imports 'linecache'!

I'll look at how we can simply throw the warning away if a
deadlock would occur.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-12 00:23

Message:
Logged In: YES 
user_id=33168

I can't think of any other/better solution.  Any idea if
there are there other ticking bombs like this?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=683658&group_id=5470


From noreply@sourceforge.net  Tue Feb 25 17:44:16 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 25 Feb 2003 09:44:16 -0800
Subject: [Python-bugs-list] [ python-Bugs-683658 ] PyErr_Warn may cause import deadlock
Message-ID: <E18nj7o-0001K8-00@sc8-sf-web3.sourceforge.net>

Bugs item #683658, was opened at 2003-02-09 19:26
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=683658&group_id=5470

Category: Python Interpreter Core
Group: Python 2.3
Status: Open
Resolution: Remind
>Priority: 7
Submitted By: Mark Hammond (mhammond)
Assigned to: Guido van Rossum (gvanrossum)
Summary: PyErr_Warn may cause import deadlock

Initial Comment:
PyErr_Warn() does an implicit import.  Thus, if
PyErr_Warn() is called on a thread while the main
thread holds the import lock, and the main thread then
subsequently waits for the child thread, Python will
deadlock.  The builtin 'apply' now calls PyErr_Warn(),
so simply calling 'apply' on another thread may cause
the deadlock.

Attaching a sample case.  Executing 'python -c "import
hang_apply"' will cause Python to deadlock.  Commenting
out the call to "apply" in that file will prevent the
deadlock (even though the apply() in question is
effectively a no-op)

The example is a little contrived, but is extracted
from real code of mine that saw this hang.  The code
path is:

* Main thread acquires import lock to import the module.
* Module import spawns a new thread.  This new thread
calls apply()
* builtin_apply() calls PyErr_Warn
* PyErr_Warn does an import of the warnings module, and
attempts to acquire the import lock.
* Main thread still waiting for child thread to
complete, but still holds import lock.

Note that removing the call to PyErr_Warn() in
builtin_apply also solves this particular hang -
however, it seems like this is a potential time bomb. 
A potential solution would be to prevent PyErr_Warn
from doing an import - this would mean importing
'warnings' at startup, and keeping a static reference
in errors.c.  Other thoughts welcome.


----------------------------------------------------------------------

>Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-25 12:44

Message:
Logged In: YES 
user_id=6380

Good point. But loading site.py first means that warnings
issued as a result of importing site.py are not handled
properly (they will be dumped to stderr, which is better
than nothing). Well, I have to revisit this anyway; I'll
make sure to take this issue into account as well.

----------------------------------------------------------------------

Comment By: Just van Rossum (jvr)
Date: 2003-02-25 12:20

Message:
Logged In: YES 
user_id=92689

Here's an issue with the patch I just ran into: it seems this patch causes the first Python module to be imported to be warnings.py instead of site.py. This means site.py doesn't have full control of the import mechanism anymore, as all modules loaded by warnings.py will already be loaded. Can't the PyModule_WarningsModule global be set _after_ site.py has run?

Use case: I'm building standalone applications in which the import bootstrapping happens in a custom site.py, assuming it will be the first Python module that will be imported. It sets up sys.path appropriately. Now it turns out that when my site.py gets run (eg.) os.py is already loaded from my system-wide installation, instead of from the zip file embedded in the application. This means I can't reliably test whether my built app really works when there's no Python installation available.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-19 11:36

Message:
Logged In: YES 
user_id=6380

I'm not happy with this solution, since it breaks for
multiple interpreters. So keeping this open. Will think
about what to do instead after 2.3a2 is released.

----------------------------------------------------------------------

Comment By: Mark Hammond (mhammond)
Date: 2003-02-18 19:42

Message:
Logged In: YES 
user_id=14198

Checking in a patch that avoids the import, thereby avoiding
the hang.  As suggested, the "linecache" import in warnings
had to be moved.

Even though I checked it in, I attach the patch here, and
have assigned it to Guido for review.  If happy, just mark
it as closed.

/cvsroot/python/python/dist/src/Python/errors.c,v  <--  errors.c
new revision: 2.76; previous revision: 2.75
/cvsroot/python/python/dist/src/Python/pythonrun.c,v  <-- 
pythonrun.c
new revision: 2.178; previous revision: 2.177
/cvsroot/python/python/dist/src/Lib/warnings.py,v  <-- 
warnings.py
new revision: 1.19; previous revision: 1.18


----------------------------------------------------------------------

Comment By: Mark Hammond (mhammond)
Date: 2003-02-15 18:17

Message:
Logged In: YES 
user_id=14198

Yes, the linecache import was the problem.  I abandonded
that approach mainly to avoid yet another import at startup.
 Should 'warnings' get new features, all required imports in
the future will also need to be imported at process startup.
 It also struck me as a time bomb.

I will make an alternative patch that moves the imports.


----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-15 16:01

Message:
Logged In: YES 
user_id=6380

Oh, and the import of linecache by warnings can be put at
the top of warnings, if that's an issue.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-15 10:46

Message:
Logged In: YES 
user_id=6380

No time to review everything here, but maybe PyErr_Warn
should not do an import? The import could be done at startup
time (when site.py is also imported) and saved in the
interpreter state.

----------------------------------------------------------------------

Comment By: Mark Hammond (mhammond)
Date: 2003-02-15 00:02

Message:
Logged In: YES 
user_id=14198

The best I can come up with here is exposing the import lock
to the C API, with a "wait" flag.

This will allow a C function to reliably determine if an
"import" work block, or acquire the lock if not.  It can
then complete its import before releasing the lock.  If the
import would block, then it can take whatever action is
necessary - in the case of PyErr_Warn, it dumps the warning
to stdout.

Attaching a patch.  It exposes (to the core, but not in
headers) two functions:

extern int PyImport_LockImport(int wait);
extern void PyImport_UnlockImport(void);

PyErr_Warn then uses these.

If we do go this route, it makes sense to make these
functions truly public (ie, add them to the headers), and
cleanup import.c appropriately.  I didn't do this in the
interests of keeping the patch small so it can be easily
digested.

Comments?  Other ideas?

----------------------------------------------------------------------

Comment By: Mark Hammond (mhammond)
Date: 2003-02-14 00:16

Message:
Logged In: YES 
user_id=14198

This simple strategy doesn't work - avoiding the import of
'warnings' works fine, until 'warnings' imports 'linecache'!

I'll look at how we can simply throw the warning away if a
deadlock would occur.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-11 18:23

Message:
Logged In: YES 
user_id=33168

I can't think of any other/better solution.  Any idea if
there are there other ticking bombs like this?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=683658&group_id=5470


From noreply@sourceforge.net  Tue Feb 25 18:27:20 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 25 Feb 2003 10:27:20 -0800
Subject: [Python-bugs-list] [ python-Bugs-683658 ] PyErr_Warn may cause import deadlock
Message-ID: <E18njnU-0001HF-00@sc8-sf-web2.sourceforge.net>

Bugs item #683658, was opened at 2003-02-10 01:26
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=683658&group_id=5470

Category: Python Interpreter Core
Group: Python 2.3
Status: Open
Resolution: Remind
Priority: 7
Submitted By: Mark Hammond (mhammond)
Assigned to: Guido van Rossum (gvanrossum)
Summary: PyErr_Warn may cause import deadlock

Initial Comment:
PyErr_Warn() does an implicit import.  Thus, if
PyErr_Warn() is called on a thread while the main
thread holds the import lock, and the main thread then
subsequently waits for the child thread, Python will
deadlock.  The builtin 'apply' now calls PyErr_Warn(),
so simply calling 'apply' on another thread may cause
the deadlock.

Attaching a sample case.  Executing 'python -c "import
hang_apply"' will cause Python to deadlock.  Commenting
out the call to "apply" in that file will prevent the
deadlock (even though the apply() in question is
effectively a no-op)

The example is a little contrived, but is extracted
from real code of mine that saw this hang.  The code
path is:

* Main thread acquires import lock to import the module.
* Module import spawns a new thread.  This new thread
calls apply()
* builtin_apply() calls PyErr_Warn
* PyErr_Warn does an import of the warnings module, and
attempts to acquire the import lock.
* Main thread still waiting for child thread to
complete, but still holds import lock.

Note that removing the call to PyErr_Warn() in
builtin_apply also solves this particular hang -
however, it seems like this is a potential time bomb. 
A potential solution would be to prevent PyErr_Warn
from doing an import - this would mean importing
'warnings' at startup, and keeping a static reference
in errors.c.  Other thoughts welcome.


----------------------------------------------------------------------

>Comment By: Just van Rossum (jvr)
Date: 2003-02-25 19:27

Message:
Logged In: YES 
user_id=92689

Additional info regarding the current situation (with the patch in place):
if warnings.py can't be imported before site.py is run, _all_ warnings issued from C will be printed to stderr, even if warnings.py _can_ be imported later.

I think it's fair to assume that any warning issued during the import of site.py signifies that something is pretty broken, so having them dumped to stderr isn't all that bad.

If you and Mark agree, I'll move the import of warnings.py to after initsite().

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-25 18:44

Message:
Logged In: YES 
user_id=6380

Good point. But loading site.py first means that warnings
issued as a result of importing site.py are not handled
properly (they will be dumped to stderr, which is better
than nothing). Well, I have to revisit this anyway; I'll
make sure to take this issue into account as well.

----------------------------------------------------------------------

Comment By: Just van Rossum (jvr)
Date: 2003-02-25 18:20

Message:
Logged In: YES 
user_id=92689

Here's an issue with the patch I just ran into: it seems this patch causes the first Python module to be imported to be warnings.py instead of site.py. This means site.py doesn't have full control of the import mechanism anymore, as all modules loaded by warnings.py will already be loaded. Can't the PyModule_WarningsModule global be set _after_ site.py has run?

Use case: I'm building standalone applications in which the import bootstrapping happens in a custom site.py, assuming it will be the first Python module that will be imported. It sets up sys.path appropriately. Now it turns out that when my site.py gets run (eg.) os.py is already loaded from my system-wide installation, instead of from the zip file embedded in the application. This means I can't reliably test whether my built app really works when there's no Python installation available.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-19 17:36

Message:
Logged In: YES 
user_id=6380

I'm not happy with this solution, since it breaks for
multiple interpreters. So keeping this open. Will think
about what to do instead after 2.3a2 is released.

----------------------------------------------------------------------

Comment By: Mark Hammond (mhammond)
Date: 2003-02-19 01:42

Message:
Logged In: YES 
user_id=14198

Checking in a patch that avoids the import, thereby avoiding
the hang.  As suggested, the "linecache" import in warnings
had to be moved.

Even though I checked it in, I attach the patch here, and
have assigned it to Guido for review.  If happy, just mark
it as closed.

/cvsroot/python/python/dist/src/Python/errors.c,v  <--  errors.c
new revision: 2.76; previous revision: 2.75
/cvsroot/python/python/dist/src/Python/pythonrun.c,v  <-- 
pythonrun.c
new revision: 2.178; previous revision: 2.177
/cvsroot/python/python/dist/src/Lib/warnings.py,v  <-- 
warnings.py
new revision: 1.19; previous revision: 1.18


----------------------------------------------------------------------

Comment By: Mark Hammond (mhammond)
Date: 2003-02-16 00:17

Message:
Logged In: YES 
user_id=14198

Yes, the linecache import was the problem.  I abandonded
that approach mainly to avoid yet another import at startup.
 Should 'warnings' get new features, all required imports in
the future will also need to be imported at process startup.
 It also struck me as a time bomb.

I will make an alternative patch that moves the imports.


----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-15 22:01

Message:
Logged In: YES 
user_id=6380

Oh, and the import of linecache by warnings can be put at
the top of warnings, if that's an issue.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-15 16:46

Message:
Logged In: YES 
user_id=6380

No time to review everything here, but maybe PyErr_Warn
should not do an import? The import could be done at startup
time (when site.py is also imported) and saved in the
interpreter state.

----------------------------------------------------------------------

Comment By: Mark Hammond (mhammond)
Date: 2003-02-15 06:02

Message:
Logged In: YES 
user_id=14198

The best I can come up with here is exposing the import lock
to the C API, with a "wait" flag.

This will allow a C function to reliably determine if an
"import" work block, or acquire the lock if not.  It can
then complete its import before releasing the lock.  If the
import would block, then it can take whatever action is
necessary - in the case of PyErr_Warn, it dumps the warning
to stdout.

Attaching a patch.  It exposes (to the core, but not in
headers) two functions:

extern int PyImport_LockImport(int wait);
extern void PyImport_UnlockImport(void);

PyErr_Warn then uses these.

If we do go this route, it makes sense to make these
functions truly public (ie, add them to the headers), and
cleanup import.c appropriately.  I didn't do this in the
interests of keeping the patch small so it can be easily
digested.

Comments?  Other ideas?

----------------------------------------------------------------------

Comment By: Mark Hammond (mhammond)
Date: 2003-02-14 06:16

Message:
Logged In: YES 
user_id=14198

This simple strategy doesn't work - avoiding the import of
'warnings' works fine, until 'warnings' imports 'linecache'!

I'll look at how we can simply throw the warning away if a
deadlock would occur.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-12 00:23

Message:
Logged In: YES 
user_id=33168

I can't think of any other/better solution.  Any idea if
there are there other ticking bombs like this?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=683658&group_id=5470


From noreply@sourceforge.net  Tue Feb 25 18:31:32 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 25 Feb 2003 10:31:32 -0800
Subject: [Python-bugs-list] [ python-Bugs-683658 ] PyErr_Warn may cause import deadlock
Message-ID: <E18njrY-000209-00@sc8-sf-web4.sourceforge.net>

Bugs item #683658, was opened at 2003-02-09 19:26
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=683658&group_id=5470

Category: Python Interpreter Core
Group: Python 2.3
Status: Open
Resolution: Remind
Priority: 7
Submitted By: Mark Hammond (mhammond)
Assigned to: Guido van Rossum (gvanrossum)
Summary: PyErr_Warn may cause import deadlock

Initial Comment:
PyErr_Warn() does an implicit import.  Thus, if
PyErr_Warn() is called on a thread while the main
thread holds the import lock, and the main thread then
subsequently waits for the child thread, Python will
deadlock.  The builtin 'apply' now calls PyErr_Warn(),
so simply calling 'apply' on another thread may cause
the deadlock.

Attaching a sample case.  Executing 'python -c "import
hang_apply"' will cause Python to deadlock.  Commenting
out the call to "apply" in that file will prevent the
deadlock (even though the apply() in question is
effectively a no-op)

The example is a little contrived, but is extracted
from real code of mine that saw this hang.  The code
path is:

* Main thread acquires import lock to import the module.
* Module import spawns a new thread.  This new thread
calls apply()
* builtin_apply() calls PyErr_Warn
* PyErr_Warn does an import of the warnings module, and
attempts to acquire the import lock.
* Main thread still waiting for child thread to
complete, but still holds import lock.

Note that removing the call to PyErr_Warn() in
builtin_apply also solves this particular hang -
however, it seems like this is a potential time bomb. 
A potential solution would be to prevent PyErr_Warn
from doing an import - this would mean importing
'warnings' at startup, and keeping a static reference
in errors.c.  Other thoughts welcome.


----------------------------------------------------------------------

>Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-25 13:31

Message:
Logged In: YES 
user_id=6380

OK, as a stopgap measure that sounds like a good idea.

(I wonder if -S should also suppress loading warnings? Or is
that too drastic?)


----------------------------------------------------------------------

Comment By: Just van Rossum (jvr)
Date: 2003-02-25 13:27

Message:
Logged In: YES 
user_id=92689

Additional info regarding the current situation (with the patch in place):
if warnings.py can't be imported before site.py is run, _all_ warnings issued from C will be printed to stderr, even if warnings.py _can_ be imported later.

I think it's fair to assume that any warning issued during the import of site.py signifies that something is pretty broken, so having them dumped to stderr isn't all that bad.

If you and Mark agree, I'll move the import of warnings.py to after initsite().

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-25 12:44

Message:
Logged In: YES 
user_id=6380

Good point. But loading site.py first means that warnings
issued as a result of importing site.py are not handled
properly (they will be dumped to stderr, which is better
than nothing). Well, I have to revisit this anyway; I'll
make sure to take this issue into account as well.

----------------------------------------------------------------------

Comment By: Just van Rossum (jvr)
Date: 2003-02-25 12:20

Message:
Logged In: YES 
user_id=92689

Here's an issue with the patch I just ran into: it seems this patch causes the first Python module to be imported to be warnings.py instead of site.py. This means site.py doesn't have full control of the import mechanism anymore, as all modules loaded by warnings.py will already be loaded. Can't the PyModule_WarningsModule global be set _after_ site.py has run?

Use case: I'm building standalone applications in which the import bootstrapping happens in a custom site.py, assuming it will be the first Python module that will be imported. It sets up sys.path appropriately. Now it turns out that when my site.py gets run (eg.) os.py is already loaded from my system-wide installation, instead of from the zip file embedded in the application. This means I can't reliably test whether my built app really works when there's no Python installation available.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-19 11:36

Message:
Logged In: YES 
user_id=6380

I'm not happy with this solution, since it breaks for
multiple interpreters. So keeping this open. Will think
about what to do instead after 2.3a2 is released.

----------------------------------------------------------------------

Comment By: Mark Hammond (mhammond)
Date: 2003-02-18 19:42

Message:
Logged In: YES 
user_id=14198

Checking in a patch that avoids the import, thereby avoiding
the hang.  As suggested, the "linecache" import in warnings
had to be moved.

Even though I checked it in, I attach the patch here, and
have assigned it to Guido for review.  If happy, just mark
it as closed.

/cvsroot/python/python/dist/src/Python/errors.c,v  <--  errors.c
new revision: 2.76; previous revision: 2.75
/cvsroot/python/python/dist/src/Python/pythonrun.c,v  <-- 
pythonrun.c
new revision: 2.178; previous revision: 2.177
/cvsroot/python/python/dist/src/Lib/warnings.py,v  <-- 
warnings.py
new revision: 1.19; previous revision: 1.18


----------------------------------------------------------------------

Comment By: Mark Hammond (mhammond)
Date: 2003-02-15 18:17

Message:
Logged In: YES 
user_id=14198

Yes, the linecache import was the problem.  I abandonded
that approach mainly to avoid yet another import at startup.
 Should 'warnings' get new features, all required imports in
the future will also need to be imported at process startup.
 It also struck me as a time bomb.

I will make an alternative patch that moves the imports.


----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-15 16:01

Message:
Logged In: YES 
user_id=6380

Oh, and the import of linecache by warnings can be put at
the top of warnings, if that's an issue.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-15 10:46

Message:
Logged In: YES 
user_id=6380

No time to review everything here, but maybe PyErr_Warn
should not do an import? The import could be done at startup
time (when site.py is also imported) and saved in the
interpreter state.

----------------------------------------------------------------------

Comment By: Mark Hammond (mhammond)
Date: 2003-02-15 00:02

Message:
Logged In: YES 
user_id=14198

The best I can come up with here is exposing the import lock
to the C API, with a "wait" flag.

This will allow a C function to reliably determine if an
"import" work block, or acquire the lock if not.  It can
then complete its import before releasing the lock.  If the
import would block, then it can take whatever action is
necessary - in the case of PyErr_Warn, it dumps the warning
to stdout.

Attaching a patch.  It exposes (to the core, but not in
headers) two functions:

extern int PyImport_LockImport(int wait);
extern void PyImport_UnlockImport(void);

PyErr_Warn then uses these.

If we do go this route, it makes sense to make these
functions truly public (ie, add them to the headers), and
cleanup import.c appropriately.  I didn't do this in the
interests of keeping the patch small so it can be easily
digested.

Comments?  Other ideas?

----------------------------------------------------------------------

Comment By: Mark Hammond (mhammond)
Date: 2003-02-14 00:16

Message:
Logged In: YES 
user_id=14198

This simple strategy doesn't work - avoiding the import of
'warnings' works fine, until 'warnings' imports 'linecache'!

I'll look at how we can simply throw the warning away if a
deadlock would occur.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-11 18:23

Message:
Logged In: YES 
user_id=33168

I can't think of any other/better solution.  Any idea if
there are there other ticking bombs like this?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=683658&group_id=5470


From noreply@sourceforge.net  Tue Feb 25 18:48:58 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 25 Feb 2003 10:48:58 -0800
Subject: [Python-bugs-list] [ python-Bugs-693094 ] Python does not build --with-pydebug on Tru64 with vendor cc
Message-ID: <E18nk8Q-0002eF-00@sc8-sf-web4.sourceforge.net>

Bugs item #693094, was opened at 2003-02-25 11:48
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=693094&group_id=5470

Category: Build
Group: Platform-specific
Status: Open
Resolution: None
Priority: 5
Submitted By: Stephan A. Terre (sfiedler)
Assigned to: Nobody/Anonymous (nobody)
Summary: Python does not build --with-pydebug on Tru64 with vendor cc

Initial Comment:
The default standards-compliance level for the
Compaq/HP compiler defines the assert macro using the
if statement (if...then) rather than the ternary if
expression (?:). As a result, when Python is configured
--with-pydebug (so NDEBUG is not defined), the macro
Py_SAFE_DOWNCAST expands to illegal C (containing a
statement inside an expression).

This can be fixed by asserting the -std switch to the
vendor compiler. Testing of 2.2.2 CVS and 2.3 CVS
indicates that there is no bad side effect to asserting
-std. 

I hope to develop a fix for this in configure.in
sometime in the next couple of days. When I have done
so, I will attach it to this bug.

For the record, here are the variations of the -std
switch on the vendor compiler:

  -std0
      Selects the K & R language mode. This is the
default.  Enforces the K &
      R programming style, with certain ANSI extensions
in areas where the K
      & R behavior is undefined or ambiguous. In
general, -std0 compiles most
      pre-ANSI C programs and produces expected
results.  The -std0 option
      causes the __STDC__ macro to be undefined.


  -std
      Selects the relaxed ANSI language mode. Enforces
the ANSI C standard,
      but allows some common programming practices
disallowed by the stan-
      dard.

      This option does not restrict the Tru64 UNIX name
space (for example,
      the names defined in system header files).  To
restrict that name space
      so that only ANSI C reserved names are visible
from the ANSI C header
      files, use the _ANSI_C_SOURCE macro. See
standards(5) for more details.

      This option causes the macro __STDC_ to be
defined to 0.

  -std1
      Selects the strict ANSI language mode. Strictly
enforces the ANSI C
      standard and all its prohibitions (such as those
that apply to the han-
      dling of void types, the definition of lvalues in
expressions, the mix-
      ing of integrals and pointers, and the
modification of rvalues).

      This option does not restrict the Tru64 UNIX name
space (for example,
      the names defined in system header files).  To
restrict that name space
      so that only ANSI C reserved names are visible
from the ANSI C header
      files, use the _ANSI_C_SOURCE macro. See
standards(5) for more details.

      This option causes the macro __STDC_ to be
defined to 1. Note that this
      option also affects linker-defined symbols.  See
ld(1) for more infor-
      mation.

      This option turns on ANSI aliasing rules
(-ansi_alias option).



----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=693094&group_id=5470


From noreply@sourceforge.net  Tue Feb 25 18:58:50 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 25 Feb 2003 10:58:50 -0800
Subject: [Python-bugs-list] [ python-Bugs-683658 ] PyErr_Warn may cause import deadlock
Message-ID: <E18nkHy-0002qP-00@sc8-sf-web1.sourceforge.net>

Bugs item #683658, was opened at 2003-02-10 01:26
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=683658&group_id=5470

Category: Python Interpreter Core
Group: Python 2.3
Status: Open
Resolution: Remind
Priority: 7
Submitted By: Mark Hammond (mhammond)
Assigned to: Guido van Rossum (gvanrossum)
Summary: PyErr_Warn may cause import deadlock

Initial Comment:
PyErr_Warn() does an implicit import.  Thus, if
PyErr_Warn() is called on a thread while the main
thread holds the import lock, and the main thread then
subsequently waits for the child thread, Python will
deadlock.  The builtin 'apply' now calls PyErr_Warn(),
so simply calling 'apply' on another thread may cause
the deadlock.

Attaching a sample case.  Executing 'python -c "import
hang_apply"' will cause Python to deadlock.  Commenting
out the call to "apply" in that file will prevent the
deadlock (even though the apply() in question is
effectively a no-op)

The example is a little contrived, but is extracted
from real code of mine that saw this hang.  The code
path is:

* Main thread acquires import lock to import the module.
* Module import spawns a new thread.  This new thread
calls apply()
* builtin_apply() calls PyErr_Warn
* PyErr_Warn does an import of the warnings module, and
attempts to acquire the import lock.
* Main thread still waiting for child thread to
complete, but still holds import lock.

Note that removing the call to PyErr_Warn() in
builtin_apply also solves this particular hang -
however, it seems like this is a potential time bomb. 
A potential solution would be to prevent PyErr_Warn
from doing an import - this would mean importing
'warnings' at startup, and keeping a static reference
in errors.c.  Other thoughts welcome.


----------------------------------------------------------------------

>Comment By: Just van Rossum (jvr)
Date: 2003-02-25 19:58

Message:
Logged In: YES 
user_id=92689

Only if PyErr_Warn() would still also attempt to do the import if PyModule_WarningsModule is NULL. But that means that deadlocks can still occur when -S is used. But I _do_ like it that no file system imports are done by Python itself with -S. And I actually don't like it at all that warnings.py always gets imported, even if it's never needed.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-25 19:31

Message:
Logged In: YES 
user_id=6380

OK, as a stopgap measure that sounds like a good idea.

(I wonder if -S should also suppress loading warnings? Or is
that too drastic?)


----------------------------------------------------------------------

Comment By: Just van Rossum (jvr)
Date: 2003-02-25 19:27

Message:
Logged In: YES 
user_id=92689

Additional info regarding the current situation (with the patch in place):
if warnings.py can't be imported before site.py is run, _all_ warnings issued from C will be printed to stderr, even if warnings.py _can_ be imported later.

I think it's fair to assume that any warning issued during the import of site.py signifies that something is pretty broken, so having them dumped to stderr isn't all that bad.

If you and Mark agree, I'll move the import of warnings.py to after initsite().

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-25 18:44

Message:
Logged In: YES 
user_id=6380

Good point. But loading site.py first means that warnings
issued as a result of importing site.py are not handled
properly (they will be dumped to stderr, which is better
than nothing). Well, I have to revisit this anyway; I'll
make sure to take this issue into account as well.

----------------------------------------------------------------------

Comment By: Just van Rossum (jvr)
Date: 2003-02-25 18:20

Message:
Logged In: YES 
user_id=92689

Here's an issue with the patch I just ran into: it seems this patch causes the first Python module to be imported to be warnings.py instead of site.py. This means site.py doesn't have full control of the import mechanism anymore, as all modules loaded by warnings.py will already be loaded. Can't the PyModule_WarningsModule global be set _after_ site.py has run?

Use case: I'm building standalone applications in which the import bootstrapping happens in a custom site.py, assuming it will be the first Python module that will be imported. It sets up sys.path appropriately. Now it turns out that when my site.py gets run (eg.) os.py is already loaded from my system-wide installation, instead of from the zip file embedded in the application. This means I can't reliably test whether my built app really works when there's no Python installation available.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-19 17:36

Message:
Logged In: YES 
user_id=6380

I'm not happy with this solution, since it breaks for
multiple interpreters. So keeping this open. Will think
about what to do instead after 2.3a2 is released.

----------------------------------------------------------------------

Comment By: Mark Hammond (mhammond)
Date: 2003-02-19 01:42

Message:
Logged In: YES 
user_id=14198

Checking in a patch that avoids the import, thereby avoiding
the hang.  As suggested, the "linecache" import in warnings
had to be moved.

Even though I checked it in, I attach the patch here, and
have assigned it to Guido for review.  If happy, just mark
it as closed.

/cvsroot/python/python/dist/src/Python/errors.c,v  <--  errors.c
new revision: 2.76; previous revision: 2.75
/cvsroot/python/python/dist/src/Python/pythonrun.c,v  <-- 
pythonrun.c
new revision: 2.178; previous revision: 2.177
/cvsroot/python/python/dist/src/Lib/warnings.py,v  <-- 
warnings.py
new revision: 1.19; previous revision: 1.18


----------------------------------------------------------------------

Comment By: Mark Hammond (mhammond)
Date: 2003-02-16 00:17

Message:
Logged In: YES 
user_id=14198

Yes, the linecache import was the problem.  I abandonded
that approach mainly to avoid yet another import at startup.
 Should 'warnings' get new features, all required imports in
the future will also need to be imported at process startup.
 It also struck me as a time bomb.

I will make an alternative patch that moves the imports.


----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-15 22:01

Message:
Logged In: YES 
user_id=6380

Oh, and the import of linecache by warnings can be put at
the top of warnings, if that's an issue.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-15 16:46

Message:
Logged In: YES 
user_id=6380

No time to review everything here, but maybe PyErr_Warn
should not do an import? The import could be done at startup
time (when site.py is also imported) and saved in the
interpreter state.

----------------------------------------------------------------------

Comment By: Mark Hammond (mhammond)
Date: 2003-02-15 06:02

Message:
Logged In: YES 
user_id=14198

The best I can come up with here is exposing the import lock
to the C API, with a "wait" flag.

This will allow a C function to reliably determine if an
"import" work block, or acquire the lock if not.  It can
then complete its import before releasing the lock.  If the
import would block, then it can take whatever action is
necessary - in the case of PyErr_Warn, it dumps the warning
to stdout.

Attaching a patch.  It exposes (to the core, but not in
headers) two functions:

extern int PyImport_LockImport(int wait);
extern void PyImport_UnlockImport(void);

PyErr_Warn then uses these.

If we do go this route, it makes sense to make these
functions truly public (ie, add them to the headers), and
cleanup import.c appropriately.  I didn't do this in the
interests of keeping the patch small so it can be easily
digested.

Comments?  Other ideas?

----------------------------------------------------------------------

Comment By: Mark Hammond (mhammond)
Date: 2003-02-14 06:16

Message:
Logged In: YES 
user_id=14198

This simple strategy doesn't work - avoiding the import of
'warnings' works fine, until 'warnings' imports 'linecache'!

I'll look at how we can simply throw the warning away if a
deadlock would occur.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-12 00:23

Message:
Logged In: YES 
user_id=33168

I can't think of any other/better solution.  Any idea if
there are there other ticking bombs like this?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=683658&group_id=5470


From noreply@sourceforge.net  Tue Feb 25 19:37:45 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 25 Feb 2003 11:37:45 -0800
Subject: [Python-bugs-list] [ python-Bugs-693121 ] Set == non-Set is a TypeError
Message-ID: <E18nktd-0004VI-00@sc8-sf-web1.sourceforge.net>

Bugs item #693121, was opened at 2003-02-25 14:37
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=693121&group_id=5470

Category: Python Library
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Chris Reedy (creedy)
Assigned to: Nobody/Anonymous (nobody)
Summary: Set == non-Set is a TypeError

Initial Comment:
Comparing Sets to non-Sets results in a TypeError. For
example:

>>> from sets import Set
>>> x = Set([1])
>>> x == 2
TypeError: Binary operation only permitted between sets

This seems to be inconsistent  with other Python
behavior. For example:

>>> (1,2,3) == 2
0
>>> "abcd" == 2
0

Assuming that the standard behavior is what is desired,
the implementation of __eq__ and other comparison
operators in sets.py should be changed to return
NotImplemented when the other object is not a Set.

Note: Looking at the code, I'm not sure whether the
implementation of __lt__, __le__, etc. should also be
changed to not return a Type Error.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=693121&group_id=5470


From noreply@sourceforge.net  Tue Feb 25 20:12:13 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 25 Feb 2003 12:12:13 -0800
Subject: [Python-bugs-list] [ python-Bugs-688907 ] Applet support is broken
Message-ID: <E18nlQz-0008UE-00@sc8-sf-web3.sourceforge.net>

Bugs item #688907, was opened at 2003-02-18 19:40
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=688907&group_id=5470

Category: Macintosh
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Jack Jansen (jackjansen)
Assigned to: Just van Rossum (jvr)
Summary: Applet support is broken

Initial Comment:
Applet support is broken in two ways:
- there is no sys.argv emulation anymore, and applets always see the -psnXXXX argument when started from the finder.
- when an applet is started from the commandline it is non-functional (no window manager connection).

Also, the building of the Idle applet doesn't work due to name conflicts.

----------------------------------------------------------------------

>Comment By: Just van Rossum (jvr)
Date: 2003-02-25 21:12

Message:
Logged In: YES 
user_id=92689

I've found a solution: site[customize].py, I have a patch ready:
bundlebuilder.py adds a sitecustomize.pyc to the bundle which sets sys.executable according to the PYTHONEXECUTABLE environment variable.
Standalone apps already have a custom site.py, so the sys.executable munging happens there. I'll just check it in I suppose...

Leaving this item open until Jack confirms everything works as it should.

----------------------------------------------------------------------

Comment By: Just van Rossum (jvr)
Date: 2003-02-25 16:43

Message:
Logged In: YES 
user_id=92689

But we can only set it if there's a wrapper, and that's not always the case. Unless we add code to the Python _core_ that sets sys.executable if the environment variable is set. I doubt that would be accepted.

Or would there at the C level be a different way to distinguish between argv[0] as set by execve() and the actual executable?

Name: I would like to avoid accidental breakage, so longer is better...

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-02-25 16:35

Message:
Logged In: YES 
user_id=45365

Good idea, but I would suggest that we *do* set sys.executable. It may be used deep down in other modules. And as per Guido's dictum sys.executable is expected to point to a python interpreter.

As to the name of the environment variable, why not simply PYTHON?

----------------------------------------------------------------------

Comment By: Just van Rossum (jvr)
Date: 2003-02-25 16:21

Message:
Logged In: YES 
user_id=92689

Here's a further idea: let the bootstrap script instead set an environment variable called PYTHONEXECUTABLE. The argv support code could use this to properly set sys.executable, and apps without the need for argv emulation (ie. any not-100% braindead GUI app) would also be able to get at the right value. Perhaps it's even better to _not_ set sys.executable in wrapper code, but change the client code to:

if "PYTHONEXECUTABLE" in os.environ:
    executable = os.environ["PYTHONEXECUTABLE"]
else:
    executable = sys.executable

That way the client code is guaranteed to work correctly independent of whether the argv wrapper was used.

Perhaps a less generic name should be chosen, eg. PYTHONBUNDLEEXECUTABLE.

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-02-25 11:48

Message:
Logged In: YES 
user_id=45365

Idea: we solve this for 50% and document it for 50%.

The "solve" bit involves applets that have the sys.argv support. This code is generated anyway, so we generate one extra line that sets sys.executable. It can do so, because we know what we want to put in there (bundlebuilder has just created the symlink anyway).

The document bit means that we have to state somewhere that if you create an applet without sys.argv emulation you are responsible yourself for setting it correctly. We do this in the IDE, so if we clearly flag the code that does this we've alreadly done quite a bit of "documenting".

----------------------------------------------------------------------

Comment By: Just van Rossum (jvr)
Date: 2003-02-25 09:54

Message:
Logged In: YES 
user_id=92689

I have a fix ready, but as I wrote in private, this will cause sys.executable to be different (ie. it will be the same as CFBundleExecutable). This apparently breaks your stuff, as well as bundlebuilder itself if it is run from a bundle it created itself...

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-02-24 17:40

Message:
Logged In: YES 
user_id=45365

Building the Idle applet has been fixed (Mac/OSX/Makefile rev 1.36).

So now the python-based applet main program is the only thing that needs to be done.

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-02-19 00:48

Message:
Logged In: YES 
user_id=45365

Just thinks he can fix the commandline problem with a Python script, so assigning to him. Will you please also update Misc/NEWS if you do?

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-02-19 00:44

Message:
Logged In: YES 
user_id=45365

sys.argv emulation works again, and that's the main part of this bug. A note has been added to NEWS that starting from the finder is broken in 2.3a2.

Building of Idle I cannot test right now.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=688907&group_id=5470


From noreply@sourceforge.net  Tue Feb 25 20:20:42 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 25 Feb 2003 12:20:42 -0800
Subject: [Python-bugs-list] [ python-Bugs-683658 ] PyErr_Warn may cause import deadlock
Message-ID: <E18nlZC-0000Py-00@sc8-sf-web3.sourceforge.net>

Bugs item #683658, was opened at 2003-02-09 19:26
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=683658&group_id=5470

Category: Python Interpreter Core
Group: Python 2.3
Status: Open
Resolution: Remind
Priority: 7
Submitted By: Mark Hammond (mhammond)
Assigned to: Guido van Rossum (gvanrossum)
Summary: PyErr_Warn may cause import deadlock

Initial Comment:
PyErr_Warn() does an implicit import.  Thus, if
PyErr_Warn() is called on a thread while the main
thread holds the import lock, and the main thread then
subsequently waits for the child thread, Python will
deadlock.  The builtin 'apply' now calls PyErr_Warn(),
so simply calling 'apply' on another thread may cause
the deadlock.

Attaching a sample case.  Executing 'python -c "import
hang_apply"' will cause Python to deadlock.  Commenting
out the call to "apply" in that file will prevent the
deadlock (even though the apply() in question is
effectively a no-op)

The example is a little contrived, but is extracted
from real code of mine that saw this hang.  The code
path is:

* Main thread acquires import lock to import the module.
* Module import spawns a new thread.  This new thread
calls apply()
* builtin_apply() calls PyErr_Warn
* PyErr_Warn does an import of the warnings module, and
attempts to acquire the import lock.
* Main thread still waiting for child thread to
complete, but still holds import lock.

Note that removing the call to PyErr_Warn() in
builtin_apply also solves this particular hang -
however, it seems like this is a potential time bomb. 
A potential solution would be to prevent PyErr_Warn
from doing an import - this would mean importing
'warnings' at startup, and keeping a static reference
in errors.c.  Other thoughts welcome.


----------------------------------------------------------------------

>Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-25 15:20

Message:
Logged In: YES 
user_id=6380

OK, implement that.

----------------------------------------------------------------------

Comment By: Just van Rossum (jvr)
Date: 2003-02-25 13:58

Message:
Logged In: YES 
user_id=92689

Only if PyErr_Warn() would still also attempt to do the import if PyModule_WarningsModule is NULL. But that means that deadlocks can still occur when -S is used. But I _do_ like it that no file system imports are done by Python itself with -S. And I actually don't like it at all that warnings.py always gets imported, even if it's never needed.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-25 13:31

Message:
Logged In: YES 
user_id=6380

OK, as a stopgap measure that sounds like a good idea.

(I wonder if -S should also suppress loading warnings? Or is
that too drastic?)


----------------------------------------------------------------------

Comment By: Just van Rossum (jvr)
Date: 2003-02-25 13:27

Message:
Logged In: YES 
user_id=92689

Additional info regarding the current situation (with the patch in place):
if warnings.py can't be imported before site.py is run, _all_ warnings issued from C will be printed to stderr, even if warnings.py _can_ be imported later.

I think it's fair to assume that any warning issued during the import of site.py signifies that something is pretty broken, so having them dumped to stderr isn't all that bad.

If you and Mark agree, I'll move the import of warnings.py to after initsite().

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-25 12:44

Message:
Logged In: YES 
user_id=6380

Good point. But loading site.py first means that warnings
issued as a result of importing site.py are not handled
properly (they will be dumped to stderr, which is better
than nothing). Well, I have to revisit this anyway; I'll
make sure to take this issue into account as well.

----------------------------------------------------------------------

Comment By: Just van Rossum (jvr)
Date: 2003-02-25 12:20

Message:
Logged In: YES 
user_id=92689

Here's an issue with the patch I just ran into: it seems this patch causes the first Python module to be imported to be warnings.py instead of site.py. This means site.py doesn't have full control of the import mechanism anymore, as all modules loaded by warnings.py will already be loaded. Can't the PyModule_WarningsModule global be set _after_ site.py has run?

Use case: I'm building standalone applications in which the import bootstrapping happens in a custom site.py, assuming it will be the first Python module that will be imported. It sets up sys.path appropriately. Now it turns out that when my site.py gets run (eg.) os.py is already loaded from my system-wide installation, instead of from the zip file embedded in the application. This means I can't reliably test whether my built app really works when there's no Python installation available.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-19 11:36

Message:
Logged In: YES 
user_id=6380

I'm not happy with this solution, since it breaks for
multiple interpreters. So keeping this open. Will think
about what to do instead after 2.3a2 is released.

----------------------------------------------------------------------

Comment By: Mark Hammond (mhammond)
Date: 2003-02-18 19:42

Message:
Logged In: YES 
user_id=14198

Checking in a patch that avoids the import, thereby avoiding
the hang.  As suggested, the "linecache" import in warnings
had to be moved.

Even though I checked it in, I attach the patch here, and
have assigned it to Guido for review.  If happy, just mark
it as closed.

/cvsroot/python/python/dist/src/Python/errors.c,v  <--  errors.c
new revision: 2.76; previous revision: 2.75
/cvsroot/python/python/dist/src/Python/pythonrun.c,v  <-- 
pythonrun.c
new revision: 2.178; previous revision: 2.177
/cvsroot/python/python/dist/src/Lib/warnings.py,v  <-- 
warnings.py
new revision: 1.19; previous revision: 1.18


----------------------------------------------------------------------

Comment By: Mark Hammond (mhammond)
Date: 2003-02-15 18:17

Message:
Logged In: YES 
user_id=14198

Yes, the linecache import was the problem.  I abandonded
that approach mainly to avoid yet another import at startup.
 Should 'warnings' get new features, all required imports in
the future will also need to be imported at process startup.
 It also struck me as a time bomb.

I will make an alternative patch that moves the imports.


----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-15 16:01

Message:
Logged In: YES 
user_id=6380

Oh, and the import of linecache by warnings can be put at
the top of warnings, if that's an issue.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-15 10:46

Message:
Logged In: YES 
user_id=6380

No time to review everything here, but maybe PyErr_Warn
should not do an import? The import could be done at startup
time (when site.py is also imported) and saved in the
interpreter state.

----------------------------------------------------------------------

Comment By: Mark Hammond (mhammond)
Date: 2003-02-15 00:02

Message:
Logged In: YES 
user_id=14198

The best I can come up with here is exposing the import lock
to the C API, with a "wait" flag.

This will allow a C function to reliably determine if an
"import" work block, or acquire the lock if not.  It can
then complete its import before releasing the lock.  If the
import would block, then it can take whatever action is
necessary - in the case of PyErr_Warn, it dumps the warning
to stdout.

Attaching a patch.  It exposes (to the core, but not in
headers) two functions:

extern int PyImport_LockImport(int wait);
extern void PyImport_UnlockImport(void);

PyErr_Warn then uses these.

If we do go this route, it makes sense to make these
functions truly public (ie, add them to the headers), and
cleanup import.c appropriately.  I didn't do this in the
interests of keeping the patch small so it can be easily
digested.

Comments?  Other ideas?

----------------------------------------------------------------------

Comment By: Mark Hammond (mhammond)
Date: 2003-02-14 00:16

Message:
Logged In: YES 
user_id=14198

This simple strategy doesn't work - avoiding the import of
'warnings' works fine, until 'warnings' imports 'linecache'!

I'll look at how we can simply throw the warning away if a
deadlock would occur.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-11 18:23

Message:
Logged In: YES 
user_id=33168

I can't think of any other/better solution.  Any idea if
there are there other ticking bombs like this?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=683658&group_id=5470


From noreply@sourceforge.net  Tue Feb 25 20:30:40 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 25 Feb 2003 12:30:40 -0800
Subject: [Python-bugs-list] [ python-Bugs-683658 ] PyErr_Warn may cause import deadlock
Message-ID: <E18nliq-0000zs-00@sc8-sf-web3.sourceforge.net>

Bugs item #683658, was opened at 2003-02-10 01:26
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=683658&group_id=5470

Category: Python Interpreter Core
Group: Python 2.3
Status: Open
Resolution: Remind
Priority: 7
Submitted By: Mark Hammond (mhammond)
Assigned to: Guido van Rossum (gvanrossum)
Summary: PyErr_Warn may cause import deadlock

Initial Comment:
PyErr_Warn() does an implicit import.  Thus, if
PyErr_Warn() is called on a thread while the main
thread holds the import lock, and the main thread then
subsequently waits for the child thread, Python will
deadlock.  The builtin 'apply' now calls PyErr_Warn(),
so simply calling 'apply' on another thread may cause
the deadlock.

Attaching a sample case.  Executing 'python -c "import
hang_apply"' will cause Python to deadlock.  Commenting
out the call to "apply" in that file will prevent the
deadlock (even though the apply() in question is
effectively a no-op)

The example is a little contrived, but is extracted
from real code of mine that saw this hang.  The code
path is:

* Main thread acquires import lock to import the module.
* Module import spawns a new thread.  This new thread
calls apply()
* builtin_apply() calls PyErr_Warn
* PyErr_Warn does an import of the warnings module, and
attempts to acquire the import lock.
* Main thread still waiting for child thread to
complete, but still holds import lock.

Note that removing the call to PyErr_Warn() in
builtin_apply also solves this particular hang -
however, it seems like this is a potential time bomb. 
A potential solution would be to prevent PyErr_Warn
from doing an import - this would mean importing
'warnings' at startup, and keeping a static reference
in errors.c.  Other thoughts welcome.


----------------------------------------------------------------------

>Comment By: Just van Rossum (jvr)
Date: 2003-02-25 21:30

Message:
Logged In: YES 
user_id=92689

Well, since you said the issue needs to be revisited anyway, I'll just leave it at moving the warnings.py import after initsite().

What I don't like is that warnings.py is imported preemptively to begin with, not just with -S. Whether that's resolvable I don't know, but if it is, it would be better than not importing warnings.py with -S.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-25 21:20

Message:
Logged In: YES 
user_id=6380

OK, implement that.

----------------------------------------------------------------------

Comment By: Just van Rossum (jvr)
Date: 2003-02-25 19:58

Message:
Logged In: YES 
user_id=92689

Only if PyErr_Warn() would still also attempt to do the import if PyModule_WarningsModule is NULL. But that means that deadlocks can still occur when -S is used. But I _do_ like it that no file system imports are done by Python itself with -S. And I actually don't like it at all that warnings.py always gets imported, even if it's never needed.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-25 19:31

Message:
Logged In: YES 
user_id=6380

OK, as a stopgap measure that sounds like a good idea.

(I wonder if -S should also suppress loading warnings? Or is
that too drastic?)


----------------------------------------------------------------------

Comment By: Just van Rossum (jvr)
Date: 2003-02-25 19:27

Message:
Logged In: YES 
user_id=92689

Additional info regarding the current situation (with the patch in place):
if warnings.py can't be imported before site.py is run, _all_ warnings issued from C will be printed to stderr, even if warnings.py _can_ be imported later.

I think it's fair to assume that any warning issued during the import of site.py signifies that something is pretty broken, so having them dumped to stderr isn't all that bad.

If you and Mark agree, I'll move the import of warnings.py to after initsite().

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-25 18:44

Message:
Logged In: YES 
user_id=6380

Good point. But loading site.py first means that warnings
issued as a result of importing site.py are not handled
properly (they will be dumped to stderr, which is better
than nothing). Well, I have to revisit this anyway; I'll
make sure to take this issue into account as well.

----------------------------------------------------------------------

Comment By: Just van Rossum (jvr)
Date: 2003-02-25 18:20

Message:
Logged In: YES 
user_id=92689

Here's an issue with the patch I just ran into: it seems this patch causes the first Python module to be imported to be warnings.py instead of site.py. This means site.py doesn't have full control of the import mechanism anymore, as all modules loaded by warnings.py will already be loaded. Can't the PyModule_WarningsModule global be set _after_ site.py has run?

Use case: I'm building standalone applications in which the import bootstrapping happens in a custom site.py, assuming it will be the first Python module that will be imported. It sets up sys.path appropriately. Now it turns out that when my site.py gets run (eg.) os.py is already loaded from my system-wide installation, instead of from the zip file embedded in the application. This means I can't reliably test whether my built app really works when there's no Python installation available.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-19 17:36

Message:
Logged In: YES 
user_id=6380

I'm not happy with this solution, since it breaks for
multiple interpreters. So keeping this open. Will think
about what to do instead after 2.3a2 is released.

----------------------------------------------------------------------

Comment By: Mark Hammond (mhammond)
Date: 2003-02-19 01:42

Message:
Logged In: YES 
user_id=14198

Checking in a patch that avoids the import, thereby avoiding
the hang.  As suggested, the "linecache" import in warnings
had to be moved.

Even though I checked it in, I attach the patch here, and
have assigned it to Guido for review.  If happy, just mark
it as closed.

/cvsroot/python/python/dist/src/Python/errors.c,v  <--  errors.c
new revision: 2.76; previous revision: 2.75
/cvsroot/python/python/dist/src/Python/pythonrun.c,v  <-- 
pythonrun.c
new revision: 2.178; previous revision: 2.177
/cvsroot/python/python/dist/src/Lib/warnings.py,v  <-- 
warnings.py
new revision: 1.19; previous revision: 1.18


----------------------------------------------------------------------

Comment By: Mark Hammond (mhammond)
Date: 2003-02-16 00:17

Message:
Logged In: YES 
user_id=14198

Yes, the linecache import was the problem.  I abandonded
that approach mainly to avoid yet another import at startup.
 Should 'warnings' get new features, all required imports in
the future will also need to be imported at process startup.
 It also struck me as a time bomb.

I will make an alternative patch that moves the imports.


----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-15 22:01

Message:
Logged In: YES 
user_id=6380

Oh, and the import of linecache by warnings can be put at
the top of warnings, if that's an issue.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-15 16:46

Message:
Logged In: YES 
user_id=6380

No time to review everything here, but maybe PyErr_Warn
should not do an import? The import could be done at startup
time (when site.py is also imported) and saved in the
interpreter state.

----------------------------------------------------------------------

Comment By: Mark Hammond (mhammond)
Date: 2003-02-15 06:02

Message:
Logged In: YES 
user_id=14198

The best I can come up with here is exposing the import lock
to the C API, with a "wait" flag.

This will allow a C function to reliably determine if an
"import" work block, or acquire the lock if not.  It can
then complete its import before releasing the lock.  If the
import would block, then it can take whatever action is
necessary - in the case of PyErr_Warn, it dumps the warning
to stdout.

Attaching a patch.  It exposes (to the core, but not in
headers) two functions:

extern int PyImport_LockImport(int wait);
extern void PyImport_UnlockImport(void);

PyErr_Warn then uses these.

If we do go this route, it makes sense to make these
functions truly public (ie, add them to the headers), and
cleanup import.c appropriately.  I didn't do this in the
interests of keeping the patch small so it can be easily
digested.

Comments?  Other ideas?

----------------------------------------------------------------------

Comment By: Mark Hammond (mhammond)
Date: 2003-02-14 06:16

Message:
Logged In: YES 
user_id=14198

This simple strategy doesn't work - avoiding the import of
'warnings' works fine, until 'warnings' imports 'linecache'!

I'll look at how we can simply throw the warning away if a
deadlock would occur.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-12 00:23

Message:
Logged In: YES 
user_id=33168

I can't think of any other/better solution.  Any idea if
there are there other ticking bombs like this?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=683658&group_id=5470


From noreply@sourceforge.net  Tue Feb 25 22:03:23 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 25 Feb 2003 14:03:23 -0800
Subject: [Python-bugs-list] [ python-Bugs-625698 ] Errors with recursive objects
Message-ID: <E18nnAZ-0005Nk-00@sc8-sf-web3.sourceforge.net>

Bugs item #625698, was opened at 2002-10-19 19:27
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=625698&group_id=5470

Category: Python Interpreter Core
Group: None
Status: Open
Resolution: Accepted
Priority: 5
Submitted By: Erik Andersén (erik_andersen)
Assigned to: Jeremy Hylton (jhylton)
Summary: Errors with recursive objects

Initial Comment:
List and dictionaries put Python into an infinite loop if they contain 
more than one reference to itself

>>> d = {}
>>> d[1] = 
d
>>> d == d  # OK Python can handle one 
recursion
1
>>> d[2] = d 
>>> d == d  # Crash with 
two

Lists are similar

>>> l=[]
>>> 
l.append(l)
>>> l==l  # OK
1
>>> l.append(l)
>>> 
l==l # Crash

>>> l>l  # Also crash

Tested with 
ActivePython 2.2.1 under Windows 98
and (the first part) 
Python 2.2.2 under Linux


----------------------------------------------------------------------

Comment By: Michael Stone (mbrierst)
Date: 2003-02-25 22:03

Message:
Logged In: YES 
user_id=670441

I have posted patch 693221 which I believe will
quickly evaluate all examples given on this page.
There's some discussion there.
Feel free to argue with it.

----------------------------------------------------------------------

Comment By: Armin Rigo (arigo)
Date: 2003-01-29 14:11

Message:
Logged In: YES 
user_id=4771

Bisimulation is a cool tool that only works in
"monotonic" settings, i.e. when the set of pairs of
objects known to be equal is always growing and
objects cannot become different when other objects
are discovered to be equal.  So the whole
bisimulation thing is inconsistent with user-defined
comparisons because these can use negations
(there is no negation in the standard list and dict
compares).

I think the language definition should stick to the
simple recursive definition of comparison, and
thus be allowed to enter an infinite recursion.
The interpreter should be Ctrl-C interruptible and
raise a "RuntimeError: maximum recursion depth
exceeded".  This is similar to instance_call() in
classobject.c.

In particular I think the interpreter should not try
to detect loops explicitely.  Consider the case
of someone who really wants to implement a
bisimulation thing:

>>> class X:
...   def __eq__(self, other):
...      if (self, other) in already_comparing:
...         return True
...      else:
...         ...

We must allow the same object's __eq__()
method to be entered twice without raising
any exception.  I would say that the problem
is similar to:

>>> def f():
...    f()

It is not the interpreter's role to detect the loop
and raise an exception, but only to crash nicely
when the recursion depth becomes critically large.


----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2003-01-23 18:57

Message:
Logged In: YES 
user_id=31435

I'll attach a patch with a simple recursion latch.  It handles 
the small examples fine.  Make them bigger and sloth 
returns, though; e.g., this one takes more than 20 seconds 
after the patch (but longer than the lifetime of the universe 
without the patch):

N = 1000

print "building a"
a = []
for i in range(N): a.append(a)

print "building b"
b = []
for i in range(N): b.append(b)

print "comparing"
print a == b

So long as inprogress entries don't outlive the comparison 
call that created them, it still leaves exponentially-
compounding overhead to keep on recreating the entries that 
used to exist.  But if an inprogress entry does outlive the 
comparison that created it, then the cached addresses may 
end up attached to objects other than the original 
comparands, and how to get each thread to eventually clear 
its inprogress dict becomes a puzzle and/or an expense.

----------------------------------------------------------------------

Comment By: Jeremy Hylton (jhylton)
Date: 2003-01-23 18:15

Message:
Logged In: YES 
user_id=31392

Maybe I need to read the code again, but IIRC the code
doesn't assume that id(a) == id(b) implies a == b.  Rather
it uses id(a) and id(b) to look for recursive calls to
compare.  It assumes that a == b will always produce the
same result when invoked recursively within a comparison.

I think a decent solution is to create a latch for the
nesting limit.  If a comparison ever exceeds the recursion
limit, the check is done until the comparison is finished.


----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2003-01-22 03:09

Message:
Logged In: YES 
user_id=31435

llit, you should study the current code before suggesting an 
algorithm.  The current code would be blazing fast if it were 
looking for recursion from the start too.  But the very 
process of building that dict with the ids of the objects as 
keys (which the current algorithm also does) costs more 
than most comparisons cost in real life.  That's why the 
current code doesn't even try to catch recursion until the 
comparison level exceeds 20, and that's where the sloth 
comes from.  Stick a gimmick like that in your algorithm, 
and you'll have similar speed problems.

Also note that id(a)  == id(b) doesn't imply a == b (see 
the other note about NaN for an example).

----------------------------------------------------------------------

Comment By: Till Plewe (llit)
Date: 2003-01-21 06:01

Message:
Logged In: YES 
user_id=540660

How about checking for equality by trying to build a
bisimulation
which "proves" equality. 

Time complexity is O(nodes(a) x nodes(b))  

SAMPLE PYTHON CODE
===============================
#! /usr/bin/env python

from types import *
SeqType=[ListType,DictType,TupleType]

def isequal(a,b):
    if id(a)==id(b): return True
    compare={(id(a),id(b)):(a,b)}
    bisimilar={}
    
    while compare:
        (ida,idb),(oba,obb)=compare.popitem()
        if ida==idb:
            continue
        elif type(oba)==type(obb):
            if type(oba) not in SeqType:
                if oba!=obb:
                    return False
                else:
                    continue
            elif len(oba)!=len(obb):
                return False
            elif (ida,idb) not in bisimilar:
                bisimilar[(ida,idb)]=1
                if type(oba)==DictType:
                    for x in oba: #oba and obb have the same
number of keys
                        if x not in obb:
                            return False
                        else:
                           
compare[(id(oba[x]),id(obb[x]))]=(oba[x],obb[x])
                else:
                    for x,y in zip(oba,obb):
                        compare[(id(x),id(y))]=(x,y)
        else:
            return False
    return bisimilar


----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2003-01-20 20:44

Message:
Logged In: YES 
user_id=31435

Cool.  He's right that they can take a horrendously long time.  
The 2-element list example takes c*2**20 units of time, for 
some suitable constant c, and if it were extended to 3 
elements, it would take c*3**20 units of time, thousands of 
times longer.

As Erik's latest example shows, the outcome isn't always 
particularly well defined either.  An alternative to speeding this 
silliness is to raise an exception instead when recursive 
objects are detected.  There was some hack value in doing 
the graph isomorphism bit, but no real practical value I can 
see.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-01-20 20:20

Message:
Logged In: YES 
user_id=33168

Sorry Tim, I was going to reply to one of your first msgs
and forgot.  Erik sent me mail which said there wasn't a
crash, just that "the statements take horrendously long time
to execute."

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2003-01-20 20:09

Message:
Logged In: YES 
user_id=31435

BTW, Erik, what did you mean by "crash"?  Nobody else has 
seen a crash here.  You started the report by saying "infinite 
loop", which isn't the same thing as a crash.  There isn't an 
infinite loop here either, although the algorithm can take 
astronomical amounts of time to finish (so may appear to be 
in an infinite loop).

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2003-01-20 19:50

Message:
Logged In: YES 
user_id=31435

Equailty for self-consistent recursive objects does a graph 
isomorphism computation.

In your self-inconsistent example, it so happens that False is 
returned, although True would be returned if the compile-time 
constant (in object.c) NESTING_LIMIT were odd instead of 
even.

But if you worry about this, you need to get out more <wink>.

----------------------------------------------------------------------

Comment By: Erik Andersén (erik_andersen)
Date: 2003-01-20 19:24

Message:
Logged In: YES 
user_id=364358

Excuse me for interupting the efficiency discussion, but what is the 
definition of equality for recursive containers.  I am particularly worrided 
about cases like 

>>> class A:
... 	def __eq__(self,other): 
return not self.u==other.u
... 	
>>> a=A()
>>> l=[a]
>>> 
a.u=l
>>> l==l # ????

If l==l then a!=a and l[0]!=l[0], so 
l!=l.
If l!=l then a==a and l[0]==l[0], so l==l !!!!!!!

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2003-01-20 16:12

Message:
Logged In: YES 
user_id=31435

I deleted my patch -- it didn't make sense.

Martin, AFAIK the docs say nothing about the relationship (if 
any) between "is" and "==".  It was deliberate intent that "is" 
not imply "==" for rich comparisons, though, in part so that 
IEEE-754 rules for NaN could be implemented by users.

>>> class NaN:
...     def __eq__(self, other): return False
...
>>> n = NaN()
>>> n is n
True
>>> n == n
False
>>>

----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2003-01-20 08:20

Message:
Logged In: YES 
user_id=21627

Where does the documentation say that "is" may not imply ==?
The check for recursion assumes this implication, anyway: if
we meet the very same object, we infer that they are equal.

What is the purpose of clear_inprogress_dict in your patch?

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2003-01-20 04:28

Message:
Logged In: YES 
user_id=31435

The patch assumes "is" implies "==", which isn't necessarily 
so for rich comparisons.

The attached patch cuts the time about in half, via:

1. Getting rid of the per-thread inprogress dicts, in favor of 
a shared inprogress dict.

2. Restoring the intent of the code that tuples be exempted 
(tuples can't be recursive).  Rev 2.65 of tupleobject.c gave 
tuples a tp_as_mapping slot, breaking the intent of the 
code.

3. Changing the inprogress dict to map a tuple to the 
smallest value of compare_nesting at which the tuple was 
seen.  delete_token() was changed to leave the token alone 
if the current compare_nesting value is greater than that.  
This allows tuples to stay in the dict longer.

4. Cutting NESTING_LIMIT from 20 to 19.  This has a huge 
effect, because the algorithm is still basically exponential-
time.

The problem remaining is that the inprogress dict is still 
useless (and empty) so long as compare_nesting is less 
than NESTING_LIMIT.  This can leave an exponential 
amount of work to be done to fight back up to 
NESTING_LIMIT.

Leaving stuff in inprogress longer, and consulting inprogress 
regardless of the value of compare_nesting (provided that's 
gone over NESTING_LIMIT at least once) slashes the time 
to triviality.  However, it isn't safe:  we can't know that the 
addresses in the dict keys refer to the same objects then.  
We could if started storing into inprogress from the very 
start, but that would be a big speed hit for usual-case 
comparisons.  I'd be happy to knock NESTING_LIMIT down 
more, though.

Assigned to Jeremy, as IIRC this was his code originally.

----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2003-01-19 17:30

Message:
Logged In: YES 
user_id=21627

The patch is fine, except that a comment needs to be added
explaining what this does and why it does that.

It should be understood that this can't fix the very similar
case

a = []
a.append(a)
a.append(a)

b = []
b.append(b)
b.append(b)

a==b

This would still take a long time, right?

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-01-19 16:05

Message:
Logged In: YES 
user_id=33168

Attached is a patch which fixes the problem for me.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-01-08 00:59

Message:
Logged In: YES 
user_id=33168

Erik, do you still have this problem with 2.2.2?  I can't
reproduce your problem under Linux.  mly can't reproduce the
problems with 2.2.1 on Windows (2000).  Do you only have the
problem on Windows?  

----------------------------------------------------------------------

Comment By: Magnus Lyckå (mly)
Date: 2002-10-26 23:12

Message:
Logged In: YES 
user_id=95217

I tried the above under 

Python 2.2.1 (#34, Sep 27 2002, 18:37:42) [MSC 32 bit
(Intel)] on win32
Type "help", "copyright", "credits" or "license" for more
information.
(ActiveState on Win 2000)

and 

Python 2.1.1 (#1, Aug 20 2001, 20:23:29)
[GCC 2.96 20000731 (Mandrake Linux 8.1 2.96-0.60mdk)] on
linux-i386

In both cases it works correctly. No crash.
It takes a lot of time though, Several seconds
on Duron 700 w/ Win2k, and tens of seconds 
on K6-233 w/ Linux.



----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=625698&group_id=5470


From noreply@sourceforge.net  Tue Feb 25 22:07:26 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 25 Feb 2003 14:07:26 -0800
Subject: [Python-bugs-list] [ python-Bugs-688907 ] Applet support is broken
Message-ID: <E18nnEU-0003Pm-00@sc8-sf-web1.sourceforge.net>

Bugs item #688907, was opened at 2003-02-18 19:40
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=688907&group_id=5470

Category: Macintosh
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Jack Jansen (jackjansen)
Assigned to: Just van Rossum (jvr)
Summary: Applet support is broken

Initial Comment:
Applet support is broken in two ways:
- there is no sys.argv emulation anymore, and applets always see the -psnXXXX argument when started from the finder.
- when an applet is started from the commandline it is non-functional (no window manager connection).

Also, the building of the Idle applet doesn't work due to name conflicts.

----------------------------------------------------------------------

>Comment By: Jack Jansen (jackjansen)
Date: 2003-02-25 23:07

Message:
Logged In: YES 
user_id=45365

Two remarks:
- I really disagree with the sitecustomize.pyc. It obscures any real sitecustomize.py that the python admin may have installed, leading to weird behaviour (scripts will see the real sitecustomize but applets not). I think we should really handle this only in the argv emulator, which is going to be used for all "dumb" applets, and let "smart" applets handle it themselves by adding the two-liner
if os.environ.has_key("PYTHONEXECUTABLE"):
    sys.executable = os.environ["PYTHONEXECUTABLE"]

- I don't like the #!/usr/bin/env construct, why not use simply the same Python that Contents/MacOS/Python links to? If that Python goes away the applet will stop working anyway, so I see no reason to let the applet depend on possibly two different Pythons.

----------------------------------------------------------------------

Comment By: Just van Rossum (jvr)
Date: 2003-02-25 21:12

Message:
Logged In: YES 
user_id=92689

I've found a solution: site[customize].py, I have a patch ready:
bundlebuilder.py adds a sitecustomize.pyc to the bundle which sets sys.executable according to the PYTHONEXECUTABLE environment variable.
Standalone apps already have a custom site.py, so the sys.executable munging happens there. I'll just check it in I suppose...

Leaving this item open until Jack confirms everything works as it should.

----------------------------------------------------------------------

Comment By: Just van Rossum (jvr)
Date: 2003-02-25 16:43

Message:
Logged In: YES 
user_id=92689

But we can only set it if there's a wrapper, and that's not always the case. Unless we add code to the Python _core_ that sets sys.executable if the environment variable is set. I doubt that would be accepted.

Or would there at the C level be a different way to distinguish between argv[0] as set by execve() and the actual executable?

Name: I would like to avoid accidental breakage, so longer is better...

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-02-25 16:35

Message:
Logged In: YES 
user_id=45365

Good idea, but I would suggest that we *do* set sys.executable. It may be used deep down in other modules. And as per Guido's dictum sys.executable is expected to point to a python interpreter.

As to the name of the environment variable, why not simply PYTHON?

----------------------------------------------------------------------

Comment By: Just van Rossum (jvr)
Date: 2003-02-25 16:21

Message:
Logged In: YES 
user_id=92689

Here's a further idea: let the bootstrap script instead set an environment variable called PYTHONEXECUTABLE. The argv support code could use this to properly set sys.executable, and apps without the need for argv emulation (ie. any not-100% braindead GUI app) would also be able to get at the right value. Perhaps it's even better to _not_ set sys.executable in wrapper code, but change the client code to:

if "PYTHONEXECUTABLE" in os.environ:
    executable = os.environ["PYTHONEXECUTABLE"]
else:
    executable = sys.executable

That way the client code is guaranteed to work correctly independent of whether the argv wrapper was used.

Perhaps a less generic name should be chosen, eg. PYTHONBUNDLEEXECUTABLE.

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-02-25 11:48

Message:
Logged In: YES 
user_id=45365

Idea: we solve this for 50% and document it for 50%.

The "solve" bit involves applets that have the sys.argv support. This code is generated anyway, so we generate one extra line that sets sys.executable. It can do so, because we know what we want to put in there (bundlebuilder has just created the symlink anyway).

The document bit means that we have to state somewhere that if you create an applet without sys.argv emulation you are responsible yourself for setting it correctly. We do this in the IDE, so if we clearly flag the code that does this we've alreadly done quite a bit of "documenting".

----------------------------------------------------------------------

Comment By: Just van Rossum (jvr)
Date: 2003-02-25 09:54

Message:
Logged In: YES 
user_id=92689

I have a fix ready, but as I wrote in private, this will cause sys.executable to be different (ie. it will be the same as CFBundleExecutable). This apparently breaks your stuff, as well as bundlebuilder itself if it is run from a bundle it created itself...

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-02-24 17:40

Message:
Logged In: YES 
user_id=45365

Building the Idle applet has been fixed (Mac/OSX/Makefile rev 1.36).

So now the python-based applet main program is the only thing that needs to be done.

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-02-19 00:48

Message:
Logged In: YES 
user_id=45365

Just thinks he can fix the commandline problem with a Python script, so assigning to him. Will you please also update Misc/NEWS if you do?

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-02-19 00:44

Message:
Logged In: YES 
user_id=45365

sys.argv emulation works again, and that's the main part of this bug. A note has been added to NEWS that starting from the finder is broken in 2.3a2.

Building of Idle I cannot test right now.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=688907&group_id=5470


From noreply@sourceforge.net  Tue Feb 25 22:08:37 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 25 Feb 2003 14:08:37 -0800
Subject: [Python-bugs-list] [ python-Bugs-693230 ] PackageManager needs to check write permission
Message-ID: <E18nnFd-0003Tp-00@sc8-sf-web1.sourceforge.net>

Bugs item #693230, was opened at 2003-02-25 23:08
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=693230&group_id=5470

Category: Macintosh
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Jack Jansen (jackjansen)
Assigned to: Jack Jansen (jackjansen)
Summary: PackageManager needs to check write permission

Initial Comment:
PackageManager needs to check that the user has write permission to site-packages (or whatever the install directory is set to, once it is settable).

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=693230&group_id=5470


From noreply@sourceforge.net  Tue Feb 25 22:25:51 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 25 Feb 2003 14:25:51 -0800
Subject: [Python-bugs-list] [ python-Bugs-688907 ] Applet support is broken
Message-ID: <E18nnWJ-00051v-00@sc8-sf-web4.sourceforge.net>

Bugs item #688907, was opened at 2003-02-18 19:40
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=688907&group_id=5470

Category: Macintosh
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Jack Jansen (jackjansen)
Assigned to: Just van Rossum (jvr)
Summary: Applet support is broken

Initial Comment:
Applet support is broken in two ways:
- there is no sys.argv emulation anymore, and applets always see the -psnXXXX argument when started from the finder.
- when an applet is started from the commandline it is non-functional (no window manager connection).

Also, the building of the Idle applet doesn't work due to name conflicts.

----------------------------------------------------------------------

>Comment By: Jack Jansen (jackjansen)
Date: 2003-02-25 23:25

Message:
Logged In: YES 
user_id=45365

Something that just struct me: by putting the full pathname to a known interpreter into the #! line we also regain 10.1 compatibility again!

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-02-25 23:07

Message:
Logged In: YES 
user_id=45365

Two remarks:
- I really disagree with the sitecustomize.pyc. It obscures any real sitecustomize.py that the python admin may have installed, leading to weird behaviour (scripts will see the real sitecustomize but applets not). I think we should really handle this only in the argv emulator, which is going to be used for all "dumb" applets, and let "smart" applets handle it themselves by adding the two-liner
if os.environ.has_key("PYTHONEXECUTABLE"):
    sys.executable = os.environ["PYTHONEXECUTABLE"]

- I don't like the #!/usr/bin/env construct, why not use simply the same Python that Contents/MacOS/Python links to? If that Python goes away the applet will stop working anyway, so I see no reason to let the applet depend on possibly two different Pythons.

----------------------------------------------------------------------

Comment By: Just van Rossum (jvr)
Date: 2003-02-25 21:12

Message:
Logged In: YES 
user_id=92689

I've found a solution: site[customize].py, I have a patch ready:
bundlebuilder.py adds a sitecustomize.pyc to the bundle which sets sys.executable according to the PYTHONEXECUTABLE environment variable.
Standalone apps already have a custom site.py, so the sys.executable munging happens there. I'll just check it in I suppose...

Leaving this item open until Jack confirms everything works as it should.

----------------------------------------------------------------------

Comment By: Just van Rossum (jvr)
Date: 2003-02-25 16:43

Message:
Logged In: YES 
user_id=92689

But we can only set it if there's a wrapper, and that's not always the case. Unless we add code to the Python _core_ that sets sys.executable if the environment variable is set. I doubt that would be accepted.

Or would there at the C level be a different way to distinguish between argv[0] as set by execve() and the actual executable?

Name: I would like to avoid accidental breakage, so longer is better...

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-02-25 16:35

Message:
Logged In: YES 
user_id=45365

Good idea, but I would suggest that we *do* set sys.executable. It may be used deep down in other modules. And as per Guido's dictum sys.executable is expected to point to a python interpreter.

As to the name of the environment variable, why not simply PYTHON?

----------------------------------------------------------------------

Comment By: Just van Rossum (jvr)
Date: 2003-02-25 16:21

Message:
Logged In: YES 
user_id=92689

Here's a further idea: let the bootstrap script instead set an environment variable called PYTHONEXECUTABLE. The argv support code could use this to properly set sys.executable, and apps without the need for argv emulation (ie. any not-100% braindead GUI app) would also be able to get at the right value. Perhaps it's even better to _not_ set sys.executable in wrapper code, but change the client code to:

if "PYTHONEXECUTABLE" in os.environ:
    executable = os.environ["PYTHONEXECUTABLE"]
else:
    executable = sys.executable

That way the client code is guaranteed to work correctly independent of whether the argv wrapper was used.

Perhaps a less generic name should be chosen, eg. PYTHONBUNDLEEXECUTABLE.

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-02-25 11:48

Message:
Logged In: YES 
user_id=45365

Idea: we solve this for 50% and document it for 50%.

The "solve" bit involves applets that have the sys.argv support. This code is generated anyway, so we generate one extra line that sets sys.executable. It can do so, because we know what we want to put in there (bundlebuilder has just created the symlink anyway).

The document bit means that we have to state somewhere that if you create an applet without sys.argv emulation you are responsible yourself for setting it correctly. We do this in the IDE, so if we clearly flag the code that does this we've alreadly done quite a bit of "documenting".

----------------------------------------------------------------------

Comment By: Just van Rossum (jvr)
Date: 2003-02-25 09:54

Message:
Logged In: YES 
user_id=92689

I have a fix ready, but as I wrote in private, this will cause sys.executable to be different (ie. it will be the same as CFBundleExecutable). This apparently breaks your stuff, as well as bundlebuilder itself if it is run from a bundle it created itself...

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-02-24 17:40

Message:
Logged In: YES 
user_id=45365

Building the Idle applet has been fixed (Mac/OSX/Makefile rev 1.36).

So now the python-based applet main program is the only thing that needs to be done.

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-02-19 00:48

Message:
Logged In: YES 
user_id=45365

Just thinks he can fix the commandline problem with a Python script, so assigning to him. Will you please also update Misc/NEWS if you do?

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-02-19 00:44

Message:
Logged In: YES 
user_id=45365

sys.argv emulation works again, and that's the main part of this bug. A note has been added to NEWS that starting from the finder is broken in 2.3a2.

Building of Idle I cannot test right now.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=688907&group_id=5470


From noreply@sourceforge.net  Tue Feb 25 22:44:39 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 25 Feb 2003 14:44:39 -0800
Subject: [Python-bugs-list] [ python-Bugs-693255 ] 2.3a2 site.py non-existing dirs
Message-ID: <E18nnoV-0005FE-00@sc8-sf-web1.sourceforge.net>

Bugs item #693255, was opened at 2003-02-25 15:44
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=693255&group_id=5470

Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: James P Rutledge (jrut)
Assigned to: Nobody/Anonymous (nobody)
Summary: 2.3a2 site.py non-existing dirs

Initial Comment:
In Python 2.3a2 the site.py leaves non-existing
directories in sys.path.

On my Debian Linux system, using Python 2.3a2,
the sys.path, after site.py is executed during
interpreter initialization, includes the entry

   /usr/local/lib/python23.zip

although no such directory currently exists on
my system.

The module documentation contained in site.py does
state

  "Non-existing directories (or non-directories)
  are never added to sys.path"


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=693255&group_id=5470


From noreply@sourceforge.net  Tue Feb 25 23:12:36 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 25 Feb 2003 15:12:36 -0800
Subject: [Python-bugs-list] [ python-Bugs-688007 ] Bundlebuilder needs to pre-convert resource files
Message-ID: <E18noFY-0000Gk-00@sc8-sf-web3.sourceforge.net>

Bugs item #688007, was opened at 2003-02-17 15:21
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=688007&group_id=5470

Category: Macintosh
Group: None
>Status: Closed
>Resolution: Fixed
Priority: 5
Submitted By: Jack Jansen (jackjansen)
Assigned to: Jack Jansen (jackjansen)
Summary: Bundlebuilder needs to pre-convert resource files

Initial Comment:
If bundlebuilder encounters an applesingle-encoded resource file it should pre-convert this to a datafork-based resource file.

At the moment the file is copied to the Resources directory verbatim, and this can cause problems if you use "sudo make frameworkinstall" to install Python: the resource folder will be mode 755 owned by root, and any normal user trying to run the applet will get a crash when the applet tries to unpack the applesingle .rsrc file into the datafork resource file .rsrc.df.rsrc.

----------------------------------------------------------------------

>Comment By: Jack Jansen (jackjansen)
Date: 2003-02-26 00:12

Message:
Logged In: YES 
user_id=45365

Finally fixed in buildtools rev. 1.8. The workaround in bundlebuilder has gone.

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-02-24 17:41

Message:
Logged In: YES 
user_id=45365

I'll pick this up again. Now I know about the (src, dst) possibility of bundlebuilder it should be easy to do the conversion in buildtools.

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-02-19 00:42

Message:
Logged In: YES 
user_id=45365

With some hacking, yes. buildtools would have to create the datafork-based resource file, but it would have to keep its original name, so it should be copied to a temp directory somewhere....

Can this wait until after 2.3a2?

----------------------------------------------------------------------

Comment By: Just van Rossum (jvr)
Date: 2003-02-18 20:17

Message:
Logged In: YES 
user_id=92689

Ok, next try: can't BuildApplet and/or buildtools.py do the conversion?

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-02-18 16:50

Message:
Logged In: YES 
user_id=45365

The FSRef APIs for data fork stuff is available under OS9, but (a) this won't work for applets, (b) ResEdit and Resorcerer don't understand this and (c) I don't think CodeWarrior handles it. So until OS9 support is deprecated wer're stuck with resource forks:-(

----------------------------------------------------------------------

Comment By: Just van Rossum (jvr)
Date: 2003-02-18 16:08

Message:
Logged In: YES 
user_id=92689

Aren't datafork res files supported in OS9? If so, this might be an opportunity to get rid of dataforks (as well as macresource.py) altogether...

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-02-18 15:04

Message:
Logged In: YES 
user_id=45365

I agree on a theoretical level, but not on a practical one. The .rsrc files in the CVS repository are AppleSingle encoded, and I can't think of another way to keep them useable on both MacOS9 and MacOSX.

Any suggestions?

----------------------------------------------------------------------

Comment By: Just van Rossum (jvr)
Date: 2003-02-18 13:32

Message:
Logged In: YES 
user_id=92689

Yes, it must be compatible with 2.2 (it also ships with PyObjC). bundlebuilder.py was deliberately written to have very few dependencies: plistlib.py is the only dependency outside the std lib.

I'm not sure I like this patch to begin with: for one it seems it's doing this resource processing at the wrong abstraction level, it adds a special case for something that I don't like being treated specially within bundlebuilder. It would be much better if this preprocessing was done elsewhere.

In other words: I think the bug is in attempting to write the converted file to begin with, not with bundlebuilder not doing the conversion.

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-02-17 17:56

Message:
Logged In: YES 
user_id=45365

Fixed in bundlebuilder.py rev 1.8, macresource rev. 1.3.

Just, I'm assigning this to you because I'm not sure whether you still keep bundlebuilder 2.2-compatible. If so you may need to disable this bit of magic if macresource.install isn't available. Please close the report when you're happy.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=688007&group_id=5470


From noreply@sourceforge.net  Tue Feb 25 23:13:50 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 25 Feb 2003 15:13:50 -0800
Subject: [Python-bugs-list] [ python-Bugs-688011 ] macresource should handle readonly applesingle files
Message-ID: <E18noGk-0000LK-00@sc8-sf-web3.sourceforge.net>

Bugs item #688011, was opened at 2003-02-17 15:23
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=688011&group_id=5470

Category: Macintosh
Group: None
>Status: Closed
>Resolution: Fixed
Priority: 5
Submitted By: Jack Jansen (jackjansen)
Assigned to: Jack Jansen (jackjansen)
Summary: macresource should handle readonly applesingle files

Initial Comment:
If macresource.need() encounters an applesingle file it will unpack it and store the resource data into a .rsrc.df.rsrc file. If the directory
is not writeable this fails.

Macresource should in stead issue a warning and use a temporary file.

----------------------------------------------------------------------

>Comment By: Jack Jansen (jackjansen)
Date: 2003-02-26 00:13

Message:
Logged In: YES 
user_id=45365

Fixed in macresource.py rev. 1.4, although on seconds thoughts I don't give a warning.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=688011&group_id=5470


From noreply@sourceforge.net  Tue Feb 25 23:24:43 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 25 Feb 2003 15:24:43 -0800
Subject: [Python-bugs-list] [ python-Bugs-675928 ] Have exception arguments keep their type
Message-ID: <E18noRH-0000iB-00@sc8-sf-web3.sourceforge.net>

Bugs item #675928, was opened at 2003-01-27 22:06
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=675928&group_id=5470

Category: Python Interpreter Core
Group: Feature Request
Status: Open
Resolution: None
Priority: 5
Submitted By: Brett Cannon (bcannon)
Assigned to: Nobody/Anonymous (nobody)
Summary: Have exception arguments keep their type

Initial Comment:
If you execute the following code::

try:
    raise Exception('a string')
except Exception, err:
    print type(err)

it prints out that ``err`` is a type 'instance'.  It would be nice if it returned the type of the actual argument (in this case, type 'str').

----------------------------------------------------------------------

>Comment By: Brett Cannon (bcannon)
Date: 2003-02-25 15:24

Message:
Logged In: YES 
user_id=357491

I played around today and discovered what was confusing me on all of this.  Executing the following code::

 exc = None
 try: raise Exception('blah')
 except Exception, thing: global exc; exc = thing

which causes type(exc) == <type 'instance'>.  But if you change the 'except' line to::

 except Exception, (thing,): ...

then type(exc) == <type 'str'>.  Obviously when you have a tuple as the target it is unpacking the arguments in the exception (i.e., it is just ``<target> = <raised exception>``).  I just wasn't viewing it as an assignment.  I guess the line "exception's parameter is assigned to the target" from the reference manual threw me; I thought it literally meant the parameters to the exception period, not only if there was assignment unpacking.

So since this seems to be another case of my quirky common sense not thinking like most people, should this bug report be closed or deleted?

----------------------------------------------------------------------

Comment By: Brett Cannon (bcannon)
Date: 2003-01-28 12:57

Message:
Logged In: YES 
user_id=357491

I must be misunderstanding how exceptions handle passed-in values at instantiation.  I know you can pass in multiple values (I assume it's ``def __init__(*args, **kwargs)`` for Exception), but I was not expecting an instance type.  I think I was expecting the values from ``.args`` to be unpacked and stored in the variables instead of an actual exception object to returned in there with a smart ``__str__`` method.  Perhaps the docs should clarify that you receive actual copies of the exception object with its ``args`` attribute set to the argument?  The tutorial, as-is, says "the except clause may specify a variable after the exception name (or list) to receive the argument's value" which suggests to me that I will get the physical argument I passed into the exception; I expected tuple unpacking of the ``args`` attribute.

In other words I see why it is the way it is, but perhaps we should clarify in the tutorial that it is an exception instance storing the arguments and not the arguments themselves?

And I actually used ``str(err)`` to get my code to work, although now that I know that it actually is an instance of an exception it really isn't needed since I will just test for an exception.  =)

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2003-01-28 07:52

Message:
Logged In: YES 
user_id=31435

I'm a bit baffled by this, Brett:  when you instantiate a class, 
you can pass any number of arguments to its constructor.  
You happened to pass a single string argument when building 
an instance of Exception here, but you could have passed 
any number of arguments.  Why should the first argument be 
special?  (Or maybe you think the last argument should be 
special <wink>).  Or what if you didn't pass any arguments at 
all?

Having "the detail" bound to the instance object raised is a 
feature, and a major one.

If you're in the habit of passing a single string to Exception 
directly, note that you can already get it back via doing

    str(err)

in the "except" clause.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=675928&group_id=5470


From noreply@sourceforge.net  Tue Feb 25 23:47:51 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 25 Feb 2003 15:47:51 -0800
Subject: [Python-bugs-list] [ python-Bugs-693293 ] Make EasyDialogs dialogs movable modal
Message-ID: <E18nonf-0006TV-00@sc8-sf-web2.sourceforge.net>

Bugs item #693293, was opened at 2003-02-26 00:47
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=693293&group_id=5470

Category: Macintosh
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Jack Jansen (jackjansen)
Assigned to: Jack Jansen (jackjansen)
Summary: Make EasyDialogs dialogs movable modal

Initial Comment:
The EasyDialogs dialogs should be movable modal. On MacOSX this should at least take care of screen refresh too.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=693293&group_id=5470


From noreply@sourceforge.net  Wed Feb 26 02:56:26 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 25 Feb 2003 18:56:26 -0800
Subject: [Python-bugs-list] [ python-Bugs-693255 ] 2.3a2 site.py non-existing dirs
Message-ID: <E18nrkA-0005DU-00@sc8-sf-web1.sourceforge.net>

Bugs item #693255, was opened at 2003-02-25 15:44
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=693255&group_id=5470

Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: James P Rutledge (jrut)
Assigned to: Nobody/Anonymous (nobody)
Summary: 2.3a2 site.py non-existing dirs

Initial Comment:
In Python 2.3a2 the site.py leaves non-existing
directories in sys.path.

On my Debian Linux system, using Python 2.3a2,
the sys.path, after site.py is executed during
interpreter initialization, includes the entry

   /usr/local/lib/python23.zip

although no such directory currently exists on
my system.

The module documentation contained in site.py does
state

  "Non-existing directories (or non-directories)
  are never added to sys.path"


----------------------------------------------------------------------

>Comment By: James P Rutledge (jrut)
Date: 2003-02-25 19:56

Message:
Logged In: YES 
user_id=720847

Additional info -- the site.py used in
Python 2.2.2 explicitly removes non-existing
and non-directory files from sys.path.

The Python 2.3 site.py does not have that
feature.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=693255&group_id=5470


From noreply@sourceforge.net  Wed Feb 26 03:03:42 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 25 Feb 2003 19:03:42 -0800
Subject: [Python-bugs-list] [ python-Bugs-692988 ] test_timeout.py needs 'network' resource
Message-ID: <E18nrrC-0007vG-00@sc8-sf-web3.sourceforge.net>

Bugs item #692988, was opened at 2003-02-25 11:01
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=692988&group_id=5470

Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Cesar Eduardo Barros (cesarb)
Assigned to: Nobody/Anonymous (nobody)
Summary: test_timeout.py needs 'network' resource

Initial Comment:
test_timeout.py fails in two tests if the network is
down. This is because it tries to connect to
www.google.com:80. It should be skipped unless the
'network' resource is enabled.

Also, the failure mode of the testConnectTimeout is
weird; it took 10 seconds for it to timeout. Maybe
that's a real bug somewhere.

The version I used is test_timeout.py revision 1.11

----------------------------------------------------------------------

>Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-25 22:03

Message:
Logged In: YES 
user_id=33168

The first problem seems to be a DNS lookup failure
(www.google.com).  What Linux distro and version are you
using?  Did you enable ipv6?  Anything else that could help
debug this problem?  Many developers run on Linux and I
haven't heard of this problem?  Is your network very slow? 
Do you have a firewall which introduces large latencies?

----------------------------------------------------------------------

Comment By: Cesar Eduardo Barros (cesarb)
Date: 2003-02-25 11:08

Message:
Logged In: YES 
user_id=57799

Weird, the attachment didn't work. Trying again to attach
the file.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=692988&group_id=5470


From noreply@sourceforge.net  Wed Feb 26 03:06:34 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 25 Feb 2003 19:06:34 -0800
Subject: [Python-bugs-list] [ python-Bugs-692951 ] Python 2.3a2 make test fails timeout
Message-ID: <E18nrty-0003V6-00@sc8-sf-web2.sourceforge.net>

Bugs item #692951, was opened at 2003-02-25 10:16
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=692951&group_id=5470

Category: Build
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Charles Hixson (quixo)
Assigned to: Nobody/Anonymous (nobody)
Summary: Python 2.3a2 make test fails timeout

Initial Comment:
after:
./config
make
I decided to do:
make test
This mainly worked, but ended up like this:
221 tests OK.
1 test failed:
    test_timeout
22 tests skipped:
    test_aepack test_al test_bsddb3 test_bz2 test_cd
test_cl
    test_curses test_email_codecs test_gl test_imgfile
    test_linuxaudiodev test_macfs test_macostools test_nis
    test_normalization test_ossaudiodev test_pep277
test_socket_ssl
    test_socketserver test_sunaudiodev test_winreg
test_winsound
1 skip unexpected on linux2:
    test_bz2
make: *** [test] Error 1

See attached file for a more complete list of the run.

----------------------------------------------------------------------

>Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-25 22:06

Message:
Logged In: YES 
user_id=33168

This appears to be a duplicate of bug 692988.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=692951&group_id=5470


From noreply@sourceforge.net  Wed Feb 26 03:08:12 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 25 Feb 2003 19:08:12 -0800
Subject: [Python-bugs-list] [ python-Bugs-693255 ] 2.3a2 site.py non-existing dirs
Message-ID: <E18nrvY-000846-00@sc8-sf-web3.sourceforge.net>

Bugs item #693255, was opened at 2003-02-25 17:44
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=693255&group_id=5470

Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: James P Rutledge (jrut)
>Assigned to: Just van Rossum (jvr)
Summary: 2.3a2 site.py non-existing dirs

Initial Comment:
In Python 2.3a2 the site.py leaves non-existing
directories in sys.path.

On my Debian Linux system, using Python 2.3a2,
the sys.path, after site.py is executed during
interpreter initialization, includes the entry

   /usr/local/lib/python23.zip

although no such directory currently exists on
my system.

The module documentation contained in site.py does
state

  "Non-existing directories (or non-directories)
  are never added to sys.path"


----------------------------------------------------------------------

>Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-25 22:08

Message:
Logged In: YES 
user_id=33168

Just, does the doc still need to be updated?

----------------------------------------------------------------------

Comment By: James P Rutledge (jrut)
Date: 2003-02-25 21:56

Message:
Logged In: YES 
user_id=720847

Additional info -- the site.py used in
Python 2.2.2 explicitly removes non-existing
and non-directory files from sys.path.

The Python 2.3 site.py does not have that
feature.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=693255&group_id=5470


From noreply@sourceforge.net  Wed Feb 26 03:06:12 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 25 Feb 2003 19:06:12 -0800
Subject: [Python-bugs-list] [ python-Bugs-692988 ] test_timeout.py needs 'network' resource
Message-ID: <E18nrtc-0006Pj-00@sc8-sf-web4.sourceforge.net>

Bugs item #692988, was opened at 2003-02-25 11:01
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=692988&group_id=5470

Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Cesar Eduardo Barros (cesarb)
Assigned to: Nobody/Anonymous (nobody)
Summary: test_timeout.py needs 'network' resource

Initial Comment:
test_timeout.py fails in two tests if the network is
down. This is because it tries to connect to
www.google.com:80. It should be skipped unless the
'network' resource is enabled.

Also, the failure mode of the testConnectTimeout is
weird; it took 10 seconds for it to timeout. Maybe
that's a real bug somewhere.

The version I used is test_timeout.py revision 1.11

----------------------------------------------------------------------

>Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-25 22:06

Message:
Logged In: YES 
user_id=33168

Whoops, sorry my last comment was meant for bug #692951. 
This report helps clarify the other bug.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-25 22:03

Message:
Logged In: YES 
user_id=33168

The first problem seems to be a DNS lookup failure
(www.google.com).  What Linux distro and version are you
using?  Did you enable ipv6?  Anything else that could help
debug this problem?  Many developers run on Linux and I
haven't heard of this problem?  Is your network very slow? 
Do you have a firewall which introduces large latencies?

----------------------------------------------------------------------

Comment By: Cesar Eduardo Barros (cesarb)
Date: 2003-02-25 11:08

Message:
Logged In: YES 
user_id=57799

Weird, the attachment didn't work. Trying again to attach
the file.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=692988&group_id=5470


From noreply@sourceforge.net  Wed Feb 26 03:10:08 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 25 Feb 2003 19:10:08 -0800
Subject: [Python-bugs-list] [ python-Bugs-692416 ] tkinter.createfilehandler dumps core
Message-ID: <E18nrxQ-0006WK-00@sc8-sf-web4.sourceforge.net>

Bugs item #692416, was opened at 2003-02-24 13:54
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=692416&group_id=5470

Category: Tkinter
Group: Python 2.3
Status: Open
Resolution: None
>Priority: 7
Submitted By: Mike Romberg (romberg)
>Assigned to: Martin v. Löwis (loewis)
Summary: tkinter.createfilehandler dumps core

Initial Comment:
  This small example probably uses an invalid
mask and the filedescriptor for a dummy file.  But
I'm seeing this core dump even when using
a correct mask and sockets.  This is with 2.3a2.

>>> import Tkinter
>>> tk = Tkinter.Tk()
>>> fp = open('foo', 'w')
>>> def foo():
...     pass
... 
>>> Tkinter._tkinter.createfilehandler(fp.fileno(), 0, foo)
Segmentation fault (core dumped)


  Here is the traceback:

#0  Tkapp_CreateFileHandler (self=0x0, args=0x1) at
Modules/_tkinter.c:2205
#1  0x080f279a in PyCFunction_Call (func=0x0,
arg=0x40730a68, kw=0x0)
    at Objects/methodobject.c:108
#2  0x080a73cd in call_function (pp_stack=0xbffff2ec,
oparg=1)
    at Python/ceval.c:3285
#3  0x080a58ed in eval_frame (f=0x815700c) at
Python/ceval.c:2041
#4  0x080a630c in PyEval_EvalCodeEx (co=0x40569160,
globals=0x0, locals=0x1, 
    args=0x815700c, argcount=0, kws=0x0, kwcount=0,
defs=0x0, defcount=0, 
    closure=0x0) at Python/ceval.c:2588
#5  0x080a8af7 in PyEval_EvalCode (co=0x1, globals=0x1,
locals=0x1)
    at Python/ceval.c:535
#6  0x080cf25b in run_node (n=0x4015f4e8, 
    filename=0x1 <Address 0x1 out of bounds>,
globals=0x1, locals=0x1, 
    flags=0x1) at Python/pythonrun.c:1105
#7  0x080ce7ae in PyRun_InteractiveOneFlags (fp=0x1, 
    filename=0x80f530a "<stdin>", flags=0xbffff4d8) at
Python/pythonrun.c:609
#8  0x080ce5a3 in PyRun_InteractiveLoopFlags
(fp=0x421271c0, 
    filename=0x80f530a "<stdin>", flags=0x40569160) at
Python/pythonrun.c:542
#9  0x080cfae2 in PyRun_AnyFileExFlags (fp=0x421271c0, 
    filename=0x80f530a "<stdin>", closeit=0,
flags=0xbffff4d8)
    at Python/pythonrun.c:505
#10 0x08054909 in Py_Main (argc=0, argv=0xbffff544) at
Modules/main.c:446
#11 0x0805448b in main (argc=1, argv=0x1) at
Modules/python.c:23
#12 0x420158d4 in __libc_start_main () from
/lib/i686/libc.so.6


----------------------------------------------------------------------

>Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-25 22:10

Message:
Logged In: YES 
user_id=33168

Confirmed the core dump on Linux with Tk 8.3.  Hopefully
Martin will be back soon.  If not I'll try to take a look.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=692416&group_id=5470


From noreply@sourceforge.net  Wed Feb 26 04:42:57 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 25 Feb 2003 20:42:57 -0800
Subject: [Python-bugs-list] [ python-Bugs-693416 ] 2.3a2 import after os.chdir difference
Message-ID: <E18ntPF-0002Is-00@sc8-sf-web3.sourceforge.net>

Bugs item #693416, was opened at 2003-02-25 21:42
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=693416&group_id=5470

Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: James P Rutledge (jrut)
Assigned to: Nobody/Anonymous (nobody)
Summary: 2.3a2 import after os.chdir difference

Initial Comment:
In Python 2.3a2 in interactive mode an import after an
os.chdir imports the module in the new current
directory after the os.chdir.

This is the same as Python 2.2.2 does both in
interactive and non-interactive mode.

In Python 2.3a2 in non-interactive mode an import after
an os.chdir does not import the module in the new
current directory after the os.chdir.  Instead it
attempts to import a module (if present) by that name
in the previous current directory before the os.chdir.
If there is not a module by that name in the previous
current directory, there is an ImportError exception.

The above results are on a Debian Linux system using an
Intel 32 bit processor.

No PYTHONSTARTUP environment variable was set.
No PYTHONPATH environment variable was set.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=693416&group_id=5470


From noreply@sourceforge.net  Wed Feb 26 07:07:21 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 25 Feb 2003 23:07:21 -0800
Subject: [Python-bugs-list] [ python-Bugs-693470 ] licence allowed, but doesn't work
Message-ID: <E18nvez-0002Cv-00@sc8-sf-web2.sourceforge.net>

Bugs item #693470, was opened at 2003-02-26 18:07
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=693470&group_id=5470

Category: Distutils
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Stuart Bishop (zenzen)
Assigned to: Nobody/Anonymous (nobody)
Summary: licence allowed, but doesn't work

Initial Comment:
'licence' is accepted without warning as a keyword
argument to setup(), but it doesn't actually work.
Spelling it 'license' works as expected. Looks like an
attempt to accept common misspellings failed.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=693470&group_id=5470


From noreply@sourceforge.net  Wed Feb 26 07:29:55 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Tue, 25 Feb 2003 23:29:55 -0800
Subject: [Python-bugs-list] [ python-Bugs-693474 ] distutils.setup() arguments missing and incorrect
Message-ID: <E18nw0p-0004jr-00@sc8-sf-web1.sourceforge.net>

Bugs item #693474, was opened at 2003-02-26 18:29
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=693474&group_id=5470

Category: Documentation
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Stuart Bishop (zenzen)
Assigned to: Nobody/Anonymous (nobody)
Summary: distutils.setup() arguments missing and incorrect

Initial Comment:
'home_page' should be 'url'
'platform' should be 'platforms'
'download_url' is undocumented, but used by pypi

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=693474&group_id=5470


From noreply@sourceforge.net  Wed Feb 26 08:21:20 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Wed, 26 Feb 2003 00:21:20 -0800
Subject: [Python-bugs-list] [ python-Bugs-685773 ] 2 (more) bugs in turtle
Message-ID: <E18nwoa-00070m-00@sc8-sf-web1.sourceforge.net>

Bugs item #685773, was opened at 2003-02-13 01:00
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=685773&group_id=5470

Category: Python Library
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Christopher Smith (smichr)
Assigned to: Nobody/Anonymous (nobody)
Summary: 2 (more) bugs in turtle

Initial Comment:
1) After initiating filling with the fill(1) command, the next fill 
command (e.g. fill(0)) does not cause the filling to take place.  
FIlling does not occur until the next draw statement occurs.

SOLUTION:
###
At the end of the IF block in fill(), put the following lines as 
delimited below with the #-- comment:

    def fill(self, flag):
        if self._filling:
 <cut>
                        self._items.append(item)
                        
                #--cps Addition to force filling.  Filling doesn't occur until 
                #--a move command is issued, so a "move" to the 
                #--present position is being issued to force the 
                #--filling to occur.
                x,y=self._position
                self._goto(x,y)
                #--
        self._path = []
###

2)  The last line of the goto() (not the _goto()) function incorrectly 
computes the x coordinate as x-x0.  You can verify this by issuing 
a goto(0,0) command as the first turtle command:  the turtle 
wanders off of the origin.

SOLUTION The coordinate should be computed as x0+x as shown 
below (again, this is the last line of the goto() function):

        self._goto(x0+x, y0-y)



/c

----------------------------------------------------------------------

>Comment By: Christopher Smith (smichr)
Date: 2003-02-26 02:21

Message:
Logged In: YES 
user_id=514525

Regarding #1, a better patch has been to name the fill() function as _fill() 
and create a new fill() function as follows:

def fill(self,n):
  self._fill(n)
  self.forward(0) #a null move 

The previous patch from #1 (as I learned) did not fill circles properly.

/c

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=685773&group_id=5470


From noreply@sourceforge.net  Wed Feb 26 08:43:27 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Wed, 26 Feb 2003 00:43:27 -0800
Subject: [Python-bugs-list] [ python-Bugs-693255 ] 2.3a2 site.py non-existing dirs
Message-ID: <E18nx9z-00089j-00@sc8-sf-web4.sourceforge.net>

Bugs item #693255, was opened at 2003-02-25 23:44
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=693255&group_id=5470

Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: James P Rutledge (jrut)
Assigned to: Just van Rossum (jvr)
Summary: 2.3a2 site.py non-existing dirs

Initial Comment:
In Python 2.3a2 the site.py leaves non-existing
directories in sys.path.

On my Debian Linux system, using Python 2.3a2,
the sys.path, after site.py is executed during
interpreter initialization, includes the entry

   /usr/local/lib/python23.zip

although no such directory currently exists on
my system.

The module documentation contained in site.py does
state

  "Non-existing directories (or non-directories)
  are never added to sys.path"


----------------------------------------------------------------------

>Comment By: Just van Rossum (jvr)
Date: 2003-02-26 09:43

Message:
Logged In: YES 
user_id=92689

The docs are indeed wrong.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-26 04:08

Message:
Logged In: YES 
user_id=33168

Just, does the doc still need to be updated?

----------------------------------------------------------------------

Comment By: James P Rutledge (jrut)
Date: 2003-02-26 03:56

Message:
Logged In: YES 
user_id=720847

Additional info -- the site.py used in
Python 2.2.2 explicitly removes non-existing
and non-directory files from sys.path.

The Python 2.3 site.py does not have that
feature.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=693255&group_id=5470


From noreply@sourceforge.net  Wed Feb 26 08:53:44 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Wed, 26 Feb 2003 00:53:44 -0800
Subject: [Python-bugs-list] [ python-Bugs-688907 ] Applet support is broken
Message-ID: <E18nxJw-0006bk-00@sc8-sf-web2.sourceforge.net>

Bugs item #688907, was opened at 2003-02-18 19:40
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=688907&group_id=5470

Category: Macintosh
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Jack Jansen (jackjansen)
Assigned to: Just van Rossum (jvr)
Summary: Applet support is broken

Initial Comment:
Applet support is broken in two ways:
- there is no sys.argv emulation anymore, and applets always see the -psnXXXX argument when started from the finder.
- when an applet is started from the commandline it is non-functional (no window manager connection).

Also, the building of the Idle applet doesn't work due to name conflicts.

----------------------------------------------------------------------

>Comment By: Just van Rossum (jvr)
Date: 2003-02-26 09:53

Message:
Logged In: YES 
user_id=92689

#!: That's the worst idea ever, as you will not be able to move your app *at* *all*.

Instead of using env, using /usr/bin/python should be ok. I just used env in the hope it would still work when someone removes /usr/bin/python and replaces it with a Python in a different location. Hm, this is brittle also, so I suggest to just hard-code /usr/bin/python.

sitecustomize: hm, good point. Alternative: insert a hacked site.py (copy the original, add a couple of lines).

I really want to get sys.executable right, as the client code is likely not aware (or rather *shouldn't* be aware) of whether it's run with argvemulator or not. This would/should result in *all* client code using the os.environ idiom. Eg. I don't expect the IDE to use the argvemulator, yet it will create applets, hence needs sys.executable. I feel strongly this is an either/or case: either we *always* set sys.executable or we don't do it ever.

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-02-25 23:25

Message:
Logged In: YES 
user_id=45365

Something that just struct me: by putting the full pathname to a known interpreter into the #! line we also regain 10.1 compatibility again!

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-02-25 23:07

Message:
Logged In: YES 
user_id=45365

Two remarks:
- I really disagree with the sitecustomize.pyc. It obscures any real sitecustomize.py that the python admin may have installed, leading to weird behaviour (scripts will see the real sitecustomize but applets not). I think we should really handle this only in the argv emulator, which is going to be used for all "dumb" applets, and let "smart" applets handle it themselves by adding the two-liner
if os.environ.has_key("PYTHONEXECUTABLE"):
    sys.executable = os.environ["PYTHONEXECUTABLE"]

- I don't like the #!/usr/bin/env construct, why not use simply the same Python that Contents/MacOS/Python links to? If that Python goes away the applet will stop working anyway, so I see no reason to let the applet depend on possibly two different Pythons.

----------------------------------------------------------------------

Comment By: Just van Rossum (jvr)
Date: 2003-02-25 21:12

Message:
Logged In: YES 
user_id=92689

I've found a solution: site[customize].py, I have a patch ready:
bundlebuilder.py adds a sitecustomize.pyc to the bundle which sets sys.executable according to the PYTHONEXECUTABLE environment variable.
Standalone apps already have a custom site.py, so the sys.executable munging happens there. I'll just check it in I suppose...

Leaving this item open until Jack confirms everything works as it should.

----------------------------------------------------------------------

Comment By: Just van Rossum (jvr)
Date: 2003-02-25 16:43

Message:
Logged In: YES 
user_id=92689

But we can only set it if there's a wrapper, and that's not always the case. Unless we add code to the Python _core_ that sets sys.executable if the environment variable is set. I doubt that would be accepted.

Or would there at the C level be a different way to distinguish between argv[0] as set by execve() and the actual executable?

Name: I would like to avoid accidental breakage, so longer is better...

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-02-25 16:35

Message:
Logged In: YES 
user_id=45365

Good idea, but I would suggest that we *do* set sys.executable. It may be used deep down in other modules. And as per Guido's dictum sys.executable is expected to point to a python interpreter.

As to the name of the environment variable, why not simply PYTHON?

----------------------------------------------------------------------

Comment By: Just van Rossum (jvr)
Date: 2003-02-25 16:21

Message:
Logged In: YES 
user_id=92689

Here's a further idea: let the bootstrap script instead set an environment variable called PYTHONEXECUTABLE. The argv support code could use this to properly set sys.executable, and apps without the need for argv emulation (ie. any not-100% braindead GUI app) would also be able to get at the right value. Perhaps it's even better to _not_ set sys.executable in wrapper code, but change the client code to:

if "PYTHONEXECUTABLE" in os.environ:
    executable = os.environ["PYTHONEXECUTABLE"]
else:
    executable = sys.executable

That way the client code is guaranteed to work correctly independent of whether the argv wrapper was used.

Perhaps a less generic name should be chosen, eg. PYTHONBUNDLEEXECUTABLE.

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-02-25 11:48

Message:
Logged In: YES 
user_id=45365

Idea: we solve this for 50% and document it for 50%.

The "solve" bit involves applets that have the sys.argv support. This code is generated anyway, so we generate one extra line that sets sys.executable. It can do so, because we know what we want to put in there (bundlebuilder has just created the symlink anyway).

The document bit means that we have to state somewhere that if you create an applet without sys.argv emulation you are responsible yourself for setting it correctly. We do this in the IDE, so if we clearly flag the code that does this we've alreadly done quite a bit of "documenting".

----------------------------------------------------------------------

Comment By: Just van Rossum (jvr)
Date: 2003-02-25 09:54

Message:
Logged In: YES 
user_id=92689

I have a fix ready, but as I wrote in private, this will cause sys.executable to be different (ie. it will be the same as CFBundleExecutable). This apparently breaks your stuff, as well as bundlebuilder itself if it is run from a bundle it created itself...

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-02-24 17:40

Message:
Logged In: YES 
user_id=45365

Building the Idle applet has been fixed (Mac/OSX/Makefile rev 1.36).

So now the python-based applet main program is the only thing that needs to be done.

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-02-19 00:48

Message:
Logged In: YES 
user_id=45365

Just thinks he can fix the commandline problem with a Python script, so assigning to him. Will you please also update Misc/NEWS if you do?

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-02-19 00:44

Message:
Logged In: YES 
user_id=45365

sys.argv emulation works again, and that's the main part of this bug. A note has been added to NEWS that starting from the finder is broken in 2.3a2.

Building of Idle I cannot test right now.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=688907&group_id=5470


From noreply@sourceforge.net  Wed Feb 26 09:18:17 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Wed, 26 Feb 2003 01:18:17 -0800
Subject: [Python-bugs-list] [ python-Bugs-688907 ] Applet support is broken
Message-ID: <E18nxhh-0007pq-00@sc8-sf-web2.sourceforge.net>

Bugs item #688907, was opened at 2003-02-18 19:40
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=688907&group_id=5470

Category: Macintosh
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Jack Jansen (jackjansen)
Assigned to: Just van Rossum (jvr)
Summary: Applet support is broken

Initial Comment:
Applet support is broken in two ways:
- there is no sys.argv emulation anymore, and applets always see the -psnXXXX argument when started from the finder.
- when an applet is started from the commandline it is non-functional (no window manager connection).

Also, the building of the Idle applet doesn't work due to name conflicts.

----------------------------------------------------------------------

>Comment By: Just van Rossum (jvr)
Date: 2003-02-26 10:18

Message:
Logged In: YES 
user_id=92689

#!: I misread you, what you suggest would work for applets to be run on the local system, but not for standalone apps: these need to be able to run on vanilla 10.2 installs. Although I'm not entirely comfortable with depending on /usr/bin/python, removing it (without replacing it with a different version) is an extremely bad idea, just like you're not supposed to remove /usr/bin/perl, as it would break the maintainance cron jobs (and probably much else). I would be happier to just use a shell script as before, but the entire point of this exercise was to fix the fact that we can't do something equivalent to os.execve() in sh (ie. lie about argv[0]).

----------------------------------------------------------------------

Comment By: Just van Rossum (jvr)
Date: 2003-02-26 09:53

Message:
Logged In: YES 
user_id=92689

#!: That's the worst idea ever, as you will not be able to move your app *at* *all*.

Instead of using env, using /usr/bin/python should be ok. I just used env in the hope it would still work when someone removes /usr/bin/python and replaces it with a Python in a different location. Hm, this is brittle also, so I suggest to just hard-code /usr/bin/python.

sitecustomize: hm, good point. Alternative: insert a hacked site.py (copy the original, add a couple of lines).

I really want to get sys.executable right, as the client code is likely not aware (or rather *shouldn't* be aware) of whether it's run with argvemulator or not. This would/should result in *all* client code using the os.environ idiom. Eg. I don't expect the IDE to use the argvemulator, yet it will create applets, hence needs sys.executable. I feel strongly this is an either/or case: either we *always* set sys.executable or we don't do it ever.

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-02-25 23:25

Message:
Logged In: YES 
user_id=45365

Something that just struct me: by putting the full pathname to a known interpreter into the #! line we also regain 10.1 compatibility again!

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-02-25 23:07

Message:
Logged In: YES 
user_id=45365

Two remarks:
- I really disagree with the sitecustomize.pyc. It obscures any real sitecustomize.py that the python admin may have installed, leading to weird behaviour (scripts will see the real sitecustomize but applets not). I think we should really handle this only in the argv emulator, which is going to be used for all "dumb" applets, and let "smart" applets handle it themselves by adding the two-liner
if os.environ.has_key("PYTHONEXECUTABLE"):
    sys.executable = os.environ["PYTHONEXECUTABLE"]

- I don't like the #!/usr/bin/env construct, why not use simply the same Python that Contents/MacOS/Python links to? If that Python goes away the applet will stop working anyway, so I see no reason to let the applet depend on possibly two different Pythons.

----------------------------------------------------------------------

Comment By: Just van Rossum (jvr)
Date: 2003-02-25 21:12

Message:
Logged In: YES 
user_id=92689

I've found a solution: site[customize].py, I have a patch ready:
bundlebuilder.py adds a sitecustomize.pyc to the bundle which sets sys.executable according to the PYTHONEXECUTABLE environment variable.
Standalone apps already have a custom site.py, so the sys.executable munging happens there. I'll just check it in I suppose...

Leaving this item open until Jack confirms everything works as it should.

----------------------------------------------------------------------

Comment By: Just van Rossum (jvr)
Date: 2003-02-25 16:43

Message:
Logged In: YES 
user_id=92689

But we can only set it if there's a wrapper, and that's not always the case. Unless we add code to the Python _core_ that sets sys.executable if the environment variable is set. I doubt that would be accepted.

Or would there at the C level be a different way to distinguish between argv[0] as set by execve() and the actual executable?

Name: I would like to avoid accidental breakage, so longer is better...

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-02-25 16:35

Message:
Logged In: YES 
user_id=45365

Good idea, but I would suggest that we *do* set sys.executable. It may be used deep down in other modules. And as per Guido's dictum sys.executable is expected to point to a python interpreter.

As to the name of the environment variable, why not simply PYTHON?

----------------------------------------------------------------------

Comment By: Just van Rossum (jvr)
Date: 2003-02-25 16:21

Message:
Logged In: YES 
user_id=92689

Here's a further idea: let the bootstrap script instead set an environment variable called PYTHONEXECUTABLE. The argv support code could use this to properly set sys.executable, and apps without the need for argv emulation (ie. any not-100% braindead GUI app) would also be able to get at the right value. Perhaps it's even better to _not_ set sys.executable in wrapper code, but change the client code to:

if "PYTHONEXECUTABLE" in os.environ:
    executable = os.environ["PYTHONEXECUTABLE"]
else:
    executable = sys.executable

That way the client code is guaranteed to work correctly independent of whether the argv wrapper was used.

Perhaps a less generic name should be chosen, eg. PYTHONBUNDLEEXECUTABLE.

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-02-25 11:48

Message:
Logged In: YES 
user_id=45365

Idea: we solve this for 50% and document it for 50%.

The "solve" bit involves applets that have the sys.argv support. This code is generated anyway, so we generate one extra line that sets sys.executable. It can do so, because we know what we want to put in there (bundlebuilder has just created the symlink anyway).

The document bit means that we have to state somewhere that if you create an applet without sys.argv emulation you are responsible yourself for setting it correctly. We do this in the IDE, so if we clearly flag the code that does this we've alreadly done quite a bit of "documenting".

----------------------------------------------------------------------

Comment By: Just van Rossum (jvr)
Date: 2003-02-25 09:54

Message:
Logged In: YES 
user_id=92689

I have a fix ready, but as I wrote in private, this will cause sys.executable to be different (ie. it will be the same as CFBundleExecutable). This apparently breaks your stuff, as well as bundlebuilder itself if it is run from a bundle it created itself...

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-02-24 17:40

Message:
Logged In: YES 
user_id=45365

Building the Idle applet has been fixed (Mac/OSX/Makefile rev 1.36).

So now the python-based applet main program is the only thing that needs to be done.

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-02-19 00:48

Message:
Logged In: YES 
user_id=45365

Just thinks he can fix the commandline problem with a Python script, so assigning to him. Will you please also update Misc/NEWS if you do?

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-02-19 00:44

Message:
Logged In: YES 
user_id=45365

sys.argv emulation works again, and that's the main part of this bug. A note has been added to NEWS that starting from the finder is broken in 2.3a2.

Building of Idle I cannot test right now.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=688907&group_id=5470


From noreply@sourceforge.net  Wed Feb 26 12:37:56 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Wed, 26 Feb 2003 04:37:56 -0800
Subject: [Python-bugs-list] [ python-Bugs-693580 ] socket module has no default timeout
Message-ID: <E18o0ou-0000Z8-00@sc8-sf-web4.sourceforge.net>

Bugs item #693580, was opened at 2003-02-26 07:37
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=693580&group_id=5470

Category: None
Group: None
Status: Open
Resolution: None
Priority: 7
Submitted By: Steve Holden (holdenweb)
Assigned to: Nobody/Anonymous (nobody)
Summary: socket module has no default timeout

Initial Comment:
Python 2.3a2.

The new socket module is a great implementation of Tim 
O'Malley's original concept. From the documentation, 
however, it appears to be missing one feature.

With the timeoutsocket module one could call 
the "setdefaulttimeout()" function to establish a timeout 
for any socket created without explicit knowledge of the 
timeout features. This means that code using ftplib, 
smtplib and friends could be readily conditioned to use 
timeouts without modification to the client modules.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=693580&group_id=5470


From noreply@sourceforge.net  Wed Feb 26 13:44:53 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Wed, 26 Feb 2003 05:44:53 -0800
Subject: [Python-bugs-list] [ python-Feature Requests-691291 ] codecs.open(filename, 'U', 'UTF-16') corrupts text
Message-ID: <E18o1rh-0003Q1-00@sc8-sf-web2.sourceforge.net>

Feature Requests item #691291, was opened at 2003-02-22 20:21
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=691291&group_id=5470

>Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Jason Orendorff (jorend)
>Assigned to: Nobody/Anonymous (nobody)
Summary: codecs.open(filename, 'U', 'UTF-16') corrupts text

Initial Comment:
Tested in Python 2.3a1.

If I write u'Hello\r\nworld\r\n' to a file, then read
it back in 'U' mode, I should get u'Hello\nworld\n'.

However, if I do this using codecs.open() and the
UTF-16 encoding, I get u'Hello\n\nworld\n\n'.

codecs.open() is not 'U'-mode-aware.  The underlying
file is opened in universal newline mode, so the byte
'\x0d' is erroneously translated to '\x0a' before the
UTF-16 codec has a chance to decode it.

The attached unit test should show specifically what it
is that I wish would work.


----------------------------------------------------------------------

>Comment By: M.-A. Lemburg (lemburg)
Date: 2003-02-26 14:44

Message:
Logged In: YES 
user_id=38388

I'm turning this into a feature request. codecs.open()
does not support 'U' as file mode.

Assigning to Jack since he introduced the 'U' mode option.
Jack, what can we do about this ?

----------------------------------------------------------------------

Comment By: Jason Orendorff (jorend)
Date: 2003-02-22 22:17

Message:
Logged In: YES 
user_id=18139

Tested in Python 2.3a2 as well (the bug is still there).

Note that this isn't limited to UTF-16.  It will affect any
encoding that uses the byte '\x0d' to mean anything other
than u'\r'.  The most common American/European encodings are
safe (ASCII, Latin-1 and friends, and UTF-8).


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=691291&group_id=5470


From noreply@sourceforge.net  Wed Feb 26 13:46:16 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Wed, 26 Feb 2003 05:46:16 -0800
Subject: [Python-bugs-list] [ python-Bugs-486434 ] Compiler complaints in posixmodule.c
Message-ID: <E18o1t2-0003Uz-00@sc8-sf-web2.sourceforge.net>

Bugs item #486434, was opened at 2001-11-28 12:51
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=486434&group_id=5470

Category: Extension Modules
Group: None
>Status: Closed
Resolution: None
Priority: 1
Submitted By: M.-A. Lemburg (lemburg)
Assigned to: M.-A. Lemburg (lemburg)
Summary: Compiler complaints in posixmodule.c

Initial Comment:
The linker on Linux reports some warnings for 
posixmodule.c:

libpython2.2.a(posixmodule.o): In function `posix_tmpnam':
/home/lemburg/projects/Python/Dev-Python/./Modules/posixmodule.c:4486:
the use of `tmpnam_r' is dangerous, better use `mkstemp'
libpython2.2.a(posixmodule.o): In function `posix_tempnam':
/home/lemburg/projects/Python/Dev-Python/./Modules/posixmodule.c:4436:
the use of `tempnam' is dangerous, better use `mkstemp'
Perhaps we ought to follow the advice ?!


----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2001-11-30 13:33

Message:
Logged In: YES 
user_id=21627

Before they can be dropped, they must be deprecated. In this
case, I see no real reason to deprecate them: They produce a
warning indicating a potential problem. For some
applications, there may not be a problem at all, e.g. if
they write the temporary files to a directory where nobody
else has write access, or if they open the temporary file
with O_EXCL.

There *are* ways to use tempnam safely. I don't think we
should change Python just because of a stupid linker warning
(which isn't stupid in general, since it made you aware of
the problem - but it is unfortunate that it cannot be turned
off).

----------------------------------------------------------------------

Comment By: M.-A. Lemburg (lemburg)
Date: 2001-11-30 10:40

Message:
Logged In: YES 
user_id=38388

Ok. How about this: we produce warnings for the two APIs in question in 2.2 and drop their support in 2.3 ?!

I hate seeing the linker warn me about Python using dangerous system APIs -- this simply doesn't look right.

----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2001-11-29 19:30

Message:
Logged In: YES 
user_id=21627

We already expose os.tmpfile, I don't think we need mkstemp.

I don't think we should remove tmpnam; applications that use
it will get the warning (for the first time in 2.2); we
should leave it to the applications to migrate away from it.

I found the recommendation not to use mkstemp on a Debian
'testing' system; dunno whether it was added by the Debian
maintainers, or whether it is part of more recent manpage
package.

----------------------------------------------------------------------

Comment By: M.-A. Lemburg (lemburg)
Date: 2001-11-28 15:17

Message:
Logged In: YES 
user_id=38388

Hmm, the man page on SuSE Linux does not say anything about using tmpfile() instead of mkstemp().

BTW, the warnings are already in place.

I wonder whether it wouldn't be better to remove the Python APIs for these functions altogether and 
instead provide interfaces for the mkstemp() and/or tempfile().

----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2001-11-28 14:56

Message:
Logged In: YES 
user_id=21627

No. The usage of tmpnam is not dangerous; we are just
exposing it to the Python application. It may be reasonable
to produce a warning if tmpnam is called.

We cannot replace tempnam with mkstemp for the same reason
Posix couldn't: one produces a string, the other one a file
handle. 

What we could do is to expose mkstemp(3) where available. I
don't see the value of that, though: it could be done only
on systems where mkstemp is available, and we already expose
tmpfile(3). In fact, the Linux man page for mkstemp(3) says

# Don't use this function, use tmpfile(3) instead. It's 
# bet­ter defined and more portable.
If you still think there is a need for action, please
propose a patch.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=486434&group_id=5470


From noreply@sourceforge.net  Wed Feb 26 14:17:19 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Wed, 26 Feb 2003 06:17:19 -0800
Subject: [Python-bugs-list] [ python-Feature Requests-691291 ] codecs.open(filename, 'U', 'UTF-16') corrupts text
Message-ID: <E18o2N5-0004oP-00@sc8-sf-web4.sourceforge.net>

Feature Requests item #691291, was opened at 2003-02-22 20:21
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=691291&group_id=5470

Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Jason Orendorff (jorend)
>Assigned to: Jack Jansen (jackjansen)
Summary: codecs.open(filename, 'U', 'UTF-16') corrupts text

Initial Comment:
Tested in Python 2.3a1.

If I write u'Hello\r\nworld\r\n' to a file, then read
it back in 'U' mode, I should get u'Hello\nworld\n'.

However, if I do this using codecs.open() and the
UTF-16 encoding, I get u'Hello\n\nworld\n\n'.

codecs.open() is not 'U'-mode-aware.  The underlying
file is opened in universal newline mode, so the byte
'\x0d' is erroneously translated to '\x0a' before the
UTF-16 codec has a chance to decode it.

The attached unit test should show specifically what it
is that I wish would work.


----------------------------------------------------------------------

Comment By: M.-A. Lemburg (lemburg)
Date: 2003-02-26 14:44

Message:
Logged In: YES 
user_id=38388

I'm turning this into a feature request. codecs.open()
does not support 'U' as file mode.

Assigning to Jack since he introduced the 'U' mode option.
Jack, what can we do about this ?

----------------------------------------------------------------------

Comment By: Jason Orendorff (jorend)
Date: 2003-02-22 22:17

Message:
Logged In: YES 
user_id=18139

Tested in Python 2.3a2 as well (the bug is still there).

Note that this isn't limited to UTF-16.  It will affect any
encoding that uses the byte '\x0d' to mean anything other
than u'\r'.  The most common American/European encodings are
safe (ASCII, Latin-1 and friends, and UTF-8).


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=691291&group_id=5470


From noreply@sourceforge.net  Wed Feb 26 14:31:59 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Wed, 26 Feb 2003 06:31:59 -0800
Subject: [Python-bugs-list] [ python-Bugs-693416 ] 2.3a2 import after os.chdir difference
Message-ID: <E18o2bH-00061D-00@sc8-sf-web2.sourceforge.net>

Bugs item #693416, was opened at 2003-02-25 21:42
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=693416&group_id=5470

Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: James P Rutledge (jrut)
Assigned to: Nobody/Anonymous (nobody)
Summary: 2.3a2 import after os.chdir difference

Initial Comment:
In Python 2.3a2 in interactive mode an import after an
os.chdir imports the module in the new current
directory after the os.chdir.

This is the same as Python 2.2.2 does both in
interactive and non-interactive mode.

In Python 2.3a2 in non-interactive mode an import after
an os.chdir does not import the module in the new
current directory after the os.chdir.  Instead it
attempts to import a module (if present) by that name
in the previous current directory before the os.chdir.
If there is not a module by that name in the previous
current directory, there is an ImportError exception.

The above results are on a Debian Linux system using an
Intel 32 bit processor.

No PYTHONSTARTUP environment variable was set.
No PYTHONPATH environment variable was set.


----------------------------------------------------------------------

>Comment By: James P Rutledge (jrut)
Date: 2003-02-26 07:31

Message:
Logged In: YES 
user_id=720847

Additional Info -- I have now tried more than one
os.chdir before the import in non-interactive mode and
found that the words "previous current directory" in
the original description should be more accurately
expressed as "ORIGINAL current directory."


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=693416&group_id=5470


From noreply@sourceforge.net  Wed Feb 26 14:37:48 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Wed, 26 Feb 2003 06:37:48 -0800
Subject: [Python-bugs-list] [ python-Bugs-692988 ] test_timeout.py needs 'network' resource
Message-ID: <E18o2gu-0006JP-00@sc8-sf-web3.sourceforge.net>

Bugs item #692988, was opened at 2003-02-25 13:01
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=692988&group_id=5470

Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Cesar Eduardo Barros (cesarb)
Assigned to: Nobody/Anonymous (nobody)
Summary: test_timeout.py needs 'network' resource

Initial Comment:
test_timeout.py fails in two tests if the network is
down. This is because it tries to connect to
www.google.com:80. It should be skipped unless the
'network' resource is enabled.

Also, the failure mode of the testConnectTimeout is
weird; it took 10 seconds for it to timeout. Maybe
that's a real bug somewhere.

The version I used is test_timeout.py revision 1.11

----------------------------------------------------------------------

>Comment By: Cesar Eduardo Barros (cesarb)
Date: 2003-02-26 11:37

Message:
Logged In: YES 
user_id=57799

testConnectTimeout is returning an error because it assumes
every socket.error is a timeout, when it could be, for
instance, a gaierror (which seems to be the case).

Also, the testcase is timing the whole socket.connect()
call, which includes both the DNS lookup and the connect()
proper. To avoid this, it should first do a lookup of
www.google.com by other means and pass the IP, not the
hostname, to the socket.connect() call. Else, it would be
also measuring the DNS resolution time (which can be kinda
long if the nameserver isn't local).

So, we have three problems in the testcase already:
1) It should only run the two remote tests when use of the
'network' resource is enabled
2) It should not assume that any socket.error is a timeout;
it must do more checks to make sure it's really a timeout
3) It should avoid measuring the time spent in
getaddrinfo(3) by resolving the hostname to an IP before
starting the tests (this would also avoid the false failures
because of DNS failure and make them look like a testcase
setup error).

Should I make a separate bug for each of these?

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-26 00:06

Message:
Logged In: YES 
user_id=33168

Whoops, sorry my last comment was meant for bug #692951. 
This report helps clarify the other bug.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-26 00:03

Message:
Logged In: YES 
user_id=33168

The first problem seems to be a DNS lookup failure
(www.google.com).  What Linux distro and version are you
using?  Did you enable ipv6?  Anything else that could help
debug this problem?  Many developers run on Linux and I
haven't heard of this problem?  Is your network very slow? 
Do you have a firewall which introduces large latencies?

----------------------------------------------------------------------

Comment By: Cesar Eduardo Barros (cesarb)
Date: 2003-02-25 13:08

Message:
Logged In: YES 
user_id=57799

Weird, the attachment didn't work. Trying again to attach
the file.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=692988&group_id=5470


From noreply@sourceforge.net  Wed Feb 26 14:40:15 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Wed, 26 Feb 2003 06:40:15 -0800
Subject: [Python-bugs-list] [ python-Bugs-692988 ] test_timeout.py needs 'network' resource
Message-ID: <E18o2jH-0006PM-00@sc8-sf-web2.sourceforge.net>

Bugs item #692988, was opened at 2003-02-25 13:01
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=692988&group_id=5470

Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Cesar Eduardo Barros (cesarb)
Assigned to: Nobody/Anonymous (nobody)
Summary: test_timeout.py needs 'network' resource

Initial Comment:
test_timeout.py fails in two tests if the network is
down. This is because it tries to connect to
www.google.com:80. It should be skipped unless the
'network' resource is enabled.

Also, the failure mode of the testConnectTimeout is
weird; it took 10 seconds for it to timeout. Maybe
that's a real bug somewhere.

The version I used is test_timeout.py revision 1.11

----------------------------------------------------------------------

>Comment By: Cesar Eduardo Barros (cesarb)
Date: 2003-02-26 11:40

Message:
Logged In: YES 
user_id=57799

Ah, and maybe the documentation should be updated to make
explicit that the connect() timeout does not include the DNS
lookup (and should not, since it's the _socket_ timeout...).
If you want a timeout in the DNS lookup, you must do the
lookup yourself (but I don't think it's really possible to
change the resolver timeout...)

----------------------------------------------------------------------

Comment By: Cesar Eduardo Barros (cesarb)
Date: 2003-02-26 11:37

Message:
Logged In: YES 
user_id=57799

testConnectTimeout is returning an error because it assumes
every socket.error is a timeout, when it could be, for
instance, a gaierror (which seems to be the case).

Also, the testcase is timing the whole socket.connect()
call, which includes both the DNS lookup and the connect()
proper. To avoid this, it should first do a lookup of
www.google.com by other means and pass the IP, not the
hostname, to the socket.connect() call. Else, it would be
also measuring the DNS resolution time (which can be kinda
long if the nameserver isn't local).

So, we have three problems in the testcase already:
1) It should only run the two remote tests when use of the
'network' resource is enabled
2) It should not assume that any socket.error is a timeout;
it must do more checks to make sure it's really a timeout
3) It should avoid measuring the time spent in
getaddrinfo(3) by resolving the hostname to an IP before
starting the tests (this would also avoid the false failures
because of DNS failure and make them look like a testcase
setup error).

Should I make a separate bug for each of these?

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-26 00:06

Message:
Logged In: YES 
user_id=33168

Whoops, sorry my last comment was meant for bug #692951. 
This report helps clarify the other bug.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-26 00:03

Message:
Logged In: YES 
user_id=33168

The first problem seems to be a DNS lookup failure
(www.google.com).  What Linux distro and version are you
using?  Did you enable ipv6?  Anything else that could help
debug this problem?  Many developers run on Linux and I
haven't heard of this problem?  Is your network very slow? 
Do you have a firewall which introduces large latencies?

----------------------------------------------------------------------

Comment By: Cesar Eduardo Barros (cesarb)
Date: 2003-02-25 13:08

Message:
Logged In: YES 
user_id=57799

Weird, the attachment didn't work. Trying again to attach
the file.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=692988&group_id=5470


From noreply@sourceforge.net  Wed Feb 26 16:54:25 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Wed, 26 Feb 2003 08:54:25 -0800
Subject: [Python-bugs-list] [ python-Bugs-639806 ] Optional argument for dict.pop() method
Message-ID: <E18o4p7-0007tm-00@sc8-sf-web1.sourceforge.net>

Bugs item #639806, was opened at 2002-11-17 23:00
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=639806&group_id=5470

Category: Python Interpreter Core
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Fernando Pérez (fer_perez)
Assigned to: Nobody/Anonymous (nobody)
Summary: Optional argument for dict.pop() method

Initial Comment:
I think the new dict.pop() method should support an optional argument for its return value when a key is not found. 

The current behavior (raising KeyError always) IMHO violates the principle of least surprise, since the very similar get() method _does_ support an optional return value. I am filing this as a bug on suggestion by Neil Schemenauer in c.l.py.

Here is a python proof of concept which I am using , implemented as a function. If this were accepted, similar behavior would need to be coded in the pop() method itself, which I suppose is part of the C core.

#----------------------------------------------------------------------------
class NotGiven: pass

def popkey(dct,key,default=NotGiven):
    """Return dct[key] and delete dct[key]. dct is modified in-place.

    If default is given, return it if dct[key] doesn't exist, otherwise raise
    KeyError.  """

    try:
        val = dct[key]
    except KeyError:
        if default is NotGiven:
            raise
        else:
            return default
    else:
        del dct[key]
        return val
#----------------------------------------------------------------------------


----------------------------------------------------------------------

Comment By: Michael Stone (mbrierst)
Date: 2003-02-26 16:54

Message:
Logged In: YES 
user_id=670441

I'm not sure how "compelling" people will find it, but 
I submitted patch 693753 to implement this behavior.
If it is not desired, this bug should probably be closed.


----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2002-11-18 23:17

Message:
Logged In: YES 
user_id=21627

No need to worry. If somebody gets it done before, they will
just close this report, so you'll know its done. Otherwise,
we can leave it open nearly indefinitely - we are all
insanely busy, all the time :-)

----------------------------------------------------------------------

Comment By: Fernando Pérez (fer_perez)
Date: 2002-11-18 19:35

Message:
Logged In: YES 
user_id=395388

Martin, I'd love to give it a go, but right now I'm insanely busy (I shouldn't even be worrying about this :)

In about two weeks things will calm down a bit though, and I could look into it. But I don't know the interpreter core at all, so I'd probably need a bit of guidance as to what files to look at (I don't want to spend an afternoon grepping).

And  I know very little about the python interenals (for example, about how to handle python exceptions from the C side). But with a bit of guidance I can manage. So if nobody else who knows the system well and can do it in 10 minutes is interested, and if someone is willing to point me in the right direction, I can look into it two/three weeks from now.

----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2002-11-18 12:17

Message:
Logged In: YES 
user_id=21627

Fernando, I sympathize with this request in principle,
although I'm concerned about the ever-growing complexity of
dictionary methods.

Would you be interested in implementing an actual patch to
provide this feature?

----------------------------------------------------------------------

Comment By: Fernando Pérez (fer_perez)
Date: 2002-11-18 09:47

Message:
Logged In: YES 
user_id=395388

This is a repost of my reply in c.l.py, for reference here. Alex Martelli just posted a detailed reply to c.l.py, roughly along the lines of mine (with more specific arguments and code examples).

Raymond Hettinger wrote:

> Do you have use cases demonstrating the value of a default rather than an
> exception?

Sure. First, on principle: the exception option is still there, if no default is provided. What giving a default buys you is not having to trap the exception yourself. If you want a missing key to generate an exception, simply don't give a default and that's it. Since it doesn't change the existing semantics (the current form doesn't take any arguments, so there can't be any confusion), I think it's a good addition.

But you asked for code. Here's an example from a wrapper which needs to filter out a keyword dictionary given to a function before passing it downstream. It needs to remove keywords which won't be understood by the downstream function, but it knows how to make a default decision if the keyword wasn't given:

        # Filter out other options the original plot doesn't know
        hardcopy = popkey(keyw,'hardcopy',psargs['filename'] is not None)
        titles = popkey(keyw,'titles',0)

This uses my popkey() function, with the new method it would be simply

        # Filter out other options the original plot doesn't know
        hardcopy = keyw.pop('hardcopy',psargs['filename'] is not None)
        titles = keyw.pop('titles',0)

if my suggestion were accepted. I can always live with my popkey() function instead, if the crippled version is left in the language :) What I won't do is use the crippled pop() and wrap things everywhere with explicit try/except blocks. In the end that only hurts readaility and creates bloat.

This is part of the Gnuplot wrappers in IPython, if you want to see the full code (or I can give more context). IPython lives at 
http://www-hep.colorado.edu/~fperez/ipython

> Also, discuss why similarity with dict.get() applies instead of symmetry
> with list.pop() or dict.popitem().

- list.pop: I wasn't looking when that was written :) But I could argue similarly that an optional default argument would be a sensible, useful addition. No semantic break for existing code, and it saves people setting up try/except traps.

- dict.popitem: this one I'm ok with not having a default value, since it is meant to traverse the dict in essentially random order. Adding a default value would be like having a dict_with_default kind of object, which is definitely different from a regular python dict. popitem() returns a k/v pair and is typically used for exhausting a dict destructively, so it makes a lot of sense to break at the end. 

But pop(key) is much more specific, so I think it is sensible to be able to control its behavior more closely. Just like dict.get(), which allows straight through the function interface one to control what happens to missing keys. Semantically pop(key) is almost like get(key), with the difference that it deletes the k/v entry from the dict. It seems only natural then that pop() would support the same default arg behavior that get() has.

Cheers,

f.

----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2002-11-18 08:02

Message:
Logged In: YES 
user_id=80475

dict.pop(k) parallels list.remove(elem), dict.popitem(), and 
list.pop([i]) which all raise exceptions instead of returning 
a default.

Do you have compelling use cases?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=639806&group_id=5470


From noreply@sourceforge.net  Wed Feb 26 19:02:13 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Wed, 26 Feb 2003 11:02:13 -0800
Subject: [Python-bugs-list] [ python-Bugs-668662 ] Compiling C sources with absolute path bug
Message-ID: <E18o6on-0000Q4-00@sc8-sf-web4.sourceforge.net>

Bugs item #668662, was opened at 2003-01-15 13:59
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=668662&group_id=5470

Category: Distutils
Group: Python 2.3
>Status: Closed
>Resolution: Fixed
Priority: 5
Submitted By: Pearu Peterson (pearu)
Assigned to: A.M. Kuchling (akuchling)
Summary: Compiling C sources with absolute path bug

Initial Comment:
Py-2.3 distutils is broken in the following situation
when building extension modules:
If a file in C sources list is given with absolute
path, say
/path/to/foo.c then Py-2.3 distutils tries to create
the corresponding    
object file to /path/to/foo.o (earlier versions would
create it to, say, build/temp.linux-i686-2.2/foo.o).
This causes problems when
1) an user does not have permissions to write to
/path/to/, e.g. in multiuser systems
2) an user builds extension modules, say, using
different compilers or flags. Then /path/to/foo.o will
be in a way for subsequent builds.
IMHO, distutils should not create any files outside the
given
build directory, unless explicitely specified using
--build-temp.

Fix:
The problematic code seems to be in the method
ccompiler.object_filenames in file distutils/ccompiler.py.
As a fix, I suggest using the following object name for
a C source if given with absolute path: 
  ./build/temp.linux-i686-2.2/path/to/foo.o.

This can be achived by inserting the following line
just after
the line `base, ext = os.path.splitext(src_name)`:

 base = base[os.path.isabs(base):]

Pearu

----------------------------------------------------------------------

>Comment By: A.M. Kuchling (akuchling)
Date: 2003-02-26 14:02

Message:
Logged In: YES 
user_id=11375

Checked in as patch 1.56 of ccompiler.py.  I'll look into backporting to 2.2.  Thanks!

----------------------------------------------------------------------

Comment By: Pearu Peterson (pearu)
Date: 2003-02-16 15:11

Message:
Logged In: YES 
user_id=88489

Attached patch maps the base of a source file with absolute
path (and 
drive if present) to relative path before composing the
corresponding object file. This avoids creation of object
files out of the build directory. 

Pearu

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2003-01-16 08:28

Message:
Logged In: YES 
user_id=6656

Actually, I guessed after posting that something like that
could happen.

BTW, I strongly suspect this is related to the fix for this bug:

[ 599248 ] ext module generation problem

Can you come up with a patch for your suggested fix?  I'm
sure Andrew would appreciate it :-)

----------------------------------------------------------------------

Comment By: Pearu Peterson (pearu)
Date: 2003-01-15 14:48

Message:
Logged In: YES 
user_id=88489

FYI, distutils does not make this absolute path, but one can
specify
absolute paths in setup.py files, for instance.
Also, tools that create extension modules (e.g. f2py), may
ship with C sources that are installed as data files and
compiled
for each generated extension module separately (possibly
with different
compilers and flags so that pre-compilation of such sources
would not be an option).


----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2003-01-15 14:23

Message:
Logged In: YES 
user_id=6656

I'm a little boggled by how distutils would get an absolute
path, but I'm going to assign this to amk anyway :)

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=668662&group_id=5470


From noreply@sourceforge.net  Wed Feb 26 19:08:48 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Wed, 26 Feb 2003 11:08:48 -0800
Subject: [Python-bugs-list] [ python-Bugs-693474 ] distutils.setup() arguments missing and incorrect
Message-ID: <E18o6vA-0002m3-00@sc8-sf-web3.sourceforge.net>

Bugs item #693474, was opened at 2003-02-26 02:29
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=693474&group_id=5470

Category: Documentation
Group: Python 2.3
>Status: Closed
>Resolution: Fixed
Priority: 5
Submitted By: Stuart Bishop (zenzen)
>Assigned to: A.M. Kuchling (akuchling)
Summary: distutils.setup() arguments missing and incorrect

Initial Comment:
'home_page' should be 'url'
'platform' should be 'platforms'
'download_url' is undocumented, but used by pypi

----------------------------------------------------------------------

>Comment By: A.M. Kuchling (akuchling)
Date: 2003-02-26 14:08

Message:
Logged In: YES 
user_id=11375

Fixed in rev.1.50 of dist.tex; thanks!

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=693474&group_id=5470


From noreply@sourceforge.net  Wed Feb 26 19:10:52 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Wed, 26 Feb 2003 11:10:52 -0800
Subject: [Python-bugs-list] [ python-Bugs-639806 ] Optional argument for dict.pop() method
Message-ID: <E18o6xA-0006BE-00@sc8-sf-web1.sourceforge.net>

Bugs item #639806, was opened at 2002-11-17 23:00
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=639806&group_id=5470

Category: Python Interpreter Core
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Fernando Pérez (fer_perez)
Assigned to: Nobody/Anonymous (nobody)
Summary: Optional argument for dict.pop() method

Initial Comment:
I think the new dict.pop() method should support an optional argument for its return value when a key is not found. 

The current behavior (raising KeyError always) IMHO violates the principle of least surprise, since the very similar get() method _does_ support an optional return value. I am filing this as a bug on suggestion by Neil Schemenauer in c.l.py.

Here is a python proof of concept which I am using , implemented as a function. If this were accepted, similar behavior would need to be coded in the pop() method itself, which I suppose is part of the C core.

#----------------------------------------------------------------------------
class NotGiven: pass

def popkey(dct,key,default=NotGiven):
    """Return dct[key] and delete dct[key]. dct is modified in-place.

    If default is given, return it if dct[key] doesn't exist, otherwise raise
    KeyError.  """

    try:
        val = dct[key]
    except KeyError:
        if default is NotGiven:
            raise
        else:
            return default
    else:
        del dct[key]
        return val
#----------------------------------------------------------------------------


----------------------------------------------------------------------

>Comment By: Fernando Pérez (fer_perez)
Date: 2003-02-26 19:10

Message:
Logged In: YES 
user_id=395388

I've just been too busy to code the patch,
but I'm glad to see that mbrierst stepped
up to the plate.

I do find the feature rather useful, but for a
more thorough discussion, the original thread
from c.l.py can be read.  Here is A. Martelli's take
on the issue, with his usual level of care and detail:

http://groups.google.com/groups?q=dict.pop&hl=en&lr=&ie=UTF-8&oe=UTF-8&selm=Bm2C9.29398%24Yw.1473263%40news2.tin.it&rnum=1


----------------------------------------------------------------------

Comment By: Michael Stone (mbrierst)
Date: 2003-02-26 16:54

Message:
Logged In: YES 
user_id=670441

I'm not sure how "compelling" people will find it, but 
I submitted patch 693753 to implement this behavior.
If it is not desired, this bug should probably be closed.


----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2002-11-18 23:17

Message:
Logged In: YES 
user_id=21627

No need to worry. If somebody gets it done before, they will
just close this report, so you'll know its done. Otherwise,
we can leave it open nearly indefinitely - we are all
insanely busy, all the time :-)

----------------------------------------------------------------------

Comment By: Fernando Pérez (fer_perez)
Date: 2002-11-18 19:35

Message:
Logged In: YES 
user_id=395388

Martin, I'd love to give it a go, but right now I'm insanely busy (I shouldn't even be worrying about this :)

In about two weeks things will calm down a bit though, and I could look into it. But I don't know the interpreter core at all, so I'd probably need a bit of guidance as to what files to look at (I don't want to spend an afternoon grepping).

And  I know very little about the python interenals (for example, about how to handle python exceptions from the C side). But with a bit of guidance I can manage. So if nobody else who knows the system well and can do it in 10 minutes is interested, and if someone is willing to point me in the right direction, I can look into it two/three weeks from now.

----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2002-11-18 12:17

Message:
Logged In: YES 
user_id=21627

Fernando, I sympathize with this request in principle,
although I'm concerned about the ever-growing complexity of
dictionary methods.

Would you be interested in implementing an actual patch to
provide this feature?

----------------------------------------------------------------------

Comment By: Fernando Pérez (fer_perez)
Date: 2002-11-18 09:47

Message:
Logged In: YES 
user_id=395388

This is a repost of my reply in c.l.py, for reference here. Alex Martelli just posted a detailed reply to c.l.py, roughly along the lines of mine (with more specific arguments and code examples).

Raymond Hettinger wrote:

> Do you have use cases demonstrating the value of a default rather than an
> exception?

Sure. First, on principle: the exception option is still there, if no default is provided. What giving a default buys you is not having to trap the exception yourself. If you want a missing key to generate an exception, simply don't give a default and that's it. Since it doesn't change the existing semantics (the current form doesn't take any arguments, so there can't be any confusion), I think it's a good addition.

But you asked for code. Here's an example from a wrapper which needs to filter out a keyword dictionary given to a function before passing it downstream. It needs to remove keywords which won't be understood by the downstream function, but it knows how to make a default decision if the keyword wasn't given:

        # Filter out other options the original plot doesn't know
        hardcopy = popkey(keyw,'hardcopy',psargs['filename'] is not None)
        titles = popkey(keyw,'titles',0)

This uses my popkey() function, with the new method it would be simply

        # Filter out other options the original plot doesn't know
        hardcopy = keyw.pop('hardcopy',psargs['filename'] is not None)
        titles = keyw.pop('titles',0)

if my suggestion were accepted. I can always live with my popkey() function instead, if the crippled version is left in the language :) What I won't do is use the crippled pop() and wrap things everywhere with explicit try/except blocks. In the end that only hurts readaility and creates bloat.

This is part of the Gnuplot wrappers in IPython, if you want to see the full code (or I can give more context). IPython lives at 
http://www-hep.colorado.edu/~fperez/ipython

> Also, discuss why similarity with dict.get() applies instead of symmetry
> with list.pop() or dict.popitem().

- list.pop: I wasn't looking when that was written :) But I could argue similarly that an optional default argument would be a sensible, useful addition. No semantic break for existing code, and it saves people setting up try/except traps.

- dict.popitem: this one I'm ok with not having a default value, since it is meant to traverse the dict in essentially random order. Adding a default value would be like having a dict_with_default kind of object, which is definitely different from a regular python dict. popitem() returns a k/v pair and is typically used for exhausting a dict destructively, so it makes a lot of sense to break at the end. 

But pop(key) is much more specific, so I think it is sensible to be able to control its behavior more closely. Just like dict.get(), which allows straight through the function interface one to control what happens to missing keys. Semantically pop(key) is almost like get(key), with the difference that it deletes the k/v entry from the dict. It seems only natural then that pop() would support the same default arg behavior that get() has.

Cheers,

f.

----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2002-11-18 08:02

Message:
Logged In: YES 
user_id=80475

dict.pop(k) parallels list.remove(elem), dict.popitem(), and 
list.pop([i]) which all raise exceptions instead of returning 
a default.

Do you have compelling use cases?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=639806&group_id=5470


From noreply@sourceforge.net  Wed Feb 26 19:22:41 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Wed, 26 Feb 2003 11:22:41 -0800
Subject: [Python-bugs-list] [ python-Bugs-471893 ] Security review of pickle/marshal docs
Message-ID: <E18o78b-0003OQ-00@sc8-sf-web3.sourceforge.net>

Bugs item #471893, was opened at 2001-10-16 18:42
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=471893&group_id=5470

Category: Extension Modules
Group: None
Status: Open
Resolution: Fixed
Priority: 7
Submitted By: Tim Peters (tim_one)
Assigned to: A.M. Kuchling (akuchling)
Summary: Security review of pickle/marshal docs

Initial Comment:
Paul Rubin points out that the security implications 
of using marshal and/or pickle aren't clear from the 
docs.  Assigning to Jeremy as he's more sensitive to 
such issues than I am; maybe Barry would like to get 
paranoid too <wink>.

A specific example:  the pickle docs say that pickle 
doesn't support code objects, and "at least this 
avoids the possibility of smuggling Trojan horses into 
a program".  However,

1) The marshal docs don't mention this vulnerability 
at all.

while

2) The pickle docs don't spell out possible dangers 
due to things pickle does that marshal doesn't (like 
importing modules, and running class constructors).

----------------------------------------------------------------------

>Comment By: A.M. Kuchling (akuchling)
Date: 2003-02-26 14:22

Message:
Logged In: YES 
user_id=11375

OK; here's a reworked version that retitles the "Pickle security" 
section to "Subclassing Unpicklers", leaving the discussion of subclassing alone except for a few typo fixes.


----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2003-02-06 14:40

Message:
Logged In: YES 
user_id=31435

I think there are several reasons to override these methods.  
The one most relevant to this bug report is that, while Python 
has stopped pretending that pickles are secure by default, 
the choke points are still there, and motivated users can still 
expolit them.

For example, search pickle.py for __import__.  The only 
occurrence of __import__ in the Unpickler class is in method 
find_class(), and that's by design.  If a user overrides 
find_class(), the only imports the Unpickler *can* do are 
those the user explicitly performs in their own find_class() 
implementation.  So if that's a notion of "security" a user is 
happy with, they can still have it.  The docs trying to describe 
this are still valid.  It's only the "by magic" safety checks that 
have gone away (and they were buggy anyway, so no loss).

----------------------------------------------------------------------

Comment By: A.M. Kuchling (akuchling)
Date: 2003-02-06 09:29

Message:
Logged In: YES 
user_id=11375

The Cookie classes that use pickle have DeprecationWarnings in 
2.3, and should disappear in 2.4.  If {find,load}_global are 
still going to be supported, though, then they should still be documented (though, given that you shouldn't be unpickling untrusted data, why would you ever bother overriding find_global?)


----------------------------------------------------------------------

Comment By: Barry A. Warsaw (bwarsaw)
Date: 2003-02-05 18:03

Message:
Logged In: YES 
user_id=12800

I'll just mention that anybody using anything other than
Cookie.SimpleCookie is insane, and even using the Cookie
module at all to parse real-world cookies is mildly nuts
because it's way too strict.  I prefer m&m's in my cookies.

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2003-02-05 17:57

Message:
Logged In: YES 
user_id=31435

Andrew, didn't you go overboard in deleting text here?  The 
__safe_for_unpickling__ check is gone in 2.3, but most of the 
rest of the text still applies.  In particular, Cookie is still a 
problem for people inclined to worry about this, and overriding 
methods like find_global and load_global is still valuable stuff 
(the unpickler still never imports anything that doesn't come 
from an opcode triggering one of these methods).  

----------------------------------------------------------------------

Comment By: Barry A. Warsaw (bwarsaw)
Date: 2003-02-05 16:29

Message:
Logged In: YES 
user_id=12800

Karmically (no, not comically) reassigning to Tim who
started this whole thing, and who happens to be dumping his
pickles these days.

----------------------------------------------------------------------

Comment By: A.M. Kuchling (akuchling)
Date: 2003-02-05 15:40

Message:
Logged In: YES 
user_id=11375

Re-opening.  With the changes to pickling in Python 2.3, 
the security material should be replaced with a warning to 
not unpickle untrusted data.  Patch attached.



----------------------------------------------------------------------

Comment By: Barry A. Warsaw (bwarsaw)
Date: 2001-11-18 11:08

Message:
Logged In: YES 
user_id=12800

You're right of course.  I meant to fix that and forgot. 
Will do so asap.  Glad you like the rest of it, though! :)

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2001-11-18 01:56

Message:
Logged In: NO 

Barry - the new docs just went up and they're a big
improvement over the old.  However the sentence 
"The issue here is usually not that a class's constructor
will get called -- it won't by the unpickler -- but that the
class's destructor (i.e. its __del__() method) might get
called when the object is garbage collected." isn't
correct.  

There's a flag in the pickle stream that tells
the unpickler that the pickler found a __getinitargs__
method at pickling time.  If the flag is set in the pickle,
then the unpickler calls the class constructor, whether
there's a __getinitargs__ method in the class or not.

The sample exploit that I posted earlier on,
<http://www.nightsong.com/phr/python/pickletest.py>,
is an example of an artificially concocted pickle that
makes a class constructor get called.


----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2001-11-17 17:07

Message:
Logged In: NO 

Are the new docs downloadable from somewhere?  thanx  --Paul


----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2001-11-15 19:43

Message:
Logged In: NO 

Barry, can you also do something about the Cookie module,
or at least about its documentation?  If Cookie is aliased
to SmartCookie, I think that needs mention.
 --Paul

----------------------------------------------------------------------

Comment By: Barry A. Warsaw (bwarsaw)
Date: 2001-11-15 19:12

Message:
Logged In: YES 
user_id=12800

I have rewritten the pickle documentation, and updated the
marshal, cPickle, and copy_reg documentation.  I believe all
the documentation issues raised here have been fixed.  

Implementation issues will be pushed to Python 2.3, and the
plan is to write a PEP to plan future work.

----------------------------------------------------------------------

Comment By: Barry A. Warsaw (bwarsaw)
Date: 2001-11-13 16:39

Message:
Logged In: YES 
user_id=12800

Assigning to Jeremy so he'll remember to forward me Jim's
email.  Jeremy can ping-pong it to Fred if he wants, and
then reassign to me after forwarding the email.

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2001-11-13 01:18

Message:
Logged In: NO 

The find_global variable sounds encouraging and if it
fixes this problem, that's great.  I'd still feel better
if the eval call goes away.  Is removing it allowed
under the 2.2 feature freeze?  It doesn't affect anything
visible, so no tests would have to change, etc.

Paul


----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2001-11-12 19:38

Message:
Logged In: NO 

It's possible that the eval is safe, but validating that
takes a line by line code review of all the paths that
evaling a string can go through.  It's also brittle in
that maybe someone will change the evaluator (say to
support Perl-like interpolation) in a future Python version
and not remember to change the unpickler.  Something like
that has already happened with the Cookie module.  My
guess is that it happened with the pickle module--the
whole point of that quoted string check is that the 
original pickle implementer didn't trust the input. 
The stuff about unpickling class instances was added
later (maybe by another person) without remembering the
security issue. 

Using eval this way is like storing a vat of cyanide in
your child's bedroom.  Sure, maybe if you check the seals
and locks on the vat carefully enough, you can convince
yourself that your child won't be able to get to the
cyanide.  But wouldn't you feel safer just not having the
vat there at all?  That's basic safety-consciousness.
Security consciousness works the same way.  Try to keep
dangerous ingredients and attackers as far away from each
other as possible.

Paul



----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2001-11-12 13:52

Message:
Logged In: YES 
user_id=31435

Why are people (Paul, Jeremy) concerned about eval'ing 
strings?  cPickle and pickle both check that they're 
properly quoted, and this isn't sh or Perl:  Python has 
no "dynamic" gimmicks buried in string literals.  All 
eval'ing a string literal can do is produce a binary blob 
via interpeting simple escape sequences.  They're like C 
strings this way -- maybe we'll run out of memory, but 
that's it.

I would agree that Python should be refactored internally 
to supply a clean function for changing string literals 
into binary blobs, but that would be for clarity and 
efficiency, not security.

----------------------------------------------------------------------

Comment By: Jim Fulton (dcjim)
Date: 2001-11-12 13:14

Message:
Logged In: YES 
user_id=73023

It sounds like there are some documentation bugs:

- The security ramifications are not discussed, nor are the
remedies.

- The cPickle module isn't documented very well. I submitted
some
   documentation a long time ago that never got incorporated
AFAIK.
   I wish I still had it. :)

- cPickle has a feature for turning off instance support and
for 
   restricting which classes can be unpickled. You can set
the find_global
   attribute on a cPickle.Unpickler. The find_global
attribute can be 
   a function or None.  If it is None, then no instances can
be 
   unpickled. If it is a function, then it should accept a
module and name
   and return the corresponding global. It is responsible
for looking
   up the global and can raise an error to prevent a
global's use.

   See the ZEO storage server implementation for an example
   of using this hook.


  



----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2001-11-12 01:33

Message:
Logged In: NO 

I like marshal and think it's much cleaner than pickle.
I'd like to see its format cleaned up and documented
so it can be a portable transport format.  Even the
recursion issue isn't so compelling--there's only a danger
on marshalling, not unmarshalling, and the application
has control over what it marshals.  So I think the idea
of documenting marshal (bug #467384) should be kept alive.
However, it shouldn't be changed in 2.2.  

If the Cookie class is still really aliased by default to
SmartCookie, that's a bad bug and should definitely be
fixed ASAP.  The Cookie docs clearly say not to use
SmartCookie or SerialCookie.  In fact I think SmartCookie
and SerialCookie should be disabled altogether.  Any
applications using them on the open internet have a security
but and should be fixed immediately.

Here's a possible temporary approach to fixing SmartCookie
and SerialCookie: append a cryptographic message
authentication code (MAC) to each pickled cookie.  That
means the application has some secret string K as a
configuration parameter (it should be 10 or more random
8-bit characters, or 20 or more random hex digits, etc). 
Then a smart cookie would be a pickle p, followed by
SHA(p+K).  The Cookie module would validate the MAC before
calling unpickle, and raise an exception if the MAC wasn't
valid.

The security of this scheme rests on K being kept secret
from attackers, which is generally not an easy thing to
manage, but it's something to think about.

Paul

----------------------------------------------------------------------

Comment By: Jeremy Hylton (jhylton)
Date: 2001-11-10 17:07

Message:
Logged In: YES 
user_id=31392

I don't think we should be doing anything about marshal.  
Maybe we should name in pyclib or something <0.9 wink>.  It 
works fine for .pyc files, but I don't see a reason for it 
to do anymore than is necessary for that purpose.

I think the notion of an unpickler that only handles 
builtin datatypes is the most attractive option you offer, 
but Paul has a good point about eval for strings.  (It 
currently has some hacks that address specific exploits, 
but I doubt they are sufficient.)  I also wonder how hard 
it is to handle builtin types and avoid subclasses of 
builtin types.

If there are any changes to pickle, I think we need to be 
careful about how it is described.  If we claim that an 
unpickler is safe for untrusted pickles, we've made a 
fairly strong claim.  I still think such a design change 
requires a PEP that includes some requirements and use 
cases and a thorough analysis of potential exploits. 


----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2001-11-10 14:42

Message:
Logged In: NO 

See bug #467384 for discussion about marshal.  Besides the
recursion issue, marshal's format is explicitly undocumented
and subject to change--you can't rely on it to interoperate
between two different Python versions, so it's no good as
an RPC serializer.  The format has kludges (e.g. the
representation of long ints) that make it undesirable to
freeze and document it and force future versions to be
backward compatible.

Adding a pickle.loads flag to prevent instance unpickling
isn't perfect but is probably the best alternative on 
your list.  Perhaps the flag can have a value that allows
unpickling the instances by restoring the instance 
attributes rather than calling the initializer.  That's
not always the right way to unpickle an instance (that's
why the unpickler no longer works that way) but it's good
enough a lot of the time.  

There's another issue with pickle/cPickle which is that they
unpickle quoted strings by evaling them.  This is scary.
While I don't see an immediate exploit, I also haven't
examined the 1000's of lines of code I'd need to examine
to convince myself that there's NOT an exploit.  I think
the unpickler should be changed to never call eval but just
parse the string as it needs to.  

Guido seemed to think pickle might have other possible
exploits.  I don't know what he had in mind but before
declaring it safe for untrusted data I think it needs to
be gone over with a fine toothed comb.

Paul


----------------------------------------------------------------------

Comment By: Barry A. Warsaw (bwarsaw)
Date: 2001-11-10 12:37

Message:
Logged In: YES 
user_id=12800

I'm going to agree with Paul that this is a problem needing
fixing, however there are really several issues.

1. Cookie module makes it too easy to code exploits.  Cookie
exports a class, also called Cookie, which is aliased to
SmartCookie, so that a naive program will simply pass cookie
data to Cookie.Cookie() and you're screwed.  So, Cookie
module's defaults should be for more security rather than
less, and Cookie.Cookie should be aliased to SimpleCookie
instead.

2. There is no built-in safe mechanism for de-serializing
data from untrusted sources.  You can't use pickle without
overloading a "magic" method.  You can't use cPickle because
you can't do the overloading trick.  You can't use marshal
because it isn't bulletproof against recursive
datastructures.  So how /do/ you do it?

I think it just may be serious enough to deal with in Python
2.2, and I volunteer to address it (so I'll steal this bug
report).  Without looking at the code, or the level of
effort necessary, I would make the following suggestions:

1. Add to the public interface of pickle and cPickle, a flag
that either disables the unpickling of instances altogether,
or disables calling any code with unpickled data, e.g.
constructors.

2. Fix marshal to be bulletproof against recursive
datastructures.
3. Update the docs for both pickle/cPickle and marshal to
explain how to safely write de-serializers of untrusted strings.

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2001-11-09 04:21

Message:
Logged In: NO 

Well, Guido and Tim agree with you that it's not a pickle
bug.  I still feel it is one, because its docs currently
make you think you can securely load untrusted pickles, and
because it's a natural, non-obscure thing to want to do
(see pyro and the cookie module), but whatever.  If it's
not a code bug then I feel it's a significant functionality
shortcoming in the python library.

Pyro uses pickle to serialize data for RPC calls over the
internet.  A malicious client could make a hostile pickle
take over the server.  The cookie module lets web
applications store user session state in browser cookies.
Its SerialCookie and SmartCookie classes let you put
arbitrary Python objects into the user session, and
serializes them when pickle.  Again, a malicious client
can make a hostile pickle, send it in a cookie header to
the http server, and take over the server when the 
application unpickles the cookie.

The current documentation for the pickle module makes it
very clear to me that the doc writer thought it was safe
to unpickle untrusted cookies.  If pickle wasn't designed
for that, then there was a communication failure between
the designer and the doc writer.

Yes, I'm willing to help with a PEP for fixing this
situation.

Paul


----------------------------------------------------------------------

Comment By: Jeremy Hylton (jhylton)
Date: 2001-11-08 12:37

Message:
Logged In: YES 
user_id=31392

I don't think of the issue you describe as a bug in the
code.  You're suggesting a new feature for pickle.  As far
as I can tell, the original design requirements for pickle
did not include the ability to securely load a pickle from
an untrusted source.

It may be a legitimate feature request, but it's too late to
make it into Python 2.2.  I suggest we look at the design
issues for Python 2.3 and decide if it's a feature we want
to support.  I imagine a PEP may be necessary to lay out the
issues and the solution.  Do you want to have a hand in that
PEP?

I still don't understand what it means that Pyro and cookie
were bit by a bug.  It sounds like they were using pickle in
ways that pickle was not intended to support.  A careful
analysis of how those two applications use pickle would be
helpful for generating requirements.


----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2001-11-07 18:54

Message:
Logged In: NO 

IMO it's a code bug that you can't unpickle strings from
untrusted sources.  Pyro and the cookie module are examples
of programs that got bitten by this bug.  Whether it's
really a bug is a matter of opinion--I had a big email
exchange with Guido and Tim about it, and they felt it
was enough to fix the pickle documentation.

Pickle has the same problem as cPickle, but with pickle
you can subclass the pickler and override the method that
unpickles class objects, and work around the (IMO) bug.
The workaround doesn't help cPickle since cPickle can't
be subclassed.  See bug #467384 for some related discussion.

Paul


----------------------------------------------------------------------

Comment By: Jeremy Hylton (jhylton)
Date: 2001-11-07 17:02

Message:
Logged In: YES 
user_id=31392

What's the code bug?  Your last message has a lot of gloom
and doom <wink>, but I'm not sure what specific problem
you'd like to see fixed.  Are you saying that something
needs to be fixed in cPickle and not in pickle?

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2001-11-07 15:08

Message:
Logged In: NO 

Irmen de Jong points out that the standard cookie module
uses pickling for serial and smart cookies.  The 2.1.1
cookie module docs explicitly say not to use those
classes because of the security hole--that they're provided
for backward compatibility only (but with what?!  Any
server that uses those classes on open ports needs to be
changed right away).

Irmen's library, http://pyro.sourceforge.net, also uses
unpickle insecurely (he's aware of this now and is figuring
out a fix).

IMO this is really a code bug rather than a documentation
bug, and should be fixed in the code rather than the docs.
Documenting the bug rather than fixing it leaves a
deficiency in the Python library: obvious uses of pickling,
like Pyro and the cookie module, can't be implemented
using cPickle and have to resort to a slower Python
deserializer, or use marshal and have possible compatibility
problems between versions (and not be able to serialize
class instances).

Paul



----------------------------------------------------------------------

Comment By: paul rubin (phr)
Date: 2001-10-16 19:06

Message:
Logged In: YES 
user_id=72053

Certainly anyone unserializing potentially malicious data
with pickle, marshal, or anything else, should check the
results before doing anything dangerous with them (like
executing code).  However, unpickle can potentially do
damage before it even returns, by creating loading modules
and creating initialized class instances.  So pickle.loads
should never be used on untrusted strings, except possibly
in a rexec wrapper (as proposed by Tim).  Another
possibility (also by Tim) is to override the load_inst
method of the Pickler class, though I don't think you can
do that for cPickle.

A sample exploit for unpickle can be found at
<http://www.nightsong.com/phr/python/pickletest.py>.
Unpickling the test string runs penguin.__init__ contrary
to the doc's saying no initialization unless there's a
__getinitargs__ method in the class def.

The "exploding penguin" class is artificial, but
applications are vulnerable if there's an unsafe
constructor anywhere in any class of the application or in
the python library (example: the NNTP constructor opens an
IP connection to an arbitrary address, so a malicious
imported string can send a message through your firewall
when you import it).


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=471893&group_id=5470


From noreply@sourceforge.net  Wed Feb 26 22:19:59 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Wed, 26 Feb 2003 14:19:59 -0800
Subject: [Python-bugs-list] [ python-Bugs-693955 ] Can't multiply str and bool
Message-ID: <E18o9uB-0003xU-00@sc8-sf-web2.sourceforge.net>

Bugs item #693955, was opened at 2003-02-26 17:19
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=693955&group_id=5470

Category: Python Interpreter Core
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: A.M. Kuchling (akuchling)
Assigned to: Nobody/Anonymous (nobody)
Summary: Can't multiply str and bool

Initial Comment:
Bug in 2.3a2+: 'abc' * True raises a TypeError:
unsupported operand type(s) for *: 'str' and 'bool' .

site.py does:  initial_slashes = path.startswith('/');
  path = '/'*initial_slashes + path .
This now fails.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=693955&group_id=5470


From noreply@sourceforge.net  Wed Feb 26 22:58:39 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Wed, 26 Feb 2003 14:58:39 -0800
Subject: [Python-bugs-list] [ python-Bugs-693996 ] email.Parser trashes header
Message-ID: <E18oAVb-0000CT-00@sc8-sf-web1.sourceforge.net>

Bugs item #693996, was opened at 2003-02-26 17:58
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=693996&group_id=5470

Category: Python Library
Group: Python 2.2.2
Status: Open
Resolution: None
Priority: 5
Submitted By: Sam Shah (sshah)
Assigned to: Nobody/Anonymous (nobody)
Summary: email.Parser trashes header

Initial Comment:

email.Parser trashes a long-lined recipient list if it
is parsed more than once.

The attached interpreter transcript demonstrates this
problem. During the first parsing, it rewrites the
header of the message (okay). But during the second
parsing, recipient C is missing from the list.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=693996&group_id=5470


From noreply@sourceforge.net  Wed Feb 26 23:17:47 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Wed, 26 Feb 2003 15:17:47 -0800
Subject: [Python-bugs-list] [ python-Bugs-693996 ] email.Parser trashes header
Message-ID: <E18oAo7-0000rj-00@sc8-sf-web1.sourceforge.net>

Bugs item #693996, was opened at 2003-02-26 17:58
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=693996&group_id=5470

Category: Python Library
Group: Python 2.2.2
Status: Open
Resolution: None
Priority: 5
Submitted By: Sam Shah (sshah)
>Assigned to: Barry A. Warsaw (bwarsaw)
Summary: email.Parser trashes header

Initial Comment:

email.Parser trashes a long-lined recipient list if it
is parsed more than once.

The attached interpreter transcript demonstrates this
problem. During the first parsing, it rewrites the
header of the message (okay). But during the second
parsing, recipient C is missing from the list.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=693996&group_id=5470


From noreply@sourceforge.net  Thu Feb 27 01:12:17 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Wed, 26 Feb 2003 17:12:17 -0800
Subject: [Python-bugs-list] [ python-Bugs-694062 ] os.popen() hangs on {Free,Open}BSD
Message-ID: <E18oCav-0001jW-00@sc8-sf-web2.sourceforge.net>

Bugs item #694062, was opened at 2003-02-26 18:12
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=694062&group_id=5470

Category: Python Interpreter Core
Group: Platform-specific
Status: Open
Resolution: None
Priority: 5
Submitted By: Jason R. Mastaler (jasonrm)
Assigned to: Nobody/Anonymous (nobody)
Summary: os.popen() hangs on {Free,Open}BSD

Initial Comment:
I'm seeing odd behavior with os.popen() on 
certain platforms such as FreeBSD and OpenBSD. 
 
FWIW, I posted about this problem almost two 
years ago on c.l.p, and never got a resolution. 
 
See 
http://mail.python.org/pipermail/python-list/2001-May/042373.html 
 
Now, many Python and FreeBSD upgrades later, 
I see the problem still exists. Now however, I need to 
find a solution as I can't workaround it any longer. 
 
I've attached about the simplest test case I could 
produce below. You can call it from a .forward or 
.qmail file.  
 
On Free/OpenBSD, the program runs but leaves a  
python process lying around: 
 
jasonrm    142  0.0  0.2  2576 2008  ??  I     5:42PM   
0:00.02 python /tmp/hang (python2.2) 
 
On other platforms such as Linux, the program also 
runs, but no process is left hanging around.  
 
I've been able to reproduce the problem on Free/OpenBSD 
only. The problem doesn't appear on IRIX, BSD/OS or  
Linux. 
 
Please let me know if I can do anything more to help 
debug this.  

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=694062&group_id=5470


From noreply@sourceforge.net  Thu Feb 27 12:52:10 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 27 Feb 2003 04:52:10 -0800
Subject: [Python-bugs-list] [ python-Feature Requests-694320 ] Add list.join()
Message-ID: <E18oNWE-0007Un-00@sc8-sf-web3.sourceforge.net>

Feature Requests item #694320, was opened at 2003-02-27 08:52
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=694320&group_id=5470

Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Daniel (kamek)
Assigned to: Nobody/Anonymous (nobody)
Summary: Add list.join()

Initial Comment:
While there's string.join(list), I believe it'd make much 
more sense to have list.join(string). In most languages 
(and also in the string module), the first argument for join 
functions is the list, and the second is the separator. 
Not to mention code using methods directly in strings, 
like "' '.join(...)" is uglier :)

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=694320&group_id=5470


From noreply@sourceforge.net  Thu Feb 27 13:08:54 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 27 Feb 2003 05:08:54 -0800
Subject: [Python-bugs-list] [ python-Feature Requests-694339 ] Dedenting with Shift+Tab
Message-ID: <E18oNmQ-0006ox-00@sc8-sf-web2.sourceforge.net>

Feature Requests item #694339, was opened at 2003-02-27 09:08
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=694339&group_id=5470

Category: IDLE
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Daniel (kamek)
Assigned to: Nobody/Anonymous (nobody)
Summary: Dedenting with Shift+Tab

Initial Comment:
IDLE already provides quick block indenting, by 
selecting one or more lines and pressing Tab. So, it 
would make sense to have Shift+Tab to do the opposite 
work. While there's already Alt+[ (and Alt+] to indent), 
Shift+Tab is more intuitive, especially to those who are 
used to other Windows developer tools, like Visual 
Studio or TextPad.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=694339&group_id=5470


From noreply@sourceforge.net  Thu Feb 27 15:44:38 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 27 Feb 2003 07:44:38 -0800
Subject: [Python-bugs-list] [ python-Bugs-694431 ] Python 2.3a2 Build fails on HP-UX11i
Message-ID: <E18oQD8-0006lk-00@sc8-sf-web3.sourceforge.net>

Bugs item #694431, was opened at 2003-02-27 15:44
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=694431&group_id=5470

Category: Build
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Richard Townsend (rptownsend)
Assigned to: Nobody/Anonymous (nobody)
Summary: Python 2.3a2 Build fails on HP-UX11i

Initial Comment:
While building Python 2.3a2 on HP-UX11i I got the 
following error:

ld -b build/temp.hp-ux-B.11.11-9000/785-
2.3/_iconv_codec.o -L/opt/python/lib -
L/usr/local/lib -o build/lib.hp-ux-B.11.11-
9000/785-2.3/_iconv_codec.sl
/usr/lib/dld.sl: Unresolved symbol: libiconv_open 
(code)  from build/lib.hp-ux-B.11.11-9000/785-
2.3/_iconv_codec.sl
sh[3]: 25624 Abort(coredump)
*** Error exit code 134


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=694431&group_id=5470


From noreply@sourceforge.net  Thu Feb 27 14:00:02 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 27 Feb 2003 06:00:02 -0800
Subject: [Python-bugs-list] [ python-Feature Requests-694374 ] Recursive regular expressions
Message-ID: <E18oOZu-0001ha-00@sc8-sf-web4.sourceforge.net>

Feature Requests item #694374, was opened at 2003-02-27 10:00
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=694374&group_id=5470

Category: Regular Expressions
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Daniel (kamek)
Assigned to: Nobody/Anonymous (nobody)
Summary: Recursive regular expressions

Initial Comment:
It would be great if Python's regexps supported recursive 
regular expressions, just like PCRE. For that it'd need 
its formations (?R), (?<name>) and (?P>name).

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=694374&group_id=5470


From noreply@sourceforge.net  Thu Feb 27 17:49:11 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 27 Feb 2003 09:49:11 -0800
Subject: [Python-bugs-list] [ python-Bugs-693580 ] socket module has no default timeout
Message-ID: <E18oS9f-0004Ea-00@sc8-sf-web3.sourceforge.net>

Bugs item #693580, was opened at 2003-02-26 07:37
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=693580&group_id=5470

Category: None
Group: None
>Status: Deleted
>Resolution: Invalid
>Priority: 1
Submitted By: Steve Holden (holdenweb)
Assigned to: Nobody/Anonymous (nobody)
Summary: socket module has no default timeout

Initial Comment:
Python 2.3a2.

The new socket module is a great implementation of Tim 
O'Malley's original concept. From the documentation, 
however, it appears to be missing one feature.

With the timeoutsocket module one could call 
the "setdefaulttimeout()" function to establish a timeout 
for any socket created without explicit knowledge of the 
timeout features. This means that code using ftplib, 
smtplib and friends could be readily conditioned to use 
timeouts without modification to the client modules.


----------------------------------------------------------------------

>Comment By: Steve Holden (holdenweb)
Date: 2003-02-27 12:49

Message:
Logged In: YES 
user_id=88157

Sorry, Geoff Talvolva points out I was talking through the top 
of my head.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=693580&group_id=5470


From noreply@sourceforge.net  Fri Feb 28 00:58:45 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 27 Feb 2003 16:58:45 -0800
Subject: [Python-bugs-list] [ python-Bugs-693094 ] Python does not build --with-pydebug on Tru64 with vendor cc
Message-ID: <E18oYrN-0003N0-00@sc8-sf-web1.sourceforge.net>

Bugs item #693094, was opened at 2003-02-25 11:48
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=693094&group_id=5470

Category: Build
Group: Platform-specific
Status: Open
Resolution: None
Priority: 5
Submitted By: Stephan A. Terre (sfiedler)
Assigned to: Nobody/Anonymous (nobody)
Summary: Python does not build --with-pydebug on Tru64 with vendor cc

Initial Comment:
The default standards-compliance level for the
Compaq/HP compiler defines the assert macro using the
if statement (if...then) rather than the ternary if
expression (?:). As a result, when Python is configured
--with-pydebug (so NDEBUG is not defined), the macro
Py_SAFE_DOWNCAST expands to illegal C (containing a
statement inside an expression).

This can be fixed by asserting the -std switch to the
vendor compiler. Testing of 2.2.2 CVS and 2.3 CVS
indicates that there is no bad side effect to asserting
-std. 

I hope to develop a fix for this in configure.in
sometime in the next couple of days. When I have done
so, I will attach it to this bug.

For the record, here are the variations of the -std
switch on the vendor compiler:

  -std0
      Selects the K & R language mode. This is the
default.  Enforces the K &
      R programming style, with certain ANSI extensions
in areas where the K
      & R behavior is undefined or ambiguous. In
general, -std0 compiles most
      pre-ANSI C programs and produces expected
results.  The -std0 option
      causes the __STDC__ macro to be undefined.


  -std
      Selects the relaxed ANSI language mode. Enforces
the ANSI C standard,
      but allows some common programming practices
disallowed by the stan-
      dard.

      This option does not restrict the Tru64 UNIX name
space (for example,
      the names defined in system header files).  To
restrict that name space
      so that only ANSI C reserved names are visible
from the ANSI C header
      files, use the _ANSI_C_SOURCE macro. See
standards(5) for more details.

      This option causes the macro __STDC_ to be
defined to 0.

  -std1
      Selects the strict ANSI language mode. Strictly
enforces the ANSI C
      standard and all its prohibitions (such as those
that apply to the han-
      dling of void types, the definition of lvalues in
expressions, the mix-
      ing of integrals and pointers, and the
modification of rvalues).

      This option does not restrict the Tru64 UNIX name
space (for example,
      the names defined in system header files).  To
restrict that name space
      so that only ANSI C reserved names are visible
from the ANSI C header
      files, use the _ANSI_C_SOURCE macro. See
standards(5) for more details.

      This option causes the macro __STDC_ to be
defined to 1. Note that this
      option also affects linker-defined symbols.  See
ld(1) for more infor-
      mation.

      This option turns on ANSI aliasing rules
(-ansi_alias option).



----------------------------------------------------------------------

>Comment By: Stephan A. Terre (sfiedler)
Date: 2003-02-27 17:58

Message:
Logged In: YES 
user_id=246063

Attached to this bug is a patch to configure.in and configure. I
believe pretty much all of the three decisions I made need
review :)

1. I put the -std flag in BASECFLAGS an OSF1-specific case
in the
non-gcc branch. Should there be some other check here that
we're using
the vendor C compiler? If so, can anyone point me at an example?

2. Upon testing, I found that the getpgrp function has a
different
signature when compiled with -std than without. As a result, the
posixmodule didn't build. My solution was to change the
definition of
CC in configure.in to be "$CC $BASECFLAGS $OPT" after the
last change
to OPT and BASECFLAGS. My thinking is that we ought to be
running
feature and behavior tests with the flags that we'll be
using to build
the interpreter. However, if this goes against the spirit or
policy of
autoconf, we should do something else!

3. I happened to notice an XXX about Monterey, so I did a little
investigation and added to the note. Hopefully this is not a
gaffe.

I have tested the changes to configure.in on Tru64 4.0F both
with and
without the --with-pydebug flag to configure. The test suite
behaved
the same in each case as before my mods.

I also tested the changes --with-pydebug on a Red Hat 7
system; they
seemed good there too.

If I should have put this patch in the Patch Manager rather than
attaching to the bug directly, I apologize and promise never
to do it
wrongly again :)

Please let me know by posting a followup here or via email
if there is
anything else I should do for this bug. 



----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=693094&group_id=5470


From noreply@sourceforge.net  Fri Feb 28 02:55:40 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 27 Feb 2003 18:55:40 -0800
Subject: [Python-bugs-list] [ python-Bugs-694812 ] setup.py imports pwd before it's built if HOME not set
Message-ID: <E18oagW-0001ll-00@sc8-sf-web4.sourceforge.net>

Bugs item #694812, was opened at 2003-02-27 19:55
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=694812&group_id=5470

Category: Build
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Stephan A. Terre (sfiedler)
Assigned to: Nobody/Anonymous (nobody)
Summary: setup.py imports pwd before it's built if HOME not set

Initial Comment:
The function check_environ in Lib/distutils/util.py
imports the 'pwd' module if the HOME environment
variable is not set and os.name is 'posix' (as it is on
at least Linux, Solaris, and Tru64 Unix). In the
context of building Python, this happens before the pwd
module has been built. The error is reproduced below.

I can work around this easily. However, since the error
message was somewhat oblique, it was confusing to
diagnose. Perhaps there's some way to improve the
diagnostic if the underlying problem cannot be fixed.

case $MAKEFLAGS in  *-s*) CC='cc' LDSHARED='ld -shared
-expect_unresolved "*"' OPT='-DNDEBUG -O -Olimit 1500'
./python -E ./setup.py -q build;;  *) CC='cc'
LDSHARED='ld -shared -expect_unresolved "*"'
OPT='-DNDEBUG -O -Olimit 1500' ./python -E ./setup.py
build;;  esac
Traceback (most recent call last):
  File "./setup.py", line 795, in ?
    main()
  File "./setup.py", line 790, in main
    scripts = ['Tools/scripts/pydoc']
  File
"/usr/var/tmp/sat/xmpy/system/support/build/Python-2.2.2/Lib/distutils/core.py",
line 110, in setup
    dist.parse_config_files()
  File
"/usr/var/tmp/sat/xmpy/system/support/build/Python-2.2.2/Lib/distutils/dist.py",
line 310, in parse_config_files
    filenames = self.find_config_files()
  File
"/usr/var/tmp/sat/xmpy/system/support/build/Python-2.2.2/Lib/distutils/dist.py",
line 272, in find_config_files
    check_environ()
  File
"/usr/var/tmp/sat/xmpy/system/support/build/Python-2.2.2/Lib/distutils/util.py",
line 150, in check_environ
    import pwd
ImportError: No module named pwd


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=694812&group_id=5470


From noreply@sourceforge.net  Fri Feb 28 02:58:26 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 27 Feb 2003 18:58:26 -0800
Subject: [Python-bugs-list] [ python-Bugs-690012 ] 2.3a2 build fails under IRIX 6.5
Message-ID: <E18oajC-0001qz-00@sc8-sf-web4.sourceforge.net>

Bugs item #690012, was opened at 2003-02-20 07:36
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690012&group_id=5470

Category: Build
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Ralf W. Grosse-Kunstleve (rwgk)
Assigned to: Guido van Rossum (gvanrossum)
Summary: 2.3a2 build fails under IRIX 6.5

Initial Comment:
Using the latest Python 2.3a2 release
./configure; make
fails on this platform:
IRIX rattler 6.5 10120734 IP32
See attached logs.

I've also tried configure --disable-ipv6, but the result is 
the same.

The previous 2.3a1 release works without a problem 
on the exact same machine.


----------------------------------------------------------------------

>Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-27 21:58

Message:
Logged In: YES 
user_id=33168

Updated patch for review (updated patch 2).  Guido, can you
review (it's short)?
 * renaming only happens on ImportError
 * if 1 is removed in favor of checking for ImportError
 * there was no problem with the warning being printed
(level is in there), I must have been daydreaming.

I reverted the change in iconv to again raise a RuntimeError
instead of ImportError.  The make will continue with the
warning printed.  This will hopefully get us over the worst
problems with iconv.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-23 20:30

Message:
Logged In: YES 
user_id=6380

(Note, there are other compilation errors in the attached logs.)

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-23 18:52

Message:
Logged In: YES 
user_id=6380

- I think the renaming shouldn't happen at all unless the
exception is ImportError.

- The if 1 can then be replaced by if isinstance(why,
ImportError).

- The message in the else branch should be changed to
include the exception name.

- I thought I fixed the logging level of these messages, by
adding "level=3", in one of the first checkins after
releasing 2.3a2. But possibly I may have botched that. Can
you check that they really aren't printed?

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-23 16:59

Message:
Logged In: YES 
user_id=33168

Guido, I agree with your comment on python-dev that the fix
below is not best.  Attached is a patch which catches all
exceptions, and changes the warning message appropriately. 
There are 2 other issues.
 1) I don't know why the if 1: is there, should that be removed?
 2) During a normal build these warning messages aren't
printed, shouldn't they be?

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-20 18:14

Message:
Logged In: YES 
user_id=33168

In order to get the build to proceed, you can change line
210 of setup.py from:

        except ImportError, why:
to:
        except (ImportError, RuntimeError), why:

This is just before:
    self.announce('*** WARNING: renaming "%s" since
importing it' ...)



----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690012&group_id=5470


From noreply@sourceforge.net  Fri Feb 28 03:08:45 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 27 Feb 2003 19:08:45 -0800
Subject: [Python-bugs-list] [ python-Bugs-694816 ] comparing unsigned value if < 0 (in unicodedata)
Message-ID: <E18oatB-0002Bs-00@sc8-sf-web4.sourceforge.net>

Bugs item #694816, was opened at 2003-02-27 22:08
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=694816&group_id=5470

Category: Extension Modules
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Neal Norwitz (nnorwitz)
Assigned to: Martin v. Löwis (loewis)
Summary: comparing unsigned value if < 0 (in unicodedata)

Initial Comment:
Modules/unicodedata.c, Line = 36
Modules/unicodedata.c, Line = 261

It seems the code < 0 condition can be safely removed
since it should always be false.

This problem was from bug #690012.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=694816&group_id=5470


From noreply@sourceforge.net  Fri Feb 28 03:10:43 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 27 Feb 2003 19:10:43 -0800
Subject: [Python-bugs-list] [ python-Bugs-694817 ] variable set but not used in zipimport
Message-ID: <E18oav5-0001hi-00@sc8-sf-web2.sourceforge.net>

Bugs item #694817, was opened at 2003-02-27 22:10
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=694817&group_id=5470

Category: Extension Modules
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Neal Norwitz (nnorwitz)
Assigned to: Just van Rossum (jvr)
Summary: variable set but not used in zipimport

Initial Comment:

Modules/zipimport.c, Line = 648

header_end is set, but not used.  It looks like it can
be safely removed, but was something supposed to be
done with it?

Also the fseek on line 667 could use SEEK_END instead
of 2.  That will doc the intent without a comment.

This problem was from bug #690012.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=694817&group_id=5470


From noreply@sourceforge.net  Fri Feb 28 03:14:56 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 27 Feb 2003 19:14:56 -0800
Subject: [Python-bugs-list] [ python-Bugs-694816 ] comparing unsigned value if < 0 (in unicodedata)
Message-ID: <E18oazA-0001pU-00@sc8-sf-web2.sourceforge.net>

Bugs item #694816, was opened at 2003-02-27 22:08
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=694816&group_id=5470

Category: Extension Modules
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Neal Norwitz (nnorwitz)
>Assigned to: Neal Norwitz (nnorwitz)
Summary: comparing unsigned value if < 0 (in unicodedata)

Initial Comment:
Modules/unicodedata.c, Line = 36
Modules/unicodedata.c, Line = 261

It seems the code < 0 condition can be safely removed
since it should always be false.

This problem was from bug #690012.


----------------------------------------------------------------------

>Comment By: Tim Peters (tim_one)
Date: 2003-02-27 22:14

Message:
Logged In: YES 
user_id=31435

Assigned back to you:  feel free to remove the useless 
code!  I vaguely remember this, and IIRC there *used* to 
be a possibility for 'code' to be of a signed type; there isn't 
anymore.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=694816&group_id=5470


From noreply@sourceforge.net  Fri Feb 28 03:25:10 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 27 Feb 2003 19:25:10 -0800
Subject: [Python-bugs-list] [ python-Bugs-694816 ] comparing unsigned value if < 0 (in unicodedata)
Message-ID: <E18ob94-0002AB-00@sc8-sf-web3.sourceforge.net>

Bugs item #694816, was opened at 2003-02-27 22:08
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=694816&group_id=5470

Category: Extension Modules
Group: Python 2.3
>Status: Closed
>Resolution: Fixed
Priority: 5
Submitted By: Neal Norwitz (nnorwitz)
Assigned to: Neal Norwitz (nnorwitz)
Summary: comparing unsigned value if < 0 (in unicodedata)

Initial Comment:
Modules/unicodedata.c, Line = 36
Modules/unicodedata.c, Line = 261

It seems the code < 0 condition can be safely removed
since it should always be false.

This problem was from bug #690012.


----------------------------------------------------------------------

>Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-27 22:25

Message:
Logged In: YES 
user_id=33168

Checked in as: Modules/unicodedata.c 2.28.

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2003-02-27 22:14

Message:
Logged In: YES 
user_id=31435

Assigned back to you:  feel free to remove the useless 
code!  I vaguely remember this, and IIRC there *used* to 
be a possibility for 'code' to be of a signed type; there isn't 
anymore.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=694816&group_id=5470


From noreply@sourceforge.net  Fri Feb 28 03:45:28 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Thu, 27 Feb 2003 19:45:28 -0800
Subject: [Python-bugs-list] [ python-Bugs-694431 ] Python 2.3a2 Build fails on HP-UX11i
Message-ID: <E18obSi-0002q9-00@sc8-sf-web2.sourceforge.net>

Bugs item #694431, was opened at 2003-02-27 10:44
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=694431&group_id=5470

Category: Build
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Richard Townsend (rptownsend)
Assigned to: Nobody/Anonymous (nobody)
Summary: Python 2.3a2 Build fails on HP-UX11i

Initial Comment:
While building Python 2.3a2 on HP-UX11i I got the 
following error:

ld -b build/temp.hp-ux-B.11.11-9000/785-
2.3/_iconv_codec.o -L/opt/python/lib -
L/usr/local/lib -o build/lib.hp-ux-B.11.11-
9000/785-2.3/_iconv_codec.sl
/usr/lib/dld.sl: Unresolved symbol: libiconv_open 
(code)  from build/lib.hp-ux-B.11.11-9000/785-
2.3/_iconv_codec.sl
sh[3]: 25624 Abort(coredump)
*** Error exit code 134


----------------------------------------------------------------------

>Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-27 22:45

Message:
Logged In: YES 
user_id=33168

Richard, can you try the latest version from CVS.  I don't
have this problem on hp-ux-B.11.00-9000/829

I do have a problem with readline, but I think that's
specific to the machine I'm building on.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=694431&group_id=5470


From noreply@sourceforge.net  Fri Feb 28 08:55:03 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Fri, 28 Feb 2003 00:55:03 -0800
Subject: [Python-bugs-list] [ python-Bugs-694817 ] variable set but not used in zipimport
Message-ID: <E18ogIJ-0001yE-00@sc8-sf-web1.sourceforge.net>

Bugs item #694817, was opened at 2003-02-28 04:10
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=694817&group_id=5470

Category: Extension Modules
Group: Python 2.3
>Status: Closed
>Resolution: Wont Fix
Priority: 5
Submitted By: Neal Norwitz (nnorwitz)
Assigned to: Just van Rossum (jvr)
Summary: variable set but not used in zipimport

Initial Comment:

Modules/zipimport.c, Line = 648

header_end is set, but not used.  It looks like it can
be safely removed, but was something supposed to be
done with it?

Also the fseek on line 667 could use SEEK_END instead
of 2.  That will doc the intent without a comment.

This problem was from bug #690012.

----------------------------------------------------------------------

>Comment By: Just van Rossum (jvr)
Date: 2003-02-28 09:54

Message:
Logged In: YES 
user_id=92689

I've fixed the fseek() but I'll leave the header_end: I think it might be needed to fix #669036.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=694817&group_id=5470


From noreply@sourceforge.net  Fri Feb 28 11:03:41 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Fri, 28 Feb 2003 03:03:41 -0800
Subject: [Python-bugs-list] [ python-Bugs-690309 ] 2.3a2 Sol8 make fails at _iconv_codec.
Message-ID: <E18oiIn-0001NA-00@sc8-sf-web4.sourceforge.net>

Bugs item #690309, was opened at 2003-02-20 22:53
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690309&group_id=5470

Category: Build
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Inyeol Lee (inyeol)
Assigned to: Nobody/Anonymous (nobody)
Summary: 2.3a2 Sol8 make fails at _iconv_codec.

Initial Comment:
Python 2.3a2 build process fails during building
_iconv_codec
module. The message is;

[...]
gcc -shared
build/temp.solaris-2.8-sun4u-2.3/_iconv_codec.o
-L/home/inyeol/util/python/lib -L/usr/local/lib -o
build/lib.solaris-2.8-sun4u-2.3/_iconv_codec.so
Traceback (most recent call last):

  [... long traceback here...]

  File "./setup.py", line 209, in build_extension
    imp.load_dynamic(ext.name, ext_filename)
RuntimeError: can't initialize the _iconv_codec module:
iconv_open() failed
*** Error code 1
make: Fatal error: Command failed for target
`sharedmods'

My build environment is gcc2.95.3 with Solaris 8
linker.
I've built 2.2, 2,2,1, 2,2,2, 2,3a1 with the same env
and there
was no (major) issues like this. I attached my
configure and
make log messages.


----------------------------------------------------------------------

>Comment By: Walter Dörwald (doerwalter)
Date: 2003-02-28 12:03

Message:
Logged In: YES 
user_id=89016

So are there any objections to closing this bug report?

----------------------------------------------------------------------

Comment By: Alejandro Lopez-Valencia (dradul)
Date: 2003-02-24 22:13

Message:
Logged In: YES 
user_id=659006

Hmmm.... Perhaps the glibc maintainers have relaxed the naming 
rules when merging GNU iconv into the library.

I also changed ISO8859-1 to ISO-8859-1 in my local CVS copy 
and now _iconv_codec works fine under cygwin, soo I think this 
solved. 

Cheers

Al


----------------------------------------------------------------------

Comment By: Walter Dörwald (doerwalter)
Date: 2003-02-24 21:29

Message:
Logged In: YES 
user_id=89016

OK, I changed the test encoding from "ISO8859-1" to
"ISO-8859-1". But I tested this under Linux, and neither
"ASCII" nor "ISO8859-1" produced any problems for me.

----------------------------------------------------------------------

Comment By: Alejandro Lopez-Valencia (dradul)
Date: 2003-02-24 19:56

Message:
Logged In: YES 
user_id=659006

Ahh! I skimmed too fast through the code and didn't see that one! 
You see, GNU iconv is, errr, anal in the matter of ecoding 
names. And the author/maintainer is adamant on using stricly 
and only the names as specified in RFC1345, RFC1468, 
RFC1557 and related (I have a document on the subject I can 
mail you if you want). 

And... ISO8859-1 is not a valid name as per iconv_open(3) for 
GNU iconv's (any linux system has that man page). On the other 
hand, ISO-8859-1 is valid. That shoud fix the problem.


----------------------------------------------------------------------

Comment By: Walter Dörwald (doerwalter)
Date: 2003-02-24 16:10

Message:
Logged In: YES 
user_id=89016

These are just warnings, I see no error here. BTW the GNU
implementation does supports both ASCII and ISO8859-1 as
encodings, so I don't see why 1.12 should break anything

----------------------------------------------------------------------

Comment By: Alejandro Lopez-Valencia (dradul)
Date: 2003-02-23 13:25

Message:
Logged In: YES 
user_id=659006

The fix applied to _iconv_codec.c (in version 1.12) *breaks* the 
extension unnder cygwin and possibly other systems that use 
GNU iconv instead of a native library (namely, Solaris and BSD 
derivatives). The relevant errors:

building '_iconv_codec' extension
gcc -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -I. -
I/cvsroot/python/dist/src/./Include -I/usr/local/include -
I/tmp/Include -I/tmp/python -c /cvsroot/python/dist
/src/Modules/_iconv_codec.c -o build/temp.cygwin-1.3.20-i686-
2.3/_iconv_codec.o
cc1: warning: changing search order for system 
directory "/usr/local/include"
cc1: warning:   as it has already been specified as a non-system 
directory
/cvsroot/python/dist/src/Modules/_iconv_codec.c: In function 
`iconvcodec_encode'
:
/cvsroot/python/dist/src/Modules/_iconv_codec.c:147: warning: 
passing arg 2 of `
libiconv' from incompatible pointer type
/cvsroot/python/dist/src/Modules/_iconv_codec.c: In function 
`iconvcodec_decode'
:
/cvsroot/python/dist/src/Modules/_iconv_codec.c:357: warning: 
passing arg 2 of `
libiconv' from incompatible pointer type
/cvsroot/python/dist/src/Modules/_iconv_codec.c: In function 
`init_iconv_codec':

/cvsroot/python/dist/src/Modules/_iconv_codec.c:682: warning: 
passing arg 2 of `
libiconv' from incompatible pointer type
gcc -shared -Wl,--enable-auto-image-base build/temp.cygwin-
1.3.20-i686-2.3/_icon
v_codec.o -L/usr/local/lib -L. -liconv -lpython2.3 -o 
build/lib.cygwin-1.3.20-i6
86-2.3/_iconv_codec.dll
running build_scripts

----------------------------------------------------------------------

Comment By: Walter Dörwald (doerwalter)
Date: 2003-02-21 19:29

Message:
Logged In: YES 
user_id=89016

I changed the init function to use 'ISO8859-1' instead of
'ASCII' as the test encoding. Can someone test whether this
fixes anything?

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-20 23:15

Message:
Logged In: YES 
user_id=33168

In order to get the build to proceed, you can change line
210 of setup.py from:

        except ImportError, why:
to:
        except (ImportError, RuntimeError), why:

This is just before:
    self.announce('*** WARNING: renaming "%s" since
importing it' ...)

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690309&group_id=5470


From noreply@sourceforge.net  Fri Feb 28 11:10:56 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Fri, 28 Feb 2003 03:10:56 -0800
Subject: [Python-bugs-list] [ python-Bugs-690309 ] 2.3a2 Sol8 make fails at _iconv_codec.
Message-ID: <E18oiPo-0001WN-00@sc8-sf-web3.sourceforge.net>

Bugs item #690309, was opened at 2003-02-20 16:53
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690309&group_id=5470

Category: Build
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Inyeol Lee (inyeol)
Assigned to: Nobody/Anonymous (nobody)
Summary: 2.3a2 Sol8 make fails at _iconv_codec.

Initial Comment:
Python 2.3a2 build process fails during building
_iconv_codec
module. The message is;

[...]
gcc -shared
build/temp.solaris-2.8-sun4u-2.3/_iconv_codec.o
-L/home/inyeol/util/python/lib -L/usr/local/lib -o
build/lib.solaris-2.8-sun4u-2.3/_iconv_codec.so
Traceback (most recent call last):

  [... long traceback here...]

  File "./setup.py", line 209, in build_extension
    imp.load_dynamic(ext.name, ext_filename)
RuntimeError: can't initialize the _iconv_codec module:
iconv_open() failed
*** Error code 1
make: Fatal error: Command failed for target
`sharedmods'

My build environment is gcc2.95.3 with Solaris 8
linker.
I've built 2.2, 2,2,1, 2,2,2, 2,3a1 with the same env
and there
was no (major) issues like this. I attached my
configure and
make log messages.


----------------------------------------------------------------------

Comment By: Alejandro Lopez-Valencia (dradul)
Date: 2003-02-28 06:10

Message:
Logged In: YES 
user_id=659006

None whatsover.


----------------------------------------------------------------------

Comment By: Walter Dörwald (doerwalter)
Date: 2003-02-28 06:03

Message:
Logged In: YES 
user_id=89016

So are there any objections to closing this bug report?

----------------------------------------------------------------------

Comment By: Alejandro Lopez-Valencia (dradul)
Date: 2003-02-24 16:13

Message:
Logged In: YES 
user_id=659006

Hmmm.... Perhaps the glibc maintainers have relaxed the naming 
rules when merging GNU iconv into the library.

I also changed ISO8859-1 to ISO-8859-1 in my local CVS copy 
and now _iconv_codec works fine under cygwin, soo I think this 
solved. 

Cheers

Al


----------------------------------------------------------------------

Comment By: Walter Dörwald (doerwalter)
Date: 2003-02-24 15:29

Message:
Logged In: YES 
user_id=89016

OK, I changed the test encoding from "ISO8859-1" to
"ISO-8859-1". But I tested this under Linux, and neither
"ASCII" nor "ISO8859-1" produced any problems for me.

----------------------------------------------------------------------

Comment By: Alejandro Lopez-Valencia (dradul)
Date: 2003-02-24 13:56

Message:
Logged In: YES 
user_id=659006

Ahh! I skimmed too fast through the code and didn't see that one! 
You see, GNU iconv is, errr, anal in the matter of ecoding 
names. And the author/maintainer is adamant on using stricly 
and only the names as specified in RFC1345, RFC1468, 
RFC1557 and related (I have a document on the subject I can 
mail you if you want). 

And... ISO8859-1 is not a valid name as per iconv_open(3) for 
GNU iconv's (any linux system has that man page). On the other 
hand, ISO-8859-1 is valid. That shoud fix the problem.


----------------------------------------------------------------------

Comment By: Walter Dörwald (doerwalter)
Date: 2003-02-24 10:10

Message:
Logged In: YES 
user_id=89016

These are just warnings, I see no error here. BTW the GNU
implementation does supports both ASCII and ISO8859-1 as
encodings, so I don't see why 1.12 should break anything

----------------------------------------------------------------------

Comment By: Alejandro Lopez-Valencia (dradul)
Date: 2003-02-23 07:25

Message:
Logged In: YES 
user_id=659006

The fix applied to _iconv_codec.c (in version 1.12) *breaks* the 
extension unnder cygwin and possibly other systems that use 
GNU iconv instead of a native library (namely, Solaris and BSD 
derivatives). The relevant errors:

building '_iconv_codec' extension
gcc -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -I. -
I/cvsroot/python/dist/src/./Include -I/usr/local/include -
I/tmp/Include -I/tmp/python -c /cvsroot/python/dist
/src/Modules/_iconv_codec.c -o build/temp.cygwin-1.3.20-i686-
2.3/_iconv_codec.o
cc1: warning: changing search order for system 
directory "/usr/local/include"
cc1: warning:   as it has already been specified as a non-system 
directory
/cvsroot/python/dist/src/Modules/_iconv_codec.c: In function 
`iconvcodec_encode'
:
/cvsroot/python/dist/src/Modules/_iconv_codec.c:147: warning: 
passing arg 2 of `
libiconv' from incompatible pointer type
/cvsroot/python/dist/src/Modules/_iconv_codec.c: In function 
`iconvcodec_decode'
:
/cvsroot/python/dist/src/Modules/_iconv_codec.c:357: warning: 
passing arg 2 of `
libiconv' from incompatible pointer type
/cvsroot/python/dist/src/Modules/_iconv_codec.c: In function 
`init_iconv_codec':

/cvsroot/python/dist/src/Modules/_iconv_codec.c:682: warning: 
passing arg 2 of `
libiconv' from incompatible pointer type
gcc -shared -Wl,--enable-auto-image-base build/temp.cygwin-
1.3.20-i686-2.3/_icon
v_codec.o -L/usr/local/lib -L. -liconv -lpython2.3 -o 
build/lib.cygwin-1.3.20-i6
86-2.3/_iconv_codec.dll
running build_scripts

----------------------------------------------------------------------

Comment By: Walter Dörwald (doerwalter)
Date: 2003-02-21 13:29

Message:
Logged In: YES 
user_id=89016

I changed the init function to use 'ISO8859-1' instead of
'ASCII' as the test encoding. Can someone test whether this
fixes anything?

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-20 17:15

Message:
Logged In: YES 
user_id=33168

In order to get the build to proceed, you can change line
210 of setup.py from:

        except ImportError, why:
to:
        except (ImportError, RuntimeError), why:

This is just before:
    self.announce('*** WARNING: renaming "%s" since
importing it' ...)

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690309&group_id=5470


From noreply@sourceforge.net  Fri Feb 28 11:18:17 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Fri, 28 Feb 2003 03:18:17 -0800
Subject: [Python-bugs-list] [ python-Bugs-690309 ] 2.3a2 Sol8 make fails at _iconv_codec.
Message-ID: <E18oiWv-0001zi-00@sc8-sf-web4.sourceforge.net>

Bugs item #690309, was opened at 2003-02-20 22:53
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690309&group_id=5470

Category: Build
Group: Python 2.3
>Status: Closed
>Resolution: Fixed
Priority: 5
Submitted By: Inyeol Lee (inyeol)
Assigned to: Nobody/Anonymous (nobody)
Summary: 2.3a2 Sol8 make fails at _iconv_codec.

Initial Comment:
Python 2.3a2 build process fails during building
_iconv_codec
module. The message is;

[...]
gcc -shared
build/temp.solaris-2.8-sun4u-2.3/_iconv_codec.o
-L/home/inyeol/util/python/lib -L/usr/local/lib -o
build/lib.solaris-2.8-sun4u-2.3/_iconv_codec.so
Traceback (most recent call last):

  [... long traceback here...]

  File "./setup.py", line 209, in build_extension
    imp.load_dynamic(ext.name, ext_filename)
RuntimeError: can't initialize the _iconv_codec module:
iconv_open() failed
*** Error code 1
make: Fatal error: Command failed for target
`sharedmods'

My build environment is gcc2.95.3 with Solaris 8
linker.
I've built 2.2, 2,2,1, 2,2,2, 2,3a1 with the same env
and there
was no (major) issues like this. I attached my
configure and
make log messages.


----------------------------------------------------------------------

Comment By: Alejandro Lopez-Valencia (dradul)
Date: 2003-02-28 12:10

Message:
Logged In: YES 
user_id=659006

None whatsover.


----------------------------------------------------------------------

Comment By: Walter Dörwald (doerwalter)
Date: 2003-02-28 12:03

Message:
Logged In: YES 
user_id=89016

So are there any objections to closing this bug report?

----------------------------------------------------------------------

Comment By: Alejandro Lopez-Valencia (dradul)
Date: 2003-02-24 22:13

Message:
Logged In: YES 
user_id=659006

Hmmm.... Perhaps the glibc maintainers have relaxed the naming 
rules when merging GNU iconv into the library.

I also changed ISO8859-1 to ISO-8859-1 in my local CVS copy 
and now _iconv_codec works fine under cygwin, soo I think this 
solved. 

Cheers

Al


----------------------------------------------------------------------

Comment By: Walter Dörwald (doerwalter)
Date: 2003-02-24 21:29

Message:
Logged In: YES 
user_id=89016

OK, I changed the test encoding from "ISO8859-1" to
"ISO-8859-1". But I tested this under Linux, and neither
"ASCII" nor "ISO8859-1" produced any problems for me.

----------------------------------------------------------------------

Comment By: Alejandro Lopez-Valencia (dradul)
Date: 2003-02-24 19:56

Message:
Logged In: YES 
user_id=659006

Ahh! I skimmed too fast through the code and didn't see that one! 
You see, GNU iconv is, errr, anal in the matter of ecoding 
names. And the author/maintainer is adamant on using stricly 
and only the names as specified in RFC1345, RFC1468, 
RFC1557 and related (I have a document on the subject I can 
mail you if you want). 

And... ISO8859-1 is not a valid name as per iconv_open(3) for 
GNU iconv's (any linux system has that man page). On the other 
hand, ISO-8859-1 is valid. That shoud fix the problem.


----------------------------------------------------------------------

Comment By: Walter Dörwald (doerwalter)
Date: 2003-02-24 16:10

Message:
Logged In: YES 
user_id=89016

These are just warnings, I see no error here. BTW the GNU
implementation does supports both ASCII and ISO8859-1 as
encodings, so I don't see why 1.12 should break anything

----------------------------------------------------------------------

Comment By: Alejandro Lopez-Valencia (dradul)
Date: 2003-02-23 13:25

Message:
Logged In: YES 
user_id=659006

The fix applied to _iconv_codec.c (in version 1.12) *breaks* the 
extension unnder cygwin and possibly other systems that use 
GNU iconv instead of a native library (namely, Solaris and BSD 
derivatives). The relevant errors:

building '_iconv_codec' extension
gcc -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -I. -
I/cvsroot/python/dist/src/./Include -I/usr/local/include -
I/tmp/Include -I/tmp/python -c /cvsroot/python/dist
/src/Modules/_iconv_codec.c -o build/temp.cygwin-1.3.20-i686-
2.3/_iconv_codec.o
cc1: warning: changing search order for system 
directory "/usr/local/include"
cc1: warning:   as it has already been specified as a non-system 
directory
/cvsroot/python/dist/src/Modules/_iconv_codec.c: In function 
`iconvcodec_encode'
:
/cvsroot/python/dist/src/Modules/_iconv_codec.c:147: warning: 
passing arg 2 of `
libiconv' from incompatible pointer type
/cvsroot/python/dist/src/Modules/_iconv_codec.c: In function 
`iconvcodec_decode'
:
/cvsroot/python/dist/src/Modules/_iconv_codec.c:357: warning: 
passing arg 2 of `
libiconv' from incompatible pointer type
/cvsroot/python/dist/src/Modules/_iconv_codec.c: In function 
`init_iconv_codec':

/cvsroot/python/dist/src/Modules/_iconv_codec.c:682: warning: 
passing arg 2 of `
libiconv' from incompatible pointer type
gcc -shared -Wl,--enable-auto-image-base build/temp.cygwin-
1.3.20-i686-2.3/_icon
v_codec.o -L/usr/local/lib -L. -liconv -lpython2.3 -o 
build/lib.cygwin-1.3.20-i6
86-2.3/_iconv_codec.dll
running build_scripts

----------------------------------------------------------------------

Comment By: Walter Dörwald (doerwalter)
Date: 2003-02-21 19:29

Message:
Logged In: YES 
user_id=89016

I changed the init function to use 'ISO8859-1' instead of
'ASCII' as the test encoding. Can someone test whether this
fixes anything?

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-20 23:15

Message:
Logged In: YES 
user_id=33168

In order to get the build to proceed, you can change line
210 of setup.py from:

        except ImportError, why:
to:
        except (ImportError, RuntimeError), why:

This is just before:
    self.announce('*** WARNING: renaming "%s" since
importing it' ...)

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690309&group_id=5470


From noreply@sourceforge.net  Fri Feb 28 14:06:58 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Fri, 28 Feb 2003 06:06:58 -0800
Subject: [Python-bugs-list] [ python-Bugs-690012 ] 2.3a2 build fails under IRIX 6.5
Message-ID: <E18olAA-0006Vg-00@sc8-sf-web1.sourceforge.net>

Bugs item #690012, was opened at 2003-02-20 07:36
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690012&group_id=5470

Category: Build
Group: Python 2.3
Status: Open
>Resolution: Accepted
Priority: 5
Submitted By: Ralf W. Grosse-Kunstleve (rwgk)
>Assigned to: Neal Norwitz (nnorwitz)
Summary: 2.3a2 build fails under IRIX 6.5

Initial Comment:
Using the latest Python 2.3a2 release
./configure; make
fails on this platform:
IRIX rattler 6.5 10120734 IP32
See attached logs.

I've also tried configure --disable-ipv6, but the result is 
the same.

The previous 2.3a1 release works without a problem 
on the exact same machine.


----------------------------------------------------------------------

>Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-28 09:06

Message:
Logged In: YES 
user_id=6380

Yes, looks good.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-27 21:58

Message:
Logged In: YES 
user_id=33168

Updated patch for review (updated patch 2).  Guido, can you
review (it's short)?
 * renaming only happens on ImportError
 * if 1 is removed in favor of checking for ImportError
 * there was no problem with the warning being printed
(level is in there), I must have been daydreaming.

I reverted the change in iconv to again raise a RuntimeError
instead of ImportError.  The make will continue with the
warning printed.  This will hopefully get us over the worst
problems with iconv.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-23 20:30

Message:
Logged In: YES 
user_id=6380

(Note, there are other compilation errors in the attached logs.)

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-23 18:52

Message:
Logged In: YES 
user_id=6380

- I think the renaming shouldn't happen at all unless the
exception is ImportError.

- The if 1 can then be replaced by if isinstance(why,
ImportError).

- The message in the else branch should be changed to
include the exception name.

- I thought I fixed the logging level of these messages, by
adding "level=3", in one of the first checkins after
releasing 2.3a2. But possibly I may have botched that. Can
you check that they really aren't printed?

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-23 16:59

Message:
Logged In: YES 
user_id=33168

Guido, I agree with your comment on python-dev that the fix
below is not best.  Attached is a patch which catches all
exceptions, and changes the warning message appropriately. 
There are 2 other issues.
 1) I don't know why the if 1: is there, should that be removed?
 2) During a normal build these warning messages aren't
printed, shouldn't they be?

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-20 18:14

Message:
Logged In: YES 
user_id=33168

In order to get the build to proceed, you can change line
210 of setup.py from:

        except ImportError, why:
to:
        except (ImportError, RuntimeError), why:

This is just before:
    self.announce('*** WARNING: renaming "%s" since
importing it' ...)



----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690012&group_id=5470


From noreply@sourceforge.net  Fri Feb 28 15:30:41 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Fri, 28 Feb 2003 07:30:41 -0800
Subject: [Python-bugs-list] [ python-Bugs-693230 ] PackageManager needs to check write permission
Message-ID: <E18omTB-0004mC-00@sc8-sf-web3.sourceforge.net>

Bugs item #693230, was opened at 2003-02-25 23:08
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=693230&group_id=5470

Category: Macintosh
Group: None
>Status: Closed
>Resolution: Fixed
Priority: 5
Submitted By: Jack Jansen (jackjansen)
Assigned to: Jack Jansen (jackjansen)
Summary: PackageManager needs to check write permission

Initial Comment:
PackageManager needs to check that the user has write permission to site-packages (or whatever the install directory is set to, once it is settable).

----------------------------------------------------------------------

>Comment By: Jack Jansen (jackjansen)
Date: 2003-02-28 16:30

Message:
Logged In: YES 
user_id=45365

Fixed in PackageManager rev 1.4

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=693230&group_id=5470


From noreply@sourceforge.net  Fri Feb 28 15:42:34 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Fri, 28 Feb 2003 07:42:34 -0800
Subject: [Python-bugs-list] [ python-Bugs-693293 ] Make EasyDialogs dialogs movable modal
Message-ID: <E18omeg-0005eu-00@sc8-sf-web2.sourceforge.net>

Bugs item #693293, was opened at 2003-02-26 00:47
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=693293&group_id=5470

Category: Macintosh
Group: None
>Status: Closed
>Resolution: Fixed
Priority: 5
Submitted By: Jack Jansen (jackjansen)
Assigned to: Jack Jansen (jackjansen)
Summary: Make EasyDialogs dialogs movable modal

Initial Comment:
The EasyDialogs dialogs should be movable modal. On MacOSX this should at least take care of screen refresh too.

----------------------------------------------------------------------

>Comment By: Jack Jansen (jackjansen)
Date: 2003-02-28 16:42

Message:
Logged In: YES 
user_id=45365

Fixed in CVS.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=693293&group_id=5470


From noreply@sourceforge.net  Fri Feb 28 16:05:50 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Fri, 28 Feb 2003 08:05:50 -0800
Subject: [Python-bugs-list] [ python-Bugs-684975 ] IDE asks for attention when quitting
Message-ID: <E18on1C-0006lj-00@sc8-sf-web4.sourceforge.net>

Bugs item #684975, was opened at 2003-02-12 00:03
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=684975&group_id=5470

Category: Macintosh
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Jack Jansen (jackjansen)
Assigned to: Just van Rossum (jvr)
Summary: IDE asks for attention when quitting

Initial Comment:
When you quit the IDE (in MacPython-OSX) via the popup menu in the dock it somehow asks for attention (the icon jumps up and down enthusiastically). When you the bring it to front it exits normally, without showing any message or anything.

I've looked at this a couple of times but never managed to find what is going on. Do you have any idea?

----------------------------------------------------------------------

>Comment By: Jack Jansen (jackjansen)
Date: 2003-02-28 17:05

Message:
Logged In: YES 
user_id=45365

Found the problem: the unconditional call to AEInteractWithUser() in the quit appleevent handler. I think this should be done differently. I've noticed that on OSX it's always safe to call AEInteractWithUser, even when not inside an AE handler. So, the simplest solution would be to call it in all the EasyDialogs calls and be done with it. Do you think this is reasonable? If so assign back to me and I'll do it.

The alternative is to do this for IDE only. That means that the various classes should call app.interact() before posting a dialog. app.interact() would then call AEInteractWithUser.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=684975&group_id=5470


From noreply@sourceforge.net  Fri Feb 28 16:31:45 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Fri, 28 Feb 2003 08:31:45 -0800
Subject: [Python-bugs-list] [ python-Bugs-684975 ] IDE asks for attention when quitting
Message-ID: <E18onQH-00081i-00@sc8-sf-web3.sourceforge.net>

Bugs item #684975, was opened at 2003-02-12 00:03
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=684975&group_id=5470

Category: Macintosh
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Jack Jansen (jackjansen)
>Assigned to: Jack Jansen (jackjansen)
Summary: IDE asks for attention when quitting

Initial Comment:
When you quit the IDE (in MacPython-OSX) via the popup menu in the dock it somehow asks for attention (the icon jumps up and down enthusiastically). When you the bring it to front it exits normally, without showing any message or anything.

I've looked at this a couple of times but never managed to find what is going on. Do you have any idea?

----------------------------------------------------------------------

>Comment By: Just van Rossum (jvr)
Date: 2003-02-28 17:31

Message:
Logged In: YES 
user_id=92689

That sounds entirely reasonable...

----------------------------------------------------------------------

Comment By: Jack Jansen (jackjansen)
Date: 2003-02-28 17:05

Message:
Logged In: YES 
user_id=45365

Found the problem: the unconditional call to AEInteractWithUser() in the quit appleevent handler. I think this should be done differently. I've noticed that on OSX it's always safe to call AEInteractWithUser, even when not inside an AE handler. So, the simplest solution would be to call it in all the EasyDialogs calls and be done with it. Do you think this is reasonable? If so assign back to me and I'll do it.

The alternative is to do this for IDE only. That means that the various classes should call app.interact() before posting a dialog. app.interact() would then call AEInteractWithUser.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=684975&group_id=5470


From noreply@sourceforge.net  Fri Feb 28 17:34:02 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Fri, 28 Feb 2003 09:34:02 -0800
Subject: [Python-bugs-list] [ python-Bugs-690012 ] 2.3a2 build fails under IRIX 6.5
Message-ID: <E18ooOY-00031N-00@sc8-sf-web4.sourceforge.net>

Bugs item #690012, was opened at 2003-02-20 07:36
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690012&group_id=5470

Category: Build
Group: Python 2.3
>Status: Closed
Resolution: Accepted
Priority: 5
Submitted By: Ralf W. Grosse-Kunstleve (rwgk)
Assigned to: Neal Norwitz (nnorwitz)
Summary: 2.3a2 build fails under IRIX 6.5

Initial Comment:
Using the latest Python 2.3a2 release
./configure; make
fails on this platform:
IRIX rattler 6.5 10120734 IP32
See attached logs.

I've also tried configure --disable-ipv6, but the result is 
the same.

The previous 2.3a1 release works without a problem 
on the exact same machine.


----------------------------------------------------------------------

>Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-28 12:34

Message:
Logged In: YES 
user_id=33168


Checked in as:

setup.py 1.152
Modules/_iconv_codec.c 1.14Misc/NEWS 1.681

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-28 09:06

Message:
Logged In: YES 
user_id=6380

Yes, looks good.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-27 21:58

Message:
Logged In: YES 
user_id=33168

Updated patch for review (updated patch 2).  Guido, can you
review (it's short)?
 * renaming only happens on ImportError
 * if 1 is removed in favor of checking for ImportError
 * there was no problem with the warning being printed
(level is in there), I must have been daydreaming.

I reverted the change in iconv to again raise a RuntimeError
instead of ImportError.  The make will continue with the
warning printed.  This will hopefully get us over the worst
problems with iconv.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-23 20:30

Message:
Logged In: YES 
user_id=6380

(Note, there are other compilation errors in the attached logs.)

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-23 18:52

Message:
Logged In: YES 
user_id=6380

- I think the renaming shouldn't happen at all unless the
exception is ImportError.

- The if 1 can then be replaced by if isinstance(why,
ImportError).

- The message in the else branch should be changed to
include the exception name.

- I thought I fixed the logging level of these messages, by
adding "level=3", in one of the first checkins after
releasing 2.3a2. But possibly I may have botched that. Can
you check that they really aren't printed?

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-23 16:59

Message:
Logged In: YES 
user_id=33168

Guido, I agree with your comment on python-dev that the fix
below is not best.  Attached is a patch which catches all
exceptions, and changes the warning message appropriately. 
There are 2 other issues.
 1) I don't know why the if 1: is there, should that be removed?
 2) During a normal build these warning messages aren't
printed, shouldn't they be?

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-20 18:14

Message:
Logged In: YES 
user_id=33168

In order to get the build to proceed, you can change line
210 of setup.py from:

        except ImportError, why:
to:
        except (ImportError, RuntimeError), why:

This is just before:
    self.announce('*** WARNING: renaming "%s" since
importing it' ...)



----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690012&group_id=5470


From noreply@sourceforge.net  Fri Feb 28 17:35:41 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Fri, 28 Feb 2003 09:35:41 -0800
Subject: [Python-bugs-list] [ python-Bugs-690012 ] 2.3a2 build fails under IRIX 6.5
Message-ID: <E18ooQ9-0001GQ-00@sc8-sf-web3.sourceforge.net>

Bugs item #690012, was opened at 2003-02-20 07:36
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690012&group_id=5470

Category: Build
Group: Python 2.3
Status: Closed
Resolution: Accepted
Priority: 5
Submitted By: Ralf W. Grosse-Kunstleve (rwgk)
Assigned to: Neal Norwitz (nnorwitz)
Summary: 2.3a2 build fails under IRIX 6.5

Initial Comment:
Using the latest Python 2.3a2 release
./configure; make
fails on this platform:
IRIX rattler 6.5 10120734 IP32
See attached logs.

I've also tried configure --disable-ipv6, but the result is 
the same.

The previous 2.3a1 release works without a problem 
on the exact same machine.


----------------------------------------------------------------------

>Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-28 12:35

Message:
Logged In: YES 
user_id=33168

Ralf, could you test from CVS to make sure everything is ok.
 The fix was general, but I'd like to make sure beta1 is in
better shape.  Thanks.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-28 12:34

Message:
Logged In: YES 
user_id=33168


Checked in as:

setup.py 1.152
Modules/_iconv_codec.c 1.14Misc/NEWS 1.681

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-28 09:06

Message:
Logged In: YES 
user_id=6380

Yes, looks good.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-27 21:58

Message:
Logged In: YES 
user_id=33168

Updated patch for review (updated patch 2).  Guido, can you
review (it's short)?
 * renaming only happens on ImportError
 * if 1 is removed in favor of checking for ImportError
 * there was no problem with the warning being printed
(level is in there), I must have been daydreaming.

I reverted the change in iconv to again raise a RuntimeError
instead of ImportError.  The make will continue with the
warning printed.  This will hopefully get us over the worst
problems with iconv.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-23 20:30

Message:
Logged In: YES 
user_id=6380

(Note, there are other compilation errors in the attached logs.)

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-23 18:52

Message:
Logged In: YES 
user_id=6380

- I think the renaming shouldn't happen at all unless the
exception is ImportError.

- The if 1 can then be replaced by if isinstance(why,
ImportError).

- The message in the else branch should be changed to
include the exception name.

- I thought I fixed the logging level of these messages, by
adding "level=3", in one of the first checkins after
releasing 2.3a2. But possibly I may have botched that. Can
you check that they really aren't printed?

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-23 16:59

Message:
Logged In: YES 
user_id=33168

Guido, I agree with your comment on python-dev that the fix
below is not best.  Attached is a patch which catches all
exceptions, and changes the warning message appropriately. 
There are 2 other issues.
 1) I don't know why the if 1: is there, should that be removed?
 2) During a normal build these warning messages aren't
printed, shouldn't they be?

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-20 18:14

Message:
Logged In: YES 
user_id=33168

In order to get the build to proceed, you can change line
210 of setup.py from:

        except ImportError, why:
to:
        except (ImportError, RuntimeError), why:

This is just before:
    self.announce('*** WARNING: renaming "%s" since
importing it' ...)



----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=690012&group_id=5470


From noreply@sourceforge.net  Fri Feb 28 17:37:13 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Fri, 28 Feb 2003 09:37:13 -0800
Subject: [Python-bugs-list] [ python-Bugs-694431 ] Python 2.3a2 Build fails on HP-UX11i
Message-ID: <E18ooRd-00010W-00@sc8-sf-web1.sourceforge.net>

Bugs item #694431, was opened at 2003-02-27 10:44
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=694431&group_id=5470

Category: Build
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Richard Townsend (rptownsend)
>Assigned to: Neal Norwitz (nnorwitz)
Summary: Python 2.3a2 Build fails on HP-UX11i

Initial Comment:
While building Python 2.3a2 on HP-UX11i I got the 
following error:

ld -b build/temp.hp-ux-B.11.11-9000/785-
2.3/_iconv_codec.o -L/opt/python/lib -
L/usr/local/lib -o build/lib.hp-ux-B.11.11-
9000/785-2.3/_iconv_codec.sl
/usr/lib/dld.sl: Unresolved symbol: libiconv_open 
(code)  from build/lib.hp-ux-B.11.11-9000/785-
2.3/_iconv_codec.sl
sh[3]: 25624 Abort(coredump)
*** Error exit code 134


----------------------------------------------------------------------

>Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-28 12:37

Message:
Logged In: YES 
user_id=33168

This iconv_codec problem may be fixed.  Please test with
current CVS to verify.
See SF bug #690012.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-27 22:45

Message:
Logged In: YES 
user_id=33168

Richard, can you try the latest version from CVS.  I don't
have this problem on hp-ux-B.11.00-9000/829

I do have a problem with readline, but I think that's
specific to the machine I'm building on.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=694431&group_id=5470


From noreply@sourceforge.net  Fri Feb 28 17:38:28 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Fri, 28 Feb 2003 09:38:28 -0800
Subject: [Python-bugs-list] [ python-Bugs-693094 ] Python does not build --with-pydebug on Tru64 with vendor cc
Message-ID: <E18ooSq-0003HD-00@sc8-sf-web4.sourceforge.net>

Bugs item #693094, was opened at 2003-02-25 13:48
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=693094&group_id=5470

Category: Build
Group: Platform-specific
Status: Open
Resolution: None
Priority: 5
Submitted By: Stephan A. Terre (sfiedler)
>Assigned to: Neal Norwitz (nnorwitz)
Summary: Python does not build --with-pydebug on Tru64 with vendor cc

Initial Comment:
The default standards-compliance level for the
Compaq/HP compiler defines the assert macro using the
if statement (if...then) rather than the ternary if
expression (?:). As a result, when Python is configured
--with-pydebug (so NDEBUG is not defined), the macro
Py_SAFE_DOWNCAST expands to illegal C (containing a
statement inside an expression).

This can be fixed by asserting the -std switch to the
vendor compiler. Testing of 2.2.2 CVS and 2.3 CVS
indicates that there is no bad side effect to asserting
-std. 

I hope to develop a fix for this in configure.in
sometime in the next couple of days. When I have done
so, I will attach it to this bug.

For the record, here are the variations of the -std
switch on the vendor compiler:

  -std0
      Selects the K & R language mode. This is the
default.  Enforces the K &
      R programming style, with certain ANSI extensions
in areas where the K
      & R behavior is undefined or ambiguous. In
general, -std0 compiles most
      pre-ANSI C programs and produces expected
results.  The -std0 option
      causes the __STDC__ macro to be undefined.


  -std
      Selects the relaxed ANSI language mode. Enforces
the ANSI C standard,
      but allows some common programming practices
disallowed by the stan-
      dard.

      This option does not restrict the Tru64 UNIX name
space (for example,
      the names defined in system header files).  To
restrict that name space
      so that only ANSI C reserved names are visible
from the ANSI C header
      files, use the _ANSI_C_SOURCE macro. See
standards(5) for more details.

      This option causes the macro __STDC_ to be
defined to 0.

  -std1
      Selects the strict ANSI language mode. Strictly
enforces the ANSI C
      standard and all its prohibitions (such as those
that apply to the han-
      dling of void types, the definition of lvalues in
expressions, the mix-
      ing of integrals and pointers, and the
modification of rvalues).

      This option does not restrict the Tru64 UNIX name
space (for example,
      the names defined in system header files).  To
restrict that name space
      so that only ANSI C reserved names are visible
from the ANSI C header
      files, use the _ANSI_C_SOURCE macro. See
standards(5) for more details.

      This option causes the macro __STDC_ to be
defined to 1. Note that this
      option also affects linker-defined symbols.  See
ld(1) for more infor-
      mation.

      This option turns on ANSI aliasing rules
(-ansi_alias option).



----------------------------------------------------------------------

Comment By: Stephan A. Terre (sfiedler)
Date: 2003-02-27 19:58

Message:
Logged In: YES 
user_id=246063

Attached to this bug is a patch to configure.in and configure. I
believe pretty much all of the three decisions I made need
review :)

1. I put the -std flag in BASECFLAGS an OSF1-specific case
in the
non-gcc branch. Should there be some other check here that
we're using
the vendor C compiler? If so, can anyone point me at an example?

2. Upon testing, I found that the getpgrp function has a
different
signature when compiled with -std than without. As a result, the
posixmodule didn't build. My solution was to change the
definition of
CC in configure.in to be "$CC $BASECFLAGS $OPT" after the
last change
to OPT and BASECFLAGS. My thinking is that we ought to be
running
feature and behavior tests with the flags that we'll be
using to build
the interpreter. However, if this goes against the spirit or
policy of
autoconf, we should do something else!

3. I happened to notice an XXX about Monterey, so I did a little
investigation and added to the note. Hopefully this is not a
gaffe.

I have tested the changes to configure.in on Tru64 4.0F both
with and
without the --with-pydebug flag to configure. The test suite
behaved
the same in each case as before my mods.

I also tested the changes --with-pydebug on a Red Hat 7
system; they
seemed good there too.

If I should have put this patch in the Patch Manager rather than
attaching to the bug directly, I apologize and promise never
to do it
wrongly again :)

Please let me know by posting a followup here or via email
if there is
anything else I should do for this bug. 



----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=693094&group_id=5470


From noreply@sourceforge.net  Fri Feb 28 17:53:09 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Fri, 28 Feb 2003 09:53:09 -0800
Subject: [Python-bugs-list] [ python-Bugs-692988 ] test_timeout.py needs 'network' resource
Message-ID: <E18ooh3-00023i-00@sc8-sf-web3.sourceforge.net>

Bugs item #692988, was opened at 2003-02-25 11:01
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=692988&group_id=5470

Category: None
>Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Cesar Eduardo Barros (cesarb)
>Assigned to: Guido van Rossum (gvanrossum)
Summary: test_timeout.py needs 'network' resource

Initial Comment:
test_timeout.py fails in two tests if the network is
down. This is because it tries to connect to
www.google.com:80. It should be skipped unless the
'network' resource is enabled.

Also, the failure mode of the testConnectTimeout is
weird; it took 10 seconds for it to timeout. Maybe
that's a real bug somewhere.

The version I used is test_timeout.py revision 1.11

----------------------------------------------------------------------

>Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-28 12:53

Message:
Logged In: YES 
user_id=33168

Guido, any objections to making test_timeout require -u
network?  There's another bug with the same problem.

----------------------------------------------------------------------

Comment By: Cesar Eduardo Barros (cesarb)
Date: 2003-02-26 09:40

Message:
Logged In: YES 
user_id=57799

Ah, and maybe the documentation should be updated to make
explicit that the connect() timeout does not include the DNS
lookup (and should not, since it's the _socket_ timeout...).
If you want a timeout in the DNS lookup, you must do the
lookup yourself (but I don't think it's really possible to
change the resolver timeout...)

----------------------------------------------------------------------

Comment By: Cesar Eduardo Barros (cesarb)
Date: 2003-02-26 09:37

Message:
Logged In: YES 
user_id=57799

testConnectTimeout is returning an error because it assumes
every socket.error is a timeout, when it could be, for
instance, a gaierror (which seems to be the case).

Also, the testcase is timing the whole socket.connect()
call, which includes both the DNS lookup and the connect()
proper. To avoid this, it should first do a lookup of
www.google.com by other means and pass the IP, not the
hostname, to the socket.connect() call. Else, it would be
also measuring the DNS resolution time (which can be kinda
long if the nameserver isn't local).

So, we have three problems in the testcase already:
1) It should only run the two remote tests when use of the
'network' resource is enabled
2) It should not assume that any socket.error is a timeout;
it must do more checks to make sure it's really a timeout
3) It should avoid measuring the time spent in
getaddrinfo(3) by resolving the hostname to an IP before
starting the tests (this would also avoid the false failures
because of DNS failure and make them look like a testcase
setup error).

Should I make a separate bug for each of these?

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-25 22:06

Message:
Logged In: YES 
user_id=33168

Whoops, sorry my last comment was meant for bug #692951. 
This report helps clarify the other bug.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-25 22:03

Message:
Logged In: YES 
user_id=33168

The first problem seems to be a DNS lookup failure
(www.google.com).  What Linux distro and version are you
using?  Did you enable ipv6?  Anything else that could help
debug this problem?  Many developers run on Linux and I
haven't heard of this problem?  Is your network very slow? 
Do you have a firewall which introduces large latencies?

----------------------------------------------------------------------

Comment By: Cesar Eduardo Barros (cesarb)
Date: 2003-02-25 11:08

Message:
Logged In: YES 
user_id=57799

Weird, the attachment didn't work. Trying again to attach
the file.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=692988&group_id=5470


From noreply@sourceforge.net  Fri Feb 28 18:00:08 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Fri, 28 Feb 2003 10:00:08 -0800
Subject: [Python-bugs-list] [ python-Bugs-692988 ] test_timeout.py needs 'network' resource
Message-ID: <E18oono-0004fQ-00@sc8-sf-web2.sourceforge.net>

Bugs item #692988, was opened at 2003-02-25 11:01
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=692988&group_id=5470

Category: None
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Cesar Eduardo Barros (cesarb)
>Assigned to: Neal Norwitz (nnorwitz)
Summary: test_timeout.py needs 'network' resource

Initial Comment:
test_timeout.py fails in two tests if the network is
down. This is because it tries to connect to
www.google.com:80. It should be skipped unless the
'network' resource is enabled.

Also, the failure mode of the testConnectTimeout is
weird; it took 10 seconds for it to timeout. Maybe
that's a real bug somewhere.

The version I used is test_timeout.py revision 1.11

----------------------------------------------------------------------

>Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-28 13:00

Message:
Logged In: YES 
user_id=6380

sure, good idea.


----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-28 12:53

Message:
Logged In: YES 
user_id=33168

Guido, any objections to making test_timeout require -u
network?  There's another bug with the same problem.

----------------------------------------------------------------------

Comment By: Cesar Eduardo Barros (cesarb)
Date: 2003-02-26 09:40

Message:
Logged In: YES 
user_id=57799

Ah, and maybe the documentation should be updated to make
explicit that the connect() timeout does not include the DNS
lookup (and should not, since it's the _socket_ timeout...).
If you want a timeout in the DNS lookup, you must do the
lookup yourself (but I don't think it's really possible to
change the resolver timeout...)

----------------------------------------------------------------------

Comment By: Cesar Eduardo Barros (cesarb)
Date: 2003-02-26 09:37

Message:
Logged In: YES 
user_id=57799

testConnectTimeout is returning an error because it assumes
every socket.error is a timeout, when it could be, for
instance, a gaierror (which seems to be the case).

Also, the testcase is timing the whole socket.connect()
call, which includes both the DNS lookup and the connect()
proper. To avoid this, it should first do a lookup of
www.google.com by other means and pass the IP, not the
hostname, to the socket.connect() call. Else, it would be
also measuring the DNS resolution time (which can be kinda
long if the nameserver isn't local).

So, we have three problems in the testcase already:
1) It should only run the two remote tests when use of the
'network' resource is enabled
2) It should not assume that any socket.error is a timeout;
it must do more checks to make sure it's really a timeout
3) It should avoid measuring the time spent in
getaddrinfo(3) by resolving the hostname to an IP before
starting the tests (this would also avoid the false failures
because of DNS failure and make them look like a testcase
setup error).

Should I make a separate bug for each of these?

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-25 22:06

Message:
Logged In: YES 
user_id=33168

Whoops, sorry my last comment was meant for bug #692951. 
This report helps clarify the other bug.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-25 22:03

Message:
Logged In: YES 
user_id=33168

The first problem seems to be a DNS lookup failure
(www.google.com).  What Linux distro and version are you
using?  Did you enable ipv6?  Anything else that could help
debug this problem?  Many developers run on Linux and I
haven't heard of this problem?  Is your network very slow? 
Do you have a firewall which introduces large latencies?

----------------------------------------------------------------------

Comment By: Cesar Eduardo Barros (cesarb)
Date: 2003-02-25 11:08

Message:
Logged In: YES 
user_id=57799

Weird, the attachment didn't work. Trying again to attach
the file.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=692988&group_id=5470


From noreply@sourceforge.net  Fri Feb 28 19:52:15 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Fri, 28 Feb 2003 11:52:15 -0800
Subject: [Python-bugs-list] [ python-Bugs-672614 ] python -S still displays 'Type "help", "cop...'
Message-ID: <E18oqYJ-0001MU-00@sc8-sf-web4.sourceforge.net>

Bugs item #672614, was opened at 2003-01-22 18:26
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=672614&group_id=5470

Category: Extension Modules
Group: Python 2.2
Status: Open
Resolution: None
Priority: 1
Submitted By: Douglas Napoleone (derivin)
Assigned to: Nobody/Anonymous (nobody)
Summary: python -S still displays 'Type "help", "cop...'

Initial Comment:
$ python -S
Python 2.2 (#28, Dec 21 2001, 12:21:22) [MSC 32 bit 
(Intel)] on win32
Type "help", "copyright", "credits" or "license" for more 
information.
>>> help
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
NameError: name 'help' is not defined


site.py sets up the builtins for help, credits, etc.
where this is printed in Py_Main() it is already known 
wether or not site.py was going to be imported or not.

I know its a real nit-pick, but it has bugged me for some 
time.

should be set to a LOW priority.

(and yes, the bug is still there in 2.3a1)






----------------------------------------------------------------------

Comment By: Michael Stone (mbrierst)
Date: 2003-02-28 19:52

Message:
Logged In: YES 
user_id=670441

Douglas, I went ahead and made your patch into a diff -c at
patch number 695250, since it seemed like you
weren't going to get to as quickly as you thought.

I would have attached it here, but of course in sourceforge
I cannot.

----------------------------------------------------------------------

Comment By: Douglas Napoleone (derivin)
Date: 2003-01-26 07:31

Message:
Logged In: YES 
user_id=541557

um sure....

its just:
if(!Py_NoSiteFlag)
{
    fprintf(stderr, "%s\n" COPYRIGHT);
}

I'll put up a dicc -c patch on monday after I test it...
seen to many patches that dont compile ^_^


----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2003-01-26 01:08

Message:
Logged In: YES 
user_id=21627

Would you like to work on a patch?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=672614&group_id=5470


From noreply@sourceforge.net  Fri Feb 28 20:08:08 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Fri, 28 Feb 2003 12:08:08 -0800
Subject: [Python-bugs-list] [ python-Bugs-692988 ] test_timeout.py needs 'network' resource
Message-ID: <E18oqng-0008CJ-00@sc8-sf-web1.sourceforge.net>

Bugs item #692988, was opened at 2003-02-25 11:01
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=692988&group_id=5470

Category: None
Group: Python 2.3
>Status: Closed
>Resolution: Fixed
Priority: 5
Submitted By: Cesar Eduardo Barros (cesarb)
Assigned to: Neal Norwitz (nnorwitz)
Summary: test_timeout.py needs 'network' resource

Initial Comment:
test_timeout.py fails in two tests if the network is
down. This is because it tries to connect to
www.google.com:80. It should be skipped unless the
'network' resource is enabled.

Also, the failure mode of the testConnectTimeout is
weird; it took 10 seconds for it to timeout. Maybe
that's a real bug somewhere.

The version I used is test_timeout.py revision 1.11

----------------------------------------------------------------------

>Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-28 15:08

Message:
Logged In: YES 
user_id=33168

test_timeout now requires -u network to run.  See bug 692951.

Checked in as:
 * Lib/test/regrtest.py 1.132
 * Lib/test/test_timeout.py 1.12
 * Misc/NEWS 1.682


----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2003-02-28 13:00

Message:
Logged In: YES 
user_id=6380

sure, good idea.


----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-28 12:53

Message:
Logged In: YES 
user_id=33168

Guido, any objections to making test_timeout require -u
network?  There's another bug with the same problem.

----------------------------------------------------------------------

Comment By: Cesar Eduardo Barros (cesarb)
Date: 2003-02-26 09:40

Message:
Logged In: YES 
user_id=57799

Ah, and maybe the documentation should be updated to make
explicit that the connect() timeout does not include the DNS
lookup (and should not, since it's the _socket_ timeout...).
If you want a timeout in the DNS lookup, you must do the
lookup yourself (but I don't think it's really possible to
change the resolver timeout...)

----------------------------------------------------------------------

Comment By: Cesar Eduardo Barros (cesarb)
Date: 2003-02-26 09:37

Message:
Logged In: YES 
user_id=57799

testConnectTimeout is returning an error because it assumes
every socket.error is a timeout, when it could be, for
instance, a gaierror (which seems to be the case).

Also, the testcase is timing the whole socket.connect()
call, which includes both the DNS lookup and the connect()
proper. To avoid this, it should first do a lookup of
www.google.com by other means and pass the IP, not the
hostname, to the socket.connect() call. Else, it would be
also measuring the DNS resolution time (which can be kinda
long if the nameserver isn't local).

So, we have three problems in the testcase already:
1) It should only run the two remote tests when use of the
'network' resource is enabled
2) It should not assume that any socket.error is a timeout;
it must do more checks to make sure it's really a timeout
3) It should avoid measuring the time spent in
getaddrinfo(3) by resolving the hostname to an IP before
starting the tests (this would also avoid the false failures
because of DNS failure and make them look like a testcase
setup error).

Should I make a separate bug for each of these?

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-25 22:06

Message:
Logged In: YES 
user_id=33168

Whoops, sorry my last comment was meant for bug #692951. 
This report helps clarify the other bug.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-25 22:03

Message:
Logged In: YES 
user_id=33168

The first problem seems to be a DNS lookup failure
(www.google.com).  What Linux distro and version are you
using?  Did you enable ipv6?  Anything else that could help
debug this problem?  Many developers run on Linux and I
haven't heard of this problem?  Is your network very slow? 
Do you have a firewall which introduces large latencies?

----------------------------------------------------------------------

Comment By: Cesar Eduardo Barros (cesarb)
Date: 2003-02-25 11:08

Message:
Logged In: YES 
user_id=57799

Weird, the attachment didn't work. Trying again to attach
the file.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=692988&group_id=5470


From noreply@sourceforge.net  Fri Feb 28 20:07:10 2003
From: noreply@sourceforge.net (SourceForge.net)
Date: Fri, 28 Feb 2003 12:07:10 -0800
Subject: [Python-bugs-list] [ python-Bugs-692951 ] Python 2.3a2 make test fails timeout
Message-ID: <E18oqmk-0001wd-00@sc8-sf-web2.sourceforge.net>

Bugs item #692951, was opened at 2003-02-25 10:16
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=692951&group_id=5470

Category: Build
Group: Python 2.3
>Status: Closed
>Resolution: Fixed
Priority: 5
Submitted By: Charles Hixson (quixo)
>Assigned to: Neal Norwitz (nnorwitz)
Summary: Python 2.3a2 make test fails timeout

Initial Comment:
after:
./config
make
I decided to do:
make test
This mainly worked, but ended up like this:
221 tests OK.
1 test failed:
    test_timeout
22 tests skipped:
    test_aepack test_al test_bsddb3 test_bz2 test_cd
test_cl
    test_curses test_email_codecs test_gl test_imgfile
    test_linuxaudiodev test_macfs test_macostools test_nis
    test_normalization test_ossaudiodev test_pep277
test_socket_ssl
    test_socketserver test_sunaudiodev test_winreg
test_winsound
1 skip unexpected on linux2:
    test_bz2
make: *** [test] Error 1

See attached file for a more complete list of the run.

----------------------------------------------------------------------

>Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-28 15:07

Message:
Logged In: YES 
user_id=33168

test_timeout now requires -u network to run.  See bug 692988.

Checked in as:
 * Lib/test/regrtest.py 1.132
 * Lib/test/test_timeout.py 1.12
 * Misc/NEWS 1.682

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2003-02-25 22:06

Message:
Logged In: YES 
user_id=33168

This appears to be a duplicate of bug 692988.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=692951&group_id=5470