From noreply at sourceforge.net  Sun Apr  1 02:53:15 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Sat, 31 Mar 2007 17:53:15 -0700
Subject: [ python-Bugs-1689617 ] Intel icc build fails with optimizations -O2
Message-ID: <E1HXoJn-0005a8-48@sc8-sf-web24.sourceforge.net>

Bugs item #1689617, was opened at 2007-03-28 04:26
Message generated for change (Comment added) made by mforbes
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1689617&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Build
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Michael Forbes (mforbes)
Assigned to: Nobody/Anonymous (nobody)
Summary: Intel icc build fails with optimizations -O2

Initial Comment:
When building python 2.5 with the free Intel compilers for non-commercial use (icc), the build works without optimizations, but fails when optimizations -O2 and -O3 are used.

The compilation proceedes without errors (though there are lot's of warnings and remarks) and makes a python executable, but when the python setup starts, the intepreter cannot import certain packages and cannot add two strings.

case $MAKEFLAGS in \
*-s*)  CC='icc -pthread' LDSHARED='icc -pthread -shared' OPT='-DNDEBUG -g -O2 -Ob2 -w1' ./python -E ./setup.py -q build;; \
*)  CC='icc -pthread' LDSHARED='icc -pthread -shared' OPT='-DNDEBUG -g -O2 -Ob2 -w1' ./python -E ./setup.py build;; \
esac
'import site' failed; use -v for traceback
Traceback (most recent call last):
  File "./setup.py", line 6, in <module>
    import sys, os, imp, re, optparse
  File "/int/apps/mmf/src/Python-2.5_intel/Lib/optparse.py", line 71, in <module>
    import textwrap
  File "/int/apps/mmf/src/Python-2.5_intel/Lib/textwrap.py", line 10, in <module>
    import string, re
  File "/int/apps/mmf/src/Python-2.5_intel/Lib/string.py", line 26, in <module>
    letters = lowercase + uppercase
SystemError: error return without exception set
make: *** [sharedmods] Error 1

For example:
$ ./python
'import site' failed; use -v for traceback
Python 2.5 (r25:51908, Mar 27 2007, 20:10:22) 
[GCC Intel(R) C++ gcc 3.4 mode] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> a = "123"
>>> a + "12"
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
SystemError: error return without exception set
>>> 

Note that both the import of the site file fails and the addition fails.

Python was configured with the following options:
./configure --with-gcc=icc\
            --with-cxx-main=icc\
            --prefix=/int/apps/mmf/apps/Python-2.5_intel/\
            OPT="-g -O2 -Ob2 -w1"
----------------------------
When compiled with fewer optimiztions, everything seems to work.
./configure --with-gcc=icc\
            --with-cxx-main=icc\
            --prefix=/int/apps/mmf/apps/Python-2.5_intel/\
            OPT="-g -O1 -Ob2 -w1"



----------------------------------------------------------------------

>Comment By: Michael Forbes (mforbes)
Date: 2007-04-01 00:53

Message:
Logged In: YES 
user_id=253921
Originator: YES

I have narrowed the problem a bit to the file Python/ceval.c.  If you
build this manually without optimizations, then everything works fine
("make test" works).
icc -pthread -c -fno-strict-aliasing -DNDEBUG -g -w1 -O1 \
-I. -I./Include -DPy_BUILD_CORE -o Python/ceval.o Python/ceval.c

I have included the warnings the compiler spits out (but it spits out
similar warnings everywhere, so I doubt that these are the problem) and
attached the "optimization report".  I tried turning off as much as
possible (inlining, loop unrolling etc.) but could still not get it to
work.  There are some threads about loop unrolling issues with icc, but
these were for -O3 optimizations:

http://softwarecommunity.intel.com/isn/community/en-us/forums/thread/321812.aspx

If I compile with -O1 and -Ob1 then it breaks and I get the optimization
report attached.  The wierd thing is that if I use -O1 and -finline, I get
an empty optimization report and it works.  TFM states that "-finline [is
the] Same as -Ob1."  This at least is a compiler problem!  I suspect that
-Ob1 is doing more than just inlining (it looks that way from the report)
so I am not exactly sure where the problem is here yet.
 
icc -pthread -c -fno-strict-aliasing -DNDEBUG -g -Wall -O1\
 -Ob1 -opt-report -opt-report-level=max -opt-report-file=opt.rep.txt\
  -I. -I./Include   -DPy_BUILD_CORE -o Python/ceval.o Python/ceval.c

Python/ceval.c(198): remark #869: parameter "self" was never referenced
  PyEval_GetCallStats(PyObject *self)
                                ^
Python/ceval.c(2710): remark #981: operands are evaluated in unspecified
order
                                            PyString_AsString(keyword));
                                            ^
Python/ceval.c(2706): remark #981: operands are evaluated in unspecified
order
                                        PyErr_Format(PyExc_TypeError,
                                        ^
Python/ceval.c(2722): remark #981: operands are evaluated in unspecified
order
                                             PyString_AsString(keyword));
                                             ^
Python/ceval.c(2717): remark #981: operands are evaluated in unspecified
order
                                        PyErr_Format(PyExc_TypeError,
                                        ^
Python/ceval.c(3690): remark #981: operands are evaluated in unspecified
order
                                     PyEval_GetFuncDesc(func),
                                     ^
Python/ceval.c(3686): remark #981: operands are evaluated in unspecified
order
                          PyErr_Format(PyExc_TypeError,
                          ^
Python/ceval.c(3800): remark #981: operands are evaluated in unspecified
order
                                     PyEval_GetFuncDesc(func));
                                     ^
Python/ceval.c(3796): remark #981: operands are evaluated in unspecified
order
                        PyErr_Format(PyExc_TypeError,
                        ^
Python/ceval.c(3815): remark #981: operands are evaluated in unspecified
order
                                                    
PyEval_GetFuncDesc(func));
                                                     ^
Python/ceval.c(3811): remark #981: operands are evaluated in unspecified
order
                                        PyErr_Format(PyExc_TypeError,
                                        ^
File Added: opt.rep.txt

----------------------------------------------------------------------

Comment By: Michael Forbes (mforbes)
Date: 2007-03-31 20:31

Message:
Logged In: YES 
user_id=253921
Originator: YES

It does not appear to be a problem with aliasing (icc accepts the
-fno-strict-aliasing flag and it is set by default using ./configure.  It
is another question of whether or not icc behaves this flag!).

This looks similar to a previous problem but I am not using any processor
specific optimizations:
http://mail.python.org/pipermail/python-list/2005-March/312145.html


----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2007-03-31 19:56

Message:
Logged In: YES 
user_id=33168
Originator: NO

My guess is that it relates to strict aliasing (which Python violates). 
Can you find the equivalent flag to gcc's -fno-strict-aliasing in icc?

----------------------------------------------------------------------

Comment By: Michael Forbes (mforbes)
Date: 2007-03-29 22:06

Message:
Logged In: YES 
user_id=253921
Originator: YES

I have spoken with some people who have had similar issues with the intel
compilers in other applications, so it looks like it might be a compiler
bug.  I will look into this.

Can anyone confirm this behaviour? (I only have access to one linux box).

----------------------------------------------------------------------

Comment By: Martin v. L?wis (loewis)
Date: 2007-03-28 11:32

Message:
Logged In: YES 
user_id=21627
Originator: NO

Can you debug this further to determine a specific problem? It could be a
compiler bug also, for all I can tell.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1689617&group_id=5470

From noreply at sourceforge.net  Sun Apr  1 09:04:21 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Sun, 01 Apr 2007 00:04:21 -0700
Subject: [ python-Bugs-1622664 ] language reference index links are broken
Message-ID: <E1HXu6v-0002NS-5Q@sc8-sf-web24.sourceforge.net>

Bugs item #1622664, was opened at 2006-12-26 16:15
Message generated for change (Comment added) made by drewp
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1622664&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Documentation
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Drew Perttula (drewp)
Assigned to: Nobody/Anonymous (nobody)
Summary: language reference index links are broken

Initial Comment:
http://docs.python.org/ref/genindex.html

For example, this text "globals() (built-in function)" links to http://docs.python.org/ref/exec.html#l2h-571, which is completely wrong.

Some links are correct, but many others are broken.

The page footer says "Release 2.5, documentation updated on 19th September, 2006."


----------------------------------------------------------------------

>Comment By: Drew Perttula (drewp)
Date: 2007-03-31 23:04

Message:
Logged In: YES 
user_id=127598
Originator: YES

Oh sorry, you're right. I was of course expecting
http://docs.python.org/lib/built-in-funcs.html#l2h-34 but that's in the
other doc section. 

Thanks for reviewing my report- ok to close.


----------------------------------------------------------------------

Comment By: Facundo Batista (facundobatista)
Date: 2007-03-31 04:05

Message:
Logged In: YES 
user_id=752496
Originator: NO

Where do you think it should point to?

Take note that you're looking into the language reference, not the library
reference.

Regards,

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1622664&group_id=5470

From noreply at sourceforge.net  Sun Apr  1 09:19:30 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Sun, 01 Apr 2007 00:19:30 -0700
Subject: [ python-Bugs-1622664 ] language reference index links are broken
Message-ID: <E1HXuLa-0005AL-PK@sc8-sf-web23.sourceforge.net>

Bugs item #1622664, was opened at 2006-12-27 01:15
Message generated for change (Settings changed) made by gbrandl
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1622664&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Documentation
Group: None
>Status: Closed
>Resolution: Wont Fix
Priority: 5
Private: No
Submitted By: Drew Perttula (drewp)
Assigned to: Nobody/Anonymous (nobody)
Summary: language reference index links are broken

Initial Comment:
http://docs.python.org/ref/genindex.html

For example, this text "globals() (built-in function)" links to http://docs.python.org/ref/exec.html#l2h-571, which is completely wrong.

Some links are correct, but many others are broken.

The page footer says "Release 2.5, documentation updated on 19th September, 2006."


----------------------------------------------------------------------

Comment By: Drew Perttula (drewp)
Date: 2007-04-01 07:04

Message:
Logged In: YES 
user_id=127598
Originator: YES

Oh sorry, you're right. I was of course expecting
http://docs.python.org/lib/built-in-funcs.html#l2h-34 but that's in the
other doc section. 

Thanks for reviewing my report- ok to close.


----------------------------------------------------------------------

Comment By: Facundo Batista (facundobatista)
Date: 2007-03-31 12:05

Message:
Logged In: YES 
user_id=752496
Originator: NO

Where do you think it should point to?

Take note that you're looking into the language reference, not the library
reference.

Regards,

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1622664&group_id=5470

From noreply at sourceforge.net  Sun Apr  1 12:05:12 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Sun, 01 Apr 2007 03:05:12 -0700
Subject: [ python-Bugs-1692280 ] lambda function segfault
Message-ID: <E1HXwvw-0001S8-2U@sc8-sf-web21.sourceforge.net>

Bugs item #1692280, was opened at 2007-04-01 18:05
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1692280&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Interpreter Core
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Dasn (dasn)
Assigned to: Nobody/Anonymous (nobody)
Summary: lambda function segfault

Initial Comment:
>>> lambda ((x,y)): 'ok'
Segment fault (core dumped).

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1692280&group_id=5470

From noreply at sourceforge.net  Sun Apr  1 12:11:22 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Sun, 01 Apr 2007 03:11:22 -0700
Subject: [ python-Bugs-1692280 ] lambda function segfault
Message-ID: <E1HXx1u-00026k-6g@sc8-sf-web21.sourceforge.net>

Bugs item #1692280, was opened at 2007-04-01 18:05
Message generated for change (Settings changed) made by dasn
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1692280&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Interpreter Core
Group: Python 2.5
Status: Open
Resolution: None
>Priority: 9
Private: No
Submitted By: Dasn (dasn)
Assigned to: Nobody/Anonymous (nobody)
Summary: lambda function segfault

Initial Comment:
>>> lambda ((x,y)): 'ok'
Segment fault (core dumped).

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1692280&group_id=5470

From noreply at sourceforge.net  Sun Apr  1 12:15:39 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Sun, 01 Apr 2007 03:15:39 -0700
Subject: [ python-Bugs-1692280 ] lambda function segfault
Message-ID: <E1HXx63-0002XH-RL@sc8-sf-web21.sourceforge.net>

Bugs item #1692280, was opened at 2007-04-01 10:05
Message generated for change (Comment added) made by gbrandl
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1692280&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Interpreter Core
Group: Python 2.5
>Status: Closed
>Resolution: Out of Date
Priority: 9
Private: No
Submitted By: Dasn (dasn)
Assigned to: Nobody/Anonymous (nobody)
Summary: lambda function segfault

Initial Comment:
>>> lambda ((x,y)): 'ok'
Segment fault (core dumped).

----------------------------------------------------------------------

>Comment By: Georg Brandl (gbrandl)
Date: 2007-04-01 10:15

Message:
Logged In: YES 
user_id=849994
Originator: NO

Thanks, this is already fixed in SVN and will be part of 2.5.1, which is
scheduled to be released soon.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1692280&group_id=5470

From noreply at sourceforge.net  Mon Apr  2 01:00:56 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Sun, 01 Apr 2007 16:00:56 -0700
Subject: [ python-Feature Requests-1692592 ] Stripping debugging symbols from
	compiled C extensions
Message-ID: <E1HY92e-0005Bh-DL@sc8-sf-web21.sourceforge.net>

Feature Requests item #1692592, was opened at 2007-04-02 01:00
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1692592&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Distutils
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Viktor Ferenczi (complex)
Assigned to: Nobody/Anonymous (nobody)
Summary: Stripping debugging symbols from compiled C extensions

Initial Comment:
It would be nice to automatically strip debugging symbols from compiled library files (such as .so files) if a C extension module is not build in debug mode. This could somehow reduce memory footprint and storage requirements for extension modules.

Distutils already does this for cygwin and emx compilers with the following code in cygwinccompiler.py and emxccompiler.py:

# who wants symbols and a many times larger output file
# should explicitly switch the debug mode on
# otherwise we let dllwrap/ld strip the output file
# (On my machine: 10KB < stripped_file < ??100KB
#   unstripped_file = stripped_file + XXX KB
#  ( XXX=254 for a typical python extension))
if not debug:
    extra_preargs.append("-s")

This code should be somehow integrated into base compiler classes such as UnixCCompiler. I've added the following at the beginning of UnixCCompiler.link function:

if not debug:
    if extra_preargs is None:
        extra_preargs = []
    extra_preargs.append("-s")

This works for me with gcc under Linux (Debian Sarge). I does not provide a patch, since this could be the best solution for this.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1692592&group_id=5470

From noreply at sourceforge.net  Mon Apr  2 01:53:38 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Sun, 01 Apr 2007 16:53:38 -0700
Subject: [ python-Bugs-1689458 ] pdb unable to jump to first statement
Message-ID: <E1HY9re-0002T2-Rr@sc8-sf-web21.sourceforge.net>

Bugs item #1689458, was opened at 2007-03-27 16:07
Message generated for change (Comment added) made by gotgenes
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1689458&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
>Category: Python Interpreter Core
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Chris Lasher (gotgenes)
Assigned to: Nobody/Anonymous (nobody)
Summary: pdb unable to jump to first statement

Initial Comment:
The Python debugger is unable to "jump" back to the first executable statement in a frame once that statement has been executed. For example:

chris at feathers:~/development/playground$ python -m pdb simple.py 
> /home/chris/development/playground/simple.py(3)?()
-> a = 1
(Pdb) next
> /home/chris/development/playground/simple.py(4)?()
-> b = 2
(Pdb) jump 3
> /home/chris/development/playground/simple.py(3)?()
-> a = 1
(Pdb) list
  1     #!/usr/bin/env python
  2  
  3     a = 1
  4  -> b = 2
  5  
  6     c = a + b
  7  
  8     print c
[EOF]
(Pdb) next
> /home/chris/development/playground/simple.py(6)?()
-> c = a + b

One can see that after supposedly "jump"ing to line 3 at the second command, when "list"ing the line, the debugger is actually at line 4. The "next" command further demonstrates this since it re-executes line 4 and moves to line 6.

This issue was raised on comp.lang.python. (For example, see
<http://groups.google.com/group/comp.lang.python/browse_thread/thread/7960201616873f41/e9623c08e3618051>
or if that link is munged, refer to
<http://tinyurl.com/324feu>

Duncan Booth offers the following:
[quote]
I verified (with a print statement in pdb) that assigning to self.curframe.f_lineno sets self.curframe.f_lineno and sel.curframe.f_lasti incorrectly

...

The problem looks to be in frameobject.c:

        addr = 0;
        line = f->f_code->co_firstlineno;
        new_lasti = -1;
        for (offset = 0; offset < lnotab_len; offset += 2) {
                addr += lnotab[offset];
                line += lnotab[offset+1];
                if (line >= new_lineno) {
                        new_lasti = addr;
                        new_lineno = line;
                        break;
                }
        }

The first bytes in lnotab are the length and line increment for line 3 (i.e. 6, 1). If line==f->f_code->co_firstlineno it should set new_lasti=0, new_lineno=line but the loop still executes once which increments new_lasti and new_lineno to the next line (6, 4).
[/quote]

And Rocky Bernstein offers the following:
[quote]
Best as I can tell, it looks like a bug in Python. pdb, pydb, rpdb2 all handle the "jump" command by changing the frame f_lineno value. When the corresponding code pointer has offset 0 (or equivalently and more simlply as you put it, is the first statement) this doesn't seem to work properly.
[/quote]

----------------------------------------------------------------------

>Comment By: Chris Lasher (gotgenes)
Date: 2007-04-01 18:53

Message:
Logged In: YES 
user_id=1180453
Originator: YES

I'm changing the category. Apparently this is a bug in the Python core,
itself, not pdb.

>From Rocky Bernstein on comp.lang.python
[quote]
At present it looks like the bug is in Python, and its handling after
setting the current frame's f_lineno when it refers to byte code offset 0,
not pdb.
[/quote]

----------------------------------------------------------------------

Comment By: Rocky Bernstein (rockyb)
Date: 2007-03-31 05:32

Message:
Logged In: YES 
user_id=158581
Originator: NO

Although a single file unit test would be nice, below is a short simple
program that I think clearly shows the bug. Alas, as a follow-up comment I
don't see a way to attach files so I have to paste it inline. However with
this example and the information from Duncan Booth, I think the problem and
how to fix it is pretty clear.

file: jumpbug.py

#!/usr/bin/env python
import inspect, linecache, sys
def tracer(frame, event, arg):
    global z
    (filename, line_no) = inspect.getframeinfo(frame)[0:2]
    print "Event %s at line %d:" % (event, line_no)
    print "\t", linecache.getline(filename, line_no),
    print "----------------------"
    try: 
        if z == 0:
            if line_no == 4:
                print "***We jumped back to line 4 but should have gone to
2**"
                sys.exit(1)
            frame.f_lineno = 2 # And 3 is broken too.
    except NameError:
        pass
    return tracer # This helps emacs figure indentation out
sys.settrace(tracer)
execfile("jumpbug2.py")
#END first file

file jumpbug2.py:

#!/usr/bin/env python
x = 2  # This statement gets skipped the 2nd time around
q = 1  # This statement gets skipped too!
try:   # tracer() will exit here if z == 0 and line_no == 4
    y = z   
except NameError:
    z = 0
print "When tracing via tracer(), f_lineno will be set to 2 here."
print "You should never get here when tracing"


file jumpbug2.py:

----------------------------------------------------------------------

Comment By: Chris Lasher (gotgenes)
Date: 2007-03-28 20:24

Message:
Logged In: YES 
user_id=1180453
Originator: YES

Truthfully, I don't have enough know-how to write a full test case or a
patch. I have put forth a request on the comp.lang.python thread (link in
original report) for those that would to please do so.

----------------------------------------------------------------------

Comment By: Collin Winter (collinwinter)
Date: 2007-03-28 16:54

Message:
Logged In: YES 
user_id=1344176
Originator: NO

Could you work up a full test case for this? Or better yet, a patch?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1689458&group_id=5470

From noreply at sourceforge.net  Mon Apr  2 02:06:32 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Sun, 01 Apr 2007 17:06:32 -0700
Subject: [ python-Bugs-1689458 ] pdb unable to jump to first statement
Message-ID: <E1HYA48-0004Lb-8e@sc8-sf-web23.sourceforge.net>

Bugs item #1689458, was opened at 2007-03-27 16:07
Message generated for change (Comment added) made by gotgenes
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1689458&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Interpreter Core
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Chris Lasher (gotgenes)
Assigned to: Nobody/Anonymous (nobody)
Summary: pdb unable to jump to first statement

Initial Comment:
The Python debugger is unable to "jump" back to the first executable statement in a frame once that statement has been executed. For example:

chris at feathers:~/development/playground$ python -m pdb simple.py 
> /home/chris/development/playground/simple.py(3)?()
-> a = 1
(Pdb) next
> /home/chris/development/playground/simple.py(4)?()
-> b = 2
(Pdb) jump 3
> /home/chris/development/playground/simple.py(3)?()
-> a = 1
(Pdb) list
  1     #!/usr/bin/env python
  2  
  3     a = 1
  4  -> b = 2
  5  
  6     c = a + b
  7  
  8     print c
[EOF]
(Pdb) next
> /home/chris/development/playground/simple.py(6)?()
-> c = a + b

One can see that after supposedly "jump"ing to line 3 at the second command, when "list"ing the line, the debugger is actually at line 4. The "next" command further demonstrates this since it re-executes line 4 and moves to line 6.

This issue was raised on comp.lang.python. (For example, see
<http://groups.google.com/group/comp.lang.python/browse_thread/thread/7960201616873f41/e9623c08e3618051>
or if that link is munged, refer to
<http://tinyurl.com/324feu>

Duncan Booth offers the following:
[quote]
I verified (with a print statement in pdb) that assigning to self.curframe.f_lineno sets self.curframe.f_lineno and sel.curframe.f_lasti incorrectly

...

The problem looks to be in frameobject.c:

        addr = 0;
        line = f->f_code->co_firstlineno;
        new_lasti = -1;
        for (offset = 0; offset < lnotab_len; offset += 2) {
                addr += lnotab[offset];
                line += lnotab[offset+1];
                if (line >= new_lineno) {
                        new_lasti = addr;
                        new_lineno = line;
                        break;
                }
        }

The first bytes in lnotab are the length and line increment for line 3 (i.e. 6, 1). If line==f->f_code->co_firstlineno it should set new_lasti=0, new_lineno=line but the loop still executes once which increments new_lasti and new_lineno to the next line (6, 4).
[/quote]

And Rocky Bernstein offers the following:
[quote]
Best as I can tell, it looks like a bug in Python. pdb, pydb, rpdb2 all handle the "jump" command by changing the frame f_lineno value. When the corresponding code pointer has offset 0 (or equivalently and more simlply as you put it, is the first statement) this doesn't seem to work properly.
[/quote]

----------------------------------------------------------------------

>Comment By: Chris Lasher (gotgenes)
Date: 2007-04-01 19:06

Message:
Logged In: YES 
user_id=1180453
Originator: YES

File Added: jumpbug.py

----------------------------------------------------------------------

Comment By: Chris Lasher (gotgenes)
Date: 2007-04-01 18:53

Message:
Logged In: YES 
user_id=1180453
Originator: YES

I'm changing the category. Apparently this is a bug in the Python core,
itself, not pdb.

>From Rocky Bernstein on comp.lang.python
[quote]
At present it looks like the bug is in Python, and its handling after
setting the current frame's f_lineno when it refers to byte code offset 0,
not pdb.
[/quote]

----------------------------------------------------------------------

Comment By: Rocky Bernstein (rockyb)
Date: 2007-03-31 05:32

Message:
Logged In: YES 
user_id=158581
Originator: NO

Although a single file unit test would be nice, below is a short simple
program that I think clearly shows the bug. Alas, as a follow-up comment I
don't see a way to attach files so I have to paste it inline. However with
this example and the information from Duncan Booth, I think the problem and
how to fix it is pretty clear.

file: jumpbug.py

#!/usr/bin/env python
import inspect, linecache, sys
def tracer(frame, event, arg):
    global z
    (filename, line_no) = inspect.getframeinfo(frame)[0:2]
    print "Event %s at line %d:" % (event, line_no)
    print "\t", linecache.getline(filename, line_no),
    print "----------------------"
    try: 
        if z == 0:
            if line_no == 4:
                print "***We jumped back to line 4 but should have gone to
2**"
                sys.exit(1)
            frame.f_lineno = 2 # And 3 is broken too.
    except NameError:
        pass
    return tracer # This helps emacs figure indentation out
sys.settrace(tracer)
execfile("jumpbug2.py")
#END first file

file jumpbug2.py:

#!/usr/bin/env python
x = 2  # This statement gets skipped the 2nd time around
q = 1  # This statement gets skipped too!
try:   # tracer() will exit here if z == 0 and line_no == 4
    y = z   
except NameError:
    z = 0
print "When tracing via tracer(), f_lineno will be set to 2 here."
print "You should never get here when tracing"


file jumpbug2.py:

----------------------------------------------------------------------

Comment By: Chris Lasher (gotgenes)
Date: 2007-03-28 20:24

Message:
Logged In: YES 
user_id=1180453
Originator: YES

Truthfully, I don't have enough know-how to write a full test case or a
patch. I have put forth a request on the comp.lang.python thread (link in
original report) for those that would to please do so.

----------------------------------------------------------------------

Comment By: Collin Winter (collinwinter)
Date: 2007-03-28 16:54

Message:
Logged In: YES 
user_id=1344176
Originator: NO

Could you work up a full test case for this? Or better yet, a patch?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1689458&group_id=5470

From noreply at sourceforge.net  Mon Apr  2 02:07:32 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Sun, 01 Apr 2007 17:07:32 -0700
Subject: [ python-Bugs-1689458 ] pdb unable to jump to first statement
Message-ID: <E1HYA56-0004S7-63@sc8-sf-web23.sourceforge.net>

Bugs item #1689458, was opened at 2007-03-27 16:07
Message generated for change (Comment added) made by gotgenes
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1689458&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Interpreter Core
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Chris Lasher (gotgenes)
Assigned to: Nobody/Anonymous (nobody)
Summary: pdb unable to jump to first statement

Initial Comment:
The Python debugger is unable to "jump" back to the first executable statement in a frame once that statement has been executed. For example:

chris at feathers:~/development/playground$ python -m pdb simple.py 
> /home/chris/development/playground/simple.py(3)?()
-> a = 1
(Pdb) next
> /home/chris/development/playground/simple.py(4)?()
-> b = 2
(Pdb) jump 3
> /home/chris/development/playground/simple.py(3)?()
-> a = 1
(Pdb) list
  1     #!/usr/bin/env python
  2  
  3     a = 1
  4  -> b = 2
  5  
  6     c = a + b
  7  
  8     print c
[EOF]
(Pdb) next
> /home/chris/development/playground/simple.py(6)?()
-> c = a + b

One can see that after supposedly "jump"ing to line 3 at the second command, when "list"ing the line, the debugger is actually at line 4. The "next" command further demonstrates this since it re-executes line 4 and moves to line 6.

This issue was raised on comp.lang.python. (For example, see
<http://groups.google.com/group/comp.lang.python/browse_thread/thread/7960201616873f41/e9623c08e3618051>
or if that link is munged, refer to
<http://tinyurl.com/324feu>

Duncan Booth offers the following:
[quote]
I verified (with a print statement in pdb) that assigning to self.curframe.f_lineno sets self.curframe.f_lineno and sel.curframe.f_lasti incorrectly

...

The problem looks to be in frameobject.c:

        addr = 0;
        line = f->f_code->co_firstlineno;
        new_lasti = -1;
        for (offset = 0; offset < lnotab_len; offset += 2) {
                addr += lnotab[offset];
                line += lnotab[offset+1];
                if (line >= new_lineno) {
                        new_lasti = addr;
                        new_lineno = line;
                        break;
                }
        }

The first bytes in lnotab are the length and line increment for line 3 (i.e. 6, 1). If line==f->f_code->co_firstlineno it should set new_lasti=0, new_lineno=line but the loop still executes once which increments new_lasti and new_lineno to the next line (6, 4).
[/quote]

And Rocky Bernstein offers the following:
[quote]
Best as I can tell, it looks like a bug in Python. pdb, pydb, rpdb2 all handle the "jump" command by changing the frame f_lineno value. When the corresponding code pointer has offset 0 (or equivalently and more simlply as you put it, is the first statement) this doesn't seem to work properly.
[/quote]

----------------------------------------------------------------------

>Comment By: Chris Lasher (gotgenes)
Date: 2007-04-01 19:07

Message:
Logged In: YES 
user_id=1180453
Originator: YES

File Added: jumpbug2.py

----------------------------------------------------------------------

Comment By: Chris Lasher (gotgenes)
Date: 2007-04-01 19:06

Message:
Logged In: YES 
user_id=1180453
Originator: YES

File Added: jumpbug.py

----------------------------------------------------------------------

Comment By: Chris Lasher (gotgenes)
Date: 2007-04-01 18:53

Message:
Logged In: YES 
user_id=1180453
Originator: YES

I'm changing the category. Apparently this is a bug in the Python core,
itself, not pdb.

>From Rocky Bernstein on comp.lang.python
[quote]
At present it looks like the bug is in Python, and its handling after
setting the current frame's f_lineno when it refers to byte code offset 0,
not pdb.
[/quote]

----------------------------------------------------------------------

Comment By: Rocky Bernstein (rockyb)
Date: 2007-03-31 05:32

Message:
Logged In: YES 
user_id=158581
Originator: NO

Although a single file unit test would be nice, below is a short simple
program that I think clearly shows the bug. Alas, as a follow-up comment I
don't see a way to attach files so I have to paste it inline. However with
this example and the information from Duncan Booth, I think the problem and
how to fix it is pretty clear.

file: jumpbug.py

#!/usr/bin/env python
import inspect, linecache, sys
def tracer(frame, event, arg):
    global z
    (filename, line_no) = inspect.getframeinfo(frame)[0:2]
    print "Event %s at line %d:" % (event, line_no)
    print "\t", linecache.getline(filename, line_no),
    print "----------------------"
    try: 
        if z == 0:
            if line_no == 4:
                print "***We jumped back to line 4 but should have gone to
2**"
                sys.exit(1)
            frame.f_lineno = 2 # And 3 is broken too.
    except NameError:
        pass
    return tracer # This helps emacs figure indentation out
sys.settrace(tracer)
execfile("jumpbug2.py")
#END first file

file jumpbug2.py:

#!/usr/bin/env python
x = 2  # This statement gets skipped the 2nd time around
q = 1  # This statement gets skipped too!
try:   # tracer() will exit here if z == 0 and line_no == 4
    y = z   
except NameError:
    z = 0
print "When tracing via tracer(), f_lineno will be set to 2 here."
print "You should never get here when tracing"


file jumpbug2.py:

----------------------------------------------------------------------

Comment By: Chris Lasher (gotgenes)
Date: 2007-03-28 20:24

Message:
Logged In: YES 
user_id=1180453
Originator: YES

Truthfully, I don't have enough know-how to write a full test case or a
patch. I have put forth a request on the comp.lang.python thread (link in
original report) for those that would to please do so.

----------------------------------------------------------------------

Comment By: Collin Winter (collinwinter)
Date: 2007-03-28 16:54

Message:
Logged In: YES 
user_id=1344176
Originator: NO

Could you work up a full test case for this? Or better yet, a patch?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1689458&group_id=5470

From noreply at sourceforge.net  Mon Apr  2 09:35:56 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Mon, 02 Apr 2007 00:35:56 -0700
Subject: [ python-Bugs-1692695 ] webbrowser module cannot open the right URL
Message-ID: <E1HYH52-0002xh-EB@sc8-sf-web22.sourceforge.net>

Bugs item #1692695, was opened at 2007-04-02 08:35
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1692695&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Pan Yongzhi (fossilet)
Assigned to: Nobody/Anonymous (nobody)
Summary: webbrowser module cannot open the right URL

Initial Comment:
My python version is:
tux 0 ~ $ ipython 
Python 2.5 (release25-maint, Mar 29 2007, 11:14:31) 
Type "copyright", "credits" or "license" for more information.

I use the example given at http://docs.python.org/lib/module-webbrowser.html :
In [1]: import webbrowser

In [2]: url = 'http://www.python.org'

In [3]: webbrowser.open_new_tab(url + '/doc')

The location bar of firefox shows:

file:///home/tux/%22http://www.python.org/doc%22

and firefox cannot open the URL.

This problem also occurs with local dir URLs. webbrowser.open also has this problem. In python 2.4.4 things are OK.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1692695&group_id=5470

From noreply at sourceforge.net  Mon Apr  2 09:46:22 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Mon, 02 Apr 2007 00:46:22 -0700
Subject: [ python-Bugs-1692695 ] webbrowser module cannot open the right URL
Message-ID: <E1HYHF8-00049g-9Y@sc8-sf-web22.sourceforge.net>

Bugs item #1692695, was opened at 2007-04-02 07:35
Message generated for change (Comment added) made by gbrandl
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1692695&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.5
>Status: Closed
>Resolution: Out of Date
Priority: 5
Private: No
Submitted By: Pan Yongzhi (fossilet)
Assigned to: Nobody/Anonymous (nobody)
Summary: webbrowser module cannot open the right URL

Initial Comment:
My python version is:
tux 0 ~ $ ipython 
Python 2.5 (release25-maint, Mar 29 2007, 11:14:31) 
Type "copyright", "credits" or "license" for more information.

I use the example given at http://docs.python.org/lib/module-webbrowser.html :
In [1]: import webbrowser

In [2]: url = 'http://www.python.org'

In [3]: webbrowser.open_new_tab(url + '/doc')

The location bar of firefox shows:

file:///home/tux/%22http://www.python.org/doc%22

and firefox cannot open the URL.

This problem also occurs with local dir URLs. webbrowser.open also has this problem. In python 2.4.4 things are OK.

----------------------------------------------------------------------

>Comment By: Georg Brandl (gbrandl)
Date: 2007-04-02 07:46

Message:
Logged In: YES 
user_id=849994
Originator: NO

This should be fixed in 2.5.1 which comes out in a few days. If not,
please reopen.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1692695&group_id=5470

From noreply at sourceforge.net  Mon Apr  2 17:27:11 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Mon, 02 Apr 2007 08:27:11 -0700
Subject: [ python-Bugs-1693050 ] \w not helpful for non-Roman scripts
Message-ID: <E1HYOR5-00019B-SG@sc8-sf-web21.sourceforge.net>

Bugs item #1693050, was opened at 2007-04-02 09:27
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1693050&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Unicode
Group: Python 2.4
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: nlmiles (nathanlmiles)
Assigned to: M.-A. Lemburg (lemburg)
Summary: \w not helpful for non-Roman scripts

Initial Comment:
When I try to use r"\w+(?u)" to find words in a unicode Devanagari text bad things happen. Words get chopped into small pieces. I think this is likely because vowel signs such as 093e are not considered to match \w.

I think that if you wish \w to be useful for Indic
scipts \w will need to be exanded to unclude unicode character categories Mc, Mn, Me.

I am using Python 2.4.4 on Windows XP SP2.

I ran the following script to see the characters which I think ought to match \w but don't

import re
import unicodedata

text = ""
for i in range(0x901,0x939): text += unichr(i)
for i in range(0x93c,0x93d): text += unichr(i)
for i in range(0x93e,0x94d): text += unichr(i)
for i in range(0x950,0x954): text += unichr(i)
for i in range(0x958,0x963): text += unichr(i)
        
parts = re.findall("\W(?u)", text)
for ch in parts:
    print "%04x" % ord(ch), unicodedata.category(ch)

The odd character here is 0904. Its categorization seems to imply that you are using the uncode 3.0 database but perhaps later versions of Python are using the current 5.0 database.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1693050&group_id=5470

From noreply at sourceforge.net  Mon Apr  2 17:29:07 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Mon, 02 Apr 2007 08:29:07 -0700
Subject: [ python-Bugs-1686386 ] Python SEGFAULT on invalid superclass access
Message-ID: <E1HYOSx-0001Ny-L7@sc8-sf-web21.sourceforge.net>

Bugs item #1686386, was opened at 2007-03-22 17:00
Message generated for change (Settings changed) made by rhettinger
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1686386&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Interpreter Core
Group: Python 2.5
Status: Open
Resolution: None
>Priority: 7
Private: No
Submitted By: toxik (ludvig.ericson)
Assigned to: Nobody/Anonymous (nobody)
Summary: Python SEGFAULT on invalid superclass access

Initial Comment:
When one uses a class that has derived BaseException in one way or another and uses an invalid super() and calls a function upon that object, Python dies with SIGSEGV.

Reproduce code:
>>> class X(BaseException):
...     def __init__(self):
...             super(X, self).__init__(self)
... 
>>> X()
Segmentation fault

I could reproduce this on two different Python 2.5 installations.

This is as much as I could get from gdb:
Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread -1211660624 (LWP 30234)]
0xb7ea601c in _PyObject_GC_Malloc () from /usr/lib/libpython2.5.so.1.0
(gdb) bt
#0  0xb7ea601c in _PyObject_GC_Malloc () from /usr/lib/libpython2.5.so.1.0
#1  0xb7ea613b in _PyObject_GC_NewVar () from /usr/lib/libpython2.5.so.1.0
#2  0xb7e4abe4 in PyTuple_New () from /usr/lib/libpython2.5.so.1.0
#3  0xb7e4b48d in ?? () from /usr/lib/libpython2.5.so.1.0
#4  0x00000001 in ?? ()
#5  0x00000000 in ?? ()


----------------------------------------------------------------------

Comment By: Georg Brandl (gbrandl)
Date: 2007-03-22 17:37

Message:
Logged In: YES 
user_id=849994
Originator: NO

This is not new in 2.5. That is does not work with super() in 2.4 is
because in 2.4 exceptions are old-style classes.
Look at this:
>>> class X(Exception):
...  def __init__(self):
...   Exception.__init__(self, self)
...
>>> x=X()
>>> str(x)
[1]    4396 segmentation fault  python2.4

The problem is that str(x) calls str(x) etc.

----------------------------------------------------------------------

Comment By: toxik (ludvig.ericson)
Date: 2007-03-22 17:09

Message:
Logged In: YES 
user_id=1373326
Originator: YES

It might be added that this works (throws an exception) in python 2.4
(though, BaseException does not exist there):
TypeError: super() argument 1 must be type, not classobj


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1686386&group_id=5470

From noreply at sourceforge.net  Mon Apr  2 17:38:01 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Mon, 02 Apr 2007 08:38:01 -0700
Subject: [ python-Bugs-1693050 ] \w not helpful for non-Roman scripts
Message-ID: <E1HYObZ-0002Vf-D7@sc8-sf-web22.sourceforge.net>

Bugs item #1693050, was opened at 2007-04-02 17:27
Message generated for change (Comment added) made by lemburg
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1693050&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
>Category: Regular Expressions
Group: Python 2.4
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: nlmiles (nathanlmiles)
>Assigned to: Nobody/Anonymous (nobody)
Summary: \w not helpful for non-Roman scripts

Initial Comment:
When I try to use r"\w+(?u)" to find words in a unicode Devanagari text bad things happen. Words get chopped into small pieces. I think this is likely because vowel signs such as 093e are not considered to match \w.

I think that if you wish \w to be useful for Indic
scipts \w will need to be exanded to unclude unicode character categories Mc, Mn, Me.

I am using Python 2.4.4 on Windows XP SP2.

I ran the following script to see the characters which I think ought to match \w but don't

import re
import unicodedata

text = ""
for i in range(0x901,0x939): text += unichr(i)
for i in range(0x93c,0x93d): text += unichr(i)
for i in range(0x93e,0x94d): text += unichr(i)
for i in range(0x950,0x954): text += unichr(i)
for i in range(0x958,0x963): text += unichr(i)
        
parts = re.findall("\W(?u)", text)
for ch in parts:
    print "%04x" % ord(ch), unicodedata.category(ch)

The odd character here is 0904. Its categorization seems to imply that you are using the uncode 3.0 database but perhaps later versions of Python are using the current 5.0 database.

----------------------------------------------------------------------

>Comment By: M.-A. Lemburg (lemburg)
Date: 2007-04-02 17:38

Message:
Logged In: YES 
user_id=38388
Originator: NO

Python 2.4 is using Unicode 3.2. Python 2.5 ships with Unicode 4.1.

We're likely to ship Unicode 5.x with Python 2.6 or a later release.

Regarding the char classes: I don't think Mc, Mn and Me should be
considered parts of a word. Those are marks which usually separate words.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1693050&group_id=5470

From noreply at sourceforge.net  Mon Apr  2 18:19:37 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Mon, 02 Apr 2007 09:19:37 -0700
Subject: [ python-Bugs-1693079 ] Can't save empty array in shelve
Message-ID: <E1HYPFp-0006iW-R7@sc8-sf-web24.sourceforge.net>

Bugs item #1693079, was opened at 2007-04-02 18:19
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1693079&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: IWL (iwl)
Assigned to: Nobody/Anonymous (nobody)
Summary: Can't save empty array in shelve

Initial Comment:

iwl wrote: 
> I've tryed to save some data containing empty arrays (array('f')) in a 
> shelve. 
> It looks like the shelve has some problems with empty arrays, get 
> allways: 
> TypeError: ("'NoneType' object is not iterable", <type 'array.array'>, 
> ('f', None))- 
> Messages when dealing with the readed back shelve. 
> Seems like I have to avoid empty arrays in a shelve. 


That seems to be a bug in the underlying pickling mechanism: 


>>> from array import array 
>>> from cPickle import loads, dumps 
>>> loads(dumps(array("f", [1]))) 
array('f', [1.0]) 
>>> loads(dumps(array("f"))) 


Traceback (most recent call last): 
  File "<stdin>", line 1, in <module> 
TypeError: ("'NoneType' object is not iterable", <type 'array.array'>, ('f', 
None)) 

Please submit a bug report. 


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1693079&group_id=5470

From noreply at sourceforge.net  Mon Apr  2 18:59:31 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Mon, 02 Apr 2007 09:59:31 -0700
Subject: [ python-Bugs-1693079 ] Can't save empty array in shelve
Message-ID: <E1HYPsR-0002v6-Uk@sc8-sf-web21.sourceforge.net>

Bugs item #1693079, was opened at 2007-04-02 11:19
Message generated for change (Settings changed) made by rhettinger
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1693079&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: IWL (iwl)
>Assigned to: Raymond Hettinger (rhettinger)
Summary: Can't save empty array in shelve

Initial Comment:

iwl wrote: 
> I've tryed to save some data containing empty arrays (array('f')) in a 
> shelve. 
> It looks like the shelve has some problems with empty arrays, get 
> allways: 
> TypeError: ("'NoneType' object is not iterable", <type 'array.array'>, 
> ('f', None))- 
> Messages when dealing with the readed back shelve. 
> Seems like I have to avoid empty arrays in a shelve. 


That seems to be a bug in the underlying pickling mechanism: 


>>> from array import array 
>>> from cPickle import loads, dumps 
>>> loads(dumps(array("f", [1]))) 
array('f', [1.0]) 
>>> loads(dumps(array("f"))) 


Traceback (most recent call last): 
  File "<stdin>", line 1, in <module> 
TypeError: ("'NoneType' object is not iterable", <type 'array.array'>, ('f', 
None)) 

Please submit a bug report. 


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1693079&group_id=5470

From noreply at sourceforge.net  Mon Apr  2 19:29:48 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Mon, 02 Apr 2007 10:29:48 -0700
Subject: [ python-Bugs-1693079 ] Can't save empty array in shelve
Message-ID: <E1HYQLk-0005yd-QB@sc8-sf-web24.sourceforge.net>

Bugs item #1693079, was opened at 2007-04-02 11:19
Message generated for change (Comment added) made by rhettinger
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1693079&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.5
>Status: Closed
>Resolution: Fixed
Priority: 5
Private: No
Submitted By: IWL (iwl)
Assigned to: Raymond Hettinger (rhettinger)
Summary: Can't save empty array in shelve

Initial Comment:

iwl wrote: 
> I've tryed to save some data containing empty arrays (array('f')) in a 
> shelve. 
> It looks like the shelve has some problems with empty arrays, get 
> allways: 
> TypeError: ("'NoneType' object is not iterable", <type 'array.array'>, 
> ('f', None))- 
> Messages when dealing with the readed back shelve. 
> Seems like I have to avoid empty arrays in a shelve. 


That seems to be a bug in the underlying pickling mechanism: 


>>> from array import array 
>>> from cPickle import loads, dumps 
>>> loads(dumps(array("f", [1]))) 
array('f', [1.0]) 
>>> loads(dumps(array("f"))) 


Traceback (most recent call last): 
  File "<stdin>", line 1, in <module> 
TypeError: ("'NoneType' object is not iterable", <type 'array.array'>, ('f', 
None)) 

Please submit a bug report. 


----------------------------------------------------------------------

>Comment By: Raymond Hettinger (rhettinger)
Date: 2007-04-02 12:29

Message:
Logged In: YES 
user_id=80475
Originator: NO

Fixed.  See versions 54657 and 54658.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1693079&group_id=5470

From noreply at sourceforge.net  Mon Apr  2 21:02:54 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Mon, 02 Apr 2007 12:02:54 -0700
Subject: [ python-Bugs-1678380 ] 0.0 and -0.0 identified,
	with surprising results
Message-ID: <E1HYRnq-00046L-KV@sc8-sf-web23.sourceforge.net>

Bugs item #1678380, was opened at 2007-03-11 12:16
Message generated for change (Comment added) made by rhettinger
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1678380&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Interpreter Core
Group: None
Status: Open
>Resolution: Accepted
Priority: 5
Private: No
Submitted By: Mark Dickinson (marketdickinson)
>Assigned to: Alex Martelli (aleax)
Summary: 0.0 and -0.0 identified, with surprising results

Initial Comment:
The identification of -0.0 and 0.0 in scripts leads to some surprising
results.  In particular, code that behaves one way in the interpreter can behave differently in a script.  For example:

Python 2.6a0 (trunk:54183M, Mar  6 2007, 20:16:00) 
[GCC 4.0.1 (Apple Computer, Inc. build 5367)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from math import atan2;
>>> x = -0.
>>> y = 0.
>>> print atan2(y, -1.)
3.14159265359

But:

>>> exec("from math import atan2; x = -0.; y = 0.; print atan2(y, -1.)")
-3.14159265359

A simpler example:

>>> x, y = -0., 0.
>>> x, y
(-0.0, -0.0)
>>> id(x) == id(y)
True

But:

>>> x = -0.
>>> y = 0.
>>> x, y
(-0.0, 0.0)

This occurs both on SuSE Linux 9.3/i686 and on OS X 10.4.8/PowerPC.


----------------------------------------------------------------------

>Comment By: Raymond Hettinger (rhettinger)
Date: 2007-04-02 14:02

Message:
Logged In: YES 
user_id=80475
Originator: NO

The distinction between +0.0 and -0.0 was important to Tim for get the
branch cuts to work correctly.  That was the reason for the special-casing
in earlier versions in pre-ast versions of compile.c and in the peephole
optimizer.

Alex's patch looks correct.  It should go into Py2.5.1.

----------------------------------------------------------------------

Comment By: Alex Martelli (aleax)
Date: 2007-03-12 09:57

Message:
Logged In: YES 
user_id=60314
Originator: NO

and yet, peephole.c does specialcase -0 (with a comment and all!-)
avoiding constant-folding optimization for it -- making things work fine in
Python 2.4.x for high enough x -- it's just that peephole.c's efforts are
defeated by a similar optimization applied without specialcase checking in
ast.c in Python 2.5.  This is inconsistent: it makes no sense to take SOME
of the precautions needed to avoid an issue but not ALL of them, since this
makes the issue appear anyway, so those precautions that ARE taken are
there for no purpose (except a miniscule slowdown and enlargement of the
interpreter:-).  Either we save those few lines of code in peephole.c or
add the few lines to ast.c that I suggest in my patch 1678668 -- the
current situation makes no sense.

----------------------------------------------------------------------

Comment By: Martin v. L?wis (loewis)
Date: 2007-03-12 04:04

Message:
Logged In: YES 
user_id=21627
Originator: NO

I also expressed myself badly. I not only meant that programs should not
rely on +0 and -0 being different things across platforms, but that they
should also not rely on them being either always different or always the
same in a single program. If Python would randomly chose to interpret +0 as
-0, or would do so for every third occurence, I still couldn't see a
problem.

----------------------------------------------------------------------

Comment By: Alex Martelli (aleax)
Date: 2007-03-11 20:54

Message:
Logged In: YES 
user_id=60314
Originator: NO

Oops,sorry, I meant patch 1678668 (copy-and-pasted the wrong ID:-).

Alex


----------------------------------------------------------------------

Comment By: Alex Martelli (aleax)
Date: 2007-03-11 20:51

Message:
Logged In: YES 
user_id=60314
Originator: NO

Also see my patch 1678380 which fixes this bug (and checks for it in a new
unittest).


----------------------------------------------------------------------

Comment By: Mark Dickinson (marketdickinson)
Date: 2007-03-11 20:15

Message:
Logged In: YES 
user_id=703403
Originator: YES

I expressed myself badly.  I apologise.  This really isn't about +0. and
-0. 
being different, or not.  I'm perfectly comfortable with the idea that +0.

and -0. may or may not be distinguishable on any given platform.

The surprise is the other way around: two *identical* calls to atan(0.,
-1.)
(or to repr(0.) for that matter) give *different* results, depending
solely 
on whether a -0. literal has appeared earlier on in the code unit being
compiled.

So if the first float zero literal encountered in a source file just
happens to be a
-0. rather than a 0., the meaning of str(0.) later on suddenly becomes
"-0.0"
rather than "0.0".  I'd like to be able to rely on str(0.) meaning "0.0"
without
having to worry about whether there might be a -0. literal appearing in
some
faraway and otherwise completely irrelevant portion of the file.


----------------------------------------------------------------------

Comment By: Martin v. L?wis (loewis)
Date: 2007-03-11 17:27

Message:
Logged In: YES 
user_id=21627
Originator: NO

marketdickinson, you should ask this question (is this really acceptable)
on python-dev. I find it perfectly acceptable. No program should rely on -0
and +0 being two different things (and thus also not relying on atan2
giving two different results).

----------------------------------------------------------------------

Comment By: Gabriel Genellina (gagenellina)
Date: 2007-03-11 15:10

Message:
Logged In: YES 
user_id=479790
Originator: NO

It appears to be a problem in the way compile.c handles literals.
See http://mail.python.org/pipermail/python-list/2007-March/430302.html

----------------------------------------------------------------------

Comment By: Mark Dickinson (marketdickinson)
Date: 2007-03-11 14:21

Message:
Logged In: YES 
user_id=703403
Originator: YES

May I beg for reconsideration.
Is the following really considered acceptable?

>>> x = -0.; atan2(0., -1)
-3.1415926535897931
>>> x = -(0.); atan2(0., -1)
3.1415926535897931
>>> atan2(0., -1)
3.1415926535897931

A single x = -0. at the start of a script can have
side effects several hundred lines later, even when
the variable x is never referred to again.  I guess
the advice should be:  "To avoid surprises, -0. should
never appear in any script."

----------------------------------------------------------------------

Comment By: Martin v. L?wis (loewis)
Date: 2007-03-11 13:35

Message:
Logged In: YES 
user_id=21627
Originator: NO

This is not a bug, at least not one that will be fixed. Details of the
floating-point can vary across platforms, and they may behave in suprising
ways in various contexts. Users shouldn't rely on Python differentiating
between -0 and +0.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1678380&group_id=5470

From noreply at sourceforge.net  Mon Apr  2 21:03:18 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Mon, 02 Apr 2007 12:03:18 -0700
Subject: [ python-Bugs-1678380 ] 0.0 and -0.0 identified,
	with surprising results
Message-ID: <E1HYRoE-0004CL-Dg@sc8-sf-web23.sourceforge.net>

Bugs item #1678380, was opened at 2007-03-11 12:16
Message generated for change (Settings changed) made by rhettinger
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1678380&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Interpreter Core
Group: None
Status: Open
Resolution: Accepted
>Priority: 6
Private: No
Submitted By: Mark Dickinson (marketdickinson)
Assigned to: Alex Martelli (aleax)
Summary: 0.0 and -0.0 identified, with surprising results

Initial Comment:
The identification of -0.0 and 0.0 in scripts leads to some surprising
results.  In particular, code that behaves one way in the interpreter can behave differently in a script.  For example:

Python 2.6a0 (trunk:54183M, Mar  6 2007, 20:16:00) 
[GCC 4.0.1 (Apple Computer, Inc. build 5367)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from math import atan2;
>>> x = -0.
>>> y = 0.
>>> print atan2(y, -1.)
3.14159265359

But:

>>> exec("from math import atan2; x = -0.; y = 0.; print atan2(y, -1.)")
-3.14159265359

A simpler example:

>>> x, y = -0., 0.
>>> x, y
(-0.0, -0.0)
>>> id(x) == id(y)
True

But:

>>> x = -0.
>>> y = 0.
>>> x, y
(-0.0, 0.0)

This occurs both on SuSE Linux 9.3/i686 and on OS X 10.4.8/PowerPC.


----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2007-04-02 14:02

Message:
Logged In: YES 
user_id=80475
Originator: NO

The distinction between +0.0 and -0.0 was important to Tim for get the
branch cuts to work correctly.  That was the reason for the special-casing
in earlier versions in pre-ast versions of compile.c and in the peephole
optimizer.

Alex's patch looks correct.  It should go into Py2.5.1.

----------------------------------------------------------------------

Comment By: Alex Martelli (aleax)
Date: 2007-03-12 09:57

Message:
Logged In: YES 
user_id=60314
Originator: NO

and yet, peephole.c does specialcase -0 (with a comment and all!-)
avoiding constant-folding optimization for it -- making things work fine in
Python 2.4.x for high enough x -- it's just that peephole.c's efforts are
defeated by a similar optimization applied without specialcase checking in
ast.c in Python 2.5.  This is inconsistent: it makes no sense to take SOME
of the precautions needed to avoid an issue but not ALL of them, since this
makes the issue appear anyway, so those precautions that ARE taken are
there for no purpose (except a miniscule slowdown and enlargement of the
interpreter:-).  Either we save those few lines of code in peephole.c or
add the few lines to ast.c that I suggest in my patch 1678668 -- the
current situation makes no sense.

----------------------------------------------------------------------

Comment By: Martin v. L?wis (loewis)
Date: 2007-03-12 04:04

Message:
Logged In: YES 
user_id=21627
Originator: NO

I also expressed myself badly. I not only meant that programs should not
rely on +0 and -0 being different things across platforms, but that they
should also not rely on them being either always different or always the
same in a single program. If Python would randomly chose to interpret +0 as
-0, or would do so for every third occurence, I still couldn't see a
problem.

----------------------------------------------------------------------

Comment By: Alex Martelli (aleax)
Date: 2007-03-11 20:54

Message:
Logged In: YES 
user_id=60314
Originator: NO

Oops,sorry, I meant patch 1678668 (copy-and-pasted the wrong ID:-).

Alex


----------------------------------------------------------------------

Comment By: Alex Martelli (aleax)
Date: 2007-03-11 20:51

Message:
Logged In: YES 
user_id=60314
Originator: NO

Also see my patch 1678380 which fixes this bug (and checks for it in a new
unittest).


----------------------------------------------------------------------

Comment By: Mark Dickinson (marketdickinson)
Date: 2007-03-11 20:15

Message:
Logged In: YES 
user_id=703403
Originator: YES

I expressed myself badly.  I apologise.  This really isn't about +0. and
-0. 
being different, or not.  I'm perfectly comfortable with the idea that +0.

and -0. may or may not be distinguishable on any given platform.

The surprise is the other way around: two *identical* calls to atan(0.,
-1.)
(or to repr(0.) for that matter) give *different* results, depending
solely 
on whether a -0. literal has appeared earlier on in the code unit being
compiled.

So if the first float zero literal encountered in a source file just
happens to be a
-0. rather than a 0., the meaning of str(0.) later on suddenly becomes
"-0.0"
rather than "0.0".  I'd like to be able to rely on str(0.) meaning "0.0"
without
having to worry about whether there might be a -0. literal appearing in
some
faraway and otherwise completely irrelevant portion of the file.


----------------------------------------------------------------------

Comment By: Martin v. L?wis (loewis)
Date: 2007-03-11 17:27

Message:
Logged In: YES 
user_id=21627
Originator: NO

marketdickinson, you should ask this question (is this really acceptable)
on python-dev. I find it perfectly acceptable. No program should rely on -0
and +0 being two different things (and thus also not relying on atan2
giving two different results).

----------------------------------------------------------------------

Comment By: Gabriel Genellina (gagenellina)
Date: 2007-03-11 15:10

Message:
Logged In: YES 
user_id=479790
Originator: NO

It appears to be a problem in the way compile.c handles literals.
See http://mail.python.org/pipermail/python-list/2007-March/430302.html

----------------------------------------------------------------------

Comment By: Mark Dickinson (marketdickinson)
Date: 2007-03-11 14:21

Message:
Logged In: YES 
user_id=703403
Originator: YES

May I beg for reconsideration.
Is the following really considered acceptable?

>>> x = -0.; atan2(0., -1)
-3.1415926535897931
>>> x = -(0.); atan2(0., -1)
3.1415926535897931
>>> atan2(0., -1)
3.1415926535897931

A single x = -0. at the start of a script can have
side effects several hundred lines later, even when
the variable x is never referred to again.  I guess
the advice should be:  "To avoid surprises, -0. should
never appear in any script."

----------------------------------------------------------------------

Comment By: Martin v. L?wis (loewis)
Date: 2007-03-11 13:35

Message:
Logged In: YES 
user_id=21627
Originator: NO

This is not a bug, at least not one that will be fixed. Details of the
floating-point can vary across platforms, and they may behave in suprising
ways in various contexts. Users shouldn't rely on Python differentiating
between -0 and +0.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1678380&group_id=5470

From noreply at sourceforge.net  Mon Apr  2 21:11:31 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Mon, 02 Apr 2007 12:11:31 -0700
Subject: [ python-Bugs-1671137 ] slice obj with no start index is 0 instead of
	None sometimes
Message-ID: <E1HYRwB-00056Y-AH@sc8-sf-web23.sourceforge.net>

Bugs item #1671137, was opened at 2007-02-28 13:58
Message generated for change (Comment added) made by rhettinger
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1671137&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Interpreter Core
Group: Python 2.6
Status: Open
>Resolution: Wont Fix
Priority: 5
Private: No
Submitted By: Mike Verdone (jyzude)
Assigned to: Nobody/Anonymous (nobody)
Summary: slice obj with no start index is 0 instead of None sometimes

Initial Comment:
Slice objects returned by the slice ":42" return different slice objects depending on whether the entire slice operation is simple or extended. This bit of code explains it best:

class SliceBug:
    def __getitem__(self, sliceArg):
        print sliceArg

s = SliceBug()

s[:42]
s[:42,]


s[:42] produces slice(0, 42, None)
s[:42,] produces (slice(None, 42, None),)

Note that this bug only occurs on classes that do no inherit from object ("old style" classes). If you change the class to make it inherit from "object" both slices have None as their start index. Oddly enough in Python 3000 it still only happens on "old style" classes even though supposedly they are the same as new style classes. I have also reproduced the bug in Python 2.6, 2.4, 2.3, and 2.2. Seems to be a long standing bug/feature.

----------------------------------------------------------------------

>Comment By: Raymond Hettinger (rhettinger)
Date: 2007-04-02 14:11

Message:
Logged In: YES 
user_id=80475
Originator: NO

Fixing this small inconsistency has very little upside but runs the risk
of breaking code that has been running fine for years.  I think we should
punt.  If you agree, please close this bug report.

----------------------------------------------------------------------

Comment By: Mike Verdone (jyzude)
Date: 2007-02-28 14:04

Message:
Logged In: YES 
user_id=584997
Originator: YES

Correction! This is fixed in Python 3000. I just have too many windows
open and too many branches checked out.
File Added: slicebug.py

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1671137&group_id=5470

From noreply at sourceforge.net  Tue Apr  3 00:17:00 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Mon, 02 Apr 2007 15:17:00 -0700
Subject: [ python-Bugs-1691411 ] Duplicate "preferences" menu item/Tk Aqua
	8.4.14
Message-ID: <E1HYUpg-00042V-NM@sc8-sf-web21.sourceforge.net>

Bugs item #1691411, was opened at 2007-03-30 11:12
Message generated for change (Comment added) made by wordtech
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1691411&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: IDLE
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Kevin Walzer (wordtech)
Assigned to: Nobody/Anonymous (nobody)
Summary: Duplicate "preferences" menu item/Tk Aqua 8.4.14

Initial Comment:
Version 8.4.14 of  Tcl/Tk Aqua (for OS X) hard-codes a "Preferences" item in the Apple menu. As a result, IDLE now has two "Preferences" items--the active one that is coded by default, and an inactive item that is hard-coded by Tk. 

The way to work around this is to check the version of Tk that is present, in this fashion:

tkversion=Tkinter.Tk().tk.eval('info patchlevel') 

If it is 8.4.14 or greater, then you should bind IDLE's "preferences" dialog to the hard-coded menu item in Tk (and remove the "preferences" menu entry that comes with IDLE). It can be done in this fashion:

 Tkinter.Tk().createcommand('::tk::mac::ShowPreferences', IDLEpreferencesfunction) ##substitute the correction function name here

If tkversion <= '8.4.13', leave things as they are.

I have tried to put together a patch for this, but I'm not sure where the correct place to add this code is. I've looked at Bindings.py and macosxSupport.py, but it doesn't work as expected (in macosxSupport.py, it simply ignores the code; in Bindings.py, it removes IDLE's preferences item, but doesn't activate the hard-coded one). So I'm leaving this to someone with a greater knowledge of IDLE's internal structure to apply. 

----------------------------------------------------------------------

>Comment By: Kevin Walzer (wordtech)
Date: 2007-04-02 17:17

Message:
Logged In: YES 
user_id=794827
Originator: YES

Patch # 1693258 submitted April 2, 2007 to address this problem.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1691411&group_id=5470

From noreply at sourceforge.net  Tue Apr  3 00:54:42 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Mon, 02 Apr 2007 15:54:42 -0700
Subject: [ python-Bugs-1647541 ] SystemError with re.match(array)
Message-ID: <E1HYVQA-00086Z-LG@sc8-sf-web21.sourceforge.net>

Bugs item #1647541, was opened at 2007-01-29 19:04
Message generated for change (Comment added) made by rhettinger
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1647541&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Extension Modules
>Group: Python 2.5
>Status: Closed
>Resolution: Fixed
Priority: 4
Private: No
Submitted By: Armin Rigo (arigo)
Assigned to: Armin Rigo (arigo)
Summary: SystemError with re.match(array)

Initial Comment:
An small issue which I guess is to be found in
the implementation of the buffer interface
for zero-length arrays:

>>> a = array.array("c")
>>> r = re.compile("bla")
>>> r.match(a)
SystemError: error return without exception set

----------------------------------------------------------------------

>Comment By: Raymond Hettinger (rhettinger)
Date: 2007-04-02 17:54

Message:
Logged In: YES 
user_id=80475
Originator: NO

Fixed in versions 54662 and 54663.
Now returns a buffer of length zero other than NULL.


----------------------------------------------------------------------

Comment By: Armin Rigo (arigo)
Date: 2007-01-30 08:37

Message:
Logged In: YES 
user_id=4771
Originator: YES

It seems to me that an empty array should be
equivalent to an empty string.  Accessing it as a
buffer should return a buffer of length 0, not
raise ValueError.

In all cases, the fix in _sre.c is sensible.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2007-01-30 00:21

Message:
Logged In: YES 
user_id=33168
Originator: NO

Armin, what do you think of the attached patch?
File Added: empty-array.diff

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1647541&group_id=5470

From noreply at sourceforge.net  Tue Apr  3 01:12:09 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Mon, 02 Apr 2007 16:12:09 -0700
Subject: [ python-Bugs-1683368 ] object.__init__ shouldn't allow args/kwds
Message-ID: <E1HYVh3-0001Ln-Nh@sc8-sf-web22.sourceforge.net>

Bugs item #1683368, was opened at 2007-03-18 22:32
Message generated for change (Comment added) made by rhettinger
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1683368&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Interpreter Core
Group: Python 3000
Status: Open
Resolution: Fixed
Priority: 5
Private: No
Submitted By: Blake Ross (blakeross)
Assigned to: Guido van Rossum (gvanrossum)
Summary: object.__init__ shouldn't allow args/kwds

Initial Comment:
object.__init__ currently allows any amount of args and keywords even though they're ignored. This is inconsistent with other built-ins, like list, that are stricter about what they'll accept. It's also inconsistent with object.__new__, which does throw if any are provided (if the default __init__ is to be used).

To reproduce:

object.__init__(object(), foo, bar=3)

This is a slight irritation when using cooperative super calling. I'd like each class' __init__ to cherry-pick keyword params it accepts and pass the remaining ones up the chain, but right now I can't rely on object.__init__ to throw if there are remaining keywords by that point.

----------------------------------------------------------------------

>Comment By: Raymond Hettinger (rhettinger)
Date: 2007-04-02 18:12

Message:
Logged In: YES 
user_id=80475
Originator: NO

FWIW, this change will be somewhat pervasive and will affect anything
inheriting object.__init__ including immutable builtins (like tuple, float,
and frozenset) as well as user-defined new-style classes that do not define
their own __init__ method (perhaps using new instead).  Here are the
warnings being thrown-off by the current test suite:

/py26/Lib/test/test_array.py:731: DeprecationWarning: object.__init__()
takes no parameters
  array.array.__init__(self, 'c', s)

/py26/Lib/copy_reg.py:51: DeprecationWarning: object.__init__() takes no
parameters
  base.__init__(obj, state)

/py26/Lib/test/test_descr.py:2308: DeprecationWarning: object.__init__()
takes no parameters
  float.__init__(self, value)


----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2007-03-23 00:01

Message:
Logged In: YES 
user_id=6380
Originator: NO

Committed revision 54539.

The committed version issues warnings rather than errors when both methods
are overridden, to avoid too much breakage.

The string.py change was necessary to avoid spurious warnings (with no
module/lineno!) and breakage of test_subprocess.py.  Something fishy's
going on -- is string.Template() used by the warnings module or by
site.py???

I'm leaving this bug open but changing the category to Py3k so remind me
it needs to be merged and then changed there.

----------------------------------------------------------------------

Comment By: Adam Olsen (rhamphoryncus)
Date: 2007-03-21 23:46

Message:
Logged In: YES 
user_id=12364
Originator: NO

>> I think it would also help if calling a method via super() didn't
allow
>> positional arguments.
>
> That's absurd, *except* for __init__(), where it could make sense
> depending on the style of cooperation used.  But not enough to enforce
this
> in the language; in Py3k you will be able to enforce this on a
per-class
> basis.

The vast majority of "positional" arguments can also be given via name. 
The rare exceptions (primarily C functions) may not cooperate well anyway,
so you're trading a relatively obscure limitation for better error
detection.

Perhaps not that important though, since it could be taught as bad style
unless absolutely needed.


>> Having two mixins with the same method name and
>> without a common parent class is just not sane.
>
> Right.  This is a cornerstone of cooperative multiple inheritance that
> sometimes seems to be forgotten; there is a big difference between
defining
> a method and extending a method, and only extending methods can make
super
> calls.
>
> The __init__ case is an exception, because there's no requirement that
a
> subclass have a signature compatible with the superclass (if you don't
get
> this, read up on constructors in C++ or Java).

I understand the desire for it to be an exception, I fail to see how it
actually is one.  The namespace/signature conflicts exist just the same.

The only way I can see to handle incompatible signatures is to add a flag
that says "I am the *ONLY* class allowed to subclass X" (triggering an
error if violated), have super() entirely bypass it, and then call
X.__init__() directly.  Even that doesn't handle X's superclasses being
subclassed more than once, and it looks pretty complicated/obscure anyway.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2007-03-21 23:28

Message:
Logged In: YES 
user_id=6380
Originator: NO

> I think it would also help if calling a method via super() didn't allow
> positional arguments.

That's absurd, *except* for __init__(), where it could make sense
depending on the style of cooperation used.  But not enough to enforce this
in the language; in Py3k you will be able to enforce this on a per-class
basis.

> Having two mixins with the same method name and
> without a common parent class is just not sane.

Right.  This is a cornerstone of cooperative multiple inheritance that
sometimes seems to be forgotten; there is a big difference between defining
a method and extending a method, and only extending methods can make super
calls.

The __init__ case is an exception, because there's no requirement that a
subclass have a signature compatible with the superclass (if you don't get
this, read up on constructors in C++ or Java).

----------------------------------------------------------------------

Comment By: Adam Olsen (rhamphoryncus)
Date: 2007-03-21 19:44

Message:
Logged In: YES 
user_id=12364
Originator: NO

I think the avoidance of super() is largely *because* of this kind of
leniency.  Consider this snippet on python 2.3:

class Base(object):
    def __init__(self, foo=None, *args, **kwargs):
        super(Base, self).__init__(foo, *args, **kwargs)
Base(foo='bar')
Base('bar', 42)
Base('bar', 42, x=7)

All pass silently, no error checking.  Now consider a python 3.0 version,
with a strict object.__init__:

class Base:
    def __init__(self, foo=None, *, y='hi', **kwargs):
        super(Base, self).__init__(**kwargs)
Base(foo='bar')  # Valid, accepted
Base('bar', 42)  # Raises exception
Base('bar', x=7)  # Raises exception

The error checking added by this bug/patch and the error checking added by
PEP 3102 (keyword-only arguments) make super a lot more sane.

I think it would also help if calling a method via super() didn't allow
positional arguments.  If the base class's arguments can't be given as
keyword args then you probably should call it explicitly, rather than
relying on super()'s MRO.

I was also going to suggest super() should automagically create empty
methods your parent classes don't have, but then I realized you really
should have a base class that asserts the lack of such an automagic
method:

class Base(object):
    def mymethod(self, myonlyarg='hello world'):
        assert not hasattr(super(Base, self), 'mymethod')

By the time you reach this base class you will have stripped off any extra
arguments that your subclasses added, leaving you with nothing to pass up
(and nothing to pass to).  Having two mixins with the same method name and
without a common parent class is just not sane.

----------------------------------------------------------------------

Comment By: Terry J. Reedy (tjreedy)
Date: 2007-03-21 18:16

Message:
Logged In: YES 
user_id=593130
Originator: NO

I ask myself, what should I expect from the documentation...

>>> object.__init__.__doc__
'x.__init__(...) initializes x; see x.__class__.__doc__ for signature'
>>> object.__class__
<type 'type'>
>>> type.__doc__
"type(object) -> the object's type\ntype(name, bases, dict) -> a new
type"

and I still don't know ;-).

----------------------------------------------------------------------

Comment By: Blake Ross (blakeross)
Date: 2007-03-21 17:48

Message:
Logged In: YES 
user_id=1747060
Originator: YES

Holding the strict version for 3 makes sense to me. Let me know if you
need anything more on my end... thanks for the fast turnaround.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2007-03-21 17:42

Message:
Logged In: YES 
user_id=6380
Originator: NO

Well, but since it's been like this for a long time, I don't want to
gratuitously break code. At least not in 2.6. So I'm rejecting the stricter
patch for 2.6. (However, if you want to submit patches that would fix these
breakages anyway, be my guest.)

----------------------------------------------------------------------

Comment By: Blake Ross (blakeross)
Date: 2007-03-21 17:03

Message:
Logged In: YES 
user_id=1747060
Originator: YES

Looks good. I skimmed briefly the tests you mentioned. The issue with
test_array appears to be exactly the kind of bug this is intended to
identify: it calls array.__init__(...), but array doesn't have its own
initializer, so object's is used.

I'd guess that the others are failing due to whatever the problem with
pickling is (test_descr uses pickling). I haven't looked into that yet.

I'm sure cooperative super calling of __init__ isn't all that common (it
seems like the mechanism itself isn't used much yet, and may not be until
it's done via keyword) but it doesn't seem like such a bad practice,
especially when mixins are in the picture. There doesn't seem to be a great
alternative.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2007-03-20 16:24

Message:
Logged In: YES 
user_id=6380
Originator: NO

I should mention that if we can't get the strict version of this in 2.6,
we should be able to get it into 3.0.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2007-03-20 15:54

Message:
Logged In: YES 
user_id=6380
Originator: NO

Here's a stricter version. Unfortunately it breaks a couple of standard
modules; this is a confirmation of my doubts whether the style of
cooperative super calling of __init__ that you use is really the most
common or "best practice".

So far I have only fixed string.py (which would otherwise prevent
extensions from being built); I haven't looked into why the other tests
fail: test_array, test_cpickle, test_descr, test_pickle (and maybe more?).

My conclusion: this would probably break too much code to be worth it.  So
I'll have to revert to the previous version.  But anyway, here it is for
your perusal.
File Added: new_init_strict.patch

----------------------------------------------------------------------

Comment By: Blake Ross (blakeross)
Date: 2007-03-19 20:27

Message:
Logged In: YES 
user_id=1747060
Originator: YES

I think making the check more rigid is a good idea, since this should
throw:

class a(object):
   def __init__(self, foo):
       super(a, self).__init__(foo)
   def __new__(cls, foo):
       return object.__new__(cls)
a(1)

(minor typo in the patch: "solution it" -> "solution is")

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2007-03-19 18:35

Message:
Logged In: YES 
user_id=6380
Originator: NO

This smells enough like a new feature that it couldn't go into 2.5.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2007-03-19 18:31

Message:
Logged In: YES 
user_id=6380
Originator: NO

Attached is a patch that implements this proposal, adding copious
commentary.  It doesn't seem to break anything in the test suite.

I wonder if we should even make the check more rigid: check the argument
list if either the current method *is* overridden or the other one *is not*
overridden.  This would make super calls check the arguments even if the
other method is overridden.  What do you think?
File Added: new_init.patch

----------------------------------------------------------------------

Comment By: Blake Ross (blakeross)
Date: 2007-03-19 17:45

Message:
Logged In: YES 
user_id=1747060
Originator: YES

Makes sense. I don't think we can ever be completely correct here since
we're inferring intent from the presence of __init__/__new__ that's liable
to be wrong in some cases, but it's likely correct often enough that it's
worth doing.

If I understand correctly, we want to be more forgiving iff one of the two
methods is used, so it seems like we should be complaining if both are used
*or* if neither is used. After all, I could add a __new__ to my coop use
case and I'd still want object to complain. If that's the case, both
object_new and object_init should be complaining if ((tp->tp_new ==
object_new && tp->tp_init == object_init) || (tp->tp_new != object_new &&
tp->tp_init != object_init)).

Of course, for the paranoid, there's always the risk that __new__ will
modify these class functions and change the outcome :) For instance, if a
class had a __new__ and no __init__ and its __new__ changed __new__ back to
object.__new__, object_init on that run would be fooled into thinking it's
using the defaults for both and would complain. I think this could only be
fixed in type_call, which is rather ugly...but then, this *is* a special
case of the "call __init__ after __new__" behavior, and we're trying to
solve it within the methods themselves. Perhaps this last point is academic
enough to be ignored...I don't know why anyone would do this, although the
language makes it possible.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2007-03-19 15:01

Message:
Logged In: YES 
user_id=6380
Originator: NO

I'll try to explain why I did it this way.  I was considering the single
inheritance case implementing an Immutable object, which overrides __new__
but has no need to override __init__ (since it's too late to do anything in
__init__ for an Immutable object).  Since the __init__ still gets called it
would be annoying to have to override it just to make the error go away if
there was a check in __init__.  The other case is overriding __init__
without overriding __new__, which is the most common way of doing Mutable
objects; here you wouldn't want __new__ to complain about extra args.  So
the only time when you'd want complaints is if both __new__ and __init__
are the defaults, in which case it doesn't really matter whether you
implement this in __init__ or in __new__, so I arbitrarily chose __new__.

I wasn't thinking of your use case at the time though (cooperative super
calls to __init__, which still isn't something I engage in on a day-to-day
basis).  I wonder if the right thing to do wouldn't be to implement the
same check both in __init__ and in __new__.

Am I makign sense?

----------------------------------------------------------------------

Comment By: Georg Brandl (gbrandl)
Date: 2007-03-19 03:56

Message:
Logged In: YES 
user_id=849994
Originator: NO

I don't really understand either why object_new() checks the arguments,
not object_init():

"""
static int
object_init(PyObject *self, PyObject *args, PyObject *kwds)
{
	return 0;
}

/* If we don't have a tp_new for a new-style class, new will use this
one.
   Therefore this should take no arguments/keywords.  However, this new
may
   also be inherited by objects that define a tp_init but no tp_new. 
These
   objects WILL pass argumets to tp_new, because it gets the same args as
   tp_init.  So only allow arguments if we aren't using the default init,
in
   which case we expect init to handle argument parsing. */
static PyObject *
object_new(PyTypeObject *type, PyObject *args, PyObject *kwds)
{
	if (type->tp_init == object_init && (PyTuple_GET_SIZE(args) ||
	     (kwds && PyDict_Check(kwds) && PyDict_Size(kwds)))) {
		PyErr_SetString(PyExc_TypeError,
				"default __new__ takes no parameters");
		return NULL;
	}
	return type->tp_alloc(type, 0);
}
"""

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1683368&group_id=5470

From noreply at sourceforge.net  Tue Apr  3 01:55:58 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Mon, 02 Apr 2007 16:55:58 -0700
Subject: [ python-Bugs-1683368 ] object.__init__ shouldn't allow args/kwds
Message-ID: <E1HYWNS-0004zJ-0X@sc8-sf-web24.sourceforge.net>

Bugs item #1683368, was opened at 2007-03-18 23:32
Message generated for change (Comment added) made by gvanrossum
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1683368&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Interpreter Core
Group: Python 3000
Status: Open
Resolution: Fixed
Priority: 5
Private: No
Submitted By: Blake Ross (blakeross)
Assigned to: Guido van Rossum (gvanrossum)
Summary: object.__init__ shouldn't allow args/kwds

Initial Comment:
object.__init__ currently allows any amount of args and keywords even though they're ignored. This is inconsistent with other built-ins, like list, that are stricter about what they'll accept. It's also inconsistent with object.__new__, which does throw if any are provided (if the default __init__ is to be used).

To reproduce:

object.__init__(object(), foo, bar=3)

This is a slight irritation when using cooperative super calling. I'd like each class' __init__ to cherry-pick keyword params it accepts and pass the remaining ones up the chain, but right now I can't rely on object.__init__ to throw if there are remaining keywords by that point.

----------------------------------------------------------------------

>Comment By: Guido van Rossum (gvanrossum)
Date: 2007-04-02 19:55

Message:
Logged In: YES 
user_id=6380
Originator: NO

That's one way of looking at it. You could also say that it found two
legitimate problems:

- since array doesn't define __init__() there's no point in calling it

- similarly, float doesn't define __init__()

The copy_reg warning is more subtle, and needs a work-around.  I've
checked in all three fixes.

----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2007-04-02 19:12

Message:
Logged In: YES 
user_id=80475
Originator: NO

FWIW, this change will be somewhat pervasive and will affect anything
inheriting object.__init__ including immutable builtins (like tuple, float,
and frozenset) as well as user-defined new-style classes that do not define
their own __init__ method (perhaps using new instead).  Here are the
warnings being thrown-off by the current test suite:

/py26/Lib/test/test_array.py:731: DeprecationWarning: object.__init__()
takes no parameters
  array.array.__init__(self, 'c', s)

/py26/Lib/copy_reg.py:51: DeprecationWarning: object.__init__() takes no
parameters
  base.__init__(obj, state)

/py26/Lib/test/test_descr.py:2308: DeprecationWarning: object.__init__()
takes no parameters
  float.__init__(self, value)


----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2007-03-23 01:01

Message:
Logged In: YES 
user_id=6380
Originator: NO

Committed revision 54539.

The committed version issues warnings rather than errors when both methods
are overridden, to avoid too much breakage.

The string.py change was necessary to avoid spurious warnings (with no
module/lineno!) and breakage of test_subprocess.py.  Something fishy's
going on -- is string.Template() used by the warnings module or by
site.py???

I'm leaving this bug open but changing the category to Py3k so remind me
it needs to be merged and then changed there.

----------------------------------------------------------------------

Comment By: Adam Olsen (rhamphoryncus)
Date: 2007-03-22 00:46

Message:
Logged In: YES 
user_id=12364
Originator: NO

>> I think it would also help if calling a method via super() didn't
allow
>> positional arguments.
>
> That's absurd, *except* for __init__(), where it could make sense
> depending on the style of cooperation used.  But not enough to enforce
this
> in the language; in Py3k you will be able to enforce this on a
per-class
> basis.

The vast majority of "positional" arguments can also be given via name. 
The rare exceptions (primarily C functions) may not cooperate well anyway,
so you're trading a relatively obscure limitation for better error
detection.

Perhaps not that important though, since it could be taught as bad style
unless absolutely needed.


>> Having two mixins with the same method name and
>> without a common parent class is just not sane.
>
> Right.  This is a cornerstone of cooperative multiple inheritance that
> sometimes seems to be forgotten; there is a big difference between
defining
> a method and extending a method, and only extending methods can make
super
> calls.
>
> The __init__ case is an exception, because there's no requirement that
a
> subclass have a signature compatible with the superclass (if you don't
get
> this, read up on constructors in C++ or Java).

I understand the desire for it to be an exception, I fail to see how it
actually is one.  The namespace/signature conflicts exist just the same.

The only way I can see to handle incompatible signatures is to add a flag
that says "I am the *ONLY* class allowed to subclass X" (triggering an
error if violated), have super() entirely bypass it, and then call
X.__init__() directly.  Even that doesn't handle X's superclasses being
subclassed more than once, and it looks pretty complicated/obscure anyway.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2007-03-22 00:28

Message:
Logged In: YES 
user_id=6380
Originator: NO

> I think it would also help if calling a method via super() didn't allow
> positional arguments.

That's absurd, *except* for __init__(), where it could make sense
depending on the style of cooperation used.  But not enough to enforce this
in the language; in Py3k you will be able to enforce this on a per-class
basis.

> Having two mixins with the same method name and
> without a common parent class is just not sane.

Right.  This is a cornerstone of cooperative multiple inheritance that
sometimes seems to be forgotten; there is a big difference between defining
a method and extending a method, and only extending methods can make super
calls.

The __init__ case is an exception, because there's no requirement that a
subclass have a signature compatible with the superclass (if you don't get
this, read up on constructors in C++ or Java).

----------------------------------------------------------------------

Comment By: Adam Olsen (rhamphoryncus)
Date: 2007-03-21 20:44

Message:
Logged In: YES 
user_id=12364
Originator: NO

I think the avoidance of super() is largely *because* of this kind of
leniency.  Consider this snippet on python 2.3:

class Base(object):
    def __init__(self, foo=None, *args, **kwargs):
        super(Base, self).__init__(foo, *args, **kwargs)
Base(foo='bar')
Base('bar', 42)
Base('bar', 42, x=7)

All pass silently, no error checking.  Now consider a python 3.0 version,
with a strict object.__init__:

class Base:
    def __init__(self, foo=None, *, y='hi', **kwargs):
        super(Base, self).__init__(**kwargs)
Base(foo='bar')  # Valid, accepted
Base('bar', 42)  # Raises exception
Base('bar', x=7)  # Raises exception

The error checking added by this bug/patch and the error checking added by
PEP 3102 (keyword-only arguments) make super a lot more sane.

I think it would also help if calling a method via super() didn't allow
positional arguments.  If the base class's arguments can't be given as
keyword args then you probably should call it explicitly, rather than
relying on super()'s MRO.

I was also going to suggest super() should automagically create empty
methods your parent classes don't have, but then I realized you really
should have a base class that asserts the lack of such an automagic
method:

class Base(object):
    def mymethod(self, myonlyarg='hello world'):
        assert not hasattr(super(Base, self), 'mymethod')

By the time you reach this base class you will have stripped off any extra
arguments that your subclasses added, leaving you with nothing to pass up
(and nothing to pass to).  Having two mixins with the same method name and
without a common parent class is just not sane.

----------------------------------------------------------------------

Comment By: Terry J. Reedy (tjreedy)
Date: 2007-03-21 19:16

Message:
Logged In: YES 
user_id=593130
Originator: NO

I ask myself, what should I expect from the documentation...

>>> object.__init__.__doc__
'x.__init__(...) initializes x; see x.__class__.__doc__ for signature'
>>> object.__class__
<type 'type'>
>>> type.__doc__
"type(object) -> the object's type\ntype(name, bases, dict) -> a new
type"

and I still don't know ;-).

----------------------------------------------------------------------

Comment By: Blake Ross (blakeross)
Date: 2007-03-21 18:48

Message:
Logged In: YES 
user_id=1747060
Originator: YES

Holding the strict version for 3 makes sense to me. Let me know if you
need anything more on my end... thanks for the fast turnaround.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2007-03-21 18:42

Message:
Logged In: YES 
user_id=6380
Originator: NO

Well, but since it's been like this for a long time, I don't want to
gratuitously break code. At least not in 2.6. So I'm rejecting the stricter
patch for 2.6. (However, if you want to submit patches that would fix these
breakages anyway, be my guest.)

----------------------------------------------------------------------

Comment By: Blake Ross (blakeross)
Date: 2007-03-21 18:03

Message:
Logged In: YES 
user_id=1747060
Originator: YES

Looks good. I skimmed briefly the tests you mentioned. The issue with
test_array appears to be exactly the kind of bug this is intended to
identify: it calls array.__init__(...), but array doesn't have its own
initializer, so object's is used.

I'd guess that the others are failing due to whatever the problem with
pickling is (test_descr uses pickling). I haven't looked into that yet.

I'm sure cooperative super calling of __init__ isn't all that common (it
seems like the mechanism itself isn't used much yet, and may not be until
it's done via keyword) but it doesn't seem like such a bad practice,
especially when mixins are in the picture. There doesn't seem to be a great
alternative.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2007-03-20 17:24

Message:
Logged In: YES 
user_id=6380
Originator: NO

I should mention that if we can't get the strict version of this in 2.6,
we should be able to get it into 3.0.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2007-03-20 16:54

Message:
Logged In: YES 
user_id=6380
Originator: NO

Here's a stricter version. Unfortunately it breaks a couple of standard
modules; this is a confirmation of my doubts whether the style of
cooperative super calling of __init__ that you use is really the most
common or "best practice".

So far I have only fixed string.py (which would otherwise prevent
extensions from being built); I haven't looked into why the other tests
fail: test_array, test_cpickle, test_descr, test_pickle (and maybe more?).

My conclusion: this would probably break too much code to be worth it.  So
I'll have to revert to the previous version.  But anyway, here it is for
your perusal.
File Added: new_init_strict.patch

----------------------------------------------------------------------

Comment By: Blake Ross (blakeross)
Date: 2007-03-19 21:27

Message:
Logged In: YES 
user_id=1747060
Originator: YES

I think making the check more rigid is a good idea, since this should
throw:

class a(object):
   def __init__(self, foo):
       super(a, self).__init__(foo)
   def __new__(cls, foo):
       return object.__new__(cls)
a(1)

(minor typo in the patch: "solution it" -> "solution is")

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2007-03-19 19:35

Message:
Logged In: YES 
user_id=6380
Originator: NO

This smells enough like a new feature that it couldn't go into 2.5.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2007-03-19 19:31

Message:
Logged In: YES 
user_id=6380
Originator: NO

Attached is a patch that implements this proposal, adding copious
commentary.  It doesn't seem to break anything in the test suite.

I wonder if we should even make the check more rigid: check the argument
list if either the current method *is* overridden or the other one *is not*
overridden.  This would make super calls check the arguments even if the
other method is overridden.  What do you think?
File Added: new_init.patch

----------------------------------------------------------------------

Comment By: Blake Ross (blakeross)
Date: 2007-03-19 18:45

Message:
Logged In: YES 
user_id=1747060
Originator: YES

Makes sense. I don't think we can ever be completely correct here since
we're inferring intent from the presence of __init__/__new__ that's liable
to be wrong in some cases, but it's likely correct often enough that it's
worth doing.

If I understand correctly, we want to be more forgiving iff one of the two
methods is used, so it seems like we should be complaining if both are used
*or* if neither is used. After all, I could add a __new__ to my coop use
case and I'd still want object to complain. If that's the case, both
object_new and object_init should be complaining if ((tp->tp_new ==
object_new && tp->tp_init == object_init) || (tp->tp_new != object_new &&
tp->tp_init != object_init)).

Of course, for the paranoid, there's always the risk that __new__ will
modify these class functions and change the outcome :) For instance, if a
class had a __new__ and no __init__ and its __new__ changed __new__ back to
object.__new__, object_init on that run would be fooled into thinking it's
using the defaults for both and would complain. I think this could only be
fixed in type_call, which is rather ugly...but then, this *is* a special
case of the "call __init__ after __new__" behavior, and we're trying to
solve it within the methods themselves. Perhaps this last point is academic
enough to be ignored...I don't know why anyone would do this, although the
language makes it possible.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2007-03-19 16:01

Message:
Logged In: YES 
user_id=6380
Originator: NO

I'll try to explain why I did it this way.  I was considering the single
inheritance case implementing an Immutable object, which overrides __new__
but has no need to override __init__ (since it's too late to do anything in
__init__ for an Immutable object).  Since the __init__ still gets called it
would be annoying to have to override it just to make the error go away if
there was a check in __init__.  The other case is overriding __init__
without overriding __new__, which is the most common way of doing Mutable
objects; here you wouldn't want __new__ to complain about extra args.  So
the only time when you'd want complaints is if both __new__ and __init__
are the defaults, in which case it doesn't really matter whether you
implement this in __init__ or in __new__, so I arbitrarily chose __new__.

I wasn't thinking of your use case at the time though (cooperative super
calls to __init__, which still isn't something I engage in on a day-to-day
basis).  I wonder if the right thing to do wouldn't be to implement the
same check both in __init__ and in __new__.

Am I makign sense?

----------------------------------------------------------------------

Comment By: Georg Brandl (gbrandl)
Date: 2007-03-19 04:56

Message:
Logged In: YES 
user_id=849994
Originator: NO

I don't really understand either why object_new() checks the arguments,
not object_init():

"""
static int
object_init(PyObject *self, PyObject *args, PyObject *kwds)
{
	return 0;
}

/* If we don't have a tp_new for a new-style class, new will use this
one.
   Therefore this should take no arguments/keywords.  However, this new
may
   also be inherited by objects that define a tp_init but no tp_new. 
These
   objects WILL pass argumets to tp_new, because it gets the same args as
   tp_init.  So only allow arguments if we aren't using the default init,
in
   which case we expect init to handle argument parsing. */
static PyObject *
object_new(PyTypeObject *type, PyObject *args, PyObject *kwds)
{
	if (type->tp_init == object_init && (PyTuple_GET_SIZE(args) ||
	     (kwds && PyDict_Check(kwds) && PyDict_Size(kwds)))) {
		PyErr_SetString(PyExc_TypeError,
				"default __new__ takes no parameters");
		return NULL;
	}
	return type->tp_alloc(type, 0);
}
"""

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1683368&group_id=5470

From noreply at sourceforge.net  Tue Apr  3 02:02:38 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Mon, 02 Apr 2007 17:02:38 -0700
Subject: [ python-Bugs-1602378 ] Incorrect docs for bisect_left
Message-ID: <E1HYWTu-0005iW-2E@sc8-sf-web24.sourceforge.net>

Bugs item #1602378, was opened at 2006-11-24 12:07
Message generated for change (Settings changed) made by rhettinger
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1602378&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Documentation
Group: Python 2.5
>Status: Closed
>Resolution: Fixed
Priority: 4
Private: No
Submitted By: Daniel Eloff (eloff)
Assigned to: Raymond Hettinger (rhettinger)
Summary: Incorrect docs for bisect_left

Initial Comment:
Quote:
Return the index where to insert item x in list a, assuming a is sorted.
    
The return value i is such that all e in a[:i] have e < x, and all e in a[i:] have e >= x.  So if x already appears in the list, i points just before the leftmost x already there.

The last sentence is incorrect, and it contradicts what comes earlier. Not knowing which is correct, I had to test it in the shell.

>>> from bisect import *
>>> l = [1,2,3,3,3,4,5]
>>> bisect_left(l,3)
2
>>> l[2:]
[3, 3, 3, 4, 5]

It should be changed to read "i points at the leftmost x already there"

-Dan

----------------------------------------------------------------------

>Comment By: Raymond Hettinger (rhettinger)
Date: 2007-04-02 19:02

Message:
Logged In: YES 
user_id=80475
Originator: NO

Fixed.  See versions 54665 and 64666.

----------------------------------------------------------------------

Comment By: Daniel Eloff (eloff)
Date: 2006-12-03 17:39

Message:
Logged In: YES 
user_id=730918
Originator: YES

That makes more sense, but if that's explained anywhere it was hidden away
where I've never discovered it. I would be in favor of you're suggested fix
to the line in question.

-Dan

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2006-11-24 14:16

Message:
Logged In: YES 
user_id=31435
Originator: NO

The docs are written from the POV that indices in Python point /between/
array elements, which is the easiest way to understand slices, and that
there are n+1 non-negative indices that "make obvious sense" in slices of a
sequence with n elements:  index 0 points "just before" element a[0], and
index n "just after" element a[n-1], while for 0 < i < n-1, index i points
"just before" element [i] /and/ "just after" element a[i-1].

This is also the sanest way to understand the return value of the bisect
functions, which again can return n+1 values given a sequence with n
elements.

That said, I agree the docs are cryptic if you don't understand that
first.  I'm not sure this is the best place to explain it.  The specific
line in question could be "repaired" by saying a[i] is the leftmost x
already there, identifying one of the n elements instead of one of the n+1
sequence positions.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1602378&group_id=5470

From noreply at sourceforge.net  Tue Apr  3 03:40:29 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Mon, 02 Apr 2007 18:40:29 -0700
Subject: [ python-Bugs-1382213 ] Tutorial section 9.5.1 ignores MRO for
	new-style classes
Message-ID: <E1HYY0b-0000XR-Ex@sc8-sf-web21.sourceforge.net>

Bugs item #1382213, was opened at 2005-12-16 02:08
Message generated for change (Comment added) made by rhettinger
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1382213&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Documentation
Group: Python 2.4
>Status: Closed
>Resolution: Fixed
Priority: 5
Private: No
Submitted By: GaryD (gazzadee)
Assigned to: Raymond Hettinger (rhettinger)
Summary: Tutorial section 9.5.1 ignores MRO for new-style classes

Initial Comment:
Section 9.5.1 ("Multiple Inheritance") of the tutorial
(as viewed on http://www.python.org/doc/2.4.2/tut/)
discusses the Method Resolution Order (MRO) for classes
with multiple inheritance.

However, the 2nd paragraph incorrectly states that "The
only rule necessary to explain the semantics is the
resolution rule used for class attribute references.
This is depth-first, left-to-right". Whilst this is
true for old-style classes, new-style classes use a
different MRO (as documented elsewhere - eg.
http://www.python.org/2.3/mro.html)

----------------------------------------------------------------------

>Comment By: Raymond Hettinger (rhettinger)
Date: 2007-04-02 20:40

Message:
Logged In: YES 
user_id=80475
Originator: NO

Fixed.  See versions 54667 and 54668.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1382213&group_id=5470

From noreply at sourceforge.net  Tue Apr  3 04:20:02 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Mon, 2 Apr 2007 19:20:02 -0700
Subject: [ python-Bugs-1683316 ] select.select() injecting spurious text in
	stdout
Message-ID: <200704030220.l332K2bw020925@sc8-sf-db2-new-b.sourceforge.net>

Bugs item #1683316, was opened at 2007-03-18 17:34
Message generated for change (Comment added) made by sf-robot
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1683316&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.4
>Status: Closed
Resolution: Invalid
Priority: 5
Private: No
Submitted By: Peter Williams (peter_ono)
Assigned to: Nobody/Anonymous (nobody)
Summary: select.select() injecting spurious text in stdout

Initial Comment:
I'm using a function (see attachment) similar to that described on Page 386 (Section 9.12) of the Python Cookbook (Second Edition) to capture the stdout and stderr streams of programs run with popen2.Popen3.  This involves using select.select() to detect the availability of data on the two streams.  Sometimes, what looks like a debugging message:

"EXCEPTION IN SAFE SELECT 9\n"

gets injected into the stdout stream.  As far as I can tell this has only started occuring after updating to version 2.4.4.

No exception occurs and the string just silently appears in stdout.  Apart from the bogus text in stdout everything seems to be working correctly and I'm able to work around the problem by looking for the string and removing whenever it occurs.

It looks to me like a debugging message that somebody forgot to remove.

----------------------------------------------------------------------

>Comment By: SourceForge Robot (sf-robot)
Date: 2007-04-02 19:20

Message:
Logged In: YES 
user_id=1312539
Originator: NO

This Tracker item was closed automatically by the system. It was
previously set to a Pending status, and the original submitter
did not respond within 14 days (the time period specified by
the administrator of this Tracker).

----------------------------------------------------------------------

Comment By: Georg Brandl (gbrandl)
Date: 2007-03-19 02:12

Message:
Logged In: YES 
user_id=849994
Originator: NO

I can find no occurrence of the message or parts of it in the current 2.5
or 2.6 branches.

Please make sure that it is Python that emits that message.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1683316&group_id=5470

From noreply at sourceforge.net  Tue Apr  3 09:15:53 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Tue, 03 Apr 2007 00:15:53 -0700
Subject: [ python-Bugs-1666807 ] Incorrect file path reported by
	inspect.getabsfile()
Message-ID: <E1HYdFB-0002q4-LV@sc8-sf-web22.sourceforge.net>

Bugs item #1666807, was opened at 2007-02-23 08:08
Message generated for change (Comment added) made by zseil
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1666807&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.5
>Status: Closed
>Resolution: Duplicate
Priority: 5
Private: No
Submitted By: Fernando P?rez (fer_perez)
Assigned to: Nobody/Anonymous (nobody)
Summary: Incorrect file path reported by inspect.getabsfile()

Initial Comment:
The following code demonstrates the problem succinctly:

###
import inspect,sys

print 'Version info:',sys.version_info
print

f1 = inspect.getabsfile(inspect)
f2 = inspect.getabsfile(inspect.iscode)
print 'File for `inspect`       :',f1
print 'File for `inspect.iscode`:',f2
print 'Do these match?',f1==f2
if f1==f2:
    print 'OK'
else:
    print 'BUG - this is a bug in this version of Python'

###  EOF

Running this on my system (Linux, Ubuntu Edgy) with 2.3, 2.4 and 2.5 produces:

tlon[bin]> ./python2.3 ~/code/python/inspect_bug.py
Version info: (2, 3, 6, 'final', 0)

File for `inspect`       : /home/fperez/tmp/local/lib/python2.3/inspect.py
File for `inspect.iscode`: /home/fperez/tmp/local/lib/python2.3/inspect.py
Do these match? True
OK
tlon[bin]> python2.4 ~/code/python/inspect_bug.py
Version info: (2, 4, 4, 'candidate', 1)

File for `inspect`       : /usr/lib/python2.4/inspect.py
File for `inspect.iscode`: /home/fperez/tmp/local/bin/inspect.py
Do these match? False
BUG - this is a bug in this version of Python
tlon[bin]> python2.5 ~/code/python/inspect_bug.py
Version info: (2, 5, 0, 'final', 0)

File for `inspect`       : /usr/lib/python2.5/inspect.py
File for `inspect.iscode`: /home/fperez/tmp/local/bin/inspect.py
Do these match? False
BUG - this is a bug in this version of Python


###

The problem arises in the fact that inspect relies, for functions (at least), on the func_code.co_filename attribute to contain a complete path.  This changed between 2.3 and 2.4, but the inspect module was never updated.  This code:

###
import inspect,sys

print 'Python version info:',sys.version_info
print 'File info for `inspect.iscode function`:'
print ' ',inspect.iscode.func_code.co_filename
print
### EOF

shows the problem:

tlon[bin]> ./python2.3 ~/code/python/inspect_bug_details.py
Python version info: (2, 3, 6, 'final', 0)
File info for `inspect.iscode function`:
  /home/fperez/tmp/local//lib/python2.3/inspect.py

tlon[bin]> python2.5 ~/code/python/inspect_bug_details.py
Python version info: (2, 5, 0, 'final', 0)
File info for `inspect.iscode function`:
  inspect.py

###

(2.4 has the same issue).

Basically, if the func_code.co_filename attribute now stores only the final filename without the full path, then the logic in the inspect module needs to be changed to accomodate this so that correct paths are reported to the user like they were in the 2.3 days.

----------------------------------------------------------------------

>Comment By: ??iga Seilnacht (zseil)
Date: 2007-04-03 09:15

Message:
Logged In: YES 
user_id=1326842
Originator: NO

Ok, this is the same problem as reported in bug #1180193:

http://www.python.org/sf/1180193

The reporter of that bug is willing to write a patch, so
I think it is better to close this one as a duplicate. 

----------------------------------------------------------------------

Comment By: Fernando P?rez (fer_perez)
Date: 2007-03-19 00:49

Message:
Logged In: YES 
user_id=395388
Originator: YES

Yes, though at least in this report we seem to have narrowed down the
problem a little better.

I'm perfectly willing to believe that Ubuntu is somehow mis-building their
shipped Python, but it would be great if the Python build itself could be
hardened against this being possible in the first place.

Unfortunately I don't know how to better track the problem, but I'll be
glad to provide info from my local system upon request.

----------------------------------------------------------------------

Comment By: ??iga Seilnacht (zseil)
Date: 2007-03-16 15:06

Message:
Logged In: YES 
user_id=1326842
Originator: NO

It looks like this is a bug in Ubuntu build process.
The logging package had the same problem:
http://www.python.org/sf/1669498
http://www.python.org/sf/1633605
http://www.python.org/sf/1616422


----------------------------------------------------------------------

Comment By: Fernando P?rez (fer_perez)
Date: 2007-03-09 03:00

Message:
Logged In: YES 
user_id=395388
Originator: YES

As I mentioned, on hand-built Pythons I don't get the bug at all.  So it
may be a problem with how Ubuntu builds its Python, since I can reproduce
the problem with both 2.4 and 2.5, but only with the default ones provided
by Ubuntu Edgy.  I don't know enough about their packaging system to know
how to retrieve build info.

There may be something odd in their build, but it would be nice if this
simply couldn't happen at all.  If anyone knows how to retrieve the
relevant info from an ubuntu build, I'll be happy to try and provide it.

----------------------------------------------------------------------

Comment By: Collin Winter (collinwinter)
Date: 2007-03-09 01:04

Message:
Logged In: YES 
user_id=1344176
Originator: NO

I haven't been able to reproduce this with Python 2.3.5, 2.4.4, 2.5.0 or
SVN HEAD (all hand-built on Slackware Linux).

What options are you providing to ./configure?

----------------------------------------------------------------------

Comment By: Fernando P?rez (fer_perez)
Date: 2007-02-24 08:22

Message:
Logged In: YES 
user_id=395388
Originator: YES

No, in my case the original tests with 2.4 and 2.5 were done with the
Ubuntu-provided (Edgy) versions, unmodified from their apt-get install
state.  But your comments are interesting.  I just rebuilt 2.5 by hand from
source on the same system, and this is what I get:

tlon[bin]> ./python2.5 ~/code/python/inspect_bug.py
Version info: (2, 5, 0, 'final', 0)

File for `inspect`       :
/home/fperez/tmp/local/lib/python2.5/inspect.py
File for `inspect.iscode`:
/home/fperez/tmp/local/lib/python2.5/inspect.py
Do these match? True
OK


tlon[bin]> ./python2.5 ~/code/python/inspect_bug_details.py
Python version info: (2, 5, 0, 'final', 0)
File info for `inspect.iscode function`:
  /home/fperez/tmp/local//lib/python2.5/inspect.py

So basically it seems that there's a difference in the value of the
func_code.co_filename object attribute depending on how the build was
made.

At this point I'm not really sure if this should be considered a Python
bug or an Ubuntu one...  Perhaps the Python build process can be made more
robust, I simply don't know.

But the end-user behavior /is/ a bug, so it would be nice to know how to
fix it.


Thanks for your info!

----------------------------------------------------------------------

Comment By: Jerry Seutter (jseutter)
Date: 2007-02-24 07:51

Message:
Logged In: YES 
user_id=1727609
Originator: NO

Hi,

On a custom-compiled Python 2.5 on Ubuntu Dapper, I get:
yello at outback:~/code/python $ python bug_1666807.py
Version info: (2, 5, 0, 'final', 0)

File for `inspect` : /usr/local/lib/python2.5/inspect.py
File for `inspect.iscode`: /usr/local/lib/python2.5/inspect.py
Do these match? True
OK


On a system python 2.4.3 on another Ubuntu Dapper system, I get:
yello at adelaide:~/code/python $ python bug_1666807.py
Version info: (2, 4, 3, 'final', 0)

File for `inspect` : /usr/lib/python2.4/inspect.py
File for `inspect.iscode`: /usr/lib/python2.4/inspect.py
Do these match? True
OK

However, I can recreate this issue on another system (CentOS4) with a
custom install where a symlink is involved.  I get: 
[prompt goes here python]$ python bug_1666807.py
Version info: (2, 2, 3, 'final', 0)

File for `inspect` : /xxx/yyyy/PythonHome/lib/python2.2/inspect.py
File for `inspect.iscode`: /xxx/yyyy/Python-2.2/lib/python2.2/inspect.py
Do these match? 0
BUG - this is a bug in this version of Python

Is a symlink involved on the system where you saw this behaviour?

----------------------------------------------------------------------

Comment By: Fernando P?rez (fer_perez)
Date: 2007-02-23 17:57

Message:
Logged In: YES 
user_id=395388
Originator: YES

Note: a colleague just tested this under Python 2.4 for Mac OSX (intel),
and the problem didn't show up there:

import inspect
print 'File for `code`         :',inspect.getabsfile(code)
print 'File for `code.interact`:',inspect.getabsfile(code.interact)

Gives:

File for `code`         :
/Library/Frameworks/Python.framework/Versions/2.4/lib/python2.4/code.py
File for `code.interact`:
/Library/Frameworks/Python.framework/Versions/2.4/lib/python2.4/code.py

My original tests were done under Ubuntu Edgy Eft, using the
Ubuntu-provided 2.4 and 2.5, and a hand-compiled 2.3.6 from the source
download at python.org.

HTH,

f

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1666807&group_id=5470

From noreply at sourceforge.net  Tue Apr  3 09:16:11 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Tue, 03 Apr 2007 00:16:11 -0700
Subject: [ python-Bugs-1051638 ] incorrect traceback filename from pyc
Message-ID: <E1HYdFT-0002vt-Dq@sc8-sf-web22.sourceforge.net>

Bugs item #1051638, was opened at 2004-10-21 19:24
Message generated for change (Comment added) made by zseil
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1051638&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Parser/Compiler
Group: Python 2.3
>Status: Closed
>Resolution: Duplicate
Priority: 5
Private: No
Submitted By: Kevin Quick (kquick)
Assigned to: Nobody/Anonymous (nobody)
Summary: incorrect traceback filename from pyc

Initial Comment:
The .pyc file apparently caches the entire path of the source file during 
compilation, causing it to report improper path information if the 
resulting code is moved and an exception occurs.

$ python
Python 2.3.3 (#1, Oct 18 2004, 16:10:24) 
[GCC 3.3.4 20040623 (Gentoo Linux 3.3.4-r1, ssp-3.3.2-2, pie-8.7.6)] 
on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> 


----------------------------------------------------------------------

>Comment By: ?iga Seilnacht (zseil)
Date: 2007-04-03 09:16

Message:
Logged In: YES 
user_id=1326842
Originator: NO

This is a duplicate of bug #1180193:

http://www.python.org/sf/1180193


----------------------------------------------------------------------

Comment By: Kevin Quick (kquick)
Date: 2004-10-30 08:01

Message:
Logged In: YES 
user_id=6133

OK, I've done some more investigation and have some more details and 
clarification.  I've attached another python script that demonstrates
these 
additional clarifications:

First, it runs sample scripts in a test directory.  The sample scripts
generate an 
exception, print that exception with traceback.print_exc, then raise to
let the 
interpreter print the exception as it fails the script.  The control
script 
verifies that the output of traceback.print_exc matches the interpreter's

traceback output.  So far, so good.

Step two demonstrates the first problem:  the original directory is simply

renamed; the previous location does not exist.  The same sample scripts
are 
run, and the output now shows that traceback.print_exc() output is 
*different* than the interpreter's traceback.  Specifically, the
interpreter 
prints the new, correct path, but the traceback.print_exc() prints the old

path.

Step three demonstrates another problem.  The scripts are run from the new

directory, but the old directory now exists, along files having the
original .py 
names, but those files are now much shorter; the lines that will be
referenced 
in the traceback do not exist.  In the output from this step, note that
the 
output from traceback.print_exc() and the interpreter traceback are again

*different* from each other.  In this case, both are fooled into reporting
file 
paths in the old directory, but for a non-existent line the interpreter
prints a 
blank line but traceback.print_exc() doesn't print a line at all.

And finally, the .py files in the original directory are filled with
nonsense (in 
python terms).  This test case shows that the output from traceback.
print_exc() and the interpreter *do match*, but both are fooled into
showing 
file paths in the old directory and the nonsense from the new versions of
the 
files.

IMHO, I think that the output from the traceback.print_exc() and the 
interpreter traceback should be the same in *all* scenarios, and I also
think 
that the path used for importing the module should be used in traceback 
reporting.

Rather than encoding the full path of the .py source in the .pyc, the
compiler 
should just encode the name portion, and use the path from the .pyc import

and look for the .py in the same directory as the .pyc; if that's garbage,
then 
so be it, but this will 99% of the time be more correct than the current 
behavior.


----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2004-10-24 02:40

Message:
Logged In: YES 
user_id=80475

IMO, this is not a bug.  A pyc file has no way of knowing
the future locations of the source that originally created it.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1051638&group_id=5470

From noreply at sourceforge.net  Tue Apr  3 09:16:27 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Tue, 03 Apr 2007 00:16:27 -0700
Subject: [ python-Bugs-1180193 ] broken pyc files
Message-ID: <E1HYdFj-0002wE-6h@sc8-sf-web22.sourceforge.net>

Bugs item #1180193, was opened at 2005-04-10 15:10
Message generated for change (Comment added) made by zseil
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1180193&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Interpreter Core
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Armin Rigo (arigo)
Assigned to: Nobody/Anonymous (nobody)
Summary: broken pyc files

Initial Comment:
In a number of situations, the .pyc files can become "corrupted" in a subtle way: the co_filename attribute of the code objects it contains become wrong.  This can occur if we move or rename directories, or if we access the same set of files from two different locations (e.g. over NFS).

This corruption doesn't prevent the .pyc files from working, but the interpreter looses the reference to the source file.  It causes trouble in tracebacks, in the inspect module, etc.

A simple fix would be to use the following logic when importing a .py file: if there is a corresponding .pyc file, in addition to checking the timestamp, check the co_filename attribute of the loaded object.  If it doesn't point to the original .py file, discard the code object and ignore the .pyc file.

Alternatively, we could force all co_filenames to point to the .py file when loading the .pyc file.

I'll write a patch for whichever alternative seems better.

----------------------------------------------------------------------

>Comment By: ?iga Seilnacht (zseil)
Date: 2007-04-03 09:16

Message:
Logged In: YES 
user_id=1326842
Originator: NO

This problem is reported quite often in the tracker,
although it shows up in different places:

http://www.python.org/sf/1666807
http://www.python.org/sf/1051638

I closed those bugs as duplicates of this one.

The logging package is also affected:

http://www.python.org/sf/1669498
http://www.python.org/sf/1633605
http://www.python.org/sf/1616422


----------------------------------------------------------------------

Comment By: Armin Rigo (arigo)
Date: 2007-03-28 15:40

Message:
Logged In: YES 
user_id=4771
Originator: YES

What I called "corruption" is the situation
where both the .py and the .pyc files are
present, but the filename stored in the .pyc
co_filenames is no longer the valid absolute
path of the corresponding .py file, for any
reason (renaming, NFS views, etc.).

This situation causes the tracebacks and the
inspect module to fail to locate the .py file,
which I consider a bug.

----------------------------------------------------------------------

Comment By: Martin v. L?wis (loewis)
Date: 2007-03-28 14:01

Message:
Logged In: YES 
user_id=21627
Originator: NO

I fail to see the corruption. It is quite desirable and normal to only
ship pyc files - that the file name they refer to is actually present is
not a requirement at all.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1180193&group_id=5470

From noreply at sourceforge.net  Tue Apr  3 09:46:20 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Tue, 03 Apr 2007 00:46:20 -0700
Subject: [ python-Bugs-1508369 ] logging module formatter problem with
	%(filename)s
Message-ID: <E1HYdie-00046z-6c@sc8-sf-web23.sourceforge.net>

Bugs item #1508369, was opened at 2006-06-19 04:08
Message generated for change (Comment added) made by zseil
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1508369&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Macintosh
Group: Python 2.4
Status: Open
Resolution: Fixed
Priority: 5
Private: No
Submitted By: David Hess (david_k_hess)
Assigned to: Ronald Oussoren (ronaldoussoren)
Summary: logging module formatter problem with %(filename)s

Initial Comment:

With Python 2.4.3 installed via the .dmg on to MacOS, when the python 
modules are compiled, they are compiled by specifying the path in such a 
way that it contains a double slash. This causes the logging module to not 
be able to figure out which module is the correct one to replace for %
(filename)s.

The following is the crux of the issue:

>>> logging.debug.func_code.co_filename
'/Library/Frameworks/Python.framework/Versions/2.4//lib/python2.4/
logging/__init__.py'
>>> logging.__file__
'/Library/Frameworks/Python.framework/Versions/2.4/lib/python2.4/
logging/__init__.pyc'
>>> 

These two strings need to match for %(filename)s to work.

I deleted /Library/Frameworks/Python.framework/Versions/2.4/lib/
python2.4/logging/__init__.pyc and recompiled it by just importing 
logging. That fixed the problem.

I assume the fix will be in the installer somewhere.



----------------------------------------------------------------------

>Comment By: ?iga Seilnacht (zseil)
Date: 2007-04-03 09:46

Message:
Logged In: YES 
user_id=1326842
Originator: NO

The 2.4 branch is not maintained anymore.  Can this
bug be closed?

----------------------------------------------------------------------

Comment By: Ronald Oussoren (ronaldoussoren)
Date: 2006-06-25 23:16

Message:
Logged In: YES 
user_id=580910

Fixed in revision #47093 for python2.5. 

I'm not closing the bug yet because I haven't backported this to 2.4 yet.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1508369&group_id=5470

From noreply at sourceforge.net  Tue Apr  3 09:56:00 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Tue, 03 Apr 2007 00:56:00 -0700
Subject: [ python-Bugs-1671137 ] slice obj with no start index is 0 instead of
	None sometimes
Message-ID: <E1HYds0-00034G-SV@sc8-sf-web24.sourceforge.net>

Bugs item #1671137, was opened at 2007-02-28 11:58
Message generated for change (Comment added) made by jyzude
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1671137&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Interpreter Core
Group: Python 2.6
>Status: Closed
Resolution: Wont Fix
Priority: 5
Private: No
Submitted By: Mike Verdone (jyzude)
Assigned to: Nobody/Anonymous (nobody)
Summary: slice obj with no start index is 0 instead of None sometimes

Initial Comment:
Slice objects returned by the slice ":42" return different slice objects depending on whether the entire slice operation is simple or extended. This bit of code explains it best:

class SliceBug:
    def __getitem__(self, sliceArg):
        print sliceArg

s = SliceBug()

s[:42]
s[:42,]


s[:42] produces slice(0, 42, None)
s[:42,] produces (slice(None, 42, None),)

Note that this bug only occurs on classes that do no inherit from object ("old style" classes). If you change the class to make it inherit from "object" both slices have None as their start index. Oddly enough in Python 3000 it still only happens on "old style" classes even though supposedly they are the same as new style classes. I have also reproduced the bug in Python 2.6, 2.4, 2.3, and 2.2. Seems to be a long standing bug/feature.

----------------------------------------------------------------------

>Comment By: Mike Verdone (jyzude)
Date: 2007-04-03 01:56

Message:
Logged In: YES 
user_id=584997
Originator: YES

I concur. I'll close the bug.

----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2007-04-02 13:11

Message:
Logged In: YES 
user_id=80475
Originator: NO

Fixing this small inconsistency has very little upside but runs the risk
of breaking code that has been running fine for years.  I think we should
punt.  If you agree, please close this bug report.

----------------------------------------------------------------------

Comment By: Mike Verdone (jyzude)
Date: 2007-02-28 12:04

Message:
Logged In: YES 
user_id=584997
Originator: YES

Correction! This is fixed in Python 3000. I just have too many windows
open and too many branches checked out.
File Added: slicebug.py

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1671137&group_id=5470

From noreply at sourceforge.net  Tue Apr  3 10:44:40 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Tue, 03 Apr 2007 01:44:40 -0700
Subject: [ python-Feature Requests-1506296 ] Add some dicts to datetime module
Message-ID: <E1HYed6-00045P-34@sc8-sf-web23.sourceforge.net>

Feature Requests item #1506296, was opened at 2006-06-14 21:35
Message generated for change (Comment added) made by zseil
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1506296&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Gregory Petrosyan (gregory_p)
Assigned to: Nobody/Anonymous (nobody)
Summary: Add some dicts to datetime module

Initial Comment:
I think it would be nice to have dicts like

weekdays = {0:'Monday', ...}
isoweekdays = {1:'Monday', ...}
months = {1:'January', ...}

in the datetime module. IMO they are rather usefull.

--
Regards, Gregory.



----------------------------------------------------------------------

>Comment By: Ziga Seilnacht (zseil)
Date: 2007-04-03 10:44

Message:
Logged In: YES 
user_id=1326842
Originator: NO

The calendar module already has something similar;
calendar.day_name, calendar.day_abbr,
calendar.month_name, calendar.month_abbr.

If you think there is still something missing, it
would be better to add it there.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1506296&group_id=5470

From noreply at sourceforge.net  Tue Apr  3 13:31:27 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Tue, 03 Apr 2007 04:31:27 -0700
Subject: [ python-Bugs-1180193 ] broken pyc files
Message-ID: <E1HYhEV-00006E-MP@sc8-sf-web22.sourceforge.net>

Bugs item #1180193, was opened at 2005-04-10 13:10
Message generated for change (Comment added) made by arigo
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1180193&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Interpreter Core
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Armin Rigo (arigo)
Assigned to: Nobody/Anonymous (nobody)
Summary: broken pyc files

Initial Comment:
In a number of situations, the .pyc files can become "corrupted" in a subtle way: the co_filename attribute of the code objects it contains become wrong.  This can occur if we move or rename directories, or if we access the same set of files from two different locations (e.g. over NFS).

This corruption doesn't prevent the .pyc files from working, but the interpreter looses the reference to the source file.  It causes trouble in tracebacks, in the inspect module, etc.

A simple fix would be to use the following logic when importing a .py file: if there is a corresponding .pyc file, in addition to checking the timestamp, check the co_filename attribute of the loaded object.  If it doesn't point to the original .py file, discard the code object and ignore the .pyc file.

Alternatively, we could force all co_filenames to point to the .py file when loading the .pyc file.

I'll write a patch for whichever alternative seems better.

----------------------------------------------------------------------

>Comment By: Armin Rigo (arigo)
Date: 2007-04-03 11:31

Message:
Logged In: YES 
user_id=4771
Originator: YES

If you ask me, I think that when the importing
system finds both a .py and a .pyc for a module,
then it should ignore all co_filename and replace
them with the real path of the .py file.  I can't
see any point of not doing so.

There are many other quirks caused by .pyc files
accidentally remaining around, but we cannot fix them
all as long as the .pyc files are at the same time
a cache for performance reason and a redistributable
program format (e.g. if "rm x.py" or "svn up" deletes
a .py file, then the module is still importable via
the .pyc left behind, a great way to oversee the fact
that imports elsewhere in the project need to be
updated).

----------------------------------------------------------------------

Comment By: Ziga Seilnacht (zseil)
Date: 2007-04-03 07:16

Message:
Logged In: YES 
user_id=1326842
Originator: NO

This problem is reported quite often in the tracker,
although it shows up in different places:

http://www.python.org/sf/1666807
http://www.python.org/sf/1051638

I closed those bugs as duplicates of this one.

The logging package is also affected:

http://www.python.org/sf/1669498
http://www.python.org/sf/1633605
http://www.python.org/sf/1616422


----------------------------------------------------------------------

Comment By: Armin Rigo (arigo)
Date: 2007-03-28 13:40

Message:
Logged In: YES 
user_id=4771
Originator: YES

What I called "corruption" is the situation
where both the .py and the .pyc files are
present, but the filename stored in the .pyc
co_filenames is no longer the valid absolute
path of the corresponding .py file, for any
reason (renaming, NFS views, etc.).

This situation causes the tracebacks and the
inspect module to fail to locate the .py file,
which I consider a bug.

----------------------------------------------------------------------

Comment By: Martin v. L?wis (loewis)
Date: 2007-03-28 12:01

Message:
Logged In: YES 
user_id=21627
Originator: NO

I fail to see the corruption. It is quite desirable and normal to only
ship pyc files - that the file name they refer to is actually present is
not a requirement at all.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1180193&group_id=5470

From noreply at sourceforge.net  Tue Apr  3 14:13:52 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Tue, 03 Apr 2007 05:13:52 -0700
Subject: [ python-Bugs-1693546 ] email.Message set_param rfc2231 encoding
	incorrect
Message-ID: <E1HYhtY-0000S7-C2@sc8-sf-web22.sourceforge.net>

Bugs item #1693546, was opened at 2007-04-03 14:13
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1693546&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Anders Hammarquist (iko)
Assigned to: Nobody/Anonymous (nobody)
Summary: email.Message set_param rfc2231 encoding incorrect

Initial Comment:
Reading the ABNF for the extended parameter values in RFC2231 reveals that they should not be quoted:

   extended-initial-value := [charset] "'" [language] "'"
                             extended-other-values

   extended-other-values := *(ext-octet / attribute-char)

however, set_param (really _formatparam) will always quote them unless requote is set to false. The fix is probably to force quote in _formatparam to False if faced with RFC2231 values, since they should never be quoted (and should not contain characters requiring quoting),

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1693546&group_id=5470

From noreply at sourceforge.net  Tue Apr  3 15:46:13 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Tue, 03 Apr 2007 06:46:13 -0700
Subject: [ python-Bugs-1180193 ] broken pyc files
Message-ID: <E1HYjKv-0003xB-HZ@sc8-sf-web23.sourceforge.net>

Bugs item #1180193, was opened at 2005-04-10 15:10
Message generated for change (Comment added) made by zseil
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1180193&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Interpreter Core
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Armin Rigo (arigo)
Assigned to: Nobody/Anonymous (nobody)
Summary: broken pyc files

Initial Comment:
In a number of situations, the .pyc files can become "corrupted" in a subtle way: the co_filename attribute of the code objects it contains become wrong.  This can occur if we move or rename directories, or if we access the same set of files from two different locations (e.g. over NFS).

This corruption doesn't prevent the .pyc files from working, but the interpreter looses the reference to the source file.  It causes trouble in tracebacks, in the inspect module, etc.

A simple fix would be to use the following logic when importing a .py file: if there is a corresponding .pyc file, in addition to checking the timestamp, check the co_filename attribute of the loaded object.  If it doesn't point to the original .py file, discard the code object and ignore the .pyc file.

Alternatively, we could force all co_filenames to point to the .py file when loading the .pyc file.

I'll write a patch for whichever alternative seems better.

----------------------------------------------------------------------

>Comment By: Ziga Seilnacht (zseil)
Date: 2007-04-03 15:46

Message:
Logged In: YES 
user_id=1326842
Originator: NO

Wouldn't your first solution be simpler? Changing all
co_filenames would require either changing various
marhal.c functions, or traversing the code object
returned by import.c/read_compiled_module().

Discarding the compiled code when the file names don't
match would be simpler and only require minor changes
in import.c/load_source_module().


----------------------------------------------------------------------

Comment By: Armin Rigo (arigo)
Date: 2007-04-03 13:31

Message:
Logged In: YES 
user_id=4771
Originator: YES

If you ask me, I think that when the importing
system finds both a .py and a .pyc for a module,
then it should ignore all co_filename and replace
them with the real path of the .py file.  I can't
see any point of not doing so.

There are many other quirks caused by .pyc files
accidentally remaining around, but we cannot fix them
all as long as the .pyc files are at the same time
a cache for performance reason and a redistributable
program format (e.g. if "rm x.py" or "svn up" deletes
a .py file, then the module is still importable via
the .pyc left behind, a great way to oversee the fact
that imports elsewhere in the project need to be
updated).

----------------------------------------------------------------------

Comment By: Ziga Seilnacht (zseil)
Date: 2007-04-03 09:16

Message:
Logged In: YES 
user_id=1326842
Originator: NO

This problem is reported quite often in the tracker,
although it shows up in different places:

http://www.python.org/sf/1666807
http://www.python.org/sf/1051638

I closed those bugs as duplicates of this one.

The logging package is also affected:

http://www.python.org/sf/1669498
http://www.python.org/sf/1633605
http://www.python.org/sf/1616422


----------------------------------------------------------------------

Comment By: Armin Rigo (arigo)
Date: 2007-03-28 15:40

Message:
Logged In: YES 
user_id=4771
Originator: YES

What I called "corruption" is the situation
where both the .py and the .pyc files are
present, but the filename stored in the .pyc
co_filenames is no longer the valid absolute
path of the corresponding .py file, for any
reason (renaming, NFS views, etc.).

This situation causes the tracebacks and the
inspect module to fail to locate the .py file,
which I consider a bug.

----------------------------------------------------------------------

Comment By: Martin v. L?wis (loewis)
Date: 2007-03-28 14:01

Message:
Logged In: YES 
user_id=21627
Originator: NO

I fail to see the corruption. It is quite desirable and normal to only
ship pyc files - that the file name they refer to is actually present is
not a requirement at all.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1180193&group_id=5470

From noreply at sourceforge.net  Tue Apr  3 17:05:13 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Tue, 03 Apr 2007 08:05:13 -0700
Subject: [ python-Bugs-1669349 ] make install fails if no previous Python
	installation
Message-ID: <E1HYkZN-0004LA-T1@sc8-sf-web24.sourceforge.net>

Bugs item #1669349, was opened at 2007-02-26 15:22
Message generated for change (Comment added) made by sbassi
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1669349&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Installation
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Matthias S. Benkmann (mbenkmann)
Assigned to: Nobody/Anonymous (nobody)
Summary: make install fails if no previous Python installation

Initial Comment:
When installing Python 2.5 on a completely Python-less system 'make install' failed. The error that caused the failure was

Compiling /usr/lib/python2.5/test/test_multibytecodec.py ...
Sorry: UnicodeError: ("\\N escapes not supported (can't load unicodedata module)",)
[snip]
Compiling /usr/lib/python2.5/zipfile.py ...
make: *** [libinstall] Error 1

'find -name unicodedata.so' showed me that the library did not exist in the build tree. However, after a 'make -i install' to force make to continue despite the error, unicodedata.so was there. So apparently the library is not built until a later stage of make install. And indeed, subsequent 'make install' calls without -i were successful. 

It is important to note that if you have a previous Python installation (at least of the same version), 'make install' will go through, because it'll load the library from there. So if you want to reproduce the issue you will have to install from a freshly unpacked tarball on a system with no Python installation.



----------------------------------------------------------------------

Comment By: Sebastian Bassi (sbassi)
Date: 2007-04-03 12:05

Message:
Logged In: YES 
user_id=179653
Originator: NO

The same problem arises when installing Python using "make altinstall",
even with another Python installed. 

----------------------------------------------------------------------

Comment By: Simon Percivall (percivall)
Date: 2007-03-21 08:26

Message:
Logged In: YES 
user_id=329382
Originator: NO

It shouldn't be a problem changing the Makefile to do sharedinstall before
libinstall in the altinstall target. I'm guessing that would solve the
problem.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1669349&group_id=5470

From noreply at sourceforge.net  Tue Apr  3 17:50:10 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Tue, 03 Apr 2007 08:50:10 -0700
Subject: [ python-Bugs-1669349 ] make install fails if no previous Python
	installation
Message-ID: <E1HYlGs-0005Ul-LS@sc8-sf-web21.sourceforge.net>

Bugs item #1669349, was opened at 2007-02-26 15:22
Message generated for change (Comment added) made by sbassi
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1669349&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Installation
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Matthias S. Benkmann (mbenkmann)
Assigned to: Nobody/Anonymous (nobody)
Summary: make install fails if no previous Python installation

Initial Comment:
When installing Python 2.5 on a completely Python-less system 'make install' failed. The error that caused the failure was

Compiling /usr/lib/python2.5/test/test_multibytecodec.py ...
Sorry: UnicodeError: ("\\N escapes not supported (can't load unicodedata module)",)
[snip]
Compiling /usr/lib/python2.5/zipfile.py ...
make: *** [libinstall] Error 1

'find -name unicodedata.so' showed me that the library did not exist in the build tree. However, after a 'make -i install' to force make to continue despite the error, unicodedata.so was there. So apparently the library is not built until a later stage of make install. And indeed, subsequent 'make install' calls without -i were successful. 

It is important to note that if you have a previous Python installation (at least of the same version), 'make install' will go through, because it'll load the library from there. So if you want to reproduce the issue you will have to install from a freshly unpacked tarball on a system with no Python installation.



----------------------------------------------------------------------

Comment By: Sebastian Bassi (sbassi)
Date: 2007-04-03 12:50

Message:
Logged In: YES 
user_id=179653
Originator: NO

Another workaround (by Jaroslaw Zabiello):

After executing
./configure
you have to edito
Modules/Setup
file and uncomment the following line:
#zlib zlibmodule.c -I$(prefix)/include -L$(exec_prefix)/lib -lz
Then continue with normal make; make install.

----------------------------------------------------------------------

Comment By: Sebastian Bassi (sbassi)
Date: 2007-04-03 12:05

Message:
Logged In: YES 
user_id=179653
Originator: NO

The same problem arises when installing Python using "make altinstall",
even with another Python installed. 

----------------------------------------------------------------------

Comment By: Simon Percivall (percivall)
Date: 2007-03-21 08:26

Message:
Logged In: YES 
user_id=329382
Originator: NO

It shouldn't be a problem changing the Makefile to do sharedinstall before
libinstall in the altinstall target. I'm guessing that would solve the
problem.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1669349&group_id=5470

From noreply at sourceforge.net  Tue Apr  3 18:17:31 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Tue, 03 Apr 2007 09:17:31 -0700
Subject: [ python-Bugs-1693753 ] Portability issie: os.rename behaves
	differently on win32
Message-ID: <E1HYlhL-0004cv-N0@sc8-sf-web23.sourceforge.net>

Bugs item #1693753, was opened at 2007-04-03 23:17
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1693753&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Platform-specific
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: cs_ (cs_)
Assigned to: Nobody/Anonymous (nobody)
Summary: Portability issie: os.rename behaves differently on win32

Initial Comment:
os.rename(src, dst) fails on Windows if dst already exist and regular file. On Unixes, it does not fail, and silently overwrites dst. It's better to use MoveFileEx(src, dst, MOVEFILE_REPLACE_EXISTING) when moving regular files under Win32 to make it behave consistent way.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1693753&group_id=5470

From noreply at sourceforge.net  Tue Apr  3 21:55:41 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Tue, 03 Apr 2007 12:55:41 -0700
Subject: [ python-Bugs-1657034 ] 'Ok' key in options dialog does nothing
Message-ID: <E1HYp6T-0002mp-Us@sc8-sf-web21.sourceforge.net>

Bugs item #1657034, was opened at 2007-02-11 01:35
Message generated for change (Comment added) made by torhu
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1657034&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: IDLE
Group: Python 2.5
>Status: Closed
>Resolution: Works For Me
Priority: 5
Private: No
Submitted By: torhu (torhu)
Assigned to: Nobody/Anonymous (nobody)
Summary: 'Ok' key in options dialog does nothing

Initial Comment:
IDLE 1.2, winxp sp2 US.

I get the same results with idlelib from svn rev. 53721.


Steps to reproduce:
1. Open Options -> Configure IDLE...
2. Click on the Ok button.
3. Nothing happens, you have to click Cancel to close the options window.


Here's a way that crashes IDLE on my machine:
1. Open Options -> Configure IDLE...
2. Set Indentation width to 8 (was 4 by default).
3. Click on Apply.
4. Click on Ok (nothing happens).
5. Click on Cancel - IDLE exits.


Workaround:
Replace the Python25/Lib/idlelib directory with the one that comes with python 2.4.3.

----------------------------------------------------------------------

>Comment By: torhu (torhu)
Date: 2007-04-03 21:55

Message:
Logged In: YES 
user_id=1038085
Originator: YES

Reinstalling Windows seems to have fixed the problem for me.  Not sure
what caused it.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1657034&group_id=5470

From noreply at sourceforge.net  Tue Apr  3 23:47:53 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Tue, 03 Apr 2007 14:47:53 -0700
Subject: [ python-Bugs-1686597 ] descrintro: error describing __new__ behavior
Message-ID: <E1HYqr3-0004qK-3a@sc8-sf-web24.sourceforge.net>

Bugs item #1686597, was opened at 2007-03-23 03:47
Message generated for change (Comment added) made by bediviere
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1686597&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Documentation
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Gabriel Genellina (gagenellina)
Assigned to: Nobody/Anonymous (nobody)
Summary: descrintro: error describing __new__ behavior

Initial Comment:
descrintro (http://www.python.org/download/releases/2.2.3/descrintro/#__new__) linked from http://www.python.org/doc/newstyle/ still says:

"__new__ must return an object. [...] If you return an object of a different class, its __init__ method will be called."

But since Revision 26220 - Modified Sat Apr 6 01:05:01 2002 UTC (4 years, 11 months ago) by gvanrossum:

"Changed new-style class instantiation so that when C's __new__ method returns something that's not a C instance, its __init__ is not called.  [SF bug #537450]"

That is, exactly the opposite as said on descrintro. The documentation for __new__ in the Reference manual, section 3.4.1, is correct and says: "If __new__() does not return an instance of cls, then the new instance's __init__() method will not be invoked."

Note that the modified behavior was already present on version 2.2.3 (and later) so updating the document currently at /download/releases/2.2.3/ would be fine.



----------------------------------------------------------------------

Comment By: Steven Bethard (bediviere)
Date: 2007-04-03 15:47

Message:
Logged In: YES 
user_id=945502
Originator: NO

Note that this is correct in the documentation:
http://docs.python.org/ref/customization.html

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1686597&group_id=5470

From noreply at sourceforge.net  Wed Apr  4 00:01:13 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Tue, 03 Apr 2007 15:01:13 -0700
Subject: [ python-Bugs-1684991 ] Explain __method__ lookup semantics for
	new-style classes
Message-ID: <E1HYr3x-0006tx-Eh@sc8-sf-web23.sourceforge.net>

Bugs item #1684991, was opened at 2007-03-21 02:53
Message generated for change (Comment added) made by bediviere
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1684991&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Documentation
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Georg Brandl (gbrandl)
Assigned to: Nobody/Anonymous (nobody)
Summary: Explain __method__ lookup semantics for new-style classes

Initial Comment:
__method__s (and next) are looked up on the type, not the instance. This isn't documented properly.

----------------------------------------------------------------------

Comment By: Steven Bethard (bediviere)
Date: 2007-04-03 16:01

Message:
Logged In: YES 
user_id=945502
Originator: NO

This is not true for all __special__ methods, e.g. __enter__ and
__exit__:

>>> class C(object):
...     pass
...    
>>> def enter(*args):
...     print 'enter', args
...     
>>> def exit(*args):
...     print 'exit', args
...     
>>> c = C()
>>> c.__enter__ = enter
>>> c.__exit__ = exit
>>> with c:
...     print 'hi'
...     
enter ()
hi
exit (None, None, None)

The documentation should say something like "When interpreting syntax that
invokes a __special__ method, Python looks for the __special__ method on
the instance's class. As an implementation detail, the lookup for some
__special__ methods may also check the instance first, but this behavior
should not be relied upon." This should probably go into the Reference
Manual section 3.4: http://docs.python.org/ref/specialnames.html

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1684991&group_id=5470

From noreply at sourceforge.net  Wed Apr  4 00:44:09 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Tue, 03 Apr 2007 15:44:09 -0700
Subject: [ python-Bugs-1693941 ] Puzzled over list append bvehavior
Message-ID: <E1HYrjV-0004Nk-WD@sc8-sf-web21.sourceforge.net>

Bugs item #1693941, was opened at 2007-04-03 15:44
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1693941&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: Python 2.6
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: elgordo (azgordo)
Assigned to: Nobody/Anonymous (nobody)
Summary: Puzzled over list append bvehavior

Initial Comment:
See attached IDLE session

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1693941&group_id=5470

From noreply at sourceforge.net  Wed Apr  4 00:47:52 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Tue, 03 Apr 2007 15:47:52 -0700
Subject: [ python-Bugs-1693941 ] Puzzled over list append bvehavior
Message-ID: <E1HYrn6-0004ix-Vz@sc8-sf-web21.sourceforge.net>

Bugs item #1693941, was opened at 2007-04-03 17:44
Message generated for change (Comment added) made by rhettinger
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1693941&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: Python 2.6
>Status: Closed
>Resolution: Invalid
Priority: 5
Private: No
Submitted By: elgordo (azgordo)
Assigned to: Nobody/Anonymous (nobody)
Summary: Puzzled over list append bvehavior

Initial Comment:
See attached IDLE session

----------------------------------------------------------------------

>Comment By: Raymond Hettinger (rhettinger)
Date: 2007-04-03 17:47

Message:
Logged In: YES 
user_id=80475
Originator: NO

Sorry, this isn't a bug.  The append method modifies the list in-place and
returns None.  

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1693941&group_id=5470

From noreply at sourceforge.net  Wed Apr  4 00:55:19 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Tue, 03 Apr 2007 15:55:19 -0700
Subject: [ python-Bugs-1647489 ] zero-length match confuses re.finditer()
Message-ID: <E1HYruJ-0005ac-A5@sc8-sf-web21.sourceforge.net>

Bugs item #1647489, was opened at 2007-01-29 17:35
Message generated for change (Settings changed) made by rhettinger
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1647489&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Regular Expressions
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Jacques Frechet (jfrechet)
>Assigned to: Gustavo Niemeyer (niemeyer)
Summary: zero-length match confuses re.finditer()

Initial Comment:
Hi!

re.finditer() seems to incorrectly increment the current position immediately after matching a zero-length substring.  For example:

>>> [m.groups() for m in re.finditer(r'(^z*)|(\w+)', 'abc')]
[('', None), (None, 'bc')]

What happened to the 'a'?  I expected this result:

[('', None), (None, 'abc')]

Perl agrees with me:

% perl -le 'print defined($1)?"\"$1\"":"undef",",",defined($2)?"\"$2\"":"undef" while "abc" =~ /(z*)|(\w+)/g' 
"",undef
undef,"abc"
"",undef

Similarly, if I remove the ^:

>>> [m.groups() for m in re.finditer(r'(z*)|(\w+)', 'abc')]
[('', None), ('', None), ('', None), ('', None)]

Now all of the letters have fallen through the cracks!  I expected this result:

[('', None), (None, 'abc'), ('', None)]

Again, perl agrees:

% perl -le 'print defined($1)?"\"$1\"":"undef",",",defined($2)?"\"$2\"":"undef" while "abc" =~ /(z*)|(\w+)/g' 
"",undef
undef,"abc"
"",undef

If this bug has already been reported, I apologize -- I wasn't able to find it here.  I haven't looked at the code for the re module, but this seems like the sort of bug that might have been accidentally introduced in order to try to prevent the same zero-length match from being returned forever.

Thanks,
Jacques

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1647489&group_id=5470

From noreply at sourceforge.net  Wed Apr  4 01:37:12 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Tue, 03 Apr 2007 16:37:12 -0700
Subject: [ python-Bugs-1514428 ] NaN comparison in Decimal broken
Message-ID: <E1HYsYq-0003mi-Bd@sc8-sf-web22.sourceforge.net>

Bugs item #1514428, was opened at 2006-06-29 11:19
Message generated for change (Settings changed) made by rhettinger
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1514428&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.5
Status: Open
Resolution: None
>Priority: 4
Private: No
Submitted By: Nick Maclaren (nmm)
Assigned to: Tim Peters (tim_one)
Summary: NaN comparison in Decimal broken

Initial Comment:
Methinks this is a bit off :-)  True should be False.


Python 2.5b1 (trunk:47059, Jun 29 2006, 14:26:46)
[GCC 4.1.0 (SUSE Linux)] on linux2
>>> import decimal
>>> d = decimal.Decimal
>>> inf = d("inf")
>>> nan = d("nan")
>>> nan > inf
True
>>> nan < inf
False
>>> inf > nan
True
>>> inf < nan
False
b

----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2007-01-05 21:05

Message:
Logged In: YES 
user_id=80475
Originator: NO

The Decimal Arithmetic Specification says that NaN comparisons should
return NaN.  The decimal module correctly implements this through the
compare() method:

    >>> nan.compare(nan)
    Decimal('NaN')

Since python's < and > operators return a boolean result, the standard is
silent on what should be done.  The current implementation uses the __cmp__
method which can only return -1, 0, or 1, so there is not a direct way to
make both < and > both return False.

If you want to go beyond the standard and have both < and > return False
for all NaN comparisons, then the __cmp__ implementation would need to be
replaced with rich comparisons.  I'm not sure that this is desirable.  IMO,
that would be no better than the current arbitrary choice where all
comparisons involving NaN report self > other.  If someone has an
application that would be harmed by the current implementation, then it
should almost certainly be use the standard compliant compare() method
instead of the boolean < and > operators.

Tim, what say you?

----------------------------------------------------------------------

Comment By: CharlesMerriam (charlesmerriam)
Date: 2006-08-23 03:43

Message:
Logged In: YES 
user_id=1581732

More specifically, any comparison with a NaN should equal
False, even inf, per IEEE 754.  A good starting point to
convince oneself of this is http://en.wikipedia.org/wiki/NaN.


----------------------------------------------------------------------

Comment By: Nick Maclaren (nmm)
Date: 2006-07-13 05:35

Message:
Logged In: YES 
user_id=42444

It's still there in Beta 2.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1514428&group_id=5470

From noreply at sourceforge.net  Wed Apr  4 01:38:41 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Tue, 03 Apr 2007 16:38:41 -0700
Subject: [ python-Bugs-1519638 ] Unmatched Group issue
Message-ID: <E1HYsaH-0003tk-VO@sc8-sf-web22.sourceforge.net>

Bugs item #1519638, was opened at 2006-07-09 13:34
Message generated for change (Settings changed) made by rhettinger
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1519638&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Regular Expressions
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Bobby Xiao (nneonneo)
>Assigned to: Fredrik Lundh (effbot)
Summary: Unmatched Group issue

Initial Comment:
Using sre.sub[n], an "unmatched group" error can occur.

The test I used is this pattern:

sre.sub("foo(?:b(ar)|baz)","\\1","foobaz")

This will cause the following backtrace to occur:

Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "lib/python2.4/sre.py", line 142, in sub
    return _compile(pattern, 0).sub(repl, string, count)
  File "lib/python2.4/sre.py", line 260, in filter
    return sre_parse.expand_template(template, match)
  File "lib/python2.4/sre_parse.py", line 782, in expand_template
    raise error, "unmatched group"
sre_constants.error: unmatched group

Python Version 2.4.3, Mac OS X (behaviour has been verified on 
Windows 2.4.3 as well).

This behaviour, while by design, is unwanted because this type of 
matching usually requests that a blank match be returned (i.e. the 
example should return '')

The example that I was trying resembles the following:

sre.sub("User: (?:Registered User #(\d+)|Guest)","%USERID|\1%",data)

The intended behaviour is that the function returns "" when the user is 
a guest and the user number if the user is a registered member.

However, when this function encounters a Guest, it raises an exception 
and terminates, which is not what is wanted.

Perl and other regex engines behave as I have described, substituting 
empty strings for unmatched groups. The code fix is relatively simple, 
and would really help out for these types of things.

----------------------------------------------------------------------

Comment By: Bobby Xiao (nneonneo)
Date: 2007-02-16 21:56

Message:
Logged In: YES 
user_id=393491
Originator: YES

AFAIK the findall function works as desired in this respect: empty matches
will return empty strings.

----------------------------------------------------------------------

Comment By: Matt Chaput (mchaput)
Date: 2007-02-15 13:35

Message:
Logged In: YES 
user_id=1249840
Originator: NO

The current behavior also makes the "sub" function useless when you need
to backreference a group that might not capture, since you have no chance
to deal with the exception.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1519638&group_id=5470

From noreply at sourceforge.net  Wed Apr  4 02:21:24 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Tue, 03 Apr 2007 17:21:24 -0700
Subject: [ python-Bugs-1684991 ] Explain __method__ lookup semantics for
	new-style classes
Message-ID: <E1HYtFc-0008Sd-5P@sc8-sf-web22.sourceforge.net>

Bugs item #1684991, was opened at 2007-03-21 03:53
Message generated for change (Comment added) made by rhettinger
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1684991&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Documentation
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Georg Brandl (gbrandl)
Assigned to: Nobody/Anonymous (nobody)
Summary: Explain __method__ lookup semantics for new-style classes

Initial Comment:
__method__s (and next) are looked up on the type, not the instance. This isn't documented properly.

----------------------------------------------------------------------

>Comment By: Raymond Hettinger (rhettinger)
Date: 2007-04-03 19:21

Message:
Logged In: YES 
user_id=80475
Originator: NO

When writing docs, always cover the general case first and thoroughly.
Keep the recommendations constructive, positive and useful (as opposed
to wording like "this behavior should not be relied upon".  A doc
patch needs to add clarity -- if it doesn't, leave it out.

----------------------------------------------------------------------

Comment By: Steven Bethard (bediviere)
Date: 2007-04-03 17:01

Message:
Logged In: YES 
user_id=945502
Originator: NO

This is not true for all __special__ methods, e.g. __enter__ and
__exit__:

>>> class C(object):
...     pass
...    
>>> def enter(*args):
...     print 'enter', args
...     
>>> def exit(*args):
...     print 'exit', args
...     
>>> c = C()
>>> c.__enter__ = enter
>>> c.__exit__ = exit
>>> with c:
...     print 'hi'
...     
enter ()
hi
exit (None, None, None)

The documentation should say something like "When interpreting syntax that
invokes a __special__ method, Python looks for the __special__ method on
the instance's class. As an implementation detail, the lookup for some
__special__ methods may also check the instance first, but this behavior
should not be relied upon." This should probably go into the Reference
Manual section 3.4: http://docs.python.org/ref/specialnames.html

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1684991&group_id=5470

From noreply at sourceforge.net  Wed Apr  4 02:23:07 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Tue, 03 Apr 2007 17:23:07 -0700
Subject: [ python-Bugs-1649098 ] non-standard: array[0]
Message-ID: <E1HYtHH-0000EA-Si@sc8-sf-web22.sourceforge.net>

Bugs item #1649098, was opened at 2007-01-31 13:25
Message generated for change (Settings changed) made by rhettinger
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1649098&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: Python 2.5
Status: Open
Resolution: None
>Priority: 6
Private: No
Submitted By: Johannes Abt (jabt)
Assigned to: Thomas Heller (theller)
Summary: non-standard: array[0]

Initial Comment:
in Modules/_ctypes/ctypes.h:

typedef struct {
[..]
        ffi_type *atypes[0];
} ffi_info;

AFAIK, arrays must be of size > 0.
_Most_ compilers accepts this, but not all
(especially my HP-UX compiler).

Please change *atypes[0] to *atypes[1]!

Bye,
Johannes


----------------------------------------------------------------------

Comment By: Thomas Heller (theller)
Date: 2007-02-28 15:43

Message:
Logged In: YES 
user_id=11105
Originator: NO

I can, and probably will, change this in Modules/_ctypes/ctypes.h. 
However, I'm afraid it will not help the OP too much because it seems he
cannot sucessfully compile the libffi sources with this HP-UX compiler
anyway because of at least *some* other problems.

I have the impression that libffi is GCC-specific, and unless someone
provides a complete patch for the HP-UX (or other) compilers it will
probably stay this way.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2007-02-25 17:24

Message:
Logged In: YES 
user_id=33168
Originator: NO

Thomas, could you take a look?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1649098&group_id=5470

From noreply at sourceforge.net  Wed Apr  4 02:24:59 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Tue, 03 Apr 2007 17:24:59 -0700
Subject: [ python-Bugs-1594966 ] doctest simple usage recipe is misleading
Message-ID: <E1HYtJ5-0000Lr-LF@sc8-sf-web22.sourceforge.net>

Bugs item #1594966, was opened at 2006-11-12 03:31
Message generated for change (Settings changed) made by rhettinger
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1594966&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Documentation
Group: Python 2.5
Status: Open
Resolution: None
>Priority: 2
Private: No
Submitted By: Ken Rimey (yemir)
Assigned to: Tim Peters (tim_one)
Summary: doctest simple usage recipe is misleading

Initial Comment:

"23.2.1 Simple Usage: Checking Examples in Docstrings" sets up a trap 
for the unsuspecting developer:

    http://docs.python.org/lib/doctest-simple-testmod.html

That page recommends adding the following code to the end of a 
module using doctest:

    def _test():
        import doctest
        doctest.testmod()

    if __name__ == "__main__":
        _test()

The problem is that a reasonable person will figure that _test() has 
been defined for convenience in executing the tests from other Python 
code as follows:

    import M
    M._test()

However, that executes the doctests found in __main__, not M!

I think the recommended recipe should instead be as follows:

    if __name__ == "__main__":
        import doctest
        doctest.testmod()


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1594966&group_id=5470

From noreply at sourceforge.net  Wed Apr  4 02:45:20 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Tue, 03 Apr 2007 17:45:20 -0700
Subject: [ python-Bugs-1563759 ] struct.unpack doens't support buffer protocol
	objects
Message-ID: <E1HYtcm-0002ZS-O6@sc8-sf-web22.sourceforge.net>

Bugs item #1563759, was opened at 2006-09-22 17:17
Message generated for change (Comment added) made by rhettinger
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1563759&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.5
Status: Open
Resolution: None
>Priority: 7
Private: No
Submitted By: Adal Chiriliuc (adalx)
Assigned to: Nobody/Anonymous (nobody)
Summary: struct.unpack doens't support buffer protocol objects

Initial Comment:
If you pass an object which supports the buffer
protocol to struct.unpack it will fail because it
specifically checks for a string.

You should use PyObject_AsReadBuffer instead.

If this code is performance critical, you could add an
unpack_buffer method or something like that.

----------------------------------------------------------------------

>Comment By: Raymond Hettinger (rhettinger)
Date: 2007-04-03 19:45

Message:
Logged In: YES 
user_id=80475
Originator: NO

This was due to Bob Ippolito's commit in revision 46184.

----------------------------------------------------------------------

Comment By: Adal Chiriliuc (adalx)
Date: 2006-09-23 07:57

Message:
Logged In: YES 
user_id=1067739

test_struct_run.py works in 2.4, throws exception in 2.5

----------------------------------------------------------------------

Comment By: Adal Chiriliuc (adalx)
Date: 2006-09-23 07:56

Message:
Logged In: YES 
user_id=1067739

The actual code which broke used a Pointer extension class
implemented in C++.

I reproduced the problem using array.array. Using array in
this way (without calling tostring) looks a bit weird.


----------------------------------------------------------------------

Comment By: Martin v. L?wis (loewis)
Date: 2006-09-23 07:12

Message:
Logged In: YES 
user_id=21627

Ah, so you say that was working previously? It's a bug,
then. Can you provide a test case?

----------------------------------------------------------------------

Comment By: Adal Chiriliuc (adalx)
Date: 2006-09-23 06:37

Message:
Logged In: YES 
user_id=1067739

Well, because it broke previously working code and there is
no warning in the documentation about that.

In the mean-time, I've found out about pack_into and
unpack_from which accept buffer like objects. Note that they
are not documented in the struct module section, only
mentioned in the "What's new" chapter.

----------------------------------------------------------------------

Comment By: Martin v. L?wis (loewis)
Date: 2006-09-23 06:19

Message:
Logged In: YES 
user_id=21627

Why is this a bug?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1563759&group_id=5470

From noreply at sourceforge.net  Wed Apr  4 03:13:56 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Tue, 03 Apr 2007 18:13:56 -0700
Subject: [ python-Bugs-1684991 ] Explain __method__ lookup semantics for
	new-style classes
Message-ID: <E1HYu4S-0003mj-LI@sc8-sf-web21.sourceforge.net>

Bugs item #1684991, was opened at 2007-03-21 02:53
Message generated for change (Comment added) made by bediviere
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1684991&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Documentation
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Georg Brandl (gbrandl)
Assigned to: Nobody/Anonymous (nobody)
Summary: Explain __method__ lookup semantics for new-style classes

Initial Comment:
__method__s (and next) are looked up on the type, not the instance. This isn't documented properly.

----------------------------------------------------------------------

Comment By: Steven Bethard (bediviere)
Date: 2007-04-03 19:13

Message:
Logged In: YES 
user_id=945502
Originator: NO

Not sure exactly what you're proposing Raymond, but you could cut mine to
"When interpreting syntax that invokes a __special__ method, Python looks
for the __special__ method on the instance's class" if that's what you
want.

----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2007-04-03 18:21

Message:
Logged In: YES 
user_id=80475
Originator: NO

When writing docs, always cover the general case first and thoroughly.
Keep the recommendations constructive, positive and useful (as opposed
to wording like "this behavior should not be relied upon".  A doc
patch needs to add clarity -- if it doesn't, leave it out.

----------------------------------------------------------------------

Comment By: Steven Bethard (bediviere)
Date: 2007-04-03 16:01

Message:
Logged In: YES 
user_id=945502
Originator: NO

This is not true for all __special__ methods, e.g. __enter__ and
__exit__:

>>> class C(object):
...     pass
...    
>>> def enter(*args):
...     print 'enter', args
...     
>>> def exit(*args):
...     print 'exit', args
...     
>>> c = C()
>>> c.__enter__ = enter
>>> c.__exit__ = exit
>>> with c:
...     print 'hi'
...     
enter ()
hi
exit (None, None, None)

The documentation should say something like "When interpreting syntax that
invokes a __special__ method, Python looks for the __special__ method on
the instance's class. As an implementation detail, the lookup for some
__special__ methods may also check the instance first, but this behavior
should not be relied upon." This should probably go into the Reference
Manual section 3.4: http://docs.python.org/ref/specialnames.html

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1684991&group_id=5470

From noreply at sourceforge.net  Wed Apr  4 08:45:03 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Tue, 03 Apr 2007 23:45:03 -0700
Subject: [ python-Bugs-1690840 ] xmlrpclib methods submit call on __str__,
	__repr__
Message-ID: <E1HYzEt-0005HR-RW@sc8-sf-web21.sourceforge.net>

Bugs item #1690840, was opened at 2007-03-29 19:14
Message generated for change (Comment added) made by ghazel
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1690840&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Greg Hazel (ghazel)
Assigned to: Nobody/Anonymous (nobody)
Summary: xmlrpclib methods submit call on __str__, __repr__

Initial Comment:
Notice how trying to print a method causes it to submit the call:


Python 2.5 (r25:51908, Sep 19 2006, 09:52:17) [MSC v.1310 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from xmlrpclib import ServerProxy
>>> s = ServerProxy("http://google.com")
>>> print s.somecall
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "c:\python25\lib\xmlrpclib.py", line 1147, in __call__
    return self.__send(self.__name, args)
  File "c:\python25\lib\xmlrpclib.py", line 1437, in __request
    verbose=self.__verbose
  File "c:\python25\lib\xmlrpclib.py", line 1191, in request
    headers
xmlrpclib.ProtocolError: <ProtocolError for google.com/RPC2: 501 Not Implemented


Similarly:

>>> f = s.somecall
>>> locals()
{'f': Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "c:\python25\lib\xmlrpclib.py", line 1147, in __call__
    return self.__send(self.__name, args)
  File "c:\python25\lib\xmlrpclib.py", line 1437, in __request
    verbose=self.__verbose
  File "c:\python25\lib\xmlrpclib.py", line 1191, in request
    headers
xmlrpclib.ProtocolError: <ProtocolError for google.com/RPC2: 501 Not Implemented>



----------------------------------------------------------------------

>Comment By: Greg Hazel (ghazel)
Date: 2007-04-04 06:45

Message:
Logged In: YES 
user_id=731668
Originator: YES

This would be reasonable:

>>> from xmlrpclib import ServerProxy
>>> s = ServerProxy("http://google.com")
>>> print s.somecall
<xmlrpclib._Method object at 0x00C08470>

Similarly:

>>> f = s.somecall
>>> locals()
{'f': <xmlrpclib._Method object at 0x00BA8030>, '__builtins__': <module
'__built
in__' (built-in)>, 's': <ServerProxy for google.com/RPC2>, '__name__':
'__main__
', 'ServerProxy': <class xmlrpclib.ServerProxy at 0x00C61AE0>, '__doc__':
None}

Bonus points for:
<xmlrpclib _Method somecall of <ServerProxy for google.com/RPC2>>

----------------------------------------------------------------------

Comment By: Collin Winter (collinwinter)
Date: 2007-03-30 02:59

Message:
Logged In: YES 
user_id=1344176
Originator: NO

What would you rather seem them print?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1690840&group_id=5470

From noreply at sourceforge.net  Wed Apr  4 12:03:57 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Wed, 04 Apr 2007 03:03:57 -0700
Subject: [ python-Bugs-1694155 ] Python 2.5 installer ended prematurely
Message-ID: <E1HZ2LN-0001T4-JG@sc8-sf-web21.sourceforge.net>

Bugs item #1694155, was opened at 2007-04-04 10:03
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1694155&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Installation
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Miclav (miclav)
Assigned to: Nobody/Anonymous (nobody)
Summary: Python 2.5 installer ended prematurely

Initial Comment:
Hello,

I could not install Python 2.5 on 2 machines running Windows XP. I wanted to install it into P:\Programs\languages\python

When I run it from P:\Proginst\languages\python, the installation fails very early : just after the "Customize Python 2.5" panel, when I click on "Next", I get almost instantly a panel "Python 2.5 installer ended prematurely" with a Finish button to exit installer.

When I run the msi file from C:\ as indicated in other bug report on the same subject, I can go further but it fails also : I get first a panel "Install Python 2.5", and some files are copied (as implied by the green status bar) but after a while it stops with a panel saying "The specified path is too long : P:\Programs" and 2 buttons to retry or cancel installation. Retry brings again to this panel so I must cancel. 

I was unable to find a solution, and I generated a python.log file (for the installation from C:\) that I join as attached file.

 On a third machine with Windows 2000 I succeeded in installing Python 2.5 in P:\Programs\languages\python from P:\Proginst\languages\python, so maybe the problem comes from WinXP? In both cases (XP and 2000) I use Zone Alarm as firewall (for XP, with XP firewall desactivated).

The only solution I found up to now is to install Python 2.2 (which uses another installer). I would be very happy if you could switch to an Open Source installation program (not commercial) so that everything in the behaviour of installation program can be known and under control. From the python.log file, I suspect that msi installer in WinXP might use some spy program, or it may in future versions of Windows. The installation with msi installer is much too fragile, and it might make fragile all Open Source programs based on Python : before trying to install Python, I had  tried to install the programm Leo, which is an editor written in Python that installs Python as a first step if it is not already installed. And I failed also to install Leo, with the same error (path P:\programs too long").

I would suggest to switch to NSIS ("Nullsoft Installer"), which is Open Source. I use it to distribute my own programs, and I found it excellent. I found further that many other Open Source programs are using this installer, and it always worked like a charm on all my machines. It seems much more robust that msi installer.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1694155&group_id=5470

From noreply at sourceforge.net  Wed Apr  4 17:10:25 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Wed, 04 Apr 2007 08:10:25 -0700
Subject: [ python-Bugs-1690840 ] xmlrpclib methods submit call on __str__,
	__repr__
Message-ID: <E1HZ77x-0002oX-Hi@sc8-sf-web21.sourceforge.net>

Bugs item #1690840, was opened at 2007-03-29 15:14
Message generated for change (Comment added) made by collinwinter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1690840&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Greg Hazel (ghazel)
>Assigned to: Fredrik Lundh (effbot)
Summary: xmlrpclib methods submit call on __str__, __repr__

Initial Comment:
Notice how trying to print a method causes it to submit the call:


Python 2.5 (r25:51908, Sep 19 2006, 09:52:17) [MSC v.1310 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from xmlrpclib import ServerProxy
>>> s = ServerProxy("http://google.com")
>>> print s.somecall
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "c:\python25\lib\xmlrpclib.py", line 1147, in __call__
    return self.__send(self.__name, args)
  File "c:\python25\lib\xmlrpclib.py", line 1437, in __request
    verbose=self.__verbose
  File "c:\python25\lib\xmlrpclib.py", line 1191, in request
    headers
xmlrpclib.ProtocolError: <ProtocolError for google.com/RPC2: 501 Not Implemented


Similarly:

>>> f = s.somecall
>>> locals()
{'f': Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "c:\python25\lib\xmlrpclib.py", line 1147, in __call__
    return self.__send(self.__name, args)
  File "c:\python25\lib\xmlrpclib.py", line 1437, in __request
    verbose=self.__verbose
  File "c:\python25\lib\xmlrpclib.py", line 1191, in request
    headers
xmlrpclib.ProtocolError: <ProtocolError for google.com/RPC2: 501 Not Implemented>



----------------------------------------------------------------------

>Comment By: Collin Winter (collinwinter)
Date: 2007-04-04 11:10

Message:
Logged In: YES 
user_id=1344176
Originator: NO

Fredrik, you originally wrote xmlrpclib: any objections to stringifying
_Method objects to something like "<xmlrpclib _Method somecall at
0x4636e346>"?

----------------------------------------------------------------------

Comment By: Greg Hazel (ghazel)
Date: 2007-04-04 02:45

Message:
Logged In: YES 
user_id=731668
Originator: YES

This would be reasonable:

>>> from xmlrpclib import ServerProxy
>>> s = ServerProxy("http://google.com")
>>> print s.somecall
<xmlrpclib._Method object at 0x00C08470>

Similarly:

>>> f = s.somecall
>>> locals()
{'f': <xmlrpclib._Method object at 0x00BA8030>, '__builtins__': <module
'__built
in__' (built-in)>, 's': <ServerProxy for google.com/RPC2>, '__name__':
'__main__
', 'ServerProxy': <class xmlrpclib.ServerProxy at 0x00C61AE0>, '__doc__':
None}

Bonus points for:
<xmlrpclib _Method somecall of <ServerProxy for google.com/RPC2>>

----------------------------------------------------------------------

Comment By: Collin Winter (collinwinter)
Date: 2007-03-29 22:59

Message:
Logged In: YES 
user_id=1344176
Originator: NO

What would you rather seem them print?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1690840&group_id=5470

From noreply at sourceforge.net  Wed Apr  4 17:11:16 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Wed, 04 Apr 2007 08:11:16 -0700
Subject: [ python-Bugs-1693753 ] Portability issue: os.rename behaves
	differently on win32
Message-ID: <E1HZ78m-0003cF-7P@sc8-sf-web21.sourceforge.net>

Bugs item #1693753, was opened at 2007-04-03 12:17
Message generated for change (Settings changed) made by collinwinter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1693753&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Platform-specific
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: cs_ (cs_)
Assigned to: Nobody/Anonymous (nobody)
>Summary: Portability issue: os.rename behaves differently on win32

Initial Comment:
os.rename(src, dst) fails on Windows if dst already exist and regular file. On Unixes, it does not fail, and silently overwrites dst. It's better to use MoveFileEx(src, dst, MOVEFILE_REPLACE_EXISTING) when moving regular files under Win32 to make it behave consistent way.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1693753&group_id=5470

From noreply at sourceforge.net  Wed Apr  4 18:55:39 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Wed, 04 Apr 2007 09:55:39 -0700
Subject: [ python-Bugs-1694155 ] Python 2.5 installer ended prematurely
Message-ID: <E1HZ8ln-00075s-AE@sc8-sf-web21.sourceforge.net>

Bugs item #1694155, was opened at 2007-04-04 05:03
Message generated for change (Settings changed) made by rhettinger
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1694155&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Installation
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Miclav (miclav)
>Assigned to: Martin v. L?wis (loewis)
Summary: Python 2.5 installer ended prematurely

Initial Comment:
Hello,

I could not install Python 2.5 on 2 machines running Windows XP. I wanted to install it into P:\Programs\languages\python

When I run it from P:\Proginst\languages\python, the installation fails very early : just after the "Customize Python 2.5" panel, when I click on "Next", I get almost instantly a panel "Python 2.5 installer ended prematurely" with a Finish button to exit installer.

When I run the msi file from C:\ as indicated in other bug report on the same subject, I can go further but it fails also : I get first a panel "Install Python 2.5", and some files are copied (as implied by the green status bar) but after a while it stops with a panel saying "The specified path is too long : P:\Programs" and 2 buttons to retry or cancel installation. Retry brings again to this panel so I must cancel. 

I was unable to find a solution, and I generated a python.log file (for the installation from C:\) that I join as attached file.

 On a third machine with Windows 2000 I succeeded in installing Python 2.5 in P:\Programs\languages\python from P:\Proginst\languages\python, so maybe the problem comes from WinXP? In both cases (XP and 2000) I use Zone Alarm as firewall (for XP, with XP firewall desactivated).

The only solution I found up to now is to install Python 2.2 (which uses another installer). I would be very happy if you could switch to an Open Source installation program (not commercial) so that everything in the behaviour of installation program can be known and under control. From the python.log file, I suspect that msi installer in WinXP might use some spy program, or it may in future versions of Windows. The installation with msi installer is much too fragile, and it might make fragile all Open Source programs based on Python : before trying to install Python, I had  tried to install the programm Leo, which is an editor written in Python that installs Python as a first step if it is not already installed. And I failed also to install Leo, with the same error (path P:\programs too long").

I would suggest to switch to NSIS ("Nullsoft Installer"), which is Open Source. I use it to distribute my own programs, and I found it excellent. I found further that many other Open Source programs are using this installer, and it always worked like a charm on all my machines. It seems much more robust that msi installer.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1694155&group_id=5470

From noreply at sourceforge.net  Wed Apr  4 19:11:28 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Wed, 04 Apr 2007 10:11:28 -0700
Subject: [ python-Bugs-1368768 ] clearing up dictionary keys/set member docs
Message-ID: <E1HZ916-0004bM-Tq@sc8-sf-web24.sourceforge.net>

Bugs item #1368768, was opened at 2005-11-29 02:40
Message generated for change (Comment added) made by sonderblade
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1368768&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Documentation
Group: Feature Request
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Mike Meyer (mwm)
Assigned to: Nobody/Anonymous (nobody)
Summary: clearing up dictionary keys/set member docs

Initial Comment:
The documentation for dictionaries says "A dictionary's keys are 
almost arbitrary values. Only values containing lists, dictionaries or 
other mutable types (that are compared by value rather than by 
object identity) may not be used as keys." This is wrong. tuples are 
an immutable type, but not all tuples can be used as keys.

The set documentation says "A set object is an unordered collection 
of immutable values.". This is also wrong - at least for common 
definitions of immutable.

Immutability is a convenient way of dealing with builtin types, but is 
a red herring. It's whether or not the object has a hash value that 
matters, and it's the behavior of that hash value (coupled with 
comparison) that determine whether or not things behave as 
expected.

The __hash__ documentation deals with these issues. I suggest 
replacing the current descriptions with one that references hashing, 
and a footnote pointing to the __hash__ docs for details:

Any hashable object(1) can be used as a dictionary key/set element. 
Lists, sets and dicts are not hashable, and can not be used. Tuples 
can be used if all the things they contain are hashable. Instances of 
all other built-in types and most user-defined classes are hashable.

(1) Objects for which the hash() function returns an appropriate 
value. See the __hash__ documentation for details.

----------------------------------------------------------------------

Comment By: Bj?rn Lindqvist (sonderblade)
Date: 2007-04-04 19:11

Message:
Logged In: YES 
user_id=51702
Originator: NO

That patch have been committed, this bug should be closed.

----------------------------------------------------------------------

Comment By: Collin Winter (collinwinter)
Date: 2006-01-05 13:28

Message:
Logged In: YES 
user_id=1344176

I have submitted patch #1397711 which resolves this issue.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1368768&group_id=5470

From noreply at sourceforge.net  Wed Apr  4 19:34:41 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Wed, 04 Apr 2007 10:34:41 -0700
Subject: [ python-Bugs-1694442 ] Failure to build on AIX 5.3
Message-ID: <E1HZ9NZ-0002t2-VZ@sc8-sf-web21.sourceforge.net>

Bugs item #1694442, was opened at 2007-04-04 10:34
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1694442&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Jaman Swearingen (tiriss)
Assigned to: Nobody/Anonymous (nobody)
Summary: Failure to build on AIX 5.3

Initial Comment:
AIX 5.3.0.0 gcc 4.0.0 tk8.4 tcl8.4 xlc_r 6.0.0 cc_r 6.0.0

Regardless of which of the compilers I use gcc, xlc_r, or cc_r, I get the following error after I hit make:

creating build/temp.aix-5.3-2.5/opt/freeware/src/packages/Python-2.5/Modules
gcc -pthread -fno-strict-aliasing -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -I. -I/opt/freeware/src/packages/Python-2.5/./Include -I./Include -I. -I/usr/local/include -I/opt/freeware/src/packages/Python-2.5/Include -I/opt/freeware/src/packages/Python-2.5 -c /opt/freeware/src/packages/Python-2.5/Modules/_struct.c -o build/temp.aix-5.3-2.5/opt/freeware/src/packages/Python-2.5/Modules/_struct.o
creating build/lib.aix-5.3-2.5
./Modules/ld_so_aix gcc -pthread -bI:Modules/python.exp build/temp.aix-5.3-2.5/opt/freeware/src/packages/Python-2.5/Modules/_struct.o -L/usr/local/lib -lpython2.5 -o build/lib.aix-5.3-2.5/_struct.so
collect2: library libpython2.5 not found
*** WARNING: renaming "_struct" since importing it failed:      0509-022 Cannot load module build/lib.aix-5.3-2.5.
        0509-026 System error: A file or directory in the path name does not exist.
error: No such file or directory

For some reason when I use xlc_r or gcc, it fails to find the local libpython2.5.a library that is residing in the current directory.  When I use cc_r, it fails to  find libtcl8.4.so and libtk8.4.so even though there are copies of those in the local . directory.

Any help would be appreciated.  Here are the configures I used:

./configure --with-gcc="xlc_r -q64" --with-cxx="xlC_r -q64" \
         --disable-ipv6 AR="ar -X64" --enable-shared

or 
./configure --with-gcc --with-cxx=g++ \
            --disable-ipv6 --enable-shared

or
configure --disable-ipv6

Let me know if there is other information you need.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1694442&group_id=5470

From noreply at sourceforge.net  Wed Apr  4 19:46:28 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Wed, 04 Apr 2007 10:46:28 -0700
Subject: [ python-Bugs-1368768 ] clearing up dictionary keys/set member docs
Message-ID: <E1HZ9Yy-0007Ko-SJ@sc8-sf-web23.sourceforge.net>

Bugs item #1368768, was opened at 2005-11-28 20:40
Message generated for change (Settings changed) made by rhettinger
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1368768&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Documentation
Group: Feature Request
>Status: Closed
>Resolution: Fixed
Priority: 5
Private: No
Submitted By: Mike Meyer (mwm)
Assigned to: Nobody/Anonymous (nobody)
Summary: clearing up dictionary keys/set member docs

Initial Comment:
The documentation for dictionaries says "A dictionary's keys are 
almost arbitrary values. Only values containing lists, dictionaries or 
other mutable types (that are compared by value rather than by 
object identity) may not be used as keys." This is wrong. tuples are 
an immutable type, but not all tuples can be used as keys.

The set documentation says "A set object is an unordered collection 
of immutable values.". This is also wrong - at least for common 
definitions of immutable.

Immutability is a convenient way of dealing with builtin types, but is 
a red herring. It's whether or not the object has a hash value that 
matters, and it's the behavior of that hash value (coupled with 
comparison) that determine whether or not things behave as 
expected.

The __hash__ documentation deals with these issues. I suggest 
replacing the current descriptions with one that references hashing, 
and a footnote pointing to the __hash__ docs for details:

Any hashable object(1) can be used as a dictionary key/set element. 
Lists, sets and dicts are not hashable, and can not be used. Tuples 
can be used if all the things they contain are hashable. Instances of 
all other built-in types and most user-defined classes are hashable.

(1) Objects for which the hash() function returns an appropriate 
value. See the __hash__ documentation for details.

----------------------------------------------------------------------

Comment By: Bj?rn Lindqvist (sonderblade)
Date: 2007-04-04 12:11

Message:
Logged In: YES 
user_id=51702
Originator: NO

That patch have been committed, this bug should be closed.

----------------------------------------------------------------------

Comment By: Collin Winter (collinwinter)
Date: 2006-01-05 07:28

Message:
Logged In: YES 
user_id=1344176

I have submitted patch #1397711 which resolves this issue.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1368768&group_id=5470

From noreply at sourceforge.net  Wed Apr  4 20:11:52 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Wed, 04 Apr 2007 11:11:52 -0700
Subject: [ python-Bugs-1694155 ] Python 2.5 installer ended prematurely
Message-ID: <E1HZ9xY-0002fa-NT@sc8-sf-web24.sourceforge.net>

Bugs item #1694155, was opened at 2007-04-04 12:03
Message generated for change (Comment added) made by loewis
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1694155&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Installation
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Miclav (miclav)
Assigned to: Martin v. L?wis (loewis)
Summary: Python 2.5 installer ended prematurely

Initial Comment:
Hello,

I could not install Python 2.5 on 2 machines running Windows XP. I wanted to install it into P:\Programs\languages\python

When I run it from P:\Proginst\languages\python, the installation fails very early : just after the "Customize Python 2.5" panel, when I click on "Next", I get almost instantly a panel "Python 2.5 installer ended prematurely" with a Finish button to exit installer.

When I run the msi file from C:\ as indicated in other bug report on the same subject, I can go further but it fails also : I get first a panel "Install Python 2.5", and some files are copied (as implied by the green status bar) but after a while it stops with a panel saying "The specified path is too long : P:\Programs" and 2 buttons to retry or cancel installation. Retry brings again to this panel so I must cancel. 

I was unable to find a solution, and I generated a python.log file (for the installation from C:\) that I join as attached file.

 On a third machine with Windows 2000 I succeeded in installing Python 2.5 in P:\Programs\languages\python from P:\Proginst\languages\python, so maybe the problem comes from WinXP? In both cases (XP and 2000) I use Zone Alarm as firewall (for XP, with XP firewall desactivated).

The only solution I found up to now is to install Python 2.2 (which uses another installer). I would be very happy if you could switch to an Open Source installation program (not commercial) so that everything in the behaviour of installation program can be known and under control. From the python.log file, I suspect that msi installer in WinXP might use some spy program, or it may in future versions of Windows. The installation with msi installer is much too fragile, and it might make fragile all Open Source programs based on Python : before trying to install Python, I had  tried to install the programm Leo, which is an editor written in Python that installs Python as a first step if it is not already installed. And I failed also to install Leo, with the same error (path P:\programs too long").

I would suggest to switch to NSIS ("Nullsoft Installer"), which is Open Source. I use it to distribute my own programs, and I found it excellent. I found further that many other Open Source programs are using this installer, and it always worked like a charm on all my machines. It seems much more robust that msi installer.


----------------------------------------------------------------------

>Comment By: Martin v. L?wis (loewis)
Date: 2007-04-04 20:11

Message:
Logged In: YES 
user_id=21627
Originator: NO

Is P a subst'ed drive, by any chance?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1694155&group_id=5470

From noreply at sourceforge.net  Wed Apr  4 20:31:35 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Wed, 04 Apr 2007 11:31:35 -0700
Subject: [ python-Bugs-1686475 ] os.stat() WindowsError 13 when file in use
Message-ID: <E1HZAGd-0003jd-9C@sc8-sf-web23.sourceforge.net>

Bugs item #1686475, was opened at 2007-03-23 03:31
Message generated for change (Comment added) made by loewis
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1686475&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.5
>Status: Closed
>Resolution: Fixed
Priority: 5
Private: No
Submitted By: d roberts (dcroberts)
Assigned to: Martin v. L?wis (loewis)
Summary: os.stat() WindowsError 13 when file in use

Initial Comment:
Using 2.5 Sept 19, 2006 on Windows XP, 32-bit.

os.stat() on an "in use" file (i.e., open by another process) erroneously returns WindowsError [Error 13]. The file stats should be available on an open file, even if you cannot read/write the file itself.

Python 2.4 correctly returns the stats on the file.

----------------
CORRECT in 2.4:
Python 2.4.4 (#71, Oct 18 2006, 08:34:43) [MSC v.1310 32 bit (Intel)] on win32
>>> import os
>>> os.stat('c:\\hiberfil.sys')
(33206, 0L, 2, 1, 0, 0, 804311040L, 1173962381, 1173962381, 1069302780)

-----------------
ERROR in 2.5:
Python 2.5 (r25:51908, Sep 19 2006, 09:52:17) [MSC v.1310 32 bit (Intel)] on win32
>>> import os
>>> os.stat("c:\\hiberfil.sys")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
WindowsError: [Error 13] The process cannot access the file because it is being
used by another process: 'c:\\hiberfil.sys'

----------------------------------------------------------------------

>Comment By: Martin v. L?wis (loewis)
Date: 2007-04-04 20:31

Message:
Logged In: YES 
user_id=21627
Originator: NO

I have now committed this patch as r54685 and r54686.

----------------------------------------------------------------------

Comment By: Martin v. L?wis (loewis)
Date: 2007-03-30 08:29

Message:
Logged In: YES 
user_id=21627
Originator: NO

Attached is a patch that makes it fall back to FindFirstFile if
GetFileAttributesEx fail with ERROR_SHARING_VIOLATION.
File Added: stat.diff

----------------------------------------------------------------------

Comment By: d roberts (dcroberts)
Date: 2007-03-23 18:49

Message:
Logged In: YES 
user_id=1750755
Originator: YES

Taking your hint, I just looked at the code in 42230 which uses the Win32
GetFileAttributesEx() function. This function does indeed return an error
for an in-use file. Not sure if this is a "feature" or "bug" in the
underlying Windows function. I did try FindFirstFile(), which is able to
return info on an in-use file. Not sure if this is a reasonable alternative
to GFAE(), but wanted to make folks aware of the possibility. It's probably
slower, since it can handle wild-cards, and returns a handle for subsequent
calls to FindNextFile(). The handle must be closed with FindClose(). I'm
not sure what underlying mechanism FindFirstFile() uses, but I'm assuming
it accesses the "directory" to get its information, rather than trying to
access the file itself. FWIW.

----------------------------------------------------------------------

Comment By: Martin v. L?wis (loewis)
Date: 2007-03-23 11:24

Message:
Logged In: YES 
user_id=21627
Originator: NO

It certainly is; Python is likely opening the file in the wrong mode now.
I'll investigate, although contributions would be welcome.

----------------------------------------------------------------------

Comment By: Georg Brandl (gbrandl)
Date: 2007-03-23 11:15

Message:
Logged In: YES 
user_id=849994
Originator: NO

May this be related to rev. 42230 - "Drop C library for stat/fstat on
Windows." ?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1686475&group_id=5470

From noreply at sourceforge.net  Wed Apr  4 22:32:41 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Wed, 04 Apr 2007 13:32:41 -0700
Subject: [ python-Bugs-1563759 ] struct.unpack doens't support buffer protocol
	objects
Message-ID: <E1HZC9p-0005Os-Te@sc8-sf-web21.sourceforge.net>

Bugs item #1563759, was opened at 2006-09-22 17:17
Message generated for change (Comment added) made by rhettinger
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1563759&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.5
>Status: Closed
>Resolution: Fixed
Priority: 7
Private: No
Submitted By: Adal Chiriliuc (adalx)
Assigned to: Nobody/Anonymous (nobody)
Summary: struct.unpack doens't support buffer protocol objects

Initial Comment:
If you pass an object which supports the buffer
protocol to struct.unpack it will fail because it
specifically checks for a string.

You should use PyObject_AsReadBuffer instead.

If this code is performance critical, you could add an
unpack_buffer method or something like that.

----------------------------------------------------------------------

>Comment By: Raymond Hettinger (rhettinger)
Date: 2007-04-04 15:32

Message:
Logged In: YES 
user_id=80475
Originator: NO

Fixed.  See revision 54690.

----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2007-04-03 19:45

Message:
Logged In: YES 
user_id=80475
Originator: NO

This was due to Bob Ippolito's commit in revision 46184.

----------------------------------------------------------------------

Comment By: Adal Chiriliuc (adalx)
Date: 2006-09-23 07:57

Message:
Logged In: YES 
user_id=1067739

test_struct_run.py works in 2.4, throws exception in 2.5

----------------------------------------------------------------------

Comment By: Adal Chiriliuc (adalx)
Date: 2006-09-23 07:56

Message:
Logged In: YES 
user_id=1067739

The actual code which broke used a Pointer extension class
implemented in C++.

I reproduced the problem using array.array. Using array in
this way (without calling tostring) looks a bit weird.


----------------------------------------------------------------------

Comment By: Martin v. L?wis (loewis)
Date: 2006-09-23 07:12

Message:
Logged In: YES 
user_id=21627

Ah, so you say that was working previously? It's a bug,
then. Can you provide a test case?

----------------------------------------------------------------------

Comment By: Adal Chiriliuc (adalx)
Date: 2006-09-23 06:37

Message:
Logged In: YES 
user_id=1067739

Well, because it broke previously working code and there is
no warning in the documentation about that.

In the mean-time, I've found out about pack_into and
unpack_from which accept buffer like objects. Note that they
are not documented in the struct module section, only
mentioned in the "What's new" chapter.

----------------------------------------------------------------------

Comment By: Martin v. L?wis (loewis)
Date: 2006-09-23 06:19

Message:
Logged In: YES 
user_id=21627

Why is this a bug?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1563759&group_id=5470

From noreply at sourceforge.net  Wed Apr  4 23:17:43 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Wed, 04 Apr 2007 14:17:43 -0700
Subject: [ python-Bugs-1678102 ] zlib.crc32() not cross-platform
Message-ID: <E1HZCrP-0005t6-Vv@sc8-sf-web24.sourceforge.net>

Bugs item #1678102, was opened at 2007-03-10 18:07
Message generated for change (Settings changed) made by tjreedy
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1678102&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.5
>Status: Closed
>Resolution: Invalid
Priority: 5
Private: No
Submitted By: Ben Collver (bencollver)
Assigned to: Nobody/Anonymous (nobody)
Summary: zlib.crc32() not cross-platform

Initial Comment:
The zlib.crc32() function sometimes produces different results for the same input on big and little-endian processors.  Same for zlib.adler32().

sparc64:
>>> import zlib
>>> print zlib.adler32("--------------------------------------------------", 1)
> 3763407051
>>> print zlib.crc32("--------------------------------------------------", 1)
3044228349

i386:
>>> import zlib
>>> print zlib.adler32("--------------------------------------------------", 1)
> -531560245
>>> print zlib.crc32("--------------------------------------------------", 1)
-1250738947


----------------------------------------------------------------------

Comment By: Gabriel Genellina (gagenellina)
Date: 2007-03-13 18:52

Message:
Logged In: YES 
user_id=479790
Originator: NO

py> -531560245 & 0xffffffff
3763407051L

It's the same number (actually, the same bit pattern). The i386 version is
signed, the other unsigned. The i386 platform uses a 32 bit "int"; the
sparc64 uses 64 bits (I presume). 3763407051 doesnt fit in 31bits, so it's
seen as a negative number.



----------------------------------------------------------------------

Comment By: Ben Collver (bencollver)
Date: 2007-03-10 18:13

Message:
Logged In: YES 
user_id=778667
Originator: YES

The extra > characters before the first results come from me pasting the
results to my irc client, then copying from there and pasting here.  Sorry
for any confusion.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1678102&group_id=5470

From noreply at sourceforge.net  Wed Apr  4 23:23:01 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Wed, 04 Apr 2007 14:23:01 -0700
Subject: [ python-Feature Requests-1662581 ] the re module can perform poorly:
	O(2**n) versus O(n**2)
Message-ID: <E1HZCwX-0002OI-In@sc8-sf-web21.sourceforge.net>

Feature Requests item #1662581, was opened at 2007-02-17 18:39
Message generated for change (Settings changed) made by tjreedy
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1662581&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
>Category: Performance
>Group: None
Status: Open
Resolution: None
Priority: 3
Private: No
Submitted By: Gregory P. Smith (greg)
Assigned to: Nobody/Anonymous (nobody)
Summary: the re module can perform poorly: O(2**n) versus O(n**2)

Initial Comment:
in short, the re module can degenerate to really really horrid performance.  See this for how and why:

 http://swtch.com/~rsc/regexp/regexp1.html

exponential decline instead of squared.

I don't have a patch so i'm filing this bug as a starting point for future work.  The Modules/_sre.c files implementation could be updated to use the parallel stepping Thompson approach instead of recursive backtracking.

filing this as a bug until me or someone else comes up with a patch.

----------------------------------------------------------------------

Comment By: Gregory P. Smith (greg)
Date: 2007-02-22 17:30

Message:
Logged In: YES 
user_id=413
Originator: YES

yeah this is better as a feature request.  certianly low priority either
way.

-nothing- I propose doing would change the syntax or behaviour of existing
regular expressions at all.  Doing so would be a disaster.  thompson nfa
does not imply changing the behaviour.

anyways its a lot more than a simple "patch" to change the re module to
not use backtracking so i expect this to languish unless someone has a of
free time and motivation all at once. :)


----------------------------------------------------------------------

Comment By: Josiah Carlson (josiahcarlson)
Date: 2007-02-22 03:51

Message:
Logged In: YES 
user_id=341410
Originator: NO

I would file this under "feature request"; the current situation isn't so
much buggy, as slow.  While you can produce a segfault with the current
regular expression engine (due to stack overflow), you can do the same
thing with regular Python on Linux (with sys.setrecursionlimit), ctypes,
etc., and none of those are considered as buggy.

My only concern with such a change is that it may or may not change the
semantics of the repeat operators '*' and '+', which are currently defined
as "greedy".  If I skimmed the article correctly late at night, switching
to a Thompson family regular expression engine may result in those
operators no longer being greedy.  Please correct me if I am wrong.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1662581&group_id=5470

From noreply at sourceforge.net  Thu Apr  5 01:44:52 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Wed, 04 Apr 2007 16:44:52 -0700
Subject: [ python-Bugs-1694663 ] Overloading  int.__pow__ does not work
Message-ID: <E1HZF9o-0007c6-8N@sc8-sf-web22.sourceforge.net>

Bugs item #1694663, was opened at 2007-04-04 19:44
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1694663&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Terry J. Reedy (tjreedy)
Assigned to: Nobody/Anonymous (nobody)
Summary: Overloading  int.__pow__ does not work

Initial Comment:
>From c.l.p: in response to glitch report, 2nd person showed that it is specific to __pow__

>>> class MyInt(int):
...     __sub__ = int.__add__
# similar lines for 9 other methods omitted
...     __or__ = int.__add__
...     __pow__ = int.__add__
...
>>> i = MyInt(42)
>>> i + 3
45
>>> i - 3
45
# similar outputs omitted
>>> i | 3
45
>>> i ** 3
74088

Another person (3rd) confirmed presence in 2.5 
Python 2.5 (r25:51908, Jan 21 2007, 03:10:25)
[GCC 3.4.6 20060404 (Red Hat 3.4.6-3)] 


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1694663&group_id=5470

From noreply at sourceforge.net  Thu Apr  5 03:47:15 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Wed, 04 Apr 2007 18:47:15 -0700
Subject: [ python-Bugs-1694663 ] Overloading  int.__pow__ does not work
Message-ID: <E1HZH4F-0005bQ-Mq@sc8-sf-web21.sourceforge.net>

Bugs item #1694663, was opened at 2007-04-05 01:44
Message generated for change (Comment added) made by zseil
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1694663&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Terry J. Reedy (tjreedy)
Assigned to: Nobody/Anonymous (nobody)
Summary: Overloading  int.__pow__ does not work

Initial Comment:
>From c.l.p: in response to glitch report, 2nd person showed that it is specific to __pow__

>>> class MyInt(int):
...     __sub__ = int.__add__
# similar lines for 9 other methods omitted
...     __or__ = int.__add__
...     __pow__ = int.__add__
...
>>> i = MyInt(42)
>>> i + 3
45
>>> i - 3
45
# similar outputs omitted
>>> i | 3
45
>>> i ** 3
74088

Another person (3rd) confirmed presence in 2.5 
Python 2.5 (r25:51908, Jan 21 2007, 03:10:25)
[GCC 3.4.6 20060404 (Red Hat 3.4.6-3)] 


----------------------------------------------------------------------

>Comment By: Ziga Seilnacht (zseil)
Date: 2007-04-05 03:47

Message:
Logged In: YES 
user_id=1326842
Originator: NO

Here is a patch (with tests) that should fix this.
There was another problem when the slot wrapper
came from a different type:

>>> class MyInt(int):
...     __mul__ = float.__add__
...
>>> MyInt(3) * 3
9

This now raises a TypeError.

File Added: mixing_slots.diff

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1694663&group_id=5470

From noreply at sourceforge.net  Thu Apr  5 06:54:33 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Wed, 04 Apr 2007 21:54:33 -0700
Subject: [ python-Bugs-1694663 ] Overloading  int.__pow__ does not work
Message-ID: <E1HZJzV-0005Ab-Aq@sc8-sf-web23.sourceforge.net>

Bugs item #1694663, was opened at 2007-04-04 18:44
Message generated for change (Comment added) made by rhettinger
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1694663&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
>Group: Python 2.5
Status: Open
Resolution: None
>Priority: 3
Private: No
Submitted By: Terry J. Reedy (tjreedy)
>Assigned to: Raymond Hettinger (rhettinger)
Summary: Overloading  int.__pow__ does not work

Initial Comment:
>From c.l.p: in response to glitch report, 2nd person showed that it is specific to __pow__

>>> class MyInt(int):
...     __sub__ = int.__add__
# similar lines for 9 other methods omitted
...     __or__ = int.__add__
...     __pow__ = int.__add__
...
>>> i = MyInt(42)
>>> i + 3
45
>>> i - 3
45
# similar outputs omitted
>>> i | 3
45
>>> i ** 3
74088

Another person (3rd) confirmed presence in 2.5 
Python 2.5 (r25:51908, Jan 21 2007, 03:10:25)
[GCC 3.4.6 20060404 (Red Hat 3.4.6-3)] 


----------------------------------------------------------------------

>Comment By: Raymond Hettinger (rhettinger)
Date: 2007-04-04 23:54

Message:
Logged In: YES 
user_id=80475
Originator: NO

I'll take a look at your patch.

The root problem is that the wrapper functions will return NULL if they
are fed the wrong number of arguments (i.e. the int.__add__ wrapper expects
exactly two arguments but the ** call provides three) -- you will see a
similar result if int.__neg__ or int.__invert__ are assigned to __add__.  
It looks like an upstream step is seeing the NULL and deciding that it
needs to look to skip the given method call and instead try the base the
class.

Am lowering the priority because there is no good use case for deliberate
argument mismatches in the override.  Am leaving the bug open because we
want the appropriate error message to surface.



----------------------------------------------------------------------

Comment By: Ziga Seilnacht (zseil)
Date: 2007-04-04 20:47

Message:
Logged In: YES 
user_id=1326842
Originator: NO

Here is a patch (with tests) that should fix this.
There was another problem when the slot wrapper
came from a different type:

>>> class MyInt(int):
...     __mul__ = float.__add__
...
>>> MyInt(3) * 3
9

This now raises a TypeError.

File Added: mixing_slots.diff

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1694663&group_id=5470

From noreply at sourceforge.net  Thu Apr  5 07:00:16 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Wed, 04 Apr 2007 22:00:16 -0700
Subject: [ python-Bugs-1686597 ] descrintro: error describing __new__ behavior
Message-ID: <E1HZK52-0000nL-VL@sc8-sf-web21.sourceforge.net>

Bugs item #1686597, was opened at 2007-03-23 06:47
Message generated for change (Comment added) made by gagenellina
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1686597&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Documentation
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Gabriel Genellina (gagenellina)
Assigned to: Nobody/Anonymous (nobody)
Summary: descrintro: error describing __new__ behavior

Initial Comment:
descrintro (http://www.python.org/download/releases/2.2.3/descrintro/#__new__) linked from http://www.python.org/doc/newstyle/ still says:

"__new__ must return an object. [...] If you return an object of a different class, its __init__ method will be called."

But since Revision 26220 - Modified Sat Apr 6 01:05:01 2002 UTC (4 years, 11 months ago) by gvanrossum:

"Changed new-style class instantiation so that when C's __new__ method returns something that's not a C instance, its __init__ is not called.  [SF bug #537450]"

That is, exactly the opposite as said on descrintro. The documentation for __new__ in the Reference manual, section 3.4.1, is correct and says: "If __new__() does not return an instance of cls, then the new instance's __init__() method will not be invoked."

Note that the modified behavior was already present on version 2.2.3 (and later) so updating the document currently at /download/releases/2.2.3/ would be fine.



----------------------------------------------------------------------

>Comment By: Gabriel Genellina (gagenellina)
Date: 2007-04-05 02:00

Message:
Logged In: YES 
user_id=479790
Originator: YES

Yes, but descrintro is the very first reference under
http://www.python.org/doc/newstyle/ "and should be your starting point."
(citing that page)

Surely there are other documents a bit outdated, but being descrintro in
such a prominent place, should have a correct content.

----------------------------------------------------------------------

Comment By: Steven Bethard (bediviere)
Date: 2007-04-03 18:47

Message:
Logged In: YES 
user_id=945502
Originator: NO

Note that this is correct in the documentation:
http://docs.python.org/ref/customization.html

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1686597&group_id=5470

From noreply at sourceforge.net  Thu Apr  5 09:52:15 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Thu, 05 Apr 2007 00:52:15 -0700
Subject: [ python-Bugs-1686597 ] descrintro: error describing __new__ behavior
Message-ID: <E1HZMlT-0002Gc-O8@sc8-sf-web21.sourceforge.net>

Bugs item #1686597, was opened at 2007-03-23 03:47
Message generated for change (Comment added) made by bediviere
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1686597&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Documentation
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Gabriel Genellina (gagenellina)
Assigned to: Nobody/Anonymous (nobody)
Summary: descrintro: error describing __new__ behavior

Initial Comment:
descrintro (http://www.python.org/download/releases/2.2.3/descrintro/#__new__) linked from http://www.python.org/doc/newstyle/ still says:

"__new__ must return an object. [...] If you return an object of a different class, its __init__ method will be called."

But since Revision 26220 - Modified Sat Apr 6 01:05:01 2002 UTC (4 years, 11 months ago) by gvanrossum:

"Changed new-style class instantiation so that when C's __new__ method returns something that's not a C instance, its __init__ is not called.  [SF bug #537450]"

That is, exactly the opposite as said on descrintro. The documentation for __new__ in the Reference manual, section 3.4.1, is correct and says: "If __new__() does not return an instance of cls, then the new instance's __init__() method will not be invoked."

Note that the modified behavior was already present on version 2.2.3 (and later) so updating the document currently at /download/releases/2.2.3/ would be fine.



----------------------------------------------------------------------

Comment By: Steven Bethard (bediviere)
Date: 2007-04-05 01:52

Message:
Logged In: YES 
user_id=945502
Originator: NO

Sorry, I wasn't suggesting that it shouldn't be corrected.  Only that
there's already text available to correct it in the Reference.

----------------------------------------------------------------------

Comment By: Gabriel Genellina (gagenellina)
Date: 2007-04-04 23:00

Message:
Logged In: YES 
user_id=479790
Originator: YES

Yes, but descrintro is the very first reference under
http://www.python.org/doc/newstyle/ "and should be your starting point."
(citing that page)

Surely there are other documents a bit outdated, but being descrintro in
such a prominent place, should have a correct content.

----------------------------------------------------------------------

Comment By: Steven Bethard (bediviere)
Date: 2007-04-03 15:47

Message:
Logged In: YES 
user_id=945502
Originator: NO

Note that this is correct in the documentation:
http://docs.python.org/ref/customization.html

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1686597&group_id=5470

From noreply at sourceforge.net  Thu Apr  5 10:11:03 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Thu, 05 Apr 2007 01:11:03 -0700
Subject: [ python-Bugs-1694833 ] Bad documentation for existing imp methods
Message-ID: <E1HZN3f-0005gS-DW@sc8-sf-web22.sourceforge.net>

Bugs item #1694833, was opened at 2007-04-05 10:11
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1694833&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Documentation
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Christoph Zwerschke (cito)
Assigned to: Nobody/Anonymous (nobody)
Summary: Bad documentation for existing imp methods

Initial Comment:
The documentation of find_module() and load_module() in the Standard Libary Reference (see http://docs.python.org/lib/module-imp.html) is incomplete/wrong.

Docu for find_module():
    
"Try to find the module "name" on the search path "path". ... If search is successful, the return value is a triple (file, pathname, description) where "file" is an open file ..."

First, the docu leaves it open whether it works only for ordinary modules or packages. It should be made clearer that it works for packages as well. In the last paragraph, the docu indirectly indicates that it works for packages, but this should be explained already in the first sentence.

Second, if "name" is a package, then "file" will not be an open file, but None, and pathname will be the path for the package (not None). The docu does not mention this case but claims that if file is None, then pathname will also be None which is wrong for a package.

Third, the docu switches arbitrarily between the "filename" and "pathname" though it always refers to the same object. This should be made consistent in the docu of find_module() itself, but also between find_module() and load_module(), since the latter takes the return values of the former as input.

Similarly, the documentation for load_modules() does not mention that it allows "file" to be None and "filename" to be the directory of a package.

Some code showing this behavior of find_module() and load_module() when they are used on a package:

from imp import find_module, load_module
package = 'imp'
args = find_module(package)
print args
print load_module(package, *args)


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1694833&group_id=5470

From noreply at sourceforge.net  Thu Apr  5 10:11:48 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Thu, 05 Apr 2007 01:11:48 -0700
Subject: [ python-Bugs-1694833 ] Bad documentation for existing imp methods
Message-ID: <E1HZN4O-0005ig-AC@sc8-sf-web22.sourceforge.net>

Bugs item #1694833, was opened at 2007-04-05 10:11
Message generated for change (Settings changed) made by cito
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1694833&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Documentation
Group: Python 2.5
Status: Open
Resolution: None
>Priority: 3
Private: No
Submitted By: Christoph Zwerschke (cito)
Assigned to: Nobody/Anonymous (nobody)
Summary: Bad documentation for existing imp methods

Initial Comment:
The documentation of find_module() and load_module() in the Standard Libary Reference (see http://docs.python.org/lib/module-imp.html) is incomplete/wrong.

Docu for find_module():
    
"Try to find the module "name" on the search path "path". ... If search is successful, the return value is a triple (file, pathname, description) where "file" is an open file ..."

First, the docu leaves it open whether it works only for ordinary modules or packages. It should be made clearer that it works for packages as well. In the last paragraph, the docu indirectly indicates that it works for packages, but this should be explained already in the first sentence.

Second, if "name" is a package, then "file" will not be an open file, but None, and pathname will be the path for the package (not None). The docu does not mention this case but claims that if file is None, then pathname will also be None which is wrong for a package.

Third, the docu switches arbitrarily between the "filename" and "pathname" though it always refers to the same object. This should be made consistent in the docu of find_module() itself, but also between find_module() and load_module(), since the latter takes the return values of the former as input.

Similarly, the documentation for load_modules() does not mention that it allows "file" to be None and "filename" to be the directory of a package.

Some code showing this behavior of find_module() and load_module() when they are used on a package:

from imp import find_module, load_module
package = 'imp'
args = find_module(package)
print args
print load_module(package, *args)


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1694833&group_id=5470

From noreply at sourceforge.net  Thu Apr  5 14:48:48 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Thu, 05 Apr 2007 05:48:48 -0700
Subject: [ python-Bugs-1694950 ] Bad documentation of SMTPException
Message-ID: <E1HZROS-0005Hs-1g@sc8-sf-web23.sourceforge.net>

Bugs item #1694950, was opened at 2007-04-05 14:48
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1694950&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Documentation
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Christoph Zwerschke (cito)
Assigned to: Nobody/Anonymous (nobody)
Summary: Bad documentation of SMTPException

Initial Comment:
The documentation of the SMTP object
http://docs.python.org/lib/SMTP-objects.html
lists "SMTPError" as a possible exception.
It should be "SMTPException" instead of "SMTPError".

Also, the "SMTPAuthenticationError" needs to be listed on the first page of the smtlib documentation
http://docs.python.org/lib/module-smtplib.html

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1694950&group_id=5470

From noreply at sourceforge.net  Thu Apr  5 14:48:59 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Thu, 05 Apr 2007 05:48:59 -0700
Subject: [ python-Bugs-1694950 ] Bad documentation of SMTPException
Message-ID: <E1HZROd-0003jt-7n@sc8-sf-web24.sourceforge.net>

Bugs item #1694950, was opened at 2007-04-05 14:48
Message generated for change (Settings changed) made by cito
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1694950&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Documentation
Group: Python 2.5
Status: Open
Resolution: None
>Priority: 2
Private: No
Submitted By: Christoph Zwerschke (cito)
Assigned to: Nobody/Anonymous (nobody)
Summary: Bad documentation of SMTPException

Initial Comment:
The documentation of the SMTP object
http://docs.python.org/lib/SMTP-objects.html
lists "SMTPError" as a possible exception.
It should be "SMTPException" instead of "SMTPError".

Also, the "SMTPAuthenticationError" needs to be listed on the first page of the smtlib documentation
http://docs.python.org/lib/module-smtplib.html

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1694950&group_id=5470

From noreply at sourceforge.net  Thu Apr  5 20:45:24 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Thu, 05 Apr 2007 11:45:24 -0700
Subject: [ python-Bugs-1695179 ] Failure to build on AIX 5.3
Message-ID: <E1HZWxY-0003IT-5f@sc8-sf-web21.sourceforge.net>

Bugs item #1695179, was opened at 2007-04-05 11:45
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1695179&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Jaman Swearingen (tiriss)
Assigned to: Nobody/Anonymous (nobody)
Summary: Failure to build on AIX 5.3

Initial Comment:
AIX 5.3.0.0 gcc 4.0.0 tk8.4 tcl8.4 xlc_r 6.0.0 cc_r 6.0.0

Regardless of which of the compilers I use gcc, xlc_r, or cc_r, I get the following error after I hit make:

creating build/temp.aix-5.3-2.5/opt/freeware/src/packages/Python-2.5/Modules
gcc -pthread -fno-strict-aliasing -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -I. -I/opt/freeware/src/packages/Python-2.5/./Include -I./Include -I. -I/usr/local/include -I/opt/freeware/src/packages/Python-2.5/Include -I/opt/freeware/src/packages/Python-2.5 -c /opt/freeware/src/packages/Python-2.5/Modules/_struct.c -o build/temp.aix-5.3-2.5/opt/freeware/src/packages/Python-2.5/Modules/_struct.o
creating build/lib.aix-5.3-2.5
./Modules/ld_so_aix gcc -pthread -bI:Modules/python.exp build/temp.aix-5.3-2.5/opt/freeware/src/packages/Python-2.5/Modules/_struct.o -L/usr/local/lib -lpython2.5 -o build/lib.aix-5.3-2.5/_struct.so
collect2: library libpython2.5 not found
*** WARNING: renaming "_struct" since importing it failed:      0509-022 Cannot load module build/lib.aix-5.3-2.5.
        0509-026 System error: A file or directory in the path name does not exist.
error: No such file or directory

For some reason when I use xlc_r or gcc, it fails to find the local libpython2.5.a library that is residing in the current directory.  When I use cc_r, it fails to  find libtcl8.4.so and libtk8.4.so even though there are copies of those in the local . directory.

Any help would be appreciated.  Here are the configures I used:

./configure --with-gcc="xlc_r -q64" --with-cxx="xlC_r -q64" \
         --disable-ipv6 AR="ar -X64" --enable-shared

or 
./configure --with-gcc --with-cxx=g++ \
            --disable-ipv6 --enable-shared

or
configure --disable-ipv6

Let me know if there is other information you need.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1695179&group_id=5470

From noreply at sourceforge.net  Thu Apr  5 22:59:26 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Thu, 05 Apr 2007 13:59:26 -0700
Subject: [ python-Bugs-1695272 ] generator expression produces incomplete list
Message-ID: <E1HZZ3G-0007S3-Pq@sc8-sf-web24.sourceforge.net>

Bugs item #1695272, was opened at 2007-04-05 20:59
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1695272&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Interpreter Core
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: wplapper (wplapper)
Assigned to: Nobody/Anonymous (nobody)
Summary: generator expression produces incomplete list

Initial Comment:
(eliminate(values, s, d2) for d2 in values[s] if d2 != d)   
is producing incomplete results for the above statement.

If the statement split into two parts, the results are correct.
    xxx = [d2 for d2 in values[s] if d2 != d]
    zzz = (eliminate(values, s, d2) for d2 in xxx)
    return all(zzz)

First create temporary list xxx and then apply generator expression zzz.


The problem has been observed for python 2.4.3 and 2.5.0.0, on Windows, Solaris (x86) and on Linux. For me, the error pattern is consistent for all test cases I ran.

This is the output of the attach script:
wrong
assign.s    0 d 2 val [1, 2, 3, 4, 5, 6, 7, 8, 9]
eliminate.s 0 d 1 val [1, 2, 3, 4, 5, 6, 7, 8, 9]
eliminate.s 0 d 3 val [2, 3, 4, 5, 6, 7, 8, 9]
eliminate.s 0 d 5 val [2, 4, 5, 6, 7, 8, 9]
eliminate.s 0 d 7 val [2, 4, 6, 7, 8, 9]
eliminate.s 0 d 9 val [2, 4, 6, 8, 9]

right
assign.s    0 d 2 val [1, 2, 3, 4, 5, 6, 7, 8, 9]
eliminate.s 0 d 1 val [1, 2, 3, 4, 5, 6, 7, 8, 9]
eliminate.s 0 d 3 val [2, 3, 4, 5, 6, 7, 8, 9]
eliminate.s 0 d 4 val [2, 4, 5, 6, 7, 8, 9]
eliminate.s 0 d 5 val [2, 5, 6, 7, 8, 9]
eliminate.s 0 d 6 val [2, 6, 7, 8, 9]
eliminate.s 0 d 7 val [2, 7, 8, 9]
eliminate.s 0 d 8 val [2, 8, 9]
eliminate.s 0 d 9 val [2, 9]
 
The test program is supposed to remove ALL numbers from the list, apart from "2". In the wrong case, only odd numbers are removed.

Best regards,
Winfried

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1695272&group_id=5470

From noreply at sourceforge.net  Thu Apr  5 23:09:33 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Thu, 05 Apr 2007 14:09:33 -0700
Subject: [ python-Bugs-1695272 ] generator expression produces incomplete list
Message-ID: <E1HZZD3-0001d5-1K@sc8-sf-web22.sourceforge.net>

Bugs item #1695272, was opened at 2007-04-05 20:59
Message generated for change (Comment added) made by gbrandl
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1695272&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Interpreter Core
Group: Python 2.5
>Status: Closed
>Resolution: Invalid
Priority: 5
Private: No
Submitted By: wplapper (wplapper)
Assigned to: Nobody/Anonymous (nobody)
Summary: generator expression produces incomplete list

Initial Comment:
(eliminate(values, s, d2) for d2 in values[s] if d2 != d)   
is producing incomplete results for the above statement.

If the statement split into two parts, the results are correct.
    xxx = [d2 for d2 in values[s] if d2 != d]
    zzz = (eliminate(values, s, d2) for d2 in xxx)
    return all(zzz)

First create temporary list xxx and then apply generator expression zzz.


The problem has been observed for python 2.4.3 and 2.5.0.0, on Windows, Solaris (x86) and on Linux. For me, the error pattern is consistent for all test cases I ran.

This is the output of the attach script:
wrong
assign.s    0 d 2 val [1, 2, 3, 4, 5, 6, 7, 8, 9]
eliminate.s 0 d 1 val [1, 2, 3, 4, 5, 6, 7, 8, 9]
eliminate.s 0 d 3 val [2, 3, 4, 5, 6, 7, 8, 9]
eliminate.s 0 d 5 val [2, 4, 5, 6, 7, 8, 9]
eliminate.s 0 d 7 val [2, 4, 6, 7, 8, 9]
eliminate.s 0 d 9 val [2, 4, 6, 8, 9]

right
assign.s    0 d 2 val [1, 2, 3, 4, 5, 6, 7, 8, 9]
eliminate.s 0 d 1 val [1, 2, 3, 4, 5, 6, 7, 8, 9]
eliminate.s 0 d 3 val [2, 3, 4, 5, 6, 7, 8, 9]
eliminate.s 0 d 4 val [2, 4, 5, 6, 7, 8, 9]
eliminate.s 0 d 5 val [2, 5, 6, 7, 8, 9]
eliminate.s 0 d 6 val [2, 6, 7, 8, 9]
eliminate.s 0 d 7 val [2, 7, 8, 9]
eliminate.s 0 d 8 val [2, 8, 9]
eliminate.s 0 d 9 val [2, 9]
 
The test program is supposed to remove ALL numbers from the list, apart from "2". In the wrong case, only odd numbers are removed.

Best regards,
Winfried

----------------------------------------------------------------------

>Comment By: Georg Brandl (gbrandl)
Date: 2007-04-05 21:09

Message:
Logged In: YES 
user_id=849994
Originator: NO

This is a programming error.
In the "wrong" function, the "values" dict is mutated by the eliminate()
function while it is iterated over.
This always gives unexpected results.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1695272&group_id=5470

From noreply at sourceforge.net  Fri Apr  6 01:09:42 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Thu, 05 Apr 2007 16:09:42 -0700
Subject: [ python-Bugs-1685000 ] DoS asyncore vulnerability
Message-ID: <E1HZb5K-0008MJ-KR@sc8-sf-web22.sourceforge.net>

Bugs item #1685000, was opened at 2007-03-21 10:15
Message generated for change (Comment added) made by sgala
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1685000&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: billiejoex (billiejoex)
Assigned to: Nobody/Anonymous (nobody)
Summary: DoS asyncore vulnerability

Initial Comment:
DoS asyncore vulnerability

asyncore, independently if used with select() or poll(), suffers a DoS-type vulnerability when a high number of simultaneous connections to handle simultaneously is reached.
The number of maximum connections is system-dependent as well as the type of error raised.
I attached two simple Proof of Concept scripts demonstrating such bug.
If you want to try the behaviours listed below run the attached "asyncore_server.py" and "asyncore_client.py" scripts on your local workstation.

On my Windows XP system (Python 2.5), independently if asyncore has been used to develop a server or a client, the error is raised by select() inside asyncore's "poll" function when 512 (socket_map's elements) simultaneous connections are reached. 
Here's the traceback I get:

[...]
connections: 510
connections: 511
connections: 512
Traceback (most recent call last):
  File "C:\scripts\asyncore_server.py", line 38, in <module>
    asyncore.loop()
  File "C:\Python25\lib\asyncore.py", line 191, in loop
    poll_fun(timeout, map)
  File "C:\Python25\lib\asyncore.py", line 121, in poll
    r, w, e = select.select(r, w, e, timeout)
ValueError: too many file descriptors in select()


On my Linux Ubuntu 6.10 (kernel 2.6.17-10, Python 2.5) different type of errors are raised depending on the application (client or server).
In an asyncore-based client the error is raised by socket module (dispatcher's "self.socket" attribute) inside 'connect' method of 'dispatcher' class:

[...]
connections: 1018
connections: 1019
connections: 1020
connections: 1021
Traceback (most recent call last):
  File "asyncore_client.py", line 31, in <module>
  File "asyncore.py", line 191, in loop
  File "asyncore.py", line 138, in poll
  File "asyncore.py", line 80, in write
  File "asyncore.py", line 76, in write
  File "asyncore.py", line 395, in handle_write_event
  File "asyncore_client.py", line 24, in handle_connect
  File "asyncore_client.py", line 9, in __init__
  File "asyncore.py", line 257, in create_socket
  File "socket.py", line 156, in __init__
socket.error: (24, 'Too many open files')


On an asyncore-based server the error is raised by socket module (dispatcher's "self.socket" attribute) inside 'accept' method of 'dispatcher' class:

[...]
connections: 1019
connections: 1020
connections: 1021
Traceback (most recent call last):
  File "asyncore_server.py", line 38, in <module>
  File "asyncore.py", line 191, in loop
  File "asyncore.py", line 132, in poll
  File "asyncore.py", line 72, in read
  File "asyncore.py", line 68, in read
  File "asyncore.py", line 384, in handle_read_event
  File "asyncore_server.py", line 16, in handle_accept
  File "asyncore.py", line 321, in accept
  File "socket.py", line 170, in accept
socket.error: (24, 'Too many open files')


----------------------------------------------------------------------

Comment By: Santiago Gala (sgala)
Date: 2007-04-06 01:09

Message:
Logged In: YES 
user_id=178886
Originator: NO

THe limit of resources that an OS can deal with is limited due to
resources, both globally and per user, precisely to avoid DoS attacks by a
uses.

In the case of linux, you can control it with "ulimit -n XXXX" (for the
user that runs the test). The default here is 1024 (and the maximum, unless
it is globally raised).

I imagine windows will have similar limits, and similar (registry, etc.)
ways to handle them.

----------------------------------------------------------------------

Comment By: Sam Rushing (rushing)
Date: 2007-03-30 19:22

Message:
Logged In: YES 
user_id=73736
Originator: NO

Turns out medusa doesn't have a socket counter class, that was some other
project I was thinking of.

Putting a try/except in place doesn't really help the problem... if you
fail to create a new socket what action will you take?

A better approach is to have a configurable limit on the number of open
connections, and then have a server-specific reaction to exceeding that
limit.  For example, an SMTP server might respond with a 4XX greeting and
close the connection.

An additional problem on Unix is that running out of descriptors affects
more than just sockets.  Once you hit the FD limit you can't open files,
or do anything that requires a descriptor.


----------------------------------------------------------------------

Comment By: billiejoex (billiejoex)
Date: 2007-03-29 16:03

Message:
Logged In: YES 
user_id=1357589
Originator: YES

> The problem is that there's no portable way to know what the limit
> on file descriptors is.

Why don't put a try/except statement before creating a new socket's
file-descriptor?
I believe that such problem shouldn't be understimated. 
Actually asyncore is the only asynchronous module present in Python's
stdlib.
If asyncore suffers a DoS vulnerability it just means that writing a
secure client/server application with Python without using a third-party
library isn't possible.

I wrote a modified version of asyncore that solves the problem with select
 (affecting Windows) but still can't find a way to solve the problem with
socket's file-descriptors (affecting Unix).

----------------------------------------------------------------------

Comment By: Sam Rushing (rushing)
Date: 2007-03-23 20:59

Message:
Logged In: YES 
user_id=73736
Originator: NO

The problem is that there's no portable way to know what the limit
on file descriptors is.  The 'classic' for select/poll is the FD_SETSIZE
macro.  But on some operating systems there is no such limit. [e.g.,
win32
does not use the 'lowest-free-int' model common to unix].

I believe that in Medusa there was a derived class or extension that
counted
the number of open sockets, and limited it, using something like a
semaphore.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1685000&group_id=5470

From noreply at sourceforge.net  Fri Apr  6 14:10:58 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Fri, 06 Apr 2007 05:10:58 -0700
Subject: [ python-Bugs-1685000 ] DoS asyncore vulnerability
Message-ID: <E1HZnHO-0003Ik-Dm@sc8-sf-web22.sourceforge.net>

Bugs item #1685000, was opened at 2007-03-21 10:15
Message generated for change (Comment added) made by billiejoex
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1685000&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.5
Status: Open
Resolution: None
>Priority: 9
Private: No
Submitted By: billiejoex (billiejoex)
Assigned to: Nobody/Anonymous (nobody)
Summary: DoS asyncore vulnerability

Initial Comment:
DoS asyncore vulnerability

asyncore, independently if used with select() or poll(), suffers a DoS-type vulnerability when a high number of simultaneous connections to handle simultaneously is reached.
The number of maximum connections is system-dependent as well as the type of error raised.
I attached two simple Proof of Concept scripts demonstrating such bug.
If you want to try the behaviours listed below run the attached "asyncore_server.py" and "asyncore_client.py" scripts on your local workstation.

On my Windows XP system (Python 2.5), independently if asyncore has been used to develop a server or a client, the error is raised by select() inside asyncore's "poll" function when 512 (socket_map's elements) simultaneous connections are reached. 
Here's the traceback I get:

[...]
connections: 510
connections: 511
connections: 512
Traceback (most recent call last):
  File "C:\scripts\asyncore_server.py", line 38, in <module>
    asyncore.loop()
  File "C:\Python25\lib\asyncore.py", line 191, in loop
    poll_fun(timeout, map)
  File "C:\Python25\lib\asyncore.py", line 121, in poll
    r, w, e = select.select(r, w, e, timeout)
ValueError: too many file descriptors in select()


On my Linux Ubuntu 6.10 (kernel 2.6.17-10, Python 2.5) different type of errors are raised depending on the application (client or server).
In an asyncore-based client the error is raised by socket module (dispatcher's "self.socket" attribute) inside 'connect' method of 'dispatcher' class:

[...]
connections: 1018
connections: 1019
connections: 1020
connections: 1021
Traceback (most recent call last):
  File "asyncore_client.py", line 31, in <module>
  File "asyncore.py", line 191, in loop
  File "asyncore.py", line 138, in poll
  File "asyncore.py", line 80, in write
  File "asyncore.py", line 76, in write
  File "asyncore.py", line 395, in handle_write_event
  File "asyncore_client.py", line 24, in handle_connect
  File "asyncore_client.py", line 9, in __init__
  File "asyncore.py", line 257, in create_socket
  File "socket.py", line 156, in __init__
socket.error: (24, 'Too many open files')


On an asyncore-based server the error is raised by socket module (dispatcher's "self.socket" attribute) inside 'accept' method of 'dispatcher' class:

[...]
connections: 1019
connections: 1020
connections: 1021
Traceback (most recent call last):
  File "asyncore_server.py", line 38, in <module>
  File "asyncore.py", line 191, in loop
  File "asyncore.py", line 132, in poll
  File "asyncore.py", line 72, in read
  File "asyncore.py", line 68, in read
  File "asyncore.py", line 384, in handle_read_event
  File "asyncore_server.py", line 16, in handle_accept
  File "asyncore.py", line 321, in accept
  File "socket.py", line 170, in accept
socket.error: (24, 'Too many open files')


----------------------------------------------------------------------

>Comment By: billiejoex (billiejoex)
Date: 2007-04-06 14:10

Message:
Logged In: YES 
user_id=1357589
Originator: YES

> Putting a try/except in place doesn't really help the problem... 
> if you fail to create a new socket what action will you take?

Calling a new dispatcher's method (for example:
"handle_exceeded_connection") could be an idea. By default it could close
the current session but it can be overriden if the user want to take some
other action.

> A better approach is to have a configurable limit on the number of 
> open connections, and then have a server-specific reaction to 
> exceeding that limit.

It doesn't work if I create additional file-objects during the execution
of the loop...

----------------------------------------------------------------------

Comment By: Santiago Gala (sgala)
Date: 2007-04-06 01:09

Message:
Logged In: YES 
user_id=178886
Originator: NO

THe limit of resources that an OS can deal with is limited due to
resources, both globally and per user, precisely to avoid DoS attacks by a
uses.

In the case of linux, you can control it with "ulimit -n XXXX" (for the
user that runs the test). The default here is 1024 (and the maximum, unless
it is globally raised).

I imagine windows will have similar limits, and similar (registry, etc.)
ways to handle them.

----------------------------------------------------------------------

Comment By: Sam Rushing (rushing)
Date: 2007-03-30 19:22

Message:
Logged In: YES 
user_id=73736
Originator: NO

Turns out medusa doesn't have a socket counter class, that was some other
project I was thinking of.

Putting a try/except in place doesn't really help the problem... if you
fail to create a new socket what action will you take?

A better approach is to have a configurable limit on the number of open
connections, and then have a server-specific reaction to exceeding that
limit.  For example, an SMTP server might respond with a 4XX greeting and
close the connection.

An additional problem on Unix is that running out of descriptors affects
more than just sockets.  Once you hit the FD limit you can't open files,
or do anything that requires a descriptor.


----------------------------------------------------------------------

Comment By: billiejoex (billiejoex)
Date: 2007-03-29 16:03

Message:
Logged In: YES 
user_id=1357589
Originator: YES

> The problem is that there's no portable way to know what the limit
> on file descriptors is.

Why don't put a try/except statement before creating a new socket's
file-descriptor?
I believe that such problem shouldn't be understimated. 
Actually asyncore is the only asynchronous module present in Python's
stdlib.
If asyncore suffers a DoS vulnerability it just means that writing a
secure client/server application with Python without using a third-party
library isn't possible.

I wrote a modified version of asyncore that solves the problem with select
 (affecting Windows) but still can't find a way to solve the problem with
socket's file-descriptors (affecting Unix).

----------------------------------------------------------------------

Comment By: Sam Rushing (rushing)
Date: 2007-03-23 20:59

Message:
Logged In: YES 
user_id=73736
Originator: NO

The problem is that there's no portable way to know what the limit
on file descriptors is.  The 'classic' for select/poll is the FD_SETSIZE
macro.  But on some operating systems there is no such limit. [e.g.,
win32
does not use the 'lowest-free-int' model common to unix].

I believe that in Medusa there was a derived class or extension that
counted
the number of open sockets, and limited it, using something like a
semaphore.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1685000&group_id=5470

From noreply at sourceforge.net  Fri Apr  6 14:13:50 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Fri, 06 Apr 2007 05:13:50 -0700
Subject: [ python-Bugs-1685000 ] asyncore DoS vulnerability
Message-ID: <E1HZnKA-0003cZ-EC@sc8-sf-web22.sourceforge.net>

Bugs item #1685000, was opened at 2007-03-21 10:15
Message generated for change (Settings changed) made by billiejoex
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1685000&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.5
Status: Open
Resolution: None
Priority: 9
Private: No
Submitted By: billiejoex (billiejoex)
Assigned to: Nobody/Anonymous (nobody)
>Summary: asyncore DoS vulnerability

Initial Comment:
DoS asyncore vulnerability

asyncore, independently if used with select() or poll(), suffers a DoS-type vulnerability when a high number of simultaneous connections to handle simultaneously is reached.
The number of maximum connections is system-dependent as well as the type of error raised.
I attached two simple Proof of Concept scripts demonstrating such bug.
If you want to try the behaviours listed below run the attached "asyncore_server.py" and "asyncore_client.py" scripts on your local workstation.

On my Windows XP system (Python 2.5), independently if asyncore has been used to develop a server or a client, the error is raised by select() inside asyncore's "poll" function when 512 (socket_map's elements) simultaneous connections are reached. 
Here's the traceback I get:

[...]
connections: 510
connections: 511
connections: 512
Traceback (most recent call last):
  File "C:\scripts\asyncore_server.py", line 38, in <module>
    asyncore.loop()
  File "C:\Python25\lib\asyncore.py", line 191, in loop
    poll_fun(timeout, map)
  File "C:\Python25\lib\asyncore.py", line 121, in poll
    r, w, e = select.select(r, w, e, timeout)
ValueError: too many file descriptors in select()


On my Linux Ubuntu 6.10 (kernel 2.6.17-10, Python 2.5) different type of errors are raised depending on the application (client or server).
In an asyncore-based client the error is raised by socket module (dispatcher's "self.socket" attribute) inside 'connect' method of 'dispatcher' class:

[...]
connections: 1018
connections: 1019
connections: 1020
connections: 1021
Traceback (most recent call last):
  File "asyncore_client.py", line 31, in <module>
  File "asyncore.py", line 191, in loop
  File "asyncore.py", line 138, in poll
  File "asyncore.py", line 80, in write
  File "asyncore.py", line 76, in write
  File "asyncore.py", line 395, in handle_write_event
  File "asyncore_client.py", line 24, in handle_connect
  File "asyncore_client.py", line 9, in __init__
  File "asyncore.py", line 257, in create_socket
  File "socket.py", line 156, in __init__
socket.error: (24, 'Too many open files')


On an asyncore-based server the error is raised by socket module (dispatcher's "self.socket" attribute) inside 'accept' method of 'dispatcher' class:

[...]
connections: 1019
connections: 1020
connections: 1021
Traceback (most recent call last):
  File "asyncore_server.py", line 38, in <module>
  File "asyncore.py", line 191, in loop
  File "asyncore.py", line 132, in poll
  File "asyncore.py", line 72, in read
  File "asyncore.py", line 68, in read
  File "asyncore.py", line 384, in handle_read_event
  File "asyncore_server.py", line 16, in handle_accept
  File "asyncore.py", line 321, in accept
  File "socket.py", line 170, in accept
socket.error: (24, 'Too many open files')


----------------------------------------------------------------------

Comment By: billiejoex (billiejoex)
Date: 2007-04-06 14:10

Message:
Logged In: YES 
user_id=1357589
Originator: YES

> Putting a try/except in place doesn't really help the problem... 
> if you fail to create a new socket what action will you take?

Calling a new dispatcher's method (for example:
"handle_exceeded_connection") could be an idea. By default it could close
the current session but it can be overriden if the user want to take some
other action.

> A better approach is to have a configurable limit on the number of 
> open connections, and then have a server-specific reaction to 
> exceeding that limit.

It doesn't work if I create additional file-objects during the execution
of the loop...

----------------------------------------------------------------------

Comment By: Santiago Gala (sgala)
Date: 2007-04-06 01:09

Message:
Logged In: YES 
user_id=178886
Originator: NO

THe limit of resources that an OS can deal with is limited due to
resources, both globally and per user, precisely to avoid DoS attacks by a
uses.

In the case of linux, you can control it with "ulimit -n XXXX" (for the
user that runs the test). The default here is 1024 (and the maximum, unless
it is globally raised).

I imagine windows will have similar limits, and similar (registry, etc.)
ways to handle them.

----------------------------------------------------------------------

Comment By: Sam Rushing (rushing)
Date: 2007-03-30 19:22

Message:
Logged In: YES 
user_id=73736
Originator: NO

Turns out medusa doesn't have a socket counter class, that was some other
project I was thinking of.

Putting a try/except in place doesn't really help the problem... if you
fail to create a new socket what action will you take?

A better approach is to have a configurable limit on the number of open
connections, and then have a server-specific reaction to exceeding that
limit.  For example, an SMTP server might respond with a 4XX greeting and
close the connection.

An additional problem on Unix is that running out of descriptors affects
more than just sockets.  Once you hit the FD limit you can't open files,
or do anything that requires a descriptor.


----------------------------------------------------------------------

Comment By: billiejoex (billiejoex)
Date: 2007-03-29 16:03

Message:
Logged In: YES 
user_id=1357589
Originator: YES

> The problem is that there's no portable way to know what the limit
> on file descriptors is.

Why don't put a try/except statement before creating a new socket's
file-descriptor?
I believe that such problem shouldn't be understimated. 
Actually asyncore is the only asynchronous module present in Python's
stdlib.
If asyncore suffers a DoS vulnerability it just means that writing a
secure client/server application with Python without using a third-party
library isn't possible.

I wrote a modified version of asyncore that solves the problem with select
 (affecting Windows) but still can't find a way to solve the problem with
socket's file-descriptors (affecting Unix).

----------------------------------------------------------------------

Comment By: Sam Rushing (rushing)
Date: 2007-03-23 20:59

Message:
Logged In: YES 
user_id=73736
Originator: NO

The problem is that there's no portable way to know what the limit
on file descriptors is.  The 'classic' for select/poll is the FD_SETSIZE
macro.  But on some operating systems there is no such limit. [e.g.,
win32
does not use the 'lowest-free-int' model common to unix].

I believe that in Medusa there was a derived class or extension that
counted
the number of open sockets, and limited it, using something like a
semaphore.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1685000&group_id=5470

From noreply at sourceforge.net  Fri Apr  6 17:28:00 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Fri, 06 Apr 2007 08:28:00 -0700
Subject: [ python-Bugs-1695688 ] Unexpected feature vs. integer and True values
Message-ID: <E1HZqM4-0004Mv-U3@sc8-sf-web22.sourceforge.net>

Bugs item #1695688, was opened at 2007-04-06 15:28
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1695688&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Interpreter Core
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: dev at bacademy (bacademy)
Assigned to: Nobody/Anonymous (nobody)
Summary: Unexpected feature vs. integer and True values

Initial Comment:
Integer 1 interpreted as True.
I didn't expect the following:

Python 2.5 (r25:51908, Apr  4 2007, 10:19:35)
[GCC 4.1.1 20060724 (prerelease) (4.1.1-3mdk)] on linux2
Type "help", "copyright", "credits" or "license" for more information.

>>> v = 2
>>> 0.0 if v == True else float(v)
2.0
>>> v = 1
>>> 0.0 if v == True else float(v)
0.0

I know how to fix this, by writing

0.0 if v is True else float(v)

but to casual observer it is not obvious...






----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1695688&group_id=5470

From noreply at sourceforge.net  Fri Apr  6 17:37:16 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Fri, 06 Apr 2007 08:37:16 -0700
Subject: [ python-Bugs-1695179 ] Failure to build on AIX 5.3
Message-ID: <E1HZqV2-0002E4-1M@sc8-sf-web21.sourceforge.net>

Bugs item #1695179, was opened at 2007-04-05 11:45
Message generated for change (Settings changed) made by tiriss
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1695179&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
>Status: Deleted
Resolution: None
Priority: 5
Private: No
Submitted By: Jaman Swearingen (tiriss)
Assigned to: Nobody/Anonymous (nobody)
Summary: Failure to build on AIX 5.3

Initial Comment:
AIX 5.3.0.0 gcc 4.0.0 tk8.4 tcl8.4 xlc_r 6.0.0 cc_r 6.0.0

Regardless of which of the compilers I use gcc, xlc_r, or cc_r, I get the following error after I hit make:

creating build/temp.aix-5.3-2.5/opt/freeware/src/packages/Python-2.5/Modules
gcc -pthread -fno-strict-aliasing -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -I. -I/opt/freeware/src/packages/Python-2.5/./Include -I./Include -I. -I/usr/local/include -I/opt/freeware/src/packages/Python-2.5/Include -I/opt/freeware/src/packages/Python-2.5 -c /opt/freeware/src/packages/Python-2.5/Modules/_struct.c -o build/temp.aix-5.3-2.5/opt/freeware/src/packages/Python-2.5/Modules/_struct.o
creating build/lib.aix-5.3-2.5
./Modules/ld_so_aix gcc -pthread -bI:Modules/python.exp build/temp.aix-5.3-2.5/opt/freeware/src/packages/Python-2.5/Modules/_struct.o -L/usr/local/lib -lpython2.5 -o build/lib.aix-5.3-2.5/_struct.so
collect2: library libpython2.5 not found
*** WARNING: renaming "_struct" since importing it failed:      0509-022 Cannot load module build/lib.aix-5.3-2.5.
        0509-026 System error: A file or directory in the path name does not exist.
error: No such file or directory

For some reason when I use xlc_r or gcc, it fails to find the local libpython2.5.a library that is residing in the current directory.  When I use cc_r, it fails to  find libtcl8.4.so and libtk8.4.so even though there are copies of those in the local . directory.

Any help would be appreciated.  Here are the configures I used:

./configure --with-gcc="xlc_r -q64" --with-cxx="xlC_r -q64" \
         --disable-ipv6 AR="ar -X64" --enable-shared

or 
./configure --with-gcc --with-cxx=g++ \
            --disable-ipv6 --enable-shared

or
configure --disable-ipv6

Let me know if there is other information you need.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1695179&group_id=5470

From noreply at sourceforge.net  Fri Apr  6 18:32:15 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Fri, 06 Apr 2007 09:32:15 -0700
Subject: [ python-Bugs-1695718 ] PEP 302 broken
Message-ID: <E1HZrMF-0002SG-NE@sc8-sf-web24.sourceforge.net>

Bugs item #1695718, was opened at 2007-04-06 12:32
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1695718&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Interpreter Core
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: phil (philipdumont)
Assigned to: Nobody/Anonymous (nobody)
Summary: PEP 302 broken

Initial Comment:
The product I'm working on uses a PEP 302 importer hook.  It's a home-grown ltihooks clone.  (We don't use ltihooks because it's GPLed, and we don't want to be.)

Our importer worked on 2.4.3, is broken on 2.5.0.

What's broken is that if the hook's find_module() method returns None for a given module name, say 'spam', then that is supposed to cause the import machinery to fall back on the regular unhooked import behavior -- that is, find 'spam.py' (or 'spam.pyc', 'spam.pyo') in the directory in question.  This used to be what happened, but no longer.

Tracing through the code, the problem seems to be occurring due to the 'continue' at line 1289 (in the 2.5 tarball) of Python/import.c.

Slogging through SVN (aside: this would have been easier if your ViewSVN supported annotate/blame -- any chance you could add that?), it appears that the offending continue statement was added in revision 46372, whose checkin comment claims that it was done for performance reasons.  I'm all for performance improvements, but not at the expense of breakage.

Attached is a tarball with some files that reproduce the problem.  (The LibtoolImporter.py file is a stripped-down toy version of what we are really using.)  Unwind the tarball, cd to the directory, and run script.py.  Here's what I get:



shell prompt> pwd
/home/phil/pep302_bug
shell prompt> ls -CF
eggs.la  LibtoolImporter.py  script.py*  spam.py
shell prompt> python2.4.3 script.py
.la file(s) found in /home/phil/pep302_bug, LibtoolImporter will be used.
LibtoolImporter.find_module() couldn't find spam.la or spammodule.la
in /home/phil/pep302_bug.  Returning None.  This is *supposed* to cause a
fallback to the default import code looking for spam.py
in /home/phil/pep302_bug
module spam loaded
shell prompt> python2.5 script.py
.la file(s) found in /home/phil/pep302_bug, LibtoolImporter will be used.
LibtoolImporter.find_module() couldn't find spam.la or spammodule.la
in /home/phil/pep302_bug.  Returning None.  This is *supposed* to cause a
fallback to the default import code looking for spam.py
in /home/phil/pep302_bug
Traceback (most recent call last):
  File "script.py", line 4, in <module>
    import spam
ImportError: No module named spam
shell prompt>

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1695718&group_id=5470

From noreply at sourceforge.net  Fri Apr  6 20:01:13 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Fri, 06 Apr 2007 11:01:13 -0700
Subject: [ python-Bugs-1695688 ] Unexpected feature vs. integer and True values
Message-ID: <E1HZskL-0003Zq-0L@sc8-sf-web24.sourceforge.net>

Bugs item #1695688, was opened at 2007-04-06 15:28
Message generated for change (Comment added) made by gbrandl
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1695688&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Interpreter Core
Group: None
>Status: Closed
>Resolution: Invalid
Priority: 5
Private: No
Submitted By: dev at bacademy (bacademy)
Assigned to: Nobody/Anonymous (nobody)
Summary: Unexpected feature vs. integer and True values

Initial Comment:
Integer 1 interpreted as True.
I didn't expect the following:

Python 2.5 (r25:51908, Apr  4 2007, 10:19:35)
[GCC 4.1.1 20060724 (prerelease) (4.1.1-3mdk)] on linux2
Type "help", "copyright", "credits" or "license" for more information.

>>> v = 2
>>> 0.0 if v == True else float(v)
2.0
>>> v = 1
>>> 0.0 if v == True else float(v)
0.0

I know how to fix this, by writing

0.0 if v is True else float(v)

but to casual observer it is not obvious...






----------------------------------------------------------------------

>Comment By: Georg Brandl (gbrandl)
Date: 2007-04-06 18:01

Message:
Logged In: YES 
user_id=849994
Originator: NO

It may be, but it is not a bug (bool inherits from int) and documented (I
think...).

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1695688&group_id=5470

From noreply at sourceforge.net  Fri Apr  6 21:21:47 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Fri, 06 Apr 2007 12:21:47 -0700
Subject: [ python-Bugs-1695718 ] PEP 302 broken
Message-ID: <E1HZu0J-0003mb-FV@sc8-sf-web24.sourceforge.net>

Bugs item #1695718, was opened at 2007-04-06 11:32
Message generated for change (Comment added) made by rhettinger
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1695718&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Interpreter Core
Group: Python 2.5
Status: Open
Resolution: None
>Priority: 6
Private: No
Submitted By: phil (philipdumont)
>Assigned to: Georg Brandl (gbrandl)
Summary: PEP 302 broken

Initial Comment:
The product I'm working on uses a PEP 302 importer hook.  It's a home-grown ltihooks clone.  (We don't use ltihooks because it's GPLed, and we don't want to be.)

Our importer worked on 2.4.3, is broken on 2.5.0.

What's broken is that if the hook's find_module() method returns None for a given module name, say 'spam', then that is supposed to cause the import machinery to fall back on the regular unhooked import behavior -- that is, find 'spam.py' (or 'spam.pyc', 'spam.pyo') in the directory in question.  This used to be what happened, but no longer.

Tracing through the code, the problem seems to be occurring due to the 'continue' at line 1289 (in the 2.5 tarball) of Python/import.c.

Slogging through SVN (aside: this would have been easier if your ViewSVN supported annotate/blame -- any chance you could add that?), it appears that the offending continue statement was added in revision 46372, whose checkin comment claims that it was done for performance reasons.  I'm all for performance improvements, but not at the expense of breakage.

Attached is a tarball with some files that reproduce the problem.  (The LibtoolImporter.py file is a stripped-down toy version of what we are really using.)  Unwind the tarball, cd to the directory, and run script.py.  Here's what I get:



shell prompt> pwd
/home/phil/pep302_bug
shell prompt> ls -CF
eggs.la  LibtoolImporter.py  script.py*  spam.py
shell prompt> python2.4.3 script.py
.la file(s) found in /home/phil/pep302_bug, LibtoolImporter will be used.
LibtoolImporter.find_module() couldn't find spam.la or spammodule.la
in /home/phil/pep302_bug.  Returning None.  This is *supposed* to cause a
fallback to the default import code looking for spam.py
in /home/phil/pep302_bug
module spam loaded
shell prompt> python2.5 script.py
.la file(s) found in /home/phil/pep302_bug, LibtoolImporter will be used.
LibtoolImporter.find_module() couldn't find spam.la or spammodule.la
in /home/phil/pep302_bug.  Returning None.  This is *supposed* to cause a
fallback to the default import code looking for spam.py
in /home/phil/pep302_bug
Traceback (most recent call last):
  File "script.py", line 4, in <module>
    import spam
ImportError: No module named spam
shell prompt>

----------------------------------------------------------------------

>Comment By: Raymond Hettinger (rhettinger)
Date: 2007-04-06 14:21

Message:
Logged In: YES 
user_id=80475
Originator: NO

Georg, I believe this was your checkin.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1695718&group_id=5470

From noreply at sourceforge.net  Fri Apr  6 22:56:53 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Fri, 06 Apr 2007 13:56:53 -0700
Subject: [ python-Bugs-1695688 ] Unexpected feature vs. integer and True values
Message-ID: <E1HZvUL-0005ds-4n@sc8-sf-web24.sourceforge.net>

Bugs item #1695688, was opened at 2007-04-06 15:28
Message generated for change (Comment added) made by gbrandl
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1695688&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Interpreter Core
Group: None
Status: Closed
Resolution: Invalid
Priority: 5
Private: No
Submitted By: dev at bacademy (bacademy)
Assigned to: Nobody/Anonymous (nobody)
Summary: Unexpected feature vs. integer and True values

Initial Comment:
Integer 1 interpreted as True.
I didn't expect the following:

Python 2.5 (r25:51908, Apr  4 2007, 10:19:35)
[GCC 4.1.1 20060724 (prerelease) (4.1.1-3mdk)] on linux2
Type "help", "copyright", "credits" or "license" for more information.

>>> v = 2
>>> 0.0 if v == True else float(v)
2.0
>>> v = 1
>>> 0.0 if v == True else float(v)
0.0

I know how to fix this, by writing

0.0 if v is True else float(v)

but to casual observer it is not obvious...






----------------------------------------------------------------------

>Comment By: Georg Brandl (gbrandl)
Date: 2007-04-06 20:56

Message:
Logged In: YES 
user_id=849994
Originator: NO

It may be, but it is not a bug (bool inherits from int) and documented (I
think...).

----------------------------------------------------------------------

Comment By: Georg Brandl (gbrandl)
Date: 2007-04-06 18:01

Message:
Logged In: YES 
user_id=849994
Originator: NO

It may be, but it is not a bug (bool inherits from int) and documented (I
think...).

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1695688&group_id=5470

From noreply at sourceforge.net  Sat Apr  7 01:27:17 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Fri, 06 Apr 2007 16:27:17 -0700
Subject: [ python-Bugs-1694663 ] Overloading  int.__pow__ does not work
Message-ID: <E1HZxpt-0000o9-KA@sc8-sf-web22.sourceforge.net>

Bugs item #1694663, was opened at 2007-04-05 01:44
Message generated for change (Comment added) made by zseil
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1694663&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: Python 2.5
Status: Open
Resolution: None
Priority: 3
Private: No
Submitted By: Terry J. Reedy (tjreedy)
Assigned to: Raymond Hettinger (rhettinger)
Summary: Overloading  int.__pow__ does not work

Initial Comment:
>From c.l.p: in response to glitch report, 2nd person showed that it is specific to __pow__

>>> class MyInt(int):
...     __sub__ = int.__add__
# similar lines for 9 other methods omitted
...     __or__ = int.__add__
...     __pow__ = int.__add__
...
>>> i = MyInt(42)
>>> i + 3
45
>>> i - 3
45
# similar outputs omitted
>>> i | 3
45
>>> i ** 3
74088

Another person (3rd) confirmed presence in 2.5 
Python 2.5 (r25:51908, Jan 21 2007, 03:10:25)
[GCC 3.4.6 20060404 (Red Hat 3.4.6-3)] 


----------------------------------------------------------------------

>Comment By: Ziga Seilnacht (zseil)
Date: 2007-04-07 01:27

Message:
Logged In: YES 
user_id=1326842
Originator: NO

Hi Raymond,

The problem is not in conversion functions, but in
slot inheritance.  If you run the example under a
debugger you can see that

MyInt->tp_as_number->nb_power == int->->tp_as_number->nb_power.

This happens because update_one_slot() doesn't fall
back to the generic slot function if a SlotWrapper in
type's dict has a different conversion function or if
it comes from an unrelated type.  The patch simply falls
back to the generic slot function in this case.

----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2007-04-05 06:54

Message:
Logged In: YES 
user_id=80475
Originator: NO

I'll take a look at your patch.

The root problem is that the wrapper functions will return NULL if they
are fed the wrong number of arguments (i.e. the int.__add__ wrapper expects
exactly two arguments but the ** call provides three) -- you will see a
similar result if int.__neg__ or int.__invert__ are assigned to __add__.  
It looks like an upstream step is seeing the NULL and deciding that it
needs to look to skip the given method call and instead try the base the
class.

Am lowering the priority because there is no good use case for deliberate
argument mismatches in the override.  Am leaving the bug open because we
want the appropriate error message to surface.



----------------------------------------------------------------------

Comment By: Ziga Seilnacht (zseil)
Date: 2007-04-05 03:47

Message:
Logged In: YES 
user_id=1326842
Originator: NO

Here is a patch (with tests) that should fix this.
There was another problem when the slot wrapper
came from a different type:

>>> class MyInt(int):
...     __mul__ = float.__add__
...
>>> MyInt(3) * 3
9

This now raises a TypeError.

File Added: mixing_slots.diff

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1694663&group_id=5470

From noreply at sourceforge.net  Sat Apr  7 08:22:08 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Fri, 06 Apr 2007 23:22:08 -0700
Subject: [ python-Bugs-1695948 ] logging.handlers.SocketHandler.makeSocket()
	blocks app
Message-ID: <E1Ha4JM-0000qj-Mp@sc8-sf-web22.sourceforge.net>

Bugs item #1695948, was opened at 2007-04-07 14:22
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1695948&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: jtdeng (jtdeng)
Assigned to: Nobody/Anonymous (nobody)
Summary: logging.handlers.SocketHandler.makeSocket() blocks app

Initial Comment:
Python Version: 
===============
Any Python Version

OS Platform:
============
Debian Linux 2.6.8-2-386 #1 Tue Aug 16 12:46:35 UTC 2005 i686 GNU/Linux
Windows XP SP2

Problem:
========
Member function makeSocket() of logging.handlers.SocketHandler creates a socket with no default timeout, and this may block the app on Linux.

def makeSocket(self):
    """
    A factory method which allows subclasses to define the precise
    type of socket they want.
    """
    s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    s.connect((self.host, self.port))
    return s 


if the log receiver on the destination host is not running, the log sender will block the app on socket.connect(), but on windows, socket.connect() will return immediately. So I propose to provide a default timeout value for makeSocket() like below:

def makeSocket(self, timeout=1):
    """
    A factory method which allows subclasses to define the precise
    type of socket they want.
    """
    s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    s.settimeout(timeout)
    s.connect((self.host, self.port))
    return s 




----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1695948&group_id=5470

From noreply at sourceforge.net  Sat Apr  7 14:40:41 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Sat, 07 Apr 2007 05:40:41 -0700
Subject: [ python-Bugs-1696025 ] codecs.EncodedFile() - the same data and file
	encodings
Message-ID: <E1HaADh-0001Tq-FL@sc8-sf-web24.sourceforge.net>

Bugs item #1696025, was opened at 2007-04-07 15:40
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1696025&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Alexandr V. Demeshko (adem)
Assigned to: Nobody/Anonymous (nobody)
Summary: codecs.EncodedFile() - the same data and file encodings

Initial Comment:
Python version: 2.5 for MS Windows

In Lib/codecs.py in lines 827-829 instead of:

    info = lookup(data_encoding)
    sr = StreamRecoder(file, info.encode, info.decode,
                       info.streamreader, info.streamwriter, errors)

should be something like:

    data_info = lookup(data_encoding)
    file_info = lookup(file_encoding)
    sr = StreamRecoder(file, data_info.encode, data_info.decode,
                       file_info.streamreader, file_info.streamwriter, errors)


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1696025&group_id=5470

From noreply at sourceforge.net  Sat Apr  7 23:38:28 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Sat, 07 Apr 2007 14:38:28 -0700
Subject: [ python-Bugs-1696199 ] Add collections.counts()
Message-ID: <E1HaIc8-0005sJ-SD@sc8-sf-web23.sourceforge.net>

Bugs item #1696199, was opened at 2007-04-07 15:38
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1696199&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.6
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Steven Bethard (bediviere)
Assigned to: Nobody/Anonymous (nobody)
Summary: Add collections.counts()

Initial Comment:
As suggested in http://mail.python.org/pipermail/python-list/2007-April/433986.html this is a patch to add a counts() function to the collections module. Usage looks like:

>>> items = 'acabbacba'
>>> item_counts = counts(items)
>>> for item in 'abcd':
...     print item, item_counts[item]
...
a 4
b 3
c 2
d 0

Yes, it's only a 4-line function, but it's a frequently re-written 4-line function.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1696199&group_id=5470

From noreply at sourceforge.net  Sun Apr  8 01:39:25 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Sat, 07 Apr 2007 16:39:25 -0700
Subject: [ python-Bugs-1696199 ] Add collections.counts()
Message-ID: <E1HaKVB-0002uQ-J7@sc8-sf-web23.sourceforge.net>

Bugs item #1696199, was opened at 2007-04-07 17:38
Message generated for change (Comment added) made by gvanrossum
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1696199&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.6
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Steven Bethard (bediviere)
Assigned to: Nobody/Anonymous (nobody)
Summary: Add collections.counts()

Initial Comment:
As suggested in http://mail.python.org/pipermail/python-list/2007-April/433986.html this is a patch to add a counts() function to the collections module. Usage looks like:

>>> items = 'acabbacba'
>>> item_counts = counts(items)
>>> for item in 'abcd':
...     print item, item_counts[item]
...
a 4
b 3
c 2
d 0

Yes, it's only a 4-line function, but it's a frequently re-written 4-line function.

----------------------------------------------------------------------

>Comment By: Guido van Rossum (gvanrossum)
Date: 2007-04-07 19:39

Message:
Logged In: YES 
user_id=6380
Originator: NO

Does it have to be a defaultdict?  I.e. is it important that
item_counts['d'] not raise KeyError?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1696199&group_id=5470

From noreply at sourceforge.net  Sun Apr  8 01:48:26 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Sat, 07 Apr 2007 16:48:26 -0700
Subject: [ python-Bugs-1696199 ] Add collections.counts()
Message-ID: <E1HaKdu-0006qh-Ew@sc8-sf-web21.sourceforge.net>

Bugs item #1696199, was opened at 2007-04-07 15:38
Message generated for change (Comment added) made by bediviere
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1696199&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.6
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Steven Bethard (bediviere)
Assigned to: Nobody/Anonymous (nobody)
Summary: Add collections.counts()

Initial Comment:
As suggested in http://mail.python.org/pipermail/python-list/2007-April/433986.html this is a patch to add a counts() function to the collections module. Usage looks like:

>>> items = 'acabbacba'
>>> item_counts = counts(items)
>>> for item in 'abcd':
...     print item, item_counts[item]
...
a 4
b 3
c 2
d 0

Yes, it's only a 4-line function, but it's a frequently re-written 4-line function.

----------------------------------------------------------------------

>Comment By: Steven Bethard (bediviere)
Date: 2007-04-07 17:48

Message:
Logged In: YES 
user_id=945502
Originator: YES

I think it's okay if it's not a defaultdict.  That was the simplest
implementation, but I certainly have no problem calling d.get() when
necessary. Should I change the implementation to use a dict()?

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2007-04-07 17:39

Message:
Logged In: YES 
user_id=6380
Originator: NO

Does it have to be a defaultdict?  I.e. is it important that
item_counts['d'] not raise KeyError?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1696199&group_id=5470

From noreply at sourceforge.net  Sun Apr  8 02:23:30 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Sat, 07 Apr 2007 17:23:30 -0700
Subject: [ python-Bugs-1696199 ] Add collections.counts()
Message-ID: <E1HaLBq-0000QU-Pk@sc8-sf-web22.sourceforge.net>

Bugs item #1696199, was opened at 2007-04-07 17:38
Message generated for change (Comment added) made by gvanrossum
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1696199&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.6
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Steven Bethard (bediviere)
Assigned to: Nobody/Anonymous (nobody)
Summary: Add collections.counts()

Initial Comment:
As suggested in http://mail.python.org/pipermail/python-list/2007-April/433986.html this is a patch to add a counts() function to the collections module. Usage looks like:

>>> items = 'acabbacba'
>>> item_counts = counts(items)
>>> for item in 'abcd':
...     print item, item_counts[item]
...
a 4
b 3
c 2
d 0

Yes, it's only a 4-line function, but it's a frequently re-written 4-line function.

----------------------------------------------------------------------

>Comment By: Guido van Rossum (gvanrossum)
Date: 2007-04-07 20:23

Message:
Logged In: YES 
user_id=6380
Originator: NO

I guess it's simplicity of implementation vs. simplicity of use.  And I'm
not even sure which is easier to use.  It's just that defaultdicts are a
very new thing and still feel "weird" -- even though I pushed for the
implementation based on popular demand I'm not a user myself.  Perhaps ask
around on python-dev?



----------------------------------------------------------------------

Comment By: Steven Bethard (bediviere)
Date: 2007-04-07 19:48

Message:
Logged In: YES 
user_id=945502
Originator: YES

I think it's okay if it's not a defaultdict.  That was the simplest
implementation, but I certainly have no problem calling d.get() when
necessary. Should I change the implementation to use a dict()?

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2007-04-07 19:39

Message:
Logged In: YES 
user_id=6380
Originator: NO

Does it have to be a defaultdict?  I.e. is it important that
item_counts['d'] not raise KeyError?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1696199&group_id=5470

From noreply at sourceforge.net  Sun Apr  8 06:18:55 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Sat, 07 Apr 2007 21:18:55 -0700
Subject: [ python-Bugs-1290505 ] strptime(): can't switch locales more than
	once
Message-ID: <E1HaOrf-0002oB-Vp@sc8-sf-web23.sourceforge.net>

Bugs item #1290505, was opened at 2005-09-13 15:50
Message generated for change (Comment added) made by bcannon
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1290505&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.5
>Status: Closed
>Resolution: Fixed
Priority: 5
Private: No
Submitted By: Adam Monsen (meonkeys)
Assigned to: Brett Cannon (bcannon)
Summary: strptime(): can't switch locales more than once

Initial Comment:
After calling strptime() once, it appears that
subsequent efforts to modify the locale settings (so
dates strings in different locales can be parsed) throw
a ValueError. I'm pasting everything here since spacing
is irrelevant:

import locale, time
print locale.getdefaultlocale()        # ('en_US', 'utf')
print locale.getlocale(locale.LC_TIME) # (None, None)
# save old locale
old_loc = locale.getlocale(locale.LC_TIME)
locale.setlocale(locale.LC_TIME, 'nl_NL')
print locale.getlocale(locale.LC_TIME) # ('nl_NL',
'ISO8859-1')
# parse local date
date = '10 augustus 2005 om 17:26'
format = '%d %B %Y om %H:%M'
dateTuple = time.strptime(date, format)
# switch back to previous locale
locale.setlocale(locale.LC_TIME, old_loc)
print locale.getlocale(locale.LC_TIME) # (None, None)
date = '10 August 2005 at 17:26'
format = '%d %B %Y at %H:%M'
dateTuple = time.strptime(date, format)

The output I get from this script is:

('en_US', 'utf')
(None, None)
('nl_NL', 'ISO8859-1')
(None, None)
Traceback (most recent call last):
  File "switching.py", line 17, in ?
    dateTuple = time.strptime(date, format)
  File "/usr/lib/python2.4/_strptime.py", line 292, in
strptime
    raise ValueError("time data did not match format: 
data=%s  fmt=%s" %
ValueError: time data did not match format:  data=10
August 2005 at 17:26  fmt=%d %B %Y at %H:%M


One workaround I found is by manually busting the
regular expression cache in _strptime:

import _strptime
_strptime._cache_lock.acquire()
_strptime._TimeRE_cache = _strptime.TimeRE()
_strptime._regex_cache = {}
_strptime._cache_lock.release()

If I do all that, I can change the LC_TIME part of the
locale as many times as I choose.

If this isn't a bug, this should at least be in the
documentation for the locale module and/or strptime().

----------------------------------------------------------------------

>Comment By: Brett Cannon (bcannon)
Date: 2007-04-07 21:18

Message:
Logged In: YES 
user_id=357491
Originator: NO

r54646 (along with a fix for the test in r54647) has the change.  This is
in the trunk.  I went with my test since if that test is wrong my whole
understanding of how time.strptime works is wrong in terms of caching.  =)

I will backport to the 2.5 branch once 2.5.1 is done since I missed the
deadline.  Thanks, Kovar, for all the help.

----------------------------------------------------------------------

Comment By: Brett Cannon (bcannon)
Date: 2007-03-29 13:03

Message:
Logged In: YES 
user_id=357491
Originator: NO

The test was checking that the TimeRE instance is recreated when the
locale changes.  You do have a valid point about the 'if' check; should
have put the setlocale call in an try/except block and just returned if an
exception was raised.

As for the %d usage of strptime, that is just to force a call into
strptime and thus trigger the new instance of TimeRE.  That is why the test
checks the id of the objects; don't really care about strptime directly
failing.  Did the test not fail properly even when you removed the 'if' but
left everything else alone?

----------------------------------------------------------------------

Comment By: Javier Sanz (kovan)
Date: 2007-03-29 09:53

Message:
Logged In: YES 
user_id=1426755
Originator: NO

I've been looking at the test case, and I noticed that isn't actually
checking anything, because locale.getlocale(locale.LC_TIME) is returning
(None,None), which is ok and just means that the default locale (which is
the C locale, not the system locale) is being used.
After removing that 'if' I also changed de_DE by es_ES to fit my system,
and strptime('10', '%d') by strptime('Fri', '%a') and strptime('vie','%a');
because '10' is '10' in all -occidental- languages, and the test would not
fail when the wrong locale is being used.

Once I made these changes to the test case, it successfully failed when
using the non-patched _strptime.py, AND ran ok when using the patched
version.

This is the test case I ended up using:



    def test_TimeRE_recreation(self):
        # The TimeRE instance should be recreated upon changing the
locale.
        locale_info = locale.getlocale(locale.LC_TIME)
        locale.setlocale(locale.LC_TIME, ('en_US', 'UTF8'))
        try:
            _strptime.strptime('Fri', '%a')
            first_time_re_id = id(_strptime._TimeRE_cache)
            locale.setlocale(locale.LC_TIME, ('es_ES', 'UTF8'))
            _strptime.strptime('vie', '%a')
            second_time_re_id = id(_strptime._TimeRE_cache)
            self.failIfEqual(first_time_re_id, second_time_re_id)
        finally:
            locale.setlocale(locale.LC_TIME, locale_info)


----------------------------------------------------------------------

Comment By: Brett Cannon (bcannon)
Date: 2007-03-28 19:07

Message:
Logged In: YES 
user_id=357491
Originator: NO

I have uploaded a patch for test_strptime that adds a test to make sure
that the TimeRE instance is recreated if the locale changes (went with
en_US and de_DE, but could easily be other locales if there are other ones
that are more common).  Let me know if the test runs fine and works.  Even
better is if it fails without the fix.
File Added: strptime_timere_test.diff

----------------------------------------------------------------------

Comment By: Javier Sanz (kovan)
Date: 2007-03-28 16:44

Message:
Logged In: YES 
user_id=1426755
Originator: NO

I'll be glad to help in whatever I can.

----------------------------------------------------------------------

Comment By: Brett Cannon (bcannon)
Date: 2007-03-28 16:40

Message:
Logged In: YES 
user_id=357491
Originator: NO

The power of procrastination in the morning.  =)  I am going to try to
come up with a test case for this.  I might ask, kovan, if you can run the
test case to make sure it works.

----------------------------------------------------------------------

Comment By: Javier Sanz (kovan)
Date: 2007-03-28 15:55

Message:
Logged In: YES 
user_id=1426755
Originator: NO

I applied the patch, and it works now :). 
Thanks bcannon for the quick responses.

----------------------------------------------------------------------

Comment By: Brett Cannon (bcannon)
Date: 2007-03-28 11:39

Message:
Logged In: YES 
user_id=357491
Originator: NO

kovan, can you please apply the patch I have uploaded to your copy of
_strptime and let me know if that fixes it?  I am oS X and switching
locales doesn't work for me so I don't have an easy way to test this.
File Added: strptime_cache.diff

----------------------------------------------------------------------

Comment By: Javier Sanz (kovan)
Date: 2007-03-28 00:06

Message:
Logged In: YES 
user_id=1426755
Originator: NO

This is the code:

def parseTime(strTime, format = "%a %b %d %H:%M:%S"):# example: Mon Aug 7
21:08:52                        

    locale.setlocale(locale.LC_TIME, ('en_US','UTF8'))    
    format = "%Y " + format
    strTime = str(datetime.now().year) + " " +strTime

    import _strptime
    _strptime._cache_lock.acquire()
    _strptime._TimeRE_cache = _strptime.TimeRE()
    _strptime._regex_cache = {}
    _strptime._cache_lock.release()    

    tuple = strptime(strTime, format)     
    return datetime(*tuple[0:6])


If I remove the code to clear the cache and add "print
format_regex.pattern" statement to _strptime.py after "format_regex =
time_re.compile(format)", I get 

(?P<Y>\d\d\d\d)\s*(?P<a>mi\???\???|s\???\???b|lun|mar|jue|vie|dom)\s*(?P<b>ene|feb|mar|abr|may|jun|jul|ago|sep|oct|nov|dic)\s*(?P<d>3[0-1]|[1-2]\d|0[1-9]|[1-9]|

[1-9])\s*(?P<H>2[0-3]|[0-1]\d|\d):(?P<M>[0-5]\d|\d):(?P<S>6[0-1]|[0-5]\d|\d)

which is in my system's locale (es), and it should be in english.

----------------------------------------------------------------------

Comment By: Brett Cannon (bcannon)
Date: 2007-03-27 19:35

Message:
Logged In: YES 
user_id=357491
Originator: NO

Can you show some code that recreatess the problem?

----------------------------------------------------------------------

Comment By: Javier Sanz (kovan)
Date: 2007-03-27 18:06

Message:
Logged In: YES 
user_id=1426755
Originator: NO

I think I'm having this issue with Python 2.5, as I can only make strptime
take into account locale.setlocale() calls if I clear strptime's internal
regexp cache between the calls to setlocal() and strptime().

----------------------------------------------------------------------

Comment By: Brett Cannon (bcannon)
Date: 2005-09-14 19:42

Message:
Logged In: YES 
user_id=357491

OK, the problem was that the cache for the locale
information in terms of dates and time was being invalidated
and recreated, but the regex cache was not being touched.  I
has now been fixed in rev. 1.41 for 2.5 and in rev. 1.38.2.3
for 2.4 .

Thanks for reporting this, Adam.

----------------------------------------------------------------------

Comment By: Adam Monsen (meonkeys)
Date: 2005-09-13 15:57

Message:
Logged In: YES 
user_id=259388

I think there were some long lines in my code. Attaching
test case.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1290505&group_id=5470

From noreply at sourceforge.net  Sun Apr  8 06:30:17 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Sat, 07 Apr 2007 21:30:17 -0700
Subject: [ python-Bugs-1569057 ] Using .next() on file open in write mode
	writes junk to file
Message-ID: <E1HaP2f-0004mz-Pq@sc8-sf-web24.sourceforge.net>

Bugs item #1569057, was opened at 2006-10-01 21:49
Message generated for change (Comment added) made by bcannon
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1569057&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Documentation
Group: Python 2.5
>Status: Closed
>Resolution: Fixed
Priority: 5
Private: No
Submitted By: andrei kulakov (rainy)
Assigned to: Brett Cannon (bcannon)
Summary: Using .next() on file open in write mode writes junk to file

Initial Comment:
When you open a file in write mode and use .next(), it 
prints an error and writes many lines of junk to file. 
I tested on windows and python 2.5:

>>> f = open('test', 'w') 
>>> f.fileno() 
4 
>>> f.write('1\n') 
>>> f.write('2\n3\n4\n') 
>>> f.next() 


Traceback (most recent call last): 
  File "<pyshell#8>", line 1, in <module> 
    f.next() 
IOError: [Errno 9] Bad file descriptor 

>>> f.close() 
>>> f = open('test') 
>>> f.next() 
'1\n' 
>>> f.next() 
'2\n' 
>>> f.next() 
'3\n' 
>>> f.next() 
'4\n' 
>>> f.next() 


'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x
00\x00\x00\x00\x00\x?
00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00
\x00\x00\x00\x00\x00\?
x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00
\x00\x00\x00 
...many more lines of junk...'



----------------------------------------------------------------------

>Comment By: Brett Cannon (bcannon)
Date: 2007-04-07 21:30

Message:
Logged In: YES 
user_id=357491
Originator: NO

r54712 has the documentation note.

----------------------------------------------------------------------

Comment By: Brett Cannon (bcannon)
Date: 2007-03-20 17:44

Message:
Logged In: YES 
user_id=357491
Originator: NO

I think the docs should just say switching is undefined and leave it at
that.  Calling 'next' on a file open for writing just makes no sense
anyway.  And who knows what the OS will do if you try to misuse a file
descriptor.

Other option is to raise an exception when 'next' is called on a file
opened for writing (TypeError?).  I think that makes more sense. 
Unfortunately having to check the mode every time you call 'next' might be
considered unacceptable.  Anyone have an opinion?

Either way I will go with one of the two solutions.

----------------------------------------------------------------------

Comment By: andrei kulakov (rainy)
Date: 2006-10-07 09:04

Message:
Logged In: YES 
user_id=511010

I tried it on win2k sp4 running python 2.5 and right now 
tried it on winXP running python 2.4 - same error. Also, at 
least one user on python newsgroup tried it and got a 
slightly different result: he did not get the error on
 .next() but the junk was written to the file. I paste my 
latest attempt right out of IDLE, without any modification 
(my first paste was also straight from idle). Except that I 
trim the junk characters, because there's more than a 
screen of 'em.

IDLE 1.1.2      
>>> f = open('test', 'w')
>>> f.write('1\n')
>>> f.next()

Traceback (most recent call last):
  File "<pyshell#2>", line 1, in -toplevel-
    f.next()
IOError: [Errno 9] Bad file descriptor
>>> f.close()
>>> f = open('test')
>>> f.next()
'1\n'
>>> f.next()
"\x95\x00\xc8\xe ......."


Please let me know if you need to know anything else...

----------------------------------------------------------------------

Comment By: Skip Montanaro (montanaro)
Date: 2006-10-07 03:38

Message:
Logged In: YES 
user_id=44345

Works for me.  (Nearly current build from SVN.) I find it implausible that
your 
explanation of failing to flush the file is the cause of the problem since
closing 
the file will cause it to be flushed.  You didn't open the file for "w+"
so there's no 
opportunity to switch between writing and reading.

What platform are you using?


----------------------------------------------------------------------

Comment By: andrei kulakov (rainy)
Date: 2006-10-03 18:23

Message:
Logged In: YES 
user_id=511010

Python newsgroup folks said that this is normal because when 
switching between write/read modes without doing flush() 
first, result is undefined. There was a suggestion from 
Terry Ready to add this to documentation on file methods:

"When switching from reading or writing (or vice versa), 
call flush(), or 
the result will be undefined."

thx, -andrei

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1569057&group_id=5470

From noreply at sourceforge.net  Sun Apr  8 06:55:27 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Sat, 07 Apr 2007 21:55:27 -0700
Subject: [ python-Bugs-1695718 ] PEP 302 broken
Message-ID: <E1HaPR1-0008SF-3l@sc8-sf-web21.sourceforge.net>

Bugs item #1695718, was opened at 2007-04-06 09:32
Message generated for change (Comment added) made by bcannon
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1695718&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Interpreter Core
Group: Python 2.5
Status: Open
Resolution: None
Priority: 6
Private: No
Submitted By: phil (philipdumont)
Assigned to: Georg Brandl (gbrandl)
Summary: PEP 302 broken

Initial Comment:
The product I'm working on uses a PEP 302 importer hook.  It's a home-grown ltihooks clone.  (We don't use ltihooks because it's GPLed, and we don't want to be.)

Our importer worked on 2.4.3, is broken on 2.5.0.

What's broken is that if the hook's find_module() method returns None for a given module name, say 'spam', then that is supposed to cause the import machinery to fall back on the regular unhooked import behavior -- that is, find 'spam.py' (or 'spam.pyc', 'spam.pyo') in the directory in question.  This used to be what happened, but no longer.

Tracing through the code, the problem seems to be occurring due to the 'continue' at line 1289 (in the 2.5 tarball) of Python/import.c.

Slogging through SVN (aside: this would have been easier if your ViewSVN supported annotate/blame -- any chance you could add that?), it appears that the offending continue statement was added in revision 46372, whose checkin comment claims that it was done for performance reasons.  I'm all for performance improvements, but not at the expense of breakage.

Attached is a tarball with some files that reproduce the problem.  (The LibtoolImporter.py file is a stripped-down toy version of what we are really using.)  Unwind the tarball, cd to the directory, and run script.py.  Here's what I get:



shell prompt> pwd
/home/phil/pep302_bug
shell prompt> ls -CF
eggs.la  LibtoolImporter.py  script.py*  spam.py
shell prompt> python2.4.3 script.py
.la file(s) found in /home/phil/pep302_bug, LibtoolImporter will be used.
LibtoolImporter.find_module() couldn't find spam.la or spammodule.la
in /home/phil/pep302_bug.  Returning None.  This is *supposed* to cause a
fallback to the default import code looking for spam.py
in /home/phil/pep302_bug
module spam loaded
shell prompt> python2.5 script.py
.la file(s) found in /home/phil/pep302_bug, LibtoolImporter will be used.
LibtoolImporter.find_module() couldn't find spam.la or spammodule.la
in /home/phil/pep302_bug.  Returning None.  This is *supposed* to cause a
fallback to the default import code looking for spam.py
in /home/phil/pep302_bug
Traceback (most recent call last):
  File "script.py", line 4, in <module>
    import spam
ImportError: No module named spam
shell prompt>

----------------------------------------------------------------------

>Comment By: Brett Cannon (bcannon)
Date: 2007-04-07 21:55

Message:
Logged In: YES 
user_id=357491
Originator: NO

I don't agree with this interpretation of PEP 302 for this instance.  If
you read the PEP there is no explicit mention that if an importer for an
entry on sys.path fails that it falls back on the default import behaviour.
 The only mention of using the default behaviour is if a value of None is
stored in sys.path_importer_cache (which also occurs when no entry on
sys.path_hooks returns a usable importer).

In my interpretation of PEP 302 (and how I implemented it for my pure
Python import implementation), if an importer exists for an entry on
sys.path then it essentially "owns" that entry.  The default import
semantics only kick in for unclaimed sys.path entries in my view.

Now I could be wrong and if I am I hope Phil can point out where in PEP
302 I am wrong.  Otherwise we have either undocumented behaviour that
changed (and that is always messy) or implied semantics that got fixed.

In other words, this probably needs to go to python-dev if Phil can't
point out exactly where PEP 302 says the semantics he is expecting is
supposed to be that way.

----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2007-04-06 12:21

Message:
Logged In: YES 
user_id=80475
Originator: NO

Georg, I believe this was your checkin.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1695718&group_id=5470

From noreply at sourceforge.net  Sun Apr  8 08:59:15 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Sat, 07 Apr 2007 23:59:15 -0700
Subject: [ python-Bugs-1696199 ] Add collections.counts()
Message-ID: <E1HaRMp-0006yC-7V@sc8-sf-web23.sourceforge.net>

Bugs item #1696199, was opened at 2007-04-07 16:38
Message generated for change (Settings changed) made by rhettinger
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1696199&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.6
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Steven Bethard (bediviere)
>Assigned to: Raymond Hettinger (rhettinger)
Summary: Add collections.counts()

Initial Comment:
As suggested in http://mail.python.org/pipermail/python-list/2007-April/433986.html this is a patch to add a counts() function to the collections module. Usage looks like:

>>> items = 'acabbacba'
>>> item_counts = counts(items)
>>> for item in 'abcd':
...     print item, item_counts[item]
...
a 4
b 3
c 2
d 0

Yes, it's only a 4-line function, but it's a frequently re-written 4-line function.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2007-04-07 19:23

Message:
Logged In: YES 
user_id=6380
Originator: NO

I guess it's simplicity of implementation vs. simplicity of use.  And I'm
not even sure which is easier to use.  It's just that defaultdicts are a
very new thing and still feel "weird" -- even though I pushed for the
implementation based on popular demand I'm not a user myself.  Perhaps ask
around on python-dev?



----------------------------------------------------------------------

Comment By: Steven Bethard (bediviere)
Date: 2007-04-07 18:48

Message:
Logged In: YES 
user_id=945502
Originator: YES

I think it's okay if it's not a defaultdict.  That was the simplest
implementation, but I certainly have no problem calling d.get() when
necessary. Should I change the implementation to use a dict()?

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2007-04-07 18:39

Message:
Logged In: YES 
user_id=6380
Originator: NO

Does it have to be a defaultdict?  I.e. is it important that
item_counts['d'] not raise KeyError?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1696199&group_id=5470

From noreply at sourceforge.net  Sun Apr  8 09:40:04 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Sun, 08 Apr 2007 00:40:04 -0700
Subject: [ python-Bugs-1696025 ] codecs.EncodedFile() - the same data and file
	encodings
Message-ID: <E1HaS0K-0006LN-Nv@sc8-sf-web22.sourceforge.net>

Bugs item #1696025, was opened at 2007-04-07 14:40
Message generated for change (Comment added) made by doerwalter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1696025&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: None
>Status: Closed
>Resolution: Out of Date
Priority: 5
Private: No
Submitted By: Alexandr V. Demeshko (adem)
Assigned to: Nobody/Anonymous (nobody)
Summary: codecs.EncodedFile() - the same data and file encodings

Initial Comment:
Python version: 2.5 for MS Windows

In Lib/codecs.py in lines 827-829 instead of:

    info = lookup(data_encoding)
    sr = StreamRecoder(file, info.encode, info.decode,
                       info.streamreader, info.streamwriter, errors)

should be something like:

    data_info = lookup(data_encoding)
    file_info = lookup(file_encoding)
    sr = StreamRecoder(file, data_info.encode, data_info.decode,
                       file_info.streamreader, file_info.streamwriter, errors)


----------------------------------------------------------------------

>Comment By: Walter D?rwald (doerwalter)
Date: 2007-04-08 09:40

Message:
Logged In: YES 
user_id=89016
Originator: NO

This has already been fixed in svn (r52517/r52518).

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1696025&group_id=5470

From noreply at sourceforge.net  Sun Apr  8 13:44:54 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Sun, 08 Apr 2007 04:44:54 -0700
Subject: [ python-Bugs-1696390 ] Strange performance with unicode and lists
Message-ID: <E1HaVpG-000252-54@sc8-sf-web23.sourceforge.net>

Bugs item #1696390, was opened at 2007-04-08 15:44
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1696390&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Unicode
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Vsevolod (sv75)
Assigned to: M.-A. Lemburg (lemburg)
Summary: Strange performance with unicode and lists

Initial Comment:
I found some strange error (?) when using list of unicodes. Here is a small sample:

# Alex in Russian
l = [u'\u0410\u043b\u0435\u043a\u0441'] 
# print method 1
for s in l:
    print s.encode("utf-8")
# print method 2
print [s.encode("utf-8") for s in l]

Output is:
?????     
['\xd0\x90\xd0\xbb\xd0\xb5\xd0\xba\xd1\x81']

I suppose than output of both methods should be equal. Or maybe I'm wrong? 

Python versions: 2.4.4, 2.5 (both version)
OS: Ubuntu 6.10, Debian Etch

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1696390&group_id=5470

From noreply at sourceforge.net  Sun Apr  8 14:34:42 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Sun, 08 Apr 2007 05:34:42 -0700
Subject: [ python-Bugs-1696390 ] Strange performance with unicode and lists
Message-ID: <E1HaWbS-0003ps-4t@sc8-sf-web24.sourceforge.net>

Bugs item #1696390, was opened at 2007-04-08 11:44
Message generated for change (Comment added) made by gbrandl
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1696390&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Unicode
Group: Python 2.5
>Status: Closed
>Resolution: Invalid
Priority: 5
Private: No
Submitted By: Vsevolod (sv75)
Assigned to: M.-A. Lemburg (lemburg)
Summary: Strange performance with unicode and lists

Initial Comment:
I found some strange error (?) when using list of unicodes. Here is a small sample:

# Alex in Russian
l = [u'\u0410\u043b\u0435\u043a\u0441'] 
# print method 1
for s in l:
    print s.encode("utf-8")
# print method 2
print [s.encode("utf-8") for s in l]

Output is:
?????     
['\xd0\x90\xd0\xbb\xd0\xb5\xd0\xba\xd1\x81']

I suppose than output of both methods should be equal. Or maybe I'm wrong? 

Python versions: 2.4.4, 2.5 (both version)
OS: Ubuntu 6.10, Debian Etch

----------------------------------------------------------------------

>Comment By: Georg Brandl (gbrandl)
Date: 2007-04-08 12:34

Message:
Logged In: YES 
user_id=849994
Originator: NO

No, the output is correct. The __str__() of a list does a __repr__() of
its contents,
to avoid confusion e.g. in this situation:

>>> l = ["a, b, c", "d, e, f"]

If list.__str__ did __str__ on its items, it would look like this:

>>> print l
[a, b, c, d, e, f]

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1696390&group_id=5470

From noreply at sourceforge.net  Sun Apr  8 14:58:07 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Sun, 08 Apr 2007 05:58:07 -0700
Subject: [ python-Bugs-1695718 ] PEP 302 broken
Message-ID: <E1HaWy7-0008WE-CH@sc8-sf-web22.sourceforge.net>

Bugs item #1695718, was opened at 2007-04-06 16:32
Message generated for change (Comment added) made by gbrandl
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1695718&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Interpreter Core
Group: Python 2.5
>Status: Closed
>Resolution: Wont Fix
Priority: 6
Private: No
Submitted By: phil (philipdumont)
Assigned to: Georg Brandl (gbrandl)
Summary: PEP 302 broken

Initial Comment:
The product I'm working on uses a PEP 302 importer hook.  It's a home-grown ltihooks clone.  (We don't use ltihooks because it's GPLed, and we don't want to be.)

Our importer worked on 2.4.3, is broken on 2.5.0.

What's broken is that if the hook's find_module() method returns None for a given module name, say 'spam', then that is supposed to cause the import machinery to fall back on the regular unhooked import behavior -- that is, find 'spam.py' (or 'spam.pyc', 'spam.pyo') in the directory in question.  This used to be what happened, but no longer.

Tracing through the code, the problem seems to be occurring due to the 'continue' at line 1289 (in the 2.5 tarball) of Python/import.c.

Slogging through SVN (aside: this would have been easier if your ViewSVN supported annotate/blame -- any chance you could add that?), it appears that the offending continue statement was added in revision 46372, whose checkin comment claims that it was done for performance reasons.  I'm all for performance improvements, but not at the expense of breakage.

Attached is a tarball with some files that reproduce the problem.  (The LibtoolImporter.py file is a stripped-down toy version of what we are really using.)  Unwind the tarball, cd to the directory, and run script.py.  Here's what I get:



shell prompt> pwd
/home/phil/pep302_bug
shell prompt> ls -CF
eggs.la  LibtoolImporter.py  script.py*  spam.py
shell prompt> python2.4.3 script.py
.la file(s) found in /home/phil/pep302_bug, LibtoolImporter will be used.
LibtoolImporter.find_module() couldn't find spam.la or spammodule.la
in /home/phil/pep302_bug.  Returning None.  This is *supposed* to cause a
fallback to the default import code looking for spam.py
in /home/phil/pep302_bug
module spam loaded
shell prompt> python2.5 script.py
.la file(s) found in /home/phil/pep302_bug, LibtoolImporter will be used.
LibtoolImporter.find_module() couldn't find spam.la or spammodule.la
in /home/phil/pep302_bug.  Returning None.  This is *supposed* to cause a
fallback to the default import code looking for spam.py
in /home/phil/pep302_bug
Traceback (most recent call last):
  File "script.py", line 4, in <module>
    import spam
ImportError: No module named spam
shell prompt>

----------------------------------------------------------------------

>Comment By: Georg Brandl (gbrandl)
Date: 2007-04-08 12:58

Message:
Logged In: YES 
user_id=849994
Originator: NO

You're correct, Brett.

I found the implicit fallback at the Iceland sprint and we decided to
change
that in 2.5:

http://mail.python.org/pipermail/python-dev/2006-May/065174.html

PEP 302 has even been edited to say:

    [...] Note
    that if the callable returns an importer object for a specific
    sys.path entry, the builtin import machinery will not be invoked
    to handle that entry any longer, even if the importer object later
    fails to find a specific module.

Closing as "Won't fix", if more need for discussion arises, please take
this to python-dev.

----------------------------------------------------------------------

Comment By: Brett Cannon (bcannon)
Date: 2007-04-08 04:55

Message:
Logged In: YES 
user_id=357491
Originator: NO

I don't agree with this interpretation of PEP 302 for this instance.  If
you read the PEP there is no explicit mention that if an importer for an
entry on sys.path fails that it falls back on the default import behaviour.
 The only mention of using the default behaviour is if a value of None is
stored in sys.path_importer_cache (which also occurs when no entry on
sys.path_hooks returns a usable importer).

In my interpretation of PEP 302 (and how I implemented it for my pure
Python import implementation), if an importer exists for an entry on
sys.path then it essentially "owns" that entry.  The default import
semantics only kick in for unclaimed sys.path entries in my view.

Now I could be wrong and if I am I hope Phil can point out where in PEP
302 I am wrong.  Otherwise we have either undocumented behaviour that
changed (and that is always messy) or implied semantics that got fixed.

In other words, this probably needs to go to python-dev if Phil can't
point out exactly where PEP 302 says the semantics he is expecting is
supposed to be that way.

----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2007-04-06 19:21

Message:
Logged In: YES 
user_id=80475
Originator: NO

Georg, I believe this was your checkin.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1695718&group_id=5470

From noreply at sourceforge.net  Mon Apr  9 07:56:02 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Sun, 08 Apr 2007 22:56:02 -0700
Subject: [ python-Bugs-1695948 ] logging.handlers.SocketHandler.makeSocket()
	blocks app
Message-ID: <E1HamrC-0006qP-MD@sc8-sf-web22.sourceforge.net>

Bugs item #1695948, was opened at 2007-04-06 23:22
Message generated for change (Comment added) made by nnorwitz
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1695948&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: jtdeng (jtdeng)
>Assigned to: Vinay Sajip (vsajip)
Summary: logging.handlers.SocketHandler.makeSocket() blocks app

Initial Comment:
Python Version: 
===============
Any Python Version

OS Platform:
============
Debian Linux 2.6.8-2-386 #1 Tue Aug 16 12:46:35 UTC 2005 i686 GNU/Linux
Windows XP SP2

Problem:
========
Member function makeSocket() of logging.handlers.SocketHandler creates a socket with no default timeout, and this may block the app on Linux.

def makeSocket(self):
    """
    A factory method which allows subclasses to define the precise
    type of socket they want.
    """
    s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    s.connect((self.host, self.port))
    return s 


if the log receiver on the destination host is not running, the log sender will block the app on socket.connect(), but on windows, socket.connect() will return immediately. So I propose to provide a default timeout value for makeSocket() like below:

def makeSocket(self, timeout=1):
    """
    A factory method which allows subclasses to define the precise
    type of socket they want.
    """
    s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    s.settimeout(timeout)
    s.connect((self.host, self.port))
    return s 




----------------------------------------------------------------------

>Comment By: Neal Norwitz (nnorwitz)
Date: 2007-04-08 22:56

Message:
Logged In: YES 
user_id=33168
Originator: NO

Vinay, could you take a look?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1695948&group_id=5470

From noreply at sourceforge.net  Mon Apr  9 07:58:11 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Sun, 08 Apr 2007 22:58:11 -0700
Subject: [ python-Bugs-1696740 ] README is referencing obsolete?
	http://starship.python.net
Message-ID: <E1HamtH-0000c0-UP@sc8-sf-web24.sourceforge.net>

Bugs item #1696740, was opened at 2007-04-09 07:58
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1696740&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Documentation
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Jiri Navratil (plavcik)
Assigned to: Nobody/Anonymous (nobody)
Summary: README is referencing obsolete? http://starship.python.net

Initial Comment:
In README file obtained 09.04.2004 from Python SVN HEAD via svn is referenced 

There's also a Python community web site at
http://starship.python.net/.

I can't open this site. I'm getting

unable to find IP address for starship.python.net

Is this information obsolete?


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1696740&group_id=5470

From noreply at sourceforge.net  Mon Apr  9 15:40:05 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Mon, 09 Apr 2007 06:40:05 -0700
Subject: [ python-Bugs-1695718 ] PEP 302 broken
Message-ID: <E1Hau6H-00012b-1G@sc8-sf-web22.sourceforge.net>

Bugs item #1695718, was opened at 2007-04-06 12:32
Message generated for change (Comment added) made by philipdumont
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1695718&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Interpreter Core
Group: Python 2.5
Status: Closed
Resolution: Wont Fix
Priority: 6
Private: No
Submitted By: phil (philipdumont)
Assigned to: Georg Brandl (gbrandl)
Summary: PEP 302 broken

Initial Comment:
The product I'm working on uses a PEP 302 importer hook.  It's a home-grown ltihooks clone.  (We don't use ltihooks because it's GPLed, and we don't want to be.)

Our importer worked on 2.4.3, is broken on 2.5.0.

What's broken is that if the hook's find_module() method returns None for a given module name, say 'spam', then that is supposed to cause the import machinery to fall back on the regular unhooked import behavior -- that is, find 'spam.py' (or 'spam.pyc', 'spam.pyo') in the directory in question.  This used to be what happened, but no longer.

Tracing through the code, the problem seems to be occurring due to the 'continue' at line 1289 (in the 2.5 tarball) of Python/import.c.

Slogging through SVN (aside: this would have been easier if your ViewSVN supported annotate/blame -- any chance you could add that?), it appears that the offending continue statement was added in revision 46372, whose checkin comment claims that it was done for performance reasons.  I'm all for performance improvements, but not at the expense of breakage.

Attached is a tarball with some files that reproduce the problem.  (The LibtoolImporter.py file is a stripped-down toy version of what we are really using.)  Unwind the tarball, cd to the directory, and run script.py.  Here's what I get:



shell prompt> pwd
/home/phil/pep302_bug
shell prompt> ls -CF
eggs.la  LibtoolImporter.py  script.py*  spam.py
shell prompt> python2.4.3 script.py
.la file(s) found in /home/phil/pep302_bug, LibtoolImporter will be used.
LibtoolImporter.find_module() couldn't find spam.la or spammodule.la
in /home/phil/pep302_bug.  Returning None.  This is *supposed* to cause a
fallback to the default import code looking for spam.py
in /home/phil/pep302_bug
module spam loaded
shell prompt> python2.5 script.py
.la file(s) found in /home/phil/pep302_bug, LibtoolImporter will be used.
LibtoolImporter.find_module() couldn't find spam.la or spammodule.la
in /home/phil/pep302_bug.  Returning None.  This is *supposed* to cause a
fallback to the default import code looking for spam.py
in /home/phil/pep302_bug
Traceback (most recent call last):
  File "script.py", line 4, in <module>
    import spam
ImportError: No module named spam
shell prompt>

----------------------------------------------------------------------

>Comment By: phil (philipdumont)
Date: 2007-04-09 09:40

Message:
Logged In: YES 
user_id=1364034
Originator: YES

Ok.  I concede that I misread the PEP.  And I won't dispute the NO CHANGE
resolution.  But before I drop it entirely, I wish to make one more
comment.

>From PEP 302: "The hooks proposed in this PEP...would allow the
__import__-based tools to get rid of most of their import.c emulation
code."

This implementation change causes hooks that handle directories containing
both custom modules and "normal" modules to have to do some of that
import.c emulation.  Which is what I guess I will do.

----------------------------------------------------------------------

Comment By: Georg Brandl (gbrandl)
Date: 2007-04-08 08:58

Message:
Logged In: YES 
user_id=849994
Originator: NO

You're correct, Brett.

I found the implicit fallback at the Iceland sprint and we decided to
change
that in 2.5:

http://mail.python.org/pipermail/python-dev/2006-May/065174.html

PEP 302 has even been edited to say:

    [...] Note
    that if the callable returns an importer object for a specific
    sys.path entry, the builtin import machinery will not be invoked
    to handle that entry any longer, even if the importer object later
    fails to find a specific module.

Closing as "Won't fix", if more need for discussion arises, please take
this to python-dev.

----------------------------------------------------------------------

Comment By: Brett Cannon (bcannon)
Date: 2007-04-08 00:55

Message:
Logged In: YES 
user_id=357491
Originator: NO

I don't agree with this interpretation of PEP 302 for this instance.  If
you read the PEP there is no explicit mention that if an importer for an
entry on sys.path fails that it falls back on the default import behaviour.
 The only mention of using the default behaviour is if a value of None is
stored in sys.path_importer_cache (which also occurs when no entry on
sys.path_hooks returns a usable importer).

In my interpretation of PEP 302 (and how I implemented it for my pure
Python import implementation), if an importer exists for an entry on
sys.path then it essentially "owns" that entry.  The default import
semantics only kick in for unclaimed sys.path entries in my view.

Now I could be wrong and if I am I hope Phil can point out where in PEP
302 I am wrong.  Otherwise we have either undocumented behaviour that
changed (and that is always messy) or implied semantics that got fixed.

In other words, this probably needs to go to python-dev if Phil can't
point out exactly where PEP 302 says the semantics he is expecting is
supposed to be that way.

----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2007-04-06 15:21

Message:
Logged In: YES 
user_id=80475
Originator: NO

Georg, I believe this was your checkin.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1695718&group_id=5470

From noreply at sourceforge.net  Mon Apr  9 16:16:23 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Mon, 09 Apr 2007 07:16:23 -0700
Subject: [ python-Bugs-1615275 ] tempile.TemporaryFile differences between
	linux and windows
Message-ID: <E1HaufP-0004z2-0b@sc8-sf-web22.sourceforge.net>

Bugs item #1615275, was opened at 2006-12-13 16:20
Message generated for change (Comment added) made by draghuram
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1615275&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Extension Modules
Group: Platform-specific
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: hirzel (hirzel)
Assigned to: Nobody/Anonymous (nobody)
Summary: tempile.TemporaryFile differences between linux and windows

Initial Comment:
This bug came up when trying to write a numpy array to a tempfile.TemporaryFile() using the numpy 'tofile' method on windows using python 2.4.  

with a numpy array 'a', and a TemporaryFile 'f', 
on windows:
>>> a.tofile(f)

throws an IOError, where on Linux it does not.
On windows, you must use a.tofile(f.file)

The cause of this difference is that in windows, tempfile.TemporaryFile() returns <type 'instance'> that has a 'file'  attribute of <type 'file'>, whereas in linux tempfile.TemporaryFile() returns <type 'file'> and there is no 'file' attribute.  

Ideally, the windows version would align with linux, and the module documentation and TemporaryFile() would return a <type 'file'>.  If this is not possible, it seems like the linux version and docs should be changed to match the windows version to align cross-platform behavior.  At least, that seems to be the shared opinion of this thread from the mailing list: numpy-discussion.  http://www.mail-archive.com/numpy-discussion at scipy.org/msg00271.html

To my knowledge, while platform differences in tempfile have been reported in the past, this one has not.



----------------------------------------------------------------------

Comment By: Raghuram Devarakonda (draghuram)
Date: 2007-04-09 10:16

Message:
Logged In: YES 
user_id=984087
Originator: NO


I posted in python-dev about this bug asking if TemporaryFile() can also
return a wrapper on all platforms but immediately realized that always
returning a wrapper instead of a file object would break any existing code
that depends on the return value being a file object. There were no other
comments from any one on the dev list either. I think the best bet is to
update the document with the following details.

1) "NamedTemporaryFile" returns a file-like object. The actual file object
can be accessed with "file" member of the returned instance.

2) "TemporaryFile" on windows is same as "NamedTemporaryFile".

3) cross-platform code that needs access to file object (such as
array.tofile) should check for the return object type and behave
accordingly.


----------------------------------------------------------------------

Comment By: Raghuram Devarakonda (draghuram)
Date: 2007-03-29 14:36

Message:
Logged In: YES 
user_id=984087
Originator: NO


After looking at tempfile.py, the reason for the difference in behaviour
is clear. On windows, "TemporaryFile" is an alias for "NamedTemporaryFile".
NamedTemporaryFile() returns a wrapper instance with file-like interface
but which is not actually a file. This is not a problem when file
operations like "write" and "close" are used directly. But array.tofile()
explicitly checks for file type object and hence fails with
NamedTemporaryFile(). Same is the reason for numpy failure as reported by
OP (I haven't explicitly analyzed numpy failure but gleaned this info from
the discussion thread in the initial post). 

Even though the reason is clear, I think the end result is a bit
unsatisfactory. array.tofile() (and numpy's tofile()) need to pass
different parameters depending on the platform. One possible solution is
for "TemporaryFile" to return a wrapper as well. Then, tofile() can be
called with TemporaryFile().file on all platforms. 






----------------------------------------------------------------------

Comment By: Raghuram Devarakonda (draghuram)
Date: 2007-03-28 10:43

Message:
Logged In: YES 
user_id=984087
Originator: NO


I used the following code to reproduce the problem on windows XP.

----------
import array
import tempfile

testarray = array.array('B')
testarray.fromstring("\x00\x00\x00")
f = tempfile.TemporaryFile()
testarray.tofile(f)
-----------

This works fine on linux but on windows, it gives the following error:

-------------
Traceback (most recent call last):
  File "c:\rags\tofile.py", line 7, in <module>
    testarray.tofile(f)
TypeError: arg must be open file
-------------

Changing "f" to "f.file" seems to work, though, as explained in the
initial post. So this may be the same problem as OP reported even though I
am getting TypeError and he mentioned IOError. 

I tested with 2.4 and 2.5 as I don't know how to set up python development
environment on windows (yet). I will see if I can set that up first before
working on the "fix". 

Raghu.





----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1615275&group_id=5470

From noreply at sourceforge.net  Mon Apr  9 18:13:36 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Mon, 09 Apr 2007 09:13:36 -0700
Subject: [ python-Bugs-1685000 ] asyncore DoS vulnerability
Message-ID: <E1HawUq-0003rq-VR@sc8-sf-web24.sourceforge.net>

Bugs item #1685000, was opened at 2007-03-21 02:15
Message generated for change (Comment added) made by josiahcarlson
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1685000&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.5
Status: Open
Resolution: None
Priority: 9
Private: No
Submitted By: billiejoex (billiejoex)
Assigned to: Nobody/Anonymous (nobody)
Summary: asyncore DoS vulnerability

Initial Comment:
DoS asyncore vulnerability

asyncore, independently if used with select() or poll(), suffers a DoS-type vulnerability when a high number of simultaneous connections to handle simultaneously is reached.
The number of maximum connections is system-dependent as well as the type of error raised.
I attached two simple Proof of Concept scripts demonstrating such bug.
If you want to try the behaviours listed below run the attached "asyncore_server.py" and "asyncore_client.py" scripts on your local workstation.

On my Windows XP system (Python 2.5), independently if asyncore has been used to develop a server or a client, the error is raised by select() inside asyncore's "poll" function when 512 (socket_map's elements) simultaneous connections are reached. 
Here's the traceback I get:

[...]
connections: 510
connections: 511
connections: 512
Traceback (most recent call last):
  File "C:\scripts\asyncore_server.py", line 38, in <module>
    asyncore.loop()
  File "C:\Python25\lib\asyncore.py", line 191, in loop
    poll_fun(timeout, map)
  File "C:\Python25\lib\asyncore.py", line 121, in poll
    r, w, e = select.select(r, w, e, timeout)
ValueError: too many file descriptors in select()


On my Linux Ubuntu 6.10 (kernel 2.6.17-10, Python 2.5) different type of errors are raised depending on the application (client or server).
In an asyncore-based client the error is raised by socket module (dispatcher's "self.socket" attribute) inside 'connect' method of 'dispatcher' class:

[...]
connections: 1018
connections: 1019
connections: 1020
connections: 1021
Traceback (most recent call last):
  File "asyncore_client.py", line 31, in <module>
  File "asyncore.py", line 191, in loop
  File "asyncore.py", line 138, in poll
  File "asyncore.py", line 80, in write
  File "asyncore.py", line 76, in write
  File "asyncore.py", line 395, in handle_write_event
  File "asyncore_client.py", line 24, in handle_connect
  File "asyncore_client.py", line 9, in __init__
  File "asyncore.py", line 257, in create_socket
  File "socket.py", line 156, in __init__
socket.error: (24, 'Too many open files')


On an asyncore-based server the error is raised by socket module (dispatcher's "self.socket" attribute) inside 'accept' method of 'dispatcher' class:

[...]
connections: 1019
connections: 1020
connections: 1021
Traceback (most recent call last):
  File "asyncore_server.py", line 38, in <module>
  File "asyncore.py", line 191, in loop
  File "asyncore.py", line 132, in poll
  File "asyncore.py", line 72, in read
  File "asyncore.py", line 68, in read
  File "asyncore.py", line 384, in handle_read_event
  File "asyncore_server.py", line 16, in handle_accept
  File "asyncore.py", line 321, in accept
  File "socket.py", line 170, in accept
socket.error: (24, 'Too many open files')


----------------------------------------------------------------------

Comment By: Josiah Carlson (josiahcarlson)
Date: 2007-04-09 09:13

Message:
Logged In: YES 
user_id=341410
Originator: NO

Assign the "bug" to me, I'm the maintainer for asyncore/asynchat.

With that said, since a user needs to override
asyncore.dispatcher.handle_accept() anyways, which necessarily needs to
call asyncore.dispatcher.accept(), the subclass is free to check the number
of sockets in its socket map before creating a new instance of whatever
subclass of asyncore.dispatcher the user has written.

Also, the number of file handles that select can handle on Windows is a
compile-time constant, and has nothing to do with the actual number of open
file handles.  Take and run the following source file on Windows and see
how the total number of open sockets can be significantly larger than the
number of sockets passed to select():

import socket
import asyncore
import random

class new_map(dict):
    def items(self):
        r = [(i,j) for i,j in dict.items(self) if not random.randrange(4)
and j != h]
        r.append((h._fileno, h))
        print len(r), len(asyncore.socket_map)
        return r

asyncore.socket_map = new_map()

class listener(asyncore.dispatcher):
    def handle_accept(self):
        x = self.accept()
        if x:
            conn, addr = x
            connection(conn)

class connection(asyncore.dispatcher):
    def writable(self):
        0
    def handle_connect(self):
        pass

if __name__ == '__main__':
    h = listener()
    h.create_socket(socket.AF_INET, socket.SOCK_STREAM)
    h.bind(('127.0.0.1', 10001))
    h.listen(5)
    while len(asyncore.socket_map) < 4096:
        a = connection()
        a.create_socket(socket.AF_INET, socket.SOCK_STREAM)
        a.connect_ex(('127.0.0.1', 10001))
        asyncore.poll()

The tail end of a run in Windows:

476 1934
501 1936
516 1938
Traceback (most recent call last):
  File "D:\MYDOCS\Projects\python\async_socket.py", line 37, in ?
    asyncore.poll()
  File "C:\python23\lib\asyncore.py", line 108, in poll
    r, w, e = select.select(r, w, e, timeout)
ValueError: too many file descriptors in select()

With a proper definition of new_map, you can handle an unlimited number of
sockets with asyncore by choosing blocks of sockets to return in its
items() call.  Note that I used random out of convenience, a proper
implementation could distribute the sockets based on fileno, time inserted,
etc.  You can do this on linux, but only if you have used ulimit, as sgala
stated.

I would also mention that a better approach is to create an access time
mapping with blacklisting support to accept/close sockets that have
hammered your server.  It would also make sense to handle socket timeouts
(no read/write on a socket for X seconds).

Regardless, none of these things are within the world of problems that
base asyncore is intended to solve.  It's a bare-bones async socket
implementation.  If you want more features, use twisted or write your own
subclasses and offer it up on the Python Cookbook (activestate.com) or in
the Python Package Index (python.org/pypi).  If the community finds that
they are useful and productive features, we may consider it for inclusion
in the standard library.

Until then, suggested close.  Will close as "will not fix" if it is
assigned to me.

----------------------------------------------------------------------

Comment By: billiejoex (billiejoex)
Date: 2007-04-06 05:10

Message:
Logged In: YES 
user_id=1357589
Originator: YES

> Putting a try/except in place doesn't really help the problem... 
> if you fail to create a new socket what action will you take?

Calling a new dispatcher's method (for example:
"handle_exceeded_connection") could be an idea. By default it could close
the current session but it can be overriden if the user want to take some
other action.

> A better approach is to have a configurable limit on the number of 
> open connections, and then have a server-specific reaction to 
> exceeding that limit.

It doesn't work if I create additional file-objects during the execution
of the loop...

----------------------------------------------------------------------

Comment By: Santiago Gala (sgala)
Date: 2007-04-05 16:09

Message:
Logged In: YES 
user_id=178886
Originator: NO

THe limit of resources that an OS can deal with is limited due to
resources, both globally and per user, precisely to avoid DoS attacks by a
uses.

In the case of linux, you can control it with "ulimit -n XXXX" (for the
user that runs the test). The default here is 1024 (and the maximum, unless
it is globally raised).

I imagine windows will have similar limits, and similar (registry, etc.)
ways to handle them.

----------------------------------------------------------------------

Comment By: Sam Rushing (rushing)
Date: 2007-03-30 10:22

Message:
Logged In: YES 
user_id=73736
Originator: NO

Turns out medusa doesn't have a socket counter class, that was some other
project I was thinking of.

Putting a try/except in place doesn't really help the problem... if you
fail to create a new socket what action will you take?

A better approach is to have a configurable limit on the number of open
connections, and then have a server-specific reaction to exceeding that
limit.  For example, an SMTP server might respond with a 4XX greeting and
close the connection.

An additional problem on Unix is that running out of descriptors affects
more than just sockets.  Once you hit the FD limit you can't open files,
or do anything that requires a descriptor.


----------------------------------------------------------------------

Comment By: billiejoex (billiejoex)
Date: 2007-03-29 07:03

Message:
Logged In: YES 
user_id=1357589
Originator: YES

> The problem is that there's no portable way to know what the limit
> on file descriptors is.

Why don't put a try/except statement before creating a new socket's
file-descriptor?
I believe that such problem shouldn't be understimated. 
Actually asyncore is the only asynchronous module present in Python's
stdlib.
If asyncore suffers a DoS vulnerability it just means that writing a
secure client/server application with Python without using a third-party
library isn't possible.

I wrote a modified version of asyncore that solves the problem with select
 (affecting Windows) but still can't find a way to solve the problem with
socket's file-descriptors (affecting Unix).

----------------------------------------------------------------------

Comment By: Sam Rushing (rushing)
Date: 2007-03-23 12:59

Message:
Logged In: YES 
user_id=73736
Originator: NO

The problem is that there's no portable way to know what the limit
on file descriptors is.  The 'classic' for select/poll is the FD_SETSIZE
macro.  But on some operating systems there is no such limit. [e.g.,
win32
does not use the 'lowest-free-int' model common to unix].

I believe that in Medusa there was a derived class or extension that
counted
the number of open sockets, and limited it, using something like a
semaphore.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1685000&group_id=5470

From noreply at sourceforge.net  Mon Apr  9 18:18:17 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Mon, 09 Apr 2007 09:18:17 -0700
Subject: [ python-Bugs-1695948 ] logging.handlers.SocketHandler.makeSocket()
	blocks app
Message-ID: <E1HawZN-0001Ye-Io@sc8-sf-web22.sourceforge.net>

Bugs item #1695948, was opened at 2007-04-07 06:22
Message generated for change (Comment added) made by vsajip
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1695948&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.5
>Status: Closed
>Resolution: Fixed
Priority: 5
Private: No
Submitted By: jtdeng (jtdeng)
Assigned to: Vinay Sajip (vsajip)
Summary: logging.handlers.SocketHandler.makeSocket() blocks app

Initial Comment:
Python Version: 
===============
Any Python Version

OS Platform:
============
Debian Linux 2.6.8-2-386 #1 Tue Aug 16 12:46:35 UTC 2005 i686 GNU/Linux
Windows XP SP2

Problem:
========
Member function makeSocket() of logging.handlers.SocketHandler creates a socket with no default timeout, and this may block the app on Linux.

def makeSocket(self):
    """
    A factory method which allows subclasses to define the precise
    type of socket they want.
    """
    s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    s.connect((self.host, self.port))
    return s 


if the log receiver on the destination host is not running, the log sender will block the app on socket.connect(), but on windows, socket.connect() will return immediately. So I propose to provide a default timeout value for makeSocket() like below:

def makeSocket(self, timeout=1):
    """
    A factory method which allows subclasses to define the precise
    type of socket they want.
    """
    s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    s.settimeout(timeout)
    s.connect((self.host, self.port))
    return s 




----------------------------------------------------------------------

>Comment By: Vinay Sajip (vsajip)
Date: 2007-04-09 16:18

Message:
Logged In: YES 
user_id=308438
Originator: NO

Change (not exactly as above) checked into SVN trunk. New functionality,
so not backported.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2007-04-09 05:56

Message:
Logged In: YES 
user_id=33168
Originator: NO

Vinay, could you take a look?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1695948&group_id=5470

From noreply at sourceforge.net  Mon Apr  9 19:42:05 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Mon, 09 Apr 2007 10:42:05 -0700
Subject: [ python-Bugs-1695718 ] PEP 302 broken
Message-ID: <E1HaxsT-0002Ez-TN@sc8-sf-web22.sourceforge.net>

Bugs item #1695718, was opened at 2007-04-06 09:32
Message generated for change (Comment added) made by bcannon
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1695718&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Interpreter Core
Group: Python 2.5
Status: Closed
Resolution: Wont Fix
Priority: 6
Private: No
Submitted By: phil (philipdumont)
Assigned to: Georg Brandl (gbrandl)
Summary: PEP 302 broken

Initial Comment:
The product I'm working on uses a PEP 302 importer hook.  It's a home-grown ltihooks clone.  (We don't use ltihooks because it's GPLed, and we don't want to be.)

Our importer worked on 2.4.3, is broken on 2.5.0.

What's broken is that if the hook's find_module() method returns None for a given module name, say 'spam', then that is supposed to cause the import machinery to fall back on the regular unhooked import behavior -- that is, find 'spam.py' (or 'spam.pyc', 'spam.pyo') in the directory in question.  This used to be what happened, but no longer.

Tracing through the code, the problem seems to be occurring due to the 'continue' at line 1289 (in the 2.5 tarball) of Python/import.c.

Slogging through SVN (aside: this would have been easier if your ViewSVN supported annotate/blame -- any chance you could add that?), it appears that the offending continue statement was added in revision 46372, whose checkin comment claims that it was done for performance reasons.  I'm all for performance improvements, but not at the expense of breakage.

Attached is a tarball with some files that reproduce the problem.  (The LibtoolImporter.py file is a stripped-down toy version of what we are really using.)  Unwind the tarball, cd to the directory, and run script.py.  Here's what I get:



shell prompt> pwd
/home/phil/pep302_bug
shell prompt> ls -CF
eggs.la  LibtoolImporter.py  script.py*  spam.py
shell prompt> python2.4.3 script.py
.la file(s) found in /home/phil/pep302_bug, LibtoolImporter will be used.
LibtoolImporter.find_module() couldn't find spam.la or spammodule.la
in /home/phil/pep302_bug.  Returning None.  This is *supposed* to cause a
fallback to the default import code looking for spam.py
in /home/phil/pep302_bug
module spam loaded
shell prompt> python2.5 script.py
.la file(s) found in /home/phil/pep302_bug, LibtoolImporter will be used.
LibtoolImporter.find_module() couldn't find spam.la or spammodule.la
in /home/phil/pep302_bug.  Returning None.  This is *supposed* to cause a
fallback to the default import code looking for spam.py
in /home/phil/pep302_bug
Traceback (most recent call last):
  File "script.py", line 4, in <module>
    import spam
ImportError: No module named spam
shell prompt>

----------------------------------------------------------------------

>Comment By: Brett Cannon (bcannon)
Date: 2007-04-09 10:42

Message:
Logged In: YES 
user_id=357491
Originator: NO

I am planning to work on the import machinery in general for this year so
hopefully things will improve.

----------------------------------------------------------------------

Comment By: phil (philipdumont)
Date: 2007-04-09 06:40

Message:
Logged In: YES 
user_id=1364034
Originator: YES

Ok.  I concede that I misread the PEP.  And I won't dispute the NO CHANGE
resolution.  But before I drop it entirely, I wish to make one more
comment.

>From PEP 302: "The hooks proposed in this PEP...would allow the
__import__-based tools to get rid of most of their import.c emulation
code."

This implementation change causes hooks that handle directories containing
both custom modules and "normal" modules to have to do some of that
import.c emulation.  Which is what I guess I will do.

----------------------------------------------------------------------

Comment By: Georg Brandl (gbrandl)
Date: 2007-04-08 05:58

Message:
Logged In: YES 
user_id=849994
Originator: NO

You're correct, Brett.

I found the implicit fallback at the Iceland sprint and we decided to
change
that in 2.5:

http://mail.python.org/pipermail/python-dev/2006-May/065174.html

PEP 302 has even been edited to say:

    [...] Note
    that if the callable returns an importer object for a specific
    sys.path entry, the builtin import machinery will not be invoked
    to handle that entry any longer, even if the importer object later
    fails to find a specific module.

Closing as "Won't fix", if more need for discussion arises, please take
this to python-dev.

----------------------------------------------------------------------

Comment By: Brett Cannon (bcannon)
Date: 2007-04-07 21:55

Message:
Logged In: YES 
user_id=357491
Originator: NO

I don't agree with this interpretation of PEP 302 for this instance.  If
you read the PEP there is no explicit mention that if an importer for an
entry on sys.path fails that it falls back on the default import behaviour.
 The only mention of using the default behaviour is if a value of None is
stored in sys.path_importer_cache (which also occurs when no entry on
sys.path_hooks returns a usable importer).

In my interpretation of PEP 302 (and how I implemented it for my pure
Python import implementation), if an importer exists for an entry on
sys.path then it essentially "owns" that entry.  The default import
semantics only kick in for unclaimed sys.path entries in my view.

Now I could be wrong and if I am I hope Phil can point out where in PEP
302 I am wrong.  Otherwise we have either undocumented behaviour that
changed (and that is always messy) or implied semantics that got fixed.

In other words, this probably needs to go to python-dev if Phil can't
point out exactly where PEP 302 says the semantics he is expecting is
supposed to be that way.

----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2007-04-06 12:21

Message:
Logged In: YES 
user_id=80475
Originator: NO

Georg, I believe this was your checkin.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1695718&group_id=5470

From noreply at sourceforge.net  Mon Apr  9 21:58:04 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Mon, 09 Apr 2007 12:58:04 -0700
Subject: [ python-Bugs-1697169 ] package.pth file name not used as described. 
Message-ID: <E1Hb004-00028P-CF@sc8-sf-web23.sourceforge.net>

Bugs item #1697169, was opened at 2007-04-09 14:58
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1697169&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Documentation
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: cfk (carlfk)
Assigned to: Nobody/Anonymous (nobody)
Summary: package.pth file name not used as described. 

Initial Comment:
"...whose name has the form package.pth" http://www.python.org/doc/lib/module-site.html 

as far as I can tell, that is incorrect.  the .pth file can be named anything - only the name of the dir listed in it is used as a package/module name.

It is implemented in site.py : 

def addpackage(sitedir, name, known_paths):

    fullname = os.path.join(sitedir, name)

        f = open(fullname, "rU")

        for line in f:

            dir, dircase = makepath(sitedir, line)
            if not dircase in known_paths and os.path.exists(dir):
                sys.path.append(dir)


Notice name is not added to sys.path.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1697169&group_id=5470

From noreply at sourceforge.net  Mon Apr  9 22:06:42 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Mon, 09 Apr 2007 13:06:42 -0700
Subject: [ python-Bugs-1697175 ] winreg module for cygwin?
Message-ID: <E1Hb08Q-0000za-2P@sc8-sf-web22.sourceforge.net>

Bugs item #1697175, was opened at 2007-04-09 20:06
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1697175&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Windows
Group: Platform-specific
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Zooko O'Whielacronx (zooko)
Assigned to: Nobody/Anonymous (nobody)
Summary: winreg module for cygwin?

Initial Comment:
Please make the winreg module work on cygwin.  Nowadays cygwin has a /proc/registry file system, which might provide an easy way to do it.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1697175&group_id=5470

From noreply at sourceforge.net  Mon Apr  9 23:23:56 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Mon, 09 Apr 2007 14:23:56 -0700
Subject: [ python-Bugs-1697215 ] execute
Message-ID: <E1Hb1LA-0002sy-Qt@sc8-sf-web23.sourceforge.net>

"execute sitedir if it starts with 'import'" doesn'
Mime-Version: 1.0
Content-Type: text/plain; charset="UTF-8"
X-SourceForge-Tracker-unixname: python
X-SourceForge-Tracker-trackerid: 105470
X-SourceForge-Tracker-itemid: 1697215
X-SourceForge-Tracker-itemstatus: Open
X-SourceForge-Tracker-itemassignee: nobody
X-SourceForge-Tracker-itemupdate-reason: Tracker Item Submitted
X-SourceForge-Tracker-itemupdate-username: Item Submitter

Bugs item #1697215, was opened at 2007-04-09 16:23
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1697215&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Documentation
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: cfk (carlfk)
Assigned to: Nobody/Anonymous (nobody)
Summary: execute
"execute sitedir if it starts with 'import'" doesn'

Initial Comment:
docstring says "execute sitedir" - but that isn't what gets executed. 

suggested: 
    """Add a new path to known_paths by combining sitedir and 'name' 
or execute lines in name.pth that start with 'import'"""

site.py

def addpackage(sitedir, name, known_paths):

    """Add a new path to known_paths by combining sitedir and 'name' or execute sitedir if it starts with 'import'"""

    fullname = os.path.join(sitedir, name)
        f = open(fullname, "rU")
        for line in f:
            if line.startswith("import"):
                exec line
                                      

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1697215&group_id=5470

From noreply at sourceforge.net  Tue Apr 10 01:05:22 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Mon, 09 Apr 2007 16:05:22 -0700
Subject: [ python-Bugs-1697248 ] Property with -> annotation triggers assert
Message-ID: <E1Hb2vK-00068y-JI@sc8-sf-web24.sourceforge.net>

Bugs item #1697248, was opened at 2007-04-09 19:05
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1697248&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Parser/Compiler
Group: Python 3000
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Guido van Rossum (gvanrossum)
Assigned to: Nobody/Anonymous (nobody)
Summary: Property with -> annotation triggers assert

Initial Comment:
Example:

# x.py
class C:
  @property
  def p(self) -> int:
    return 0

$ ./python -E x.py
python: ../Python/ast.c:724: ast_for_arguments: Assertion `((n)->n_type) == 263 || ((n)->n_type) == 267' failed.
Aborted
$

Removing "@property" or "-> int" from the example avoids the problem.  Adding argument annotations inside the argument list doesn't trigger it.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1697248&group_id=5470

From noreply at sourceforge.net  Tue Apr 10 08:48:34 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Mon, 09 Apr 2007 23:48:34 -0700
Subject: [ python-Bugs-1681974 ] mkdtemp fails on Windows if username has
	non-ASCII character
Message-ID: <E1HbA9a-0001tB-Ca@sc8-sf-web22.sourceforge.net>

Bugs item #1681974, was opened at 2007-03-16 12:03
Message generated for change (Comment added) made by niemisto
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1681974&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Markus Niemist? (niemisto)
Assigned to: Nobody/Anonymous (nobody)
Summary: mkdtemp fails on Windows if username has non-ASCII character

Initial Comment:
mkdtemp fails miserably on Windows if Windows user name has any non-ASCII characters, like ?? or ??, in it. mkdtemp throws an encoding error.

This seems to be because the default temp dir in Windows is "c:\documents and settings\<user name>\local settings\temp". Now if the user name has non-ASCII characters ASCII decoder cannot handle it and creating temp directories won't work.

As a work around I have used the following code:

tempdir = unicode(tempfile.gettempdir(), 'mbcs')
mkdtemp(suffix='foo', dir=tempdir)

This applies for both Python 2.4 and Python 2.5.

----------------------------------------------------------------------

>Comment By: Markus Niemist? (niemisto)
Date: 2007-04-10 09:48

Message:
Logged In: YES 
user_id=459586
Originator: YES

Here is traceback. Sorry it took so long.

Traceback (most recent call last):
  File "c:\util\home\xxx\xxx.py", line 350, in OnOpen
    dir = tempfile.mkdtemp(prefix=u'test')
  File "C:\python24\lib\tempfile.py", line 326, in mkdtemp
    file = _os.path.join(dir, prefix + name + suffix)
  File "c:\python24\lib\ntpath.py", line 102, in join
    path += "\\" + b
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe4 in position 12:
ordinal not in range(128)


----------------------------------------------------------------------

Comment By: Georg Brandl (gbrandl)
Date: 2007-03-21 13:42

Message:
Logged In: YES 
user_id=849994
Originator: NO

Could you indicate where exactly what error is raised?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1681974&group_id=5470

From noreply at sourceforge.net  Tue Apr 10 15:21:43 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Tue, 10 Apr 2007 06:21:43 -0700
Subject: [ python-Bugs-223616 ] compiler package needs better documentation.
Message-ID: <E1HbGI3-00061b-3s@sc8-sf-web23.sourceforge.net>

Bugs item #223616, was opened at 2000-11-27 15:33
Message generated for change (Comment added) made by facundobatista
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=223616&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Documentation
Group: Feature Request
>Status: Closed
>Resolution: Fixed
Priority: 6
Private: No
Submitted By: Fred L. Drake, Jr. (fdrake)
Assigned to: Jeremy Hylton (jhylton)
Summary: compiler package needs better documentation.

Initial Comment:
The compiler package is not documented; this needs to be done before tools are likely to be written on top of it.

----------------------------------------------------------------------

>Comment By: Facundo Batista (facundobatista)
Date: 2007-04-10 10:21

Message:
Logged In: YES 
user_id=752496
Originator: NO

The SVN log reflects that the docs *have* been moved into the Library
Reference, and that they have been corrected and polished.

Fred, you said they needed polishing in December 2001. From then,
corrections have been made from Georg, Raymond, Brett and edloper.

I'm closing this, but feel free to reopen it if you think that this docs
*still* need more polishing.

Regards,

----------------------------------------------------------------------

Comment By: Facundo Batista (facundobatista)
Date: 2007-03-28 18:00

Message:
Logged In: YES 
user_id=752496
Originator: NO

This is one of the oldest bugs we have...

As the compiler package surely got revised with the inclusion of AST, and
the bug was only to remember about polishing... this still needs to be
open?


----------------------------------------------------------------------

Comment By: Jeremy Hylton (jhylton)
Date: 2002-03-01 19:38

Message:
Logged In: YES 
user_id=31392

jeremy, get your butt in gear on this!


----------------------------------------------------------------------

Comment By: Fred L. Drake, Jr. (fdrake)
Date: 2001-12-21 15:19

Message:
Logged In: YES 
user_id=3066

I'll note that Jeremy has written basic docs, and they are
in the Library Reference.  But he really needs to polish
them a bit as well, so this stays open but gets a new summary.

----------------------------------------------------------------------

Comment By: Fred L. Drake, Jr. (fdrake)
Date: 2001-08-10 18:56

Message:
Logged In: YES 
user_id=3066

Jeremy said he'd work on this next week, so I'm bumping the
priority as a reminder.

----------------------------------------------------------------------

Comment By: Fred L. Drake, Jr. (fdrake)
Date: 2001-01-08 00:48

Message:
Jeremy -- is there anything that can be done to make it easier for you to
get this done?  I think these should really be documented and moved into
the library.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=223616&group_id=5470

From noreply at sourceforge.net  Tue Apr 10 16:00:11 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Tue, 10 Apr 2007 07:00:11 -0700
Subject: [ python-Bugs-223616 ] compiler package needs better documentation.
Message-ID: <E1HbGtH-0000cS-8C@sc8-sf-web22.sourceforge.net>

Bugs item #223616, was opened at 2000-11-27 13:33
Message generated for change (Comment added) made by fdrake
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=223616&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Documentation
Group: Feature Request
Status: Closed
Resolution: Fixed
Priority: 6
Private: No
Submitted By: Fred L. Drake, Jr. (fdrake)
Assigned to: Jeremy Hylton (jhylton)
Summary: compiler package needs better documentation.

Initial Comment:
The compiler package is not documented; this needs to be done before tools are likely to be written on top of it.

----------------------------------------------------------------------

>Comment By: Fred L. Drake, Jr. (fdrake)
Date: 2007-04-10 10:00

Message:
Logged In: YES 
user_id=3066
Originator: YES

I think this is fine, Facundo.  All docs need polishing, but that's not a
bug so long as they're complete, accurate, and readable.

Thanks for checking on this.

----------------------------------------------------------------------

Comment By: Facundo Batista (facundobatista)
Date: 2007-04-10 09:21

Message:
Logged In: YES 
user_id=752496
Originator: NO

The SVN log reflects that the docs *have* been moved into the Library
Reference, and that they have been corrected and polished.

Fred, you said they needed polishing in December 2001. From then,
corrections have been made from Georg, Raymond, Brett and edloper.

I'm closing this, but feel free to reopen it if you think that this docs
*still* need more polishing.

Regards,

----------------------------------------------------------------------

Comment By: Facundo Batista (facundobatista)
Date: 2007-03-28 17:00

Message:
Logged In: YES 
user_id=752496
Originator: NO

This is one of the oldest bugs we have...

As the compiler package surely got revised with the inclusion of AST, and
the bug was only to remember about polishing... this still needs to be
open?


----------------------------------------------------------------------

Comment By: Jeremy Hylton (jhylton)
Date: 2002-03-01 17:38

Message:
Logged In: YES 
user_id=31392

jeremy, get your butt in gear on this!


----------------------------------------------------------------------

Comment By: Fred L. Drake, Jr. (fdrake)
Date: 2001-12-21 13:19

Message:
Logged In: YES 
user_id=3066

I'll note that Jeremy has written basic docs, and they are
in the Library Reference.  But he really needs to polish
them a bit as well, so this stays open but gets a new summary.

----------------------------------------------------------------------

Comment By: Fred L. Drake, Jr. (fdrake)
Date: 2001-08-10 17:56

Message:
Logged In: YES 
user_id=3066

Jeremy said he'd work on this next week, so I'm bumping the
priority as a reminder.

----------------------------------------------------------------------

Comment By: Fred L. Drake, Jr. (fdrake)
Date: 2001-01-07 22:48

Message:
Jeremy -- is there anything that can be done to make it easier for you to
get this done?  I think these should really be documented and moved into
the library.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=223616&group_id=5470

From noreply at sourceforge.net  Tue Apr 10 16:06:53 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Tue, 10 Apr 2007 07:06:53 -0700
Subject: [ python-Bugs-1696740 ] README is referencing obsolete?
	http://starship.python.net
Message-ID: <E1HbGzl-0002i8-Ut@sc8-sf-web23.sourceforge.net>

Bugs item #1696740, was opened at 2007-04-09 02:58
Message generated for change (Comment added) made by gagenellina
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1696740&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Documentation
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Jiri Navratil (plavcik)
Assigned to: Nobody/Anonymous (nobody)
Summary: README is referencing obsolete? http://starship.python.net

Initial Comment:
In README file obtained 09.04.2004 from Python SVN HEAD via svn is referenced 

There's also a Python community web site at
http://starship.python.net/.

I can't open this site. I'm getting

unable to find IP address for starship.python.net

Is this information obsolete?


----------------------------------------------------------------------

Comment By: Gabriel Genellina (gagenellina)
Date: 2007-04-10 11:06

Message:
Logged In: YES 
user_id=479790
Originator: NO

No, the site had some problems but it's up and running again.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1696740&group_id=5470

From noreply at sourceforge.net  Tue Apr 10 17:55:38 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Tue, 10 Apr 2007 08:55:38 -0700
Subject: [ python-Bugs-1697782 ] types.InstanceType is missing but used by
	pydoc
Message-ID: <E1HbIh0-0004zB-JI@sc8-sf-web22.sourceforge.net>

Bugs item #1697782, was opened at 2007-04-10 17:55
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1697782&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 3000
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Christian Heimes (tiran)
Assigned to: Nobody/Anonymous (nobody)
Summary: types.InstanceType is missing but used by pydoc

Initial Comment:
>>> help(callable.__call__)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/heimes/dev/python/p3yk/Lib/site.py", line 348, in __call__
    return pydoc.help(*args, **kwds)
  File "/home/heimes/dev/python/p3yk/Lib/pydoc.py", line 1658, in __call__
    self.help(request)
  File "/home/heimes/dev/python/p3yk/Lib/pydoc.py", line 1702, in help
    else: doc(request, 'Help on %s:')
  File "/home/heimes/dev/python/p3yk/Lib/pydoc.py", line 1477, in doc
    desc = describe(object)
  File "/home/heimes/dev/python/p3yk/Lib/pydoc.py", line 1436, in describe
    if type(thing) is types.InstanceType:
AttributeError: 'module' object has no attribute 'InstanceType'


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1697782&group_id=5470

From noreply at sourceforge.net  Tue Apr 10 19:06:27 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Tue, 10 Apr 2007 10:06:27 -0700
Subject: [ python-Bugs-1697820 ] __getslice__ still used in built-in types
Message-ID: <E1HbJnX-0007t1-HZ@sc8-sf-web24.sourceforge.net>

Bugs item #1697820, was opened at 2007-04-10 19:06
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1697820&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Documentation
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Torsten Bronger (bronger)
Assigned to: Nobody/Anonymous (nobody)
Summary: __getslice__ still used in built-in types

Initial Comment:
In the Python language reference <http://docs.python.org/ref/sequence-methods.html>, after the sentence

"""Deprecated since release 2.0. Support slice objects as parameters to the __getitem__() method."""

I suggest the following addition:

"""However, built-in types in CPython currently still implement __getslice__.  Therefore, you have to override it in derived classes when implementing slicing."""

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1697820&group_id=5470

From noreply at sourceforge.net  Tue Apr 10 21:47:49 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Tue, 10 Apr 2007 12:47:49 -0700
Subject: [ python-Bugs-1697916 ] Segfaults on memory error
Message-ID: <E1HbMJh-00081c-9h@sc8-sf-web21.sourceforge.net>

Bugs item #1697916, was opened at 2007-04-10 21:47
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1697916&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Interpreter Core
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: STINNER Victor (haypo)
Assigned to: Nobody/Anonymous (nobody)
Summary: Segfaults on memory error

Initial Comment:
Hi, I'm playing with resource.setrlimit(resource.RLIMIT_AS) to limit memory usage during fuzzing tests. It works quite well but Python crashs sometimes (with SEGFAULT).

I downloaded Python source code and recompiled it with EXTRA_FLAGS="-g -O0" to find errors. I found three bugs and wrote a patch for all of them.

Comments:
* Objects/exceptions.c:33: allocation may returns NULL on memory error
* Objects/longobject.c:2511: long_divrem() may allocate new long integers but l_divmod() doesn't check that div and mod are not NULL
* Objects/object.c:1284: problem with NULL mro. I don't understand how mro works, but I think that the error may be catched when mro is assigned. Problem: where is it done? in Objects/typeobject.c?

So don't apply my patch directly: fix for object.c may be wrong.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1697916&group_id=5470

From noreply at sourceforge.net  Tue Apr 10 22:58:04 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Tue, 10 Apr 2007 13:58:04 -0700
Subject: [ python-Bugs-1697943 ] msgfmt cannot cope with BOM
Message-ID: <E1HbNPg-0001CU-DM@sc8-sf-web23.sourceforge.net>

Bugs item #1697943, was opened at 2007-04-10 22:58
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1697943&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Demos and Tools
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Christoph Zwerschke (cito)
Assigned to: Nobody/Anonymous (nobody)
Summary: msgfmt cannot cope with BOM

Initial Comment:
If a .po file has a BOM (byte order mark) at the beginning, as is often the case for utf-8 files created on Windows, msgfmt.py complines about a syntax error.

The attached patch fixes this problem.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1697943&group_id=5470

From noreply at sourceforge.net  Wed Apr 11 08:26:00 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Tue, 10 Apr 2007 23:26:00 -0700
Subject: [ python-Bugs-1698167 ] xml.etree document element.tag
Message-ID: <E1HbWHI-0008Bm-1Z@sc8-sf-web23.sourceforge.net>

Bugs item #1698167, was opened at 2007-04-11 06:25
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1698167&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Documentation
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: paul rubin (phr)
Assigned to: Nobody/Anonymous (nobody)
Summary: xml.etree document element.tag

Initial Comment:
The xml.etree docs vaguely mention an implementation-dependent Element interface without describing it in any detail.  I could not figure out from the docs how to get the tag name of an element returned from (say) the getiterator interface.  That is, for an element like <foo/>, I wanted the string "foo".  Examining the library source showed that e.tag does the job, at least some of the time, and that was enough to get my app working.  Could the actual situation please be documented--thanks.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1698167&group_id=5470

From noreply at sourceforge.net  Wed Apr 11 14:58:45 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Wed, 11 Apr 2007 05:58:45 -0700
Subject: [ python-Bugs-1698398 ] wrong % of params for format string in
	ZipFile.printdir()
Message-ID: <E1HbcPN-0002GS-Ny@sc8-sf-web23.sourceforge.net>

Bugs item #1698398, was opened at 2007-04-11 14:58
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1698398&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Szabolcs Berecz (szabihernyo)
Assigned to: Nobody/Anonymous (nobody)
Summary: wrong % of params for format string in ZipFile.printdir()

Initial Comment:
In zipfile.py:448
date = "%d-%02d-%02d %02d:%02d:%02d" % zinfo.date_time
should be changed to
date = "%d-%02d-%02d %02d:%02d:%02d" % zinfo.date_time[:6]

checked with python2.[45] on windows and linux


>>> from zipfile import ZipFile
>>> from StringIO import StringIO
>>> s = StringIO()
>>> zf = ZipFile(s, 'w')
>>> zf.writestr('file.ext', '123')
>>> zf.printdir()
File Name                                             Modified             Size
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.5/zipfile.py", line 448, in printdir
    date = "%d-%02d-%02d %02d:%02d:%02d" % zinfo.date_time
TypeError: int argument required
>>> zf.filelist[0].date_time
(2007, 4, 11, 13, 38, 58, 2, 101, 1)
>>>


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1698398&group_id=5470

From noreply at sourceforge.net  Wed Apr 11 15:34:53 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Wed, 11 Apr 2007 06:34:53 -0700
Subject: [ python-Bugs-1698398 ] wrong % of params for format string in
	ZipFile.printdir()
Message-ID: <E1HbcyL-0006D5-94@sc8-sf-web23.sourceforge.net>

Bugs item #1698398, was opened at 2007-04-11 07:58
Message generated for change (Comment added) made by alanmcintyre
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1698398&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Szabolcs Berecz (szabihernyo)
Assigned to: Nobody/Anonymous (nobody)
Summary: wrong % of params for format string in ZipFile.printdir()

Initial Comment:
In zipfile.py:448
date = "%d-%02d-%02d %02d:%02d:%02d" % zinfo.date_time
should be changed to
date = "%d-%02d-%02d %02d:%02d:%02d" % zinfo.date_time[:6]

checked with python2.[45] on windows and linux


>>> from zipfile import ZipFile
>>> from StringIO import StringIO
>>> s = StringIO()
>>> zf = ZipFile(s, 'w')
>>> zf.writestr('file.ext', '123')
>>> zf.printdir()
File Name                                             Modified             Size
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.5/zipfile.py", line 448, in printdir
    date = "%d-%02d-%02d %02d:%02d:%02d" % zinfo.date_time
TypeError: int argument required
>>> zf.filelist[0].date_time
(2007, 4, 11, 13, 38, 58, 2, 101, 1)
>>>


----------------------------------------------------------------------

Comment By: Alan McIntyre (alanmcintyre)
Date: 2007-04-11 08:34

Message:
Logged In: YES 
user_id=1115903
Originator: NO

The same problem appears to be present in 2.6 as well.  The date_time
attribute of ZipInfo is a time.struct_time in 2.4 and 2.6 (I don't have 2.5
available to check at the moment), not a tuple.  I'm assuming this could be
fixed by changing that line to:

date = "%d-%02d-%02d %02d:%02d:%02d" % tuple(zinfo.date_time)[:6]

At the moment I can't connect to the svn server; when I can I'll submit a
patch for the trunk and 2.5.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1698398&group_id=5470

From noreply at sourceforge.net  Wed Apr 11 15:52:17 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Wed, 11 Apr 2007 06:52:17 -0700
Subject: [ python-Bugs-1697782 ] types.InstanceType is missing but used by
	pydoc
Message-ID: <E1HbdFB-0003dE-Iw@sc8-sf-web22.sourceforge.net>

Bugs item #1697782, was opened at 2007-04-10 17:55
Message generated for change (Comment added) made by tiran
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1697782&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 3000
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Christian Heimes (tiran)
Assigned to: Nobody/Anonymous (nobody)
Summary: types.InstanceType is missing but used by pydoc

Initial Comment:
>>> help(callable.__call__)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/heimes/dev/python/p3yk/Lib/site.py", line 348, in __call__
    return pydoc.help(*args, **kwds)
  File "/home/heimes/dev/python/p3yk/Lib/pydoc.py", line 1658, in __call__
    self.help(request)
  File "/home/heimes/dev/python/p3yk/Lib/pydoc.py", line 1702, in help
    else: doc(request, 'Help on %s:')
  File "/home/heimes/dev/python/p3yk/Lib/pydoc.py", line 1477, in doc
    desc = describe(object)
  File "/home/heimes/dev/python/p3yk/Lib/pydoc.py", line 1436, in describe
    if type(thing) is types.InstanceType:
AttributeError: 'module' object has no attribute 'InstanceType'


----------------------------------------------------------------------

>Comment By: Christian Heimes (tiran)
Date: 2007-04-11 15:52

Message:
Logged In: YES 
user_id=560817
Originator: YES

types.InstanceType is still used by several stdlib modules.

FIX: svn cp the types.py module from Python 2.6 to the p3yk branh

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1697782&group_id=5470

From noreply at sourceforge.net  Wed Apr 11 17:59:01 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Wed, 11 Apr 2007 08:59:01 -0700
Subject: [ python-Bugs-1698398 ] wrong % of params for format string in
	ZipFile.printdir()
Message-ID: <E1HbfDp-0001tP-Cx@sc8-sf-web22.sourceforge.net>

Bugs item #1698398, was opened at 2007-04-11 07:58
Message generated for change (Settings changed) made by rhettinger
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1698398&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
>Group: Python 2.5
Status: Open
Resolution: None
>Priority: 6
Private: No
Submitted By: Szabolcs Berecz (szabihernyo)
>Assigned to: Raymond Hettinger (rhettinger)
Summary: wrong % of params for format string in ZipFile.printdir()

Initial Comment:
In zipfile.py:448
date = "%d-%02d-%02d %02d:%02d:%02d" % zinfo.date_time
should be changed to
date = "%d-%02d-%02d %02d:%02d:%02d" % zinfo.date_time[:6]

checked with python2.[45] on windows and linux


>>> from zipfile import ZipFile
>>> from StringIO import StringIO
>>> s = StringIO()
>>> zf = ZipFile(s, 'w')
>>> zf.writestr('file.ext', '123')
>>> zf.printdir()
File Name                                             Modified             Size
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.5/zipfile.py", line 448, in printdir
    date = "%d-%02d-%02d %02d:%02d:%02d" % zinfo.date_time
TypeError: int argument required
>>> zf.filelist[0].date_time
(2007, 4, 11, 13, 38, 58, 2, 101, 1)
>>>


----------------------------------------------------------------------

Comment By: Alan McIntyre (alanmcintyre)
Date: 2007-04-11 08:34

Message:
Logged In: YES 
user_id=1115903
Originator: NO

The same problem appears to be present in 2.6 as well.  The date_time
attribute of ZipInfo is a time.struct_time in 2.4 and 2.6 (I don't have 2.5
available to check at the moment), not a tuple.  I'm assuming this could be
fixed by changing that line to:

date = "%d-%02d-%02d %02d:%02d:%02d" % tuple(zinfo.date_time)[:6]

At the moment I can't connect to the svn server; when I can I'll submit a
patch for the trunk and 2.5.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1698398&group_id=5470

From noreply at sourceforge.net  Wed Apr 11 17:59:01 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Wed, 11 Apr 2007 08:59:01 -0700
Subject: [ python-Bugs-1431091 ] CSV Sniffer fails to report mismatch of
	column counts
Message-ID: <E1HbfDp-0001tS-DT@sc8-sf-web22.sourceforge.net>

Bugs item #1431091, was opened at 2006-02-13 23:47
Message generated for change (Comment added) made by tekkaman
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1431091&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Extension Modules
Group: Python 2.4
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Vinko (vvrsalob)
Assigned to: Andrew McNamara (andrewmcnamara)
Summary: CSV Sniffer fails to report mismatch of column counts

Initial Comment:
If one line of a CSV file is missing one or more
commas, the delimiter detection code of the Sniffer
class fails, setting delimiter to an empty string.

This leads to a totally misleading error when using
has_header(). 

This code shows the problem (Python 2.4.2, FC3 and
Ubuntu Breezy):

import csv

str1 = "a,b,c,d\r\n1,2,foo bar,dead
beef\r\nthis,line,is,ok\r\n"
str2 = "a,b,c,d\r\n1,2,foo bar,dead beef\r\nthis,line
is,not\r\n"

s = csv.Sniffer()

d1 = s.sniff(str1)
d2 = s.sniff(str2)

for line in str1.split('\r\n'):
    print line.count(',')

print d1.delimiter
print s.has_header(str1)

for line in str2.split('\r\n'):
    print line.count(',')

print d2.delimiter
print s.has_header(str2)


----------------------------------------------------------------------

Comment By: Simone Leo (tekkaman)
Date: 2007-04-11 17:59

Message:
Logged In: YES 
user_id=891539
Originator: NO

Problem is still there as of Python 2.4.3.

Trying to read in a file whose lines have a different number of fields, I
get:

Traceback (most recent call last):
  File "../myscript.py", line 59, in ?
    main()
  File "../myscript.py", line 30, in main
    reader = csv.reader(fin, dialect)
TypeError: bad argument type for built-in operation

where "dialect" has been sniffed by feeding the first two lines to the
Sniffer.

What I expect is to either:
  1. get different sized rows, with no exception raised
  2. get a csv.Error instead of the TypeError above

Thanks

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1431091&group_id=5470

From noreply at sourceforge.net  Wed Apr 11 17:59:44 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Wed, 11 Apr 2007 08:59:44 -0700
Subject: [ python-Bugs-1697916 ] Segfaults on memory error
Message-ID: <E1HbfEW-000206-3c@sc8-sf-web22.sourceforge.net>

Bugs item #1697916, was opened at 2007-04-10 14:47
Message generated for change (Settings changed) made by rhettinger
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1697916&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Interpreter Core
Group: Python 2.5
Status: Open
Resolution: None
>Priority: 6
Private: No
Submitted By: STINNER Victor (haypo)
>Assigned to: Raymond Hettinger (rhettinger)
Summary: Segfaults on memory error

Initial Comment:
Hi, I'm playing with resource.setrlimit(resource.RLIMIT_AS) to limit memory usage during fuzzing tests. It works quite well but Python crashs sometimes (with SEGFAULT).

I downloaded Python source code and recompiled it with EXTRA_FLAGS="-g -O0" to find errors. I found three bugs and wrote a patch for all of them.

Comments:
* Objects/exceptions.c:33: allocation may returns NULL on memory error
* Objects/longobject.c:2511: long_divrem() may allocate new long integers but l_divmod() doesn't check that div and mod are not NULL
* Objects/object.c:1284: problem with NULL mro. I don't understand how mro works, but I think that the error may be catched when mro is assigned. Problem: where is it done? in Objects/typeobject.c?

So don't apply my patch directly: fix for object.c may be wrong.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1697916&group_id=5470

From noreply at sourceforge.net  Wed Apr 11 18:00:14 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Wed, 11 Apr 2007 09:00:14 -0700
Subject: [ python-Bugs-1698167 ] xml.etree document element.tag
Message-ID: <E1HbfF0-00026B-2C@sc8-sf-web22.sourceforge.net>

Bugs item #1698167, was opened at 2007-04-11 01:25
Message generated for change (Settings changed) made by rhettinger
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1698167&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Documentation
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: paul rubin (phr)
>Assigned to: Fredrik Lundh (effbot)
Summary: xml.etree document element.tag

Initial Comment:
The xml.etree docs vaguely mention an implementation-dependent Element interface without describing it in any detail.  I could not figure out from the docs how to get the tag name of an element returned from (say) the getiterator interface.  That is, for an element like <foo/>, I wanted the string "foo".  Examining the library source showed that e.tag does the job, at least some of the time, and that was enough to get my app working.  Could the actual situation please be documented--thanks.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1698167&group_id=5470

From noreply at sourceforge.net  Wed Apr 11 18:07:37 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Wed, 11 Apr 2007 09:07:37 -0700
Subject: [ python-Bugs-1697943 ] msgfmt cannot cope with BOM
Message-ID: <E1HbfM9-00049t-MW@sc8-sf-web24.sourceforge.net>

Bugs item #1697943, was opened at 2007-04-10 15:58
Message generated for change (Comment added) made by rhettinger
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1697943&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Demos and Tools
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Christoph Zwerschke (cito)
>Assigned to: Martin v. L?wis (loewis)
Summary: msgfmt cannot cope with BOM

Initial Comment:
If a .po file has a BOM (byte order mark) at the beginning, as is often the case for utf-8 files created on Windows, msgfmt.py complines about a syntax error.

The attached patch fixes this problem.

----------------------------------------------------------------------

>Comment By: Raymond Hettinger (rhettinger)
Date: 2007-04-11 11:07

Message:
Logged In: YES 
user_id=80475
Originator: NO

Martin, is this your code?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1697943&group_id=5470

From noreply at sourceforge.net  Wed Apr 11 18:11:51 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Wed, 11 Apr 2007 09:11:51 -0700
Subject: [ python-Bugs-1697916 ] Segfaults on memory error
Message-ID: <E1HbfQF-0003Km-S8@sc8-sf-web22.sourceforge.net>

Bugs item #1697916, was opened at 2007-04-10 19:47
Message generated for change (Comment added) made by gbrandl
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1697916&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Interpreter Core
Group: Python 2.5
Status: Open
Resolution: None
Priority: 6
Private: No
Submitted By: STINNER Victor (haypo)
Assigned to: Raymond Hettinger (rhettinger)
Summary: Segfaults on memory error

Initial Comment:
Hi, I'm playing with resource.setrlimit(resource.RLIMIT_AS) to limit memory usage during fuzzing tests. It works quite well but Python crashs sometimes (with SEGFAULT).

I downloaded Python source code and recompiled it with EXTRA_FLAGS="-g -O0" to find errors. I found three bugs and wrote a patch for all of them.

Comments:
* Objects/exceptions.c:33: allocation may returns NULL on memory error
* Objects/longobject.c:2511: long_divrem() may allocate new long integers but l_divmod() doesn't check that div and mod are not NULL
* Objects/object.c:1284: problem with NULL mro. I don't understand how mro works, but I think that the error may be catched when mro is assigned. Problem: where is it done? in Objects/typeobject.c?

So don't apply my patch directly: fix for object.c may be wrong.

----------------------------------------------------------------------

>Comment By: Georg Brandl (gbrandl)
Date: 2007-04-11 16:11

Message:
Logged In: YES 
user_id=849994
Originator: NO

Status update: Fixed the first two bugs locally, and a third one
discovered by Victor on #python-dev.

The mro one is unclear -- we've asked him to reproduce it and poke around
in gdb to see why tp_mro is NULL, which it shouldn't be.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1697916&group_id=5470

From noreply at sourceforge.net  Wed Apr 11 18:31:12 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Wed, 11 Apr 2007 09:31:12 -0700
Subject: [ python-Bugs-1672568 ] silent error in
	email.message.Message.get_payload
Message-ID: <E1Hbfiy-0006nJ-A5@sc8-sf-web24.sourceforge.net>

Bugs item #1672568, was opened at 2007-03-02 12:04
Message generated for change (Settings changed) made by rhettinger
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1672568&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Not a Bug
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Renaud Blanch (blanch)
>Assigned to: Barry A. Warsaw (bwarsaw)
Summary: silent error in email.message.Message.get_payload

Initial Comment:
I rencently had trouble finding a bug in an email processing script because of an error that pass silently in the email module.
The get_payload() method in the email.message module always return something when the decode argument is set to True.
This behaviour is well documented, but is their a reason to catch the decoding errors ?
Why not let them pop-up to the calling code ?

renaud


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1672568&group_id=5470

From noreply at sourceforge.net  Wed Apr 11 18:52:52 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Wed, 11 Apr 2007 09:52:52 -0700
Subject: [ python-Bugs-1583862 ] yield+break stops tracing
Message-ID: <E1Hbg3w-0007mV-PR@sc8-sf-web22.sourceforge.net>

Bugs item #1583862, was opened at 2006-10-24 10:55
Message generated for change (Comment added) made by rhettinger
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1583862&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Interpreter Core
Group: Python 2.5
Status: Open
>Resolution: Wont Fix
Priority: 5
Private: No
Submitted By: Lukas Lalinsky (luks)
Assigned to: Nobody/Anonymous (nobody)
Summary: yield+break stops tracing

Initial Comment:
Here is an example script:

def myiter():
    for i in range(10):
        yield i
for i in myiter():
    break
print "foo"

Now, if I try to trace it:

$ python -m trace --trace --count test.py
 --- modulename: threading, funcname: settrace
threading.py(70):     _trace_hook = func
 --- modulename: test, funcname: <module>
test.py(1): def myiter():
test.py(5): for i in myiter():
 --- modulename: test, funcname: myiter
test.py(2):     for i in range(10):
test.py(3):         yield i
test.py(6):     break
c:\python25\lib\ntpath.py:190: RuntimeWarning:
tp_compare didn't return -1 or -2 for exception
  if i<=max(p.rfind('/'), p.rfind('\\')):
foo

It stops tracing after the `break` statement. The line
after, `print "foo"`, is not traced nor included in the
coverage output.

I'm not sure RuntimeWarning from ntpath.py is relevant
here, because if I use the trace module in some other
situation it doesn't print it. IMO, it's just a side
effect of some different problem.


----------------------------------------------------------------------

>Comment By: Raymond Hettinger (rhettinger)
Date: 2007-04-11 11:52

Message:
Logged In: YES 
user_id=80475
Originator: NO

Using Py2.5.1, the problem seems to have gone away.
Lukas, can you please verify that it is solved and then close.


=======================================================================
rhettinger at localhost ~ $ py25/python -m trace --trace --count test.py
 --- modulename: threading, funcname: settrace
threading.py(70):     _trace_hook = func
 --- modulename: trace, funcname: <module>
<string>(1):   --- modulename: trace, funcname: <module>
test.py(1): def myiter():
test.py(6): for i in myiter():
 --- modulename: trace, funcname: myiter
test.py(2):     for i in range(10):
test.py(3):         yield i
test.py(7):     break
 --- modulename: trace, funcname: myiter
test.py(9): print 'foo'
foo


----------------------------------------------------------------------

Comment By: Lukas Lalinsky (luks)
Date: 2006-10-24 11:05

Message:
Logged In: YES 
user_id=587716

Oh, I forgot. This bug is specific to Python 2.5. It works
fine in 2.4.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1583862&group_id=5470

From noreply at sourceforge.net  Wed Apr 11 18:53:32 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Wed, 11 Apr 2007 09:53:32 -0700
Subject: [ python-Bugs-1678710 ] dead link in tkinter documentation
Message-ID: <E1Hbg4a-0007so-EB@sc8-sf-web22.sourceforge.net>

Bugs item #1678710, was opened at 2007-03-11 22:34
Message generated for change (Settings changed) made by rhettinger
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1678710&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Documentation
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Ben Collver (bencollver)
>Assigned to: Fred L. Drake, Jr. (fdrake)
Summary: dead link in tkinter documentation

Initial Comment:
On the following page, there is a dead link.
http://www.python.org/doc/2.3.5/lib/node642.html

The dead link is to the following URL.
http://www.python.org/doc/2.3.5/lib/classes/ClassPacker.html

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1678710&group_id=5470

From noreply at sourceforge.net  Wed Apr 11 18:53:32 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Wed, 11 Apr 2007 09:53:32 -0700
Subject: [ python-Bugs-1697916 ] Segfaults on memory error
Message-ID: <E1Hbg4a-0001HW-UG@sc8-sf-web23.sourceforge.net>

Bugs item #1697916, was opened at 2007-04-10 21:47
Message generated for change (Comment added) made by haypo
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1697916&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Interpreter Core
Group: Python 2.5
>Status: Closed
Resolution: None
Priority: 6
Private: No
Submitted By: STINNER Victor (haypo)
Assigned to: Raymond Hettinger (rhettinger)
Summary: Segfaults on memory error

Initial Comment:
Hi, I'm playing with resource.setrlimit(resource.RLIMIT_AS) to limit memory usage during fuzzing tests. It works quite well but Python crashs sometimes (with SEGFAULT).

I downloaded Python source code and recompiled it with EXTRA_FLAGS="-g -O0" to find errors. I found three bugs and wrote a patch for all of them.

Comments:
* Objects/exceptions.c:33: allocation may returns NULL on memory error
* Objects/longobject.c:2511: long_divrem() may allocate new long integers but l_divmod() doesn't check that div and mod are not NULL
* Objects/object.c:1284: problem with NULL mro. I don't understand how mro works, but I think that the error may be catched when mro is assigned. Problem: where is it done? in Objects/typeobject.c?

So don't apply my patch directly: fix for object.c may be wrong.

----------------------------------------------------------------------

>Comment By: STINNER Victor (haypo)
Date: 2007-04-11 18:53

Message:
Logged In: YES 
user_id=365388
Originator: YES

Ignore my bug about NULL mro since i'm not able to reproduce it. Thanks
for the 3 other fixes ;-)

----------------------------------------------------------------------

Comment By: Georg Brandl (gbrandl)
Date: 2007-04-11 18:11

Message:
Logged In: YES 
user_id=849994
Originator: NO

Status update: Fixed the first two bugs locally, and a third one
discovered by Victor on #python-dev.

The mro one is unclear -- we've asked him to reproduce it and poke around
in gdb to see why tp_mro is NULL, which it shouldn't be.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1697916&group_id=5470

From noreply at sourceforge.net  Wed Apr 11 18:55:54 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Wed, 11 Apr 2007 09:55:54 -0700
Subject: [ python-Bugs-1633648 ] incomplete numerical comparisons
Message-ID: <E1Hbg6s-00087D-Kw@sc8-sf-web22.sourceforge.net>

Bugs item #1633648, was opened at 2007-01-11 19:18
Message generated for change (Comment added) made by rhettinger
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1633648&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Interpreter Core
Group: Python 2.5
Status: Open
Resolution: None
>Priority: 6
Private: No
Submitted By: Matthias Klose (doko)
Assigned to: Nobody/Anonymous (nobody)
Summary: incomplete numerical comparisons

Initial Comment:
[forwarded from http://bugs.debian.org/334022]

bug submitter writes:

I've been tracking down the regression failure in python-pgsql under
python2.[45], and here's what it comes down to.

Python-pgsql includes a short int type named PgInt2, which allows itself
to be coerced into all of the usual numeric types.

The regression that fails is when a PgInt2 is compared with a float.  In
this case python determines that the comparison is not implemented.

The problem is this:

- Under python2.[45], the float type includes tp_richcompare but not
  tp_compare.

- When calling try_rich_to_3way_compare(), python does not try any kind
  of numeric coercion, and so the comparison fails.

- When calling try_3way_compare(), python successfully coerces the
  PgInt2 into a float, but then the comparison fails because the float
  type has no tp_compare routine.

Presumably what would fix things would be one of the following:

- Bring back the trivial float_compare() routine, which was removed with
  python2.[45] when they brought in the new float_richcompare() instead;

- In try_3way_compare(), if the coercion succeeds and neither object
  has a tp_compare routine, try tp_richcompare before failing completely.

Does either of these solutions seem feasible?

Thanks - Ben.


----------------------------------------------------------------------

>Comment By: Raymond Hettinger (rhettinger)
Date: 2007-04-11 11:55

Message:
Logged In: YES 
user_id=80475
Originator: NO

Raising the priority so this report doesn't get lost.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1633648&group_id=5470

From noreply at sourceforge.net  Wed Apr 11 19:09:53 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Wed, 11 Apr 2007 10:09:53 -0700
Subject: [ python-Bugs-1613130 ] str.split creates new string even if pattern
	not found
Message-ID: <E1HbgKP-00034t-Iq@sc8-sf-web23.sourceforge.net>

Bugs item #1613130, was opened at 2006-12-11 08:03
Message generated for change (Comment added) made by rhettinger
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1613130&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Performance
Group: Python 2.5
Status: Open
Resolution: None
>Priority: 1
Private: No
Submitted By: Antoine Pitrou (pitrou)
Assigned to: Fredrik Lundh (effbot)
Summary: str.split creates new string even if pattern not found

Initial Comment:
Hello,

Several string methods avoid allocating a new string when the operation result is trivially the same as one of the parameters (e.g. replacing a non-existing substring). However, split() does not exhibit this optimization, it always constructs a new string even if no splitting occurs:

$ python
Python 2.5 (r25:51908, Oct  6 2006, 15:22:41) 
[GCC 4.1.2 20060928 (prerelease) (Ubuntu 4.1.1-13ubuntu4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> s = "abcde" * 2
>>> id(s)
3084139400L
>>> id(str(s))
3084139400L
>>> id("" + s)
3084139400L
>>> id(s.strip())
3084139400L
>>> id(s.replace("g", "h"))
3084139400L
>>> [id(x) for x in s.partition("h")]
[3084139400L, 3084271768L, 3084271768L]
>>> [id(x) for x in s.split("h")]
[3084139360L]


----------------------------------------------------------------------

>Comment By: Raymond Hettinger (rhettinger)
Date: 2007-04-11 12:09

Message:
Logged In: YES 
user_id=80475
Originator: NO

Dropping the priority.  This pay-off is near zero and likely not worth the
cost of making the code more complex than it already is.

----------------------------------------------------------------------

Comment By: Georg Brandl (gbrandl)
Date: 2006-12-12 11:21

Message:
Logged In: YES 
user_id=849994
Originator: NO

Sounds like this is best assigned to Fredrik.

----------------------------------------------------------------------

Comment By: Antoine Pitrou (pitrou)
Date: 2006-12-12 06:35

Message:
Logged In: YES 
user_id=133955
Originator: YES

Ok, I did a patch which partially adds the optimization (the patch is at
home, I can't post it right now). I have a few questions though:
 - there is a USE_FAST flag which can bring some speedups when a
multicharacter separator is used; however, it is not enabled by default, is
there a reason for this?
 - where and by whom is maintained stringbench.py, so that I can propose
additional tests for it (namely, tests for unmatched split())?
 - split() implementation is duplicated between str and unicode (the
unicode versions having less optimizations), would it be useful to
"stringlib'ify" split()?
 - rsplit() does quite similar things as split(), has anyone tried to
factor similar parts? do you see any caveats doing so?


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1613130&group_id=5470

From noreply at sourceforge.net  Wed Apr 11 19:12:01 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Wed, 11 Apr 2007 10:12:01 -0700
Subject: [ python-Bugs-1621111 ] IDLE crashes on OS X 10.4 when "Preferences"
	selected
Message-ID: <E1HbgMT-0003J5-4D@sc8-sf-web23.sourceforge.net>

Bugs item #1621111, was opened at 2006-12-22 23:03
Message generated for change (Comment added) made by rhettinger
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1621111&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Macintosh
Group: Python 2.5
Status: Open
Resolution: None
>Priority: 6
Private: No
Submitted By: Mick L (mehum)
Assigned to: Ronald Oussoren (ronaldoussoren)
Summary: IDLE crashes on OS X 10.4 when "Preferences" selected

Initial Comment:
When I select 'preferences' (on IDLE 1.2, Python 2.5)  or 'Configure IDLE' (IDLE 1.1.4, Python 2.4.4 and IDLE 1.0, Python 2.3.3), Python crashes as soon as the dialog box appears.

Terminal reports a "Segmentation fault" upon the crash.

My system is an iMac G5 with 1 GB ram running OS X v 10.4.8, Tcl/Tk 8.4.10, and seems otherwise stable.  

IDLE also appears stable until I select this option.

The Python crash log is attached.

Please, can anyone suggest what may be causing this problem?  Thank you.



----------------------------------------------------------------------

>Comment By: Raymond Hettinger (rhettinger)
Date: 2007-04-11 12:12

Message:
Logged In: YES 
user_id=80475
Originator: NO

Bumping up the priority so this doesn't get lost.  My guess is that IDLE
is tickling some problem with the TkInter bindings.

----------------------------------------------------------------------

Comment By: Fran?ois Scholder (f_scholder)
Date: 2007-02-02 16:55

Message:
Logged In: YES 
user_id=1609867
Originator: NO

Erratum:

A similar problem arises with WinXP when trying to configure IDLE via the
"Options>Configure IDLE..." menu.

Clicking on the 'Ok' or 'Apply' buttons of the dialog seemingly has no
effect. However when closing the window by clicking on either the window
close button (top right 'X') or the 'Cancel' button, several bahaviors
are
observed depending on how many times one has clicked on the 'Ok' and/or
---->'Apply'<---- buttons before:
- 0 clicks => expected behavior (dialog is closed, IDLE session is
resumed
and working fine)
- 1 or 2 clicks => dialog is closed but the IDLE shell behaves as a text
editor instead of an interpreter
- 3 or more clicks => IDLE crashes...

Note: I'm running Python 2.5 (r25:51908, Sep 19 2006, 09:52:17) [MSC
v.1310 32 bit (Intel)] on win32 -> WinXP SP2

----------------------------------------------------------------------

Comment By: Fran?ois Scholder (f_scholder)
Date: 2007-02-02 16:48

Message:
Logged In: YES 
user_id=1609867
Originator: NO

A similar problem arises with WinXP when trying to configure IDLE via the
"Options>Configure IDLE..." menu.

Clicking on the 'Ok' or 'Apply' buttons of the dialog seemingly has no
effect. However when closing the window by clicking on either the window
close button (top right 'X') or the 'Cancel' button, several bahaviors are
observed depending on how many times one has clicked on the 'Ok' and/or
'Cancel' buttons before:
- 0 clicks => expected behavior (dialog is closed, IDLE session is resumed
and working fine)
- 1 or 2 clicks => dialog is closed but the IDLE shell behaves as a text
editor instead of an interpreter
- 3 or more clicks => IDLE crashes...

Note: I'm running Python 2.5 (r25:51908, Sep 19 2006, 09:52:17) [MSC
v.1310 32 bit (Intel)] on win32 -> WinXP SP2

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1621111&group_id=5470

From noreply at sourceforge.net  Wed Apr 11 19:19:46 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Wed, 11 Apr 2007 10:19:46 -0700
Subject: [ python-Bugs-1607041 ] Condition.wait timeout fails when system
	clock changes
Message-ID: <E1HbgTy-0002Jd-Dr@sc8-sf-web22.sourceforge.net>

Bugs item #1607041, was opened at 2006-12-01 13:45
Message generated for change (Comment added) made by rhettinger
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1607041&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.4
>Status: Closed
>Resolution: Wont Fix
Priority: 5
Private: No
Submitted By: BC (hashstat)
Assigned to: Nobody/Anonymous (nobody)
Summary: Condition.wait timeout fails when system clock changes

Initial Comment:
If the system clock is adjusted after Condition.wait is called with a timeout, the timeout does not expire as expected.  This appears to be due to the threading.Condition class using the system clock to calculate the timeout expiration without taking system clock changes into account.  Let me illustrate this with an example:


   import threading

   c = threading.Condition()
   c.acquire()
   try:
      ...
      c.wait(3600.0)
      ...
   finally:
      c.release()


Let's say the above snippet is executed at 08:00.  Assuming the condition is never notified, the timeout should expire somewhere close to 09:00.  At 08:30 someone notices that the clock is an hour fast and sets it back to 07:30.  The timeout still expire around 09:00 causing a wait of 2 hours instead of the requested 1 hour.

Now let's say instead that the clock was moved ahead to 09:30 at 08:30.  The timeout would expire immediately after only a 30 minute wait.

----------------------------------------------------------------------

>Comment By: Raymond Hettinger (rhettinger)
Date: 2007-04-11 12:19

Message:
Logged In: YES 
user_id=80475
Originator: NO

I don't think this is the right thing to do.  Detecting clock changes is
tricky (all you can do is detect backward movement).  Then, deciding what
to do about it also involves a certain amount of guesswork (at how much the
time changed and what the user really wants to occur).  Also, changing
clocks may affect many parts of the system (log entries being out of order,
etc) and it is not entirely clear what the interactions should be with the
Condition timeout.  IOW, when the clock changes, you've likely got bigger
problems.  Refuse the temptation to guess.


----------------------------------------------------------------------

Comment By: Jp Calderone (kuran)
Date: 2006-12-01 17:50

Message:
Logged In: YES 
user_id=366566
Originator: NO

Yes, I misunderstood.  Please disregard my comments. :)


----------------------------------------------------------------------

Comment By: BC (hashstat)
Date: 2006-12-01 17:35

Message:
Logged In: YES 
user_id=939959
Originator: YES

I submitted patch# 1607149 to add checking for clock variations in the
wait method when called with a timeout.

----------------------------------------------------------------------

Comment By: BC (hashstat)
Date: 2006-12-01 16:57

Message:
Logged In: YES 
user_id=939959
Originator: YES

Apparently my description wasn't as clear as I thought.  I don't care
about the clock times in the example.  They were only used to illustrate
the point.  What I do care about is that if a use c.wait(3600) ("wait 60
minutes"), then my thread should wake up in roughly 60 minutes without
regard to changes in the system clock.

With the current Condition implementation, no matter what timeout is used,
setting the system clock ahead reduces or eliminates the wait while setting
the system clock back increases the wait.  So if the clock is set back one
hour in the middle of a 1 microsecond wait (c.wait(1)), wait will return in
an hour and 1 microsecond.  Is this the way it should work?

----------------------------------------------------------------------

Comment By: Jp Calderone (kuran)
Date: 2006-12-01 16:06

Message:
Logged In: YES 
user_id=366566
Originator: NO

Why should it work that way?

c.wait(3600) doesn't say "wait until 0900", it says "wait 60 minutes".

If you want c.waitUntil(9 oclock), maybe that would be a good API
addition, but it definitely should be a separate method.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1607041&group_id=5470

From noreply at sourceforge.net  Wed Apr 11 20:41:42 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Wed, 11 Apr 2007 11:41:42 -0700
Subject: [ python-Feature Requests-1191699 ] make slices pickable
Message-ID: <E1HbhlG-0006fJ-Ow@sc8-sf-web21.sourceforge.net>

Feature Requests item #1191699, was opened at 2005-04-28 08:44
Message generated for change (Comment added) made by rhettinger
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1191699&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Interpreter Core
>Group: Python 2.6
>Status: Closed
>Resolution: Accepted
Priority: 5
Private: No
Submitted By: Sebastien de Menten (sdementen)
Assigned to: Raymond Hettinger (rhettinger)
Summary: make slices pickable

Initial Comment:
As suggested  by the summary, provide pickability of 
slices by default.
Currently one has to use copy_reg module to register 
slices as pickables.

----------------------------------------------------------------------

>Comment By: Raymond Hettinger (rhettinger)
Date: 2007-04-11 13:41

Message:
Logged In: YES 
user_id=80475
Originator: NO

Done.  See revision 54760.

----------------------------------------------------------------------

Comment By: Connelly (connelly)
Date: 2006-07-16 13:55

Message:
Logged In: YES 
user_id=1039782

Additional use case: I needed picklable slices when logging method calls
to a 
database:  __setitem__(self, i, x) could not be logged because the
database 
module attempted to pickle objects such as ('__setitem__', slice(1, 2, 3),
[]).

For other Python users who have run into this problem: the code needed to

make slices picklable by copy_reg is available at:

  http://mail.python.org/pipermail/python-list/2004-November/
248988.html


----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2005-06-02 19:50

Message:
Logged In: YES 
user_id=80475

Okay, sounds reasonable.  Will implement when I have time.

----------------------------------------------------------------------

Comment By: Sebastien de Menten (sdementen)
Date: 2005-04-30 13:02

Message:
Logged In: YES 
user_id=820079

Use case for pickable slices. 
Let A be a numeric array of size M x N. We want to consider 
sub-arrays in this array like A[:4, :] (the first 4 lines of the 
array). 
If we want to store both the array and the information of 
sub-arrays structures, we need to store slices (we can also 
store start/end indices of the array ... but this is call a slice 
so it would be better to have pickable slices). 
 
In fact, whenever one wants to write generic algorithm 
working on "sublist of lists" or "subarrays of arrays" or 
"sub-items of collection of items", it is nicer to use slice 
objects explicitly and so store them also explicitly. 
 

----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2005-04-30 07:32

Message:
Logged In: YES 
user_id=80475

Use cases?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1191699&group_id=5470

From noreply at sourceforge.net  Wed Apr 11 21:25:21 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Wed, 11 Apr 2007 12:25:21 -0700
Subject: [ python-Bugs-1697782 ] types.InstanceType is missing but used by
	pydoc
Message-ID: <E1HbiRV-00038R-GF@sc8-sf-web21.sourceforge.net>

Bugs item #1697782, was opened at 2007-04-10 15:55
Message generated for change (Comment added) made by gbrandl
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1697782&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 3000
>Status: Closed
>Resolution: Fixed
Priority: 5
Private: No
Submitted By: Christian Heimes (tiran)
Assigned to: Nobody/Anonymous (nobody)
Summary: types.InstanceType is missing but used by pydoc

Initial Comment:
>>> help(callable.__call__)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/heimes/dev/python/p3yk/Lib/site.py", line 348, in __call__
    return pydoc.help(*args, **kwds)
  File "/home/heimes/dev/python/p3yk/Lib/pydoc.py", line 1658, in __call__
    self.help(request)
  File "/home/heimes/dev/python/p3yk/Lib/pydoc.py", line 1702, in help
    else: doc(request, 'Help on %s:')
  File "/home/heimes/dev/python/p3yk/Lib/pydoc.py", line 1477, in doc
    desc = describe(object)
  File "/home/heimes/dev/python/p3yk/Lib/pydoc.py", line 1436, in describe
    if type(thing) is types.InstanceType:
AttributeError: 'module' object has no attribute 'InstanceType'


----------------------------------------------------------------------

>Comment By: Georg Brandl (gbrandl)
Date: 2007-04-11 19:25

Message:
Logged In: YES 
user_id=849994
Originator: NO

Fixed all instances of types.InstanceType in rev. 54761.

----------------------------------------------------------------------

Comment By: Christian Heimes (tiran)
Date: 2007-04-11 13:52

Message:
Logged In: YES 
user_id=560817
Originator: YES

types.InstanceType is still used by several stdlib modules.

FIX: svn cp the types.py module from Python 2.6 to the p3yk branh

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1697782&group_id=5470

From noreply at sourceforge.net  Wed Apr 11 22:11:14 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Wed, 11 Apr 2007 13:11:14 -0700
Subject: [ python-Bugs-1697916 ] Segfaults on memory error
Message-ID: <E1Hbj9u-0004GS-Hy@sc8-sf-web22.sourceforge.net>

Bugs item #1697916, was opened at 2007-04-10 19:47
Message generated for change (Comment added) made by gbrandl
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1697916&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Interpreter Core
Group: Python 2.5
>Status: Open
Resolution: None
Priority: 6
Private: No
Submitted By: STINNER Victor (haypo)
>Assigned to: Georg Brandl (gbrandl)
Summary: Segfaults on memory error

Initial Comment:
Hi, I'm playing with resource.setrlimit(resource.RLIMIT_AS) to limit memory usage during fuzzing tests. It works quite well but Python crashs sometimes (with SEGFAULT).

I downloaded Python source code and recompiled it with EXTRA_FLAGS="-g -O0" to find errors. I found three bugs and wrote a patch for all of them.

Comments:
* Objects/exceptions.c:33: allocation may returns NULL on memory error
* Objects/longobject.c:2511: long_divrem() may allocate new long integers but l_divmod() doesn't check that div and mod are not NULL
* Objects/object.c:1284: problem with NULL mro. I don't understand how mro works, but I think that the error may be catched when mro is assigned. Problem: where is it done? in Objects/typeobject.c?

So don't apply my patch directly: fix for object.c may be wrong.

----------------------------------------------------------------------

>Comment By: Georg Brandl (gbrandl)
Date: 2007-04-11 20:11

Message:
Logged In: YES 
user_id=849994
Originator: NO

Reopening until I can backport this to the 2.5 branch.

----------------------------------------------------------------------

Comment By: STINNER Victor (haypo)
Date: 2007-04-11 16:53

Message:
Logged In: YES 
user_id=365388
Originator: YES

Ignore my bug about NULL mro since i'm not able to reproduce it. Thanks
for the 3 other fixes ;-)

----------------------------------------------------------------------

Comment By: Georg Brandl (gbrandl)
Date: 2007-04-11 16:11

Message:
Logged In: YES 
user_id=849994
Originator: NO

Status update: Fixed the first two bugs locally, and a third one
discovered by Victor on #python-dev.

The mro one is unclear -- we've asked him to reproduce it and poke around
in gdb to see why tp_mro is NULL, which it shouldn't be.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1697916&group_id=5470

From noreply at sourceforge.net  Wed Apr 11 22:11:23 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Wed, 11 Apr 2007 13:11:23 -0700
Subject: [ python-Bugs-1697782 ] types.InstanceType is missing but used by
	pydoc
Message-ID: <E1HbjA3-0004Gf-5F@sc8-sf-web22.sourceforge.net>

Bugs item #1697782, was opened at 2007-04-10 15:55
Message generated for change (Comment added) made by gbrandl
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1697782&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 3000
>Status: Open
Resolution: Fixed
Priority: 5
Private: No
Submitted By: Christian Heimes (tiran)
>Assigned to: Georg Brandl (gbrandl)
Summary: types.InstanceType is missing but used by pydoc

Initial Comment:
>>> help(callable.__call__)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/heimes/dev/python/p3yk/Lib/site.py", line 348, in __call__
    return pydoc.help(*args, **kwds)
  File "/home/heimes/dev/python/p3yk/Lib/pydoc.py", line 1658, in __call__
    self.help(request)
  File "/home/heimes/dev/python/p3yk/Lib/pydoc.py", line 1702, in help
    else: doc(request, 'Help on %s:')
  File "/home/heimes/dev/python/p3yk/Lib/pydoc.py", line 1477, in doc
    desc = describe(object)
  File "/home/heimes/dev/python/p3yk/Lib/pydoc.py", line 1436, in describe
    if type(thing) is types.InstanceType:
AttributeError: 'module' object has no attribute 'InstanceType'


----------------------------------------------------------------------

>Comment By: Georg Brandl (gbrandl)
Date: 2007-04-11 20:11

Message:
Logged In: YES 
user_id=849994
Originator: NO

Reopening until I can backport this to the 2.5 branch.

----------------------------------------------------------------------

Comment By: Georg Brandl (gbrandl)
Date: 2007-04-11 19:25

Message:
Logged In: YES 
user_id=849994
Originator: NO

Fixed all instances of types.InstanceType in rev. 54761.

----------------------------------------------------------------------

Comment By: Christian Heimes (tiran)
Date: 2007-04-11 13:52

Message:
Logged In: YES 
user_id=560817
Originator: YES

types.InstanceType is still used by several stdlib modules.

FIX: svn cp the types.py module from Python 2.6 to the p3yk branh

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1697782&group_id=5470

From noreply at sourceforge.net  Wed Apr 11 22:24:56 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Wed, 11 Apr 2007 13:24:56 -0700
Subject: [ python-Bugs-1621111 ] IDLE crashes on OS X 10.4 when "Preferences"
	selected
Message-ID: <E1HbjNA-0007Vu-Rn@sc8-sf-web23.sourceforge.net>

Bugs item #1621111, was opened at 2006-12-23 05:03
Message generated for change (Comment added) made by ronaldoussoren
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1621111&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Macintosh
Group: Python 2.5
Status: Open
Resolution: None
Priority: 6
Private: No
Submitted By: Mick L (mehum)
Assigned to: Ronald Oussoren (ronaldoussoren)
Summary: IDLE crashes on OS X 10.4 when "Preferences" selected

Initial Comment:
When I select 'preferences' (on IDLE 1.2, Python 2.5)  or 'Configure IDLE' (IDLE 1.1.4, Python 2.4.4 and IDLE 1.0, Python 2.3.3), Python crashes as soon as the dialog box appears.

Terminal reports a "Segmentation fault" upon the crash.

My system is an iMac G5 with 1 GB ram running OS X v 10.4.8, Tcl/Tk 8.4.10, and seems otherwise stable.  

IDLE also appears stable until I select this option.

The Python crash log is attached.

Please, can anyone suggest what may be causing this problem?  Thank you.



----------------------------------------------------------------------

>Comment By: Ronald Oussoren (ronaldoussoren)
Date: 2007-04-11 22:24

Message:
Logged In: YES 
user_id=580910
Originator: NO

It seems to be caused by Tcl/Tk, I cannot reproduce this with the system
version of Tk

This is on 10.4.8/intel, with python 2.5.1rc1. I don't think the change in
python version fixes this issue.

Mick: could you please test what happens when you temporarily move
/Library/Frameworks/Tcl.framework and /Library/Frameworks/Tk.framework to
another folder.

----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2007-04-11 19:12

Message:
Logged In: YES 
user_id=80475
Originator: NO

Bumping up the priority so this doesn't get lost.  My guess is that IDLE
is tickling some problem with the TkInter bindings.

----------------------------------------------------------------------

Comment By: Fran?ois Scholder (f_scholder)
Date: 2007-02-02 22:55

Message:
Logged In: YES 
user_id=1609867
Originator: NO

Erratum:

A similar problem arises with WinXP when trying to configure IDLE via the
"Options>Configure IDLE..." menu.

Clicking on the 'Ok' or 'Apply' buttons of the dialog seemingly has no
effect. However when closing the window by clicking on either the window
close button (top right 'X') or the 'Cancel' button, several bahaviors
are
observed depending on how many times one has clicked on the 'Ok' and/or
---->'Apply'<---- buttons before:
- 0 clicks => expected behavior (dialog is closed, IDLE session is
resumed
and working fine)
- 1 or 2 clicks => dialog is closed but the IDLE shell behaves as a text
editor instead of an interpreter
- 3 or more clicks => IDLE crashes...

Note: I'm running Python 2.5 (r25:51908, Sep 19 2006, 09:52:17) [MSC
v.1310 32 bit (Intel)] on win32 -> WinXP SP2

----------------------------------------------------------------------

Comment By: Fran?ois Scholder (f_scholder)
Date: 2007-02-02 22:48

Message:
Logged In: YES 
user_id=1609867
Originator: NO

A similar problem arises with WinXP when trying to configure IDLE via the
"Options>Configure IDLE..." menu.

Clicking on the 'Ok' or 'Apply' buttons of the dialog seemingly has no
effect. However when closing the window by clicking on either the window
close button (top right 'X') or the 'Cancel' button, several bahaviors are
observed depending on how many times one has clicked on the 'Ok' and/or
'Cancel' buttons before:
- 0 clicks => expected behavior (dialog is closed, IDLE session is resumed
and working fine)
- 1 or 2 clicks => dialog is closed but the IDLE shell behaves as a text
editor instead of an interpreter
- 3 or more clicks => IDLE crashes...

Note: I'm running Python 2.5 (r25:51908, Sep 19 2006, 09:52:17) [MSC
v.1310 32 bit (Intel)] on win32 -> WinXP SP2

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1621111&group_id=5470

From noreply at sourceforge.net  Thu Apr 12 00:13:22 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Wed, 11 Apr 2007 15:13:22 -0700
Subject: [ python-Bugs-1697943 ] msgfmt cannot cope with BOM
Message-ID: <E1Hbl46-0004q3-HW@sc8-sf-web21.sourceforge.net>

Bugs item #1697943, was opened at 2007-04-10 22:58
Message generated for change (Comment added) made by loewis
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1697943&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Demos and Tools
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Christoph Zwerschke (cito)
Assigned to: Martin v. L?wis (loewis)
Summary: msgfmt cannot cope with BOM

Initial Comment:
If a .po file has a BOM (byte order mark) at the beginning, as is often the case for utf-8 files created on Windows, msgfmt.py complines about a syntax error.

The attached patch fixes this problem.

----------------------------------------------------------------------

>Comment By: Martin v. L?wis (loewis)
Date: 2007-04-12 00:13

Message:
Logged In: YES 
user_id=21627
Originator: NO

It's my code, but I will need to establish first whether it's a bug. That
depends on what the PO specification says, and, if is it silent on the
matter, what GNU gettext does.

----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2007-04-11 18:07

Message:
Logged In: YES 
user_id=80475
Originator: NO

Martin, is this your code?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1697943&group_id=5470

From noreply at sourceforge.net  Thu Apr 12 05:28:04 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Wed, 11 Apr 2007 20:28:04 -0700
Subject: [ python-Bugs-1698398 ] wrong % of params for format string in
	ZipFile.printdir()
Message-ID: <E1Hbpye-0002l6-7A@sc8-sf-web23.sourceforge.net>

Bugs item #1698398, was opened at 2007-04-11 07:58
Message generated for change (Comment added) made by alanmcintyre
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1698398&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.5
Status: Open
Resolution: None
Priority: 6
Private: No
Submitted By: Szabolcs Berecz (szabihernyo)
Assigned to: Raymond Hettinger (rhettinger)
Summary: wrong % of params for format string in ZipFile.printdir()

Initial Comment:
In zipfile.py:448
date = "%d-%02d-%02d %02d:%02d:%02d" % zinfo.date_time
should be changed to
date = "%d-%02d-%02d %02d:%02d:%02d" % zinfo.date_time[:6]

checked with python2.[45] on windows and linux


>>> from zipfile import ZipFile
>>> from StringIO import StringIO
>>> s = StringIO()
>>> zf = ZipFile(s, 'w')
>>> zf.writestr('file.ext', '123')
>>> zf.printdir()
File Name                                             Modified             Size
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.5/zipfile.py", line 448, in printdir
    date = "%d-%02d-%02d %02d:%02d:%02d" % zinfo.date_time
TypeError: int argument required
>>> zf.filelist[0].date_time
(2007, 4, 11, 13, 38, 58, 2, 101, 1)
>>>


----------------------------------------------------------------------

Comment By: Alan McIntyre (alanmcintyre)
Date: 2007-04-11 22:28

Message:
Logged In: YES 
user_id=1115903
Originator: NO

Patches to fix this have been posted for 2.5 (#1698915) and 2.6 (#1698917)

----------------------------------------------------------------------

Comment By: Alan McIntyre (alanmcintyre)
Date: 2007-04-11 08:34

Message:
Logged In: YES 
user_id=1115903
Originator: NO

The same problem appears to be present in 2.6 as well.  The date_time
attribute of ZipInfo is a time.struct_time in 2.4 and 2.6 (I don't have 2.5
available to check at the moment), not a tuple.  I'm assuming this could be
fixed by changing that line to:

date = "%d-%02d-%02d %02d:%02d:%02d" % tuple(zinfo.date_time)[:6]

At the moment I can't connect to the svn server; when I can I'll submit a
patch for the trunk and 2.5.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1698398&group_id=5470

From noreply at sourceforge.net  Thu Apr 12 06:13:45 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Wed, 11 Apr 2007 21:13:45 -0700
Subject: [ python-Feature Requests-1193128 ] 'str'.translate(None) => identity
	translation
Message-ID: <E1Hbqgr-0003uO-32@sc8-sf-web22.sourceforge.net>

Feature Requests item #1193128, was opened at 2005-04-30 15:31
Message generated for change (Comment added) made by rhettinger
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1193128&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Interpreter Core
Group: None
>Status: Closed
>Resolution: Accepted
Priority: 5
Private: No
Submitted By: Bengt Richter (bokr)
Assigned to: Raymond Hettinger (rhettinger)
Summary: 'str'.translate(None) => identity translation

Initial Comment:
This feature would permit passing None as the first
argument to str.translate in place of  an identity
translation
table like ''.join([[chr(i) for i in xrange(256)])

This should be handy for deleting characters, e.g., as in

     s = s.translate(None, delchars)

----------------------------------------------------------------------

>Comment By: Raymond Hettinger (rhettinger)
Date: 2007-04-11 23:13

Message:
Logged In: YES 
user_id=80475
Originator: NO

Done.  See revision 54772.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1193128&group_id=5470

From noreply at sourceforge.net  Thu Apr 12 06:56:06 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Wed, 11 Apr 2007 21:56:06 -0700
Subject: [ python-Bugs-1678710 ] dead link in tkinter documentation
Message-ID: <E1HbrLq-0006BT-6L@sc8-sf-web24.sourceforge.net>

Bugs item #1678710, was opened at 2007-03-11 23:34
Message generated for change (Comment added) made by fdrake
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1678710&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Documentation
Group: Python 2.3
>Status: Closed
>Resolution: Wont Fix
Priority: 5
Private: No
Submitted By: Ben Collver (bencollver)
Assigned to: Fred L. Drake, Jr. (fdrake)
Summary: dead link in tkinter documentation

Initial Comment:
On the following page, there is a dead link.
http://www.python.org/doc/2.3.5/lib/node642.html

The dead link is to the following URL.
http://www.python.org/doc/2.3.5/lib/classes/ClassPacker.html

----------------------------------------------------------------------

>Comment By: Fred L. Drake, Jr. (fdrake)
Date: 2007-04-12 00:56

Message:
Logged In: YES 
user_id=3066
Originator: NO

This has been fixed in more recent versions of the documentation.

Version 2.3.x is no longer in maintenance; this won't be fixed for older
versions; sorry.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1678710&group_id=5470

From noreply at sourceforge.net  Thu Apr 12 07:44:17 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Wed, 11 Apr 2007 22:44:17 -0700
Subject: [ python-Bugs-1698944 ] dtdparser discards last line
Message-ID: <E1Hbs6T-00050y-9g@sc8-sf-web22.sourceforge.net>

Bugs item #1698944, was opened at 2007-04-11 22:44
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1698944&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: L. Peter Deutsch (lpd)
Assigned to: Nobody/Anonymous (nobody)
Summary: dtdparser discards last line

Initial Comment:
The DTD parser in xml.parsers.xmlproc.dtdparser sometimes simply ignores the last line of the input file. Here is the Python program I used to test it:

------begin
from xml.parsers.xmlproc.dtdparser import DTDParser,\
 DTDConsumer, DTDConsumerPE
import tracer

def parse_dtd(dtd):
    parser = DTDParser()
    handler = DTDConsumerPE()
    parser.set_dtd_consumer(handler)
    parser.feed(dtd)

tracer.trace(DTDConsumer)

parse_dtd('''\
<!ENTITY % score PUBLIC

	"-//Recordare//ELEMENTS MusicXML 1.1 Score//EN"

	"t.dtd">

%score;



''')
------end

where the file t.dtd contained:

------begin
<!ELEMENT opus EMPTY>
------end

The tracer package (my own code) indicated that the opus element was never parsed. The same was true if I added a blank line after the %score; entity reference. However, if I added *two* blank lines after the entity reference, the element declaration *was* processed.

I would note that while the bug exists in the original xmlproc package (at least in xmlproc 0.70, downloadable from http://www.garshol.priv.no/download/software/xmlproc/), the bug does *not* exist in the PyXML package (downloadable from http://sourceforge.net/projects/pyxml).


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1698944&group_id=5470

From noreply at sourceforge.net  Thu Apr 12 10:27:32 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Thu, 12 Apr 2007 01:27:32 -0700
Subject: [ python-Bugs-1621111 ] IDLE crashes on OS X 10.4 when "Preferences"
	selected
Message-ID: <E1HbueS-0001gG-Np@sc8-sf-web21.sourceforge.net>

Bugs item #1621111, was opened at 2006-12-23 13:03
Message generated for change (Comment added) made by mehum
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1621111&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Macintosh
Group: Python 2.5
Status: Open
Resolution: None
Priority: 6
Private: No
Submitted By: Mick L (mehum)
Assigned to: Ronald Oussoren (ronaldoussoren)
Summary: IDLE crashes on OS X 10.4 when "Preferences" selected

Initial Comment:
When I select 'preferences' (on IDLE 1.2, Python 2.5)  or 'Configure IDLE' (IDLE 1.1.4, Python 2.4.4 and IDLE 1.0, Python 2.3.3), Python crashes as soon as the dialog box appears.

Terminal reports a "Segmentation fault" upon the crash.

My system is an iMac G5 with 1 GB ram running OS X v 10.4.8, Tcl/Tk 8.4.10, and seems otherwise stable.  

IDLE also appears stable until I select this option.

The Python crash log is attached.

Please, can anyone suggest what may be causing this problem?  Thank you.



----------------------------------------------------------------------

>Comment By: Mick L (mehum)
Date: 2007-04-12 17:27

Message:
Logged In: YES 
user_id=1674554
Originator: YES

OK... I followed your advice Ronald (moved the Tcl and Tk frameworks out
of Library), and it worked!  IDLE no longer crashes! Thank you so much!

With the frameworks moved, it apparently reverts to Tcl/Tk 8.4.7

The only oddity is that there is a _very_ long delay (beachball) when
opening preferences / configure IDLE; but now that the font is legibly
large, I doubt I shall need to go back there.

Thank you again,
Kind regards, Mick L

----------------------------------------------------------------------

Comment By: Ronald Oussoren (ronaldoussoren)
Date: 2007-04-12 05:24

Message:
Logged In: YES 
user_id=580910
Originator: NO

It seems to be caused by Tcl/Tk, I cannot reproduce this with the system
version of Tk

This is on 10.4.8/intel, with python 2.5.1rc1. I don't think the change in
python version fixes this issue.

Mick: could you please test what happens when you temporarily move
/Library/Frameworks/Tcl.framework and /Library/Frameworks/Tk.framework to
another folder.

----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2007-04-12 02:12

Message:
Logged In: YES 
user_id=80475
Originator: NO

Bumping up the priority so this doesn't get lost.  My guess is that IDLE
is tickling some problem with the TkInter bindings.

----------------------------------------------------------------------

Comment By: Fran?ois Scholder (f_scholder)
Date: 2007-02-03 06:55

Message:
Logged In: YES 
user_id=1609867
Originator: NO

Erratum:

A similar problem arises with WinXP when trying to configure IDLE via the
"Options>Configure IDLE..." menu.

Clicking on the 'Ok' or 'Apply' buttons of the dialog seemingly has no
effect. However when closing the window by clicking on either the window
close button (top right 'X') or the 'Cancel' button, several bahaviors
are
observed depending on how many times one has clicked on the 'Ok' and/or
---->'Apply'<---- buttons before:
- 0 clicks => expected behavior (dialog is closed, IDLE session is
resumed
and working fine)
- 1 or 2 clicks => dialog is closed but the IDLE shell behaves as a text
editor instead of an interpreter
- 3 or more clicks => IDLE crashes...

Note: I'm running Python 2.5 (r25:51908, Sep 19 2006, 09:52:17) [MSC
v.1310 32 bit (Intel)] on win32 -> WinXP SP2

----------------------------------------------------------------------

Comment By: Fran?ois Scholder (f_scholder)
Date: 2007-02-03 06:48

Message:
Logged In: YES 
user_id=1609867
Originator: NO

A similar problem arises with WinXP when trying to configure IDLE via the
"Options>Configure IDLE..." menu.

Clicking on the 'Ok' or 'Apply' buttons of the dialog seemingly has no
effect. However when closing the window by clicking on either the window
close button (top right 'X') or the 'Cancel' button, several bahaviors are
observed depending on how many times one has clicked on the 'Ok' and/or
'Cancel' buttons before:
- 0 clicks => expected behavior (dialog is closed, IDLE session is resumed
and working fine)
- 1 or 2 clicks => dialog is closed but the IDLE shell behaves as a text
editor instead of an interpreter
- 3 or more clicks => IDLE crashes...

Note: I'm running Python 2.5 (r25:51908, Sep 19 2006, 09:52:17) [MSC
v.1310 32 bit (Intel)] on win32 -> WinXP SP2

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1621111&group_id=5470

From noreply at sourceforge.net  Thu Apr 12 11:10:45 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Thu, 12 Apr 2007 02:10:45 -0700
Subject: [ python-Bugs-1697943 ] msgfmt cannot cope with BOM
Message-ID: <E1HbvKH-0006Ou-DI@sc8-sf-web21.sourceforge.net>

Bugs item #1697943, was opened at 2007-04-10 22:58
Message generated for change (Comment added) made by cito
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1697943&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Demos and Tools
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Christoph Zwerschke (cito)
Assigned to: Martin v. L?wis (loewis)
Summary: msgfmt cannot cope with BOM

Initial Comment:
If a .po file has a BOM (byte order mark) at the beginning, as is often the case for utf-8 files created on Windows, msgfmt.py complines about a syntax error.

The attached patch fixes this problem.

----------------------------------------------------------------------

>Comment By: Christoph Zwerschke (cito)
Date: 2007-04-12 11:10

Message:
Logged In: YES 
user_id=193957
Originator: YES

It may well be that GNU gettext also chokes on a BOM, because they aren't
used under Linux. But I think as a Python tool it should be more
Windows-tolerant. The annoying thing is that you get a syntax error, but
everything looks right because the BOM is usually invisible. Such error
messages are really frustrating. Either the BOM should be silently ignored
(as in the patch) or the users should get a friendly error message asking
them to save the file without BOM. If GNU behaves badly to Windows users,
that's not an excuse to do the same. They are already suffering enough
because of their (or their bosses') bad choice of OS ;-)



----------------------------------------------------------------------

Comment By: Martin v. L?wis (loewis)
Date: 2007-04-12 00:13

Message:
Logged In: YES 
user_id=21627
Originator: NO

It's my code, but I will need to establish first whether it's a bug. That
depends on what the PO specification says, and, if is it silent on the
matter, what GNU gettext does.

----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2007-04-11 18:07

Message:
Logged In: YES 
user_id=80475
Originator: NO

Martin, is this your code?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1697943&group_id=5470

From noreply at sourceforge.net  Thu Apr 12 11:19:44 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Thu, 12 Apr 2007 02:19:44 -0700
Subject: [ python-Bugs-1613130 ] str.split creates new string even if pattern
	not found
Message-ID: <E1HbvSy-0000Pr-FS@sc8-sf-web24.sourceforge.net>

Bugs item #1613130, was opened at 2006-12-11 14:03
Message generated for change (Comment added) made by pitrou
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1613130&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Performance
Group: Python 2.5
Status: Open
Resolution: None
Priority: 1
Private: No
Submitted By: Antoine Pitrou (pitrou)
Assigned to: Fredrik Lundh (effbot)
Summary: str.split creates new string even if pattern not found

Initial Comment:
Hello,

Several string methods avoid allocating a new string when the operation result is trivially the same as one of the parameters (e.g. replacing a non-existing substring). However, split() does not exhibit this optimization, it always constructs a new string even if no splitting occurs:

$ python
Python 2.5 (r25:51908, Oct  6 2006, 15:22:41) 
[GCC 4.1.2 20060928 (prerelease) (Ubuntu 4.1.1-13ubuntu4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> s = "abcde" * 2
>>> id(s)
3084139400L
>>> id(str(s))
3084139400L
>>> id("" + s)
3084139400L
>>> id(s.strip())
3084139400L
>>> id(s.replace("g", "h"))
3084139400L
>>> [id(x) for x in s.partition("h")]
[3084139400L, 3084271768L, 3084271768L]
>>> [id(x) for x in s.split("h")]
[3084139360L]


----------------------------------------------------------------------

>Comment By: Antoine Pitrou (pitrou)
Date: 2007-04-12 11:19

Message:
Logged In: YES 
user_id=133955
Originator: YES

Hi,

> Dropping the priority.  This pay-off is near zero and likely not worth
the
> cost of making the code more complex than it already is.

No problem!
The more interesting question actually was whether it made any sense to
factor out the split() implementation in "stringlib" so as to share the
implementation between str and unicode.

Also, as for the USE_FAST question you asked on python-dev, I may have an
answer: if you try to enable USE_FAST you'll see that some operations are
indeed faster on large strings (say 100s or 1000s of characters), but they
become slower on small strings because of the larger overhead of the search
algorithm. Thus USE_FAST could negatively impact Python programs which
process a lot of small strings.


----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2007-04-11 19:09

Message:
Logged In: YES 
user_id=80475
Originator: NO

Dropping the priority.  This pay-off is near zero and likely not worth the
cost of making the code more complex than it already is.

----------------------------------------------------------------------

Comment By: Georg Brandl (gbrandl)
Date: 2006-12-12 17:21

Message:
Logged In: YES 
user_id=849994
Originator: NO

Sounds like this is best assigned to Fredrik.

----------------------------------------------------------------------

Comment By: Antoine Pitrou (pitrou)
Date: 2006-12-12 12:35

Message:
Logged In: YES 
user_id=133955
Originator: YES

Ok, I did a patch which partially adds the optimization (the patch is at
home, I can't post it right now). I have a few questions though:
 - there is a USE_FAST flag which can bring some speedups when a
multicharacter separator is used; however, it is not enabled by default, is
there a reason for this?
 - where and by whom is maintained stringbench.py, so that I can propose
additional tests for it (namely, tests for unmatched split())?
 - split() implementation is duplicated between str and unicode (the
unicode versions having less optimizations), would it be useful to
"stringlib'ify" split()?
 - rsplit() does quite similar things as split(), has anyone tried to
factor similar parts? do you see any caveats doing so?


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1613130&group_id=5470

From noreply at sourceforge.net  Thu Apr 12 18:35:41 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Thu, 12 Apr 2007 09:35:41 -0700
Subject: [ python-Bugs-1693753 ] Portability issue: os.rename behaves
	differently on win32
Message-ID: <E1Hc2Gr-0005CP-33@sc8-sf-web23.sourceforge.net>

Bugs item #1693753, was opened at 2007-04-03 12:17
Message generated for change (Comment added) made by draghuram
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1693753&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Platform-specific
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: cs_ (cs_)
Assigned to: Nobody/Anonymous (nobody)
Summary: Portability issue: os.rename behaves differently on win32

Initial Comment:
os.rename(src, dst) fails on Windows if dst already exist and regular file. On Unixes, it does not fail, and silently overwrites dst. It's better to use MoveFileEx(src, dst, MOVEFILE_REPLACE_EXISTING) when moving regular files under Win32 to make it behave consistent way.

----------------------------------------------------------------------

Comment By: Raghuram Devarakonda (draghuram)
Date: 2007-04-12 12:35

Message:
Logged In: YES 
user_id=984087
Originator: NO


>From MSDN document and googling, it does look like
MOVEFILE_REPLACE_EXISTING guarantees atomicity for rename operation. I will
see if I can come up with a patch (I will first need to learn building
python on windows).

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1693753&group_id=5470

From noreply at sourceforge.net  Thu Apr 12 20:34:26 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Thu, 12 Apr 2007 11:34:26 -0700
Subject: [ python-Bugs-1697820 ] __getslice__ still used in built-in types
Message-ID: <E1Hc47m-00041D-8H@sc8-sf-web22.sourceforge.net>

Bugs item #1697820, was opened at 2007-04-10 12:06
Message generated for change (Comment added) made by rhettinger
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1697820&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Documentation
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Torsten Bronger (bronger)
>Assigned to: Raymond Hettinger (rhettinger)
Summary: __getslice__ still used in built-in types

Initial Comment:
In the Python language reference <http://docs.python.org/ref/sequence-methods.html>, after the sentence

"""Deprecated since release 2.0. Support slice objects as parameters to the __getitem__() method."""

I suggest the following addition:

"""However, built-in types in CPython currently still implement __getslice__.  Therefore, you have to override it in derived classes when implementing slicing."""

----------------------------------------------------------------------

>Comment By: Raymond Hettinger (rhettinger)
Date: 2007-04-12 13:34

Message:
Logged In: YES 
user_id=80475
Originator: NO

Sensible suggestion.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1697820&group_id=5470

From noreply at sourceforge.net  Fri Apr 13 00:49:03 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Thu, 12 Apr 2007 15:49:03 -0700
Subject: [ python-Bugs-1699594 ] shlex fails to parse strings correctly
Message-ID: <E1Hc86B-0003rB-Ka@sc8-sf-web24.sourceforge.net>

Bugs item #1699594, was opened at 2007-04-12 18:49
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1699594&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Collin Winter (collinwinter)
Assigned to: Nobody/Anonymous (nobody)
Summary: shlex fails to parse strings correctly

Initial Comment:
shlex fails to parse

"\!echo \"it's fine\""

and

 "\!echo \"it\'s fine\""

correctly. ValueError("No closing quotation") is raised incorrectly.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1699594&group_id=5470

From noreply at sourceforge.net  Fri Apr 13 00:50:10 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Thu, 12 Apr 2007 15:50:10 -0700
Subject: [ python-Bugs-1699594 ] shlex fails to parse strings correctly
Message-ID: <E1Hc87G-0003ya-59@sc8-sf-web24.sourceforge.net>

Bugs item #1699594, was opened at 2007-04-12 18:49
Message generated for change (Comment added) made by collinwinter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1699594&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Collin Winter (collinwinter)
Assigned to: Nobody/Anonymous (nobody)
Summary: shlex fails to parse strings correctly

Initial Comment:
shlex fails to parse

"\!echo \"it's fine\""

and

 "\!echo \"it\'s fine\""

correctly. ValueError("No closing quotation") is raised incorrectly.

----------------------------------------------------------------------

>Comment By: Collin Winter (collinwinter)
Date: 2007-04-12 18:50

Message:
Logged In: YES 
user_id=1344176
Originator: YES

This should probably be fixed at the same time as #1521950.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1699594&group_id=5470

From noreply at sourceforge.net  Fri Apr 13 09:22:54 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Fri, 13 Apr 2007 00:22:54 -0700
Subject: [ python-Bugs-1699759 ] pickle example contains errors
Message-ID: <E1HcG7S-0007PW-28@sc8-sf-web23.sourceforge.net>

Bugs item #1699759, was opened at 2007-04-13 02:22
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1699759&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Documentation
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Mark Edgington (edgimar)
Assigned to: Nobody/Anonymous (nobody)
Summary: pickle example contains errors

Initial Comment:
In the pickle documentation (see http://docs.python.org/lib/pickle-example.html ), there is an error.  At the end of the page, under the "A sample usage might be..." section, the file for dumping is opened in text-mode instead of binary mode.

This bit me because I was lazy and didn't re-read all of the pickle documentation, but based some code on this snippet.  The problem occurs under certain circumstances when pickling a type 'object' instance (newstyle) under Windows, and then attempting to unpickle it under Linux.  You get the following error: ImportError: No module named copy_reg.  This made no sense to me, and it took a long time to figure out that the problem was due to the mode in which the file was saved (what that has to do with the ImportError I still have no idea...).  If interested, I could attach a test script which is supposed to load the data to a class instance, and two pickle dumps, one which works, and the other which fails.

Perhaps a related suggestion which maybe should be in a different bug is that pickle should check to see when it writing to a filehandle if the filehandle's mode is binary or text, and to issue a warning if it is text.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1699759&group_id=5470

From noreply at sourceforge.net  Fri Apr 13 12:26:11 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Fri, 13 Apr 2007 03:26:11 -0700
Subject: [ python-Bugs-1699853 ] locale.getlocale() output fails as
	setlocale() input 
Message-ID: <E1HcIyp-0006eR-RI@sc8-sf-web21.sourceforge.net>

Bugs item #1699853, was opened at 2007-04-13 12:26
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1699853&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Bernhard Reiter (ber)
Assigned to: Nobody/Anonymous (nobody)
Summary: locale.getlocale() output fails as setlocale() input 

Initial Comment:
This problem report about the locale module
consists of three closely related parts 
(this is why I have decided to put it in one report).
a) the example in the docs is wrong / missleading
b) under some locale settings python as a defect
c) a test case for the locale module, showing b)
   but useful as general start for a test module.

Details:
        a)
        Section example:
                The line
                >>> loc = locale.getlocale(locale.LC_ALL) # get current locale
                contradicts that getlocale should not be called with
                        LC_ALL, as stated in the description of getlocale.
                Suggestion is to change the example to be more useful
                as getting the locale as first action is not really useful.
                It should be "C" anyway which will lead to (None, None)
                so the value is already known. It would make more sense to

                first set the default locale to the user preferences:
import locale
locale.setlocale(locale.LC_ALL,'')
loc = locale.getlocale(locale.LC_NUMERIC)
locale.setlocale(locale.LC_NUMERIC,"C")
# convert a string here
locale.setlocale(locale.LC_NUMERIC, loc)

                _but_ this does not work, see problem b).
                What does work is:

import
locale.setlocale(locale.LC_ALL,'')
loc = locale.setlocale(locale.LC_NUMERIC)
locale.setlocale(locale.LC_NUMERIC,"C")
# convert a string here
locale.setlocale(locale.LC_NUMERIC, loc)

Note that all_loc = locale.setlocale(locale.LC_ALL) might contain
several categories (see attached test_locale.py where I needed to decode
this).
'LC_CTYPE=de_DE.UTF-8;LC_NUMERIC=en_GB.utf8;LC_TIME=de_DE.UTF-8;LC_COLLATE=de_DE.UTF-8;LC_MONETARY=de_DE.UTF-8;LC_MESSAGES=de_DE.UTF-8;LC_PAPER=de_DE.UTF-8;LC_NAME=de_DE.UTF-8;LC_ADDRESS=de_DE.UTF-8;LC_TELEPHONE=de_DE.UTF-8;LC_MEASUREMENT=de_DE.UTF-8;LC_IDENTIFICATION=de_DE.UTF-8'


        b)
                The output of getlocale cannot be used as input to
                setlocale sometimes.
                Works with
                * python2.5 und python2.4 on
                  Debian GNU/Linux Etch ppc, de_DE.utf8.

                I had failures with
                * python2.3, python2.4, python2.5
                  on Debian GNU/Linux Sarge ppc, de_DE at euro
                * Windows XP SP2
                        python-2.4.4.msi    German, see:

                >>> import locale
                >>> result = locale.setlocale(locale.LC_NUMERIC,"")
                >>> print result
                German_Germany.1252
                >>> got = locale.getlocale(locale.LC_NUMERIC)
                >>> print got
                ('de_DE', '1252')
                >>> # works
                ... locale.setlocale(locale.LC_NUMERIC, result)
                'German_Germany.1252'
                >>> # fails
                ... locale.setlocale(locale.LC_NUMERIC, got)
                Traceback (most recent call last):
                  File "<stdin>", line 2, in ?
                  File "C:\Python24\lib\locale.py", line 381, in setlocale
                    return _setlocale(category, locale)
                locale.Error: unsupported locale setting
                >>>



----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1699853&group_id=5470

From noreply at sourceforge.net  Fri Apr 13 15:32:04 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Fri, 13 Apr 2007 06:32:04 -0700
Subject: [ python-Bugs-1698167 ] xml.etree document element.tag
Message-ID: <E1HcLsi-00059I-CT@sc8-sf-web21.sourceforge.net>

Bugs item #1698167, was opened at 2007-04-11 08:25
Message generated for change (Comment added) made by effbot
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1698167&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Documentation
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: paul rubin (phr)
Assigned to: Fredrik Lundh (effbot)
Summary: xml.etree document element.tag

Initial Comment:
The xml.etree docs vaguely mention an implementation-dependent Element interface without describing it in any detail.  I could not figure out from the docs how to get the tag name of an element returned from (say) the getiterator interface.  That is, for an element like <foo/>, I wanted the string "foo".  Examining the library source showed that e.tag does the job, at least some of the time, and that was enough to get my app working.  Could the actual situation please be documented--thanks.

----------------------------------------------------------------------

>Comment By: Fredrik Lundh (effbot)
Date: 2007-04-13 15:32

Message:
Logged In: YES 
user_id=38376
Originator: NO

Looks like the entire Element section is missing from the current
documentation.  Thanks for reporting this; I'll take a look when I find the
time.

In the meantime, you'll find additional documentation here:
http://effbot.org/zone/element.htm

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1698167&group_id=5470

From noreply at sourceforge.net  Fri Apr 13 15:45:50 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Fri, 13 Apr 2007 06:45:50 -0700
Subject: [ python-Bugs-1599254 ] mailbox: other programs' messages can vanish
	without trace
Message-ID: <E1HcM62-0006cN-Cl@sc8-sf-web21.sourceforge.net>

Bugs item #1599254, was opened at 2006-11-19 16:03
Message generated for change (Comment added) made by baikie
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1599254&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.5
Status: Open
Resolution: None
Priority: 7
Private: No
Submitted By: David Watson (baikie)
Assigned to: A.M. Kuchling (akuchling)
Summary: mailbox: other programs' messages can vanish without trace

Initial Comment:
The mailbox classes based on _singlefileMailbox (mbox, MMDF, Babyl) implement the flush() method by writing the new mailbox contents into a temporary file which is then renamed over the original. Unfortunately, if another program tries to deliver messages while mailbox.py is working, and uses only fcntl() locking, it will have the old file open and be blocked waiting for the lock to become available. Once mailbox.py has replaced the old file and closed it, making the lock available, the other program will write its messages into the now-deleted "old" file, consigning them to oblivion.

I've caused Postfix on Linux to lose mail this way (although I did have to turn off its use of dot-locking to do so).

A possible fix is attached.  Instead of new_file being renamed, its contents are copied back to the original file.  If file.truncate() is available, the mailbox is then truncated to size.  Otherwise, if truncation is required, it's truncated to zero length beforehand by reopening self._path with mode wb+.  In the latter case, there's a check to see if the mailbox was replaced while we weren't looking, but there's still a race condition.  Any alternative ideas?

Incidentally, this fixes a problem whereby Postfix wouldn't deliver to the replacement file as it had the execute bit set.


----------------------------------------------------------------------

>Comment By: David Watson (baikie)
Date: 2007-04-13 14:45

Message:
Logged In: YES 
user_id=1504904
Originator: YES

Here's a possible solution to the rereading problem.  It should allow
existing applications to work whether they behave properly or not, as
well as (probably) most third-party subclasses.

The idea is to compare the new table of contents to the list of known
offset pairs, and raise ExternalClashError if *any* of them have
changed or disappeared.  Any new pairs can then be added to _toc under
new keys.  To maintain the list of known pairs, a special dict
subclass is used on self._toc that records every offset pair ever set
in it - even those that are subsequently removed from the mapping.
However, if self._pending is not set when rereading, then the code
doesn't rely on this special behaviour: it just uses
self._toc.itervalues(), which will work even if a subclass has
replaced the special _toc with a normal dictionary.

Ways the code can break:

- If a subclass replaces self._toc and then the application tries to
lock the mailbox *after* making modifications (so that _update_toc()
finds self._pending set, and looks for the special attribute on _toc).

- If a subclass tries to store something other than sequences in _toc.

- If a subclass' _generate_toc() can produce offsets for messages that
don't match those they were written under.

File Added: mailbox-update-toc-again.diff

----------------------------------------------------------------------

Comment By: A.M. Kuchling (akuchling)
Date: 2007-03-28 17:56

Message:
Logged In: YES 
user_id=11375
Originator: NO

Created a branch in SVN at
svn+ssh://pythondev at svn.python.org/python/branches/amk-mailbox to work on
this for 2.5.1.  I've committed the unified2 module patch and the
test_concurrent_ad() test.


----------------------------------------------------------------------

Comment By: A.M. Kuchling (akuchling)
Date: 2007-01-24 20:48

Message:
Logged In: YES 
user_id=11375
Originator: NO

I've strengthened the warning again.

The MH bug in unified2 is straightforward: MH.remove() opens a file
object, locks it, closes the file object, and then tries to unlock it. 
Presumably the MH test case never bothered locking the mailbox before
making changes before.


----------------------------------------------------------------------

Comment By: David Watson (baikie)
Date: 2007-01-22 20:24

Message:
Logged In: YES 
user_id=1504904
Originator: YES

So what you propose to commit for 2.5 is basically mailbox-unified2
(your mailbox-unified-patch, minus the _toc clearing)?


----------------------------------------------------------------------

Comment By: A.M. Kuchling (akuchling)
Date: 2007-01-22 15:46

Message:
Logged In: YES 
user_id=11375
Originator: NO

This would be an API change, and therefore out-of-bounds for 2.5.

I suggest giving up on this for 2.5.1 and only fixing it in 2.6.   I'll
add another warning to the docs, and maybe to the module as well.



----------------------------------------------------------------------

Comment By: David Watson (baikie)
Date: 2007-01-21 22:10

Message:
Logged In: YES 
user_id=1504904
Originator: YES

Hold on, I have a plan.  If _toc is only regenerated on locking, or at
the end of a flush(), then the only way self._pending can be set at
that time is if the application has made modifications before calling
lock().  If we make that an exception-raising offence, then we can
assume that self._toc is a faithful representation of the last known
contents of the file.  That means we can preserve the existing message
keys on a reread without any of that _user_toc nonsense.

Diff attached, to apply on top of mailbox-unified2.  It's probably had
even less review and testing than the previous version, but it appears
to pass all the regression tests and doesn't change any existing
semantics.

File Added: mailbox-update-toc-new.diff

----------------------------------------------------------------------

Comment By: A.M. Kuchling (akuchling)
Date: 2007-01-21 03:16

Message:
Logged In: YES 
user_id=11375
Originator: NO

<sigh>  I'm starting to lose track of all the variations on the bug. 
Maybe we should just add more warnings to the documentation about locking
the mailbox when modifying it and not try to fix this at all.


----------------------------------------------------------------------

Comment By: David Watson (baikie)
Date: 2007-01-20 18:20

Message:
Logged In: YES 
user_id=1504904
Originator: YES

Hang on.  If a message's key changes after recreating _toc, that does
not mean that another process has modified the mailbox.  If the
application removes a message and then (inadvertently) causes _toc to
be regenerated, the keys of all subsequent messages will be
decremented by one, due only to the application's own actions.

That's what happens in the "broken locking" test case: the program
intends to remove message 0, flush, and then remove message 1, but
because _toc is regenerated in between, message 1 is renumbered as 0,
message 2 is renumbered as 1, and so the program deletes message 2
instead.  To clear _toc in such code without attempting to preserve
the message keys turns possible data loss (in the case that another
process modified the mailbox) into certain data loss.  That's what I'm
concerned about.


----------------------------------------------------------------------

Comment By: A.M. Kuchling (akuchling)
Date: 2007-01-19 15:24

Message:
Logged In: YES 
user_id=11375
Originator: NO

After reflection, I don't think the potential changing actually makes
things any worse.  _generate() always starts numbering keys with 1, so if a
message's key changes because of lock()'s, re-reading, that means someone
else has already modified the mailbox.  Without the ToC clearing, you're
already fated to have a corrupted mailbox because the new mailbox will be
written using outdated file offsets.  With the ToC clearing, you delete the
wrong message.  Neither outcome is good, but data is lost either way.  

The new behaviour is maybe a little bit better in that you're losing a
single message but still generating a well-formed mailbox, and not a
randomly jumbled mailbox.

I suggest applying the patch to clear self._toc, and noting in the
documentation that keys might possibly change after doing a lock().


----------------------------------------------------------------------

Comment By: David Watson (baikie)
Date: 2007-01-18 18:15

Message:
Logged In: YES 
user_id=1504904
Originator: YES

This version passes the new tests (it fixes the length checking bug,
and no longer clears self._toc), but goes back to failing
test_concurrent_add.

File Added: mailbox-unified2-module.diff

----------------------------------------------------------------------

Comment By: David Watson (baikie)
Date: 2007-01-18 18:14

Message:
Logged In: YES 
user_id=1504904
Originator: YES

Unfortunately, there is a problem with clearing _toc: it's basically
the one I alluded to in my comment of 2006-12-16.  Back then I thought
it could be caught by the test you issue the warning for, but the
application may instead do its second remove() *after* the lock() (so
that self._pending is not set at lock() time), using the key carried
over from before it called unlock().  As before, this would result in
the wrong message being deleted.

I've added a test case for this (diff attached), and a bug I found in
the process whereby flush() wasn't updating the length, which could
cause subsequent flushes to fail (I've got a fix for this).  These
seem to have turned up a third bug in the MH class, but I haven't
looked at that yet.

File Added: mailbox-unified2-test.diff

----------------------------------------------------------------------

Comment By: A.M. Kuchling (akuchling)
Date: 2007-01-17 21:06

Message:
Logged In: YES 
user_id=11375
Originator: NO

Add mailbox-unified-patch.

File Added: mailbox-unified-patch.diff

----------------------------------------------------------------------

Comment By: A.M. Kuchling (akuchling)
Date: 2007-01-17 21:05

Message:
Logged In: YES 
user_id=11375
Originator: NO

mailbox-pending-lock is the difference between David's copy-back-new +
fcntl-warning and my -unified patch, uploaded so that he can comment on the
changes.

The only change is to make _singleFileMailbox.lock() clear self._toc,
forcing a re-read of the mailbox file.  If _pending is true, the ToC isn't
cleared and a warning is logged.  I think this lets existing code run
(albeit possibly with a warning if the mailbox is modified before .lock()
is called), but fixes the risk of missing changes written by another
process.

Triggering a new warning is sort of an API change, but IMHO it's still
worth backporting; code that triggers this warning is at risk of losing
messages or corrupting the mailbox.

Clearing the _toc on .lock() is also sort of an API change, but it's
necessary to avoid the potential data loss.  It may lead to some redundant
scanning of mailbox files, but this price is worth paying, I think; people
probably don't have 2Gb mbox files (I hope not, anyway!) and no extra read
is done if you create the mailbox and immediately lock it before looking
anything up.

Neal: if you want to discuss this patch or want an explanation of
something, feel free to chat with me about it.

I'll wait a day or two and see if David spots any problems.  If nothing
turns up, I'll commit it to both trunk and release25-maint.



File Added: mailbox-pending-lock.diff

----------------------------------------------------------------------

Comment By: A.M. Kuchling (akuchling)
Date: 2007-01-17 20:53

Message:
Logged In: YES 
user_id=11375
Originator: NO

mailbox-unified-patch contains David's copy-back-new and fcntl-warn
patches, plus the test-mailbox patch and some additional changes to
mailbox.py from me.  (I'll upload a diff to David's diffs in a minute.)

This is the change I'd like to check in.  test_mailbox.py now passes, as
does the mailbox-break.py script I'm using.


----------------------------------------------------------------------

Comment By: A.M. Kuchling (akuchling)
Date: 2007-01-17 19:56

Message:
Logged In: YES 
user_id=11375
Originator: NO

Committed a modified version of the doc. patch in rev. 53472 (trunk) and
rev. 53474 (release25-maint).


----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2007-01-17 06:48

Message:
Logged In: YES 
user_id=33168
Originator: NO

Andrew, do you need any help with this?

----------------------------------------------------------------------

Comment By: A.M. Kuchling (akuchling)
Date: 2007-01-15 19:01

Message:
Logged In: YES 
user_id=11375
Originator: NO

Comment from Andrew MacIntyre (os2vacpp is the OS/2 that lacks
ftruncate()):

================

I actively support the OS/2 EMX port (sys.platform == "os2emx"; build
directory is PC/os2emx).  I would like to keep the VAC++ port alive, but
the reality is I don't have the resources to do so.  The VAC++ port was
the subject of discussion about removal of build support support from the
source tree for 2.6 - I don't recall there being a definitive outcome,
but if someone does delete the PC/os2vacpp directory, I'm not in a
position to argue.

AMK: (mailbox.py has a separate section of code used when file.truncate()
isn't available, and the existence of this section is bedevilling me.
It would be convenient if platforms without file.truncate() weren't a
factor; then this branch could just be removed.  In your opinion,
would it hurt OS/2 users of mailbox.py if support for platforms
without file.truncate() was removed?)

aimacintyre: No.  From what documentation I can quickly check, ftruncate()
operates
on file descriptors rather than FILE pointers.  As such I am sure that,
if it became an issue, it would not be difficult to write a ftruncate()
emulation wrapper for the underlying OS/2 APIs that implement the
required functionality.



----------------------------------------------------------------------

Comment By: David Watson (baikie)
Date: 2007-01-13 18:32

Message:
Logged In: YES 
user_id=1504904
Originator: YES

I like the warning idea - it seems appropriate if the problem is
relatively rare.  How about this?

File Added: mailbox-fcntl-warn.diff

----------------------------------------------------------------------

Comment By: A.M. Kuchling (akuchling)
Date: 2007-01-12 19:41

Message:
Logged In: YES 
user_id=11375
Originator: NO

One OS/2 port lacks truncate(), and so does RISCOS.

----------------------------------------------------------------------

Comment By: A.M. Kuchling (akuchling)
Date: 2007-01-12 18:41

Message:
Logged In: YES 
user_id=11375
Originator: NO

I realized that making flush() invalidate keys breaks the final example in
the docs, which loops over inbox.iterkeys() and removes messages, doing a
pack() after each message.

Which platforms lack file.truncate()?  Windows has it; POSIX has it, so
modern Unix variants should all have it.  Maybe mailbox should simply raise
an exception (or trigger a warning?) if truncate is missing, and we should
then assume that flush() has no effect upon keys.


----------------------------------------------------------------------

Comment By: A.M. Kuchling (akuchling)
Date: 2007-01-12 17:12

Message:
Logged In: YES 
user_id=11375
Originator: NO

So shall we document flush() as invalidating keys, then?

----------------------------------------------------------------------

Comment By: David Watson (baikie)
Date: 2007-01-06 19:57

Message:
Logged In: YES 
user_id=1504904
Originator: YES

Oops, length checking had made the first two lines of this patch
redundant; update-toc applies OK with fuzz.

File Added: mailbox-copy-back-new.diff

----------------------------------------------------------------------

Comment By: David Watson (baikie)
Date: 2007-01-06 15:30

Message:
Logged In: YES 
user_id=1504904
Originator: YES

File Added: mailbox-copy-back-53287.diff

----------------------------------------------------------------------

Comment By: David Watson (baikie)
Date: 2007-01-06 15:24

Message:
Logged In: YES 
user_id=1504904
Originator: YES

Aack, yes, that should be _next_user_key.  Attaching a fixed version.

I've been thinking, though: flush() does in fact invalidate the keys
on platforms without a file.truncate(), when the fcntl() lock is
momentarily released afterwards.  It seems hard to avoid this as,
perversely, fcntl() locks are supposed to be released automatically on
all file descriptors referring to the file whenever the process closes
any one of them - even one the lock was never set on.

So, code using mailbox.py such platforms could inadvertently be
carrying keys across an unlocked period, which is not made safe by the
update-toc patch (as it's only meant to avert disasters resulting from
doing this *and* rebuilding the table of contents, *assuming* that
another process hasn't deleted or rearranged messages).

File Added: mailbox-update-toc-fixed.diff

----------------------------------------------------------------------

Comment By: A.M. Kuchling (akuchling)
Date: 2007-01-05 19:51

Message:
Logged In: YES 
user_id=11375
Originator: NO

Question about mailbox-update-doc: the add() method still returns
self._next_key - 1; should this be 
self._next_user_key - 1?  The keys in _user_toc are the ones returned to
external users of the mailbox, right?

(A good test case would be to initialize _next_key to 0 and _next_user_key
to a different value like 123456.)

I'm still staring at the patch, trying to convince myself that it will
help -- haven't spotted any problems, but this bug is making me nervous...


----------------------------------------------------------------------

Comment By: A.M. Kuchling (akuchling)
Date: 2007-01-05 19:24

Message:
Logged In: YES 
user_id=11375
Originator: NO

As a step toward improving matters, I've attached the suggested doc patch
(for both 25-maint and trunk).  It encourages people to use Maildir :),
explicitly states that modifications should be bracketed by lock(), and
fixes the examples to match.

It does not say that keys are invalidated by doing a flush(), because
we're going to try to avoid the necessity for that.


File Added: mailbox-docs.diff

----------------------------------------------------------------------

Comment By: A.M. Kuchling (akuchling)
Date: 2006-12-20 19:48

Message:
Logged In: YES 
user_id=11375
Originator: NO

Committed length-checking.diff to trunk in rev. 53110.


----------------------------------------------------------------------

Comment By: David Watson (baikie)
Date: 2006-12-20 19:19

Message:
Logged In: YES 
user_id=1504904
Originator: YES

File Added: mailbox-test-lock.diff

----------------------------------------------------------------------

Comment By: David Watson (baikie)
Date: 2006-12-20 19:17

Message:
Logged In: YES 
user_id=1504904
Originator: YES

Yeah, I think that should definitely go in.  ExternalClashError or a
subclass sounds fine to me (although you could make a whole taxonomy
of these errors, really).  It would be good to have the code actually
keep up with other programs' changes, though; a program might just
want to count the messages at first, say, and not make changes until
much later.

I've been trying out the second option (patch attached, to apply on
top of mailbox-copy-back), regenerating _toc on locking, but
preserving existing keys.  The patch allows existing _generate_toc()s
to work unmodified, but means that _toc now holds the entire last
known contents of the mailbox file, with the 'pending' (user-visible)
mailbox state being held in a new attribute, _user_toc, which is a
mapping from keys issued to the program to the keys of _toc
(i.e. sequence numbers in the file).  When _toc is updated, any new
messages that have appeared are given keys in _user_toc that haven't
been issued before, and any messages that have disappeared are removed
from it.  The code basically assumes that messages with the same
sequence number are the same message, though, so even if most cases
are caught by the length check, programs that make
deletions/replacements before locking could still delete the wrong
messages.  This behaviour could be trapped, though, by raising an
exception in lock() if self._pending is set (after all, code like that
would be incorrect unless it could be assumed that the mailbox module
kept hashes of each message or something).

Also attached is a patch to the test case, adding a lock/unlock around
the message count to make sure _toc is up-to-date if the parent
process finishes first; without it, there are still intermittent
failures.

File Added: mailbox-update-toc.diff

----------------------------------------------------------------------

Comment By: A.M. Kuchling (akuchling)
Date: 2006-12-20 14:46

Message:
Logged In: YES 
user_id=11375
Originator: NO

Attaching a patch that adds length checking: before doing a flush() on a
single-file mailbox, seek to the end and verify its length is unchanged. 
It raises an ExternalClashError if the file's length has changed.  (Should
there be a different exception for this case, perhaps a subclass of
ExternalClashError?)

I verified that this change works by running a program that added 25
messages, pausing between each one, and then did 'echo "new line" >
/tmp/mbox' from a shell while the program was running.

I also noticed that the self._lookup() call in self.flush() wasn't
necessary, and replaced it by an assertion.

I think this change should go on both the trunk and 25-maint branches.

File Added: length-checking.diff

----------------------------------------------------------------------

Comment By: A.M. Kuchling (akuchling)
Date: 2006-12-18 17:43

Message:
Logged In: YES 
user_id=11375
Originator: NO

Eep, correct; changing the key IDs would be a big problem for existing
code.  We could say 'discard all keys' after doing lock() or unlock(), but
this is an API change that means the fix couldn't be backported to
2.5-maint.

We could make generating the ToC more complicated, preserving key IDs when
possible; that may not be too difficult, though the code might be messy.

Maybe it's best to just catch this error condition: save the size of the
mailbox, updating it in _append_message(), and then make .flush() raise an
exception if the mailbox size has unexpectedly changed.


----------------------------------------------------------------------

Comment By: David Watson (baikie)
Date: 2006-12-16 19:09

Message:
Logged In: YES 
user_id=1504904
Originator: YES

Yes, I see what you mean.  I had tried multiple flushes, but only
inside a single lock/unlock.  But this means that in the no-truncate()
code path, even this is currently unsafe, as the lock is momentarily
released after flushing.

I think _toc should be regenerated after every lock(), as with the
risk of another process replacing/deleting/rearranging the messages,
it isn't valid to carry sequence numbers from one locked period to
another anyway, or from unlocked to locked.  However, this runs the
risk of dangerously breaking code that thinks it is valid to do so,
even in the case where the mailbox was *not* modified (i.e. turning
possible failure into certain failure).  For instance, if the program
removes message 1, then as things stand, the key "1" is no longer
used, and removing message 2 will remove the message that followed 1.
If _toc is regenerated in between, however (using the current code, so
that the messages are renumbered from 0), then the old message 2
becomes message 1, and removing message 2 will therefore remove the
wrong message.  You'd also have things like pending deletions and
replacements (also unsafe generally) being forgotten.  So it would
take some work to get right, if it's to work at all...


----------------------------------------------------------------------

Comment By: A.M. Kuchling (akuchling)
Date: 2006-12-15 14:06

Message:
Logged In: YES 
user_id=11375
Originator: NO

I'm testing the fix using two Python processes running mailbox.py, and my
test case fails even with your patch.  This is due to another bug, even in
the patched version.  

mbox has a dictionary attribute, _toc, mapping message keys to positions
in the file.  flush() writes out all the messages in self._toc and
constructs a new _toc with the new file offsets.  It doesn't re-read the
file to see if new messages were added by another process.

One fix that seems to work: instead of doing 'self._toc = new_toc' after
flush() has done its work, do self._toc = None.  The ToC will be
regenerated the next time _lookup() is called, causing a re-read of all the
contents of the mbox.  Inefficient, but I see no way around the necessity
for doing this.

It's not clear to me that my suggested fix is enough, though.  Process #1
opens a mailbox, reads the ToC, and the process does something else for 5
minutes.  In the meantime, process #2 adds a file to the mbox.  Process #1
then adds a message to the mbox and writes it out; it never notices process
#2's change.

Maybe the _toc has to be regenerated every time you call lock(), because
at this point you know there will be no further updates to the mbox by any
other process.  Any unlocked usage of _toc should also really be
regenerating _toc every time, because you never know if another process has
added a message... but that would be really inefficient.







----------------------------------------------------------------------

Comment By: A.M. Kuchling (akuchling)
Date: 2006-12-15 13:17

Message:
Logged In: YES 
user_id=11375
Originator: NO

The attached patch adds a test case to test_mailbox.py that demonstrates
the problem.  No modifications to mailbox.py are needed to show data loss.

Now looking at the patch...

File Added: mailbox-test.patch

----------------------------------------------------------------------

Comment By: A.M. Kuchling (akuchling)
Date: 2006-12-12 21:04

Message:
Logged In: YES 
user_id=11375
Originator: NO

I agree with David's analysis; this is in fact a bug.  I'll try to look at
the patch.

----------------------------------------------------------------------

Comment By: David Watson (baikie)
Date: 2006-11-19 20:44

Message:
Logged In: YES 
user_id=1504904
Originator: YES

This is a bug.  The point is that the code is subverting the protection of
its own fcntl locking.  I should have pointed out that Postfix was still
using fcntl locking, and that should have been sufficient.  (In fact, it
was due to its use of fcntl locking that it chose  precisely the wrong
moment to deliver mail.)  Dot-locking does protect against this, but not
every program uses it - which is precisely the reason that the code
implements fcntl locking in the first place.


----------------------------------------------------------------------

Comment By: Martin v. L?wis (loewis)
Date: 2006-11-19 20:02

Message:
Logged In: YES 
user_id=21627
Originator: NO

Mailbox locking was invented precisely to support this kind of operation.
Why do you complain that things break if you deliberately turn off the
mechanism preventing breakage?

I fail to see a bug here.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1599254&group_id=5470

From noreply at sourceforge.net  Fri Apr 13 15:46:39 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Fri, 13 Apr 2007 06:46:39 -0700
Subject: [ python-Bugs-1599254 ] mailbox: other programs' messages can vanish
	without trace
Message-ID: <E1HcM6p-0006j4-A2@sc8-sf-web21.sourceforge.net>

Bugs item #1599254, was opened at 2006-11-19 16:03
Message generated for change (Comment added) made by baikie
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1599254&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.5
Status: Open
Resolution: None
Priority: 7
Private: No
Submitted By: David Watson (baikie)
Assigned to: A.M. Kuchling (akuchling)
Summary: mailbox: other programs' messages can vanish without trace

Initial Comment:
The mailbox classes based on _singlefileMailbox (mbox, MMDF, Babyl) implement the flush() method by writing the new mailbox contents into a temporary file which is then renamed over the original. Unfortunately, if another program tries to deliver messages while mailbox.py is working, and uses only fcntl() locking, it will have the old file open and be blocked waiting for the lock to become available. Once mailbox.py has replaced the old file and closed it, making the lock available, the other program will write its messages into the now-deleted "old" file, consigning them to oblivion.

I've caused Postfix on Linux to lose mail this way (although I did have to turn off its use of dot-locking to do so).

A possible fix is attached.  Instead of new_file being renamed, its contents are copied back to the original file.  If file.truncate() is available, the mailbox is then truncated to size.  Otherwise, if truncation is required, it's truncated to zero length beforehand by reopening self._path with mode wb+.  In the latter case, there's a check to see if the mailbox was replaced while we weren't looking, but there's still a race condition.  Any alternative ideas?

Incidentally, this fixes a problem whereby Postfix wouldn't deliver to the replacement file as it had the execute bit set.


----------------------------------------------------------------------

>Comment By: David Watson (baikie)
Date: 2007-04-13 14:46

Message:
Logged In: YES 
user_id=1504904
Originator: YES

Some new test cases for this stuff.

File Added: test_mailbox-reread.diff

----------------------------------------------------------------------

Comment By: David Watson (baikie)
Date: 2007-04-13 14:45

Message:
Logged In: YES 
user_id=1504904
Originator: YES

Here's a possible solution to the rereading problem.  It should allow
existing applications to work whether they behave properly or not, as
well as (probably) most third-party subclasses.

The idea is to compare the new table of contents to the list of known
offset pairs, and raise ExternalClashError if *any* of them have
changed or disappeared.  Any new pairs can then be added to _toc under
new keys.  To maintain the list of known pairs, a special dict
subclass is used on self._toc that records every offset pair ever set
in it - even those that are subsequently removed from the mapping.
However, if self._pending is not set when rereading, then the code
doesn't rely on this special behaviour: it just uses
self._toc.itervalues(), which will work even if a subclass has
replaced the special _toc with a normal dictionary.

Ways the code can break:

- If a subclass replaces self._toc and then the application tries to
lock the mailbox *after* making modifications (so that _update_toc()
finds self._pending set, and looks for the special attribute on _toc).

- If a subclass tries to store something other than sequences in _toc.

- If a subclass' _generate_toc() can produce offsets for messages that
don't match those they were written under.

File Added: mailbox-update-toc-again.diff

----------------------------------------------------------------------

Comment By: A.M. Kuchling (akuchling)
Date: 2007-03-28 17:56

Message:
Logged In: YES 
user_id=11375
Originator: NO

Created a branch in SVN at
svn+ssh://pythondev at svn.python.org/python/branches/amk-mailbox to work on
this for 2.5.1.  I've committed the unified2 module patch and the
test_concurrent_ad() test.


----------------------------------------------------------------------

Comment By: A.M. Kuchling (akuchling)
Date: 2007-01-24 20:48

Message:
Logged In: YES 
user_id=11375
Originator: NO

I've strengthened the warning again.

The MH bug in unified2 is straightforward: MH.remove() opens a file
object, locks it, closes the file object, and then tries to unlock it. 
Presumably the MH test case never bothered locking the mailbox before
making changes before.


----------------------------------------------------------------------

Comment By: David Watson (baikie)
Date: 2007-01-22 20:24

Message:
Logged In: YES 
user_id=1504904
Originator: YES

So what you propose to commit for 2.5 is basically mailbox-unified2
(your mailbox-unified-patch, minus the _toc clearing)?


----------------------------------------------------------------------

Comment By: A.M. Kuchling (akuchling)
Date: 2007-01-22 15:46

Message:
Logged In: YES 
user_id=11375
Originator: NO

This would be an API change, and therefore out-of-bounds for 2.5.

I suggest giving up on this for 2.5.1 and only fixing it in 2.6.   I'll
add another warning to the docs, and maybe to the module as well.



----------------------------------------------------------------------

Comment By: David Watson (baikie)
Date: 2007-01-21 22:10

Message:
Logged In: YES 
user_id=1504904
Originator: YES

Hold on, I have a plan.  If _toc is only regenerated on locking, or at
the end of a flush(), then the only way self._pending can be set at
that time is if the application has made modifications before calling
lock().  If we make that an exception-raising offence, then we can
assume that self._toc is a faithful representation of the last known
contents of the file.  That means we can preserve the existing message
keys on a reread without any of that _user_toc nonsense.

Diff attached, to apply on top of mailbox-unified2.  It's probably had
even less review and testing than the previous version, but it appears
to pass all the regression tests and doesn't change any existing
semantics.

File Added: mailbox-update-toc-new.diff

----------------------------------------------------------------------

Comment By: A.M. Kuchling (akuchling)
Date: 2007-01-21 03:16

Message:
Logged In: YES 
user_id=11375
Originator: NO

<sigh>  I'm starting to lose track of all the variations on the bug. 
Maybe we should just add more warnings to the documentation about locking
the mailbox when modifying it and not try to fix this at all.


----------------------------------------------------------------------

Comment By: David Watson (baikie)
Date: 2007-01-20 18:20

Message:
Logged In: YES 
user_id=1504904
Originator: YES

Hang on.  If a message's key changes after recreating _toc, that does
not mean that another process has modified the mailbox.  If the
application removes a message and then (inadvertently) causes _toc to
be regenerated, the keys of all subsequent messages will be
decremented by one, due only to the application's own actions.

That's what happens in the "broken locking" test case: the program
intends to remove message 0, flush, and then remove message 1, but
because _toc is regenerated in between, message 1 is renumbered as 0,
message 2 is renumbered as 1, and so the program deletes message 2
instead.  To clear _toc in such code without attempting to preserve
the message keys turns possible data loss (in the case that another
process modified the mailbox) into certain data loss.  That's what I'm
concerned about.


----------------------------------------------------------------------

Comment By: A.M. Kuchling (akuchling)
Date: 2007-01-19 15:24

Message:
Logged In: YES 
user_id=11375
Originator: NO

After reflection, I don't think the potential changing actually makes
things any worse.  _generate() always starts numbering keys with 1, so if a
message's key changes because of lock()'s, re-reading, that means someone
else has already modified the mailbox.  Without the ToC clearing, you're
already fated to have a corrupted mailbox because the new mailbox will be
written using outdated file offsets.  With the ToC clearing, you delete the
wrong message.  Neither outcome is good, but data is lost either way.  

The new behaviour is maybe a little bit better in that you're losing a
single message but still generating a well-formed mailbox, and not a
randomly jumbled mailbox.

I suggest applying the patch to clear self._toc, and noting in the
documentation that keys might possibly change after doing a lock().


----------------------------------------------------------------------

Comment By: David Watson (baikie)
Date: 2007-01-18 18:15

Message:
Logged In: YES 
user_id=1504904
Originator: YES

This version passes the new tests (it fixes the length checking bug,
and no longer clears self._toc), but goes back to failing
test_concurrent_add.

File Added: mailbox-unified2-module.diff

----------------------------------------------------------------------

Comment By: David Watson (baikie)
Date: 2007-01-18 18:14

Message:
Logged In: YES 
user_id=1504904
Originator: YES

Unfortunately, there is a problem with clearing _toc: it's basically
the one I alluded to in my comment of 2006-12-16.  Back then I thought
it could be caught by the test you issue the warning for, but the
application may instead do its second remove() *after* the lock() (so
that self._pending is not set at lock() time), using the key carried
over from before it called unlock().  As before, this would result in
the wrong message being deleted.

I've added a test case for this (diff attached), and a bug I found in
the process whereby flush() wasn't updating the length, which could
cause subsequent flushes to fail (I've got a fix for this).  These
seem to have turned up a third bug in the MH class, but I haven't
looked at that yet.

File Added: mailbox-unified2-test.diff

----------------------------------------------------------------------

Comment By: A.M. Kuchling (akuchling)
Date: 2007-01-17 21:06

Message:
Logged In: YES 
user_id=11375
Originator: NO

Add mailbox-unified-patch.

File Added: mailbox-unified-patch.diff

----------------------------------------------------------------------

Comment By: A.M. Kuchling (akuchling)
Date: 2007-01-17 21:05

Message:
Logged In: YES 
user_id=11375
Originator: NO

mailbox-pending-lock is the difference between David's copy-back-new +
fcntl-warning and my -unified patch, uploaded so that he can comment on the
changes.

The only change is to make _singleFileMailbox.lock() clear self._toc,
forcing a re-read of the mailbox file.  If _pending is true, the ToC isn't
cleared and a warning is logged.  I think this lets existing code run
(albeit possibly with a warning if the mailbox is modified before .lock()
is called), but fixes the risk of missing changes written by another
process.

Triggering a new warning is sort of an API change, but IMHO it's still
worth backporting; code that triggers this warning is at risk of losing
messages or corrupting the mailbox.

Clearing the _toc on .lock() is also sort of an API change, but it's
necessary to avoid the potential data loss.  It may lead to some redundant
scanning of mailbox files, but this price is worth paying, I think; people
probably don't have 2Gb mbox files (I hope not, anyway!) and no extra read
is done if you create the mailbox and immediately lock it before looking
anything up.

Neal: if you want to discuss this patch or want an explanation of
something, feel free to chat with me about it.

I'll wait a day or two and see if David spots any problems.  If nothing
turns up, I'll commit it to both trunk and release25-maint.



File Added: mailbox-pending-lock.diff

----------------------------------------------------------------------

Comment By: A.M. Kuchling (akuchling)
Date: 2007-01-17 20:53

Message:
Logged In: YES 
user_id=11375
Originator: NO

mailbox-unified-patch contains David's copy-back-new and fcntl-warn
patches, plus the test-mailbox patch and some additional changes to
mailbox.py from me.  (I'll upload a diff to David's diffs in a minute.)

This is the change I'd like to check in.  test_mailbox.py now passes, as
does the mailbox-break.py script I'm using.


----------------------------------------------------------------------

Comment By: A.M. Kuchling (akuchling)
Date: 2007-01-17 19:56

Message:
Logged In: YES 
user_id=11375
Originator: NO

Committed a modified version of the doc. patch in rev. 53472 (trunk) and
rev. 53474 (release25-maint).


----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2007-01-17 06:48

Message:
Logged In: YES 
user_id=33168
Originator: NO

Andrew, do you need any help with this?

----------------------------------------------------------------------

Comment By: A.M. Kuchling (akuchling)
Date: 2007-01-15 19:01

Message:
Logged In: YES 
user_id=11375
Originator: NO

Comment from Andrew MacIntyre (os2vacpp is the OS/2 that lacks
ftruncate()):

================

I actively support the OS/2 EMX port (sys.platform == "os2emx"; build
directory is PC/os2emx).  I would like to keep the VAC++ port alive, but
the reality is I don't have the resources to do so.  The VAC++ port was
the subject of discussion about removal of build support support from the
source tree for 2.6 - I don't recall there being a definitive outcome,
but if someone does delete the PC/os2vacpp directory, I'm not in a
position to argue.

AMK: (mailbox.py has a separate section of code used when file.truncate()
isn't available, and the existence of this section is bedevilling me.
It would be convenient if platforms without file.truncate() weren't a
factor; then this branch could just be removed.  In your opinion,
would it hurt OS/2 users of mailbox.py if support for platforms
without file.truncate() was removed?)

aimacintyre: No.  From what documentation I can quickly check, ftruncate()
operates
on file descriptors rather than FILE pointers.  As such I am sure that,
if it became an issue, it would not be difficult to write a ftruncate()
emulation wrapper for the underlying OS/2 APIs that implement the
required functionality.



----------------------------------------------------------------------

Comment By: David Watson (baikie)
Date: 2007-01-13 18:32

Message:
Logged In: YES 
user_id=1504904
Originator: YES

I like the warning idea - it seems appropriate if the problem is
relatively rare.  How about this?

File Added: mailbox-fcntl-warn.diff

----------------------------------------------------------------------

Comment By: A.M. Kuchling (akuchling)
Date: 2007-01-12 19:41

Message:
Logged In: YES 
user_id=11375
Originator: NO

One OS/2 port lacks truncate(), and so does RISCOS.

----------------------------------------------------------------------

Comment By: A.M. Kuchling (akuchling)
Date: 2007-01-12 18:41

Message:
Logged In: YES 
user_id=11375
Originator: NO

I realized that making flush() invalidate keys breaks the final example in
the docs, which loops over inbox.iterkeys() and removes messages, doing a
pack() after each message.

Which platforms lack file.truncate()?  Windows has it; POSIX has it, so
modern Unix variants should all have it.  Maybe mailbox should simply raise
an exception (or trigger a warning?) if truncate is missing, and we should
then assume that flush() has no effect upon keys.


----------------------------------------------------------------------

Comment By: A.M. Kuchling (akuchling)
Date: 2007-01-12 17:12

Message:
Logged In: YES 
user_id=11375
Originator: NO

So shall we document flush() as invalidating keys, then?

----------------------------------------------------------------------

Comment By: David Watson (baikie)
Date: 2007-01-06 19:57

Message:
Logged In: YES 
user_id=1504904
Originator: YES

Oops, length checking had made the first two lines of this patch
redundant; update-toc applies OK with fuzz.

File Added: mailbox-copy-back-new.diff

----------------------------------------------------------------------

Comment By: David Watson (baikie)
Date: 2007-01-06 15:30

Message:
Logged In: YES 
user_id=1504904
Originator: YES

File Added: mailbox-copy-back-53287.diff

----------------------------------------------------------------------

Comment By: David Watson (baikie)
Date: 2007-01-06 15:24

Message:
Logged In: YES 
user_id=1504904
Originator: YES

Aack, yes, that should be _next_user_key.  Attaching a fixed version.

I've been thinking, though: flush() does in fact invalidate the keys
on platforms without a file.truncate(), when the fcntl() lock is
momentarily released afterwards.  It seems hard to avoid this as,
perversely, fcntl() locks are supposed to be released automatically on
all file descriptors referring to the file whenever the process closes
any one of them - even one the lock was never set on.

So, code using mailbox.py such platforms could inadvertently be
carrying keys across an unlocked period, which is not made safe by the
update-toc patch (as it's only meant to avert disasters resulting from
doing this *and* rebuilding the table of contents, *assuming* that
another process hasn't deleted or rearranged messages).

File Added: mailbox-update-toc-fixed.diff

----------------------------------------------------------------------

Comment By: A.M. Kuchling (akuchling)
Date: 2007-01-05 19:51

Message:
Logged In: YES 
user_id=11375
Originator: NO

Question about mailbox-update-doc: the add() method still returns
self._next_key - 1; should this be 
self._next_user_key - 1?  The keys in _user_toc are the ones returned to
external users of the mailbox, right?

(A good test case would be to initialize _next_key to 0 and _next_user_key
to a different value like 123456.)

I'm still staring at the patch, trying to convince myself that it will
help -- haven't spotted any problems, but this bug is making me nervous...


----------------------------------------------------------------------

Comment By: A.M. Kuchling (akuchling)
Date: 2007-01-05 19:24

Message:
Logged In: YES 
user_id=11375
Originator: NO

As a step toward improving matters, I've attached the suggested doc patch
(for both 25-maint and trunk).  It encourages people to use Maildir :),
explicitly states that modifications should be bracketed by lock(), and
fixes the examples to match.

It does not say that keys are invalidated by doing a flush(), because
we're going to try to avoid the necessity for that.


File Added: mailbox-docs.diff

----------------------------------------------------------------------

Comment By: A.M. Kuchling (akuchling)
Date: 2006-12-20 19:48

Message:
Logged In: YES 
user_id=11375
Originator: NO

Committed length-checking.diff to trunk in rev. 53110.


----------------------------------------------------------------------

Comment By: David Watson (baikie)
Date: 2006-12-20 19:19

Message:
Logged In: YES 
user_id=1504904
Originator: YES

File Added: mailbox-test-lock.diff

----------------------------------------------------------------------

Comment By: David Watson (baikie)
Date: 2006-12-20 19:17

Message:
Logged In: YES 
user_id=1504904
Originator: YES

Yeah, I think that should definitely go in.  ExternalClashError or a
subclass sounds fine to me (although you could make a whole taxonomy
of these errors, really).  It would be good to have the code actually
keep up with other programs' changes, though; a program might just
want to count the messages at first, say, and not make changes until
much later.

I've been trying out the second option (patch attached, to apply on
top of mailbox-copy-back), regenerating _toc on locking, but
preserving existing keys.  The patch allows existing _generate_toc()s
to work unmodified, but means that _toc now holds the entire last
known contents of the mailbox file, with the 'pending' (user-visible)
mailbox state being held in a new attribute, _user_toc, which is a
mapping from keys issued to the program to the keys of _toc
(i.e. sequence numbers in the file).  When _toc is updated, any new
messages that have appeared are given keys in _user_toc that haven't
been issued before, and any messages that have disappeared are removed
from it.  The code basically assumes that messages with the same
sequence number are the same message, though, so even if most cases
are caught by the length check, programs that make
deletions/replacements before locking could still delete the wrong
messages.  This behaviour could be trapped, though, by raising an
exception in lock() if self._pending is set (after all, code like that
would be incorrect unless it could be assumed that the mailbox module
kept hashes of each message or something).

Also attached is a patch to the test case, adding a lock/unlock around
the message count to make sure _toc is up-to-date if the parent
process finishes first; without it, there are still intermittent
failures.

File Added: mailbox-update-toc.diff

----------------------------------------------------------------------

Comment By: A.M. Kuchling (akuchling)
Date: 2006-12-20 14:46

Message:
Logged In: YES 
user_id=11375
Originator: NO

Attaching a patch that adds length checking: before doing a flush() on a
single-file mailbox, seek to the end and verify its length is unchanged. 
It raises an ExternalClashError if the file's length has changed.  (Should
there be a different exception for this case, perhaps a subclass of
ExternalClashError?)

I verified that this change works by running a program that added 25
messages, pausing between each one, and then did 'echo "new line" >
/tmp/mbox' from a shell while the program was running.

I also noticed that the self._lookup() call in self.flush() wasn't
necessary, and replaced it by an assertion.

I think this change should go on both the trunk and 25-maint branches.

File Added: length-checking.diff

----------------------------------------------------------------------

Comment By: A.M. Kuchling (akuchling)
Date: 2006-12-18 17:43

Message:
Logged In: YES 
user_id=11375
Originator: NO

Eep, correct; changing the key IDs would be a big problem for existing
code.  We could say 'discard all keys' after doing lock() or unlock(), but
this is an API change that means the fix couldn't be backported to
2.5-maint.

We could make generating the ToC more complicated, preserving key IDs when
possible; that may not be too difficult, though the code might be messy.

Maybe it's best to just catch this error condition: save the size of the
mailbox, updating it in _append_message(), and then make .flush() raise an
exception if the mailbox size has unexpectedly changed.


----------------------------------------------------------------------

Comment By: David Watson (baikie)
Date: 2006-12-16 19:09

Message:
Logged In: YES 
user_id=1504904
Originator: YES

Yes, I see what you mean.  I had tried multiple flushes, but only
inside a single lock/unlock.  But this means that in the no-truncate()
code path, even this is currently unsafe, as the lock is momentarily
released after flushing.

I think _toc should be regenerated after every lock(), as with the
risk of another process replacing/deleting/rearranging the messages,
it isn't valid to carry sequence numbers from one locked period to
another anyway, or from unlocked to locked.  However, this runs the
risk of dangerously breaking code that thinks it is valid to do so,
even in the case where the mailbox was *not* modified (i.e. turning
possible failure into certain failure).  For instance, if the program
removes message 1, then as things stand, the key "1" is no longer
used, and removing message 2 will remove the message that followed 1.
If _toc is regenerated in between, however (using the current code, so
that the messages are renumbered from 0), then the old message 2
becomes message 1, and removing message 2 will therefore remove the
wrong message.  You'd also have things like pending deletions and
replacements (also unsafe generally) being forgotten.  So it would
take some work to get right, if it's to work at all...


----------------------------------------------------------------------

Comment By: A.M. Kuchling (akuchling)
Date: 2006-12-15 14:06

Message:
Logged In: YES 
user_id=11375
Originator: NO

I'm testing the fix using two Python processes running mailbox.py, and my
test case fails even with your patch.  This is due to another bug, even in
the patched version.  

mbox has a dictionary attribute, _toc, mapping message keys to positions
in the file.  flush() writes out all the messages in self._toc and
constructs a new _toc with the new file offsets.  It doesn't re-read the
file to see if new messages were added by another process.

One fix that seems to work: instead of doing 'self._toc = new_toc' after
flush() has done its work, do self._toc = None.  The ToC will be
regenerated the next time _lookup() is called, causing a re-read of all the
contents of the mbox.  Inefficient, but I see no way around the necessity
for doing this.

It's not clear to me that my suggested fix is enough, though.  Process #1
opens a mailbox, reads the ToC, and the process does something else for 5
minutes.  In the meantime, process #2 adds a file to the mbox.  Process #1
then adds a message to the mbox and writes it out; it never notices process
#2's change.

Maybe the _toc has to be regenerated every time you call lock(), because
at this point you know there will be no further updates to the mbox by any
other process.  Any unlocked usage of _toc should also really be
regenerating _toc every time, because you never know if another process has
added a message... but that would be really inefficient.







----------------------------------------------------------------------

Comment By: A.M. Kuchling (akuchling)
Date: 2006-12-15 13:17

Message:
Logged In: YES 
user_id=11375
Originator: NO

The attached patch adds a test case to test_mailbox.py that demonstrates
the problem.  No modifications to mailbox.py are needed to show data loss.

Now looking at the patch...

File Added: mailbox-test.patch

----------------------------------------------------------------------

Comment By: A.M. Kuchling (akuchling)
Date: 2006-12-12 21:04

Message:
Logged In: YES 
user_id=11375
Originator: NO

I agree with David's analysis; this is in fact a bug.  I'll try to look at
the patch.

----------------------------------------------------------------------

Comment By: David Watson (baikie)
Date: 2006-11-19 20:44

Message:
Logged In: YES 
user_id=1504904
Originator: YES

This is a bug.  The point is that the code is subverting the protection of
its own fcntl locking.  I should have pointed out that Postfix was still
using fcntl locking, and that should have been sufficient.  (In fact, it
was due to its use of fcntl locking that it chose  precisely the wrong
moment to deliver mail.)  Dot-locking does protect against this, but not
every program uses it - which is precisely the reason that the code
implements fcntl locking in the first place.


----------------------------------------------------------------------

Comment By: Martin v. L?wis (loewis)
Date: 2006-11-19 20:02

Message:
Logged In: YES 
user_id=21627
Originator: NO

Mailbox locking was invented precisely to support this kind of operation.
Why do you complain that things break if you deliberately turn off the
mechanism preventing breakage?

I fail to see a bug here.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1599254&group_id=5470

From noreply at sourceforge.net  Fri Apr 13 15:47:46 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Fri, 13 Apr 2007 06:47:46 -0700
Subject: [ python-Bugs-1599254 ] mailbox: other programs' messages can vanish
	without trace
Message-ID: <E1HcM7u-0006pr-VP@sc8-sf-web21.sourceforge.net>

Bugs item #1599254, was opened at 2006-11-19 16:03
Message generated for change (Comment added) made by baikie
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1599254&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.5
Status: Open
Resolution: None
Priority: 7
Private: No
Submitted By: David Watson (baikie)
Assigned to: A.M. Kuchling (akuchling)
Summary: mailbox: other programs' messages can vanish without trace

Initial Comment:
The mailbox classes based on _singlefileMailbox (mbox, MMDF, Babyl) implement the flush() method by writing the new mailbox contents into a temporary file which is then renamed over the original. Unfortunately, if another program tries to deliver messages while mailbox.py is working, and uses only fcntl() locking, it will have the old file open and be blocked waiting for the lock to become available. Once mailbox.py has replaced the old file and closed it, making the lock available, the other program will write its messages into the now-deleted "old" file, consigning them to oblivion.

I've caused Postfix on Linux to lose mail this way (although I did have to turn off its use of dot-locking to do so).

A possible fix is attached.  Instead of new_file being renamed, its contents are copied back to the original file.  If file.truncate() is available, the mailbox is then truncated to size.  Otherwise, if truncation is required, it's truncated to zero length beforehand by reopening self._path with mode wb+.  In the latter case, there's a check to see if the mailbox was replaced while we weren't looking, but there's still a race condition.  Any alternative ideas?

Incidentally, this fixes a problem whereby Postfix wouldn't deliver to the replacement file as it had the execute bit set.


----------------------------------------------------------------------

>Comment By: David Watson (baikie)
Date: 2007-04-13 14:47

Message:
Logged In: YES 
user_id=1504904
Originator: YES

This fixes the Babyl breakage.  Perhaps it should be in the superclass?

File Added: mailbox-babyl-fix.diff

----------------------------------------------------------------------

Comment By: David Watson (baikie)
Date: 2007-04-13 14:46

Message:
Logged In: YES 
user_id=1504904
Originator: YES

Some new test cases for this stuff.

File Added: test_mailbox-reread.diff

----------------------------------------------------------------------

Comment By: David Watson (baikie)
Date: 2007-04-13 14:45

Message:
Logged In: YES 
user_id=1504904
Originator: YES

Here's a possible solution to the rereading problem.  It should allow
existing applications to work whether they behave properly or not, as
well as (probably) most third-party subclasses.

The idea is to compare the new table of contents to the list of known
offset pairs, and raise ExternalClashError if *any* of them have
changed or disappeared.  Any new pairs can then be added to _toc under
new keys.  To maintain the list of known pairs, a special dict
subclass is used on self._toc that records every offset pair ever set
in it - even those that are subsequently removed from the mapping.
However, if self._pending is not set when rereading, then the code
doesn't rely on this special behaviour: it just uses
self._toc.itervalues(), which will work even if a subclass has
replaced the special _toc with a normal dictionary.

Ways the code can break:

- If a subclass replaces self._toc and then the application tries to
lock the mailbox *after* making modifications (so that _update_toc()
finds self._pending set, and looks for the special attribute on _toc).

- If a subclass tries to store something other than sequences in _toc.

- If a subclass' _generate_toc() can produce offsets for messages that
don't match those they were written under.

File Added: mailbox-update-toc-again.diff

----------------------------------------------------------------------

Comment By: A.M. Kuchling (akuchling)
Date: 2007-03-28 17:56

Message:
Logged In: YES 
user_id=11375
Originator: NO

Created a branch in SVN at
svn+ssh://pythondev at svn.python.org/python/branches/amk-mailbox to work on
this for 2.5.1.  I've committed the unified2 module patch and the
test_concurrent_ad() test.


----------------------------------------------------------------------

Comment By: A.M. Kuchling (akuchling)
Date: 2007-01-24 20:48

Message:
Logged In: YES 
user_id=11375
Originator: NO

I've strengthened the warning again.

The MH bug in unified2 is straightforward: MH.remove() opens a file
object, locks it, closes the file object, and then tries to unlock it. 
Presumably the MH test case never bothered locking the mailbox before
making changes before.


----------------------------------------------------------------------

Comment By: David Watson (baikie)
Date: 2007-01-22 20:24

Message:
Logged In: YES 
user_id=1504904
Originator: YES

So what you propose to commit for 2.5 is basically mailbox-unified2
(your mailbox-unified-patch, minus the _toc clearing)?


----------------------------------------------------------------------

Comment By: A.M. Kuchling (akuchling)
Date: 2007-01-22 15:46

Message:
Logged In: YES 
user_id=11375
Originator: NO

This would be an API change, and therefore out-of-bounds for 2.5.

I suggest giving up on this for 2.5.1 and only fixing it in 2.6.   I'll
add another warning to the docs, and maybe to the module as well.



----------------------------------------------------------------------

Comment By: David Watson (baikie)
Date: 2007-01-21 22:10

Message:
Logged In: YES 
user_id=1504904
Originator: YES

Hold on, I have a plan.  If _toc is only regenerated on locking, or at
the end of a flush(), then the only way self._pending can be set at
that time is if the application has made modifications before calling
lock().  If we make that an exception-raising offence, then we can
assume that self._toc is a faithful representation of the last known
contents of the file.  That means we can preserve the existing message
keys on a reread without any of that _user_toc nonsense.

Diff attached, to apply on top of mailbox-unified2.  It's probably had
even less review and testing than the previous version, but it appears
to pass all the regression tests and doesn't change any existing
semantics.

File Added: mailbox-update-toc-new.diff

----------------------------------------------------------------------

Comment By: A.M. Kuchling (akuchling)
Date: 2007-01-21 03:16

Message:
Logged In: YES 
user_id=11375
Originator: NO

<sigh>  I'm starting to lose track of all the variations on the bug. 
Maybe we should just add more warnings to the documentation about locking
the mailbox when modifying it and not try to fix this at all.


----------------------------------------------------------------------

Comment By: David Watson (baikie)
Date: 2007-01-20 18:20

Message:
Logged In: YES 
user_id=1504904
Originator: YES

Hang on.  If a message's key changes after recreating _toc, that does
not mean that another process has modified the mailbox.  If the
application removes a message and then (inadvertently) causes _toc to
be regenerated, the keys of all subsequent messages will be
decremented by one, due only to the application's own actions.

That's what happens in the "broken locking" test case: the program
intends to remove message 0, flush, and then remove message 1, but
because _toc is regenerated in between, message 1 is renumbered as 0,
message 2 is renumbered as 1, and so the program deletes message 2
instead.  To clear _toc in such code without attempting to preserve
the message keys turns possible data loss (in the case that another
process modified the mailbox) into certain data loss.  That's what I'm
concerned about.


----------------------------------------------------------------------

Comment By: A.M. Kuchling (akuchling)
Date: 2007-01-19 15:24

Message:
Logged In: YES 
user_id=11375
Originator: NO

After reflection, I don't think the potential changing actually makes
things any worse.  _generate() always starts numbering keys with 1, so if a
message's key changes because of lock()'s, re-reading, that means someone
else has already modified the mailbox.  Without the ToC clearing, you're
already fated to have a corrupted mailbox because the new mailbox will be
written using outdated file offsets.  With the ToC clearing, you delete the
wrong message.  Neither outcome is good, but data is lost either way.  

The new behaviour is maybe a little bit better in that you're losing a
single message but still generating a well-formed mailbox, and not a
randomly jumbled mailbox.

I suggest applying the patch to clear self._toc, and noting in the
documentation that keys might possibly change after doing a lock().


----------------------------------------------------------------------

Comment By: David Watson (baikie)
Date: 2007-01-18 18:15

Message:
Logged In: YES 
user_id=1504904
Originator: YES

This version passes the new tests (it fixes the length checking bug,
and no longer clears self._toc), but goes back to failing
test_concurrent_add.

File Added: mailbox-unified2-module.diff

----------------------------------------------------------------------

Comment By: David Watson (baikie)
Date: 2007-01-18 18:14

Message:
Logged In: YES 
user_id=1504904
Originator: YES

Unfortunately, there is a problem with clearing _toc: it's basically
the one I alluded to in my comment of 2006-12-16.  Back then I thought
it could be caught by the test you issue the warning for, but the
application may instead do its second remove() *after* the lock() (so
that self._pending is not set at lock() time), using the key carried
over from before it called unlock().  As before, this would result in
the wrong message being deleted.

I've added a test case for this (diff attached), and a bug I found in
the process whereby flush() wasn't updating the length, which could
cause subsequent flushes to fail (I've got a fix for this).  These
seem to have turned up a third bug in the MH class, but I haven't
looked at that yet.

File Added: mailbox-unified2-test.diff

----------------------------------------------------------------------

Comment By: A.M. Kuchling (akuchling)
Date: 2007-01-17 21:06

Message:
Logged In: YES 
user_id=11375
Originator: NO

Add mailbox-unified-patch.

File Added: mailbox-unified-patch.diff

----------------------------------------------------------------------

Comment By: A.M. Kuchling (akuchling)
Date: 2007-01-17 21:05

Message:
Logged In: YES 
user_id=11375
Originator: NO

mailbox-pending-lock is the difference between David's copy-back-new +
fcntl-warning and my -unified patch, uploaded so that he can comment on the
changes.

The only change is to make _singleFileMailbox.lock() clear self._toc,
forcing a re-read of the mailbox file.  If _pending is true, the ToC isn't
cleared and a warning is logged.  I think this lets existing code run
(albeit possibly with a warning if the mailbox is modified before .lock()
is called), but fixes the risk of missing changes written by another
process.

Triggering a new warning is sort of an API change, but IMHO it's still
worth backporting; code that triggers this warning is at risk of losing
messages or corrupting the mailbox.

Clearing the _toc on .lock() is also sort of an API change, but it's
necessary to avoid the potential data loss.  It may lead to some redundant
scanning of mailbox files, but this price is worth paying, I think; people
probably don't have 2Gb mbox files (I hope not, anyway!) and no extra read
is done if you create the mailbox and immediately lock it before looking
anything up.

Neal: if you want to discuss this patch or want an explanation of
something, feel free to chat with me about it.

I'll wait a day or two and see if David spots any problems.  If nothing
turns up, I'll commit it to both trunk and release25-maint.



File Added: mailbox-pending-lock.diff

----------------------------------------------------------------------

Comment By: A.M. Kuchling (akuchling)
Date: 2007-01-17 20:53

Message:
Logged In: YES 
user_id=11375
Originator: NO

mailbox-unified-patch contains David's copy-back-new and fcntl-warn
patches, plus the test-mailbox patch and some additional changes to
mailbox.py from me.  (I'll upload a diff to David's diffs in a minute.)

This is the change I'd like to check in.  test_mailbox.py now passes, as
does the mailbox-break.py script I'm using.


----------------------------------------------------------------------

Comment By: A.M. Kuchling (akuchling)
Date: 2007-01-17 19:56

Message:
Logged In: YES 
user_id=11375
Originator: NO

Committed a modified version of the doc. patch in rev. 53472 (trunk) and
rev. 53474 (release25-maint).


----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2007-01-17 06:48

Message:
Logged In: YES 
user_id=33168
Originator: NO

Andrew, do you need any help with this?

----------------------------------------------------------------------

Comment By: A.M. Kuchling (akuchling)
Date: 2007-01-15 19:01

Message:
Logged In: YES 
user_id=11375
Originator: NO

Comment from Andrew MacIntyre (os2vacpp is the OS/2 that lacks
ftruncate()):

================

I actively support the OS/2 EMX port (sys.platform == "os2emx"; build
directory is PC/os2emx).  I would like to keep the VAC++ port alive, but
the reality is I don't have the resources to do so.  The VAC++ port was
the subject of discussion about removal of build support support from the
source tree for 2.6 - I don't recall there being a definitive outcome,
but if someone does delete the PC/os2vacpp directory, I'm not in a
position to argue.

AMK: (mailbox.py has a separate section of code used when file.truncate()
isn't available, and the existence of this section is bedevilling me.
It would be convenient if platforms without file.truncate() weren't a
factor; then this branch could just be removed.  In your opinion,
would it hurt OS/2 users of mailbox.py if support for platforms
without file.truncate() was removed?)

aimacintyre: No.  From what documentation I can quickly check, ftruncate()
operates
on file descriptors rather than FILE pointers.  As such I am sure that,
if it became an issue, it would not be difficult to write a ftruncate()
emulation wrapper for the underlying OS/2 APIs that implement the
required functionality.



----------------------------------------------------------------------

Comment By: David Watson (baikie)
Date: 2007-01-13 18:32

Message:
Logged In: YES 
user_id=1504904
Originator: YES

I like the warning idea - it seems appropriate if the problem is
relatively rare.  How about this?

File Added: mailbox-fcntl-warn.diff

----------------------------------------------------------------------

Comment By: A.M. Kuchling (akuchling)
Date: 2007-01-12 19:41

Message:
Logged In: YES 
user_id=11375
Originator: NO

One OS/2 port lacks truncate(), and so does RISCOS.

----------------------------------------------------------------------

Comment By: A.M. Kuchling (akuchling)
Date: 2007-01-12 18:41

Message:
Logged In: YES 
user_id=11375
Originator: NO

I realized that making flush() invalidate keys breaks the final example in
the docs, which loops over inbox.iterkeys() and removes messages, doing a
pack() after each message.

Which platforms lack file.truncate()?  Windows has it; POSIX has it, so
modern Unix variants should all have it.  Maybe mailbox should simply raise
an exception (or trigger a warning?) if truncate is missing, and we should
then assume that flush() has no effect upon keys.


----------------------------------------------------------------------

Comment By: A.M. Kuchling (akuchling)
Date: 2007-01-12 17:12

Message:
Logged In: YES 
user_id=11375
Originator: NO

So shall we document flush() as invalidating keys, then?

----------------------------------------------------------------------

Comment By: David Watson (baikie)
Date: 2007-01-06 19:57

Message:
Logged In: YES 
user_id=1504904
Originator: YES

Oops, length checking had made the first two lines of this patch
redundant; update-toc applies OK with fuzz.

File Added: mailbox-copy-back-new.diff

----------------------------------------------------------------------

Comment By: David Watson (baikie)
Date: 2007-01-06 15:30

Message:
Logged In: YES 
user_id=1504904
Originator: YES

File Added: mailbox-copy-back-53287.diff

----------------------------------------------------------------------

Comment By: David Watson (baikie)
Date: 2007-01-06 15:24

Message:
Logged In: YES 
user_id=1504904
Originator: YES

Aack, yes, that should be _next_user_key.  Attaching a fixed version.

I've been thinking, though: flush() does in fact invalidate the keys
on platforms without a file.truncate(), when the fcntl() lock is
momentarily released afterwards.  It seems hard to avoid this as,
perversely, fcntl() locks are supposed to be released automatically on
all file descriptors referring to the file whenever the process closes
any one of them - even one the lock was never set on.

So, code using mailbox.py such platforms could inadvertently be
carrying keys across an unlocked period, which is not made safe by the
update-toc patch (as it's only meant to avert disasters resulting from
doing this *and* rebuilding the table of contents, *assuming* that
another process hasn't deleted or rearranged messages).

File Added: mailbox-update-toc-fixed.diff

----------------------------------------------------------------------

Comment By: A.M. Kuchling (akuchling)
Date: 2007-01-05 19:51

Message:
Logged In: YES 
user_id=11375
Originator: NO

Question about mailbox-update-doc: the add() method still returns
self._next_key - 1; should this be 
self._next_user_key - 1?  The keys in _user_toc are the ones returned to
external users of the mailbox, right?

(A good test case would be to initialize _next_key to 0 and _next_user_key
to a different value like 123456.)

I'm still staring at the patch, trying to convince myself that it will
help -- haven't spotted any problems, but this bug is making me nervous...


----------------------------------------------------------------------

Comment By: A.M. Kuchling (akuchling)
Date: 2007-01-05 19:24

Message:
Logged In: YES 
user_id=11375
Originator: NO

As a step toward improving matters, I've attached the suggested doc patch
(for both 25-maint and trunk).  It encourages people to use Maildir :),
explicitly states that modifications should be bracketed by lock(), and
fixes the examples to match.

It does not say that keys are invalidated by doing a flush(), because
we're going to try to avoid the necessity for that.


File Added: mailbox-docs.diff

----------------------------------------------------------------------

Comment By: A.M. Kuchling (akuchling)
Date: 2006-12-20 19:48

Message:
Logged In: YES 
user_id=11375
Originator: NO

Committed length-checking.diff to trunk in rev. 53110.


----------------------------------------------------------------------

Comment By: David Watson (baikie)
Date: 2006-12-20 19:19

Message:
Logged In: YES 
user_id=1504904
Originator: YES

File Added: mailbox-test-lock.diff

----------------------------------------------------------------------

Comment By: David Watson (baikie)
Date: 2006-12-20 19:17

Message:
Logged In: YES 
user_id=1504904
Originator: YES

Yeah, I think that should definitely go in.  ExternalClashError or a
subclass sounds fine to me (although you could make a whole taxonomy
of these errors, really).  It would be good to have the code actually
keep up with other programs' changes, though; a program might just
want to count the messages at first, say, and not make changes until
much later.

I've been trying out the second option (patch attached, to apply on
top of mailbox-copy-back), regenerating _toc on locking, but
preserving existing keys.  The patch allows existing _generate_toc()s
to work unmodified, but means that _toc now holds the entire last
known contents of the mailbox file, with the 'pending' (user-visible)
mailbox state being held in a new attribute, _user_toc, which is a
mapping from keys issued to the program to the keys of _toc
(i.e. sequence numbers in the file).  When _toc is updated, any new
messages that have appeared are given keys in _user_toc that haven't
been issued before, and any messages that have disappeared are removed
from it.  The code basically assumes that messages with the same
sequence number are the same message, though, so even if most cases
are caught by the length check, programs that make
deletions/replacements before locking could still delete the wrong
messages.  This behaviour could be trapped, though, by raising an
exception in lock() if self._pending is set (after all, code like that
would be incorrect unless it could be assumed that the mailbox module
kept hashes of each message or something).

Also attached is a patch to the test case, adding a lock/unlock around
the message count to make sure _toc is up-to-date if the parent
process finishes first; without it, there are still intermittent
failures.

File Added: mailbox-update-toc.diff

----------------------------------------------------------------------

Comment By: A.M. Kuchling (akuchling)
Date: 2006-12-20 14:46

Message:
Logged In: YES 
user_id=11375
Originator: NO

Attaching a patch that adds length checking: before doing a flush() on a
single-file mailbox, seek to the end and verify its length is unchanged. 
It raises an ExternalClashError if the file's length has changed.  (Should
there be a different exception for this case, perhaps a subclass of
ExternalClashError?)

I verified that this change works by running a program that added 25
messages, pausing between each one, and then did 'echo "new line" >
/tmp/mbox' from a shell while the program was running.

I also noticed that the self._lookup() call in self.flush() wasn't
necessary, and replaced it by an assertion.

I think this change should go on both the trunk and 25-maint branches.

File Added: length-checking.diff

----------------------------------------------------------------------

Comment By: A.M. Kuchling (akuchling)
Date: 2006-12-18 17:43

Message:
Logged In: YES 
user_id=11375
Originator: NO

Eep, correct; changing the key IDs would be a big problem for existing
code.  We could say 'discard all keys' after doing lock() or unlock(), but
this is an API change that means the fix couldn't be backported to
2.5-maint.

We could make generating the ToC more complicated, preserving key IDs when
possible; that may not be too difficult, though the code might be messy.

Maybe it's best to just catch this error condition: save the size of the
mailbox, updating it in _append_message(), and then make .flush() raise an
exception if the mailbox size has unexpectedly changed.


----------------------------------------------------------------------

Comment By: David Watson (baikie)
Date: 2006-12-16 19:09

Message:
Logged In: YES 
user_id=1504904
Originator: YES

Yes, I see what you mean.  I had tried multiple flushes, but only
inside a single lock/unlock.  But this means that in the no-truncate()
code path, even this is currently unsafe, as the lock is momentarily
released after flushing.

I think _toc should be regenerated after every lock(), as with the
risk of another process replacing/deleting/rearranging the messages,
it isn't valid to carry sequence numbers from one locked period to
another anyway, or from unlocked to locked.  However, this runs the
risk of dangerously breaking code that thinks it is valid to do so,
even in the case where the mailbox was *not* modified (i.e. turning
possible failure into certain failure).  For instance, if the program
removes message 1, then as things stand, the key "1" is no longer
used, and removing message 2 will remove the message that followed 1.
If _toc is regenerated in between, however (using the current code, so
that the messages are renumbered from 0), then the old message 2
becomes message 1, and removing message 2 will therefore remove the
wrong message.  You'd also have things like pending deletions and
replacements (also unsafe generally) being forgotten.  So it would
take some work to get right, if it's to work at all...


----------------------------------------------------------------------

Comment By: A.M. Kuchling (akuchling)
Date: 2006-12-15 14:06

Message:
Logged In: YES 
user_id=11375
Originator: NO

I'm testing the fix using two Python processes running mailbox.py, and my
test case fails even with your patch.  This is due to another bug, even in
the patched version.  

mbox has a dictionary attribute, _toc, mapping message keys to positions
in the file.  flush() writes out all the messages in self._toc and
constructs a new _toc with the new file offsets.  It doesn't re-read the
file to see if new messages were added by another process.

One fix that seems to work: instead of doing 'self._toc = new_toc' after
flush() has done its work, do self._toc = None.  The ToC will be
regenerated the next time _lookup() is called, causing a re-read of all the
contents of the mbox.  Inefficient, but I see no way around the necessity
for doing this.

It's not clear to me that my suggested fix is enough, though.  Process #1
opens a mailbox, reads the ToC, and the process does something else for 5
minutes.  In the meantime, process #2 adds a file to the mbox.  Process #1
then adds a message to the mbox and writes it out; it never notices process
#2's change.

Maybe the _toc has to be regenerated every time you call lock(), because
at this point you know there will be no further updates to the mbox by any
other process.  Any unlocked usage of _toc should also really be
regenerating _toc every time, because you never know if another process has
added a message... but that would be really inefficient.







----------------------------------------------------------------------

Comment By: A.M. Kuchling (akuchling)
Date: 2006-12-15 13:17

Message:
Logged In: YES 
user_id=11375
Originator: NO

The attached patch adds a test case to test_mailbox.py that demonstrates
the problem.  No modifications to mailbox.py are needed to show data loss.

Now looking at the patch...

File Added: mailbox-test.patch

----------------------------------------------------------------------

Comment By: A.M. Kuchling (akuchling)
Date: 2006-12-12 21:04

Message:
Logged In: YES 
user_id=11375
Originator: NO

I agree with David's analysis; this is in fact a bug.  I'll try to look at
the patch.

----------------------------------------------------------------------

Comment By: David Watson (baikie)
Date: 2006-11-19 20:44

Message:
Logged In: YES 
user_id=1504904
Originator: YES

This is a bug.  The point is that the code is subverting the protection of
its own fcntl locking.  I should have pointed out that Postfix was still
using fcntl locking, and that should have been sufficient.  (In fact, it
was due to its use of fcntl locking that it chose  precisely the wrong
moment to deliver mail.)  Dot-locking does protect against this, but not
every program uses it - which is precisely the reason that the code
implements fcntl locking in the first place.


----------------------------------------------------------------------

Comment By: Martin v. L?wis (loewis)
Date: 2006-11-19 20:02

Message:
Logged In: YES 
user_id=21627
Originator: NO

Mailbox locking was invented precisely to support this kind of operation.
Why do you complain that things break if you deliberately turn off the
mechanism preventing breakage?

I fail to see a bug here.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1599254&group_id=5470

From noreply at sourceforge.net  Fri Apr 13 17:06:26 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Fri, 13 Apr 2007 08:06:26 -0700
Subject: [ python-Bugs-1700132 ] import and capital letters
Message-ID: <E1HcNM2-0000L4-N2@sc8-sf-web24.sourceforge.net>

Bugs item #1700132, was opened at 2007-04-13 15:06
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1700132&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Interpreter Core
Group: Python 2.4
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: omsynge (omsynge)
Assigned to: Nobody/Anonymous (nobody)
Summary: import and capital letters

Initial Comment:
Interactive and relative paths allow unlimited
(Or at least I have not found a limit yet) number of characters to be uppercase. This is fine, but when the python interpreter reads directories from PYTHONPATH with 3 capital letters you then get a failure to locate the files.

I have replicated this issue with python 2.2 and python 2.4. These where with red hat el3 and ubuntu (Some version not sure which)

and an example

import dcacheYaimInstallerTest.logger as logger

works fine in interactive or relative paths, but not when installed via an RPM.


import dcacheYaimInstallertest.logger as logger

is just fine in both scenarios. 

This bug cost me some hours to trace and would have cost more had I not had a lot of experience of packaging, so I would be pleased if this could be fixed in all versions of Python.

Regards

Owen








----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1700132&group_id=5470

From noreply at sourceforge.net  Fri Apr 13 18:45:41 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Fri, 13 Apr 2007 09:45:41 -0700
Subject: [ python-Bugs-1685000 ] asyncore DoS vulnerability
Message-ID: <E1HcOu5-00060G-5m@sc8-sf-web24.sourceforge.net>

Bugs item #1685000, was opened at 2007-03-21 10:15
Message generated for change (Settings changed) made by billiejoex
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1685000&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.5
Status: Open
Resolution: None
Priority: 9
Private: No
Submitted By: billiejoex (billiejoex)
>Assigned to: Josiah Carlson (josiahcarlson)
Summary: asyncore DoS vulnerability

Initial Comment:
DoS asyncore vulnerability

asyncore, independently if used with select() or poll(), suffers a DoS-type vulnerability when a high number of simultaneous connections to handle simultaneously is reached.
The number of maximum connections is system-dependent as well as the type of error raised.
I attached two simple Proof of Concept scripts demonstrating such bug.
If you want to try the behaviours listed below run the attached "asyncore_server.py" and "asyncore_client.py" scripts on your local workstation.

On my Windows XP system (Python 2.5), independently if asyncore has been used to develop a server or a client, the error is raised by select() inside asyncore's "poll" function when 512 (socket_map's elements) simultaneous connections are reached. 
Here's the traceback I get:

[...]
connections: 510
connections: 511
connections: 512
Traceback (most recent call last):
  File "C:\scripts\asyncore_server.py", line 38, in <module>
    asyncore.loop()
  File "C:\Python25\lib\asyncore.py", line 191, in loop
    poll_fun(timeout, map)
  File "C:\Python25\lib\asyncore.py", line 121, in poll
    r, w, e = select.select(r, w, e, timeout)
ValueError: too many file descriptors in select()


On my Linux Ubuntu 6.10 (kernel 2.6.17-10, Python 2.5) different type of errors are raised depending on the application (client or server).
In an asyncore-based client the error is raised by socket module (dispatcher's "self.socket" attribute) inside 'connect' method of 'dispatcher' class:

[...]
connections: 1018
connections: 1019
connections: 1020
connections: 1021
Traceback (most recent call last):
  File "asyncore_client.py", line 31, in <module>
  File "asyncore.py", line 191, in loop
  File "asyncore.py", line 138, in poll
  File "asyncore.py", line 80, in write
  File "asyncore.py", line 76, in write
  File "asyncore.py", line 395, in handle_write_event
  File "asyncore_client.py", line 24, in handle_connect
  File "asyncore_client.py", line 9, in __init__
  File "asyncore.py", line 257, in create_socket
  File "socket.py", line 156, in __init__
socket.error: (24, 'Too many open files')


On an asyncore-based server the error is raised by socket module (dispatcher's "self.socket" attribute) inside 'accept' method of 'dispatcher' class:

[...]
connections: 1019
connections: 1020
connections: 1021
Traceback (most recent call last):
  File "asyncore_server.py", line 38, in <module>
  File "asyncore.py", line 191, in loop
  File "asyncore.py", line 132, in poll
  File "asyncore.py", line 72, in read
  File "asyncore.py", line 68, in read
  File "asyncore.py", line 384, in handle_read_event
  File "asyncore_server.py", line 16, in handle_accept
  File "asyncore.py", line 321, in accept
  File "socket.py", line 170, in accept
socket.error: (24, 'Too many open files')


----------------------------------------------------------------------

Comment By: Josiah Carlson (josiahcarlson)
Date: 2007-04-09 18:13

Message:
Logged In: YES 
user_id=341410
Originator: NO

Assign the "bug" to me, I'm the maintainer for asyncore/asynchat.

With that said, since a user needs to override
asyncore.dispatcher.handle_accept() anyways, which necessarily needs to
call asyncore.dispatcher.accept(), the subclass is free to check the number
of sockets in its socket map before creating a new instance of whatever
subclass of asyncore.dispatcher the user has written.

Also, the number of file handles that select can handle on Windows is a
compile-time constant, and has nothing to do with the actual number of open
file handles.  Take and run the following source file on Windows and see
how the total number of open sockets can be significantly larger than the
number of sockets passed to select():

import socket
import asyncore
import random

class new_map(dict):
    def items(self):
        r = [(i,j) for i,j in dict.items(self) if not random.randrange(4)
and j != h]
        r.append((h._fileno, h))
        print len(r), len(asyncore.socket_map)
        return r

asyncore.socket_map = new_map()

class listener(asyncore.dispatcher):
    def handle_accept(self):
        x = self.accept()
        if x:
            conn, addr = x
            connection(conn)

class connection(asyncore.dispatcher):
    def writable(self):
        0
    def handle_connect(self):
        pass

if __name__ == '__main__':
    h = listener()
    h.create_socket(socket.AF_INET, socket.SOCK_STREAM)
    h.bind(('127.0.0.1', 10001))
    h.listen(5)
    while len(asyncore.socket_map) < 4096:
        a = connection()
        a.create_socket(socket.AF_INET, socket.SOCK_STREAM)
        a.connect_ex(('127.0.0.1', 10001))
        asyncore.poll()

The tail end of a run in Windows:

476 1934
501 1936
516 1938
Traceback (most recent call last):
  File "D:\MYDOCS\Projects\python\async_socket.py", line 37, in ?
    asyncore.poll()
  File "C:\python23\lib\asyncore.py", line 108, in poll
    r, w, e = select.select(r, w, e, timeout)
ValueError: too many file descriptors in select()

With a proper definition of new_map, you can handle an unlimited number of
sockets with asyncore by choosing blocks of sockets to return in its
items() call.  Note that I used random out of convenience, a proper
implementation could distribute the sockets based on fileno, time inserted,
etc.  You can do this on linux, but only if you have used ulimit, as sgala
stated.

I would also mention that a better approach is to create an access time
mapping with blacklisting support to accept/close sockets that have
hammered your server.  It would also make sense to handle socket timeouts
(no read/write on a socket for X seconds).

Regardless, none of these things are within the world of problems that
base asyncore is intended to solve.  It's a bare-bones async socket
implementation.  If you want more features, use twisted or write your own
subclasses and offer it up on the Python Cookbook (activestate.com) or in
the Python Package Index (python.org/pypi).  If the community finds that
they are useful and productive features, we may consider it for inclusion
in the standard library.

Until then, suggested close.  Will close as "will not fix" if it is
assigned to me.

----------------------------------------------------------------------

Comment By: billiejoex (billiejoex)
Date: 2007-04-06 14:10

Message:
Logged In: YES 
user_id=1357589
Originator: YES

> Putting a try/except in place doesn't really help the problem... 
> if you fail to create a new socket what action will you take?

Calling a new dispatcher's method (for example:
"handle_exceeded_connection") could be an idea. By default it could close
the current session but it can be overriden if the user want to take some
other action.

> A better approach is to have a configurable limit on the number of 
> open connections, and then have a server-specific reaction to 
> exceeding that limit.

It doesn't work if I create additional file-objects during the execution
of the loop...

----------------------------------------------------------------------

Comment By: Santiago Gala (sgala)
Date: 2007-04-06 01:09

Message:
Logged In: YES 
user_id=178886
Originator: NO

THe limit of resources that an OS can deal with is limited due to
resources, both globally and per user, precisely to avoid DoS attacks by a
uses.

In the case of linux, you can control it with "ulimit -n XXXX" (for the
user that runs the test). The default here is 1024 (and the maximum, unless
it is globally raised).

I imagine windows will have similar limits, and similar (registry, etc.)
ways to handle them.

----------------------------------------------------------------------

Comment By: Sam Rushing (rushing)
Date: 2007-03-30 19:22

Message:
Logged In: YES 
user_id=73736
Originator: NO

Turns out medusa doesn't have a socket counter class, that was some other
project I was thinking of.

Putting a try/except in place doesn't really help the problem... if you
fail to create a new socket what action will you take?

A better approach is to have a configurable limit on the number of open
connections, and then have a server-specific reaction to exceeding that
limit.  For example, an SMTP server might respond with a 4XX greeting and
close the connection.

An additional problem on Unix is that running out of descriptors affects
more than just sockets.  Once you hit the FD limit you can't open files,
or do anything that requires a descriptor.


----------------------------------------------------------------------

Comment By: billiejoex (billiejoex)
Date: 2007-03-29 16:03

Message:
Logged In: YES 
user_id=1357589
Originator: YES

> The problem is that there's no portable way to know what the limit
> on file descriptors is.

Why don't put a try/except statement before creating a new socket's
file-descriptor?
I believe that such problem shouldn't be understimated. 
Actually asyncore is the only asynchronous module present in Python's
stdlib.
If asyncore suffers a DoS vulnerability it just means that writing a
secure client/server application with Python without using a third-party
library isn't possible.

I wrote a modified version of asyncore that solves the problem with select
 (affecting Windows) but still can't find a way to solve the problem with
socket's file-descriptors (affecting Unix).

----------------------------------------------------------------------

Comment By: Sam Rushing (rushing)
Date: 2007-03-23 20:59

Message:
Logged In: YES 
user_id=73736
Originator: NO

The problem is that there's no portable way to know what the limit
on file descriptors is.  The 'classic' for select/poll is the FD_SETSIZE
macro.  But on some operating systems there is no such limit. [e.g.,
win32
does not use the 'lowest-free-int' model common to unix].

I believe that in Medusa there was a derived class or extension that
counted
the number of open sockets, and limited it, using something like a
semaphore.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1685000&group_id=5470

From noreply at sourceforge.net  Fri Apr 13 21:36:52 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Fri, 13 Apr 2007 12:36:52 -0700
Subject: [ python-Bugs-1685000 ] asyncore DoS vulnerability
Message-ID: <E1HcRZk-0001AK-CG@sc8-sf-web23.sourceforge.net>

Bugs item #1685000, was opened at 2007-03-21 02:15
Message generated for change (Comment added) made by josiahcarlson
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1685000&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.5
>Status: Closed
>Resolution: Wont Fix
>Priority: 5
Private: No
Submitted By: billiejoex (billiejoex)
Assigned to: Josiah Carlson (josiahcarlson)
Summary: asyncore DoS vulnerability

Initial Comment:
DoS asyncore vulnerability

asyncore, independently if used with select() or poll(), suffers a DoS-type vulnerability when a high number of simultaneous connections to handle simultaneously is reached.
The number of maximum connections is system-dependent as well as the type of error raised.
I attached two simple Proof of Concept scripts demonstrating such bug.
If you want to try the behaviours listed below run the attached "asyncore_server.py" and "asyncore_client.py" scripts on your local workstation.

On my Windows XP system (Python 2.5), independently if asyncore has been used to develop a server or a client, the error is raised by select() inside asyncore's "poll" function when 512 (socket_map's elements) simultaneous connections are reached. 
Here's the traceback I get:

[...]
connections: 510
connections: 511
connections: 512
Traceback (most recent call last):
  File "C:\scripts\asyncore_server.py", line 38, in <module>
    asyncore.loop()
  File "C:\Python25\lib\asyncore.py", line 191, in loop
    poll_fun(timeout, map)
  File "C:\Python25\lib\asyncore.py", line 121, in poll
    r, w, e = select.select(r, w, e, timeout)
ValueError: too many file descriptors in select()


On my Linux Ubuntu 6.10 (kernel 2.6.17-10, Python 2.5) different type of errors are raised depending on the application (client or server).
In an asyncore-based client the error is raised by socket module (dispatcher's "self.socket" attribute) inside 'connect' method of 'dispatcher' class:

[...]
connections: 1018
connections: 1019
connections: 1020
connections: 1021
Traceback (most recent call last):
  File "asyncore_client.py", line 31, in <module>
  File "asyncore.py", line 191, in loop
  File "asyncore.py", line 138, in poll
  File "asyncore.py", line 80, in write
  File "asyncore.py", line 76, in write
  File "asyncore.py", line 395, in handle_write_event
  File "asyncore_client.py", line 24, in handle_connect
  File "asyncore_client.py", line 9, in __init__
  File "asyncore.py", line 257, in create_socket
  File "socket.py", line 156, in __init__
socket.error: (24, 'Too many open files')


On an asyncore-based server the error is raised by socket module (dispatcher's "self.socket" attribute) inside 'accept' method of 'dispatcher' class:

[...]
connections: 1019
connections: 1020
connections: 1021
Traceback (most recent call last):
  File "asyncore_server.py", line 38, in <module>
  File "asyncore.py", line 191, in loop
  File "asyncore.py", line 132, in poll
  File "asyncore.py", line 72, in read
  File "asyncore.py", line 68, in read
  File "asyncore.py", line 384, in handle_read_event
  File "asyncore_server.py", line 16, in handle_accept
  File "asyncore.py", line 321, in accept
  File "socket.py", line 170, in accept
socket.error: (24, 'Too many open files')


----------------------------------------------------------------------

>Comment By: Josiah Carlson (josiahcarlson)
Date: 2007-04-13 12:36

Message:
Logged In: YES 
user_id=341410
Originator: NO

The OP and I discussed this via email and IM.

There seems to be a few issues that the OP is concerned about.  The first
is that the number of allowable sockets/process is platform dependent
(Windows has no limit, linux can be set manually).  The second is that some
platforms limit the number of sockets that can be passed to select at a
time (Windows limits this to 512, I don't know about linux).  The third is
that the OP wants a solution to handling both a standard denial of service
attack (single client), as well as a distributed denial of service attack
(many clients).

The first issue is annoying, bit it is not within the realm of problems
that should be dealt with by base asyncore.  Just like platform floating
point handling is platform-dependent, sockets/file handles per process is
also platform-dependent, and trying to abstract it away is not reasonable.

The second issue is also annoying, but it also isn't within the realm of
problems that should be dealt with by base asyncore.  Whether or not an
application should be able to handle more than a few hundred sockets at a
time is dependent on the application, and modifying asyncore to make
assumptions about whether an application should or should not handle that
many sockets is not reasonable.

The third issue is also not reasonable for us to handle.  How to respond
to many incoming connections (from a single source, or from many source) is
also application dependent.  On a web server, maybe you just don't accept
connections when you are overloaded.  Maybe if you follow the advice of
http://www.remote.org/jochen/work/pub/zero-downtime.pdf , you handle them
all.  These are all application-specific.

Now, because how to handle the cases are all platform, application,
protocol, etc., dependent, assigning a single set of rules for asyncore to
respond to conditions of high numbers of sockets is outside the range or
problems that asyncore is intended to solve.  I would also point out that
Twisted doesn't do anything special to handle these cases.  When a Twisted
select reactor on linux comes to the file handle limit, it dies, just like
asyncore.  When a Twisted poll reactor on linux comes to the file handle
limit, it dies. On Windows, Twisted uses the Windows messaging API
"WSAEventSelect and MsgWaitForMultipleObjects, or something"
(http://twistedmatrix.com/pipermail/twisted-python/2006-April/012976.html),
but having written a poll method for Windows using this myself, I find that
select uses much less processor (I can provide you with an example).

With all of that said, since a higher-level asynchronous sockets framework
doesn't handle the case that you want asyncore to handle, I can't help but
say, once again, "asyncore wasn't intended to handle the situation that you
describe, and how *you* handle it, as an application developer, is up to
you."

Closing as won't fix.

----------------------------------------------------------------------

Comment By: Josiah Carlson (josiahcarlson)
Date: 2007-04-09 09:13

Message:
Logged In: YES 
user_id=341410
Originator: NO

Assign the "bug" to me, I'm the maintainer for asyncore/asynchat.

With that said, since a user needs to override
asyncore.dispatcher.handle_accept() anyways, which necessarily needs to
call asyncore.dispatcher.accept(), the subclass is free to check the number
of sockets in its socket map before creating a new instance of whatever
subclass of asyncore.dispatcher the user has written.

Also, the number of file handles that select can handle on Windows is a
compile-time constant, and has nothing to do with the actual number of open
file handles.  Take and run the following source file on Windows and see
how the total number of open sockets can be significantly larger than the
number of sockets passed to select():

import socket
import asyncore
import random

class new_map(dict):
    def items(self):
        r = [(i,j) for i,j in dict.items(self) if not random.randrange(4)
and j != h]
        r.append((h._fileno, h))
        print len(r), len(asyncore.socket_map)
        return r

asyncore.socket_map = new_map()

class listener(asyncore.dispatcher):
    def handle_accept(self):
        x = self.accept()
        if x:
            conn, addr = x
            connection(conn)

class connection(asyncore.dispatcher):
    def writable(self):
        0
    def handle_connect(self):
        pass

if __name__ == '__main__':
    h = listener()
    h.create_socket(socket.AF_INET, socket.SOCK_STREAM)
    h.bind(('127.0.0.1', 10001))
    h.listen(5)
    while len(asyncore.socket_map) < 4096:
        a = connection()
        a.create_socket(socket.AF_INET, socket.SOCK_STREAM)
        a.connect_ex(('127.0.0.1', 10001))
        asyncore.poll()

The tail end of a run in Windows:

476 1934
501 1936
516 1938
Traceback (most recent call last):
  File "D:\MYDOCS\Projects\python\async_socket.py", line 37, in ?
    asyncore.poll()
  File "C:\python23\lib\asyncore.py", line 108, in poll
    r, w, e = select.select(r, w, e, timeout)
ValueError: too many file descriptors in select()

With a proper definition of new_map, you can handle an unlimited number of
sockets with asyncore by choosing blocks of sockets to return in its
items() call.  Note that I used random out of convenience, a proper
implementation could distribute the sockets based on fileno, time inserted,
etc.  You can do this on linux, but only if you have used ulimit, as sgala
stated.

I would also mention that a better approach is to create an access time
mapping with blacklisting support to accept/close sockets that have
hammered your server.  It would also make sense to handle socket timeouts
(no read/write on a socket for X seconds).

Regardless, none of these things are within the world of problems that
base asyncore is intended to solve.  It's a bare-bones async socket
implementation.  If you want more features, use twisted or write your own
subclasses and offer it up on the Python Cookbook (activestate.com) or in
the Python Package Index (python.org/pypi).  If the community finds that
they are useful and productive features, we may consider it for inclusion
in the standard library.

Until then, suggested close.  Will close as "will not fix" if it is
assigned to me.

----------------------------------------------------------------------

Comment By: billiejoex (billiejoex)
Date: 2007-04-06 05:10

Message:
Logged In: YES 
user_id=1357589
Originator: YES

> Putting a try/except in place doesn't really help the problem... 
> if you fail to create a new socket what action will you take?

Calling a new dispatcher's method (for example:
"handle_exceeded_connection") could be an idea. By default it could close
the current session but it can be overriden if the user want to take some
other action.

> A better approach is to have a configurable limit on the number of 
> open connections, and then have a server-specific reaction to 
> exceeding that limit.

It doesn't work if I create additional file-objects during the execution
of the loop...

----------------------------------------------------------------------

Comment By: Santiago Gala (sgala)
Date: 2007-04-05 16:09

Message:
Logged In: YES 
user_id=178886
Originator: NO

THe limit of resources that an OS can deal with is limited due to
resources, both globally and per user, precisely to avoid DoS attacks by a
uses.

In the case of linux, you can control it with "ulimit -n XXXX" (for the
user that runs the test). The default here is 1024 (and the maximum, unless
it is globally raised).

I imagine windows will have similar limits, and similar (registry, etc.)
ways to handle them.

----------------------------------------------------------------------

Comment By: Sam Rushing (rushing)
Date: 2007-03-30 10:22

Message:
Logged In: YES 
user_id=73736
Originator: NO

Turns out medusa doesn't have a socket counter class, that was some other
project I was thinking of.

Putting a try/except in place doesn't really help the problem... if you
fail to create a new socket what action will you take?

A better approach is to have a configurable limit on the number of open
connections, and then have a server-specific reaction to exceeding that
limit.  For example, an SMTP server might respond with a 4XX greeting and
close the connection.

An additional problem on Unix is that running out of descriptors affects
more than just sockets.  Once you hit the FD limit you can't open files,
or do anything that requires a descriptor.


----------------------------------------------------------------------

Comment By: billiejoex (billiejoex)
Date: 2007-03-29 07:03

Message:
Logged In: YES 
user_id=1357589
Originator: YES

> The problem is that there's no portable way to know what the limit
> on file descriptors is.

Why don't put a try/except statement before creating a new socket's
file-descriptor?
I believe that such problem shouldn't be understimated. 
Actually asyncore is the only asynchronous module present in Python's
stdlib.
If asyncore suffers a DoS vulnerability it just means that writing a
secure client/server application with Python without using a third-party
library isn't possible.

I wrote a modified version of asyncore that solves the problem with select
 (affecting Windows) but still can't find a way to solve the problem with
socket's file-descriptors (affecting Unix).

----------------------------------------------------------------------

Comment By: Sam Rushing (rushing)
Date: 2007-03-23 12:59

Message:
Logged In: YES 
user_id=73736
Originator: NO

The problem is that there's no portable way to know what the limit
on file descriptors is.  The 'classic' for select/poll is the FD_SETSIZE
macro.  But on some operating systems there is no such limit. [e.g.,
win32
does not use the 'lowest-free-int' model common to unix].

I believe that in Medusa there was a derived class or extension that
counted
the number of open sockets, and limited it, using something like a
semaphore.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1685000&group_id=5470

From noreply at sourceforge.net  Fri Apr 13 21:53:14 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Fri, 13 Apr 2007 12:53:14 -0700
Subject: [ python-Bugs-1700304 ] pydoc.help samples sys.stdout and sys.stdin
	at import time
Message-ID: <E1HcRpa-0007Us-Dy@sc8-sf-web24.sourceforge.net>

Bugs item #1700304, was opened at 2007-04-13 12:53
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1700304&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Extension Modules
Group: Python 2.4
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: James Lingard (jchl)
Assigned to: Nobody/Anonymous (nobody)
Summary: pydoc.help samples sys.stdout and sys.stdin at import time

Initial Comment:
pydoc.help (aliased to the builtin help) uses the values of sys.stdout and sys.stdin that were in use when the pydoc module was imported.  This means that if sys.stdout and/or sys.stdin are later modified, subsequent calls to pydoc.help (or help) use the wrong stdout and stdin.

Instead, help should use the current values of sys.stdout and sys.stdin each time it is called.

Reported against Python 2.4.4.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1700304&group_id=5470

From noreply at sourceforge.net  Sat Apr 14 05:16:14 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Fri, 13 Apr 2007 20:16:14 -0700
Subject: [ python-Bugs-1700455 ] ctypes Fundamental data types
Message-ID: <E1HcYkI-0003aK-Q2@sc8-sf-web21.sourceforge.net>

Bugs item #1700455, was opened at 2007-04-14 11:16
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1700455&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Documentation
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: hg6980 (hg6980)
Assigned to: Nobody/Anonymous (nobody)
Summary: ctypes Fundamental data types

Initial Comment:
I think the sentence:-



The current memory block contents can be accessed (or changed) with the raw property, if you want to access it as NUL terminated string, use the string property


should be
The current memory block contents can be accessed (or changed) with the raw property, if you want to access it as NUL terminated string, use the value property


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1700455&group_id=5470

From noreply at sourceforge.net  Sat Apr 14 06:28:38 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Fri, 13 Apr 2007 21:28:38 -0700
Subject: [ python-Bugs-1163401 ] uncaught AttributeError deep in urllib
Message-ID: <E1HcZsM-0001id-C6@sc8-sf-web23.sourceforge.net>

Bugs item #1163401, was opened at 2005-03-15 00:39
Message generated for change (Comment added) made by nagle
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1163401&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Extension Modules
Group: Python 2.4
Status: Closed
Resolution: Duplicate
Priority: 5
Private: No
Submitted By: K Lars Lohn (lohnk)
Assigned to: Nobody/Anonymous (nobody)
Summary: uncaught AttributeError deep in urllib

Initial Comment:
Python 2.4 and Python 2.3.4 running under Suse 9.2

We're getting an AttributeError exception "AttributeError: 'NoneType' object has no attribute 'read'" within a very simple call to urllib.urlopen.

This was discovered while working on Sentry 2, the new mirror integrity checker for the Mozilla project.  We try to touch hundreds of URLs to make sure that the files are present on each of the mirrors.  One particular URL kills the call to urllib.urlopen: http://mozilla.mirrors.skynet.be/pub/ftp.mozilla.org/firefox/releases/1.0/win32/en-US/Firefox%20Setup%201.0.exe
This file probably does not exist on the mirror, however, in other cases of bad URLs, we get much more graceful failures when we try to read from the object returned by urllib.urlopen.

>>> import urllib
>>> urlReader = urllib.urlopen("http://mozilla.mirrors.skynet.be/pub/ftp.mozilla.org/firefox/releases/1.0/win32/en-US/Firefox%20Setup%201.0.exe")
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "/usr/local/lib/python2.4/urllib.py", line 77, in urlopen
    return opener.open(url)
  File "/usr/local/lib/python2.4/urllib.py", line 180, in open
    return getattr(self, name)(url)
  File "/usr/local/lib/python2.4/urllib.py", line 305, in open_http
    return self.http_error(url, fp, errcode, errmsg, headers)
  File "/usr/local/lib/python2.4/urllib.py", line 322, in http_error
    return self.http_error_default(url, fp, errcode, errmsg, headers)
  File "/usr/local/lib/python2.4/urllib.py", line 550, in http_error_default
    return addinfourl(fp, headers, "http:" + url)
  File "/usr/local/lib/python2.4/urllib.py", line 836, in __init__
    addbase.__init__(self, fp)
  File "/usr/local/lib/python2.4/urllib.py", line 786, in __init__
    self.read = self.fp.read
AttributeError: 'NoneType' object has no attribute 'read'

The  attached file is a three line scipt that demos the problem.



----------------------------------------------------------------------

Comment By: John Nagle (nagle)
Date: 2007-04-14 04:28

Message:
Logged In: YES 
user_id=5571
Originator: NO

The basic cause of the "NoneType" attribute error is a straightforward bug
in "urllib2". If an error occurs during opening that causes an error to
"http_error_default", a dummy file object is created using "addinfourl", so
as to return something that looks like an empty file, rather than raising
an exception.  But that doesn't work if "getfile()" on the httplib.HTTP
object returns "None", which is unusual but can happen.  

We're seeing this error in Python 2.4 on Windows.

We're still trying to understand exactly what network situation forces
this path, but it's quite real.  It seems to be occurring on Python 2.5 on
Linux, too.

If you override http_error_default in a subclass, you get an HTTP error of
"-1" reported when this situation occurs. 



----------------------------------------------------------------------

Comment By: Georg Brandl (birkenfeld)
Date: 2005-12-15 22:10

Message:
Logged In: YES 
user_id=1188172

Duplicate of #767111.

----------------------------------------------------------------------

Comment By: Roy Smith (roysmith)
Date: 2005-04-02 21:44

Message:
Logged In: YES 
user_id=390499

Wow, this is bizarre.  I just spend some time tracking down exactly this 
same bug and was just about to enter it when I saw this entry.  For what 
it's worth, I can reliably reproduce this exception when fetching a URL
from 
a deliberately broken server (well, at least I think it's broken; have to

double-check the HTTP spec to be sure this isn't actually allowed) which 
produces headers but no body:

(This is on Mac OSX-10.3.8, Python-2.3.4)

-------------------------------
Roy-Smiths-Computer:bug$ cat server.py
#!/usr/bin/env python

from BaseHTTPServer import *

class NullHandler (BaseHTTPRequestHandler):
    def do_GET (self):
        self.send_response (100)
        self.end_headers ()

server = HTTPServer (('', 8000), NullHandler)
server.handle_request()
------------------------------
Roy-Smiths-Computer:bug$ cat client.py
#!/usr/bin/env python

import urllib

urllib.urlopen ('http://127.0.0.1:8000')
---------------------------------
Roy-Smiths-Computer:bug$ ./client.py
Traceback (most recent call last):
  File "./client.py", line 5, in ?
    urllib.urlopen ('http://127.0.0.1:8000')
  File "/usr/local/lib/python2.3/urllib.py", line 76, in urlopen
    return opener.open(url)
  File "/usr/local/lib/python2.3/urllib.py", line 181, in open
    return getattr(self, name)(url)
  File "/usr/local/lib/python2.3/urllib.py", line 306, in open_http
    return self.http_error(url, fp, errcode, errmsg, headers)
  File "/usr/local/lib/python2.3/urllib.py", line 323, in http_error
    return self.http_error_default(url, fp, errcode, errmsg, headers)
  File "/usr/local/lib/python2.3/urllib.py", line 551, in
http_error_default
    return addinfourl(fp, headers, "http:" + url)
  File "/usr/local/lib/python2.3/urllib.py", line 837, in __init__
    addbase.__init__(self, fp)
  File "/usr/local/lib/python2.3/urllib.py", line 787, in __init__
    self.read = self.fp.read
AttributeError: 'NoneType' object has no attribute 'read'

---------------------------------

I'll give urllib2 a try and see how that works.


----------------------------------------------------------------------

Comment By: K Lars Lohn (lohnk)
Date: 2005-03-16 17:07

Message:
Logged In: YES 
user_id=1239273

I've changed over to urllib2. The only complication involved the exception
handling model: urllib2's HTTPError exceptions cannot be pickled because
they contain an open socket._fileobject.  While mildly inconvenient, the
workaround was not difficult.

----------------------------------------------------------------------

Comment By: Skip Montanaro (montanaro)
Date: 2005-03-15 19:09

Message:
Logged In: YES 
user_id=44345


Looking through the code I believe I traced the problem back
to httplib.HTTP which sets self.fp to None when it's closed.
It seems that urllib is trying to access this object after
the connection's been closed.

I realize the problem has passed for the moment, but have you 
considered using urllib2?  The urllib library still uses
httplib.HTTP
which is really only there for backward compatibility.  From
this end it would be nice to leave urllib and httplib.HTTP
alone.
New apps should probably use urllib2 which uses the newer
httplib.HTTPConnection class.


----------------------------------------------------------------------

Comment By: K Lars Lohn (lohnk)
Date: 2005-03-15 16:50

Message:
Logged In: YES 
user_id=1239273

This problem is apparently transient depending on network conditions or,
perhaps, the configuration of the server end.  On 3/14 the problem has
mysteriously vanished....

----------------------------------------------------------------------

Comment By: Jarek Zgoda (zgoda)
Date: 2005-03-15 09:52

Message:
Logged In: YES 
user_id=92222

No such error on Windows:
Python 2.4 (#60, Nov 30 2004, 11:49:19) [MSC v.1310 32 bit
(Intel)] on win32

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1163401&group_id=5470

From noreply at sourceforge.net  Sat Apr 14 10:27:02 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Sat, 14 Apr 2007 01:27:02 -0700
Subject: [ python-Bugs-1700507 ] Carbon.Scrap.PutScrapFlavor
Message-ID: <E1Hcdb4-0004KS-Fx@sc8-sf-web22.sourceforge.net>

Bugs item #1700507, was opened at 2007-04-14 01:27
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1700507&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Macintosh
Group: Platform-specific
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: dingus9 (dingus9)
Assigned to: Jack Jansen (jackjansen)
Summary: Carbon.Scrap.PutScrapFlavor

Initial Comment:
The GetScrapFlavor(), and ClearCurrentScrap() functions work fine, however the PutScrapFlavor, is not putting th e text into my clipboard. The function does not return an error nor exception from my code, as far as I can tell.

sample of code:
import Carbon.Scrap as scrap 
import MacOS

    text = 'the test text'
    scrap.ClearCurrentScrap()
    item = scrap.GetCurrentScrap()
    item.PutScrapFlavor(type, 0, text)

Result is: Previous text is cleard, and nothing is put in its place.

I am continuing to research this, as well as try it in C, and install python2.5.

Environment
Python 2.4
Mac OSX 10.4.9
Intel

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1700507&group_id=5470

From noreply at sourceforge.net  Sat Apr 14 21:06:30 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Sat, 14 Apr 2007 12:06:30 -0700
Subject: [ python-Bugs-1700455 ] ctypes Fundamental data types
Message-ID: <E1HcnZu-0002FR-38@sc8-sf-web24.sourceforge.net>

Bugs item #1700455, was opened at 2007-04-14 05:16
Message generated for change (Comment added) made by loewis
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1700455&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Documentation
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: hg6980 (hg6980)
>Assigned to: Thomas Heller (theller)
Summary: ctypes Fundamental data types

Initial Comment:
I think the sentence:-



The current memory block contents can be accessed (or changed) with the raw property, if you want to access it as NUL terminated string, use the string property


should be
The current memory block contents can be accessed (or changed) with the raw property, if you want to access it as NUL terminated string, use the value property


----------------------------------------------------------------------

>Comment By: Martin v. L?wis (loewis)
Date: 2007-04-14 21:06

Message:
Logged In: YES 
user_id=21627
Originator: NO

Thomas, can you take a look?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1700455&group_id=5470

From noreply at sourceforge.net  Sat Apr 14 21:08:25 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Sat, 14 Apr 2007 12:08:25 -0700
Subject: [ python-Bugs-1698944 ] dtdparser discards last line
Message-ID: <E1Hcnbl-0002Sd-2l@sc8-sf-web24.sourceforge.net>

Bugs item #1698944, was opened at 2007-04-12 07:44
Message generated for change (Comment added) made by loewis
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1698944&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
>Group: 3rd Party
>Status: Closed
>Resolution: Invalid
Priority: 5
Private: No
Submitted By: L. Peter Deutsch (lpd)
Assigned to: Nobody/Anonymous (nobody)
Summary: dtdparser discards last line

Initial Comment:
The DTD parser in xml.parsers.xmlproc.dtdparser sometimes simply ignores the last line of the input file. Here is the Python program I used to test it:

------begin
from xml.parsers.xmlproc.dtdparser import DTDParser,\
 DTDConsumer, DTDConsumerPE
import tracer

def parse_dtd(dtd):
    parser = DTDParser()
    handler = DTDConsumerPE()
    parser.set_dtd_consumer(handler)
    parser.feed(dtd)

tracer.trace(DTDConsumer)

parse_dtd('''\
<!ENTITY % score PUBLIC

	"-//Recordare//ELEMENTS MusicXML 1.1 Score//EN"

	"t.dtd">

%score;



''')
------end

where the file t.dtd contained:

------begin
<!ELEMENT opus EMPTY>
------end

The tracer package (my own code) indicated that the opus element was never parsed. The same was true if I added a blank line after the %score; entity reference. However, if I added *two* blank lines after the entity reference, the element declaration *was* processed.

I would note that while the bug exists in the original xmlproc package (at least in xmlproc 0.70, downloadable from http://www.garshol.priv.no/download/software/xmlproc/), the bug does *not* exist in the PyXML package (downloadable from http://sourceforge.net/projects/pyxml).


----------------------------------------------------------------------

>Comment By: Martin v. L?wis (loewis)
Date: 2007-04-14 21:08

Message:
Logged In: YES 
user_id=21627
Originator: NO

xmlproc is not part of the Python release; instead, it is part of PyXML.
Closing this report as third-party.

Alas, PyXML is not maintained anymore, so bugs in xmlproc are unlikely to
get fixed.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1698944&group_id=5470

From noreply at sourceforge.net  Sat Apr 14 21:12:25 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Sat, 14 Apr 2007 12:12:25 -0700
Subject: [ python-Bugs-1697169 ] package.pth file name not used as described. 
Message-ID: <E1Hcnfd-0008Sc-Q6@sc8-sf-web22.sourceforge.net>

Bugs item #1697169, was opened at 2007-04-09 21:58
Message generated for change (Comment added) made by loewis
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1697169&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Documentation
Group: None
>Status: Pending
>Resolution: Wont Fix
Priority: 5
Private: No
Submitted By: cfk (carlfk)
Assigned to: Nobody/Anonymous (nobody)
Summary: package.pth file name not used as described. 

Initial Comment:
"...whose name has the form package.pth" http://www.python.org/doc/lib/module-site.html 

as far as I can tell, that is incorrect.  the .pth file can be named anything - only the name of the dir listed in it is used as a package/module name.

It is implemented in site.py : 

def addpackage(sitedir, name, known_paths):

    fullname = os.path.join(sitedir, name)

        f = open(fullname, "rU")

        for line in f:

            dir, dircase = makepath(sitedir, line)
            if not dircase in known_paths and os.path.exists(dir):
                sys.path.append(dir)


Notice name is not added to sys.path.

----------------------------------------------------------------------

>Comment By: Martin v. L?wis (loewis)
Date: 2007-04-14 21:12

Message:
Logged In: YES 
user_id=21627
Originator: NO

I fail to see the problem. "package" is deliberately type-set in italics
to indicate that the name of the pth is a placeholder. Nowhere it says that
it must be the string "package", or a specific string.

Tentatively closing as won't fix.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1697169&group_id=5470

From noreply at sourceforge.net  Sat Apr 14 21:13:17 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Sat, 14 Apr 2007 12:13:17 -0700
Subject: [ python-Feature Requests-1697175 ] winreg module for cygwin?
Message-ID: <E1HcngT-00007h-Gx@sc8-sf-web22.sourceforge.net>

Feature Requests item #1697175, was opened at 2007-04-09 22:06
Message generated for change (Comment added) made by loewis
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1697175&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
>Category: Windows
>Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Zooko O'Whielacronx (zooko)
Assigned to: Nobody/Anonymous (nobody)
Summary: winreg module for cygwin?

Initial Comment:
Please make the winreg module work on cygwin.  Nowadays cygwin has a /proc/registry file system, which might provide an easy way to do it.

----------------------------------------------------------------------

>Comment By: Martin v. L?wis (loewis)
Date: 2007-04-14 21:13

Message:
Logged In: YES 
user_id=21627
Originator: NO

Reclassifying as a feature request.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1697175&group_id=5470

From noreply at sourceforge.net  Sun Apr 15 01:07:14 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Sat, 14 Apr 2007 16:07:14 -0700
Subject: [ python-Feature Requests-1700821 ] audioop module - request to add a
	note
Message-ID: <E1HcrKs-0004Mt-JU@sc8-sf-web21.sourceforge.net>

Feature Requests item #1700821, was opened at 2007-04-15 02:07
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1700821&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Documentation
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: ncryptor (ncryptor)
Assigned to: Nobody/Anonymous (nobody)
Summary: audioop module - request to add a note

Initial Comment:
The audioop function lin2lin converts sample sizes of linear pcm fragments.
However, when converting to 8-bit, there is a pitfall (that wasted for me a few hours).
In some audio formats, such as .WAV files, 16 and 32 bit samples are signed, but 8 bit samples are unsigned. So when converting from 2 to 1 byte wide samples, you need to also add 128 to the result.
Example code:

new_frames = audioop.lin2lin(frames, old_width, new_width)
if 1 == new_width:
   new_frames = audioop.bias(new_frames, 1, 128)

the same, in reverse, has to be applied when converting from 8 to 16 or 32 bit width samples.

This wasted so much time for me and is hardly documented.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1700821&group_id=5470

From noreply at sourceforge.net  Sun Apr 15 05:51:16 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Sat, 14 Apr 2007 20:51:16 -0700
Subject: [ python-Bugs-1700865 ] PCbuild8/pcbuild.sln is missing "_socket"
	sub-project
Message-ID: <E1Hcvlk-000241-L2@sc8-sf-web23.sourceforge.net>

Bugs item #1700865, was opened at 2007-04-14 23:51
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1700865&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Build
Group: Python 2.6
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Raghuram Devarakonda (draghuram)
Assigned to: Nobody/Anonymous (nobody)
Summary: PCbuild8/pcbuild.sln is missing "_socket" sub-project

Initial Comment:

As the subject says, PCbuild8/pcbuild.sln is missing entry for "_socket". However, _socket.vcproj is present in the directory.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1700865&group_id=5470

From noreply at sourceforge.net  Sun Apr 15 10:28:46 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Sun, 15 Apr 2007 01:28:46 -0700
Subject: [ python-Bugs-1700912 ] questionable results from IDLE, Windows,
	& several built-in 
Message-ID: <E1Hd06I-0006vb-71@sc8-sf-web23.sourceforge.net>

Bugs item #1700912, was opened at 2007-04-15 03:28
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1700912&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Windows
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: bavantgarde314 (bavantgarde314)
Assigned to: Nobody/Anonymous (nobody)
Summary: questionable results from IDLE, Windows, & several built-in 

Initial Comment:
Python 2.5 (r25:51908, Sep 19 2006, 09:52:17) [MSC v.1310 32 bit (Intel)] on win32

Windows(ME)


Attachment reveals questionable results from IDLE, Windows, & several built-in functions.
        
Under IDLE: function swapcase() updates 120 characters.
Under Windows (dos-screen) or a Windows ide swapcase() updates only 52 characters: ord values 65-90 for (uppercase)A-Z and 97-122 for (lowercase)a-z.
                   
Under IDLE: functions lower() & upper() return different results than swapcase()
for hex values:  8a,  8c,  8e,  9a,  9c,  9e,  9f, ff  
 or ord values  138, 140, 142, 154, 156, 158, 159, 255
 or characters   ?    ?    ?    ?    ?    ?    ?    ?




----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1700912&group_id=5470

From noreply at sourceforge.net  Sun Apr 15 11:41:55 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Sun, 15 Apr 2007 02:41:55 -0700
Subject: [ python-Bugs-1700507 ] Carbon.Scrap.PutScrapFlavor
Message-ID: <E1Hd1F5-0003nX-Ue@sc8-sf-web21.sourceforge.net>

Bugs item #1700507, was opened at 2007-04-14 01:27
Message generated for change (Comment added) made by dingus9
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1700507&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Macintosh
Group: Platform-specific
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: dingus9 (dingus9)
Assigned to: Jack Jansen (jackjansen)
Summary: Carbon.Scrap.PutScrapFlavor

Initial Comment:
The GetScrapFlavor(), and ClearCurrentScrap() functions work fine, however the PutScrapFlavor, is not putting th e text into my clipboard. The function does not return an error nor exception from my code, as far as I can tell.

sample of code:
import Carbon.Scrap as scrap 
import MacOS

    text = 'the test text'
    scrap.ClearCurrentScrap()
    item = scrap.GetCurrentScrap()
    item.PutScrapFlavor(type, 0, text)

Result is: Previous text is cleard, and nothing is put in its place.

I am continuing to research this, as well as try it in C, and install python2.5.

Environment
Python 2.4
Mac OSX 10.4.9
Intel

----------------------------------------------------------------------

>Comment By: dingus9 (dingus9)
Date: 2007-04-15 02:41

Message:
Logged In: YES 
user_id=1769255
Originator: YES

Tested with python25, function calls GetScrapFlavorData, and
PutScrapFlavorData now result in a bus error.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1700507&group_id=5470

From noreply at sourceforge.net  Sun Apr 15 19:59:18 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Sun, 15 Apr 2007 10:59:18 -0700
Subject: [ python-Bugs-1697215 ] Docstring for site.addpackage() is incorrect
Message-ID: <E1Hd90Q-0001AK-L8@sc8-sf-web24.sourceforge.net>

Bugs item #1697215, was opened at 2007-04-09 23:23
Message generated for change (Comment added) made by zseil
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1697215&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Documentation
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: cfk (carlfk)
Assigned to: Nobody/Anonymous (nobody)
>Summary: Docstring for site.addpackage() is incorrect

Initial Comment:
docstring says "execute sitedir" - but that isn't what gets executed. 

suggested: 
    """Add a new path to known_paths by combining sitedir and 'name' 
or execute lines in name.pth that start with 'import'"""

site.py

def addpackage(sitedir, name, known_paths):

    """Add a new path to known_paths by combining sitedir and 'name' or execute sitedir if it starts with 'import'"""

    fullname = os.path.join(sitedir, name)
        f = open(fullname, "rU")
        for line in f:
            if line.startswith("import"):
                exec line
                                      

----------------------------------------------------------------------

>Comment By: Ziga Seilnacht (zseil)
Date: 2007-04-15 19:59

Message:
Logged In: YES 
user_id=1326842
Originator: NO

Here is a patch.  addpackage() is actually responsible
for .pth file processing, so the current docstring was
entirely incorrect.


File Added: addpackage_docstring.diff

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1697215&group_id=5470

From noreply at sourceforge.net  Sun Apr 15 20:39:32 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Sun, 15 Apr 2007 11:39:32 -0700
Subject: [ python-Bugs-1690103 ] trace module borks __file__
Message-ID: <E1Hd9dM-0003vN-66@sc8-sf-web21.sourceforge.net>

Bugs item #1690103, was opened at 2007-03-28 21:05
Message generated for change (Comment added) made by zseil
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1690103&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Skip Montanaro (montanaro)
Assigned to: Nobody/Anonymous (nobody)
Summary: trace module borks __file__

Initial Comment:
Stick this in a file (say, printfile.py):

    print __file__

If I run it as "python printfile.py" using Python 2.4, 2.5 or 2.6 it prints as expected, e.g.:

    % python ~/tmp/printfile.py
    /Users/skip/tmp/printfile.py

If, however, I run it under control of the trace module I get something entirely different:

    % python -m trace --count ~/tmp/printfile.py
    /Users/skip/local/lib/python2.6/trace.py
    % python -m trace --trace ~/tmp/printfile.py
     --- modulename: threading, funcname: settrace
    threading.py(70):     _trace_hook = func
     --- modulename: trace, funcname: <module>
    <string>(1):   --- modulename: trace, funcname: <module>
    printfile.py(1): print __file__
    /Users/skip/local/lib/python2.6/trace.py

Definitely looks buggy to me...


----------------------------------------------------------------------

>Comment By: Ziga Seilnacht (zseil)
Date: 2007-04-15 20:39

Message:
Logged In: YES 
user_id=1326842
Originator: NO

Wouldn't it be better for trace.main() to call
Tracer.runctx() instead of Tracer.run()?  That
way you could use a copy of __main__.__dict__
for locals and globals and overwrite the filename
there.  Note that your solution would still report
the wrong filename if someone called tracer.main()
from his own main script.

----------------------------------------------------------------------

Comment By: Skip Montanaro (montanaro)
Date: 2007-03-28 22:38

Message:
Logged In: YES 
user_id=44345
Originator: YES

Hmmm...  Maybe not so hard after all.  See attached diff.  Does anyone see
a problem with simply overwriting the trace module's __file__ attribute?

File Added: trace.diff

----------------------------------------------------------------------

Comment By: Skip Montanaro (montanaro)
Date: 2007-03-28 22:01

Message:
Logged In: YES 
user_id=44345
Originator: YES

I believe this is because the trace module calls execfile()
to run the file.  I'm not sure if there's an easy fix for
this problem.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1690103&group_id=5470

From noreply at sourceforge.net  Mon Apr 16 02:02:34 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Sun, 15 Apr 2007 17:02:34 -0700
Subject: [ python-Bugs-1701192 ] symbol conflict for 'readahead'
Message-ID: <E1HdEfy-0001dn-Ul@sc8-sf-web22.sourceforge.net>

Bugs item #1701192, was opened at 2007-04-15 20:02
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1701192&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Interpreter Core
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: ajrh (ajrh)
Assigned to: Nobody/Anonymous (nobody)
Summary: symbol conflict for 'readahead'

Initial Comment:
I'm encountering the error below with python 2.5 on current Debian x86. 

src/python-2.5.1c1/Objects/fileobject.c:1845: error: conflicting types for 'readahead'
/usr/include/bits/fcntl.h:187: error: previous declaration of 'readahead' was here

It may be that's it's provoked only by my slightly custom embedded build process - perhaps the standard build does not include fcntl.h while compiling this file.

A fix is trivial - just to rename the static readahead function to something else, e.g. as below.

Rgds

A

====================================



--- fileobject.c.Orig   2007-04-15 19:47:35.000000000 -0400
+++ fileobject.c        2007-04-15 19:49:11.000000000 -0400
@@ -1841,7 +1841,7 @@
    (unless at EOF) and no more than bufsize.  Returns negative value on
    error, will set MemoryError if bufsize bytes cannot be allocated. */
 static int
-readahead(PyFileObject *f, int bufsize)
+do_readahead(PyFileObject *f, int bufsize)
 {
        Py_ssize_t chunksize;

@@ -1887,7 +1887,7 @@
        Py_ssize_t len;

        if (f->f_buf == NULL)
-               if (readahead(f, bufsize) < 0)
+               if (do_readahead(f, bufsize) < 0)
                        return NULL;

 

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1701192&group_id=5470

From noreply at sourceforge.net  Mon Apr 16 02:45:16 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Sun, 15 Apr 2007 17:45:16 -0700
Subject: [ python-Bugs-1701207 ] svnversion_init assertion failed
Message-ID: <E1HdFLI-0007Ko-86@sc8-sf-web24.sourceforge.net>

Bugs item #1701207, was opened at 2007-04-15 20:45
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1701207&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Extension Modules
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: ajrh (ajrh)
Assigned to: Nobody/Anonymous (nobody)
Summary: svnversion_init assertion failed

Initial Comment:
python: src/python-2.5.1c1/Python/sysmodule.c:1000: svnversion_init: Assertion `len >= 13' failed.

I see these assertions were added here:
http://mail.python.org/pipermail/python-checkins/2006-October/057079.html

Given the form of the subversion headurl string, the assertion doesn't appear to me at first glance to be valid.




----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1701207&group_id=5470

From noreply at sourceforge.net  Mon Apr 16 06:48:25 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Sun, 15 Apr 2007 21:48:25 -0700
Subject: [ python-Bugs-1700865 ] PCbuild8/pcbuild.sln is missing "_socket"
	sub-project
Message-ID: <E1HdJ8b-0000cY-0U@sc8-sf-web23.sourceforge.net>

Bugs item #1700865, was opened at 2007-04-15 05:51
Message generated for change (Comment added) made by loewis
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1700865&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Build
Group: Python 2.6
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Raghuram Devarakonda (draghuram)
>Assigned to: Kristj?n Valur (krisvale)
Summary: PCbuild8/pcbuild.sln is missing "_socket" sub-project

Initial Comment:

As the subject says, PCbuild8/pcbuild.sln is missing entry for "_socket". However, _socket.vcproj is present in the directory.

----------------------------------------------------------------------

>Comment By: Martin v. L??wis (loewis)
Date: 2007-04-16 06:48

Message:
Logged In: YES 
user_id=21627
Originator: NO

Kristjan, can you take a look?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1700865&group_id=5470

From noreply at sourceforge.net  Mon Apr 16 07:03:08 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Sun, 15 Apr 2007 22:03:08 -0700
Subject: [ python-Bugs-1701207 ] svnversion_init assertion failed
Message-ID: <E1HdJMq-0007If-Cr@sc8-sf-web21.sourceforge.net>

Bugs item #1701207, was opened at 2007-04-16 02:45
Message generated for change (Comment added) made by loewis
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1701207&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Extension Modules
Group: Python 2.5
Status: Open
Resolution: None
>Priority: 9
Private: No
Submitted By: ajrh (ajrh)
>Assigned to: Neal Norwitz (nnorwitz)
Summary: svnversion_init assertion failed

Initial Comment:
python: src/python-2.5.1c1/Python/sysmodule.c:1000: svnversion_init: Assertion `len >= 13' failed.

I see these assertions were added here:
http://mail.python.org/pipermail/python-checkins/2006-October/057079.html

Given the form of the subversion headurl string, the assertion doesn't appear to me at first glance to be valid.




----------------------------------------------------------------------

>Comment By: Martin v. L?wis (loewis)
Date: 2007-04-16 07:03

Message:
Logged In: YES 
user_id=21627
Originator: NO

Neal, can you take a look? The assert really does not make sense to me.
How did you arrive at 13?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1701207&group_id=5470

From noreply at sourceforge.net  Mon Apr 16 07:09:53 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Sun, 15 Apr 2007 22:09:53 -0700
Subject: [ python-Bugs-1700132 ] import and capital letters
Message-ID: <E1HdJTN-0002rb-2y@sc8-sf-web23.sourceforge.net>

Bugs item #1700132, was opened at 2007-04-13 17:06
Message generated for change (Comment added) made by loewis
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1700132&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Interpreter Core
Group: Python 2.4
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: omsynge (omsynge)
Assigned to: Nobody/Anonymous (nobody)
Summary: import and capital letters

Initial Comment:
Interactive and relative paths allow unlimited
(Or at least I have not found a limit yet) number of characters to be uppercase. This is fine, but when the python interpreter reads directories from PYTHONPATH with 3 capital letters you then get a failure to locate the files.

I have replicated this issue with python 2.2 and python 2.4. These where with red hat el3 and ubuntu (Some version not sure which)

and an example

import dcacheYaimInstallerTest.logger as logger

works fine in interactive or relative paths, but not when installed via an RPM.


import dcacheYaimInstallertest.logger as logger

is just fine in both scenarios. 

This bug cost me some hours to trace and would have cost more had I not had a lot of experience of packaging, so I would be pleased if this could be fixed in all versions of Python.

Regards

Owen








----------------------------------------------------------------------

>Comment By: Martin v. L?wis (loewis)
Date: 2007-04-16 07:09

Message:
Logged In: YES 
user_id=21627
Originator: NO

I cannot reproduce the problem. The import statement works fine for me
when I create 

/usr/lib/python2.4/site-packages/dcacheYaimInstallerTest/logger.py

Can you share the RPM file that causes the problem?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1700132&group_id=5470

From noreply at sourceforge.net  Mon Apr 16 07:34:03 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Sun, 15 Apr 2007 22:34:03 -0700
Subject: [ python-Bugs-1701207 ] svnversion_init assertion failed
Message-ID: <E1HdJql-0001PR-6r@sc8-sf-web24.sourceforge.net>

Bugs item #1701207, was opened at 2007-04-15 17:45
Message generated for change (Comment added) made by nnorwitz
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1701207&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Extension Modules
Group: Python 2.5
Status: Open
Resolution: None
Priority: 9
Private: No
Submitted By: ajrh (ajrh)
Assigned to: Neal Norwitz (nnorwitz)
Summary: svnversion_init assertion failed

Initial Comment:
python: src/python-2.5.1c1/Python/sysmodule.c:1000: svnversion_init: Assertion `len >= 13' failed.

I see these assertions were added here:
http://mail.python.org/pipermail/python-checkins/2006-October/057079.html

Given the form of the subversion headurl string, the assertion doesn't appear to me at first glance to be valid.




----------------------------------------------------------------------

>Comment By: Neal Norwitz (nnorwitz)
Date: 2007-04-15 22:34

Message:
Logged In: YES 
user_id=33168
Originator: NO

Unfortunately I don't recall.  Looking at the code, my guess is that I
added the assertion into the wrong code block.  The next code block down
requires len >= 13.  I'll move it on head, shall I remove it for 2.5.1,
then add it back in the proper place for 2.5.2?

----------------------------------------------------------------------

Comment By: Martin v. L?wis (loewis)
Date: 2007-04-15 22:03

Message:
Logged In: YES 
user_id=21627
Originator: NO

Neal, can you take a look? The assert really does not make sense to me.
How did you arrive at 13?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1701207&group_id=5470

From noreply at sourceforge.net  Mon Apr 16 08:09:19 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Sun, 15 Apr 2007 23:09:19 -0700
Subject: [ python-Bugs-1701207 ] svnversion_init assertion failed
Message-ID: <E1HdKOt-0005wY-4b@sc8-sf-web21.sourceforge.net>

Bugs item #1701207, was opened at 2007-04-15 17:45
Message generated for change (Comment added) made by nnorwitz
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1701207&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Extension Modules
Group: Python 2.5
Status: Open
Resolution: None
Priority: 9
Private: No
Submitted By: ajrh (ajrh)
Assigned to: Neal Norwitz (nnorwitz)
Summary: svnversion_init assertion failed

Initial Comment:
python: src/python-2.5.1c1/Python/sysmodule.c:1000: svnversion_init: Assertion `len >= 13' failed.

I see these assertions were added here:
http://mail.python.org/pipermail/python-checkins/2006-October/057079.html

Given the form of the subversion headurl string, the assertion doesn't appear to me at first glance to be valid.




----------------------------------------------------------------------

>Comment By: Neal Norwitz (nnorwitz)
Date: 2007-04-15 23:09

Message:
Logged In: YES 
user_id=33168
Originator: NO

Anthony agreed that the assert should be removed for 2.5.1.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2007-04-15 22:34

Message:
Logged In: YES 
user_id=33168
Originator: NO

Unfortunately I don't recall.  Looking at the code, my guess is that I
added the assertion into the wrong code block.  The next code block down
requires len >= 13.  I'll move it on head, shall I remove it for 2.5.1,
then add it back in the proper place for 2.5.2?

----------------------------------------------------------------------

Comment By: Martin v. L?wis (loewis)
Date: 2007-04-15 22:03

Message:
Logged In: YES 
user_id=21627
Originator: NO

Neal, can you take a look? The assert really does not make sense to me.
How did you arrive at 13?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1701207&group_id=5470

From noreply at sourceforge.net  Mon Apr 16 08:13:49 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Sun, 15 Apr 2007 23:13:49 -0700
Subject: [ python-Bugs-1701207 ] svnversion_init assertion failed
Message-ID: <E1HdKTF-0006OJ-C0@sc8-sf-web21.sourceforge.net>

Bugs item #1701207, was opened at 2007-04-15 17:45
Message generated for change (Comment added) made by nnorwitz
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1701207&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
>Category: Python Interpreter Core
>Group: Python 2.6
Status: Open
Resolution: None
>Priority: 7
Private: No
Submitted By: ajrh (ajrh)
Assigned to: Neal Norwitz (nnorwitz)
Summary: svnversion_init assertion failed

Initial Comment:
python: src/python-2.5.1c1/Python/sysmodule.c:1000: svnversion_init: Assertion `len >= 13' failed.

I see these assertions were added here:
http://mail.python.org/pipermail/python-checkins/2006-October/057079.html

Given the form of the subversion headurl string, the assertion doesn't appear to me at first glance to be valid.




----------------------------------------------------------------------

>Comment By: Neal Norwitz (nnorwitz)
Date: 2007-04-15 23:13

Message:
Logged In: YES 
user_id=33168
Originator: NO

Committed revision 54836. (2.5) - Remove the 2 assertions.

I still need to fix up head.  I also noticed I botched the second assert. 
- should be +.  I'll test this and verify the fix before fixing head.


----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2007-04-15 23:09

Message:
Logged In: YES 
user_id=33168
Originator: NO

Anthony agreed that the assert should be removed for 2.5.1.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2007-04-15 22:34

Message:
Logged In: YES 
user_id=33168
Originator: NO

Unfortunately I don't recall.  Looking at the code, my guess is that I
added the assertion into the wrong code block.  The next code block down
requires len >= 13.  I'll move it on head, shall I remove it for 2.5.1,
then add it back in the proper place for 2.5.2?

----------------------------------------------------------------------

Comment By: Martin v. L?wis (loewis)
Date: 2007-04-15 22:03

Message:
Logged In: YES 
user_id=21627
Originator: NO

Neal, can you take a look? The assert really does not make sense to me.
How did you arrive at 13?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1701207&group_id=5470

From noreply at sourceforge.net  Mon Apr 16 09:40:24 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Mon, 16 Apr 2007 00:40:24 -0700
Subject: [ python-Bugs-1701207 ] svnversion_init assertion failed
Message-ID: <E1HdLp2-00022U-RQ@sc8-sf-web23.sourceforge.net>

Bugs item #1701207, was opened at 2007-04-15 17:45
Message generated for change (Comment added) made by nnorwitz
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1701207&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Interpreter Core
>Group: Python 2.5
>Status: Closed
Resolution: None
>Priority: 5
Private: No
Submitted By: ajrh (ajrh)
Assigned to: Neal Norwitz (nnorwitz)
Summary: svnversion_init assertion failed

Initial Comment:
python: src/python-2.5.1c1/Python/sysmodule.c:1000: svnversion_init: Assertion `len >= 13' failed.

I see these assertions were added here:
http://mail.python.org/pipermail/python-checkins/2006-October/057079.html

Given the form of the subversion headurl string, the assertion doesn't appear to me at first glance to be valid.




----------------------------------------------------------------------

>Comment By: Neal Norwitz (nnorwitz)
Date: 2007-04-16 00:40

Message:
Logged In: YES 
user_id=33168
Originator: NO

Committed revision 54841. (trunk)

I think I'll just leave this out of 2.5 given how badly I botched this the
first time.

I did test this and ensure that both branches worked this time.  For the
first block, I triggered istag by adding tag to the headurl.  For the
second block I #undef SVNVERSION in Modules/getbuildinfo.c to verify the
exported version works too.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2007-04-15 23:13

Message:
Logged In: YES 
user_id=33168
Originator: NO

Committed revision 54836. (2.5) - Remove the 2 assertions.

I still need to fix up head.  I also noticed I botched the second assert. 
- should be +.  I'll test this and verify the fix before fixing head.


----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2007-04-15 23:09

Message:
Logged In: YES 
user_id=33168
Originator: NO

Anthony agreed that the assert should be removed for 2.5.1.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2007-04-15 22:34

Message:
Logged In: YES 
user_id=33168
Originator: NO

Unfortunately I don't recall.  Looking at the code, my guess is that I
added the assertion into the wrong code block.  The next code block down
requires len >= 13.  I'll move it on head, shall I remove it for 2.5.1,
then add it back in the proper place for 2.5.2?

----------------------------------------------------------------------

Comment By: Martin v. L?wis (loewis)
Date: 2007-04-15 22:03

Message:
Logged In: YES 
user_id=21627
Originator: NO

Neal, can you take a look? The assert really does not make sense to me.
How did you arrive at 13?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1701207&group_id=5470

From noreply at sourceforge.net  Mon Apr 16 12:05:22 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Mon, 16 Apr 2007 03:05:22 -0700
Subject: [ python-Bugs-1701389 ] utf-16 codec problems with multiple file
	append
Message-ID: <E1HdO5K-0004kB-Bh@sc8-sf-web22.sourceforge.net>

Bugs item #1701389, was opened at 2007-04-16 18:05
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1701389&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Unicode
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Iceberg Luo (iceberg4ever)
Assigned to: M.-A. Lemburg (lemburg)
Summary: utf-16 codec problems with multiple file append

Initial Comment:
This bug is similar but not exactly the same as bug215974.  (http://sourceforge.net/tracker/?group_id=5470&atid=105470&aid=215974&func=detail)

In my test, even multiple write() within an open()~close() lifespan will not cause the multi BOM phenomena mentioned in bug215974. Maybe it is because bug 215974 was somehow fixed during the past 7 years, although Lemburg classified it as WontFix. 

However, if a file is appended for more than once, by an "codecs.open('file.txt', 'a', 'utf16')", the multi BOM appears.

At the same time, the saying of "(Extra unnecessary) BOM marks are removed from the input stream by the Python UTF-16 codec" in bug215974 is not true even in today, on Python2.4.4 and Python2.5.1c1 on Windows XP.

Iceberg
------------------

PS: Did not find the "File Upload" checkbox mentioned in this web page, so I think I'd better paste the code right here...

import codecs, os

filename = "test.utf-16"
if os.path.exists(filename): os.unlink(filename)  # reset

def myOpen():
  return codecs.open(filename, "a", 'UTF-16')
def readThemBack():
  return list( codecs.open(filename, "r", 'UTF-16') )
def clumsyPatch(raw): # you can read it after your first run of this program
  for line in raw:
    if line[0] in (u'\ufffe', u'\ufeff'): # get rid of the BOMs
      yield line[1:]
    else:
      yield line

fout = myOpen()
fout.write(u"ab\n") # to simplify the problem, I only use ASCII chars here
fout.write(u"cd\n")
fout.close()
print readThemBack()
assert readThemBack() == [ u'ab\n', u'cd\n' ]
assert os.stat(filename).st_size == 14  # Only one BOM in the file

fout = myOpen()
fout.write(u"ef\n")
fout.write(u"gh\n")
fout.close()
print readThemBack()
#print list( clumsyPatch( readThemBack() ) )  # later you can enable this fix
assert readThemBack() == [ u'ab\n', u'cd\n', u'ef\n', u'gh\n' ] # fails here
assert os.stat(filename).st_size == 26  # not to mention here: multi BOM appears


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1701389&group_id=5470

From noreply at sourceforge.net  Mon Apr 16 12:45:20 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Mon, 16 Apr 2007 03:45:20 -0700
Subject: [ python-Bugs-1701409 ] Segfault in c_char_p of ctypes
Message-ID: <E1HdOi0-0000rY-8b@sc8-sf-web24.sourceforge.net>

Bugs item #1701409, was opened at 2007-04-16 12:45
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1701409&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: sebastien Martini (seb_martini)
Assigned to: Nobody/Anonymous (nobody)
Summary: Segfault in c_char_p of ctypes

Initial Comment:
Hi,

I experienced a segmentation fault (when providing a wrong argument type to c_char_p) in the ctypes module under Linux and with the version 2.5 .


sundae:~$ python
Python 2.5.1c1 (release25-maint, Apr  6 2007, 22:02:36) 
[GCC 4.1.2 (Ubuntu 4.1.2-0ubuntu4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from ctypes import *
>>> c_char_p(42)
zsh: segmentation fault (core dumped)  python
sundae:~$  


Regards,

Sebastien Martini


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1701409&group_id=5470

From noreply at sourceforge.net  Mon Apr 16 14:13:17 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Mon, 16 Apr 2007 05:13:17 -0700
Subject: [ python-Bugs-1701449 ] documentation bug in profile module
Message-ID: <E1HdQ57-0001g2-E9@sc8-sf-web22.sourceforge.net>

Bugs item #1701449, was opened at 2007-04-16 12:13
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1701449&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Documentation
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Keith Briggs (kbriggs)
Assigned to: Nobody/Anonymous (nobody)
Summary: documentation bug in profile module

Initial Comment:
At http://docs.python.org/lib/module-profile.html, it says...

This function takes a single argument that has can be passed to the exec statement

This doesn't make sense.  Should "has" be deleted?


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1701449&group_id=5470

From noreply at sourceforge.net  Mon Apr 16 14:15:55 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Mon, 16 Apr 2007 05:15:55 -0700
Subject: [ python-Bugs-1700455 ] ctypes Fundamental data types
Message-ID: <E1HdQ7f-0006MR-M6@sc8-sf-web23.sourceforge.net>

Bugs item #1700455, was opened at 2007-04-14 03:16
Message generated for change (Comment added) made by kbriggs
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1700455&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Documentation
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: hg6980 (hg6980)
Assigned to: Thomas Heller (theller)
Summary: ctypes Fundamental data types

Initial Comment:
I think the sentence:-



The current memory block contents can be accessed (or changed) with the raw property, if you want to access it as NUL terminated string, use the string property


should be
The current memory block contents can be accessed (or changed) with the raw property, if you want to access it as NUL terminated string, use the value property


----------------------------------------------------------------------

Comment By: Keith Briggs (kbriggs)
Date: 2007-04-16 12:15

Message:
Logged In: YES 
user_id=888261
Originator: NO

That may be technical correct, but such punctuation cannot be right.  It
should be...

The current memory block contents can be accessed (or changed) with the
raw
property; if you want to access it as NUL terminated string, use the
value
property.

----------------------------------------------------------------------

Comment By: Martin v. L?wis (loewis)
Date: 2007-04-14 19:06

Message:
Logged In: YES 
user_id=21627
Originator: NO

Thomas, can you take a look?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1700455&group_id=5470

From noreply at sourceforge.net  Mon Apr 16 14:28:16 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Mon, 16 Apr 2007 05:28:16 -0700
Subject: [ python-Bugs-1701452 ] Feature request: Comparing regexps
Message-ID: <E1HdQJc-0006Jp-Rj@sc8-sf-web21.sourceforge.net>

Bugs item #1701452, was opened at 2007-04-16 14:28
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1701452&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.4
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Thomas Dybdahl Ahle (thomasda)
Assigned to: Nobody/Anonymous (nobody)
Summary: Feature request: Comparing regexps

Initial Comment:
It would be very nice with a function in the re module to compare two regular expressions and see how they overlap.
Return value would perhaps be an a constant like DISJOINT, SUBSET etc.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1701452&group_id=5470

From noreply at sourceforge.net  Mon Apr 16 16:58:45 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Mon, 16 Apr 2007 07:58:45 -0700
Subject: [ python-Feature Requests-1656675 ] Drop-Handler for Python files
Message-ID: <E1HdSfF-0007Xh-Og@sc8-sf-web23.sourceforge.net>

Feature Requests item #1656675, was opened at 2007-02-10 06:36
Message generated for change (Comment added) made by jimjjewett
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1656675&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Installation
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Marek Kubica (leonidasxiv)
Assigned to: Martin v. L?wis (loewis)
Summary: Drop-Handler for Python files

Initial Comment:
Hi!

We had a dscussion about what happens when one drops a file on a python sourcecode-file in the python-forum [1]. It turned out, that nothing happens at all. So someone brought up a solution. It is not really a problem with Python but it would be nice to add this to the Windows Installer.

The proposed solution was to add the following to the registry (this is the format of the registry editor):

Windows Registry Editor Version 5.00

[HKEY_CLASSES_ROOT\Python.File\shellex\DropHandler]
@="{86C86720-42A0-1069-A2E8-08002B30309D}"

[HKEY_CLASSES_ROOT\Python.NoConFile\shellex\DropHandler]
@="{86C86720-42A0-1069-A2E8-08002B30309D}"

[HKEY_CLASSES_ROOT\Python.CompiledFile\shellex\DropHandler]
@="{86C86720-42A0-1069-A2E8-08002B30309D}"

That should be simple thing to do using the MSI-Installer.

[1] only in german, sorry: http://www.python-forum.de/topic-7493.html

----------------------------------------------------------------------

Comment By: Jim Jewett (jimjjewett)
Date: 2007-04-16 10:58

Message:
Logged In: YES 
user_id=764593
Originator: NO

Suggesting close, for insufficient documentation.

----------------------------------------------------------------------

Comment By: Jim Jewett (jimjjewett)
Date: 2007-03-06 18:50

Message:
Logged In: YES 
user_id=764593
Originator: NO

What would happen then?  For example, should it cause python to start up,
running with that file, and the dropped file as stdin?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1656675&group_id=5470

From noreply at sourceforge.net  Mon Apr 16 21:40:23 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Mon, 16 Apr 2007 12:40:23 -0700
Subject: [ python-Bugs-1701409 ] Segfault in c_char_p of ctypes
Message-ID: <E1HdX3n-0007t0-Re@sc8-sf-web23.sourceforge.net>

Bugs item #1701409, was opened at 2007-04-16 12:45
Message generated for change (Comment added) made by seb_martini
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1701409&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: sebastien Martini (seb_martini)
Assigned to: Nobody/Anonymous (nobody)
Summary: Segfault in c_char_p of ctypes

Initial Comment:
Hi,

I experienced a segmentation fault (when providing a wrong argument type to c_char_p) in the ctypes module under Linux and with the version 2.5 .


sundae:~$ python
Python 2.5.1c1 (release25-maint, Apr  6 2007, 22:02:36) 
[GCC 4.1.2 (Ubuntu 4.1.2-0ubuntu4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from ctypes import *
>>> c_char_p(42)
zsh: segmentation fault (core dumped)  python
sundae:~$  


Regards,

Sebastien Martini


----------------------------------------------------------------------

>Comment By: sebastien Martini (seb_martini)
Date: 2007-04-16 21:40

Message:
Logged In: YES 
user_id=1528211
Originator: YES

c_wchar_p also contains this bug.




----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1701409&group_id=5470

From noreply at sourceforge.net  Mon Apr 16 22:15:31 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Mon, 16 Apr 2007 13:15:31 -0700
Subject: [ python-Feature Requests-1701452 ] Feature request: Comparing regexps
Message-ID: <E1HdXbn-0003Gc-Hz@sc8-sf-web21.sourceforge.net>

Feature Requests item #1701452, was opened at 2007-04-16 12:28
Message generated for change (Settings changed) made by gbrandl
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1701452&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
>Category: Python Library
>Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Thomas Dybdahl Ahle (thomasda)
Assigned to: Nobody/Anonymous (nobody)
Summary: Feature request: Comparing regexps

Initial Comment:
It would be very nice with a function in the re module to compare two regular expressions and see how they overlap.
Return value would perhaps be an a constant like DISJOINT, SUBSET etc.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1701452&group_id=5470

From noreply at sourceforge.net  Mon Apr 16 22:43:30 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Mon, 16 Apr 2007 13:43:30 -0700
Subject: [ python-Feature Requests-1701452 ] Feature request: Comparing regexps
Message-ID: <E1HdY2s-0002XW-6Q@sc8-sf-web22.sourceforge.net>

Feature Requests item #1701452, was opened at 2007-04-16 07:28
Message generated for change (Comment added) made by rhettinger
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1701452&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Thomas Dybdahl Ahle (thomasda)
>Assigned to: Gustavo Niemeyer (niemeyer)
Summary: Feature request: Comparing regexps

Initial Comment:
It would be very nice with a function in the re module to compare two regular expressions and see how they overlap.
Return value would perhaps be an a constant like DISJOINT, SUBSET etc.

----------------------------------------------------------------------

>Comment By: Raymond Hettinger (rhettinger)
Date: 2007-04-16 15:43

Message:
Logged In: YES 
user_id=80475
Originator: NO

Can this be done in the existing implementation by comparing the racetrack
diagrams (character transitions)?

Thomas, do you know of anywhere this have been done (third-party modules
or in other languages)?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1701452&group_id=5470

From noreply at sourceforge.net  Mon Apr 16 23:09:26 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Mon, 16 Apr 2007 14:09:26 -0700
Subject: [ python-Feature Requests-1656675 ] Drop-Handler for Python files
Message-ID: <E1HdYRy-0006b5-BU@sc8-sf-web24.sourceforge.net>

Feature Requests item #1656675, was opened at 2007-02-10 12:36
Message generated for change (Comment added) made by loewis
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1656675&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Installation
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Marek Kubica (leonidasxiv)
Assigned to: Martin v. L?wis (loewis)
Summary: Drop-Handler for Python files

Initial Comment:
Hi!

We had a dscussion about what happens when one drops a file on a python sourcecode-file in the python-forum [1]. It turned out, that nothing happens at all. So someone brought up a solution. It is not really a problem with Python but it would be nice to add this to the Windows Installer.

The proposed solution was to add the following to the registry (this is the format of the registry editor):

Windows Registry Editor Version 5.00

[HKEY_CLASSES_ROOT\Python.File\shellex\DropHandler]
@="{86C86720-42A0-1069-A2E8-08002B30309D}"

[HKEY_CLASSES_ROOT\Python.NoConFile\shellex\DropHandler]
@="{86C86720-42A0-1069-A2E8-08002B30309D}"

[HKEY_CLASSES_ROOT\Python.CompiledFile\shellex\DropHandler]
@="{86C86720-42A0-1069-A2E8-08002B30309D}"

That should be simple thing to do using the MSI-Installer.

[1] only in german, sorry: http://www.python-forum.de/topic-7493.html

----------------------------------------------------------------------

>Comment By: Martin v. L?wis (loewis)
Date: 2007-04-16 23:09

Message:
Logged In: YES 
user_id=21627
Originator: NO

I plan to implement this for 2.6; I think I can make sense out of it.

With this setting, dropping file foo.txt on file bar.py will run
"python.exe bar.py foo.txt", through different levels of indirection. So
the file (path) being dropped appears as sys.argv[1].

----------------------------------------------------------------------

Comment By: Jim Jewett (jimjjewett)
Date: 2007-04-16 16:58

Message:
Logged In: YES 
user_id=764593
Originator: NO

Suggesting close, for insufficient documentation.

----------------------------------------------------------------------

Comment By: Jim Jewett (jimjjewett)
Date: 2007-03-07 00:50

Message:
Logged In: YES 
user_id=764593
Originator: NO

What would happen then?  For example, should it cause python to start up,
running with that file, and the dropped file as stdin?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1656675&group_id=5470

From noreply at sourceforge.net  Tue Apr 17 02:42:50 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Mon, 16 Apr 2007 17:42:50 -0700
Subject: [ python-Bugs-1303614 ] Bypassing __dict__ readonlyness
Message-ID: <E1HdbmU-0007jV-Gr@sc8-sf-web23.sourceforge.net>

Bugs item #1303614, was opened at 2005-09-25 01:40
Message generated for change (Comment added) made by zseil
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1303614&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Interpreter Core
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Armin Rigo (arigo)
Assigned to: Nobody/Anonymous (nobody)
Summary: Bypassing __dict__ readonlyness

Initial Comment:
The __dict__ attribute of some objects is read-only,
e.g. for type objects.  However, there is a generic
way to still access and modify it (without hacking with
gc.get_referrents(), that is).  This can lead to
obscure crashes.  Attached is an example that shows
a potential "problem" involving putting strange keys
in the __dict__ of a type.

This is probably very minor anyway.  If we wanted to
fix this, we would need a __dict__ descriptor that
looks more cleverly at the object to which it is
applied.

BTW the first person who understand why the attached
program crashes gets a free coffee.

----------------------------------------------------------------------

>Comment By: Ziga Seilnacht (zseil)
Date: 2007-04-17 02:42

Message:
Logged In: YES 
user_id=1326842
Originator: NO

Here is a patch that shold fix the deldict bug.  It also
removes the new condition for metaclasses, because it
isn't needed anymore, and fixes bug #1174712.  These two
changes were needed so that the patch could be properly
tested.

The patch does what Armin suggested; the __dict__ descriptor
looks for a builtin base with tp_dictoffset != 0, and if one
is found, it delegates to that base's __dict__ descriptor.

File Added: modify_dict_bug.diff

----------------------------------------------------------------------

Comment By: Armin Rigo (arigo)
Date: 2007-02-26 09:23

Message:
Logged In: YES 
user_id=4771
Originator: YES

456?  Now that's interesting.  Not 579?

I'm not sure if I ever thought about what the
expected answer should be, but I'd say that
you are correct: 'x.__dict__' should be empty
in the end.  Actually I don't really see how
it manages *not* to be empty in IronPython;
looks like either a very minor detail or a
deeper bug...

----------------------------------------------------------------------

Comment By: Jeremy Hylton (jhylton)
Date: 2007-02-25 23:36

Message:
Logged In: YES 
user_id=31392
Originator: NO

I tried test67.py in IronPython.  It reports that x.hello has the value
456.  Is that the correct behavior?  It seems incorrect to me.  If the
__eq__ method is called, then the object should no longer have either the
Strange() or hello attributes.  They are cleared by reseting __dict__.  Is
that right?

----------------------------------------------------------------------

Comment By: Jeremy Hylton (jhylton)
Date: 2007-02-25 23:23

Message:
Logged In: YES 
user_id=31392
Originator: NO

I tried test67.py in IronPython.  It reports that x.hello has the value
456.  Is that the correct behavior?  It seems incorrect to me.  If the
__eq__ method is called, then the object should no longer have either the
Strange() or hello attributes.  They are cleared by reseting __dict__.  Is
that right?

----------------------------------------------------------------------

Comment By: Armin Rigo (arigo)
Date: 2006-06-30 09:05

Message:
Logged In: YES 
user_id=4771

Well, try making an "empty" class Foo(object): pass, and see
what magically shows up in Foo.__dict__.keys().  Here is the
__dict__ descriptor used for instances of Foo.  This
descriptor is connected to subtype_dict() and
subtype_setdict() in typeobject.c.

INCREF/DECREFs are in theory missing around every use of the
dictionary returned by _PyObject_GetDictPtr(), because more
or less any such use could remove the dict from the object
from which _PyObject_GetDictPtr() returned from, and so
decref the dict - while it's used.  This might require some
timing, to check the performance impact.

----------------------------------------------------------------------

Comment By: Brett Cannon (bcannon)
Date: 2006-06-30 03:14

Message:
Logged In: YES 
user_id=357491

OK, then I am going to need a little guidance to dive into
this more since this is going into some murky waters for me.  =)

For the normal, non-evil case of an empty Python class
(``class Foo(object): pass``), I would think that accessing
__dict__ would fall through to the tp_getset for object, and
then fall to the same slot for type.  Now that is obviously
not happening since you get a straight dict back for that
attribute access and not a dict proxy as would be suggested
based on my logic listed above.

So, my question is, where is the code that handles fetching
Foo().__dict__?  And if you have an inkling of where I
should be looking in terms of possible refcounting additions
that would be great as well.

----------------------------------------------------------------------

Comment By: Armin Rigo (arigo)
Date: 2006-06-29 23:19

Message:
Logged In: YES 
user_id=4771

Brett: I think you're approaching the problem from the wrong
angle.  The problem is being allowed to freely tamper with
the dict stored in objects.  Getting NULL errors here and
there is only a symptom.  As I explain, the '__dict__'
descriptor object needs to do some more checks, and to be
fully safe some Py_INCREF/Py_DECREF are needed in some
critical places.

----------------------------------------------------------------------

Comment By: Brett Cannon (bcannon)
Date: 2006-06-29 19:45

Message:
Logged In: YES 
user_id=357491

For the deldict.py crasher, if you look at the traceback
there is no good place to do a check that tp_dict is sane
except in type_module() or PyDict_GetItem().  Now adding the
NULL check in type_module() will fix this specific problem,
but that seems like an ad-hoc patch.

Why don't we check for NULL in PyDict_GetItem() and return
NULL just like the PyDict_Check() test?  I am sure the
answer is "performance", but this is not PyDict_GETITEM()and
thus already is not the performance-critical version anyway.
 So why shouldn't we check for NULL there and possibly catch
other errors?

----------------------------------------------------------------------

Comment By: Brett Cannon (bcannon)
Date: 2006-06-29 19:41

Message:
Logged In: YES 
user_id=357491

Simple patch for the loosing_dict_ref.py crasher is
attached.  Just checked for possible NULL values in what is
needed by _Py_ForgetReference().  Let me know what you
think, Armin.

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2005-12-14 16:36

Message:
Logged In: YES 
user_id=6656

Yikes!

----------------------------------------------------------------------

Comment By: Armin Rigo (arigo)
Date: 2005-12-14 16:32

Message:
Logged In: YES 
user_id=4771

A proposed fix for the later crash: the __dict__ descriptor of
user-defined classes should verify if there is another __dict__ descriptor
along the solid path of the type (i.e. following "tp_base" pointers).  If
so, it should delegate to it.

----------------------------------------------------------------------

Comment By: Armin Rigo (arigo)
Date: 2005-12-14 15:49

Message:
Logged In: YES 
user_id=4771

Uh oh.  There is a much simpler crash.  The technique outlined in
deldict.py can be used to rebind or even delete the __dict__ attribute of
instances where it should normally not be allowed.

----------------------------------------------------------------------

Comment By: Armin Rigo (arigo)
Date: 2005-09-25 10:55

Message:
Logged In: YES 
user_id=4771

The bug is related to code like PyObject_GenericGetAttribute and
_PyType_Lookup which are not careful about the reference counters of the
objects they operate on.  This allows a much simpler crash (test67.py): the
__dict__ of an object can be decrefed away while lookdict() is looking at
it.  This has a simple fix -- drop some amount of Py_INCREF/Py_DECREF in
core routines like PyObject_GenericGetAttr.  We probably need to measure if
it has a performance impact, though.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1303614&group_id=5470

From noreply at sourceforge.net  Tue Apr 17 09:51:37 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Tue, 17 Apr 2007 00:51:37 -0700
Subject: [ python-Feature Requests-1701452 ] Feature request: Comparing regexps
Message-ID: <E1HdiTR-0002rV-AA@sc8-sf-web23.sourceforge.net>

Feature Requests item #1701452, was opened at 2007-04-16 14:28
Message generated for change (Comment added) made by thomasda
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1701452&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Thomas Dybdahl Ahle (thomasda)
Assigned to: Gustavo Niemeyer (niemeyer)
Summary: Feature request: Comparing regexps

Initial Comment:
It would be very nice with a function in the re module to compare two regular expressions and see how they overlap.
Return value would perhaps be an a constant like DISJOINT, SUBSET etc.

----------------------------------------------------------------------

>Comment By: Thomas Dybdahl Ahle (thomasda)
Date: 2007-04-17 09:51

Message:
Logged In: YES 
user_id=1304417
Originator: YES

I found this page with the function to do so:
http://terpstra.ca/compare.html

I also found this thread:
http://bumppo.net/lists/fun-with-perl/1999/09/msg00000.html which discusses
how to do this with some canonical formed expressions. 

A function like this would really be able to speed some applications up,
which matches a lot of strings with a number of expressions. If you know
which ones are disjoint, you can break the test when one test matches.

----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2007-04-16 22:43

Message:
Logged In: YES 
user_id=80475
Originator: NO

Can this be done in the existing implementation by comparing the racetrack
diagrams (character transitions)?

Thomas, do you know of anywhere this have been done (third-party modules
or in other languages)?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1701452&group_id=5470

From noreply at sourceforge.net  Tue Apr 17 10:29:04 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Tue, 17 Apr 2007 01:29:04 -0700
Subject: [ python-Bugs-1702036 ] Turtle isn't thread-safe (crashes)
Message-ID: <E1Hdj3g-0001p0-5g@sc8-sf-web22.sourceforge.net>

Bugs item #1702036, was opened at 2007-04-17 08:29
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1702036&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: lomm (korka)
Assigned to: Nobody/Anonymous (nobody)
Summary: Turtle isn't thread-safe (crashes)

Initial Comment:
These are a few examples of errors during code-reentry in the turtle module:

System tested: Windows XP sp2, Python 2.5

1. turtle.circle fails if the tkinter is closed while drawing.

# Code example:
import turtle
turtle.circle(100)
# close the tkinter window while the circle is drawing
# will give an "invalid command name" exception



2. using multiple inheritance, it's possible to draw more than 1 turtle running around at a time. This works part of the time, but crashes python completely on occasions.

# Code example:
import turtle, random
from threading import Thread

class Ninja(Thread, turtle.Turtle):
	'A ninja is a threaded turtle'
	
	def __init__(self):
		# constructors
		Thread.__init__(self)
		turtle.Turtle.__init__(self)
		
		# where will i go?
		self.Direction = random.randint(-180,180)
	
	def run(self):
		# that way!
		self.left(self.Direction)
		
		# march 'forward'
		for i in range(50):
			self.forward(16*random.random())
			self.left(22 - 45*random.random())




ninjas = []
for i in range(3):
	ninjas.append(Ninja())
	ninjas[-1].start()


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1702036&group_id=5470

From noreply at sourceforge.net  Tue Apr 17 17:04:52 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Tue, 17 Apr 2007 08:04:52 -0700
Subject: [ python-Bugs-1627543 ] Status bar on OSX garbled
Message-ID: <E1HdpEi-0004As-Ni@sc8-sf-web24.sourceforge.net>

Bugs item #1627543, was opened at 2007-01-03 23:49
Message generated for change (Comment added) made by gtabares
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1627543&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: IDLE
Group: Platform-specific
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: sigzero (sigzero)
Assigned to: Nobody/Anonymous (nobody)
Summary: Status bar on OSX garbled

Initial Comment:
The way that OSX windows work there is always a resizing handle in the lower right hand corner of windows. The way that IDLE currently does the statusbar is:

|Ln: 13|Col: 4

This cause the Col number to be placed over the resizer.

Something along the lines of:

|Ln: 13|Col: 4|<spaces>

would probably ensure that the resizer is not overlayed.



----------------------------------------------------------------------

Comment By: Gus Tabares (gtabares)
Date: 2007-04-17 10:04

Message:
Logged In: YES 
user_id=834009
Originator: NO

This is only reproducible when invoking IDLE from the Terminal. Invoking
IDLE from the Finder is not affected. Here is the broken code in question:

Line 299 EditorWindow.py
if macosxSupport.runningAsOSXApp():


This will always return False from the Terminal because IDLE.app is not in
sys.argv[0] (see macosxSupport.py line 9).

----------------------------------------------------------------------

Comment By: sigzero (sigzero)
Date: 2007-01-03 23:49

Message:
Logged In: YES 
user_id=1339209
Originator: YES

This is for IDLE 1.1.4 and I am using Python 2.4.4 on OSX Tiger.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1627543&group_id=5470

From noreply at sourceforge.net  Wed Apr 18 01:29:54 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Tue, 17 Apr 2007 16:29:54 -0700
Subject: [ python-Bugs-1702551 ] distutils sdist does not exclude SVN/CVS
	files on Windows
Message-ID: <E1Hdx7S-0004u7-K9@sc8-sf-web21.sourceforge.net>

Bugs item #1702551, was opened at 2007-04-18 01:29
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1702551&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Distutils
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Guy Dalberto (guy-dalberto)
Assigned to: Nobody/Anonymous (nobody)
Summary: distutils sdist does not exclude SVN/CVS files on Windows

Initial Comment:
+ Bug is specific to Windows platform

+ When creating a source distribution with :
**setup.py sdist**

+ the files in CVS/RCS/.svn folders should be     excluded, as specified by paragraph 4.1 of  distutils documentation

+ on windows, they are not excluded, because :
   + the pattern is r'/(RCS|CVS|\.svn)/.*'
   + the filepathes are in the perverse Windows style (separated by backslashes)

+ I could exclude those files by doing either :
  + add a <<global-exclude .svn/*>> in manifest.in
  + modify line 357 of distutils/command/sdist.py

+ initial line
self.filelist.exclude_pattern(r'/(RCS|CVS|\.svn)/.*', is_regex=1)

+ modified lines
reossep = (os.sep == '/' and '/' or r'\\')
self.filelist.exclude_pattern(r'%s(RCS|CVS|\.svn)%s.*' % (reossep, reossep), is_regex=1)


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1702551&group_id=5470

From noreply at sourceforge.net  Wed Apr 18 08:40:13 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Tue, 17 Apr 2007 23:40:13 -0700
Subject: [ python-Feature Requests-1702681 ] Prevent textwrap from breaking
	words at hyphens
Message-ID: <E1He3pt-0003O2-4J@sc8-sf-web23.sourceforge.net>

Feature Requests item #1702681, was opened at 2007-04-17 23:40
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1702681&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Matt Kraai (kraai)
Assigned to: Nobody/Anonymous (nobody)
Summary: Prevent textwrap from breaking words at hyphens

Initial Comment:
I'm using textwrap to wrap text that contains URLs that contain hyphens.  When it wraps these URLs after a hyphen, it breaks the URL.  I wish there was a way to prevent it from doing so.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1702681&group_id=5470

From noreply at sourceforge.net  Wed Apr 18 09:37:17 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Wed, 18 Apr 2007 00:37:17 -0700
Subject: [ python-Bugs-1701409 ] Segfault in c_char_p of ctypes
Message-ID: <E1He4j7-0001wb-F6@sc8-sf-web21.sourceforge.net>

Bugs item #1701409, was opened at 2007-04-16 03:45
Message generated for change (Comment added) made by nnorwitz
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1701409&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
>Category: Extension Modules
Group: None
Status: Open
Resolution: None
>Priority: 7
Private: No
Submitted By: sebastien Martini (seb_martini)
>Assigned to: Thomas Heller (theller)
Summary: Segfault in c_char_p of ctypes

Initial Comment:
Hi,

I experienced a segmentation fault (when providing a wrong argument type to c_char_p) in the ctypes module under Linux and with the version 2.5 .


sundae:~$ python
Python 2.5.1c1 (release25-maint, Apr  6 2007, 22:02:36) 
[GCC 4.1.2 (Ubuntu 4.1.2-0ubuntu4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from ctypes import *
>>> c_char_p(42)
zsh: segmentation fault (core dumped)  python
sundae:~$  


Regards,

Sebastien Martini


----------------------------------------------------------------------

>Comment By: Neal Norwitz (nnorwitz)
Date: 2007-04-18 00:37

Message:
Logged In: YES 
user_id=33168
Originator: NO

Thanks for the bug report Sebastien.

This bug occurs in 2.5 and in the trunk, so it's not a regression. 
Thomas, could you take a look?  Integers are specifically allowed here
(floats are not).  The problem is when trying to print the repr:

#0  PyString_FromString (str=0x2a <Address 0x2a out of bounds>) at
Objects/stringobject.c:108
#1  z_get (ptr=0x2ae1aa9d6450, size=8) at Modules/_ctypes/cfield.c:1373
#2  Simple_get_value (self=0x2ae1aa9d63f8) at
Modules/_ctypes/_ctypes.c:4007
#3  Simple_repr (self=0x2ae1aa9d63f8) at Modules/_ctypes/_ctypes.c:4095
#4  PyObject_Repr (v=0x2ae1aa9d63f8) at Objects/object.c:361


----------------------------------------------------------------------

Comment By: sebastien Martini (seb_martini)
Date: 2007-04-16 12:40

Message:
Logged In: YES 
user_id=1528211
Originator: YES

c_wchar_p also contains this bug.




----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1701409&group_id=5470

From noreply at sourceforge.net  Wed Apr 18 10:04:38 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Wed, 18 Apr 2007 01:04:38 -0700
Subject: [ python-Bugs-1701449 ] documentation bug in profile module
Message-ID: <E1He59a-0006GW-7G@sc8-sf-web24.sourceforge.net>

Bugs item #1701449, was opened at 2007-04-16 21:13
Message generated for change (Comment added) made by quiver
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1701449&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Documentation
Group: Python 2.5
>Status: Closed
>Resolution: Out of Date
Priority: 5
Private: No
Submitted By: Keith Briggs (kbriggs)
Assigned to: Nobody/Anonymous (nobody)
Summary: documentation bug in profile module

Initial Comment:
At http://docs.python.org/lib/module-profile.html, it says...

This function takes a single argument that has can be passed to the exec statement

This doesn't make sense.  Should "has" be deleted?


----------------------------------------------------------------------

>Comment By: George Yoshida (quiver)
Date: 2007-04-18 17:04

Message:
Logged In: YES 
user_id=671362
Originator: NO

This is aleady fixed in the development version of 2.5 and trunk.

With the relase of 2.5.1, the page you specified will be updated.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1701449&group_id=5470

From noreply at sourceforge.net  Wed Apr 18 11:22:54 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Wed, 18 Apr 2007 02:22:54 -0700
Subject: [ python-Bugs-1303614 ] Bypassing __dict__ readonlyness
Message-ID: <E1He6NK-0002XI-6P@sc8-sf-web22.sourceforge.net>

Bugs item #1303614, was opened at 2005-09-24 23:40
Message generated for change (Comment added) made by arigo
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1303614&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Interpreter Core
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Armin Rigo (arigo)
Assigned to: Nobody/Anonymous (nobody)
Summary: Bypassing __dict__ readonlyness

Initial Comment:
The __dict__ attribute of some objects is read-only,
e.g. for type objects.  However, there is a generic
way to still access and modify it (without hacking with
gc.get_referrents(), that is).  This can lead to
obscure crashes.  Attached is an example that shows
a potential "problem" involving putting strange keys
in the __dict__ of a type.

This is probably very minor anyway.  If we wanted to
fix this, we would need a __dict__ descriptor that
looks more cleverly at the object to which it is
applied.

BTW the first person who understand why the attached
program crashes gets a free coffee.

----------------------------------------------------------------------

>Comment By: Armin Rigo (arigo)
Date: 2007-04-18 09:22

Message:
Logged In: YES 
user_id=4771
Originator: YES

I'm skeptical about the whole revision 53997
which introduced not only the unneeded metaclass
condition, but also the strange check when
assigning to __bases__.  I don't think that this
check about __bases__ is correct or relevant or
really fixes any crash.  The link between the
bases and the metaclass of a class is tenuous
anyway: the bases are just used to compute the
metaclass if none is specified explicitly.

As I said on python-dev (with no answer) I am
thinking about reverting r53997 completely.
I'm going to check what I said above a bit more
in-depth first.

----------------------------------------------------------------------

Comment By: Ziga Seilnacht (zseil)
Date: 2007-04-17 00:42

Message:
Logged In: YES 
user_id=1326842
Originator: NO

Here is a patch that shold fix the deldict bug.  It also
removes the new condition for metaclasses, because it
isn't needed anymore, and fixes bug #1174712.  These two
changes were needed so that the patch could be properly
tested.

The patch does what Armin suggested; the __dict__ descriptor
looks for a builtin base with tp_dictoffset != 0, and if one
is found, it delegates to that base's __dict__ descriptor.

File Added: modify_dict_bug.diff

----------------------------------------------------------------------

Comment By: Armin Rigo (arigo)
Date: 2007-02-26 08:23

Message:
Logged In: YES 
user_id=4771
Originator: YES

456?  Now that's interesting.  Not 579?

I'm not sure if I ever thought about what the
expected answer should be, but I'd say that
you are correct: 'x.__dict__' should be empty
in the end.  Actually I don't really see how
it manages *not* to be empty in IronPython;
looks like either a very minor detail or a
deeper bug...

----------------------------------------------------------------------

Comment By: Jeremy Hylton (jhylton)
Date: 2007-02-25 22:36

Message:
Logged In: YES 
user_id=31392
Originator: NO

I tried test67.py in IronPython.  It reports that x.hello has the value
456.  Is that the correct behavior?  It seems incorrect to me.  If the
__eq__ method is called, then the object should no longer have either the
Strange() or hello attributes.  They are cleared by reseting __dict__.  Is
that right?

----------------------------------------------------------------------

Comment By: Jeremy Hylton (jhylton)
Date: 2007-02-25 22:23

Message:
Logged In: YES 
user_id=31392
Originator: NO

I tried test67.py in IronPython.  It reports that x.hello has the value
456.  Is that the correct behavior?  It seems incorrect to me.  If the
__eq__ method is called, then the object should no longer have either the
Strange() or hello attributes.  They are cleared by reseting __dict__.  Is
that right?

----------------------------------------------------------------------

Comment By: Armin Rigo (arigo)
Date: 2006-06-30 07:05

Message:
Logged In: YES 
user_id=4771

Well, try making an "empty" class Foo(object): pass, and see
what magically shows up in Foo.__dict__.keys().  Here is the
__dict__ descriptor used for instances of Foo.  This
descriptor is connected to subtype_dict() and
subtype_setdict() in typeobject.c.

INCREF/DECREFs are in theory missing around every use of the
dictionary returned by _PyObject_GetDictPtr(), because more
or less any such use could remove the dict from the object
from which _PyObject_GetDictPtr() returned from, and so
decref the dict - while it's used.  This might require some
timing, to check the performance impact.

----------------------------------------------------------------------

Comment By: Brett Cannon (bcannon)
Date: 2006-06-30 01:14

Message:
Logged In: YES 
user_id=357491

OK, then I am going to need a little guidance to dive into
this more since this is going into some murky waters for me.  =)

For the normal, non-evil case of an empty Python class
(``class Foo(object): pass``), I would think that accessing
__dict__ would fall through to the tp_getset for object, and
then fall to the same slot for type.  Now that is obviously
not happening since you get a straight dict back for that
attribute access and not a dict proxy as would be suggested
based on my logic listed above.

So, my question is, where is the code that handles fetching
Foo().__dict__?  And if you have an inkling of where I
should be looking in terms of possible refcounting additions
that would be great as well.

----------------------------------------------------------------------

Comment By: Armin Rigo (arigo)
Date: 2006-06-29 21:19

Message:
Logged In: YES 
user_id=4771

Brett: I think you're approaching the problem from the wrong
angle.  The problem is being allowed to freely tamper with
the dict stored in objects.  Getting NULL errors here and
there is only a symptom.  As I explain, the '__dict__'
descriptor object needs to do some more checks, and to be
fully safe some Py_INCREF/Py_DECREF are needed in some
critical places.

----------------------------------------------------------------------

Comment By: Brett Cannon (bcannon)
Date: 2006-06-29 17:45

Message:
Logged In: YES 
user_id=357491

For the deldict.py crasher, if you look at the traceback
there is no good place to do a check that tp_dict is sane
except in type_module() or PyDict_GetItem().  Now adding the
NULL check in type_module() will fix this specific problem,
but that seems like an ad-hoc patch.

Why don't we check for NULL in PyDict_GetItem() and return
NULL just like the PyDict_Check() test?  I am sure the
answer is "performance", but this is not PyDict_GETITEM()and
thus already is not the performance-critical version anyway.
 So why shouldn't we check for NULL there and possibly catch
other errors?

----------------------------------------------------------------------

Comment By: Brett Cannon (bcannon)
Date: 2006-06-29 17:41

Message:
Logged In: YES 
user_id=357491

Simple patch for the loosing_dict_ref.py crasher is
attached.  Just checked for possible NULL values in what is
needed by _Py_ForgetReference().  Let me know what you
think, Armin.

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2005-12-14 15:36

Message:
Logged In: YES 
user_id=6656

Yikes!

----------------------------------------------------------------------

Comment By: Armin Rigo (arigo)
Date: 2005-12-14 15:32

Message:
Logged In: YES 
user_id=4771

A proposed fix for the later crash: the __dict__ descriptor of
user-defined classes should verify if there is another __dict__ descriptor
along the solid path of the type (i.e. following "tp_base" pointers).  If
so, it should delegate to it.

----------------------------------------------------------------------

Comment By: Armin Rigo (arigo)
Date: 2005-12-14 14:49

Message:
Logged In: YES 
user_id=4771

Uh oh.  There is a much simpler crash.  The technique outlined in
deldict.py can be used to rebind or even delete the __dict__ attribute of
instances where it should normally not be allowed.

----------------------------------------------------------------------

Comment By: Armin Rigo (arigo)
Date: 2005-09-25 08:55

Message:
Logged In: YES 
user_id=4771

The bug is related to code like PyObject_GenericGetAttribute and
_PyType_Lookup which are not careful about the reference counters of the
objects they operate on.  This allows a much simpler crash (test67.py): the
__dict__ of an object can be decrefed away while lookdict() is looking at
it.  This has a simple fix -- drop some amount of Py_INCREF/Py_DECREF in
core routines like PyObject_GenericGetAttr.  We probably need to measure if
it has a performance impact, though.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1303614&group_id=5470

From noreply at sourceforge.net  Wed Apr 18 12:05:39 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Wed, 18 Apr 2007 03:05:39 -0700
Subject: [ python-Bugs-1699853 ] locale.getlocale() output fails as
	setlocale() input 
Message-ID: <E1He72h-0000E8-MQ@sc8-sf-web23.sourceforge.net>

Bugs item #1699853, was opened at 2007-04-13 12:26
Message generated for change (Comment added) made by iszegedi
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1699853&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Bernhard Reiter (ber)
Assigned to: Nobody/Anonymous (nobody)
Summary: locale.getlocale() output fails as setlocale() input 

Initial Comment:
This problem report about the locale module
consists of three closely related parts 
(this is why I have decided to put it in one report).
a) the example in the docs is wrong / missleading
b) under some locale settings python as a defect
c) a test case for the locale module, showing b)
   but useful as general start for a test module.

Details:
        a)
        Section example:
                The line
                >>> loc = locale.getlocale(locale.LC_ALL) # get current locale
                contradicts that getlocale should not be called with
                        LC_ALL, as stated in the description of getlocale.
                Suggestion is to change the example to be more useful
                as getting the locale as first action is not really useful.
                It should be "C" anyway which will lead to (None, None)
                so the value is already known. It would make more sense to

                first set the default locale to the user preferences:
import locale
locale.setlocale(locale.LC_ALL,'')
loc = locale.getlocale(locale.LC_NUMERIC)
locale.setlocale(locale.LC_NUMERIC,"C")
# convert a string here
locale.setlocale(locale.LC_NUMERIC, loc)

                _but_ this does not work, see problem b).
                What does work is:

import
locale.setlocale(locale.LC_ALL,'')
loc = locale.setlocale(locale.LC_NUMERIC)
locale.setlocale(locale.LC_NUMERIC,"C")
# convert a string here
locale.setlocale(locale.LC_NUMERIC, loc)

Note that all_loc = locale.setlocale(locale.LC_ALL) might contain
several categories (see attached test_locale.py where I needed to decode
this).
'LC_CTYPE=de_DE.UTF-8;LC_NUMERIC=en_GB.utf8;LC_TIME=de_DE.UTF-8;LC_COLLATE=de_DE.UTF-8;LC_MONETARY=de_DE.UTF-8;LC_MESSAGES=de_DE.UTF-8;LC_PAPER=de_DE.UTF-8;LC_NAME=de_DE.UTF-8;LC_ADDRESS=de_DE.UTF-8;LC_TELEPHONE=de_DE.UTF-8;LC_MEASUREMENT=de_DE.UTF-8;LC_IDENTIFICATION=de_DE.UTF-8'


        b)
                The output of getlocale cannot be used as input to
                setlocale sometimes.
                Works with
                * python2.5 und python2.4 on
                  Debian GNU/Linux Etch ppc, de_DE.utf8.

                I had failures with
                * python2.3, python2.4, python2.5
                  on Debian GNU/Linux Sarge ppc, de_DE at euro
                * Windows XP SP2
                        python-2.4.4.msi    German, see:

                >>> import locale
                >>> result = locale.setlocale(locale.LC_NUMERIC,"")
                >>> print result
                German_Germany.1252
                >>> got = locale.getlocale(locale.LC_NUMERIC)
                >>> print got
                ('de_DE', '1252')
                >>> # works
                ... locale.setlocale(locale.LC_NUMERIC, result)
                'German_Germany.1252'
                >>> # fails
                ... locale.setlocale(locale.LC_NUMERIC, got)
                Traceback (most recent call last):
                  File "<stdin>", line 2, in ?
                  File "C:\Python24\lib\locale.py", line 381, in setlocale
                    return _setlocale(category, locale)
                locale.Error: unsupported locale setting
                >>>



----------------------------------------------------------------------

Comment By: Istvan Szegedi (iszegedi)
Date: 2007-04-18 12:05

Message:
Logged In: YES 
user_id=1772412
Originator: NO

I could reproduce the problem on Fedora Core 5 with Python 2.4.3.

After tracing down the issue, I found the following:

The problem is in locate.py. There is a function called normalize defined
in the locate.py file. This function is invoked by setlocale function if
the incoming locale argument is not a string. (in your example this
condition is true because locale.getlocale function returns a tuple so got
variable is a tuple.) The normalize function is using an encoding_alias
table which results to translate the full locale into an incorrect version.
What happens in my environment is that there is an incoming value
en_us.utf-8  which is converted to en_us.utf and that is the return value
from normalize function. Then _setlocale low level function invoked in
setlocale function throws an exception when it receives en_us.utf  argument
and it is an unsupported locale setting.


This is the original code snippet in locale.py where it is converted in a
wrong way in normalize function:


    # Second try: langname (without encoding)
    code = locale_alias.get(langname, None)
    if code is not None:
        if '.' in code:
            langname, defenc = code.split('.')
        else:
            langname = code
            defenc = ''
        if encoding:
            encoding = encoding_alias.get(encoding, encoding)
        else:
            encoding = defenc
        if encoding:
            return langname + '.' + encoding
        else:
            return langname

    else:
        return localename


To get it fixed, I modified the code in locate.py as follows:


    # Second try: langname (without encoding)
    code = locale_alias.get(langname, None)
    if code is not None:
        if '.' in code:
            langname, defenc = code.split('.')
        else:
            langname = code
            defenc = ''
#        if encoding:
#            encoding = encoding_alias.get(encoding, encoding)
#        else:
#            encoding = defenc
	if encoding is None:
	     encoding = defenc
        if encoding:
            return langname + '.' + encoding
        else:
            return langname

    else:
        return localename


So the effect of encoding_table is skipped. Then your test_locale.py
returns OK.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1699853&group_id=5470

From noreply at sourceforge.net  Wed Apr 18 16:33:24 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Wed, 18 Apr 2007 07:33:24 -0700
Subject: [ python-Feature Requests-1701452 ] Feature request: Comparing regexps
Message-ID: <E1HeBDo-0000qC-SK@sc8-sf-web24.sourceforge.net>

Feature Requests item #1701452, was opened at 2007-04-16 14:28
Message generated for change (Comment added) made by thomasda
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1701452&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Thomas Dybdahl Ahle (thomasda)
Assigned to: Gustavo Niemeyer (niemeyer)
Summary: Feature request: Comparing regexps

Initial Comment:
It would be very nice with a function in the re module to compare two regular expressions and see how they overlap.
Return value would perhaps be an a constant like DISJOINT, SUBSET etc.

----------------------------------------------------------------------

>Comment By: Thomas Dybdahl Ahle (thomasda)
Date: 2007-04-18 16:33

Message:
Logged In: YES 
user_id=1304417
Originator: YES

I talked with the guy who wrote the ZZ comparator.

"""I can give you the source code under the GPL if you like. However, I  
think it would be difficult to port to python. It's 1100 lines of  
very SML-style mostly-uncommented code. Do you know SML?

It's available at svn://mlton.org/mltonlib/trunk/ca/terpstra/regexp.  
I would expect credit for the algorithm. :-) Let me know if you  
succeed in porting it."""

I don't know any SML though.
If anybody does or can write a python equaliant of the algorithm:

"""1. Parse the regular expression (in GNU regular expression syntax)
2. Convert that parse tree into an NFA
3. Convert the NFA into a DFA by an algorithm I invented (it's why I
wrote this program; I thought of the algorithm and found it amusing
to see it in action)

For comparing regular expressions I take the following additional  
steps
4. Combine the two DFA's (with the cross product)
4a. For finding common words, I intersect them
4b. For finding words in A, but not B, I intersect A with the negated
DFA of B
4c. ...
5. Find the minimal DFA corresponding to this intersection
6. Run a weighted depth-first search (similar to Dijkstra's) to find
the least-weighted path from the initial state to an accepting state
(the weighting makes it prefer human readable characters in the
examples)"""

It could be cool.
Otherwise I can also try to learn sml and port it.

----------------------------------------------------------------------

Comment By: Thomas Dybdahl Ahle (thomasda)
Date: 2007-04-17 09:51

Message:
Logged In: YES 
user_id=1304417
Originator: YES

I found this page with the function to do so:
http://terpstra.ca/compare.html

I also found this thread:
http://bumppo.net/lists/fun-with-perl/1999/09/msg00000.html which discusses
how to do this with some canonical formed expressions. 

A function like this would really be able to speed some applications up,
which matches a lot of strings with a number of expressions. If you know
which ones are disjoint, you can break the test when one test matches.

----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2007-04-16 22:43

Message:
Logged In: YES 
user_id=80475
Originator: NO

Can this be done in the existing implementation by comparing the racetrack
diagrams (character transitions)?

Thomas, do you know of anywhere this have been done (third-party modules
or in other languages)?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1701452&group_id=5470

From noreply at sourceforge.net  Wed Apr 18 18:56:39 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Wed, 18 Apr 2007 09:56:39 -0700
Subject: [ python-Bugs-1703110 ] Incorrect example for add_password()
Message-ID: <E1HeDSR-0007GZ-5S@sc8-sf-web23.sourceforge.net>

Bugs item #1703110, was opened at 2007-04-18 12:56
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1703110&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Documentation
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Bob Kline (bkline)
Assigned to: Nobody/Anonymous (nobody)
Summary: Incorrect example for add_password()

Initial Comment:
In the documentation for urllib2, the example [1] for using basic HTTP authentication has the following code for the call to add_password():

auth_handler.add_password('realm', 'host', 'username', 'password')

This actually worked in earlier (pre-2.4.3) versions of urllib2, but even in those older versions, the documentation for HTTPPasswordMgr.add_password() makes it clear that a URI must be passed as the second argument to the method, not a host name.

Even though the behavior in the library method has changed (passing a full URI versions of Python earlier than 2.4.3 didn't work, at least when a non-standard port was specified), the documentation has not changed (the add_password() documentation specifies a URI, even in the pre-2.4.3 versions of Python) and the examples in the documentation for urllib2 still say 'host'.

[1] http://docs.python.org/lib/urllib2-examples.html

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1703110&group_id=5470

From noreply at sourceforge.net  Wed Apr 18 21:20:26 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Wed, 18 Apr 2007 12:20:26 -0700
Subject: [ python-Bugs-1701409 ] Segfault in c_char_p of ctypes
Message-ID: <E1HeFha-0000Sb-Mj@sc8-sf-web24.sourceforge.net>

Bugs item #1701409, was opened at 2007-04-16 12:45
Message generated for change (Comment added) made by theller
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1701409&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Extension Modules
Group: None
Status: Open
Resolution: None
Priority: 7
Private: No
Submitted By: sebastien Martini (seb_martini)
Assigned to: Thomas Heller (theller)
Summary: Segfault in c_char_p of ctypes

Initial Comment:
Hi,

I experienced a segmentation fault (when providing a wrong argument type to c_char_p) in the ctypes module under Linux and with the version 2.5 .


sundae:~$ python
Python 2.5.1c1 (release25-maint, Apr  6 2007, 22:02:36) 
[GCC 4.1.2 (Ubuntu 4.1.2-0ubuntu4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from ctypes import *
>>> c_char_p(42)
zsh: segmentation fault (core dumped)  python
sundae:~$  


Regards,

Sebastien Martini


----------------------------------------------------------------------

>Comment By: Thomas Heller (theller)
Date: 2007-04-18 21:20

Message:
Logged In: YES 
user_id=11105
Originator: NO

This is a difficult issue.  The integer (which is interpreted as address)
is allowed because some libraries use 'char *' pointers initialized to
small, invalid addresses for special purposes.

On windows, printing a c_char_p(42) does not crash, it raises a ValueError
instead:
>>> from ctypes import *
>>> c_char_p(42)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: invalid string pointer 0x00B20B48
>>>

Windows does this by checking if there is a valid string at the address
(see Modules/_ctypes/cfield.c, line 1366) by calling the IsBadStringPointer
api function.  Do other platforms have a function that can do this check?

If not, I'm afraid we would have to give up on the very convenient repr of
'normal' c_char_p instances:

>>> c_char_p("foo bar")
c_char_p('foo bar')
>>>

and only print the address (at least on non-windows).

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2007-04-18 09:37

Message:
Logged In: YES 
user_id=33168
Originator: NO

Thanks for the bug report Sebastien.

This bug occurs in 2.5 and in the trunk, so it's not a regression. 
Thomas, could you take a look?  Integers are specifically allowed here
(floats are not).  The problem is when trying to print the repr:

#0  PyString_FromString (str=0x2a <Address 0x2a out of bounds>) at
Objects/stringobject.c:108
#1  z_get (ptr=0x2ae1aa9d6450, size=8) at Modules/_ctypes/cfield.c:1373
#2  Simple_get_value (self=0x2ae1aa9d63f8) at
Modules/_ctypes/_ctypes.c:4007
#3  Simple_repr (self=0x2ae1aa9d63f8) at Modules/_ctypes/_ctypes.c:4095
#4  PyObject_Repr (v=0x2ae1aa9d63f8) at Objects/object.c:361


----------------------------------------------------------------------

Comment By: sebastien Martini (seb_martini)
Date: 2007-04-16 21:40

Message:
Logged In: YES 
user_id=1528211
Originator: YES

c_wchar_p also contains this bug.




----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1701409&group_id=5470

From noreply at sourceforge.net  Wed Apr 18 21:20:56 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Wed, 18 Apr 2007 12:20:56 -0700
Subject: [ python-Bugs-1124861 ] subprocess fails on GetStdHandle in
	interactive GUI
Message-ID: <E1HeFi4-0000T7-ED@sc8-sf-web24.sourceforge.net>

Bugs item #1124861, was opened at 2005-02-17 16:23
Message generated for change (Comment added) made by grante
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1124861&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Windows
Group: Python 2.4
Status: Closed
Resolution: Fixed
Priority: 7
Private: No
Submitted By: davids (davidschein)
Assigned to: Nobody/Anonymous (nobody)
Summary: subprocess fails on GetStdHandle in interactive GUI

Initial Comment:
Using the suprocess module from with IDLE or PyWindows,
it appears that calls GetStdHandle (STD_<???>_HANDLE)
returns None, which causes an error.  (All appears fine
on Linux, the standard Python command-line, and ipython.)

For example:
>>> import subprocess
>>> p = subprocess.Popen("dir", stdout=subprocess.PIPE)

Traceback (most recent call last):
  File "<pyshell#49>", line 1, in -toplevel-
    p = subprocess.Popen("dir", stdout=subprocess.PIPE)
  File "C:\Python24\lib\subprocess.py", line 545, in
__init__
    (p2cread, p2cwrite,
  File "C:\Python24\lib\subprocess.py", line 605, in
_get_handles
    p2cread = self._make_inheritable(p2cread)
  File "C:\Python24\lib\subprocess.py", line 646, in
_make_inheritable
    DUPLICATE_SAME_ACCESS)
TypeError: an integer is required

The error originates in the mswindows implementation of
_get_handles.  You need to set one of stdin, stdout, or
strerr because the first line in the method is:
if stdin == None and stdout == None and stderr == None:
...return (None, None, None, None, None, None)

I added "if not handle: return GetCurrentProcess()" to
_make_inheritable() as below and it worked.  Of course,
I really do not know what is going on, so I am letting
go now...

def _make_inheritable(self, handle):
..."""Return a duplicate of handle, which is inheritable"""
...if not handle: return GetCurrentProcess()
...return DuplicateHandle(GetCurrentProcess(), handle,
....................................GetCurrentProcess(),
0, 1,
....................................DUPLICATE_SAME_ACCESS)


----------------------------------------------------------------------

Comment By: Grant Edwards (grante)
Date: 2007-04-18 19:20

Message:
Logged In: YES 
user_id=61937
Originator: NO

I _think_ this traceback from a program running under Pythonw 2.4.3
Enthought Edition on WinXP is caused by the same underlying bug (in this
case, stdin was set to PIPE, and stdout/stderr were both left to default):

     File "subprocess.pyc", line 533, in __init__
     File "subprocess.pyc", line 607, in _get_handles
     File "subprocess.pyc", line 634, in _make_inheritable
   WindowsError: [Errno 6] The handle is invalid



----------------------------------------------------------------------

Comment By: Peter ?strand (astrand)
Date: 2007-02-06 15:43

Message:
Logged In: YES 
user_id=344921
Originator: NO

I've applied 1124861.3.patch to both trunk (rev 53646) and the
release25-maint branch (rev 53647). 

----------------------------------------------------------------------

Comment By: Peter ?strand (astrand)
Date: 2007-01-30 20:05

Message:
Logged In: YES 
user_id=344921
Originator: NO

Please review 1124861.3.patch. 

----------------------------------------------------------------------

Comment By: Peter ?strand (astrand)
Date: 2007-01-30 20:04

Message:
Logged In: YES 
user_id=344921
Originator: NO

File Added: 1124861.3.patch

----------------------------------------------------------------------

Comment By: Peter ?strand (astrand)
Date: 2007-01-29 21:42

Message:
Logged In: YES 
user_id=344921
Originator: NO

Some ideas of possible solutions for this bug:

1) As Roger Upole suggests, throw an readable error when GetStdHandle
fails. This would not really change much, besides of subprocess being a
little less confusing. 

2) Automatically create PIPEs for those handles that fails. The PIPE could
either be left open or closed. A WriteFile in the child would get
ERROR_BROKEN_PIPE, if the parent has closed it. Not as good as
ERROR_INVALID_HANDLE, but pretty close. (Or should I say pretty closed?
:-)

3) Try to attach the handles to a NUL device, as 1238747 suggests. 

4) Hope for the best and actually pass invalid handles in
startupinfo.hStdInput, startupinfo.hStdOutput, or
startupinfo.hStdError. It would be nice if this was possible: If
GetStdHandle fails in the current process, it makes sense that GetStdHandle
will fail in the child as well. But, as far as I understand, it's not
possible or safe to pass invalid handles in the startupinfo structure. 

Currently, I'm leaning towards solution 2), with closing the parents PIPE
ends. 

----------------------------------------------------------------------

Comment By: Peter ?strand (astrand)
Date: 2007-01-22 19:36

Message:
Logged In: YES 
user_id=344921
Originator: NO

The following bugs have been marked as duplicate of this bug:

1358527
1603907
1126208
1238747



----------------------------------------------------------------------

Comment By: craig (codecraig)
Date: 2006-10-13 15:54

Message:
Logged In: YES 
user_id=1258995

On windows, this seems to work

from subprocess import *
p = Popen(cmd, stdin=PIPE, stdout=PIPE, stderr=PIPE)

....in some cases (depending on what command you are
executing, a command prompt window may appear).  Do not show
a window use this...

import win32con
p = Popen(cmd, stdin=PIPE, stdout=PIPE, stderr=PIPE,
creationflags=win32con.CREATE_NO_WINDOW)

...google for Microsoft Process Creation Flags for more info

----------------------------------------------------------------------

Comment By: Steven Bethard (bediviere)
Date: 2005-09-26 14:53

Message:
Logged In: YES 
user_id=945502

This issue was discussed on comp.lang.python[1] and Roger
Upole suggested:

"""
Basically, gui apps like VS don't have a console, so
GetStdHandle returns 0.   _subprocess.GetStdHandle
returns None if the handle is 0, which gives the original
error.  Pywin32 just returns the 0, so the process gets
one step further but still hits the above error.

Subprocess.py should probably check the
result of GetStdHandle for None (or 0)
and throw a readable error that says something like
"No standard handle available, you must specify one"
"""

[1]http://mail.python.org/pipermail/python-list/2005-September/300744.html

----------------------------------------------------------------------

Comment By: Steven Bethard (bediviere)
Date: 2005-08-13 20:37

Message:
Logged In: YES 
user_id=945502

I ran into a similar problem in Ellogon (www.ellogon.org)
which interfaces with Python through tclpython
(http://jfontain.free.fr/tclpython.htm).

My current workaround is to always set all of stdin, stdout,
and stderr to subprocess.PIPE.  I never use the stderr pipe,
but at least this keeps the broken GetStdHandle calls from
happening.

Looking at the code, I kinda think the fix should be::

    if handle is None:
        return handle
    return DuplicateHandle(GetCurrentProcess(), ...

where if handle is None, it stays None.  But I'm also
probably in over my head here.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1124861&group_id=5470

From noreply at sourceforge.net  Thu Apr 19 07:02:34 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Wed, 18 Apr 2007 22:02:34 -0700
Subject: [ python-Bugs-1700912 ] questionable results from IDLE, Windows,
	& several built-in 
Message-ID: <E1HeOmw-0002oe-Pf@sc8-sf-web24.sourceforge.net>

Bugs item #1700912, was opened at 2007-04-15 01:28
Message generated for change (Comment added) made by josiahcarlson
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1700912&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Windows
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: bavantgarde314 (bavantgarde314)
Assigned to: Nobody/Anonymous (nobody)
Summary: questionable results from IDLE, Windows, & several built-in 

Initial Comment:
Python 2.5 (r25:51908, Sep 19 2006, 09:52:17) [MSC v.1310 32 bit (Intel)] on win32

Windows(ME)


Attachment reveals questionable results from IDLE, Windows, & several built-in functions.
        
Under IDLE: function swapcase() updates 120 characters.
Under Windows (dos-screen) or a Windows ide swapcase() updates only 52 characters: ord values 65-90 for (uppercase)A-Z and 97-122 for (lowercase)a-z.
                   
Under IDLE: functions lower() & upper() return different results than swapcase()
for hex values:  8a,  8c,  8e,  9a,  9c,  9e,  9f, ff  
 or ord values  138, 140, 142, 154, 156, 158, 159, 255
 or characters   ?    ?    ?    ?    ?    ?    ?    ?




----------------------------------------------------------------------

Comment By: Josiah Carlson (josiahcarlson)
Date: 2007-04-18 22:02

Message:
Logged In: YES 
user_id=341410
Originator: NO

The behaviors of those functions depend on the locale.

Check the results of:

import locale
locale.getlocale()


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1700912&group_id=5470

From noreply at sourceforge.net  Thu Apr 19 07:12:05 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Wed, 18 Apr 2007 22:12:05 -0700
Subject: [ python-Bugs-1702036 ] Turtle isn't thread-safe (crashes)
Message-ID: <E1HeOw9-0002XS-Sn@sc8-sf-web23.sourceforge.net>

Bugs item #1702036, was opened at 2007-04-17 01:29
Message generated for change (Comment added) made by josiahcarlson
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1702036&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: lomm (korka)
Assigned to: Nobody/Anonymous (nobody)
Summary: Turtle isn't thread-safe (crashes)

Initial Comment:
These are a few examples of errors during code-reentry in the turtle module:

System tested: Windows XP sp2, Python 2.5

1. turtle.circle fails if the tkinter is closed while drawing.

# Code example:
import turtle
turtle.circle(100)
# close the tkinter window while the circle is drawing
# will give an "invalid command name" exception



2. using multiple inheritance, it's possible to draw more than 1 turtle running around at a time. This works part of the time, but crashes python completely on occasions.

# Code example:
import turtle, random
from threading import Thread

class Ninja(Thread, turtle.Turtle):
	'A ninja is a threaded turtle'
	
	def __init__(self):
		# constructors
		Thread.__init__(self)
		turtle.Turtle.__init__(self)
		
		# where will i go?
		self.Direction = random.randint(-180,180)
	
	def run(self):
		# that way!
		self.left(self.Direction)
		
		# march 'forward'
		for i in range(50):
			self.forward(16*random.random())
			self.left(22 - 45*random.random())




ninjas = []
for i in range(3):
	ninjas.append(Ninja())
	ninjas[-1].start()


----------------------------------------------------------------------

Comment By: Josiah Carlson (josiahcarlson)
Date: 2007-04-18 22:12

Message:
Logged In: YES 
user_id=341410
Originator: NO

Does Turtle claim to be thread safe in the documentation?  Should we
explicitly state that Turtle is not thread safe?  Should we also say,
"don't close the turtle window while it is drawing"?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1702036&group_id=5470

From noreply at sourceforge.net  Thu Apr 19 07:54:17 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Wed, 18 Apr 2007 22:54:17 -0700
Subject: [ python-Bugs-1703178 ] link_objects in setup.cfg crashes build
Message-ID: <E1HePaz-0000y8-Cw@sc8-sf-web24.sourceforge.net>

Bugs item #1703178, was opened at 2007-04-18 12:03
Message generated for change (Settings changed) made by nnorwitz
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1703178&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
>Category: None
>Group: Python 2.4
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Markus Schatten (mschatte)
Assigned to: Nobody/Anonymous (nobody)
Summary: link_objects in setup.cfg crashes build

Initial Comment:
When I put the following into setup.cfg:

---

[build_ext]
include_dirs = /home/mschatte/myPackages/XSB/config/i686-pc-linux-gnu:/home/mschatte/myPackages/XSB/emu
link_objects = /home/mschatte/myPackages/XSB/config/i686-pc-linux-gnu/saved.o/xsb.o

---

in order to link a c extension to another object which should reside on the users computer, I get the following error (tested on two kubuntu machines, python 2.4):

# python setup.py build
Distribution.parse_config_files():
  reading setup.cfg
options (after parsing config files):
option dict for 'build_ext' command:
  {'include_dirs': ('setup.cfg',
                    '/home/mschatte/myPackages/XSB/config/i686-pc-linux-gnu:/home/mschatte/myPackages/XSB/emu'),
   'link_objects': ('setup.cfg',
                    '/home/mschatte/myPackages/XSB/config/i686-pc-linux-gnu/saved.o/xsb.o')}
options (after parsing command line):
option dict for 'build' command:
  {}
option dict for 'build_ext' command:
  {'include_dirs': ('setup.cfg',
                    '/home/mschatte/myPackages/XSB/config/i686-pc-linux-gnu:/home/mschatte/myPackages/XSB/emu'),
   'link_objects': ('setup.cfg',
                    '/home/mschatte/myPackages/XSB/config/i686-pc-linux-gnu/saved.o/xsb.o')}
running build
Distribution.get_command_obj(): creating 'build' command object
running build_py
Distribution.get_command_obj(): creating 'build_py' command object
creating build
creating build/lib.linux-i686-2.4
copying dbms.py -> build/lib.linux-i686-2.4
copying __init__.py -> build/lib.linux-i686-2.4
copying f.py -> build/lib.linux-i686-2.4
copying interface.py -> build/lib.linux-i686-2.4
copying py2f.py -> build/lib.linux-i686-2.4
copying reasonable.py -> build/lib.linux-i686-2.4
creating build/lib.linux-i686-2.4/xsb_swig
copying xsb_swig/__init__.py -> build/lib.linux-i686-2.4/xsb_swig
copying xsb_swig/xsb.py -> build/lib.linux-i686-2.4/xsb_swig
running build_ext
Distribution.get_command_obj(): creating 'build_ext' command object
  setting options for 'build_ext' command:
    link_objects = /home/mschatte/myPackages/XSB/config/i686-pc-linux-gnu/saved.o/xsb.o (from setup.cfg)
    include_dirs = /home/mschatte/myPackages/XSB/config/i686-pc-linux-gnu:/home/mschatte/myPackages/XSB/emu (from setup.cfg)
building 'xsb_swig._xsb' extension
creating build/temp.linux-i686-2.4
creating build/temp.linux-i686-2.4/xsb_swig
gcc -pthread -fno-strict-aliasing -DNDEBUG -g -O2 -Wall -Wstrict-prototypes -fPIC -I/home/mschatte/myPackages/XSB/config/i686-pc-linux-gnu -I/home/mschatte/myPackages/XSB/emu -I/usr/include/python2.4 -c xsb_swig/swig_xsb_wrap.c -o build/temp.linux-i686-2.4/xsb_swig/swig_xsb_wrap.o
In file included from xsb_swig/swig_xsb.h:25,
                 from xsb_swig/swig_xsb_wrap.c:2346:
/home/mschatte/myPackages/XSB/emu/cinterf.h:259: warning: function declaration isn?t a prototype
/home/mschatte/myPackages/XSB/emu/cinterf.h:262: warning: function declaration isn?t a prototype
/home/mschatte/myPackages/XSB/emu/cinterf.h:285: warning: function declaration isn?t a prototype
/home/mschatte/myPackages/XSB/emu/cinterf.h:286: warning: function declaration isn?t a prototype
/home/mschatte/myPackages/XSB/emu/cinterf.h:294: warning: function declaration isn?t a prototype
/home/mschatte/myPackages/XSB/emu/cinterf.h:296: warning: function declaration isn?t a prototype
/home/mschatte/myPackages/XSB/emu/cinterf.h:300: warning: function declaration isn?t a prototype
/home/mschatte/myPackages/XSB/emu/cinterf.h:304: warning: function declaration isn?t a prototype
/home/mschatte/myPackages/XSB/emu/cinterf.h:305: warning: function declaration isn?t a prototype
In file included from xsb_swig/swig_xsb_wrap.c:2346:
xsb_swig/swig_xsb.h:101: warning: function declaration isn?t a prototype
xsb_swig/swig_xsb.h:135: warning: function declaration isn?t a prototype
xsb_swig/swig_xsb.h:137: warning: function declaration isn?t a prototype
xsb_swig/swig_xsb.h:141: warning: function declaration isn?t a prototype
xsb_swig/swig_xsb.h:145: warning: function declaration isn?t a prototype
xsb_swig/swig_xsb.h:146: warning: function declaration isn?t a prototype
xsb_swig/swig_xsb_wrap.c:2498: warning: function declaration isn?t a prototype
xsb_swig/swig_xsb_wrap.c:5124: warning: function declaration isn?t a prototype
gcc -pthread -fno-strict-aliasing -DNDEBUG -g -O2 -Wall -Wstrict-prototypes -fPIC -I/home/mschatte/myPackages/XSB/config/i686-pc-linux-gnu -I/home/mschatte/myPackages/XSB/emu -I/usr/include/python2.4 -c xsb_swig/swig_xsb.c -o build/temp.linux-i686-2.4/xsb_swig/swig_xsb.o
In file included from xsb_swig/swig_xsb.h:25,
                 from xsb_swig/swig_xsb.c:28:
/home/mschatte/myPackages/XSB/emu/cinterf.h:259: warning: function declaration isn?t a prototype
/home/mschatte/myPackages/XSB/emu/cinterf.h:262: warning: function declaration isn?t a prototype
/home/mschatte/myPackages/XSB/emu/cinterf.h:285: warning: function declaration isn?t a prototype
/home/mschatte/myPackages/XSB/emu/cinterf.h:286: warning: function declaration isn?t a prototype
/home/mschatte/myPackages/XSB/emu/cinterf.h:294: warning: function declaration isn?t a prototype
/home/mschatte/myPackages/XSB/emu/cinterf.h:296: warning: function declaration isn?t a prototype
/home/mschatte/myPackages/XSB/emu/cinterf.h:300: warning: function declaration isn?t a prototype
/home/mschatte/myPackages/XSB/emu/cinterf.h:304: warning: function declaration isn?t a prototype
/home/mschatte/myPackages/XSB/emu/cinterf.h:305: warning: function declaration isn?t a prototype
In file included from xsb_swig/swig_xsb.c:28:
xsb_swig/swig_xsb.h:101: warning: function declaration isn?t a prototype
xsb_swig/swig_xsb.h:135: warning: function declaration isn?t a prototype
xsb_swig/swig_xsb.h:137: warning: function declaration isn?t a prototype
xsb_swig/swig_xsb.h:141: warning: function declaration isn?t a prototype
xsb_swig/swig_xsb.h:145: warning: function declaration isn?t a prototype
xsb_swig/swig_xsb.h:146: warning: function declaration isn?t a prototype
xsb_swig/swig_xsb.c: In function ?intpointer?:
xsb_swig/swig_xsb.c:84: warning: function returns address of local variable
xsb_swig/swig_xsb.c: At top level:
xsb_swig/swig_xsb.c:140: warning: function declaration isn?t a prototype
xsb_swig/swig_xsb.c:174: warning: function declaration isn?t a prototype
xsb_swig/swig_xsb.c:176: warning: function declaration isn?t a prototype
xsb_swig/swig_xsb.c:180: warning: function declaration isn?t a prototype
xsb_swig/swig_xsb.c:184: warning: function declaration isn?t a prototype
xsb_swig/swig_xsb.c:185: warning: function declaration isn?t a prototype
Traceback (most recent call last):
  File "setup.py", line 11, in ?
    [ "xsb_swig/swig_xsb_wrap.c", "xsb_swig/swig_xsb.c" ]
  File "distutils/core.py", line 149, in setup
  File "distutils/dist.py", line 946, in run_commands
  File "distutils/dist.py", line 966, in run_command
  File "distutils/command/build.py", line 112, in run
  File "/usr/lib/python2.4/cmd.py", line 333, in run_command
    del help[cmd]
  File "distutils/dist.py", line 966, in run_command
  File "distutils/command/build_ext.py", line 279, in run
  File "distutils/command/build_ext.py", line 405, in build_extensions
  File "distutils/command/build_ext.py", line 502, in build_extension
  File "distutils/ccompiler.py", line 847, in link_shared_object
  File "distutils/unixccompiler.py", line 211, in link
TypeError: can only concatenate list (not "str") to list

---

maybe it isn't a bug, but link_objects is not well documented, so I wasn't able to find the appropriate syntax for setup.cfg

Best regards,

--
Markus Schatten, dipl. inf.
e-mail: markus.schatten at foi.hr
Faculty of Organization and Informatics
Vara?din, Croatia


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1703178&group_id=5470

From noreply at sourceforge.net  Thu Apr 19 08:37:03 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Wed, 18 Apr 2007 23:37:03 -0700
Subject: [ python-Feature Requests-1702681 ] Prevent textwrap from breaking
	words at hyphens
Message-ID: <E1HeQGN-0005WY-Dg@sc8-sf-web24.sourceforge.net>

Feature Requests item #1702681, was opened at 2007-04-18 01:40
Message generated for change (Comment added) made by rhettinger
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1702681&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Matt Kraai (kraai)
>Assigned to: Greg Ward (gward)
Summary: Prevent textwrap from breaking words at hyphens

Initial Comment:
I'm using textwrap to wrap text that contains URLs that contain hyphens.  When it wraps these URLs after a hyphen, it breaks the URL.  I wish there was a way to prevent it from doing so.

----------------------------------------------------------------------

>Comment By: Raymond Hettinger (rhettinger)
Date: 2007-04-19 01:37

Message:
Logged In: YES 
user_id=80475
Originator: NO

FWIW, one workaround is to "monkey patch" the module:

  textwrap.TextWrapper.wordsep_re =
re.compile(r'(\s+|(?<=[\w\!\"\'\&\.\,\?])-{2,}(?=\w))')

Another workaround is to "hide" the hyphens during wrapping and then
restore them.
  
  def myfill(text, width=70, **kwargs):
     althyphen = chr(127)
     text = text.replace('-', althyphen)
     result = wrap(text, width, **kwargs)
     return result.replace(althyphen, '-')

That being said, I'm +1 on adding a keyword argument treating hyphens as
non-breaking.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1702681&group_id=5470

From noreply at sourceforge.net  Thu Apr 19 08:44:11 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Wed, 18 Apr 2007 23:44:11 -0700
Subject: [ python-Feature Requests-1701452 ] Feature request: Comparing regexps
Message-ID: <E1HeQNH-0002cu-8i@sc8-sf-web22.sourceforge.net>

Feature Requests item #1701452, was opened at 2007-04-16 07:28
Message generated for change (Comment added) made by rhettinger
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1701452&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: None
>Status: Closed
>Resolution: Rejected
Priority: 5
Private: No
Submitted By: Thomas Dybdahl Ahle (thomasda)
Assigned to: Gustavo Niemeyer (niemeyer)
Summary: Feature request: Comparing regexps

Initial Comment:
It would be very nice with a function in the re module to compare two regular expressions and see how they overlap.
Return value would perhaps be an a constant like DISJOINT, SUBSET etc.

----------------------------------------------------------------------

>Comment By: Raymond Hettinger (rhettinger)
Date: 2007-04-19 01:44

Message:
Logged In: YES 
user_id=80475
Originator: NO

Am closing the feature request because it has very little chance of making
it into the core distribution without being a third-party module first. 
The existing implementation is already somewhat difficult to maintain and
we don't want to make that worse.  Also, the new functionality is of
interest to only a small subset of the re module users.

If you do write it, respect the GPL license on the SML code and take some
care to make sure you can license your new code in any way you see fit.



----------------------------------------------------------------------

Comment By: Thomas Dybdahl Ahle (thomasda)
Date: 2007-04-18 09:33

Message:
Logged In: YES 
user_id=1304417
Originator: YES

I talked with the guy who wrote the ZZ comparator.

"""I can give you the source code under the GPL if you like. However, I  
think it would be difficult to port to python. It's 1100 lines of  
very SML-style mostly-uncommented code. Do you know SML?

It's available at svn://mlton.org/mltonlib/trunk/ca/terpstra/regexp.  
I would expect credit for the algorithm. :-) Let me know if you  
succeed in porting it."""

I don't know any SML though.
If anybody does or can write a python equaliant of the algorithm:

"""1. Parse the regular expression (in GNU regular expression syntax)
2. Convert that parse tree into an NFA
3. Convert the NFA into a DFA by an algorithm I invented (it's why I
wrote this program; I thought of the algorithm and found it amusing
to see it in action)

For comparing regular expressions I take the following additional  
steps
4. Combine the two DFA's (with the cross product)
4a. For finding common words, I intersect them
4b. For finding words in A, but not B, I intersect A with the negated
DFA of B
4c. ...
5. Find the minimal DFA corresponding to this intersection
6. Run a weighted depth-first search (similar to Dijkstra's) to find
the least-weighted path from the initial state to an accepting state
(the weighting makes it prefer human readable characters in the
examples)"""

It could be cool.
Otherwise I can also try to learn sml and port it.

----------------------------------------------------------------------

Comment By: Thomas Dybdahl Ahle (thomasda)
Date: 2007-04-17 02:51

Message:
Logged In: YES 
user_id=1304417
Originator: YES

I found this page with the function to do so:
http://terpstra.ca/compare.html

I also found this thread:
http://bumppo.net/lists/fun-with-perl/1999/09/msg00000.html which discusses
how to do this with some canonical formed expressions. 

A function like this would really be able to speed some applications up,
which matches a lot of strings with a number of expressions. If you know
which ones are disjoint, you can break the test when one test matches.

----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2007-04-16 15:43

Message:
Logged In: YES 
user_id=80475
Originator: NO

Can this be done in the existing implementation by comparing the racetrack
diagrams (character transitions)?

Thomas, do you know of anywhere this have been done (third-party modules
or in other languages)?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1701452&group_id=5470

From noreply at sourceforge.net  Thu Apr 19 08:45:47 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Wed, 18 Apr 2007 23:45:47 -0700
Subject: [ python-Bugs-1700821 ] audioop module - request to add a note
Message-ID: <E1HeQOp-0002lB-VP@sc8-sf-web22.sourceforge.net>

Bugs item #1700821, was opened at 2007-04-14 18:07
Message generated for change (Comment added) made by rhettinger
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1700821&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
>Category: Documentation
Group: None
Status: Open
>Resolution: Accepted
Priority: 5
Private: No
Submitted By: ncryptor (ncryptor)
>Assigned to: Raymond Hettinger (rhettinger)
Summary: audioop module - request to add a note

Initial Comment:
The audioop function lin2lin converts sample sizes of linear pcm fragments.
However, when converting to 8-bit, there is a pitfall (that wasted for me a few hours).
In some audio formats, such as .WAV files, 16 and 32 bit samples are signed, but 8 bit samples are unsigned. So when converting from 2 to 1 byte wide samples, you need to also add 128 to the result.
Example code:

new_frames = audioop.lin2lin(frames, old_width, new_width)
if 1 == new_width:
   new_frames = audioop.bias(new_frames, 1, 128)

the same, in reverse, has to be applied when converting from 8 to 16 or 32 bit width samples.

This wasted so much time for me and is hardly documented.


----------------------------------------------------------------------

>Comment By: Raymond Hettinger (rhettinger)
Date: 2007-04-19 01:45

Message:
Logged In: YES 
user_id=80475
Originator: NO

I will add a note.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1700821&group_id=5470

From noreply at sourceforge.net  Thu Apr 19 08:46:43 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Wed, 18 Apr 2007 23:46:43 -0700
Subject: [ python-Feature Requests-1686390 ] csv sniff example
Message-ID: <E1HeQPj-0002ri-Bz@sc8-sf-web22.sourceforge.net>

Feature Requests item #1686390, was opened at 2007-03-22 17:04
Message generated for change (Settings changed) made by rhettinger
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1686390&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Documentation
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Thomas Guettler (guettli)
>Assigned to: Skip Montanaro (montanaro)
Summary: csv sniff example

Initial Comment:
Hi,

I think the csv module misses an example for the
nice sniff method:

import csv
dialect=csv.Sniffer().sniff(open(csvfile).read(1024))
fd=open(csvfile)
for row in csv.reader(fd, dialect):
    for col in row:
        print col,
    print


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1686390&group_id=5470

From noreply at sourceforge.net  Thu Apr 19 08:58:13 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Wed, 18 Apr 2007 23:58:13 -0700
Subject: [ python-Bugs-1701409 ] Segfault in c_char_p of ctypes
Message-ID: <E1HeQar-0006HH-O2@sc8-sf-web23.sourceforge.net>

Bugs item #1701409, was opened at 2007-04-16 03:45
Message generated for change (Comment added) made by nnorwitz
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1701409&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Extension Modules
Group: None
Status: Open
Resolution: None
>Priority: 6
Private: No
Submitted By: sebastien Martini (seb_martini)
Assigned to: Thomas Heller (theller)
Summary: Segfault in c_char_p of ctypes

Initial Comment:
Hi,

I experienced a segmentation fault (when providing a wrong argument type to c_char_p) in the ctypes module under Linux and with the version 2.5 .


sundae:~$ python
Python 2.5.1c1 (release25-maint, Apr  6 2007, 22:02:36) 
[GCC 4.1.2 (Ubuntu 4.1.2-0ubuntu4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from ctypes import *
>>> c_char_p(42)
zsh: segmentation fault (core dumped)  python
sundae:~$  


Regards,

Sebastien Martini


----------------------------------------------------------------------

>Comment By: Neal Norwitz (nnorwitz)
Date: 2007-04-18 23:58

Message:
Logged In: YES 
user_id=33168
Originator: NO

> Do other platforms have a function that can do this check?

There's no platform independent way to do this.

I don't know anything about ctypes, but what might be a way to handle this
is providing a way to convert an integer into a pointer (like a void*). 
Then all these routines could accept this void* type that you would define,
but not accept a raw integer.  That means people would still have access to
do the same thing they can currently do, but it would be clearer that they
are playing with fire by converting any int into a pointer.

There are so many ways to use ctypes to crash, I'm not sure if this is
worth fixing, except perhaps in the docs to point out the issues.

----------------------------------------------------------------------

Comment By: Thomas Heller (theller)
Date: 2007-04-18 12:20

Message:
Logged In: YES 
user_id=11105
Originator: NO

This is a difficult issue.  The integer (which is interpreted as address)
is allowed because some libraries use 'char *' pointers initialized to
small, invalid addresses for special purposes.

On windows, printing a c_char_p(42) does not crash, it raises a ValueError
instead:
>>> from ctypes import *
>>> c_char_p(42)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: invalid string pointer 0x00B20B48
>>>

Windows does this by checking if there is a valid string at the address
(see Modules/_ctypes/cfield.c, line 1366) by calling the IsBadStringPointer
api function.  Do other platforms have a function that can do this check?

If not, I'm afraid we would have to give up on the very convenient repr of
'normal' c_char_p instances:

>>> c_char_p("foo bar")
c_char_p('foo bar')
>>>

and only print the address (at least on non-windows).

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2007-04-18 00:37

Message:
Logged In: YES 
user_id=33168
Originator: NO

Thanks for the bug report Sebastien.

This bug occurs in 2.5 and in the trunk, so it's not a regression. 
Thomas, could you take a look?  Integers are specifically allowed here
(floats are not).  The problem is when trying to print the repr:

#0  PyString_FromString (str=0x2a <Address 0x2a out of bounds>) at
Objects/stringobject.c:108
#1  z_get (ptr=0x2ae1aa9d6450, size=8) at Modules/_ctypes/cfield.c:1373
#2  Simple_get_value (self=0x2ae1aa9d63f8) at
Modules/_ctypes/_ctypes.c:4007
#3  Simple_repr (self=0x2ae1aa9d63f8) at Modules/_ctypes/_ctypes.c:4095
#4  PyObject_Repr (v=0x2ae1aa9d63f8) at Objects/object.c:361


----------------------------------------------------------------------

Comment By: sebastien Martini (seb_martini)
Date: 2007-04-16 12:40

Message:
Logged In: YES 
user_id=1528211
Originator: YES

c_wchar_p also contains this bug.




----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1701409&group_id=5470

From noreply at sourceforge.net  Thu Apr 19 08:58:53 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Wed, 18 Apr 2007 23:58:53 -0700
Subject: [ python-Feature Requests-1702036 ] Turtle isn't thread-safe (crashes)
Message-ID: <E1HeQbV-0006Hy-Tz@sc8-sf-web23.sourceforge.net>

Feature Requests item #1702036, was opened at 2007-04-17 03:29
Message generated for change (Comment added) made by rhettinger
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1702036&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
>Category: Python Library
>Group: Python 2.6
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: lomm (korka)
Assigned to: Nobody/Anonymous (nobody)
Summary: Turtle isn't thread-safe (crashes)

Initial Comment:
These are a few examples of errors during code-reentry in the turtle module:

System tested: Windows XP sp2, Python 2.5

1. turtle.circle fails if the tkinter is closed while drawing.

# Code example:
import turtle
turtle.circle(100)
# close the tkinter window while the circle is drawing
# will give an "invalid command name" exception



2. using multiple inheritance, it's possible to draw more than 1 turtle running around at a time. This works part of the time, but crashes python completely on occasions.

# Code example:
import turtle, random
from threading import Thread

class Ninja(Thread, turtle.Turtle):
	'A ninja is a threaded turtle'
	
	def __init__(self):
		# constructors
		Thread.__init__(self)
		turtle.Turtle.__init__(self)
		
		# where will i go?
		self.Direction = random.randint(-180,180)
	
	def run(self):
		# that way!
		self.left(self.Direction)
		
		# march 'forward'
		for i in range(50):
			self.forward(16*random.random())
			self.left(22 - 45*random.random())




ninjas = []
for i in range(3):
	ninjas.append(Ninja())
	ninjas[-1].start()


----------------------------------------------------------------------

>Comment By: Raymond Hettinger (rhettinger)
Date: 2007-04-19 01:58

Message:
Logged In: YES 
user_id=80475
Originator: NO

As Josiah says, the current docs do not make any promises about
thread-safety.  So, this isn't a bug.

That being said, it would be nice to offer a way to have multiple turtles
running around under the control of different threads.

----------------------------------------------------------------------

Comment By: Josiah Carlson (josiahcarlson)
Date: 2007-04-19 00:12

Message:
Logged In: YES 
user_id=341410
Originator: NO

Does Turtle claim to be thread safe in the documentation?  Should we
explicitly state that Turtle is not thread safe?  Should we also say,
"don't close the turtle window while it is drawing"?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1702036&group_id=5470

From noreply at sourceforge.net  Thu Apr 19 09:00:06 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Thu, 19 Apr 2007 00:00:06 -0700
Subject: [ python-Bugs-1699759 ] pickle example contains errors
Message-ID: <E1HeQcg-0006UQ-Lb@sc8-sf-web23.sourceforge.net>

Bugs item #1699759, was opened at 2007-04-13 02:22
Message generated for change (Settings changed) made by rhettinger
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1699759&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Documentation
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Mark Edgington (edgimar)
>Assigned to: Raymond Hettinger (rhettinger)
Summary: pickle example contains errors

Initial Comment:
In the pickle documentation (see http://docs.python.org/lib/pickle-example.html ), there is an error.  At the end of the page, under the "A sample usage might be..." section, the file for dumping is opened in text-mode instead of binary mode.

This bit me because I was lazy and didn't re-read all of the pickle documentation, but based some code on this snippet.  The problem occurs under certain circumstances when pickling a type 'object' instance (newstyle) under Windows, and then attempting to unpickle it under Linux.  You get the following error: ImportError: No module named copy_reg.  This made no sense to me, and it took a long time to figure out that the problem was due to the mode in which the file was saved (what that has to do with the ImportError I still have no idea...).  If interested, I could attach a test script which is supposed to load the data to a class instance, and two pickle dumps, one which works, and the other which fails.

Perhaps a related suggestion which maybe should be in a different bug is that pickle should check to see when it writing to a filehandle if the filehandle's mode is binary or text, and to issue a warning if it is text.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1699759&group_id=5470

From noreply at sourceforge.net  Thu Apr 19 10:22:36 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Thu, 19 Apr 2007 01:22:36 -0700
Subject: [ python-Bugs-1703448 ] "t.join();
	assert t not in threading.enumerate()" fails
Message-ID: <E1HeRuW-0006pI-IB@sc8-sf-web21.sourceforge.net>

Bugs item #1703448, was opened at 2007-04-19 18:22
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1703448&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Andrew Bennetts (spiv)
Assigned to: Nobody/Anonymous (nobody)
Summary: "t.join(); assert t not in threading.enumerate()" fails

Initial Comment:
This script always fails for me, usually on the very first iteration:

----
import threading

while True:
    print '.',
    t = threading.Thread(target=lambda: None)
    t.start()
    t.join()
    alive_threads = threading.enumerate()
    assert len(alive_threads) == 1, alive_threads
----

The docs clearly say this is expected to work: Thread.join is described as "Wait until the thread terminates" and threading.enumerate's description says it will "Return a list of all currently active Thread objects ... It excludes terminated threads".  So by the time t.join() returns, the thread should not be reported by threading.enumerate anymore.

The bug appears to be that while the thread is shutting down, join() may exit too early: it waits for the __stopped flag to be set, but the thread hasn't necessarily been removed from the _active dict by that time, so enumerate will report a "stopped" thread.  (Or if you like the bug is that __stopped is can be set while the thread is still in the _active dict.)

A workaround is to filter the results of threading.enumerate() through [t for t in threading.enumerate() if t.isAlive()].

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1703448&group_id=5470

From noreply at sourceforge.net  Thu Apr 19 10:24:40 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Thu, 19 Apr 2007 01:24:40 -0700
Subject: [ python-Bugs-1699853 ] locale.getlocale() output fails as
	setlocale() input 
Message-ID: <E1HeRwW-000736-Dz@sc8-sf-web23.sourceforge.net>

Bugs item #1699853, was opened at 2007-04-13 12:26
Message generated for change (Comment added) made by iszegedi
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1699853&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Bernhard Reiter (ber)
Assigned to: Nobody/Anonymous (nobody)
Summary: locale.getlocale() output fails as setlocale() input 

Initial Comment:
This problem report about the locale module
consists of three closely related parts 
(this is why I have decided to put it in one report).
a) the example in the docs is wrong / missleading
b) under some locale settings python as a defect
c) a test case for the locale module, showing b)
   but useful as general start for a test module.

Details:
        a)
        Section example:
                The line
                >>> loc = locale.getlocale(locale.LC_ALL) # get current locale
                contradicts that getlocale should not be called with
                        LC_ALL, as stated in the description of getlocale.
                Suggestion is to change the example to be more useful
                as getting the locale as first action is not really useful.
                It should be "C" anyway which will lead to (None, None)
                so the value is already known. It would make more sense to

                first set the default locale to the user preferences:
import locale
locale.setlocale(locale.LC_ALL,'')
loc = locale.getlocale(locale.LC_NUMERIC)
locale.setlocale(locale.LC_NUMERIC,"C")
# convert a string here
locale.setlocale(locale.LC_NUMERIC, loc)

                _but_ this does not work, see problem b).
                What does work is:

import
locale.setlocale(locale.LC_ALL,'')
loc = locale.setlocale(locale.LC_NUMERIC)
locale.setlocale(locale.LC_NUMERIC,"C")
# convert a string here
locale.setlocale(locale.LC_NUMERIC, loc)

Note that all_loc = locale.setlocale(locale.LC_ALL) might contain
several categories (see attached test_locale.py where I needed to decode
this).
'LC_CTYPE=de_DE.UTF-8;LC_NUMERIC=en_GB.utf8;LC_TIME=de_DE.UTF-8;LC_COLLATE=de_DE.UTF-8;LC_MONETARY=de_DE.UTF-8;LC_MESSAGES=de_DE.UTF-8;LC_PAPER=de_DE.UTF-8;LC_NAME=de_DE.UTF-8;LC_ADDRESS=de_DE.UTF-8;LC_TELEPHONE=de_DE.UTF-8;LC_MEASUREMENT=de_DE.UTF-8;LC_IDENTIFICATION=de_DE.UTF-8'


        b)
                The output of getlocale cannot be used as input to
                setlocale sometimes.
                Works with
                * python2.5 und python2.4 on
                  Debian GNU/Linux Etch ppc, de_DE.utf8.

                I had failures with
                * python2.3, python2.4, python2.5
                  on Debian GNU/Linux Sarge ppc, de_DE at euro
                * Windows XP SP2
                        python-2.4.4.msi    German, see:

                >>> import locale
                >>> result = locale.setlocale(locale.LC_NUMERIC,"")
                >>> print result
                German_Germany.1252
                >>> got = locale.getlocale(locale.LC_NUMERIC)
                >>> print got
                ('de_DE', '1252')
                >>> # works
                ... locale.setlocale(locale.LC_NUMERIC, result)
                'German_Germany.1252'
                >>> # fails
                ... locale.setlocale(locale.LC_NUMERIC, got)
                Traceback (most recent call last):
                  File "<stdin>", line 2, in ?
                  File "C:\Python24\lib\locale.py", line 381, in setlocale
                    return _setlocale(category, locale)
                locale.Error: unsupported locale setting
                >>>



----------------------------------------------------------------------

Comment By: Istvan Szegedi (iszegedi)
Date: 2007-04-19 10:24

Message:
Logged In: YES 
user_id=1772412
Originator: NO

I could reproduce the problem on Fedora Core 5 with Python 2.4.3.

After tracing down the issue, I found the following:

The problem is in locate.py. There is a function called normalize defined
in the locate.py file. This function is invoked by setlocale function if
the incoming locale argument is not a string. (in your example this
condition is true because locale.getlocale function returns a tuple so got
variable is a tuple.) The normalize function is using an encoding_alias
table which results to translate the full locale into an incorrect version.
What happens in my environment is that there is an incoming value
en_us.utf-8  which is converted to en_us.utf and that is the return value
from normalize function. Then _setlocale low level function invoked in
setlocale function throws an exception when it receives en_us.utf  argument
and it is an unsupported locale setting.


This is the original code snippet in locale.py where it is converted in a
wrong way in normalize function:


    # Second try: langname (without encoding)
    code = locale_alias.get(langname, None)
    if code is not None:
        if '.' in code:
            langname, defenc = code.split('.')
        else:
            langname = code
            defenc = ''
        if encoding:
            encoding = encoding_alias.get(encoding, encoding)
        else:
            encoding = defenc
        if encoding:
            return langname + '.' + encoding
        else:
            return langname

    else:
        return localename


To get it fixed, I modified the code in locate.py as follows:


    # Second try: langname (without encoding)
    code = locale_alias.get(langname, None)
    if code is not None:
        if '.' in code:
            langname, defenc = code.split('.')
        else:
            langname = code
            defenc = ''
#        if encoding:
#            encoding = encoding_alias.get(encoding, encoding)
#        else:
#            encoding = defenc
	if encoding is None:
	     encoding = defenc
        if encoding:
            return langname + '.' + encoding
        else:
            return langname

    else:
        return localename


So the effect of encoding_table is skipped. Then your test_locale.py
returns OK.

----------------------------------------------------------------------

Comment By: Istvan Szegedi (iszegedi)
Date: 2007-04-18 12:05

Message:
Logged In: YES 
user_id=1772412
Originator: NO

I could reproduce the problem on Fedora Core 5 with Python 2.4.3.

After tracing down the issue, I found the following:

The problem is in locate.py. There is a function called normalize defined
in the locate.py file. This function is invoked by setlocale function if
the incoming locale argument is not a string. (in your example this
condition is true because locale.getlocale function returns a tuple so got
variable is a tuple.) The normalize function is using an encoding_alias
table which results to translate the full locale into an incorrect version.
What happens in my environment is that there is an incoming value
en_us.utf-8  which is converted to en_us.utf and that is the return value
from normalize function. Then _setlocale low level function invoked in
setlocale function throws an exception when it receives en_us.utf  argument
and it is an unsupported locale setting.


This is the original code snippet in locale.py where it is converted in a
wrong way in normalize function:


    # Second try: langname (without encoding)
    code = locale_alias.get(langname, None)
    if code is not None:
        if '.' in code:
            langname, defenc = code.split('.')
        else:
            langname = code
            defenc = ''
        if encoding:
            encoding = encoding_alias.get(encoding, encoding)
        else:
            encoding = defenc
        if encoding:
            return langname + '.' + encoding
        else:
            return langname

    else:
        return localename


To get it fixed, I modified the code in locate.py as follows:


    # Second try: langname (without encoding)
    code = locale_alias.get(langname, None)
    if code is not None:
        if '.' in code:
            langname, defenc = code.split('.')
        else:
            langname = code
            defenc = ''
#        if encoding:
#            encoding = encoding_alias.get(encoding, encoding)
#        else:
#            encoding = defenc
	if encoding is None:
	     encoding = defenc
        if encoding:
            return langname + '.' + encoding
        else:
            return langname

    else:
        return localename


So the effect of encoding_table is skipped. Then your test_locale.py
returns OK.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1699853&group_id=5470

From noreply at sourceforge.net  Thu Apr 19 10:55:06 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Thu, 19 Apr 2007 01:55:06 -0700
Subject: [ python-Bugs-1699853 ] locale.getlocale() output fails as
	setlocale() input 
Message-ID: <E1HeSPy-0003T9-Ts@sc8-sf-web24.sourceforge.net>

Bugs item #1699853, was opened at 2007-04-13 12:26
Message generated for change (Comment added) made by ber
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1699853&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Bernhard Reiter (ber)
Assigned to: Nobody/Anonymous (nobody)
Summary: locale.getlocale() output fails as setlocale() input 

Initial Comment:
This problem report about the locale module
consists of three closely related parts 
(this is why I have decided to put it in one report).
a) the example in the docs is wrong / missleading
b) under some locale settings python as a defect
c) a test case for the locale module, showing b)
   but useful as general start for a test module.

Details:
        a)
        Section example:
                The line
                >>> loc = locale.getlocale(locale.LC_ALL) # get current locale
                contradicts that getlocale should not be called with
                        LC_ALL, as stated in the description of getlocale.
                Suggestion is to change the example to be more useful
                as getting the locale as first action is not really useful.
                It should be "C" anyway which will lead to (None, None)
                so the value is already known. It would make more sense to

                first set the default locale to the user preferences:
import locale
locale.setlocale(locale.LC_ALL,'')
loc = locale.getlocale(locale.LC_NUMERIC)
locale.setlocale(locale.LC_NUMERIC,"C")
# convert a string here
locale.setlocale(locale.LC_NUMERIC, loc)

                _but_ this does not work, see problem b).
                What does work is:

import
locale.setlocale(locale.LC_ALL,'')
loc = locale.setlocale(locale.LC_NUMERIC)
locale.setlocale(locale.LC_NUMERIC,"C")
# convert a string here
locale.setlocale(locale.LC_NUMERIC, loc)

Note that all_loc = locale.setlocale(locale.LC_ALL) might contain
several categories (see attached test_locale.py where I needed to decode
this).
'LC_CTYPE=de_DE.UTF-8;LC_NUMERIC=en_GB.utf8;LC_TIME=de_DE.UTF-8;LC_COLLATE=de_DE.UTF-8;LC_MONETARY=de_DE.UTF-8;LC_MESSAGES=de_DE.UTF-8;LC_PAPER=de_DE.UTF-8;LC_NAME=de_DE.UTF-8;LC_ADDRESS=de_DE.UTF-8;LC_TELEPHONE=de_DE.UTF-8;LC_MEASUREMENT=de_DE.UTF-8;LC_IDENTIFICATION=de_DE.UTF-8'


        b)
                The output of getlocale cannot be used as input to
                setlocale sometimes.
                Works with
                * python2.5 und python2.4 on
                  Debian GNU/Linux Etch ppc, de_DE.utf8.

                I had failures with
                * python2.3, python2.4, python2.5
                  on Debian GNU/Linux Sarge ppc, de_DE at euro
                * Windows XP SP2
                        python-2.4.4.msi    German, see:

                >>> import locale
                >>> result = locale.setlocale(locale.LC_NUMERIC,"")
                >>> print result
                German_Germany.1252
                >>> got = locale.getlocale(locale.LC_NUMERIC)
                >>> print got
                ('de_DE', '1252')
                >>> # works
                ... locale.setlocale(locale.LC_NUMERIC, result)
                'German_Germany.1252'
                >>> # fails
                ... locale.setlocale(locale.LC_NUMERIC, got)
                Traceback (most recent call last):
                  File "<stdin>", line 2, in ?
                  File "C:\Python24\lib\locale.py", line 381, in setlocale
                    return _setlocale(category, locale)
                locale.Error: unsupported locale setting
                >>>



----------------------------------------------------------------------

>Comment By: Bernhard Reiter (ber)
Date: 2007-04-19 10:55

Message:
Logged In: YES 
user_id=113859
Originator: YES

Istvan,
thanks for looking into this and adding information.
I do not feel competent to judge what the solution would be,
as I do not know the design goals of getlocale().
Given the documentation the function call would only make sense
if setlocale(getlocale(LC_XYZ)) would work in all cases, especially
after the locale has been set to the user preferances with
setlocale(LC_ALL,"")
There is no simple test case that can make sure this is the case.

The workaround for current code for me is to use setlocale(LC_XYZ) only
to ask for the currently set locale and then decipher the string if the
categories have different settings. This workaround can be seen in my
proposed test_case.py. 

I believe next steps could be to get a full overview and check design and
implementation, add some testcases so that more is covered and then fix 
the implementation. We could try to find out who invented getlocale
and ask.

----------------------------------------------------------------------

Comment By: Istvan Szegedi (iszegedi)
Date: 2007-04-19 10:24

Message:
Logged In: YES 
user_id=1772412
Originator: NO

I could reproduce the problem on Fedora Core 5 with Python 2.4.3.

After tracing down the issue, I found the following:

The problem is in locate.py. There is a function called normalize defined
in the locate.py file. This function is invoked by setlocale function if
the incoming locale argument is not a string. (in your example this
condition is true because locale.getlocale function returns a tuple so got
variable is a tuple.) The normalize function is using an encoding_alias
table which results to translate the full locale into an incorrect version.
What happens in my environment is that there is an incoming value
en_us.utf-8  which is converted to en_us.utf and that is the return value
from normalize function. Then _setlocale low level function invoked in
setlocale function throws an exception when it receives en_us.utf  argument
and it is an unsupported locale setting.


This is the original code snippet in locale.py where it is converted in a
wrong way in normalize function:


    # Second try: langname (without encoding)
    code = locale_alias.get(langname, None)
    if code is not None:
        if '.' in code:
            langname, defenc = code.split('.')
        else:
            langname = code
            defenc = ''
        if encoding:
            encoding = encoding_alias.get(encoding, encoding)
        else:
            encoding = defenc
        if encoding:
            return langname + '.' + encoding
        else:
            return langname

    else:
        return localename


To get it fixed, I modified the code in locate.py as follows:


    # Second try: langname (without encoding)
    code = locale_alias.get(langname, None)
    if code is not None:
        if '.' in code:
            langname, defenc = code.split('.')
        else:
            langname = code
            defenc = ''
#        if encoding:
#            encoding = encoding_alias.get(encoding, encoding)
#        else:
#            encoding = defenc
	if encoding is None:
	     encoding = defenc
        if encoding:
            return langname + '.' + encoding
        else:
            return langname

    else:
        return localename


So the effect of encoding_table is skipped. Then your test_locale.py
returns OK.

----------------------------------------------------------------------

Comment By: Istvan Szegedi (iszegedi)
Date: 2007-04-18 12:05

Message:
Logged In: YES 
user_id=1772412
Originator: NO

I could reproduce the problem on Fedora Core 5 with Python 2.4.3.

After tracing down the issue, I found the following:

The problem is in locate.py. There is a function called normalize defined
in the locate.py file. This function is invoked by setlocale function if
the incoming locale argument is not a string. (in your example this
condition is true because locale.getlocale function returns a tuple so got
variable is a tuple.) The normalize function is using an encoding_alias
table which results to translate the full locale into an incorrect version.
What happens in my environment is that there is an incoming value
en_us.utf-8  which is converted to en_us.utf and that is the return value
from normalize function. Then _setlocale low level function invoked in
setlocale function throws an exception when it receives en_us.utf  argument
and it is an unsupported locale setting.


This is the original code snippet in locale.py where it is converted in a
wrong way in normalize function:


    # Second try: langname (without encoding)
    code = locale_alias.get(langname, None)
    if code is not None:
        if '.' in code:
            langname, defenc = code.split('.')
        else:
            langname = code
            defenc = ''
        if encoding:
            encoding = encoding_alias.get(encoding, encoding)
        else:
            encoding = defenc
        if encoding:
            return langname + '.' + encoding
        else:
            return langname

    else:
        return localename


To get it fixed, I modified the code in locate.py as follows:


    # Second try: langname (without encoding)
    code = locale_alias.get(langname, None)
    if code is not None:
        if '.' in code:
            langname, defenc = code.split('.')
        else:
            langname = code
            defenc = ''
#        if encoding:
#            encoding = encoding_alias.get(encoding, encoding)
#        else:
#            encoding = defenc
	if encoding is None:
	     encoding = defenc
        if encoding:
            return langname + '.' + encoding
        else:
            return langname

    else:
        return localename


So the effect of encoding_table is skipped. Then your test_locale.py
returns OK.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1699853&group_id=5470

From noreply at sourceforge.net  Thu Apr 19 11:02:27 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Thu, 19 Apr 2007 02:02:27 -0700
Subject: [ python-Bugs-1669349 ] make install fails if no previous Python
	installation
Message-ID: <E1HeSX5-0002mr-Bj@sc8-sf-web21.sourceforge.net>

Bugs item #1669349, was opened at 2007-02-26 18:22
Message generated for change (Comment added) made by skiolding
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1669349&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Installation
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Matthias S. Benkmann (mbenkmann)
Assigned to: Nobody/Anonymous (nobody)
Summary: make install fails if no previous Python installation

Initial Comment:
When installing Python 2.5 on a completely Python-less system 'make install' failed. The error that caused the failure was

Compiling /usr/lib/python2.5/test/test_multibytecodec.py ...
Sorry: UnicodeError: ("\\N escapes not supported (can't load unicodedata module)",)
[snip]
Compiling /usr/lib/python2.5/zipfile.py ...
make: *** [libinstall] Error 1

'find -name unicodedata.so' showed me that the library did not exist in the build tree. However, after a 'make -i install' to force make to continue despite the error, unicodedata.so was there. So apparently the library is not built until a later stage of make install. And indeed, subsequent 'make install' calls without -i were successful. 

It is important to note that if you have a previous Python installation (at least of the same version), 'make install' will go through, because it'll load the library from there. So if you want to reproduce the issue you will have to install from a freshly unpacked tarball on a system with no Python installation.



----------------------------------------------------------------------

Comment By: Adrian Silva (skiolding)
Date: 2007-04-19 09:02

Message:
Logged In: YES 
user_id=1771816
Originator: NO

The build failure is still present on svn trunk.

I'm using the first workaround posted (i.e. "changing the Makefile to do
sharedinstall before
libinstall in the altinstall target") to build from trunk.

----------------------------------------------------------------------

Comment By: Sebastian Bassi (sbassi)
Date: 2007-04-03 15:50

Message:
Logged In: YES 
user_id=179653
Originator: NO

Another workaround (by Jaroslaw Zabiello):

After executing
./configure
you have to edito
Modules/Setup
file and uncomment the following line:
#zlib zlibmodule.c -I$(prefix)/include -L$(exec_prefix)/lib -lz
Then continue with normal make; make install.

----------------------------------------------------------------------

Comment By: Sebastian Bassi (sbassi)
Date: 2007-04-03 15:05

Message:
Logged In: YES 
user_id=179653
Originator: NO

The same problem arises when installing Python using "make altinstall",
even with another Python installed. 

----------------------------------------------------------------------

Comment By: Simon Percivall (percivall)
Date: 2007-03-21 11:26

Message:
Logged In: YES 
user_id=329382
Originator: NO

It shouldn't be a problem changing the Makefile to do sharedinstall before
libinstall in the altinstall target. I'm guessing that would solve the
problem.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1669349&group_id=5470

From noreply at sourceforge.net  Thu Apr 19 12:18:42 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Thu, 19 Apr 2007 03:18:42 -0700
Subject: [ python-Bugs-1699853 ] locale.getlocale() output fails as
	setlocale() input 
Message-ID: <E1HeTis-0000Ip-La@sc8-sf-web23.sourceforge.net>

Bugs item #1699853, was opened at 2007-04-13 12:26
Message generated for change (Comment added) made by iszegedi
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1699853&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Bernhard Reiter (ber)
Assigned to: Nobody/Anonymous (nobody)
Summary: locale.getlocale() output fails as setlocale() input 

Initial Comment:
This problem report about the locale module
consists of three closely related parts 
(this is why I have decided to put it in one report).
a) the example in the docs is wrong / missleading
b) under some locale settings python as a defect
c) a test case for the locale module, showing b)
   but useful as general start for a test module.

Details:
        a)
        Section example:
                The line
                >>> loc = locale.getlocale(locale.LC_ALL) # get current locale
                contradicts that getlocale should not be called with
                        LC_ALL, as stated in the description of getlocale.
                Suggestion is to change the example to be more useful
                as getting the locale as first action is not really useful.
                It should be "C" anyway which will lead to (None, None)
                so the value is already known. It would make more sense to

                first set the default locale to the user preferences:
import locale
locale.setlocale(locale.LC_ALL,'')
loc = locale.getlocale(locale.LC_NUMERIC)
locale.setlocale(locale.LC_NUMERIC,"C")
# convert a string here
locale.setlocale(locale.LC_NUMERIC, loc)

                _but_ this does not work, see problem b).
                What does work is:

import
locale.setlocale(locale.LC_ALL,'')
loc = locale.setlocale(locale.LC_NUMERIC)
locale.setlocale(locale.LC_NUMERIC,"C")
# convert a string here
locale.setlocale(locale.LC_NUMERIC, loc)

Note that all_loc = locale.setlocale(locale.LC_ALL) might contain
several categories (see attached test_locale.py where I needed to decode
this).
'LC_CTYPE=de_DE.UTF-8;LC_NUMERIC=en_GB.utf8;LC_TIME=de_DE.UTF-8;LC_COLLATE=de_DE.UTF-8;LC_MONETARY=de_DE.UTF-8;LC_MESSAGES=de_DE.UTF-8;LC_PAPER=de_DE.UTF-8;LC_NAME=de_DE.UTF-8;LC_ADDRESS=de_DE.UTF-8;LC_TELEPHONE=de_DE.UTF-8;LC_MEASUREMENT=de_DE.UTF-8;LC_IDENTIFICATION=de_DE.UTF-8'


        b)
                The output of getlocale cannot be used as input to
                setlocale sometimes.
                Works with
                * python2.5 und python2.4 on
                  Debian GNU/Linux Etch ppc, de_DE.utf8.

                I had failures with
                * python2.3, python2.4, python2.5
                  on Debian GNU/Linux Sarge ppc, de_DE at euro
                * Windows XP SP2
                        python-2.4.4.msi    German, see:

                >>> import locale
                >>> result = locale.setlocale(locale.LC_NUMERIC,"")
                >>> print result
                German_Germany.1252
                >>> got = locale.getlocale(locale.LC_NUMERIC)
                >>> print got
                ('de_DE', '1252')
                >>> # works
                ... locale.setlocale(locale.LC_NUMERIC, result)
                'German_Germany.1252'
                >>> # fails
                ... locale.setlocale(locale.LC_NUMERIC, got)
                Traceback (most recent call last):
                  File "<stdin>", line 2, in ?
                  File "C:\Python24\lib\locale.py", line 381, in setlocale
                    return _setlocale(category, locale)
                locale.Error: unsupported locale setting
                >>>



----------------------------------------------------------------------

Comment By: Istvan Szegedi (iszegedi)
Date: 2007-04-19 12:18

Message:
Logged In: YES 
user_id=1772412
Originator: NO

Hi Bernhard,

I absolutely agree with you and I cannot  really judge my correction,
either. It was just a quick and dirty solution to see if it would fix the
problem. In fact, there are other ways to do it as well, like  to modify
the 
encoding_alias table not to translate utf-8 string into utf (and thus to
prevent it to produce an invalid locale setting for _setlocale )

In the locale.py file I found two names mentioned:

Author:  Mark-Andre Lemburg (mal at lemburg.com) 
and Fredrick Lund  (fredrick at pythonware.com) as a modifyier

so it might be a good idea to drop them a mail and ask for their comments.
Do you want to do it or shall I? If you are willing to do it, please, keep
me in the loop.

----------------------------------------------------------------------

Comment By: Bernhard Reiter (ber)
Date: 2007-04-19 10:55

Message:
Logged In: YES 
user_id=113859
Originator: YES

Istvan,
thanks for looking into this and adding information.
I do not feel competent to judge what the solution would be,
as I do not know the design goals of getlocale().
Given the documentation the function call would only make sense
if setlocale(getlocale(LC_XYZ)) would work in all cases, especially
after the locale has been set to the user preferances with
setlocale(LC_ALL,"")
There is no simple test case that can make sure this is the case.

The workaround for current code for me is to use setlocale(LC_XYZ) only
to ask for the currently set locale and then decipher the string if the
categories have different settings. This workaround can be seen in my
proposed test_case.py. 

I believe next steps could be to get a full overview and check design and
implementation, add some testcases so that more is covered and then fix 
the implementation. We could try to find out who invented getlocale
and ask.

----------------------------------------------------------------------

Comment By: Istvan Szegedi (iszegedi)
Date: 2007-04-19 10:24

Message:
Logged In: YES 
user_id=1772412
Originator: NO

I could reproduce the problem on Fedora Core 5 with Python 2.4.3.

After tracing down the issue, I found the following:

The problem is in locate.py. There is a function called normalize defined
in the locate.py file. This function is invoked by setlocale function if
the incoming locale argument is not a string. (in your example this
condition is true because locale.getlocale function returns a tuple so got
variable is a tuple.) The normalize function is using an encoding_alias
table which results to translate the full locale into an incorrect version.
What happens in my environment is that there is an incoming value
en_us.utf-8  which is converted to en_us.utf and that is the return value
from normalize function. Then _setlocale low level function invoked in
setlocale function throws an exception when it receives en_us.utf  argument
and it is an unsupported locale setting.


This is the original code snippet in locale.py where it is converted in a
wrong way in normalize function:


    # Second try: langname (without encoding)
    code = locale_alias.get(langname, None)
    if code is not None:
        if '.' in code:
            langname, defenc = code.split('.')
        else:
            langname = code
            defenc = ''
        if encoding:
            encoding = encoding_alias.get(encoding, encoding)
        else:
            encoding = defenc
        if encoding:
            return langname + '.' + encoding
        else:
            return langname

    else:
        return localename


To get it fixed, I modified the code in locate.py as follows:


    # Second try: langname (without encoding)
    code = locale_alias.get(langname, None)
    if code is not None:
        if '.' in code:
            langname, defenc = code.split('.')
        else:
            langname = code
            defenc = ''
#        if encoding:
#            encoding = encoding_alias.get(encoding, encoding)
#        else:
#            encoding = defenc
	if encoding is None:
	     encoding = defenc
        if encoding:
            return langname + '.' + encoding
        else:
            return langname

    else:
        return localename


So the effect of encoding_table is skipped. Then your test_locale.py
returns OK.

----------------------------------------------------------------------

Comment By: Istvan Szegedi (iszegedi)
Date: 2007-04-18 12:05

Message:
Logged In: YES 
user_id=1772412
Originator: NO

I could reproduce the problem on Fedora Core 5 with Python 2.4.3.

After tracing down the issue, I found the following:

The problem is in locate.py. There is a function called normalize defined
in the locate.py file. This function is invoked by setlocale function if
the incoming locale argument is not a string. (in your example this
condition is true because locale.getlocale function returns a tuple so got
variable is a tuple.) The normalize function is using an encoding_alias
table which results to translate the full locale into an incorrect version.
What happens in my environment is that there is an incoming value
en_us.utf-8  which is converted to en_us.utf and that is the return value
from normalize function. Then _setlocale low level function invoked in
setlocale function throws an exception when it receives en_us.utf  argument
and it is an unsupported locale setting.


This is the original code snippet in locale.py where it is converted in a
wrong way in normalize function:


    # Second try: langname (without encoding)
    code = locale_alias.get(langname, None)
    if code is not None:
        if '.' in code:
            langname, defenc = code.split('.')
        else:
            langname = code
            defenc = ''
        if encoding:
            encoding = encoding_alias.get(encoding, encoding)
        else:
            encoding = defenc
        if encoding:
            return langname + '.' + encoding
        else:
            return langname

    else:
        return localename


To get it fixed, I modified the code in locate.py as follows:


    # Second try: langname (without encoding)
    code = locale_alias.get(langname, None)
    if code is not None:
        if '.' in code:
            langname, defenc = code.split('.')
        else:
            langname = code
            defenc = ''
#        if encoding:
#            encoding = encoding_alias.get(encoding, encoding)
#        else:
#            encoding = defenc
	if encoding is None:
	     encoding = defenc
        if encoding:
            return langname + '.' + encoding
        else:
            return langname

    else:
        return localename


So the effect of encoding_table is skipped. Then your test_locale.py
returns OK.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1699853&group_id=5470

From noreply at sourceforge.net  Thu Apr 19 12:23:34 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Thu, 19 Apr 2007 03:23:34 -0700
Subject: [ python-Bugs-1699853 ] locale.getlocale() output fails as
	setlocale() input 
Message-ID: <E1HeTna-0003HS-VU@sc8-sf-web21.sourceforge.net>

Bugs item #1699853, was opened at 2007-04-13 12:26
Message generated for change (Comment added) made by ber
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1699853&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Bernhard Reiter (ber)
Assigned to: Nobody/Anonymous (nobody)
Summary: locale.getlocale() output fails as setlocale() input 

Initial Comment:
This problem report about the locale module
consists of three closely related parts 
(this is why I have decided to put it in one report).
a) the example in the docs is wrong / missleading
b) under some locale settings python as a defect
c) a test case for the locale module, showing b)
   but useful as general start for a test module.

Details:
        a)
        Section example:
                The line
                >>> loc = locale.getlocale(locale.LC_ALL) # get current locale
                contradicts that getlocale should not be called with
                        LC_ALL, as stated in the description of getlocale.
                Suggestion is to change the example to be more useful
                as getting the locale as first action is not really useful.
                It should be "C" anyway which will lead to (None, None)
                so the value is already known. It would make more sense to

                first set the default locale to the user preferences:
import locale
locale.setlocale(locale.LC_ALL,'')
loc = locale.getlocale(locale.LC_NUMERIC)
locale.setlocale(locale.LC_NUMERIC,"C")
# convert a string here
locale.setlocale(locale.LC_NUMERIC, loc)

                _but_ this does not work, see problem b).
                What does work is:

import
locale.setlocale(locale.LC_ALL,'')
loc = locale.setlocale(locale.LC_NUMERIC)
locale.setlocale(locale.LC_NUMERIC,"C")
# convert a string here
locale.setlocale(locale.LC_NUMERIC, loc)

Note that all_loc = locale.setlocale(locale.LC_ALL) might contain
several categories (see attached test_locale.py where I needed to decode
this).
'LC_CTYPE=de_DE.UTF-8;LC_NUMERIC=en_GB.utf8;LC_TIME=de_DE.UTF-8;LC_COLLATE=de_DE.UTF-8;LC_MONETARY=de_DE.UTF-8;LC_MESSAGES=de_DE.UTF-8;LC_PAPER=de_DE.UTF-8;LC_NAME=de_DE.UTF-8;LC_ADDRESS=de_DE.UTF-8;LC_TELEPHONE=de_DE.UTF-8;LC_MEASUREMENT=de_DE.UTF-8;LC_IDENTIFICATION=de_DE.UTF-8'


        b)
                The output of getlocale cannot be used as input to
                setlocale sometimes.
                Works with
                * python2.5 und python2.4 on
                  Debian GNU/Linux Etch ppc, de_DE.utf8.

                I had failures with
                * python2.3, python2.4, python2.5
                  on Debian GNU/Linux Sarge ppc, de_DE at euro
                * Windows XP SP2
                        python-2.4.4.msi    German, see:

                >>> import locale
                >>> result = locale.setlocale(locale.LC_NUMERIC,"")
                >>> print result
                German_Germany.1252
                >>> got = locale.getlocale(locale.LC_NUMERIC)
                >>> print got
                ('de_DE', '1252')
                >>> # works
                ... locale.setlocale(locale.LC_NUMERIC, result)
                'German_Germany.1252'
                >>> # fails
                ... locale.setlocale(locale.LC_NUMERIC, got)
                Traceback (most recent call last):
                  File "<stdin>", line 2, in ?
                  File "C:\Python24\lib\locale.py", line 381, in setlocale
                    return _setlocale(category, locale)
                locale.Error: unsupported locale setting
                >>>



----------------------------------------------------------------------

>Comment By: Bernhard Reiter (ber)
Date: 2007-04-19 12:23

Message:
Logged In: YES 
user_id=113859
Originator: YES

Feel free to drop them an email, this is a good idea.
Maybe "svn blame" or history inspection produces more names that actually
wrote
the code and the documentation.

----------------------------------------------------------------------

Comment By: Istvan Szegedi (iszegedi)
Date: 2007-04-19 12:18

Message:
Logged In: YES 
user_id=1772412
Originator: NO

Hi Bernhard,

I absolutely agree with you and I cannot  really judge my correction,
either. It was just a quick and dirty solution to see if it would fix the
problem. In fact, there are other ways to do it as well, like  to modify
the 
encoding_alias table not to translate utf-8 string into utf (and thus to
prevent it to produce an invalid locale setting for _setlocale )

In the locale.py file I found two names mentioned:

Author:  Mark-Andre Lemburg (mal at lemburg.com) 
and Fredrick Lund  (fredrick at pythonware.com) as a modifyier

so it might be a good idea to drop them a mail and ask for their comments.
Do you want to do it or shall I? If you are willing to do it, please, keep
me in the loop.

----------------------------------------------------------------------

Comment By: Bernhard Reiter (ber)
Date: 2007-04-19 10:55

Message:
Logged In: YES 
user_id=113859
Originator: YES

Istvan,
thanks for looking into this and adding information.
I do not feel competent to judge what the solution would be,
as I do not know the design goals of getlocale().
Given the documentation the function call would only make sense
if setlocale(getlocale(LC_XYZ)) would work in all cases, especially
after the locale has been set to the user preferances with
setlocale(LC_ALL,"")
There is no simple test case that can make sure this is the case.

The workaround for current code for me is to use setlocale(LC_XYZ) only
to ask for the currently set locale and then decipher the string if the
categories have different settings. This workaround can be seen in my
proposed test_case.py. 

I believe next steps could be to get a full overview and check design and
implementation, add some testcases so that more is covered and then fix 
the implementation. We could try to find out who invented getlocale
and ask.

----------------------------------------------------------------------

Comment By: Istvan Szegedi (iszegedi)
Date: 2007-04-19 10:24

Message:
Logged In: YES 
user_id=1772412
Originator: NO

I could reproduce the problem on Fedora Core 5 with Python 2.4.3.

After tracing down the issue, I found the following:

The problem is in locate.py. There is a function called normalize defined
in the locate.py file. This function is invoked by setlocale function if
the incoming locale argument is not a string. (in your example this
condition is true because locale.getlocale function returns a tuple so got
variable is a tuple.) The normalize function is using an encoding_alias
table which results to translate the full locale into an incorrect version.
What happens in my environment is that there is an incoming value
en_us.utf-8  which is converted to en_us.utf and that is the return value
from normalize function. Then _setlocale low level function invoked in
setlocale function throws an exception when it receives en_us.utf  argument
and it is an unsupported locale setting.


This is the original code snippet in locale.py where it is converted in a
wrong way in normalize function:


    # Second try: langname (without encoding)
    code = locale_alias.get(langname, None)
    if code is not None:
        if '.' in code:
            langname, defenc = code.split('.')
        else:
            langname = code
            defenc = ''
        if encoding:
            encoding = encoding_alias.get(encoding, encoding)
        else:
            encoding = defenc
        if encoding:
            return langname + '.' + encoding
        else:
            return langname

    else:
        return localename


To get it fixed, I modified the code in locate.py as follows:


    # Second try: langname (without encoding)
    code = locale_alias.get(langname, None)
    if code is not None:
        if '.' in code:
            langname, defenc = code.split('.')
        else:
            langname = code
            defenc = ''
#        if encoding:
#            encoding = encoding_alias.get(encoding, encoding)
#        else:
#            encoding = defenc
	if encoding is None:
	     encoding = defenc
        if encoding:
            return langname + '.' + encoding
        else:
            return langname

    else:
        return localename


So the effect of encoding_table is skipped. Then your test_locale.py
returns OK.

----------------------------------------------------------------------

Comment By: Istvan Szegedi (iszegedi)
Date: 2007-04-18 12:05

Message:
Logged In: YES 
user_id=1772412
Originator: NO

I could reproduce the problem on Fedora Core 5 with Python 2.4.3.

After tracing down the issue, I found the following:

The problem is in locate.py. There is a function called normalize defined
in the locate.py file. This function is invoked by setlocale function if
the incoming locale argument is not a string. (in your example this
condition is true because locale.getlocale function returns a tuple so got
variable is a tuple.) The normalize function is using an encoding_alias
table which results to translate the full locale into an incorrect version.
What happens in my environment is that there is an incoming value
en_us.utf-8  which is converted to en_us.utf and that is the return value
from normalize function. Then _setlocale low level function invoked in
setlocale function throws an exception when it receives en_us.utf  argument
and it is an unsupported locale setting.


This is the original code snippet in locale.py where it is converted in a
wrong way in normalize function:


    # Second try: langname (without encoding)
    code = locale_alias.get(langname, None)
    if code is not None:
        if '.' in code:
            langname, defenc = code.split('.')
        else:
            langname = code
            defenc = ''
        if encoding:
            encoding = encoding_alias.get(encoding, encoding)
        else:
            encoding = defenc
        if encoding:
            return langname + '.' + encoding
        else:
            return langname

    else:
        return localename


To get it fixed, I modified the code in locate.py as follows:


    # Second try: langname (without encoding)
    code = locale_alias.get(langname, None)
    if code is not None:
        if '.' in code:
            langname, defenc = code.split('.')
        else:
            langname = code
            defenc = ''
#        if encoding:
#            encoding = encoding_alias.get(encoding, encoding)
#        else:
#            encoding = defenc
	if encoding is None:
	     encoding = defenc
        if encoding:
            return langname + '.' + encoding
        else:
            return langname

    else:
        return localename


So the effect of encoding_table is skipped. Then your test_locale.py
returns OK.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1699853&group_id=5470

From noreply at sourceforge.net  Thu Apr 19 12:30:20 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Thu, 19 Apr 2007 03:30:20 -0700
Subject: [ python-Bugs-1701389 ] utf-16 codec problems with multiple file
	append
Message-ID: <E1HeTu8-0005E6-M9@sc8-sf-web24.sourceforge.net>

Bugs item #1701389, was opened at 2007-04-16 12:05
Message generated for change (Comment added) made by doerwalter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1701389&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Unicode
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Iceberg Luo (iceberg4ever)
Assigned to: M.-A. Lemburg (lemburg)
Summary: utf-16 codec problems with multiple file append

Initial Comment:
This bug is similar but not exactly the same as bug215974.  (http://sourceforge.net/tracker/?group_id=5470&atid=105470&aid=215974&func=detail)

In my test, even multiple write() within an open()~close() lifespan will not cause the multi BOM phenomena mentioned in bug215974. Maybe it is because bug 215974 was somehow fixed during the past 7 years, although Lemburg classified it as WontFix. 

However, if a file is appended for more than once, by an "codecs.open('file.txt', 'a', 'utf16')", the multi BOM appears.

At the same time, the saying of "(Extra unnecessary) BOM marks are removed from the input stream by the Python UTF-16 codec" in bug215974 is not true even in today, on Python2.4.4 and Python2.5.1c1 on Windows XP.

Iceberg
------------------

PS: Did not find the "File Upload" checkbox mentioned in this web page, so I think I'd better paste the code right here...

import codecs, os

filename = "test.utf-16"
if os.path.exists(filename): os.unlink(filename)  # reset

def myOpen():
  return codecs.open(filename, "a", 'UTF-16')
def readThemBack():
  return list( codecs.open(filename, "r", 'UTF-16') )
def clumsyPatch(raw): # you can read it after your first run of this program
  for line in raw:
    if line[0] in (u'\ufffe', u'\ufeff'): # get rid of the BOMs
      yield line[1:]
    else:
      yield line

fout = myOpen()
fout.write(u"ab\n") # to simplify the problem, I only use ASCII chars here
fout.write(u"cd\n")
fout.close()
print readThemBack()
assert readThemBack() == [ u'ab\n', u'cd\n' ]
assert os.stat(filename).st_size == 14  # Only one BOM in the file

fout = myOpen()
fout.write(u"ef\n")
fout.write(u"gh\n")
fout.close()
print readThemBack()
#print list( clumsyPatch( readThemBack() ) )  # later you can enable this fix
assert readThemBack() == [ u'ab\n', u'cd\n', u'ef\n', u'gh\n' ] # fails here
assert os.stat(filename).st_size == 26  # not to mention here: multi BOM appears


----------------------------------------------------------------------

>Comment By: Walter D?rwald (doerwalter)
Date: 2007-04-19 12:30

Message:
Logged In: YES 
user_id=89016
Originator: NO

append mode is simply not supported for codecs. How would the codec find
out the codec state that was active after the last characters where written
to the file?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1701389&group_id=5470

From noreply at sourceforge.net  Thu Apr 19 12:35:59 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Thu, 19 Apr 2007 03:35:59 -0700
Subject: [ python-Bugs-1701389 ] utf-16 codec problems with multiple file
	append
Message-ID: <E1HeTzb-00023j-TK@sc8-sf-web22.sourceforge.net>

Bugs item #1701389, was opened at 2007-04-16 12:05
Message generated for change (Comment added) made by lemburg
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1701389&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Unicode
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Iceberg Luo (iceberg4ever)
>Assigned to: Walter D?rwald (doerwalter)
Summary: utf-16 codec problems with multiple file append

Initial Comment:
This bug is similar but not exactly the same as bug215974.  (http://sourceforge.net/tracker/?group_id=5470&atid=105470&aid=215974&func=detail)

In my test, even multiple write() within an open()~close() lifespan will not cause the multi BOM phenomena mentioned in bug215974. Maybe it is because bug 215974 was somehow fixed during the past 7 years, although Lemburg classified it as WontFix. 

However, if a file is appended for more than once, by an "codecs.open('file.txt', 'a', 'utf16')", the multi BOM appears.

At the same time, the saying of "(Extra unnecessary) BOM marks are removed from the input stream by the Python UTF-16 codec" in bug215974 is not true even in today, on Python2.4.4 and Python2.5.1c1 on Windows XP.

Iceberg
------------------

PS: Did not find the "File Upload" checkbox mentioned in this web page, so I think I'd better paste the code right here...

import codecs, os

filename = "test.utf-16"
if os.path.exists(filename): os.unlink(filename)  # reset

def myOpen():
  return codecs.open(filename, "a", 'UTF-16')
def readThemBack():
  return list( codecs.open(filename, "r", 'UTF-16') )
def clumsyPatch(raw): # you can read it after your first run of this program
  for line in raw:
    if line[0] in (u'\ufffe', u'\ufeff'): # get rid of the BOMs
      yield line[1:]
    else:
      yield line

fout = myOpen()
fout.write(u"ab\n") # to simplify the problem, I only use ASCII chars here
fout.write(u"cd\n")
fout.close()
print readThemBack()
assert readThemBack() == [ u'ab\n', u'cd\n' ]
assert os.stat(filename).st_size == 14  # Only one BOM in the file

fout = myOpen()
fout.write(u"ef\n")
fout.write(u"gh\n")
fout.close()
print readThemBack()
#print list( clumsyPatch( readThemBack() ) )  # later you can enable this fix
assert readThemBack() == [ u'ab\n', u'cd\n', u'ef\n', u'gh\n' ] # fails here
assert os.stat(filename).st_size == 26  # not to mention here: multi BOM appears


----------------------------------------------------------------------

>Comment By: M.-A. Lemburg (lemburg)
Date: 2007-04-19 12:35

Message:
Logged In: YES 
user_id=38388
Originator: NO

I suggest you close this as wont fix.


----------------------------------------------------------------------

Comment By: Walter D?rwald (doerwalter)
Date: 2007-04-19 12:30

Message:
Logged In: YES 
user_id=89016
Originator: NO

append mode is simply not supported for codecs. How would the codec find
out the codec state that was active after the last characters where written
to the file?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1701389&group_id=5470

From noreply at sourceforge.net  Thu Apr 19 13:30:40 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Thu, 19 Apr 2007 04:30:40 -0700
Subject: [ python-Bugs-1701389 ] utf-16 codec problems with multiple file
	append
Message-ID: <E1HeUqW-00087j-O7@sc8-sf-web23.sourceforge.net>

Bugs item #1701389, was opened at 2007-04-16 12:05
Message generated for change (Comment added) made by doerwalter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1701389&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Unicode
Group: Python 2.5
>Status: Closed
>Resolution: Wont Fix
Priority: 5
Private: No
Submitted By: Iceberg Luo (iceberg4ever)
>Assigned to: M.-A. Lemburg (lemburg)
Summary: utf-16 codec problems with multiple file append

Initial Comment:
This bug is similar but not exactly the same as bug215974.  (http://sourceforge.net/tracker/?group_id=5470&atid=105470&aid=215974&func=detail)

In my test, even multiple write() within an open()~close() lifespan will not cause the multi BOM phenomena mentioned in bug215974. Maybe it is because bug 215974 was somehow fixed during the past 7 years, although Lemburg classified it as WontFix. 

However, if a file is appended for more than once, by an "codecs.open('file.txt', 'a', 'utf16')", the multi BOM appears.

At the same time, the saying of "(Extra unnecessary) BOM marks are removed from the input stream by the Python UTF-16 codec" in bug215974 is not true even in today, on Python2.4.4 and Python2.5.1c1 on Windows XP.

Iceberg
------------------

PS: Did not find the "File Upload" checkbox mentioned in this web page, so I think I'd better paste the code right here...

import codecs, os

filename = "test.utf-16"
if os.path.exists(filename): os.unlink(filename)  # reset

def myOpen():
  return codecs.open(filename, "a", 'UTF-16')
def readThemBack():
  return list( codecs.open(filename, "r", 'UTF-16') )
def clumsyPatch(raw): # you can read it after your first run of this program
  for line in raw:
    if line[0] in (u'\ufffe', u'\ufeff'): # get rid of the BOMs
      yield line[1:]
    else:
      yield line

fout = myOpen()
fout.write(u"ab\n") # to simplify the problem, I only use ASCII chars here
fout.write(u"cd\n")
fout.close()
print readThemBack()
assert readThemBack() == [ u'ab\n', u'cd\n' ]
assert os.stat(filename).st_size == 14  # Only one BOM in the file

fout = myOpen()
fout.write(u"ef\n")
fout.write(u"gh\n")
fout.close()
print readThemBack()
#print list( clumsyPatch( readThemBack() ) )  # later you can enable this fix
assert readThemBack() == [ u'ab\n', u'cd\n', u'ef\n', u'gh\n' ] # fails here
assert os.stat(filename).st_size == 26  # not to mention here: multi BOM appears


----------------------------------------------------------------------

>Comment By: Walter D?rwald (doerwalter)
Date: 2007-04-19 13:30

Message:
Logged In: YES 
user_id=89016
Originator: NO

Closing as "won't fix"

----------------------------------------------------------------------

Comment By: M.-A. Lemburg (lemburg)
Date: 2007-04-19 12:35

Message:
Logged In: YES 
user_id=38388
Originator: NO

I suggest you close this as wont fix.


----------------------------------------------------------------------

Comment By: Walter D?rwald (doerwalter)
Date: 2007-04-19 12:30

Message:
Logged In: YES 
user_id=89016
Originator: NO

append mode is simply not supported for codecs. How would the codec find
out the codec state that was active after the last characters where written
to the file?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1701389&group_id=5470

From noreply at sourceforge.net  Thu Apr 19 15:35:35 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Thu, 19 Apr 2007 06:35:35 -0700
Subject: [ python-Bugs-1703592 ] have a way to ignore nonexisting locales in
	locale.setlocale
Message-ID: <E1HeWnP-0007sA-Vp@sc8-sf-web21.sourceforge.net>

Bugs item #1703592, was opened at 2007-04-19 15:35
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1703592&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Extension Modules
Group: Python 2.6
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Matthias Klose (doko)
Assigned to: Martin v. L?wis (loewis)
Summary: have a way to ignore nonexisting locales in locale.setlocale

Initial Comment:
this came up on #ubuntu-devel; Debian and Ubuntu do not generate all libc locales by default, so it is likely that 

$ LC_ALL=en_US.UTF-8 python -c "import locale; locale.setlocale(locale.LC_ALL, '')"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/lib/python2.5/locale.py", line 476, in setlocale
    return _setlocale(category, locale)
locale.Error: unsupported locale setting

fails on a system (often experienced ssh'ing into a server system where your current locale doesn't exist). Examples for bug reports in various applications are:

  https://launchpad.net/bugs/91583 (apt-listchanges)
  https://launchpad.net/bugs/88638 (pitivi)
  https://launchpad.net/bugs/81556 (exaile)
  https://launchpad.net/bugs/90525 (hwdb-client)

In C, the result of the setlocale(3) call can be, and usually is ignored, if the locale cannot be set.

It is argued that the Python interface for locale.setlocale() should not raise an exception by default, if the locale is missing on the system.

That would be an behaviour change of locale.setlocale(), so the original behavour should be kept as an option (introducing an optional third parameter to raise the exception when the locale doesn't exist). Is this an appropriate change, or could another change be suggested?


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1703592&group_id=5470

From noreply at sourceforge.net  Thu Apr 19 15:42:22 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Thu, 19 Apr 2007 06:42:22 -0700
Subject: [ python-Bugs-1703448 ] "t.join();
	assert t not in threading.enumerate()" fails
Message-ID: <E1HeWty-0005Zp-O9@sc8-sf-web23.sourceforge.net>

Bugs item #1703448, was opened at 2007-04-19 03:22
Message generated for change (Settings changed) made by rhettinger
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1703448&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Andrew Bennetts (spiv)
>Assigned to: Raymond Hettinger (rhettinger)
Summary: "t.join(); assert t not in threading.enumerate()" fails

Initial Comment:
This script always fails for me, usually on the very first iteration:

----
import threading

while True:
    print '.',
    t = threading.Thread(target=lambda: None)
    t.start()
    t.join()
    alive_threads = threading.enumerate()
    assert len(alive_threads) == 1, alive_threads
----

The docs clearly say this is expected to work: Thread.join is described as "Wait until the thread terminates" and threading.enumerate's description says it will "Return a list of all currently active Thread objects ... It excludes terminated threads".  So by the time t.join() returns, the thread should not be reported by threading.enumerate anymore.

The bug appears to be that while the thread is shutting down, join() may exit too early: it waits for the __stopped flag to be set, but the thread hasn't necessarily been removed from the _active dict by that time, so enumerate will report a "stopped" thread.  (Or if you like the bug is that __stopped is can be set while the thread is still in the _active dict.)

A workaround is to filter the results of threading.enumerate() through [t for t in threading.enumerate() if t.isAlive()].

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1703448&group_id=5470

From noreply at sourceforge.net  Thu Apr 19 16:46:15 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Thu, 19 Apr 2007 07:46:15 -0700
Subject: [ python-Bugs-1303614 ] Bypassing __dict__ readonlyness
Message-ID: <E1HeXtn-0006Y2-Ta@sc8-sf-web24.sourceforge.net>

Bugs item #1303614, was opened at 2005-09-24 23:40
Message generated for change (Comment added) made by arigo
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1303614&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Interpreter Core
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Armin Rigo (arigo)
Assigned to: Nobody/Anonymous (nobody)
Summary: Bypassing __dict__ readonlyness

Initial Comment:
The __dict__ attribute of some objects is read-only,
e.g. for type objects.  However, there is a generic
way to still access and modify it (without hacking with
gc.get_referrents(), that is).  This can lead to
obscure crashes.  Attached is an example that shows
a potential "problem" involving putting strange keys
in the __dict__ of a type.

This is probably very minor anyway.  If we wanted to
fix this, we would need a __dict__ descriptor that
looks more cleverly at the object to which it is
applied.

BTW the first person who understand why the attached
program crashes gets a free coffee.

----------------------------------------------------------------------

>Comment By: Armin Rigo (arigo)
Date: 2007-04-19 14:46

Message:
Logged In: YES 
user_id=4771
Originator: YES

Reverted r53997 in revision 54875.

zseil, could you update your patch for the new
svn head?  Thanks!  It should mostly be a matter
of simplifying it.

----------------------------------------------------------------------

Comment By: Armin Rigo (arigo)
Date: 2007-04-18 09:22

Message:
Logged In: YES 
user_id=4771
Originator: YES

I'm skeptical about the whole revision 53997
which introduced not only the unneeded metaclass
condition, but also the strange check when
assigning to __bases__.  I don't think that this
check about __bases__ is correct or relevant or
really fixes any crash.  The link between the
bases and the metaclass of a class is tenuous
anyway: the bases are just used to compute the
metaclass if none is specified explicitly.

As I said on python-dev (with no answer) I am
thinking about reverting r53997 completely.
I'm going to check what I said above a bit more
in-depth first.

----------------------------------------------------------------------

Comment By: Ziga Seilnacht (zseil)
Date: 2007-04-17 00:42

Message:
Logged In: YES 
user_id=1326842
Originator: NO

Here is a patch that shold fix the deldict bug.  It also
removes the new condition for metaclasses, because it
isn't needed anymore, and fixes bug #1174712.  These two
changes were needed so that the patch could be properly
tested.

The patch does what Armin suggested; the __dict__ descriptor
looks for a builtin base with tp_dictoffset != 0, and if one
is found, it delegates to that base's __dict__ descriptor.

File Added: modify_dict_bug.diff

----------------------------------------------------------------------

Comment By: Armin Rigo (arigo)
Date: 2007-02-26 08:23

Message:
Logged In: YES 
user_id=4771
Originator: YES

456?  Now that's interesting.  Not 579?

I'm not sure if I ever thought about what the
expected answer should be, but I'd say that
you are correct: 'x.__dict__' should be empty
in the end.  Actually I don't really see how
it manages *not* to be empty in IronPython;
looks like either a very minor detail or a
deeper bug...

----------------------------------------------------------------------

Comment By: Jeremy Hylton (jhylton)
Date: 2007-02-25 22:36

Message:
Logged In: YES 
user_id=31392
Originator: NO

I tried test67.py in IronPython.  It reports that x.hello has the value
456.  Is that the correct behavior?  It seems incorrect to me.  If the
__eq__ method is called, then the object should no longer have either the
Strange() or hello attributes.  They are cleared by reseting __dict__.  Is
that right?

----------------------------------------------------------------------

Comment By: Jeremy Hylton (jhylton)
Date: 2007-02-25 22:23

Message:
Logged In: YES 
user_id=31392
Originator: NO

I tried test67.py in IronPython.  It reports that x.hello has the value
456.  Is that the correct behavior?  It seems incorrect to me.  If the
__eq__ method is called, then the object should no longer have either the
Strange() or hello attributes.  They are cleared by reseting __dict__.  Is
that right?

----------------------------------------------------------------------

Comment By: Armin Rigo (arigo)
Date: 2006-06-30 07:05

Message:
Logged In: YES 
user_id=4771

Well, try making an "empty" class Foo(object): pass, and see
what magically shows up in Foo.__dict__.keys().  Here is the
__dict__ descriptor used for instances of Foo.  This
descriptor is connected to subtype_dict() and
subtype_setdict() in typeobject.c.

INCREF/DECREFs are in theory missing around every use of the
dictionary returned by _PyObject_GetDictPtr(), because more
or less any such use could remove the dict from the object
from which _PyObject_GetDictPtr() returned from, and so
decref the dict - while it's used.  This might require some
timing, to check the performance impact.

----------------------------------------------------------------------

Comment By: Brett Cannon (bcannon)
Date: 2006-06-30 01:14

Message:
Logged In: YES 
user_id=357491

OK, then I am going to need a little guidance to dive into
this more since this is going into some murky waters for me.  =)

For the normal, non-evil case of an empty Python class
(``class Foo(object): pass``), I would think that accessing
__dict__ would fall through to the tp_getset for object, and
then fall to the same slot for type.  Now that is obviously
not happening since you get a straight dict back for that
attribute access and not a dict proxy as would be suggested
based on my logic listed above.

So, my question is, where is the code that handles fetching
Foo().__dict__?  And if you have an inkling of where I
should be looking in terms of possible refcounting additions
that would be great as well.

----------------------------------------------------------------------

Comment By: Armin Rigo (arigo)
Date: 2006-06-29 21:19

Message:
Logged In: YES 
user_id=4771

Brett: I think you're approaching the problem from the wrong
angle.  The problem is being allowed to freely tamper with
the dict stored in objects.  Getting NULL errors here and
there is only a symptom.  As I explain, the '__dict__'
descriptor object needs to do some more checks, and to be
fully safe some Py_INCREF/Py_DECREF are needed in some
critical places.

----------------------------------------------------------------------

Comment By: Brett Cannon (bcannon)
Date: 2006-06-29 17:45

Message:
Logged In: YES 
user_id=357491

For the deldict.py crasher, if you look at the traceback
there is no good place to do a check that tp_dict is sane
except in type_module() or PyDict_GetItem().  Now adding the
NULL check in type_module() will fix this specific problem,
but that seems like an ad-hoc patch.

Why don't we check for NULL in PyDict_GetItem() and return
NULL just like the PyDict_Check() test?  I am sure the
answer is "performance", but this is not PyDict_GETITEM()and
thus already is not the performance-critical version anyway.
 So why shouldn't we check for NULL there and possibly catch
other errors?

----------------------------------------------------------------------

Comment By: Brett Cannon (bcannon)
Date: 2006-06-29 17:41

Message:
Logged In: YES 
user_id=357491

Simple patch for the loosing_dict_ref.py crasher is
attached.  Just checked for possible NULL values in what is
needed by _Py_ForgetReference().  Let me know what you
think, Armin.

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2005-12-14 15:36

Message:
Logged In: YES 
user_id=6656

Yikes!

----------------------------------------------------------------------

Comment By: Armin Rigo (arigo)
Date: 2005-12-14 15:32

Message:
Logged In: YES 
user_id=4771

A proposed fix for the later crash: the __dict__ descriptor of
user-defined classes should verify if there is another __dict__ descriptor
along the solid path of the type (i.e. following "tp_base" pointers).  If
so, it should delegate to it.

----------------------------------------------------------------------

Comment By: Armin Rigo (arigo)
Date: 2005-12-14 14:49

Message:
Logged In: YES 
user_id=4771

Uh oh.  There is a much simpler crash.  The technique outlined in
deldict.py can be used to rebind or even delete the __dict__ attribute of
instances where it should normally not be allowed.

----------------------------------------------------------------------

Comment By: Armin Rigo (arigo)
Date: 2005-09-25 08:55

Message:
Logged In: YES 
user_id=4771

The bug is related to code like PyObject_GenericGetAttribute and
_PyType_Lookup which are not careful about the reference counters of the
objects they operate on.  This allows a much simpler crash (test67.py): the
__dict__ of an object can be decrefed away while lookdict() is looking at
it.  This has a simple fix -- drop some amount of Py_INCREF/Py_DECREF in
core routines like PyObject_GenericGetAttr.  We probably need to measure if
it has a performance impact, though.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1303614&group_id=5470

From noreply at sourceforge.net  Thu Apr 19 17:15:06 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Thu, 19 Apr 2007 08:15:06 -0700
Subject: [ python-Bugs-1696199 ] Add collections.counts()
Message-ID: <E1HeYLi-00065W-Uf@sc8-sf-web21.sourceforge.net>

Bugs item #1696199, was opened at 2007-04-07 15:38
Message generated for change (Comment added) made by bediviere
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1696199&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.6
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Steven Bethard (bediviere)
Assigned to: Raymond Hettinger (rhettinger)
Summary: Add collections.counts()

Initial Comment:
As suggested in http://mail.python.org/pipermail/python-list/2007-April/433986.html this is a patch to add a counts() function to the collections module. Usage looks like:

>>> items = 'acabbacba'
>>> item_counts = counts(items)
>>> for item in 'abcd':
...     print item, item_counts[item]
...
a 4
b 3
c 2
d 0

Yes, it's only a 4-line function, but it's a frequently re-written 4-line function.

----------------------------------------------------------------------

>Comment By: Steven Bethard (bediviere)
Date: 2007-04-19 09:15

Message:
Logged In: YES 
user_id=945502
Originator: YES

A summary of the python-dev thread
(http://mail.python.org/pipermail/python-dev/2007-April/072502.html)

Since the number of times an unseen item was seen is 0, most people felt
returning 0 was more natural behavior than raising KeyError.

There was some discussion of alternate names, but most people were fine
with counts().

Raymond suggested making it a classmethod of dict, but people were a
little concerned about adding to dict's already complex API, and since the
result of counts() needed to return 0s instead of raising KeyErrors, it
wouldn't really have the same behavior as a plain dict() anyway.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2007-04-07 18:23

Message:
Logged In: YES 
user_id=6380
Originator: NO

I guess it's simplicity of implementation vs. simplicity of use.  And I'm
not even sure which is easier to use.  It's just that defaultdicts are a
very new thing and still feel "weird" -- even though I pushed for the
implementation based on popular demand I'm not a user myself.  Perhaps ask
around on python-dev?



----------------------------------------------------------------------

Comment By: Steven Bethard (bediviere)
Date: 2007-04-07 17:48

Message:
Logged In: YES 
user_id=945502
Originator: YES

I think it's okay if it's not a defaultdict.  That was the simplest
implementation, but I certainly have no problem calling d.get() when
necessary. Should I change the implementation to use a dict()?

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2007-04-07 17:39

Message:
Logged In: YES 
user_id=6380
Originator: NO

Does it have to be a defaultdict?  I.e. is it important that
item_counts['d'] not raise KeyError?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1696199&group_id=5470

From noreply at sourceforge.net  Thu Apr 19 18:40:18 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Thu, 19 Apr 2007 09:40:18 -0700
Subject: [ python-Bugs-1303614 ] Bypassing __dict__ readonlyness
Message-ID: <E1HeZgA-0007Ax-Ii@sc8-sf-web21.sourceforge.net>

Bugs item #1303614, was opened at 2005-09-25 01:40
Message generated for change (Comment added) made by zseil
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1303614&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Interpreter Core
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Armin Rigo (arigo)
Assigned to: Nobody/Anonymous (nobody)
Summary: Bypassing __dict__ readonlyness

Initial Comment:
The __dict__ attribute of some objects is read-only,
e.g. for type objects.  However, there is a generic
way to still access and modify it (without hacking with
gc.get_referrents(), that is).  This can lead to
obscure crashes.  Attached is an example that shows
a potential "problem" involving putting strange keys
in the __dict__ of a type.

This is probably very minor anyway.  If we wanted to
fix this, we would need a __dict__ descriptor that
looks more cleverly at the object to which it is
applied.

BTW the first person who understand why the attached
program crashes gets a free coffee.

----------------------------------------------------------------------

>Comment By: Ziga Seilnacht (zseil)
Date: 2007-04-19 18:40

Message:
Logged In: YES 
user_id=1326842
Originator: NO

Here is the updated patch.  I also added a few more
tests, and removed the news entry for revision 53997.


File Added: modify_dict_bug2.diff

----------------------------------------------------------------------

Comment By: Armin Rigo (arigo)
Date: 2007-04-19 16:46

Message:
Logged In: YES 
user_id=4771
Originator: YES

Reverted r53997 in revision 54875.

zseil, could you update your patch for the new
svn head?  Thanks!  It should mostly be a matter
of simplifying it.

----------------------------------------------------------------------

Comment By: Armin Rigo (arigo)
Date: 2007-04-18 11:22

Message:
Logged In: YES 
user_id=4771
Originator: YES

I'm skeptical about the whole revision 53997
which introduced not only the unneeded metaclass
condition, but also the strange check when
assigning to __bases__.  I don't think that this
check about __bases__ is correct or relevant or
really fixes any crash.  The link between the
bases and the metaclass of a class is tenuous
anyway: the bases are just used to compute the
metaclass if none is specified explicitly.

As I said on python-dev (with no answer) I am
thinking about reverting r53997 completely.
I'm going to check what I said above a bit more
in-depth first.

----------------------------------------------------------------------

Comment By: Ziga Seilnacht (zseil)
Date: 2007-04-17 02:42

Message:
Logged In: YES 
user_id=1326842
Originator: NO

Here is a patch that shold fix the deldict bug.  It also
removes the new condition for metaclasses, because it
isn't needed anymore, and fixes bug #1174712.  These two
changes were needed so that the patch could be properly
tested.

The patch does what Armin suggested; the __dict__ descriptor
looks for a builtin base with tp_dictoffset != 0, and if one
is found, it delegates to that base's __dict__ descriptor.

File Added: modify_dict_bug.diff

----------------------------------------------------------------------

Comment By: Armin Rigo (arigo)
Date: 2007-02-26 09:23

Message:
Logged In: YES 
user_id=4771
Originator: YES

456?  Now that's interesting.  Not 579?

I'm not sure if I ever thought about what the
expected answer should be, but I'd say that
you are correct: 'x.__dict__' should be empty
in the end.  Actually I don't really see how
it manages *not* to be empty in IronPython;
looks like either a very minor detail or a
deeper bug...

----------------------------------------------------------------------

Comment By: Jeremy Hylton (jhylton)
Date: 2007-02-25 23:36

Message:
Logged In: YES 
user_id=31392
Originator: NO

I tried test67.py in IronPython.  It reports that x.hello has the value
456.  Is that the correct behavior?  It seems incorrect to me.  If the
__eq__ method is called, then the object should no longer have either the
Strange() or hello attributes.  They are cleared by reseting __dict__.  Is
that right?

----------------------------------------------------------------------

Comment By: Jeremy Hylton (jhylton)
Date: 2007-02-25 23:23

Message:
Logged In: YES 
user_id=31392
Originator: NO

I tried test67.py in IronPython.  It reports that x.hello has the value
456.  Is that the correct behavior?  It seems incorrect to me.  If the
__eq__ method is called, then the object should no longer have either the
Strange() or hello attributes.  They are cleared by reseting __dict__.  Is
that right?

----------------------------------------------------------------------

Comment By: Armin Rigo (arigo)
Date: 2006-06-30 09:05

Message:
Logged In: YES 
user_id=4771

Well, try making an "empty" class Foo(object): pass, and see
what magically shows up in Foo.__dict__.keys().  Here is the
__dict__ descriptor used for instances of Foo.  This
descriptor is connected to subtype_dict() and
subtype_setdict() in typeobject.c.

INCREF/DECREFs are in theory missing around every use of the
dictionary returned by _PyObject_GetDictPtr(), because more
or less any such use could remove the dict from the object
from which _PyObject_GetDictPtr() returned from, and so
decref the dict - while it's used.  This might require some
timing, to check the performance impact.

----------------------------------------------------------------------

Comment By: Brett Cannon (bcannon)
Date: 2006-06-30 03:14

Message:
Logged In: YES 
user_id=357491

OK, then I am going to need a little guidance to dive into
this more since this is going into some murky waters for me.  =)

For the normal, non-evil case of an empty Python class
(``class Foo(object): pass``), I would think that accessing
__dict__ would fall through to the tp_getset for object, and
then fall to the same slot for type.  Now that is obviously
not happening since you get a straight dict back for that
attribute access and not a dict proxy as would be suggested
based on my logic listed above.

So, my question is, where is the code that handles fetching
Foo().__dict__?  And if you have an inkling of where I
should be looking in terms of possible refcounting additions
that would be great as well.

----------------------------------------------------------------------

Comment By: Armin Rigo (arigo)
Date: 2006-06-29 23:19

Message:
Logged In: YES 
user_id=4771

Brett: I think you're approaching the problem from the wrong
angle.  The problem is being allowed to freely tamper with
the dict stored in objects.  Getting NULL errors here and
there is only a symptom.  As I explain, the '__dict__'
descriptor object needs to do some more checks, and to be
fully safe some Py_INCREF/Py_DECREF are needed in some
critical places.

----------------------------------------------------------------------

Comment By: Brett Cannon (bcannon)
Date: 2006-06-29 19:45

Message:
Logged In: YES 
user_id=357491

For the deldict.py crasher, if you look at the traceback
there is no good place to do a check that tp_dict is sane
except in type_module() or PyDict_GetItem().  Now adding the
NULL check in type_module() will fix this specific problem,
but that seems like an ad-hoc patch.

Why don't we check for NULL in PyDict_GetItem() and return
NULL just like the PyDict_Check() test?  I am sure the
answer is "performance", but this is not PyDict_GETITEM()and
thus already is not the performance-critical version anyway.
 So why shouldn't we check for NULL there and possibly catch
other errors?

----------------------------------------------------------------------

Comment By: Brett Cannon (bcannon)
Date: 2006-06-29 19:41

Message:
Logged In: YES 
user_id=357491

Simple patch for the loosing_dict_ref.py crasher is
attached.  Just checked for possible NULL values in what is
needed by _Py_ForgetReference().  Let me know what you
think, Armin.

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2005-12-14 16:36

Message:
Logged In: YES 
user_id=6656

Yikes!

----------------------------------------------------------------------

Comment By: Armin Rigo (arigo)
Date: 2005-12-14 16:32

Message:
Logged In: YES 
user_id=4771

A proposed fix for the later crash: the __dict__ descriptor of
user-defined classes should verify if there is another __dict__ descriptor
along the solid path of the type (i.e. following "tp_base" pointers).  If
so, it should delegate to it.

----------------------------------------------------------------------

Comment By: Armin Rigo (arigo)
Date: 2005-12-14 15:49

Message:
Logged In: YES 
user_id=4771

Uh oh.  There is a much simpler crash.  The technique outlined in
deldict.py can be used to rebind or even delete the __dict__ attribute of
instances where it should normally not be allowed.

----------------------------------------------------------------------

Comment By: Armin Rigo (arigo)
Date: 2005-09-25 10:55

Message:
Logged In: YES 
user_id=4771

The bug is related to code like PyObject_GenericGetAttribute and
_PyType_Lookup which are not careful about the reference counters of the
objects they operate on.  This allows a much simpler crash (test67.py): the
__dict__ of an object can be decrefed away while lookdict() is looking at
it.  This has a simple fix -- drop some amount of Py_INCREF/Py_DECREF in
core routines like PyObject_GenericGetAttr.  We probably need to measure if
it has a performance impact, though.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1303614&group_id=5470

From noreply at sourceforge.net  Thu Apr 19 19:50:16 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Thu, 19 Apr 2007 10:50:16 -0700
Subject: [ python-Bugs-1703592 ] have a way to ignore nonexisting locales in
	locale.setlocale
Message-ID: <E1Heals-00068u-KD@sc8-sf-web22.sourceforge.net>

Bugs item #1703592, was opened at 2007-04-19 15:35
Message generated for change (Comment added) made by loewis
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1703592&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Extension Modules
Group: Python 2.6
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Matthias Klose (doko)
Assigned to: Martin v. L?wis (loewis)
Summary: have a way to ignore nonexisting locales in locale.setlocale

Initial Comment:
this came up on #ubuntu-devel; Debian and Ubuntu do not generate all libc locales by default, so it is likely that 

$ LC_ALL=en_US.UTF-8 python -c "import locale; locale.setlocale(locale.LC_ALL, '')"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/lib/python2.5/locale.py", line 476, in setlocale
    return _setlocale(category, locale)
locale.Error: unsupported locale setting

fails on a system (often experienced ssh'ing into a server system where your current locale doesn't exist). Examples for bug reports in various applications are:

  https://launchpad.net/bugs/91583 (apt-listchanges)
  https://launchpad.net/bugs/88638 (pitivi)
  https://launchpad.net/bugs/81556 (exaile)
  https://launchpad.net/bugs/90525 (hwdb-client)

In C, the result of the setlocale(3) call can be, and usually is ignored, if the locale cannot be set.

It is argued that the Python interface for locale.setlocale() should not raise an exception by default, if the locale is missing on the system.

That would be an behaviour change of locale.setlocale(), so the original behavour should be kept as an option (introducing an optional third parameter to raise the exception when the locale doesn't exist). Is this an appropriate change, or could another change be suggested?


----------------------------------------------------------------------

>Comment By: Martin v. L?wis (loewis)
Date: 2007-04-19 19:50

Message:
Logged In: YES 
user_id=21627
Originator: NO

Not raising an exception if an operation failed violates the Zen of Python
(errors should never pass silently, unless explicitly silenced). So
explicit silencing is necessary, but if so, it's easy enough to explicitly
catch the exception:

try:
  locale.setlocale(..)
except locale.Error:
  pass

If the intention is that all software on Debian works successfully even if
the locale is configured incorrectly, then an automated build system should
perform all builds in a non-existing locale, and see if anything fails.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1703592&group_id=5470

From noreply at sourceforge.net  Thu Apr 19 21:33:05 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Thu, 19 Apr 2007 12:33:05 -0700
Subject: [ python-Feature Requests-1703592 ] have a way to ignore nonexisting
	locales in locale.setlocale
Message-ID: <E1HecNN-0000PC-Rg@sc8-sf-web22.sourceforge.net>

Feature Requests item #1703592, was opened at 2007-04-19 08:35
Message generated for change (Comment added) made by rhettinger
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1703592&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
>Category: Extension Modules
>Group: Python 2.6
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Matthias Klose (doko)
Assigned to: Martin v. L?wis (loewis)
Summary: have a way to ignore nonexisting locales in locale.setlocale

Initial Comment:
this came up on #ubuntu-devel; Debian and Ubuntu do not generate all libc locales by default, so it is likely that 

$ LC_ALL=en_US.UTF-8 python -c "import locale; locale.setlocale(locale.LC_ALL, '')"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/lib/python2.5/locale.py", line 476, in setlocale
    return _setlocale(category, locale)
locale.Error: unsupported locale setting

fails on a system (often experienced ssh'ing into a server system where your current locale doesn't exist). Examples for bug reports in various applications are:

  https://launchpad.net/bugs/91583 (apt-listchanges)
  https://launchpad.net/bugs/88638 (pitivi)
  https://launchpad.net/bugs/81556 (exaile)
  https://launchpad.net/bugs/90525 (hwdb-client)

In C, the result of the setlocale(3) call can be, and usually is ignored, if the locale cannot be set.

It is argued that the Python interface for locale.setlocale() should not raise an exception by default, if the locale is missing on the system.

That would be an behaviour change of locale.setlocale(), so the original behavour should be kept as an option (introducing an optional third parameter to raise the exception when the locale doesn't exist). Is this an appropriate change, or could another change be suggested?


----------------------------------------------------------------------

>Comment By: Raymond Hettinger (rhettinger)
Date: 2007-04-19 14:33

Message:
Logged In: YES 
user_id=80475
Originator: NO

-1  

I prefer explicit exception. If needed, it is not hard to catch and ignore
the exception. 

----------------------------------------------------------------------

Comment By: Martin v. L?wis (loewis)
Date: 2007-04-19 12:50

Message:
Logged In: YES 
user_id=21627
Originator: NO

Not raising an exception if an operation failed violates the Zen of Python
(errors should never pass silently, unless explicitly silenced). So
explicit silencing is necessary, but if so, it's easy enough to explicitly
catch the exception:

try:
  locale.setlocale(..)
except locale.Error:
  pass

If the intention is that all software on Debian works successfully even if
the locale is configured incorrectly, then an automated build system should
perform all builds in a non-existing locale, and see if anything fails.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1703592&group_id=5470

From noreply at sourceforge.net  Thu Apr 19 22:47:30 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Thu, 19 Apr 2007 13:47:30 -0700
Subject: [ python-Bugs-1701409 ] Segfault in c_char_p of ctypes
Message-ID: <E1HedXO-0007ma-I3@sc8-sf-web24.sourceforge.net>

Bugs item #1701409, was opened at 2007-04-16 06:45
Message generated for change (Comment added) made by belopolsky
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1701409&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Extension Modules
Group: None
Status: Open
Resolution: None
Priority: 6
Private: No
Submitted By: sebastien Martini (seb_martini)
Assigned to: Thomas Heller (theller)
Summary: Segfault in c_char_p of ctypes

Initial Comment:
Hi,

I experienced a segmentation fault (when providing a wrong argument type to c_char_p) in the ctypes module under Linux and with the version 2.5 .


sundae:~$ python
Python 2.5.1c1 (release25-maint, Apr  6 2007, 22:02:36) 
[GCC 4.1.2 (Ubuntu 4.1.2-0ubuntu4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from ctypes import *
>>> c_char_p(42)
zsh: segmentation fault (core dumped)  python
sundae:~$  


Regards,

Sebastien Martini


----------------------------------------------------------------------

Comment By: Alexander Belopolsky (belopolsky)
Date: 2007-04-19 16:47

Message:
Logged In: YES 
user_id=835142
Originator: NO

It is easy to distinguish between a c_char_p that is created from a python
string from one that is created from a python int: the former has an
_objects attribute set to a string:

>>> x,y = map(c_char_p, ('abc',42))
>>> (x._objects,y._objects)
('abc', None)

It may be reasonable to give up a "nice" repr for the cases where c_char_p
is not known to point into a python string. You can still keep "nice" str,
so users can still enjoy a crash by printing c_char_p(42). :-)

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2007-04-19 02:58

Message:
Logged In: YES 
user_id=33168
Originator: NO

> Do other platforms have a function that can do this check?

There's no platform independent way to do this.

I don't know anything about ctypes, but what might be a way to handle this
is providing a way to convert an integer into a pointer (like a void*). 
Then all these routines could accept this void* type that you would define,
but not accept a raw integer.  That means people would still have access to
do the same thing they can currently do, but it would be clearer that they
are playing with fire by converting any int into a pointer.

There are so many ways to use ctypes to crash, I'm not sure if this is
worth fixing, except perhaps in the docs to point out the issues.

----------------------------------------------------------------------

Comment By: Thomas Heller (theller)
Date: 2007-04-18 15:20

Message:
Logged In: YES 
user_id=11105
Originator: NO

This is a difficult issue.  The integer (which is interpreted as address)
is allowed because some libraries use 'char *' pointers initialized to
small, invalid addresses for special purposes.

On windows, printing a c_char_p(42) does not crash, it raises a ValueError
instead:
>>> from ctypes import *
>>> c_char_p(42)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: invalid string pointer 0x00B20B48
>>>

Windows does this by checking if there is a valid string at the address
(see Modules/_ctypes/cfield.c, line 1366) by calling the IsBadStringPointer
api function.  Do other platforms have a function that can do this check?

If not, I'm afraid we would have to give up on the very convenient repr of
'normal' c_char_p instances:

>>> c_char_p("foo bar")
c_char_p('foo bar')
>>>

and only print the address (at least on non-windows).

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2007-04-18 03:37

Message:
Logged In: YES 
user_id=33168
Originator: NO

Thanks for the bug report Sebastien.

This bug occurs in 2.5 and in the trunk, so it's not a regression. 
Thomas, could you take a look?  Integers are specifically allowed here
(floats are not).  The problem is when trying to print the repr:

#0  PyString_FromString (str=0x2a <Address 0x2a out of bounds>) at
Objects/stringobject.c:108
#1  z_get (ptr=0x2ae1aa9d6450, size=8) at Modules/_ctypes/cfield.c:1373
#2  Simple_get_value (self=0x2ae1aa9d63f8) at
Modules/_ctypes/_ctypes.c:4007
#3  Simple_repr (self=0x2ae1aa9d63f8) at Modules/_ctypes/_ctypes.c:4095
#4  PyObject_Repr (v=0x2ae1aa9d63f8) at Objects/object.c:361


----------------------------------------------------------------------

Comment By: sebastien Martini (seb_martini)
Date: 2007-04-16 15:40

Message:
Logged In: YES 
user_id=1528211
Originator: YES

c_wchar_p also contains this bug.




----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1701409&group_id=5470

From noreply at sourceforge.net  Thu Apr 19 23:51:13 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Thu, 19 Apr 2007 14:51:13 -0700
Subject: [ python-Bugs-1703952 ] ctypes: problem with large integers
Message-ID: <E1HeeX3-0000ec-Gu@sc8-sf-web22.sourceforge.net>

Bugs item #1703952, was opened at 2007-04-19 17:51
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1703952&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Extension Modules
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Alexander Belopolsky (belopolsky)
Assigned to: Nobody/Anonymous (nobody)
Summary: ctypes: problem with large integers

Initial Comment:
Python 2.5 (r25:51908, Nov 24 2006, 11:03:50)
[GCC 3.4.4 20050721 (Red Hat 3.4.4-2)] on linux2
>>> from ctypes import *
>>> c_int(2**31).value
-2147483648
>>> c_long(2**32-1).value
-1

In a 64-bit build, the situation is even worse:

>>> c_int(int(2**32)).value
0
>>> c_int(2**32).value
0

Another way to see the problem:
>>> c = CDLL(None)
>>> c.printf("%d\n", 2**32)
0
2


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1703952&group_id=5470

From noreply at sourceforge.net  Fri Apr 20 05:04:18 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Thu, 19 Apr 2007 20:04:18 -0700
Subject: [ python-Bugs-1696740 ] README is referencing obsolete?
	http://starship.python.net
Message-ID: <E1HejQ2-0000HH-3b@sc8-sf-web22.sourceforge.net>

Bugs item #1696740, was opened at 2007-04-09 01:58
Message generated for change (Settings changed) made by tjreedy
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1696740&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Documentation
Group: None
>Status: Closed
>Resolution: Invalid
Priority: 5
Private: No
Submitted By: Jiri Navratil (plavcik)
Assigned to: Nobody/Anonymous (nobody)
Summary: README is referencing obsolete? http://starship.python.net

Initial Comment:
In README file obtained 09.04.2004 from Python SVN HEAD via svn is referenced 

There's also a Python community web site at
http://starship.python.net/.

I can't open this site. I'm getting

unable to find IP address for starship.python.net

Is this information obsolete?


----------------------------------------------------------------------

Comment By: Gabriel Genellina (gagenellina)
Date: 2007-04-10 10:06

Message:
Logged In: YES 
user_id=479790
Originator: NO

No, the site had some problems but it's up and running again.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1696740&group_id=5470

From noreply at sourceforge.net  Fri Apr 20 05:39:29 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Thu, 19 Apr 2007 20:39:29 -0700
Subject: [ python-Bugs-1701389 ] utf-16 codec problems with multiple file
	append
Message-ID: <E1Hejy5-0003yY-3G@sc8-sf-web22.sourceforge.net>

Bugs item #1701389, was opened at 2007-04-16 18:05
Message generated for change (Comment added) made by iceberg4ever
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1701389&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Unicode
Group: Python 2.5
Status: Closed
>Resolution: Remind
Priority: 5
Private: No
Submitted By: Iceberg Luo (iceberg4ever)
Assigned to: M.-A. Lemburg (lemburg)
Summary: utf-16 codec problems with multiple file append

Initial Comment:
This bug is similar but not exactly the same as bug215974.  (http://sourceforge.net/tracker/?group_id=5470&atid=105470&aid=215974&func=detail)

In my test, even multiple write() within an open()~close() lifespan will not cause the multi BOM phenomena mentioned in bug215974. Maybe it is because bug 215974 was somehow fixed during the past 7 years, although Lemburg classified it as WontFix. 

However, if a file is appended for more than once, by an "codecs.open('file.txt', 'a', 'utf16')", the multi BOM appears.

At the same time, the saying of "(Extra unnecessary) BOM marks are removed from the input stream by the Python UTF-16 codec" in bug215974 is not true even in today, on Python2.4.4 and Python2.5.1c1 on Windows XP.

Iceberg
------------------

PS: Did not find the "File Upload" checkbox mentioned in this web page, so I think I'd better paste the code right here...

import codecs, os

filename = "test.utf-16"
if os.path.exists(filename): os.unlink(filename)  # reset

def myOpen():
  return codecs.open(filename, "a", 'UTF-16')
def readThemBack():
  return list( codecs.open(filename, "r", 'UTF-16') )
def clumsyPatch(raw): # you can read it after your first run of this program
  for line in raw:
    if line[0] in (u'\ufffe', u'\ufeff'): # get rid of the BOMs
      yield line[1:]
    else:
      yield line

fout = myOpen()
fout.write(u"ab\n") # to simplify the problem, I only use ASCII chars here
fout.write(u"cd\n")
fout.close()
print readThemBack()
assert readThemBack() == [ u'ab\n', u'cd\n' ]
assert os.stat(filename).st_size == 14  # Only one BOM in the file

fout = myOpen()
fout.write(u"ef\n")
fout.write(u"gh\n")
fout.close()
print readThemBack()
#print list( clumsyPatch( readThemBack() ) )  # later you can enable this fix
assert readThemBack() == [ u'ab\n', u'cd\n', u'ef\n', u'gh\n' ] # fails here
assert os.stat(filename).st_size == 26  # not to mention here: multi BOM appears


----------------------------------------------------------------------

>Comment By: Iceberg Luo (iceberg4ever)
Date: 2007-04-20 11:39

Message:
Logged In: YES 
user_id=1770538
Originator: YES

If such a bug would be fixed, either StreamWriter or StreamReader should
do something.

I can understand Doerwalter that it is somewhat not comfortable for a
StreamWriter to detect whether these is already a BOM at current file
header, especially when operating in append mode. But, IMHO, the
StreamReader should be able to detect multi BOM during its life span and
automatically ignore the non-first one, providing that a BOM is never
supposed to occur in normal content.  Not to mention that such a Reader
seems exist for a while, according to the "(extra unnecessary) BOM marks
are removed
from the input stream by the Python UTF-16 codec" in bug215974
(http://sourceforge.net/tracker/?group_id=5470&atid=105470&aid=215974&func=
detail).

Therefore I don't think a WontFix will be the proper FINAL solution for
this case.

----------------------------------------------------------------------

Comment By: Walter D?rwald (doerwalter)
Date: 2007-04-19 19:30

Message:
Logged In: YES 
user_id=89016
Originator: NO

Closing as "won't fix"

----------------------------------------------------------------------

Comment By: M.-A. Lemburg (lemburg)
Date: 2007-04-19 18:35

Message:
Logged In: YES 
user_id=38388
Originator: NO

I suggest you close this as wont fix.


----------------------------------------------------------------------

Comment By: Walter D?rwald (doerwalter)
Date: 2007-04-19 18:30

Message:
Logged In: YES 
user_id=89016
Originator: NO

append mode is simply not supported for codecs. How would the codec find
out the codec state that was active after the last characters where written
to the file?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1701389&group_id=5470

From noreply at sourceforge.net  Fri Apr 20 10:21:36 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Fri, 20 Apr 2007 01:21:36 -0700
Subject: [ python-Bugs-1704156 ] TarFile.addfile() throws a struct.error
Message-ID: <E1HeoN6-0001yH-7j@sc8-sf-web22.sourceforge.net>

Bugs item #1704156, was opened at 2007-04-20 01:21
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1704156&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: K. C. Wong (dvusboy)
Assigned to: Nobody/Anonymous (nobody)
Summary: TarFile.addfile() throws a struct.error

Initial Comment:
When adding a file to a TarFile instance using addfile(), if the file paths (name and arcname) are unicode strings, then a struct.error will the raised. Python versions prior to 2.5 do not show this behaviour.

Assuming the current directory has a file name 'mac.txt', here is an interactive session that shows the problem:

Python 2.5 (r25:51908, Apr 18 2007, 19:06:57)
[GCC 3.4.6 20060404 (Red Hat 3.4.6-3)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import tarfile
>>> t=tarfile.open('test.tar', 'w')
>>> i=t.gettarinfo(u'mac.txt', u'mac.txt')
>>> t.addfile(i, file(u'mac.txt', 'r'))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.5/tarfile.py", line 1422, in addfile
    self.fileobj.write(tarinfo.tobuf(self.posix))
  File "/usr/lib/python2.5/tarfile.py", line 871, in tobuf
    buf = struct.pack("%ds" % BLOCKSIZE, "".join(parts))
  File "/usr/lib/python2.5/struct.py", line 63, in pack
    return o.pack(*args)
struct.error: argument for 's' must be a string


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1704156&group_id=5470

From noreply at sourceforge.net  Fri Apr 20 13:06:28 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Fri, 20 Apr 2007 04:06:28 -0700
Subject: [ python-Feature Requests-1703592 ] have a way to ignore nonexisting
	locales in locale.setlocale
Message-ID: <E1Heqwe-0007Eq-6s@sc8-sf-web23.sourceforge.net>

Feature Requests item #1703592, was opened at 2007-04-19 14:35
Message generated for change (Comment added) made by cjwatson
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1703592&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Extension Modules
Group: Python 2.6
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Matthias Klose (doko)
Assigned to: Martin v. L?wis (loewis)
Summary: have a way to ignore nonexisting locales in locale.setlocale

Initial Comment:
this came up on #ubuntu-devel; Debian and Ubuntu do not generate all libc locales by default, so it is likely that 

$ LC_ALL=en_US.UTF-8 python -c "import locale; locale.setlocale(locale.LC_ALL, '')"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/lib/python2.5/locale.py", line 476, in setlocale
    return _setlocale(category, locale)
locale.Error: unsupported locale setting

fails on a system (often experienced ssh'ing into a server system where your current locale doesn't exist). Examples for bug reports in various applications are:

  https://launchpad.net/bugs/91583 (apt-listchanges)
  https://launchpad.net/bugs/88638 (pitivi)
  https://launchpad.net/bugs/81556 (exaile)
  https://launchpad.net/bugs/90525 (hwdb-client)

In C, the result of the setlocale(3) call can be, and usually is ignored, if the locale cannot be set.

It is argued that the Python interface for locale.setlocale() should not raise an exception by default, if the locale is missing on the system.

That would be an behaviour change of locale.setlocale(), so the original behavour should be kept as an option (introducing an optional third parameter to raise the exception when the locale doesn't exist). Is this an appropriate change, or could another change be suggested?


----------------------------------------------------------------------

Comment By: Colin Watson (cjwatson)
Date: 2007-04-20 12:06

Message:
Logged In: YES 
user_id=17056
Originator: NO

If this were actually a true error, I'd agree with you, but it isn't. For
most programs (that just do locale.setlocale(locale.LC_ALL, '') to switch
on internationalisation), it's a warning, not an error; in the common case
there is no reason for them to fail.

If you still insist that this has to be treated as an error, how about
adding locale.enable_i18n or something for the common case that does:

  try:
      locale.setlocale(locale.LC_ALL, '')
  except locale.Error:
      pass

Because, in practice, many programs appear not to bother catching the
exception, and because the programmer is typically using an environment
with a properly configured locale they won't notice. It's only when you're
in an environment such as sshing (with SendEnv) to a remote system that
doesn't have your locale configured that you notice that C programs
continue to function correctly, Perl programs issue a warning on stderr,
but Python programs crash. While noticing errors is a good thing in
general, it seems to go a bit far here.

----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2007-04-19 20:33

Message:
Logged In: YES 
user_id=80475
Originator: NO

-1  

I prefer explicit exception. If needed, it is not hard to catch and ignore
the exception. 

----------------------------------------------------------------------

Comment By: Martin v. L?wis (loewis)
Date: 2007-04-19 18:50

Message:
Logged In: YES 
user_id=21627
Originator: NO

Not raising an exception if an operation failed violates the Zen of Python
(errors should never pass silently, unless explicitly silenced). So
explicit silencing is necessary, but if so, it's easy enough to explicitly
catch the exception:

try:
  locale.setlocale(..)
except locale.Error:
  pass

If the intention is that all software on Debian works successfully even if
the locale is configured incorrectly, then an automated build system should
perform all builds in a non-existing locale, and see if anything fails.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1703592&group_id=5470

From noreply at sourceforge.net  Fri Apr 20 15:01:23 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Fri, 20 Apr 2007 06:01:23 -0700
Subject: [ python-Bugs-1704287 ] must run "make" before "make install"
Message-ID: <E1Hesjr-0004I3-E3@sc8-sf-web23.sourceforge.net>

Bugs item #1704287, was opened at 2007-04-20 07:01
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1704287&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Build
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Joseph VanAndel (vanandel)
Assigned to: Nobody/Anonymous (nobody)
Summary: must run "make" before "make install"

Initial Comment:
"make install" fails unless you've already run "make".
Not a show-stopper, but specifying the correct dependencies for "make install" would force everything to be built before it was installed.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1704287&group_id=5470

From noreply at sourceforge.net  Fri Apr 20 16:52:00 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Fri, 20 Apr 2007 07:52:00 -0700
Subject: [ python-Bugs-1704156 ] TarFile.addfile() throws a struct.error
Message-ID: <E1HeuSu-0000RQ-3o@sc8-sf-web23.sourceforge.net>

Bugs item #1704156, was opened at 2007-04-20 10:21
Message generated for change (Settings changed) made by gustaebel
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1704156&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: K. C. Wong (dvusboy)
>Assigned to: Lars Gust?bel (gustaebel)
Summary: TarFile.addfile() throws a struct.error

Initial Comment:
When adding a file to a TarFile instance using addfile(), if the file paths (name and arcname) are unicode strings, then a struct.error will the raised. Python versions prior to 2.5 do not show this behaviour.

Assuming the current directory has a file name 'mac.txt', here is an interactive session that shows the problem:

Python 2.5 (r25:51908, Apr 18 2007, 19:06:57)
[GCC 3.4.6 20060404 (Red Hat 3.4.6-3)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import tarfile
>>> t=tarfile.open('test.tar', 'w')
>>> i=t.gettarinfo(u'mac.txt', u'mac.txt')
>>> t.addfile(i, file(u'mac.txt', 'r'))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.5/tarfile.py", line 1422, in addfile
    self.fileobj.write(tarinfo.tobuf(self.posix))
  File "/usr/lib/python2.5/tarfile.py", line 871, in tobuf
    buf = struct.pack("%ds" % BLOCKSIZE, "".join(parts))
  File "/usr/lib/python2.5/struct.py", line 63, in pack
    return o.pack(*args)
struct.error: argument for 's' must be a string


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1704156&group_id=5470

From noreply at sourceforge.net  Fri Apr 20 20:55:48 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Fri, 20 Apr 2007 11:55:48 -0700
Subject: [ python-Bugs-1703952 ] ctypes: problem with large integers
Message-ID: <E1HeyGq-0008KP-Ir@sc8-sf-web22.sourceforge.net>

Bugs item #1703952, was opened at 2007-04-19 23:51
Message generated for change (Comment added) made by theller
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1703952&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Extension Modules
Group: Python 2.5
>Status: Closed
>Resolution: Invalid
Priority: 5
Private: No
Submitted By: Alexander Belopolsky (belopolsky)
>Assigned to: Thomas Heller (theller)
Summary: ctypes: problem with large integers

Initial Comment:
Python 2.5 (r25:51908, Nov 24 2006, 11:03:50)
[GCC 3.4.4 20050721 (Red Hat 3.4.4-2)] on linux2
>>> from ctypes import *
>>> c_int(2**31).value
-2147483648
>>> c_long(2**32-1).value
-1

In a 64-bit build, the situation is even worse:

>>> c_int(int(2**32)).value
0
>>> c_int(2**32).value
0

Another way to see the problem:
>>> c = CDLL(None)
>>> c.printf("%d\n", 2**32)
0
2


----------------------------------------------------------------------

>Comment By: Thomas Heller (theller)
Date: 2007-04-20 20:55

Message:
Logged In: YES 
user_id=11105
Originator: NO

This works as designed.  ctypes intentionally does no overflow checking
when using the c_int, c_uint, and related integer types.  Instead, only the
available bits are used - just as in C.

Closing as invalid (sorry).

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1703952&group_id=5470

From noreply at sourceforge.net  Fri Apr 20 21:25:00 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Fri, 20 Apr 2007 12:25:00 -0700
Subject: [ python-Bugs-1704156 ] TarFile.addfile() throws a struct.error
Message-ID: <E1Heyj6-0005Qk-W0@sc8-sf-web21.sourceforge.net>

Bugs item #1704156, was opened at 2007-04-20 10:21
Message generated for change (Comment added) made by gustaebel
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1704156&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: K. C. Wong (dvusboy)
Assigned to: Lars Gust?bel (gustaebel)
Summary: TarFile.addfile() throws a struct.error

Initial Comment:
When adding a file to a TarFile instance using addfile(), if the file paths (name and arcname) are unicode strings, then a struct.error will the raised. Python versions prior to 2.5 do not show this behaviour.

Assuming the current directory has a file name 'mac.txt', here is an interactive session that shows the problem:

Python 2.5 (r25:51908, Apr 18 2007, 19:06:57)
[GCC 3.4.6 20060404 (Red Hat 3.4.6-3)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import tarfile
>>> t=tarfile.open('test.tar', 'w')
>>> i=t.gettarinfo(u'mac.txt', u'mac.txt')
>>> t.addfile(i, file(u'mac.txt', 'r'))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.5/tarfile.py", line 1422, in addfile
    self.fileobj.write(tarinfo.tobuf(self.posix))
  File "/usr/lib/python2.5/tarfile.py", line 871, in tobuf
    buf = struct.pack("%ds" % BLOCKSIZE, "".join(parts))
  File "/usr/lib/python2.5/struct.py", line 63, in pack
    return o.pack(*args)
struct.error: argument for 's' must be a string


----------------------------------------------------------------------

>Comment By: Lars Gust?bel (gustaebel)
Date: 2007-04-20 21:25

Message:
Logged In: YES 
user_id=642936
Originator: NO

tarfile.py was never guaranteed to work correctly with unicode filenames.
The fact that this works with Python < 2.5 is purely accidental.

You can work around this (sticking to your example):

i = t.gettarinfo(u'mac.txt', 'mac.txt')

or:

i = t.gettarinfo('mac.txt')


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1704156&group_id=5470

From noreply at sourceforge.net  Fri Apr 20 21:37:45 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Fri, 20 Apr 2007 12:37:45 -0700
Subject: [ python-Bugs-1703952 ] ctypes: problem with large integers
Message-ID: <E1HeyvR-0004TV-2M@sc8-sf-web22.sourceforge.net>

Bugs item #1703952, was opened at 2007-04-19 17:51
Message generated for change (Comment added) made by belopolsky
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1703952&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Extension Modules
Group: Python 2.5
>Status: Open
Resolution: Invalid
Priority: 5
Private: No
Submitted By: Alexander Belopolsky (belopolsky)
Assigned to: Thomas Heller (theller)
Summary: ctypes: problem with large integers

Initial Comment:
Python 2.5 (r25:51908, Nov 24 2006, 11:03:50)
[GCC 3.4.4 20050721 (Red Hat 3.4.4-2)] on linux2
>>> from ctypes import *
>>> c_int(2**31).value
-2147483648
>>> c_long(2**32-1).value
-1

In a 64-bit build, the situation is even worse:

>>> c_int(int(2**32)).value
0
>>> c_int(2**32).value
0

Another way to see the problem:
>>> c = CDLL(None)
>>> c.printf("%d\n", 2**32)
0
2


----------------------------------------------------------------------

>Comment By: Alexander Belopolsky (belopolsky)
Date: 2007-04-20 15:37

Message:
Logged In: YES 
user_id=835142
Originator: YES

An issue remains with the implicit conversion:

On a 64-bit platform (sizeof(long)=64):

>>> c.printf("%d\n",1<<64)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ctypes.ArgumentError: argument 2: <type 'exceptions.OverflowError'>: long
int too long to convert

so it does do overflow checking, but

>>> c.printf("%d\n",(1<<64)-1)
-1
3
>>> c.printf("%d\n",(1<<32))
0
2



----------------------------------------------------------------------

Comment By: Thomas Heller (theller)
Date: 2007-04-20 14:55

Message:
Logged In: YES 
user_id=11105
Originator: NO

This works as designed.  ctypes intentionally does no overflow checking
when using the c_int, c_uint, and related integer types.  Instead, only the
available bits are used - just as in C.

Closing as invalid (sorry).

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1703952&group_id=5470

From noreply at sourceforge.net  Fri Apr 20 21:52:03 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Fri, 20 Apr 2007 12:52:03 -0700
Subject: [ python-Bugs-1701409 ] Segfault in c_char_p of ctypes
Message-ID: <E1Hez9H-0001Bf-Jc@sc8-sf-web24.sourceforge.net>

Bugs item #1701409, was opened at 2007-04-16 12:45
Message generated for change (Comment added) made by theller
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1701409&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Extension Modules
Group: None
Status: Open
Resolution: None
Priority: 6
Private: No
Submitted By: sebastien Martini (seb_martini)
Assigned to: Thomas Heller (theller)
Summary: Segfault in c_char_p of ctypes

Initial Comment:
Hi,

I experienced a segmentation fault (when providing a wrong argument type to c_char_p) in the ctypes module under Linux and with the version 2.5 .


sundae:~$ python
Python 2.5.1c1 (release25-maint, Apr  6 2007, 22:02:36) 
[GCC 4.1.2 (Ubuntu 4.1.2-0ubuntu4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from ctypes import *
>>> c_char_p(42)
zsh: segmentation fault (core dumped)  python
sundae:~$  


Regards,

Sebastien Martini


----------------------------------------------------------------------

>Comment By: Thomas Heller (theller)
Date: 2007-04-20 21:52

Message:
Logged In: YES 
user_id=11105
Originator: NO

belopolsky:  _objects is only set when the c_char_p instance is created in
the way you described.  It will not be set if the instance is modified by a
foreign function call, so this is not robust.

nnorwitz: Sure there are many ways to crash Python with ctypes, but
printing c_char_p(42) should not carsh, IMO.

I believe the best strategy would be to behave this way on Windows where
the check for a valid string pointer can be made:

>>> c_char_p(42)
c_char_p(0x2A)
>>> c_char("foo")
c_char('foo')
>>>

and this way on other systems:

>>> c_char_p(42)
c_char_p(0x2A)
>>> c_char_p("foo")
c_char_p(0x2A7F3B)
>>>

The attached patch fixes this for c_char_p, a similar patch should be
applied to c_wchar_p.
File Added: c_char_p.patch

----------------------------------------------------------------------

Comment By: Alexander Belopolsky (belopolsky)
Date: 2007-04-19 22:47

Message:
Logged In: YES 
user_id=835142
Originator: NO

It is easy to distinguish between a c_char_p that is created from a python
string from one that is created from a python int: the former has an
_objects attribute set to a string:

>>> x,y = map(c_char_p, ('abc',42))
>>> (x._objects,y._objects)
('abc', None)

It may be reasonable to give up a "nice" repr for the cases where c_char_p
is not known to point into a python string. You can still keep "nice" str,
so users can still enjoy a crash by printing c_char_p(42). :-)

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2007-04-19 08:58

Message:
Logged In: YES 
user_id=33168
Originator: NO

> Do other platforms have a function that can do this check?

There's no platform independent way to do this.

I don't know anything about ctypes, but what might be a way to handle this
is providing a way to convert an integer into a pointer (like a void*). 
Then all these routines could accept this void* type that you would define,
but not accept a raw integer.  That means people would still have access to
do the same thing they can currently do, but it would be clearer that they
are playing with fire by converting any int into a pointer.

There are so many ways to use ctypes to crash, I'm not sure if this is
worth fixing, except perhaps in the docs to point out the issues.

----------------------------------------------------------------------

Comment By: Thomas Heller (theller)
Date: 2007-04-18 21:20

Message:
Logged In: YES 
user_id=11105
Originator: NO

This is a difficult issue.  The integer (which is interpreted as address)
is allowed because some libraries use 'char *' pointers initialized to
small, invalid addresses for special purposes.

On windows, printing a c_char_p(42) does not crash, it raises a ValueError
instead:
>>> from ctypes import *
>>> c_char_p(42)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: invalid string pointer 0x00B20B48
>>>

Windows does this by checking if there is a valid string at the address
(see Modules/_ctypes/cfield.c, line 1366) by calling the IsBadStringPointer
api function.  Do other platforms have a function that can do this check?

If not, I'm afraid we would have to give up on the very convenient repr of
'normal' c_char_p instances:

>>> c_char_p("foo bar")
c_char_p('foo bar')
>>>

and only print the address (at least on non-windows).

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2007-04-18 09:37

Message:
Logged In: YES 
user_id=33168
Originator: NO

Thanks for the bug report Sebastien.

This bug occurs in 2.5 and in the trunk, so it's not a regression. 
Thomas, could you take a look?  Integers are specifically allowed here
(floats are not).  The problem is when trying to print the repr:

#0  PyString_FromString (str=0x2a <Address 0x2a out of bounds>) at
Objects/stringobject.c:108
#1  z_get (ptr=0x2ae1aa9d6450, size=8) at Modules/_ctypes/cfield.c:1373
#2  Simple_get_value (self=0x2ae1aa9d63f8) at
Modules/_ctypes/_ctypes.c:4007
#3  Simple_repr (self=0x2ae1aa9d63f8) at Modules/_ctypes/_ctypes.c:4095
#4  PyObject_Repr (v=0x2ae1aa9d63f8) at Objects/object.c:361


----------------------------------------------------------------------

Comment By: sebastien Martini (seb_martini)
Date: 2007-04-16 21:40

Message:
Logged In: YES 
user_id=1528211
Originator: YES

c_wchar_p also contains this bug.




----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1701409&group_id=5470

From noreply at sourceforge.net  Fri Apr 20 21:59:00 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Fri, 20 Apr 2007 12:59:00 -0700
Subject: [ python-Feature Requests-1703592 ] have a way to ignore nonexisting
	locales in locale.setlocale
Message-ID: <E1HezG0-0006k6-Cn@sc8-sf-web22.sourceforge.net>

Feature Requests item #1703592, was opened at 2007-04-19 15:35
Message generated for change (Comment added) made by loewis
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1703592&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Extension Modules
Group: Python 2.6
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Matthias Klose (doko)
Assigned to: Martin v. L?wis (loewis)
Summary: have a way to ignore nonexisting locales in locale.setlocale

Initial Comment:
this came up on #ubuntu-devel; Debian and Ubuntu do not generate all libc locales by default, so it is likely that 

$ LC_ALL=en_US.UTF-8 python -c "import locale; locale.setlocale(locale.LC_ALL, '')"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/lib/python2.5/locale.py", line 476, in setlocale
    return _setlocale(category, locale)
locale.Error: unsupported locale setting

fails on a system (often experienced ssh'ing into a server system where your current locale doesn't exist). Examples for bug reports in various applications are:

  https://launchpad.net/bugs/91583 (apt-listchanges)
  https://launchpad.net/bugs/88638 (pitivi)
  https://launchpad.net/bugs/81556 (exaile)
  https://launchpad.net/bugs/90525 (hwdb-client)

In C, the result of the setlocale(3) call can be, and usually is ignored, if the locale cannot be set.

It is argued that the Python interface for locale.setlocale() should not raise an exception by default, if the locale is missing on the system.

That would be an behaviour change of locale.setlocale(), so the original behavour should be kept as an option (introducing an optional third parameter to raise the exception when the locale doesn't exist). Is this an appropriate change, or could another change be suggested?


----------------------------------------------------------------------

>Comment By: Martin v. L?wis (loewis)
Date: 2007-04-20 21:59

Message:
Logged In: YES 
user_id=21627
Originator: NO

How can you say it's not an error? The function does not achieve what it
attempts to.

Adding another function with a different semantics is fine to me, although
I doubt it helps: the existing code calls setlocale, not that other
function.

----------------------------------------------------------------------

Comment By: Colin Watson (cjwatson)
Date: 2007-04-20 13:06

Message:
Logged In: YES 
user_id=17056
Originator: NO

If this were actually a true error, I'd agree with you, but it isn't. For
most programs (that just do locale.setlocale(locale.LC_ALL, '') to switch
on internationalisation), it's a warning, not an error; in the common case
there is no reason for them to fail.

If you still insist that this has to be treated as an error, how about
adding locale.enable_i18n or something for the common case that does:

  try:
      locale.setlocale(locale.LC_ALL, '')
  except locale.Error:
      pass

Because, in practice, many programs appear not to bother catching the
exception, and because the programmer is typically using an environment
with a properly configured locale they won't notice. It's only when you're
in an environment such as sshing (with SendEnv) to a remote system that
doesn't have your locale configured that you notice that C programs
continue to function correctly, Perl programs issue a warning on stderr,
but Python programs crash. While noticing errors is a good thing in
general, it seems to go a bit far here.

----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2007-04-19 21:33

Message:
Logged In: YES 
user_id=80475
Originator: NO

-1  

I prefer explicit exception. If needed, it is not hard to catch and ignore
the exception. 

----------------------------------------------------------------------

Comment By: Martin v. L?wis (loewis)
Date: 2007-04-19 19:50

Message:
Logged In: YES 
user_id=21627
Originator: NO

Not raising an exception if an operation failed violates the Zen of Python
(errors should never pass silently, unless explicitly silenced). So
explicit silencing is necessary, but if so, it's easy enough to explicitly
catch the exception:

try:
  locale.setlocale(..)
except locale.Error:
  pass

If the intention is that all software on Debian works successfully even if
the locale is configured incorrectly, then an automated build system should
perform all builds in a non-existing locale, and see if anything fails.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1703592&group_id=5470

From noreply at sourceforge.net  Fri Apr 20 22:42:02 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Fri, 20 Apr 2007 13:42:02 -0700
Subject: [ python-Bugs-1701409 ] Segfault in c_char_p of ctypes
Message-ID: <E1Hezve-0002Ml-UO@sc8-sf-web24.sourceforge.net>

Bugs item #1701409, was opened at 2007-04-16 06:45
Message generated for change (Comment added) made by belopolsky
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1701409&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Extension Modules
Group: None
Status: Open
Resolution: None
Priority: 6
Private: No
Submitted By: sebastien Martini (seb_martini)
Assigned to: Thomas Heller (theller)
Summary: Segfault in c_char_p of ctypes

Initial Comment:
Hi,

I experienced a segmentation fault (when providing a wrong argument type to c_char_p) in the ctypes module under Linux and with the version 2.5 .


sundae:~$ python
Python 2.5.1c1 (release25-maint, Apr  6 2007, 22:02:36) 
[GCC 4.1.2 (Ubuntu 4.1.2-0ubuntu4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from ctypes import *
>>> c_char_p(42)
zsh: segmentation fault (core dumped)  python
sundae:~$  


Regards,

Sebastien Martini


----------------------------------------------------------------------

Comment By: Alexander Belopolsky (belopolsky)
Date: 2007-04-20 16:42

Message:
Logged In: YES 
user_id=835142
Originator: NO

theller: [_objects] will not be set if the instance is modified by a
foreign function call, so this is not robust.

I thought about this before making my suggestion.  I believe people will
be less surprized if objects returned from an ffi call print with funny
numbers than getting c_char_p(NNNNN) after entering c_char_p("abc"). 

A few comments on the patch:

1. Since c_void_p is printed in decimal, shouldn't c_char_p be treated the
same? If hexadecimal is prefered, maybe c_void_p should be changed.

2. Currently c_char_p(0) is printed as c_char_p(None), but the patch will
change that to c_char_p(0).

----------------------------------------------------------------------

Comment By: Thomas Heller (theller)
Date: 2007-04-20 15:52

Message:
Logged In: YES 
user_id=11105
Originator: NO

belopolsky:  _objects is only set when the c_char_p instance is created in
the way you described.  It will not be set if the instance is modified by a
foreign function call, so this is not robust.

nnorwitz: Sure there are many ways to crash Python with ctypes, but
printing c_char_p(42) should not carsh, IMO.

I believe the best strategy would be to behave this way on Windows where
the check for a valid string pointer can be made:

>>> c_char_p(42)
c_char_p(0x2A)
>>> c_char("foo")
c_char('foo')
>>>

and this way on other systems:

>>> c_char_p(42)
c_char_p(0x2A)
>>> c_char_p("foo")
c_char_p(0x2A7F3B)
>>>

The attached patch fixes this for c_char_p, a similar patch should be
applied to c_wchar_p.
File Added: c_char_p.patch

----------------------------------------------------------------------

Comment By: Alexander Belopolsky (belopolsky)
Date: 2007-04-19 16:47

Message:
Logged In: YES 
user_id=835142
Originator: NO

It is easy to distinguish between a c_char_p that is created from a python
string from one that is created from a python int: the former has an
_objects attribute set to a string:

>>> x,y = map(c_char_p, ('abc',42))
>>> (x._objects,y._objects)
('abc', None)

It may be reasonable to give up a "nice" repr for the cases where c_char_p
is not known to point into a python string. You can still keep "nice" str,
so users can still enjoy a crash by printing c_char_p(42). :-)

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2007-04-19 02:58

Message:
Logged In: YES 
user_id=33168
Originator: NO

> Do other platforms have a function that can do this check?

There's no platform independent way to do this.

I don't know anything about ctypes, but what might be a way to handle this
is providing a way to convert an integer into a pointer (like a void*). 
Then all these routines could accept this void* type that you would define,
but not accept a raw integer.  That means people would still have access to
do the same thing they can currently do, but it would be clearer that they
are playing with fire by converting any int into a pointer.

There are so many ways to use ctypes to crash, I'm not sure if this is
worth fixing, except perhaps in the docs to point out the issues.

----------------------------------------------------------------------

Comment By: Thomas Heller (theller)
Date: 2007-04-18 15:20

Message:
Logged In: YES 
user_id=11105
Originator: NO

This is a difficult issue.  The integer (which is interpreted as address)
is allowed because some libraries use 'char *' pointers initialized to
small, invalid addresses for special purposes.

On windows, printing a c_char_p(42) does not crash, it raises a ValueError
instead:
>>> from ctypes import *
>>> c_char_p(42)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: invalid string pointer 0x00B20B48
>>>

Windows does this by checking if there is a valid string at the address
(see Modules/_ctypes/cfield.c, line 1366) by calling the IsBadStringPointer
api function.  Do other platforms have a function that can do this check?

If not, I'm afraid we would have to give up on the very convenient repr of
'normal' c_char_p instances:

>>> c_char_p("foo bar")
c_char_p('foo bar')
>>>

and only print the address (at least on non-windows).

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2007-04-18 03:37

Message:
Logged In: YES 
user_id=33168
Originator: NO

Thanks for the bug report Sebastien.

This bug occurs in 2.5 and in the trunk, so it's not a regression. 
Thomas, could you take a look?  Integers are specifically allowed here
(floats are not).  The problem is when trying to print the repr:

#0  PyString_FromString (str=0x2a <Address 0x2a out of bounds>) at
Objects/stringobject.c:108
#1  z_get (ptr=0x2ae1aa9d6450, size=8) at Modules/_ctypes/cfield.c:1373
#2  Simple_get_value (self=0x2ae1aa9d63f8) at
Modules/_ctypes/_ctypes.c:4007
#3  Simple_repr (self=0x2ae1aa9d63f8) at Modules/_ctypes/_ctypes.c:4095
#4  PyObject_Repr (v=0x2ae1aa9d63f8) at Objects/object.c:361


----------------------------------------------------------------------

Comment By: sebastien Martini (seb_martini)
Date: 2007-04-16 15:40

Message:
Logged In: YES 
user_id=1528211
Originator: YES

c_wchar_p also contains this bug.




----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1701409&group_id=5470

From noreply at sourceforge.net  Sat Apr 21 06:54:17 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Fri, 20 Apr 2007 21:54:17 -0700
Subject: [ python-Bugs-1704156 ] TarFile.addfile() throws a struct.error
Message-ID: <E1Hf7c1-0000py-Lo@sc8-sf-web24.sourceforge.net>

Bugs item #1704156, was opened at 2007-04-20 01:21
Message generated for change (Comment added) made by dvusboy
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1704156&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: K. C. Wong (dvusboy)
Assigned to: Lars Gust?bel (gustaebel)
Summary: TarFile.addfile() throws a struct.error

Initial Comment:
When adding a file to a TarFile instance using addfile(), if the file paths (name and arcname) are unicode strings, then a struct.error will the raised. Python versions prior to 2.5 do not show this behaviour.

Assuming the current directory has a file name 'mac.txt', here is an interactive session that shows the problem:

Python 2.5 (r25:51908, Apr 18 2007, 19:06:57)
[GCC 3.4.6 20060404 (Red Hat 3.4.6-3)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import tarfile
>>> t=tarfile.open('test.tar', 'w')
>>> i=t.gettarinfo(u'mac.txt', u'mac.txt')
>>> t.addfile(i, file(u'mac.txt', 'r'))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.5/tarfile.py", line 1422, in addfile
    self.fileobj.write(tarinfo.tobuf(self.posix))
  File "/usr/lib/python2.5/tarfile.py", line 871, in tobuf
    buf = struct.pack("%ds" % BLOCKSIZE, "".join(parts))
  File "/usr/lib/python2.5/struct.py", line 63, in pack
    return o.pack(*args)
struct.error: argument for 's' must be a string


----------------------------------------------------------------------

>Comment By: K. C. Wong (dvusboy)
Date: 2007-04-20 21:54

Message:
Logged In: YES 
user_id=414175
Originator: YES

I see the work around, and I have already implemented similar workarounds
in my code. However, I have 2 problem with this response:

1. The behaviour changed. As the documentation did not explicitly say
tarfile does not support unicode file paths, and it worked prior to 2.5,
then I would say breaking that behaviour at the least calls for a
documentation update.
2. The error message stamps from struct failing to pack a unicode string.
First of all, I did not grasp the subtle message of it being a unicode
string as opposed to a non-unicode string. You see, I actually did not
expect unicode string in the first place, it was a by-product of TEXT_DATA
from a DOM tree. I can understand why struct.pack() throws (because no
explicit encoding scheme was specified) but it was so cryptic with regard
to tarfile itself, that I had to modify tarfile to track down the reason
for the exception.
In short, I would prefer the owner of tarfile to make an explicit support
or not-supported call on unicode file path, document said decision and make
more reasonable attempt in presenting releavant exceptions.
Thank you for looking into this.

----------------------------------------------------------------------

Comment By: Lars Gust?bel (gustaebel)
Date: 2007-04-20 12:25

Message:
Logged In: YES 
user_id=642936
Originator: NO

tarfile.py was never guaranteed to work correctly with unicode filenames.
The fact that this works with Python < 2.5 is purely accidental.

You can work around this (sticking to your example):

i = t.gettarinfo(u'mac.txt', 'mac.txt')

or:

i = t.gettarinfo('mac.txt')


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1704156&group_id=5470

From noreply at sourceforge.net  Sat Apr 21 09:33:24 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Sat, 21 Apr 2007 00:33:24 -0700
Subject: [ python-Bugs-1700912 ] questionable results from IDLE, Windows,
	& several built-in 
Message-ID: <E1HfA60-0000s4-5l@sc8-sf-web24.sourceforge.net>

Bugs item #1700912, was opened at 2007-04-15 03:28
Message generated for change (Comment added) made by bavantgarde314
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1700912&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Windows
Group: Python 2.5
>Status: Closed
Resolution: None
Priority: 5
Private: No
Submitted By: bavantgarde314 (bavantgarde314)
Assigned to: Nobody/Anonymous (nobody)
Summary: questionable results from IDLE, Windows, & several built-in 

Initial Comment:
Python 2.5 (r25:51908, Sep 19 2006, 09:52:17) [MSC v.1310 32 bit (Intel)] on win32

Windows(ME)


Attachment reveals questionable results from IDLE, Windows, & several built-in functions.
        
Under IDLE: function swapcase() updates 120 characters.
Under Windows (dos-screen) or a Windows ide swapcase() updates only 52 characters: ord values 65-90 for (uppercase)A-Z and 97-122 for (lowercase)a-z.
                   
Under IDLE: functions lower() & upper() return different results than swapcase()
for hex values:  8a,  8c,  8e,  9a,  9c,  9e,  9f, ff  
 or ord values  138, 140, 142, 154, 156, 158, 159, 255
 or characters   ?    ?    ?    ?    ?    ?    ?    ?




----------------------------------------------------------------------

>Comment By: bavantgarde314 (bavantgarde314)
Date: 2007-04-21 02:33

Message:
Logged In: YES 
user_id=1769863
Originator: YES

Thanks

import locale
locale.setlocale(locale.LC_ALL, '')

solved my problems.


----------------------------------------------------------------------

Comment By: Josiah Carlson (josiahcarlson)
Date: 2007-04-19 00:02

Message:
Logged In: YES 
user_id=341410
Originator: NO

The behaviors of those functions depend on the locale.

Check the results of:

import locale
locale.getlocale()


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1700912&group_id=5470

From noreply at sourceforge.net  Sat Apr 21 09:36:06 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Sat, 21 Apr 2007 00:36:06 -0700
Subject: [ python-Bugs-1700912 ] questionable results from IDLE, Windows,
	& several built-in 
Message-ID: <E1HfA8c-0001CT-Vx@sc8-sf-web24.sourceforge.net>

Bugs item #1700912, was opened at 2007-04-15 03:28
Message generated for change (Comment added) made by bavantgarde314
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1700912&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Windows
Group: Python 2.5
Status: Closed
Resolution: None
Priority: 5
Private: No
Submitted By: bavantgarde314 (bavantgarde314)
Assigned to: Nobody/Anonymous (nobody)
Summary: questionable results from IDLE, Windows, & several built-in 

Initial Comment:
Python 2.5 (r25:51908, Sep 19 2006, 09:52:17) [MSC v.1310 32 bit (Intel)] on win32

Windows(ME)


Attachment reveals questionable results from IDLE, Windows, & several built-in functions.
        
Under IDLE: function swapcase() updates 120 characters.
Under Windows (dos-screen) or a Windows ide swapcase() updates only 52 characters: ord values 65-90 for (uppercase)A-Z and 97-122 for (lowercase)a-z.
                   
Under IDLE: functions lower() & upper() return different results than swapcase()
for hex values:  8a,  8c,  8e,  9a,  9c,  9e,  9f, ff  
 or ord values  138, 140, 142, 154, 156, 158, 159, 255
 or characters   ?    ?    ?    ?    ?    ?    ?    ?




----------------------------------------------------------------------

>Comment By: bavantgarde314 (bavantgarde314)
Date: 2007-04-21 02:36

Message:
Logged In: YES 
user_id=1769863
Originator: YES

Thanks

import locale
locale.setlocale(locale.LC_ALL, '')

solved my problems.


----------------------------------------------------------------------

Comment By: bavantgarde314 (bavantgarde314)
Date: 2007-04-21 02:33

Message:
Logged In: YES 
user_id=1769863
Originator: YES

Thanks

import locale
locale.setlocale(locale.LC_ALL, '')

solved my problems.


----------------------------------------------------------------------

Comment By: Josiah Carlson (josiahcarlson)
Date: 2007-04-19 00:02

Message:
Logged In: YES 
user_id=341410
Originator: NO

The behaviors of those functions depend on the locale.

Check the results of:

import locale
locale.getlocale()


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1700912&group_id=5470

From noreply at sourceforge.net  Sat Apr 21 12:41:53 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Sat, 21 Apr 2007 03:41:53 -0700
Subject: [ python-Bugs-1704790 ] Exception message when using subprocess.POpen
Message-ID: <E1HfD2P-0006gp-PL@sc8-sf-web24.sourceforge.net>

Bugs item #1704790, was opened at 2007-04-21 06:41
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1704790&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Sam Ruby (rubys)
Assigned to: Nobody/Anonymous (nobody)
Summary: Exception message when using subprocess.POpen

Initial Comment:
module.py:

om subprocess import Popen, PIPE
pid = Popen(['whoami'],stdout=PIPE,stderr=PIPE)
print pid.communicate()

main.py:

import module

Running module.py directly produces the desired result.  Running main.py, however, produces the following on Python 2.5.1c1 (release25-maint, Apr 12 2007, 21:00:25), i.e., the version of Python that is included in Ubuntu Feisty Fawn:

Exception exceptions.AttributeError: "'NoneType' object has no attribute 'maxint'" in <bound method Popen.__del__ of <subprocess.Popen object at 0xb7d49f0c>> ignored

The same script works in Python 2.4.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1704790&group_id=5470

From noreply at sourceforge.net  Sat Apr 21 12:42:34 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Sat, 21 Apr 2007 03:42:34 -0700
Subject: [ python-Bugs-1704790 ] Exception message when using subprocess.Popen
Message-ID: <E1HfD34-0007LE-K0@sc8-sf-web21.sourceforge.net>

Bugs item #1704790, was opened at 2007-04-21 06:41
Message generated for change (Settings changed) made by rubys
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1704790&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Sam Ruby (rubys)
Assigned to: Nobody/Anonymous (nobody)
>Summary: Exception message when using subprocess.Popen

Initial Comment:
module.py:

om subprocess import Popen, PIPE
pid = Popen(['whoami'],stdout=PIPE,stderr=PIPE)
print pid.communicate()

main.py:

import module

Running module.py directly produces the desired result.  Running main.py, however, produces the following on Python 2.5.1c1 (release25-maint, Apr 12 2007, 21:00:25), i.e., the version of Python that is included in Ubuntu Feisty Fawn:

Exception exceptions.AttributeError: "'NoneType' object has no attribute 'maxint'" in <bound method Popen.__del__ of <subprocess.Popen object at 0xb7d49f0c>> ignored

The same script works in Python 2.4.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1704790&group_id=5470

From noreply at sourceforge.net  Sat Apr 21 12:52:05 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Sat, 21 Apr 2007 03:52:05 -0700
Subject: [ python-Bugs-1704793 ] incorrect return value of
	unicodedata.lookup() - beoynd BMP
Message-ID: <E1HfDCH-0008RR-6g@sc8-sf-web21.sourceforge.net>

Bugs item #1704793, was opened at 2007-04-21 12:52
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1704793&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Unicode
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: vbr (vlbrom)
Assigned to: M.-A. Lemburg (lemburg)
Summary: incorrect return value of unicodedata.lookup() - beoynd BMP

Initial Comment:
There seem to be an incorrect handling of unicode characters beyond the BMP (code point higher than 0xFFFF) in the unicodedata module - function lookup() on narrow unicode python builds (python 2.5.1, Windows XPh)

>>> unicodedata.lookup("GOTHIC LETTER FAIHU")
u'\u0346'
(should be u'\U00010346' - the beginning of the literal is truncated - leading to the ambiguity - in this case u'\u0346' is a combining diacritics "COMBINING BRIDGE ABOVE")

on the contrary, the unicode string literals \N{name} work well.

>>> u"\N{GOTHIC LETTER FAIHU}"
u'\U00010346'

Unfortunately, I haven't been able to find the problematic pieces of sourcecode, so I'm not able to fix it. 

It seems, that initially the correct information on the given codepoint is used, but finally only the last four digits of the codepoint value are taken into account using the "narrow" unicode literal \uxxxx instead of \Uxxxxxxxx 
, while the same task is handled correctly by the unicodeescape codec used for unicode string literals.

vbr



----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1704793&group_id=5470

From noreply at sourceforge.net  Sat Apr 21 14:25:40 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Sat, 21 Apr 2007 05:25:40 -0700
Subject: [ python-Bugs-1704156 ] TarFile.addfile() throws a struct.error
Message-ID: <E1HfEeq-0000O1-99@sc8-sf-web24.sourceforge.net>

Bugs item #1704156, was opened at 2007-04-20 10:21
Message generated for change (Comment added) made by gustaebel
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1704156&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: K. C. Wong (dvusboy)
Assigned to: Lars Gust?bel (gustaebel)
Summary: TarFile.addfile() throws a struct.error

Initial Comment:
When adding a file to a TarFile instance using addfile(), if the file paths (name and arcname) are unicode strings, then a struct.error will the raised. Python versions prior to 2.5 do not show this behaviour.

Assuming the current directory has a file name 'mac.txt', here is an interactive session that shows the problem:

Python 2.5 (r25:51908, Apr 18 2007, 19:06:57)
[GCC 3.4.6 20060404 (Red Hat 3.4.6-3)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import tarfile
>>> t=tarfile.open('test.tar', 'w')
>>> i=t.gettarinfo(u'mac.txt', u'mac.txt')
>>> t.addfile(i, file(u'mac.txt', 'r'))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.5/tarfile.py", line 1422, in addfile
    self.fileobj.write(tarinfo.tobuf(self.posix))
  File "/usr/lib/python2.5/tarfile.py", line 871, in tobuf
    buf = struct.pack("%ds" % BLOCKSIZE, "".join(parts))
  File "/usr/lib/python2.5/struct.py", line 63, in pack
    return o.pack(*args)
struct.error: argument for 's' must be a string


----------------------------------------------------------------------

>Comment By: Lars Gust?bel (gustaebel)
Date: 2007-04-21 14:25

Message:
Logged In: YES 
user_id=642936
Originator: NO

I checked in a simple fix to the release25-maint branch (rev. 54908). I
will come up with a more solid approach for 2.6.

----------------------------------------------------------------------

Comment By: K. C. Wong (dvusboy)
Date: 2007-04-21 06:54

Message:
Logged In: YES 
user_id=414175
Originator: YES

I see the work around, and I have already implemented similar workarounds
in my code. However, I have 2 problem with this response:

1. The behaviour changed. As the documentation did not explicitly say
tarfile does not support unicode file paths, and it worked prior to 2.5,
then I would say breaking that behaviour at the least calls for a
documentation update.
2. The error message stamps from struct failing to pack a unicode string.
First of all, I did not grasp the subtle message of it being a unicode
string as opposed to a non-unicode string. You see, I actually did not
expect unicode string in the first place, it was a by-product of TEXT_DATA
from a DOM tree. I can understand why struct.pack() throws (because no
explicit encoding scheme was specified) but it was so cryptic with regard
to tarfile itself, that I had to modify tarfile to track down the reason
for the exception.
In short, I would prefer the owner of tarfile to make an explicit support
or not-supported call on unicode file path, document said decision and make
more reasonable attempt in presenting releavant exceptions.
Thank you for looking into this.

----------------------------------------------------------------------

Comment By: Lars Gust?bel (gustaebel)
Date: 2007-04-20 21:25

Message:
Logged In: YES 
user_id=642936
Originator: NO

tarfile.py was never guaranteed to work correctly with unicode filenames.
The fact that this works with Python < 2.5 is purely accidental.

You can work around this (sticking to your example):

i = t.gettarinfo(u'mac.txt', 'mac.txt')

or:

i = t.gettarinfo('mac.txt')


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1704156&group_id=5470

From noreply at sourceforge.net  Sat Apr 21 18:53:02 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Sat, 21 Apr 2007 09:53:02 -0700
Subject: [ python-Bugs-813986 ] robotparser interactively prompts for username
	and password
Message-ID: <E1HfIpa-00079D-8U@sc8-sf-web24.sourceforge.net>

Bugs item #813986, was opened at 2003-09-28 13:06
Message generated for change (Comment added) made by nagle
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=813986&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.3
Status: Open
Resolution: None
Priority: 6
Private: No
Submitted By: Erik Demaine (edemaine)
Assigned to: Martin v. L?wis (loewis)
Summary: robotparser interactively prompts for username and password

Initial Comment:
This is a rare occurrence, but if a /robots.txt file is
password-protected on an http server, robotparser
interactively prompts (via raw_input) for a username
and password, because that is urllib's default
behavior.  One example of such a URL, at least at the
time of this writing, is

http://www.cosc.canterbury.ac.nz/robots.txt

Given that robotparser and robots.txt is all about
*robots* (not interactive users), I don't think this
interactive behavior is terribly appropriate.  Attached
is a simple patch to robotparser.py to fix this
behavior, forcing urllib to return the 401 error that
it ought to.

Another issue is whether a 401 (Authorization Required)
URL means that everything should be allowed or
everything should be disallowed.  I'm not sure what's
&quot;right&quot;.  Reading the spec, it says 'This file must be
accessible via HTTP on the local URL &quot;/robots.txt&quot;'
which I would read to mean it should be accessible
without username/password.  On the other hand, the
current robotparser.py code says &quot;if self.errcode ==
401 or self.errcode == 403: self.disallow_all = 1&quot;
which has the opposite effect.  I'll leave deciding
which is most appropriate to the powers that be.


----------------------------------------------------------------------

Comment By: John Nagle (nagle)
Date: 2007-04-21 16:53

Message:
Logged In: YES 
user_id=5571
Originator: NO

The attached patch was never integrated into the distribution.  This is
still broken in Python 2.4 (Win32), Python 2.5 (Win32), and Python 2.5
(Linux).  

This stalled an overnight batch job for us.  Very annoying.

Reproduce with:

import robotparser
url = 'http://mueblesmoraleda.com' # whole site is password-protected.
parser = robotparser.RobotFileParser()
parser.set_url(url)
parser.read()	# Prompts for password


----------------------------------------------------------------------

Comment By: Wummel (calvin)
Date: 2003-09-29 13:24

Message:
Logged In: YES 
user_id=9205

http://www.robotstxt.org/wc/norobots-rfc.html specifies the
401 and 403 return code consequences as restricting the
whole site (ie disallow_all = 1).

For the password input, the patch looks good to me. On the
long term, robotparser.py should switch to urllib2.py
anyway, and it should handle Transfer-Encoding: gzip.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=813986&group_id=5470

From noreply at sourceforge.net  Sat Apr 21 22:29:52 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Sat, 21 Apr 2007 13:29:52 -0700
Subject: [ python-Bugs-1704793 ] incorrect return value of
	unicodedata.lookup() - beoynd BMP
Message-ID: <E1HfMDQ-00027S-G1@sc8-sf-web21.sourceforge.net>

Bugs item #1704793, was opened at 2007-04-21 10:52
Message generated for change (Comment added) made by gbrandl
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1704793&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Unicode
Group: Python 2.5
Status: Open
Resolution: None
>Priority: 6
Private: No
Submitted By: vbr (vlbrom)
Assigned to: M.-A. Lemburg (lemburg)
Summary: incorrect return value of unicodedata.lookup() - beoynd BMP

Initial Comment:
There seem to be an incorrect handling of unicode characters beyond the BMP (code point higher than 0xFFFF) in the unicodedata module - function lookup() on narrow unicode python builds (python 2.5.1, Windows XPh)

>>> unicodedata.lookup("GOTHIC LETTER FAIHU")
u'\u0346'
(should be u'\U00010346' - the beginning of the literal is truncated - leading to the ambiguity - in this case u'\u0346' is a combining diacritics "COMBINING BRIDGE ABOVE")

on the contrary, the unicode string literals \N{name} work well.

>>> u"\N{GOTHIC LETTER FAIHU}"
u'\U00010346'

Unfortunately, I haven't been able to find the problematic pieces of sourcecode, so I'm not able to fix it. 

It seems, that initially the correct information on the given codepoint is used, but finally only the last four digits of the codepoint value are taken into account using the "narrow" unicode literal \uxxxx instead of \Uxxxxxxxx 
, while the same task is handled correctly by the unicodeescape codec used for unicode string literals.

vbr



----------------------------------------------------------------------

>Comment By: Georg Brandl (gbrandl)
Date: 2007-04-21 20:29

Message:
Logged In: YES 
user_id=849994
Originator: NO

Confirmed with an linux-x86 UCS-4 build here.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1704793&group_id=5470

From noreply at sourceforge.net  Sat Apr 21 22:36:02 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Sat, 21 Apr 2007 13:36:02 -0700
Subject: [ python-Bugs-1704790 ] Exception message when using subprocess.Popen
Message-ID: <E1HfMJO-0007vV-4A@sc8-sf-web21.sourceforge.net>

Bugs item #1704790, was opened at 2007-04-21 10:41
Message generated for change (Comment added) made by gbrandl
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1704790&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.5
>Status: Closed
>Resolution: Fixed
Priority: 5
Private: No
Submitted By: Sam Ruby (rubys)
Assigned to: Nobody/Anonymous (nobody)
Summary: Exception message when using subprocess.Popen

Initial Comment:
module.py:

om subprocess import Popen, PIPE
pid = Popen(['whoami'],stdout=PIPE,stderr=PIPE)
print pid.communicate()

main.py:

import module

Running module.py directly produces the desired result.  Running main.py, however, produces the following on Python 2.5.1c1 (release25-maint, Apr 12 2007, 21:00:25), i.e., the version of Python that is included in Ubuntu Feisty Fawn:

Exception exceptions.AttributeError: "'NoneType' object has no attribute 'maxint'" in <bound method Popen.__del__ of <subprocess.Popen object at 0xb7d49f0c>> ignored

The same script works in Python 2.4.

----------------------------------------------------------------------

>Comment By: Georg Brandl (gbrandl)
Date: 2007-04-21 20:36

Message:
Logged In: YES 
user_id=849994
Originator: NO

Reproduced here; fixed in rev. 54918, 54919 (2.5).

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1704790&group_id=5470

From noreply at sourceforge.net  Sun Apr 22 01:49:33 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Sat, 21 Apr 2007 16:49:33 -0700
Subject: [ python-Bugs-1703110 ] Incorrect example for add_password()
Message-ID: <E1HfPKf-0002JB-GB@sc8-sf-web21.sourceforge.net>

Bugs item #1703110, was opened at 2007-04-18 09:56
Message generated for change (Comment added) made by nnorwitz
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1703110&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Documentation
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Bob Kline (bkline)
Assigned to: Nobody/Anonymous (nobody)
Summary: Incorrect example for add_password()

Initial Comment:
In the documentation for urllib2, the example [1] for using basic HTTP authentication has the following code for the call to add_password():

auth_handler.add_password('realm', 'host', 'username', 'password')

This actually worked in earlier (pre-2.4.3) versions of urllib2, but even in those older versions, the documentation for HTTPPasswordMgr.add_password() makes it clear that a URI must be passed as the second argument to the method, not a host name.

Even though the behavior in the library method has changed (passing a full URI versions of Python earlier than 2.4.3 didn't work, at least when a non-standard port was specified), the documentation has not changed (the add_password() documentation specifies a URI, even in the pre-2.4.3 versions of Python) and the examples in the documentation for urllib2 still say 'host'.

[1] http://docs.python.org/lib/urllib2-examples.html

----------------------------------------------------------------------

>Comment By: Neal Norwitz (nnorwitz)
Date: 2007-04-21 16:49

Message:
Logged In: YES 
user_id=33168
Originator: NO

Bob, can you submit a patch to correct these problems?  Note:  we no
longer support the 2.4 series.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1703110&group_id=5470

From noreply at sourceforge.net  Sun Apr 22 04:55:57 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Sat, 21 Apr 2007 19:55:57 -0700
Subject: [ python-Bugs-451607 ] SSL support in socket module needs tests
Message-ID: <E1HfSF3-0005gr-QX@sc8-sf-web21.sourceforge.net>

Bugs item #451607, was opened at 2001-08-16 13:00
Message generated for change (Comment added) made by facundobatista
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=451607&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Extension Modules
Group: None
>Status: Closed
>Resolution: Fixed
Priority: 5
Private: No
Submitted By: Barry A. Warsaw (bwarsaw)
Assigned to: Facundo Batista (facundobatista)
Summary: SSL support in socket module needs tests

Initial Comment:
SSL support in the socket module should have a
regression test, either in test_socket.py or in a
separate test_ssl.py file.

----------------------------------------------------------------------

>Comment By: Facundo Batista (facundobatista)
Date: 2007-04-21 23:55

Message:
Logged In: YES 
user_id=752496
Originator: NO

I implemented local tests in test_socket_ssl. If present, it starts
openssl and then tests against it.

All the SSL methods are tested.

Feel free to add more complex SSL tests, all infrastructure is now in
place.

----------------------------------------------------------------------

Comment By: Facundo Batista (facundobatista)
Date: 2007-03-30 21:45

Message:
Logged In: YES 
user_id=752496
Originator: NO

I'll try to test the socket to a (when available) local openssl server.

----------------------------------------------------------------------

Comment By: Martin v. L?wis (loewis)
Date: 2006-04-12 05:01

Message:
Logged In: YES 
user_id=21627

I don't think I will do anything about this anytime soon, so
unassigning myself.

----------------------------------------------------------------------

Comment By: Martin v. L?wis (loewis)
Date: 2002-10-10 05:51

Message:
Logged In: YES 
user_id=21627

I don't think this is relevant here: OpenSSL uses whatever
device it uses; we need not to concern ourselves with
OpenSSL's choice.

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2002-10-10 05:39

Message:
Logged In: NO 

Using /dev/random for a user level application is
inappropriate.  Use /dev/urandom instead.
/dev/random actually tries to suck entropy out of the
entropy pool, and blocks if there's
not enough.  It's best to make sure there's sufficient
initial entropy in the pool, then use
/dev/urandom which uses the existing entropy to seed a
CPRNG.  Assuming the CPRNG
is properly designed, /dev/urandom should be fine for
OpenSSL, since if someone magically
breaks the cryptography in the CPRNG then they can probably
break OpenSSL's cryptography
the same way.  --phr

----------------------------------------------------------------------

Comment By: Martin v. L?wis (loewis)
Date: 2002-10-10 05:13

Message:
Logged In: YES 
user_id=21627

Things have changed a bit since: Solaris 9 has /dev/random.

----------------------------------------------------------------------

Comment By: Luke Kenneth Casson Leighton (lkcl)
Date: 2002-10-09 15:25

Message:
Logged In: YES 
user_id=80200

yes it definitely does because on solaris, which doesn't have a
/dev/random, urlretrieve fails to operate because openssl cannot
get enough entropy for the PRNG in order to make the SSL
connection.

----------------------------------------------------------------------

Comment By: Martin v. L?wis (loewis)
Date: 2001-10-11 16:39

Message:
Logged In: YES 
user_id=21627

1) When prngd runs on a TCP socket, it still won't accept
connections from remote systems.
2) Sure, but would that simplify testing?
3) My comment was actually directed at bug #232460, which is
Solaris specific: Solaris does not have a /dev/[u]random, so
prngd is the only choice. I agree that using /dev/random is
better if available. Furthermore, OpenSSL will already make
this choice for you at installation time.

----------------------------------------------------------------------

Comment By: paul rubin (phr)
Date: 2001-10-11 15:44

Message:
Logged In: YES 
user_id=72053

1) I think it's dangerous to run a prngd on an IP socket
instead of a Unix domain socket.  It creates the possibility
of (either accidentally or by ignorance) running OpenSSL
on a separate host from the prngd, making the random numbers
vulnerable to network sniffing.  That's bad--the numbers
are cryptographic secrets and should not be exposed.

2) It's simple to set up a local SSL server with the
command line openssl s_server option.  

3) I'm not crazy about the whole prngd concept.  I haven't
looked at the CHILL interface yet, but if it's possible
to abandon prngd and get random numbers through CHILL,
that might be best.  On Linux, /dev/urandom should be used.

----------------------------------------------------------------------

Comment By: Martin v. L?wis (loewis)
Date: 2001-10-11 15:32

Message:
Logged In: YES 
user_id=21627

On PRNG problems, it seems we have two options:
- wait for, then require OpenSSL 0.9.7. It will look for a
prngd socket in default locations (/var/run/egd-pool,
/dev/egd-pool, /etc/egd-pool and /etc/entropy); then require
administrators to set up OpenSSL that it indeed finds a
prngd in these locations when needed.
- expose RAND_add. Exposing any other of the interfaces is
pointless; on our installation, prngd runs on localhost:708
instead of a Unix domain socket, and none of the other
interfaces could use such a configuration. On top of
RAND_add, we could communicate with prngd ourselves, e.g. by
using the RANDFILE environment variable.


----------------------------------------------------------------------

Comment By: Martin v. L?wis (loewis)
Date: 2001-09-03 12:12

Message:
Logged In: YES 
user_id=21627

I'd suggest an approach similar to the one in test_socket,
i.e. test only if can_fork. As for the certificates: I think
they should be generated in advance and included into the
test suite.

----------------------------------------------------------------------

Comment By: Barry A. Warsaw (bwarsaw)
Date: 2001-08-28 10:53

Message:
Logged In: YES 
user_id=12800

There is currently an extremely minimal test in
test_socket_ssl.py, but it is fairly bogus because it
requires an outside network connection.  The only test there
just tries to hit https://sf.net.

Ideally, there would be a set of regression tests that only
required local resources.  I.e. it would set up a local SSL
server, then do connects to it to test the various SSL
features in socket module.

Take a look at test_socket_ssl.py in the 2.2 cvs and see if
you can improve matters.  From what I understand, the tricky
part is generating all the necessary certificates, etc.

Also, if you intend to work on this, please log in to SF to
continue this dialog so we know who we're talking to! :)

----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2001-08-28 02:58

Message:
Logged In: NO 

I may be able to help with this.  I'm fairly familiar with
SSL in general, though not about the socket module.
What's needed?


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=451607&group_id=5470

From noreply at sourceforge.net  Sun Apr 22 06:13:15 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Sat, 21 Apr 2007 21:13:15 -0700
Subject: [ python-Bugs-1704156 ] TarFile.addfile() throws a struct.error
Message-ID: <E1HfTRr-00082G-S0@sc8-sf-web22.sourceforge.net>

Bugs item #1704156, was opened at 2007-04-20 01:21
Message generated for change (Comment added) made by dvusboy
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1704156&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: K. C. Wong (dvusboy)
Assigned to: Lars Gust?bel (gustaebel)
Summary: TarFile.addfile() throws a struct.error

Initial Comment:
When adding a file to a TarFile instance using addfile(), if the file paths (name and arcname) are unicode strings, then a struct.error will the raised. Python versions prior to 2.5 do not show this behaviour.

Assuming the current directory has a file name 'mac.txt', here is an interactive session that shows the problem:

Python 2.5 (r25:51908, Apr 18 2007, 19:06:57)
[GCC 3.4.6 20060404 (Red Hat 3.4.6-3)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import tarfile
>>> t=tarfile.open('test.tar', 'w')
>>> i=t.gettarinfo(u'mac.txt', u'mac.txt')
>>> t.addfile(i, file(u'mac.txt', 'r'))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.5/tarfile.py", line 1422, in addfile
    self.fileobj.write(tarinfo.tobuf(self.posix))
  File "/usr/lib/python2.5/tarfile.py", line 871, in tobuf
    buf = struct.pack("%ds" % BLOCKSIZE, "".join(parts))
  File "/usr/lib/python2.5/struct.py", line 63, in pack
    return o.pack(*args)
struct.error: argument for 's' must be a string


----------------------------------------------------------------------

>Comment By: K. C. Wong (dvusboy)
Date: 2007-04-21 21:13

Message:
Logged In: YES 
user_id=414175
Originator: YES

Much thanks for your effort.

----------------------------------------------------------------------

Comment By: Lars Gust?bel (gustaebel)
Date: 2007-04-21 05:25

Message:
Logged In: YES 
user_id=642936
Originator: NO

I checked in a simple fix to the release25-maint branch (rev. 54908). I
will come up with a more solid approach for 2.6.

----------------------------------------------------------------------

Comment By: K. C. Wong (dvusboy)
Date: 2007-04-20 21:54

Message:
Logged In: YES 
user_id=414175
Originator: YES

I see the work around, and I have already implemented similar workarounds
in my code. However, I have 2 problem with this response:

1. The behaviour changed. As the documentation did not explicitly say
tarfile does not support unicode file paths, and it worked prior to 2.5,
then I would say breaking that behaviour at the least calls for a
documentation update.
2. The error message stamps from struct failing to pack a unicode string.
First of all, I did not grasp the subtle message of it being a unicode
string as opposed to a non-unicode string. You see, I actually did not
expect unicode string in the first place, it was a by-product of TEXT_DATA
from a DOM tree. I can understand why struct.pack() throws (because no
explicit encoding scheme was specified) but it was so cryptic with regard
to tarfile itself, that I had to modify tarfile to track down the reason
for the exception.
In short, I would prefer the owner of tarfile to make an explicit support
or not-supported call on unicode file path, document said decision and make
more reasonable attempt in presenting releavant exceptions.
Thank you for looking into this.

----------------------------------------------------------------------

Comment By: Lars Gust?bel (gustaebel)
Date: 2007-04-20 12:25

Message:
Logged In: YES 
user_id=642936
Originator: NO

tarfile.py was never guaranteed to work correctly with unicode filenames.
The fact that this works with Python < 2.5 is purely accidental.

You can work around this (sticking to your example):

i = t.gettarinfo(u'mac.txt', 'mac.txt')

or:

i = t.gettarinfo('mac.txt')


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1704156&group_id=5470

From noreply at sourceforge.net  Sun Apr 22 08:56:50 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Sat, 21 Apr 2007 23:56:50 -0700
Subject: [ python-Bugs-1705170 ] contextmanager eats StopIteration
Message-ID: <E1HfW0A-0004rG-HE@sc8-sf-web22.sourceforge.net>

Bugs item #1705170, was opened at 2007-04-22 00:56
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1705170&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Adam Olsen (rhamphoryncus)
Assigned to: Nobody/Anonymous (nobody)
Summary: contextmanager eats StopIteration

Initial Comment:
This is the same bug recently reported by Bob Sidebotham on python-dev (http://mail.python.org/pipermail/python-dev/2007-April/072484.html), but my test-case doesn't involve an overly broad try/except block.

It also seems similar to Bug 1462485, which has been closed and marked fixed, but I can reproduce with my test-case using both python 2.5 and python-trunk (revision 54759).

The irony is that catching the exception to add debugging forces it to be instantiated, thereby avoiding the bug.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1705170&group_id=5470

From noreply at sourceforge.net  Sun Apr 22 21:14:40 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Sun, 22 Apr 2007 12:14:40 -0700
Subject: [ python-Bugs-1705362 ] cannot change cursor color in IDLE 
Message-ID: <E1HfhWC-0008E0-9M@sc8-sf-web22.sourceforge.net>

Bugs item #1705362, was opened at 2007-04-22 19:14
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1705362&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: zorkin (zorkin)
Assigned to: Nobody/Anonymous (nobody)
Summary: cannot change cursor color in IDLE 

Initial Comment:
Python 2.5.1 for windows
When I change the cursor color to another color, it still stays black, so I can't really make black background.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1705362&group_id=5470

From noreply at sourceforge.net  Sun Apr 22 21:20:48 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Sun, 22 Apr 2007 12:20:48 -0700
Subject: [ python-Bugs-1705365 ] 'nonlocal x' at top level crashes interpreter
Message-ID: <E1Hfhc8-0000Qj-5Y@sc8-sf-web22.sourceforge.net>

Bugs item #1705365, was opened at 2007-04-22 12:20
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1705365&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Interpreter Core
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: John Reese (johnreese)
Assigned to: Nobody/Anonymous (nobody)
Summary: 'nonlocal x' at top level crashes interpreter

Initial Comment:
A nonlocal declaration at the interpreter top level causes a bus error.  This happens in the p3yk branch at head (r54922).  Reporter is using MacOS 10.4.9 on Intel.  Reporter is perfectly aware that nonlocal doesn't do anything useful at the top level, but feels it should not crash the interpreter.


% gdb python3.0                                          
GNU gdb 6.3.50-20050815 (Apple version gdb-563) (Wed Jul 19 05:10:58 GMT 2006)
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "i386-apple-darwin"...Reading symbols for shared libraries .. done

(gdb) run
Starting program: /Users/jtr/bin/python3.0 
Reading symbols for shared libraries . done
Python 3.0x (p3yk:54922, Apr 22 2007, 12:15:19) 
[GCC 4.0.1 (Apple Computer, Inc. build 5363)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> nonlocal x

Program received signal EXC_BAD_ACCESS, Could not access memory.
Reason: KERN_PROTECTION_FAILURE at address: 0x00000004
PySet_Contains (anyset=0x0, key=0x532640) at Objects/setobject.c:2129
2129            if (!PyAnySet_Check(anyset)) {
(gdb) The program is running.  Exit anyway? (y or n) y


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1705365&group_id=5470

From noreply at sourceforge.net  Sun Apr 22 22:38:28 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Sun, 22 Apr 2007 13:38:28 -0700
Subject: [ python-Bugs-1705393 ] Select() failure (race condition)
Message-ID: <E1HfipI-0008MH-1P@sc8-sf-web23.sourceforge.net>

Bugs item #1705393, was opened at 2007-04-22 13:38
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1705393&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Ron Garret (rongarret)
Assigned to: Nobody/Anonymous (nobody)
Summary: Select() failure (race condition)

Initial Comment:
select.select fails on file-like objects created by socket.makefile when data exists in the file buffer but not in the underlying socket.

Here is code to reproduce the bug.  Run it, then point a web browser at port 8080 and observer that select times out indicating that no input is available even though input is still in fact available.

=======

from SocketServer import *
from socket import *
from select import select

class myHandler(StreamRequestHandler):
  def handle(self):
    while 1:
      sl = select([self.rfile],[],[],1)[0]
      print sl
      l = self.rfile.readline()
      if len(l)<3: break
      print l,
      pass
    print>>self.wfile, 'HTTP/1.0 200 OK'
    print>>self.wfile, 'Content-type: text/plain'
    print>>self.wfile
    print>>self.wfile, 'foo'

server = TCPServer(('',8080), myHandler)
server.serve_forever()


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1705393&group_id=5470

From noreply at sourceforge.net  Sun Apr 22 23:00:18 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Sun, 22 Apr 2007 14:00:18 -0700
Subject: [ python-Bugs-1703110 ] Incorrect example for add_password()
Message-ID: <E1HfjAQ-0002qW-4z@sc8-sf-web22.sourceforge.net>

Bugs item #1703110, was opened at 2007-04-18 12:56
Message generated for change (Comment added) made by bkline
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1703110&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Documentation
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Bob Kline (bkline)
Assigned to: Nobody/Anonymous (nobody)
Summary: Incorrect example for add_password()

Initial Comment:
In the documentation for urllib2, the example [1] for using basic HTTP authentication has the following code for the call to add_password():

auth_handler.add_password('realm', 'host', 'username', 'password')

This actually worked in earlier (pre-2.4.3) versions of urllib2, but even in those older versions, the documentation for HTTPPasswordMgr.add_password() makes it clear that a URI must be passed as the second argument to the method, not a host name.

Even though the behavior in the library method has changed (passing a full URI versions of Python earlier than 2.4.3 didn't work, at least when a non-standard port was specified), the documentation has not changed (the add_password() documentation specifies a URI, even in the pre-2.4.3 versions of Python) and the examples in the documentation for urllib2 still say 'host'.

[1] http://docs.python.org/lib/urllib2-examples.html

----------------------------------------------------------------------

>Comment By: Bob Kline (bkline)
Date: 2007-04-22 17:00

Message:
Logged In: YES 
user_id=291529
Originator: YES

If I'm not right in assuming that the documentation is generated directly
from urllib2.py let me know and I'll try to find the right separate file in
svn.
File Added: urllib2.py.patch

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2007-04-21 19:49

Message:
Logged In: YES 
user_id=33168
Originator: NO

Bob, can you submit a patch to correct these problems?  Note:  we no
longer support the 2.4 series.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1703110&group_id=5470

From noreply at sourceforge.net  Sun Apr 22 23:12:05 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Sun, 22 Apr 2007 14:12:05 -0700
Subject: [ python-Bugs-813986 ] robotparser interactively prompts for username
	and password
Message-ID: <E1HfjLp-0008Ka-A6@sc8-sf-web24.sourceforge.net>

Bugs item #813986, was opened at 2003-09-28 13:06
Message generated for change (Comment added) made by nagle
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=813986&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.3
Status: Open
Resolution: None
Priority: 6
Private: No
Submitted By: Erik Demaine (edemaine)
Assigned to: Martin v. L?wis (loewis)
Summary: robotparser interactively prompts for username and password

Initial Comment:
This is a rare occurrence, but if a /robots.txt file is
password-protected on an http server, robotparser
interactively prompts (via raw_input) for a username
and password, because that is urllib's default
behavior.  One example of such a URL, at least at the
time of this writing, is

http://www.cosc.canterbury.ac.nz/robots.txt

Given that robotparser and robots.txt is all about
*robots* (not interactive users), I don't think this
interactive behavior is terribly appropriate.  Attached
is a simple patch to robotparser.py to fix this
behavior, forcing urllib to return the 401 error that
it ought to.

Another issue is whether a 401 (Authorization Required)
URL means that everything should be allowed or
everything should be disallowed.  I'm not sure what's
&quot;right&quot;.  Reading the spec, it says 'This file must be
accessible via HTTP on the local URL &quot;/robots.txt&quot;'
which I would read to mean it should be accessible
without username/password.  On the other hand, the
current robotparser.py code says &quot;if self.errcode ==
401 or self.errcode == 403: self.disallow_all = 1&quot;
which has the opposite effect.  I'll leave deciding
which is most appropriate to the powers that be.


----------------------------------------------------------------------

Comment By: John Nagle (nagle)
Date: 2007-04-22 21:12

Message:
Logged In: YES 
user_id=5571
Originator: NO

I tried the patch by doing this:

import robotparser   
def prompt_user_passwd(self, host, realm):
    return None, None
robotparser.URLopener.prompt_user_passwd = prompt_user_passwd    # temp
patch


This dealt with the problem effectively; robots.txt files are being
processed normally, and if reading one causes an authentication request,
it's handled as if no password was input, without any interaction.  

So this could probably go in.

----------------------------------------------------------------------

Comment By: John Nagle (nagle)
Date: 2007-04-21 16:53

Message:
Logged In: YES 
user_id=5571
Originator: NO

The attached patch was never integrated into the distribution.  This is
still broken in Python 2.4 (Win32), Python 2.5 (Win32), and Python 2.5
(Linux).  

This stalled an overnight batch job for us.  Very annoying.

Reproduce with:

import robotparser
url = 'http://mueblesmoraleda.com' # whole site is password-protected.
parser = robotparser.RobotFileParser()
parser.set_url(url)
parser.read()	# Prompts for password


----------------------------------------------------------------------

Comment By: Wummel (calvin)
Date: 2003-09-29 13:24

Message:
Logged In: YES 
user_id=9205

http://www.robotstxt.org/wc/norobots-rfc.html specifies the
401 and 403 return code consequences as restricting the
whole site (ie disallow_all = 1).

For the password input, the patch looks good to me. On the
long term, robotparser.py should switch to urllib2.py
anyway, and it should handle Transfer-Encoding: gzip.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=813986&group_id=5470

From noreply at sourceforge.net  Mon Apr 23 00:23:14 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Sun, 22 Apr 2007 15:23:14 -0700
Subject: [ python-Bugs-1705365 ] 'nonlocal x' at top level crashes interpreter
Message-ID: <E1HfkSg-0007a9-7U@sc8-sf-web24.sourceforge.net>

Bugs item #1705365, was opened at 2007-04-22 19:20
Message generated for change (Comment added) made by gbrandl
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1705365&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Interpreter Core
>Group: Python 3000
Status: Open
Resolution: None
>Priority: 6
Private: No
Submitted By: John Reese (johnreese)
>Assigned to: Nick Coghlan (ncoghlan)
Summary: 'nonlocal x' at top level crashes interpreter

Initial Comment:
A nonlocal declaration at the interpreter top level causes a bus error.  This happens in the p3yk branch at head (r54922).  Reporter is using MacOS 10.4.9 on Intel.  Reporter is perfectly aware that nonlocal doesn't do anything useful at the top level, but feels it should not crash the interpreter.


% gdb python3.0                                          
GNU gdb 6.3.50-20050815 (Apple version gdb-563) (Wed Jul 19 05:10:58 GMT 2006)
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "i386-apple-darwin"...Reading symbols for shared libraries .. done

(gdb) run
Starting program: /Users/jtr/bin/python3.0 
Reading symbols for shared libraries . done
Python 3.0x (p3yk:54922, Apr 22 2007, 12:15:19) 
[GCC 4.0.1 (Apple Computer, Inc. build 5363)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> nonlocal x

Program received signal EXC_BAD_ACCESS, Could not access memory.
Reason: KERN_PROTECTION_FAILURE at address: 0x00000004
PySet_Contains (anyset=0x0, key=0x532640) at Objects/setobject.c:2129
2129            if (!PyAnySet_Check(anyset)) {
(gdb) The program is running.  Exit anyway? (y or n) y


----------------------------------------------------------------------

>Comment By: Georg Brandl (gbrandl)
Date: 2007-04-22 22:23

Message:
Logged In: YES 
user_id=849994
Originator: NO

Nick, that crash is in the symtable code you refactored with the setcomp
patch.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1705365&group_id=5470

From noreply at sourceforge.net  Mon Apr 23 00:25:13 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Sun, 22 Apr 2007 15:25:13 -0700
Subject: [ python-Bugs-1703110 ] Incorrect example for add_password()
Message-ID: <E1HfkUb-0007oW-CD@sc8-sf-web24.sourceforge.net>

Bugs item #1703110, was opened at 2007-04-18 16:56
Message generated for change (Comment added) made by gbrandl
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1703110&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Documentation
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Bob Kline (bkline)
Assigned to: Nobody/Anonymous (nobody)
Summary: Incorrect example for add_password()

Initial Comment:
In the documentation for urllib2, the example [1] for using basic HTTP authentication has the following code for the call to add_password():

auth_handler.add_password('realm', 'host', 'username', 'password')

This actually worked in earlier (pre-2.4.3) versions of urllib2, but even in those older versions, the documentation for HTTPPasswordMgr.add_password() makes it clear that a URI must be passed as the second argument to the method, not a host name.

Even though the behavior in the library method has changed (passing a full URI versions of Python earlier than 2.4.3 didn't work, at least when a non-standard port was specified), the documentation has not changed (the add_password() documentation specifies a URI, even in the pre-2.4.3 versions of Python) and the examples in the documentation for urllib2 still say 'host'.

[1] http://docs.python.org/lib/urllib2-examples.html

----------------------------------------------------------------------

>Comment By: Georg Brandl (gbrandl)
Date: 2007-04-22 22:25

Message:
Logged In: YES 
user_id=849994
Originator: NO

The docs are maintained separately, you'll have to look at
Doc/lib/liburllib2.tex.

----------------------------------------------------------------------

Comment By: Bob Kline (bkline)
Date: 2007-04-22 21:00

Message:
Logged In: YES 
user_id=291529
Originator: YES

If I'm not right in assuming that the documentation is generated directly
from urllib2.py let me know and I'll try to find the right separate file in
svn.
File Added: urllib2.py.patch

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2007-04-21 23:49

Message:
Logged In: YES 
user_id=33168
Originator: NO

Bob, can you submit a patch to correct these problems?  Note:  we no
longer support the 2.4 series.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1703110&group_id=5470

From noreply at sourceforge.net  Mon Apr 23 00:48:18 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Sun, 22 Apr 2007 15:48:18 -0700
Subject: [ python-Bugs-1693753 ] Portability issue: os.rename behaves
	differently on win32
Message-ID: <E1Hfkqw-0003z3-Q4@sc8-sf-web21.sourceforge.net>

Bugs item #1693753, was opened at 2007-04-03 12:17
Message generated for change (Comment added) made by draghuram
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1693753&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Platform-specific
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: cs_ (cs_)
Assigned to: Nobody/Anonymous (nobody)
Summary: Portability issue: os.rename behaves differently on win32

Initial Comment:
os.rename(src, dst) fails on Windows if dst already exist and regular file. On Unixes, it does not fail, and silently overwrites dst. It's better to use MoveFileEx(src, dst, MOVEFILE_REPLACE_EXISTING) when moving regular files under Win32 to make it behave consistent way.

----------------------------------------------------------------------

Comment By: Raghuram Devarakonda (draghuram)
Date: 2007-04-22 18:48

Message:
Logged In: YES 
user_id=984087
Originator: NO


I submitted a patch for this (1704547)

----------------------------------------------------------------------

Comment By: Raghuram Devarakonda (draghuram)
Date: 2007-04-12 12:35

Message:
Logged In: YES 
user_id=984087
Originator: NO


>From MSDN document and googling, it does look like
MOVEFILE_REPLACE_EXISTING guarantees atomicity for rename operation. I will
see if I can come up with a patch (I will first need to learn building
python on windows).

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1693753&group_id=5470

From noreply at sourceforge.net  Mon Apr 23 03:38:44 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Sun, 22 Apr 2007 18:38:44 -0700
Subject: [ python-Bugs-1703110 ] Incorrect example for add_password()
Message-ID: <E1HfnVs-0006yC-M2@sc8-sf-web23.sourceforge.net>

Bugs item #1703110, was opened at 2007-04-18 12:56
Message generated for change (Comment added) made by bkline
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1703110&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Documentation
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Bob Kline (bkline)
Assigned to: Nobody/Anonymous (nobody)
Summary: Incorrect example for add_password()

Initial Comment:
In the documentation for urllib2, the example [1] for using basic HTTP authentication has the following code for the call to add_password():

auth_handler.add_password('realm', 'host', 'username', 'password')

This actually worked in earlier (pre-2.4.3) versions of urllib2, but even in those older versions, the documentation for HTTPPasswordMgr.add_password() makes it clear that a URI must be passed as the second argument to the method, not a host name.

Even though the behavior in the library method has changed (passing a full URI versions of Python earlier than 2.4.3 didn't work, at least when a non-standard port was specified), the documentation has not changed (the add_password() documentation specifies a URI, even in the pre-2.4.3 versions of Python) and the examples in the documentation for urllib2 still say 'host'.

[1] http://docs.python.org/lib/urllib2-examples.html

----------------------------------------------------------------------

>Comment By: Bob Kline (bkline)
Date: 2007-04-22 21:38

Message:
Logged In: YES 
user_id=291529
Originator: YES

File Added: bug1703110.patch

----------------------------------------------------------------------

Comment By: Georg Brandl (gbrandl)
Date: 2007-04-22 18:25

Message:
Logged In: YES 
user_id=849994
Originator: NO

The docs are maintained separately, you'll have to look at
Doc/lib/liburllib2.tex.

----------------------------------------------------------------------

Comment By: Bob Kline (bkline)
Date: 2007-04-22 17:00

Message:
Logged In: YES 
user_id=291529
Originator: YES

If I'm not right in assuming that the documentation is generated directly
from urllib2.py let me know and I'll try to find the right separate file in
svn.
File Added: urllib2.py.patch

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2007-04-21 19:49

Message:
Logged In: YES 
user_id=33168
Originator: NO

Bob, can you submit a patch to correct these problems?  Note:  we no
longer support the 2.4 series.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1703110&group_id=5470

From noreply at sourceforge.net  Mon Apr 23 05:56:38 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Sun, 22 Apr 2007 20:56:38 -0700
Subject: [ python-Feature Requests-1705520 ] pyunit should allow __unittest in
	locals to trim stackframes
Message-ID: <E1HfpfK-0005sJ-9Q@sc8-sf-web22.sourceforge.net>

Feature Requests item #1705520, was opened at 2007-04-23 13:56
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1705520&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.6
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Robert Collins (rbcollins)
Assigned to: Nobody/Anonymous (nobody)
Summary: pyunit should allow __unittest in locals to trim stackframes

Initial Comment:
pyunit looks for __unittest = 1 in the globals of functions in an assertion stacktrace. This is used to trim the trace so that unittest implementation methods do  not show up.

It would be great if it looked in the locals of the frame as well, as this would allow subclasses of TestCase that add new assertFOO methods to have them filtered in the same manner.

e.g. (bad example :))
def assertComplexState(self, inputs):
    __unittest = 1
    self.assertEqual('42', inputs[0], 'the input %s is not the right answer' % inputs)



----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1705520&group_id=5470

From noreply at sourceforge.net  Mon Apr 23 06:53:44 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Sun, 22 Apr 2007 21:53:44 -0700
Subject: [ python-Bugs-1705393 ] Select() failure (race condition)
Message-ID: <E1HfqYa-0001PW-Ql@sc8-sf-web23.sourceforge.net>

Bugs item #1705393, was opened at 2007-04-22 17:38
Message generated for change (Comment added) made by gagenellina
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1705393&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Ron Garret (rongarret)
Assigned to: Nobody/Anonymous (nobody)
Summary: Select() failure (race condition)

Initial Comment:
select.select fails on file-like objects created by socket.makefile when data exists in the file buffer but not in the underlying socket.

Here is code to reproduce the bug.  Run it, then point a web browser at port 8080 and observer that select times out indicating that no input is available even though input is still in fact available.

=======

from SocketServer import *
from socket import *
from select import select

class myHandler(StreamRequestHandler):
  def handle(self):
    while 1:
      sl = select([self.rfile],[],[],1)[0]
      print sl
      l = self.rfile.readline()
      if len(l)<3: break
      print l,
      pass
    print>>self.wfile, 'HTTP/1.0 200 OK'
    print>>self.wfile, 'Content-type: text/plain'
    print>>self.wfile
    print>>self.wfile, 'foo'

server = TCPServer(('',8080), myHandler)
server.serve_forever()


----------------------------------------------------------------------

Comment By: Gabriel Genellina (gagenellina)
Date: 2007-04-23 01:53

Message:
Logged In: YES 
user_id=479790
Originator: NO

The handler should not do its own select() call. (And even then, select
should use the underlying socket, not the wrapping pseudo-file object).
Just removing the select() call works fine.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1705393&group_id=5470

From noreply at sourceforge.net  Mon Apr 23 07:21:17 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Sun, 22 Apr 2007 22:21:17 -0700
Subject: [ python-Bugs-1703448 ] "t.join();
	assert t not in threading.enumerate()" fails
Message-ID: <E1HfqzF-0006X6-VE@sc8-sf-web22.sourceforge.net>

Bugs item #1703448, was opened at 2007-04-19 05:22
Message generated for change (Comment added) made by gagenellina
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1703448&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Andrew Bennetts (spiv)
Assigned to: Raymond Hettinger (rhettinger)
Summary: "t.join(); assert t not in threading.enumerate()" fails

Initial Comment:
This script always fails for me, usually on the very first iteration:

----
import threading

while True:
    print '.',
    t = threading.Thread(target=lambda: None)
    t.start()
    t.join()
    alive_threads = threading.enumerate()
    assert len(alive_threads) == 1, alive_threads
----

The docs clearly say this is expected to work: Thread.join is described as "Wait until the thread terminates" and threading.enumerate's description says it will "Return a list of all currently active Thread objects ... It excludes terminated threads".  So by the time t.join() returns, the thread should not be reported by threading.enumerate anymore.

The bug appears to be that while the thread is shutting down, join() may exit too early: it waits for the __stopped flag to be set, but the thread hasn't necessarily been removed from the _active dict by that time, so enumerate will report a "stopped" thread.  (Or if you like the bug is that __stopped is can be set while the thread is still in the _active dict.)

A workaround is to filter the results of threading.enumerate() through [t for t in threading.enumerate() if t.isAlive()].

----------------------------------------------------------------------

Comment By: Gabriel Genellina (gagenellina)
Date: 2007-04-23 02:21

Message:
Logged In: YES 
user_id=479790
Originator: NO

The analysis appears to be correct, but I can't reproduce the bug with 2.5
on Windows XP SP2.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1703448&group_id=5470

From noreply at sourceforge.net  Mon Apr 23 07:48:52 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Sun, 22 Apr 2007 22:48:52 -0700
Subject: [ python-Bugs-1705362 ] cannot change cursor color in IDLE 
Message-ID: <E1HfrPw-0000z2-Oo@sc8-sf-web22.sourceforge.net>

Bugs item #1705362, was opened at 2007-04-22 16:14
Message generated for change (Comment added) made by gagenellina
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1705362&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: zorkin (zorkin)
Assigned to: Nobody/Anonymous (nobody)
Summary: cannot change cursor color in IDLE 

Initial Comment:
Python 2.5.1 for windows
When I change the cursor color to another color, it still stays black, so I can't really make black background.

----------------------------------------------------------------------

Comment By: Gabriel Genellina (gagenellina)
Date: 2007-04-23 02:48

Message:
Logged In: YES 
user_id=479790
Originator: NO

IDLE on Windows appears to be using an always-black caret (cursor) - not
the standard caret behavior, which does its "blinking" by inverting the
background color.
So something might be improved here, but I doubt you can reliably set the
caret color on Windows; see http://support.microsoft.com/kb/84054/en-us


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1705362&group_id=5470

From noreply at sourceforge.net  Mon Apr 23 07:51:56 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Sun, 22 Apr 2007 22:51:56 -0700
Subject: [ python-Bugs-1703448 ] "t.join();
	assert t not in threading.enumerate()" fails
Message-ID: <E1HfrSu-0004ZM-Ty@sc8-sf-web23.sourceforge.net>

Bugs item #1703448, was opened at 2007-04-19 05:22
Message generated for change (Comment added) made by gagenellina
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1703448&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Andrew Bennetts (spiv)
Assigned to: Raymond Hettinger (rhettinger)
Summary: "t.join(); assert t not in threading.enumerate()" fails

Initial Comment:
This script always fails for me, usually on the very first iteration:

----
import threading

while True:
    print '.',
    t = threading.Thread(target=lambda: None)
    t.start()
    t.join()
    alive_threads = threading.enumerate()
    assert len(alive_threads) == 1, alive_threads
----

The docs clearly say this is expected to work: Thread.join is described as "Wait until the thread terminates" and threading.enumerate's description says it will "Return a list of all currently active Thread objects ... It excludes terminated threads".  So by the time t.join() returns, the thread should not be reported by threading.enumerate anymore.

The bug appears to be that while the thread is shutting down, join() may exit too early: it waits for the __stopped flag to be set, but the thread hasn't necessarily been removed from the _active dict by that time, so enumerate will report a "stopped" thread.  (Or if you like the bug is that __stopped is can be set while the thread is still in the _active dict.)

A workaround is to filter the results of threading.enumerate() through [t for t in threading.enumerate() if t.isAlive()].

----------------------------------------------------------------------

Comment By: Gabriel Genellina (gagenellina)
Date: 2007-04-23 02:51

Message:
Logged In: YES 
user_id=479790
Originator: NO

The analysis appears to be correct, but I can't reproduce the bug with 2.5
on Windows XP SP2.


----------------------------------------------------------------------

Comment By: Gabriel Genellina (gagenellina)
Date: 2007-04-23 02:21

Message:
Logged In: YES 
user_id=479790
Originator: NO

The analysis appears to be correct, but I can't reproduce the bug with 2.5
on Windows XP SP2.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1703448&group_id=5470

From noreply at sourceforge.net  Mon Apr 23 08:00:25 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Sun, 22 Apr 2007 23:00:25 -0700
Subject: [ python-Bugs-1705393 ] Select() failure (race condition)
Message-ID: <E1Hfrb7-0002Gb-6v@sc8-sf-web22.sourceforge.net>

Bugs item #1705393, was opened at 2007-04-22 13:38
Message generated for change (Comment added) made by rongarret
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1705393&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Ron Garret (rongarret)
Assigned to: Nobody/Anonymous (nobody)
Summary: Select() failure (race condition)

Initial Comment:
select.select fails on file-like objects created by socket.makefile when data exists in the file buffer but not in the underlying socket.

Here is code to reproduce the bug.  Run it, then point a web browser at port 8080 and observer that select times out indicating that no input is available even though input is still in fact available.

=======

from SocketServer import *
from socket import *
from select import select

class myHandler(StreamRequestHandler):
  def handle(self):
    while 1:
      sl = select([self.rfile],[],[],1)[0]
      print sl
      l = self.rfile.readline()
      if len(l)<3: break
      print l,
      pass
    print>>self.wfile, 'HTTP/1.0 200 OK'
    print>>self.wfile, 'Content-type: text/plain'
    print>>self.wfile
    print>>self.wfile, 'foo'

server = TCPServer(('',8080), myHandler)
server.serve_forever()


----------------------------------------------------------------------

>Comment By: Ron Garret (rongarret)
Date: 2007-04-22 23:00

Message:
Logged In: YES 
user_id=1775883
Originator: YES

>Just removing the select() call works fine.

For this simple example yes, not for more complicated cases.  My
particular application is a transparent HTTP proxy which needs to be able
to serve multiple connections at once and handle keep-alive connections. 
(The proxy connects to local server processes through unix sockets, which
is why I can't use a regular HTTP proxy.  Send me email if you want more
details about why I'm doing this.)  Without writing a full-blown HTTP-aware
proxy, select (or something like it) is necessary.

Upon further study, though, I have come to the conclusion that this is not
actually a bug (since select is doing what it is advertised to do), just
very counter-intituitive behavior.  If select is going to accept file
objects as arguments it ought to do the Right Thing with them IMO.

----------------------------------------------------------------------

Comment By: Gabriel Genellina (gagenellina)
Date: 2007-04-22 21:53

Message:
Logged In: YES 
user_id=479790
Originator: NO

The handler should not do its own select() call. (And even then, select
should use the underlying socket, not the wrapping pseudo-file object).
Just removing the select() call works fine.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1705393&group_id=5470

From noreply at sourceforge.net  Mon Apr 23 09:03:37 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Mon, 23 Apr 2007 00:03:37 -0700
Subject: [ python-Bugs-1703448 ] "t.join();
	assert t not in threading.enumerate()" fails
Message-ID: <E1HfsaH-0003p2-A6@sc8-sf-web23.sourceforge.net>

Bugs item #1703448, was opened at 2007-04-19 18:22
Message generated for change (Comment added) made by spiv
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1703448&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Andrew Bennetts (spiv)
Assigned to: Raymond Hettinger (rhettinger)
Summary: "t.join(); assert t not in threading.enumerate()" fails

Initial Comment:
This script always fails for me, usually on the very first iteration:

----
import threading

while True:
    print '.',
    t = threading.Thread(target=lambda: None)
    t.start()
    t.join()
    alive_threads = threading.enumerate()
    assert len(alive_threads) == 1, alive_threads
----

The docs clearly say this is expected to work: Thread.join is described as "Wait until the thread terminates" and threading.enumerate's description says it will "Return a list of all currently active Thread objects ... It excludes terminated threads".  So by the time t.join() returns, the thread should not be reported by threading.enumerate anymore.

The bug appears to be that while the thread is shutting down, join() may exit too early: it waits for the __stopped flag to be set, but the thread hasn't necessarily been removed from the _active dict by that time, so enumerate will report a "stopped" thread.  (Or if you like the bug is that __stopped is can be set while the thread is still in the _active dict.)

A workaround is to filter the results of threading.enumerate() through [t for t in threading.enumerate() if t.isAlive()].

----------------------------------------------------------------------

>Comment By: Andrew Bennetts (spiv)
Date: 2007-04-23 17:03

Message:
Logged In: YES 
user_id=50945
Originator: YES

gagenellina: thanks for taking a look!

For what it's worth, I can reproduce this with that script using the
Python 2.4 and 2.5 that come with Ubuntu 7.04 ("Feisty Fawn"), on an i386
laptop, so hopefully other people on a similar platform can also reproduce
this.



----------------------------------------------------------------------

Comment By: Gabriel Genellina (gagenellina)
Date: 2007-04-23 15:51

Message:
Logged In: YES 
user_id=479790
Originator: NO

The analysis appears to be correct, but I can't reproduce the bug with 2.5
on Windows XP SP2.


----------------------------------------------------------------------

Comment By: Gabriel Genellina (gagenellina)
Date: 2007-04-23 15:21

Message:
Logged In: YES 
user_id=479790
Originator: NO

The analysis appears to be correct, but I can't reproduce the bug with 2.5
on Windows XP SP2.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1703448&group_id=5470

From noreply at sourceforge.net  Mon Apr 23 09:33:50 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Mon, 23 Apr 2007 00:33:50 -0700
Subject: [ python-Bugs-1705362 ] cannot change cursor color in IDLE 
Message-ID: <E1Hft3W-0003jA-6K@sc8-sf-web22.sourceforge.net>

Bugs item #1705362, was opened at 2007-04-22 19:14
Message generated for change (Comment added) made by zorkin
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1705362&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: zorkin (zorkin)
Assigned to: Nobody/Anonymous (nobody)
Summary: cannot change cursor color in IDLE 

Initial Comment:
Python 2.5.1 for windows
When I change the cursor color to another color, it still stays black, so I can't really make black background.

----------------------------------------------------------------------

>Comment By: zorkin (zorkin)
Date: 2007-04-23 07:33

Message:
Logged In: YES 
user_id=1741501
Originator: YES

The page on the link says you can invert it. As I understand, the caret
color is either black or background color. You could force it to be
black/white.

Also you should hide this option in selector - it's misleading, because I
thought i does something.

----------------------------------------------------------------------

Comment By: Gabriel Genellina (gagenellina)
Date: 2007-04-23 05:48

Message:
Logged In: YES 
user_id=479790
Originator: NO

IDLE on Windows appears to be using an always-black caret (cursor) - not
the standard caret behavior, which does its "blinking" by inverting the
background color.
So something might be improved here, but I doubt you can reliably set the
caret color on Windows; see http://support.microsoft.com/kb/84054/en-us


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1705362&group_id=5470

From noreply at sourceforge.net  Mon Apr 23 11:31:05 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Mon, 23 Apr 2007 02:31:05 -0700
Subject: [ python-Bugs-1705717 ] Error in sys.argv documenation
Message-ID: <E1Hfusz-0007UW-C1@sc8-sf-web22.sourceforge.net>

Bugs item #1705717, was opened at 2007-04-23 02:31
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1705717&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Documentation
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Kamil Kisiel (kisielk)
Assigned to: Nobody/Anonymous (nobody)
Summary: Error in sys.argv documenation

Initial Comment:
>From the current version on http://docs.python.org/lib/module-sys.html:

argv

The list of command line arguments passed to a Python script. argv[0] is the script name (it is operating system dependent whether this is a full pathname or not). If the command was executed using the -c command line option to the interpreter, argv[0] is set to the string '-c'. If no script name was passed to the Python interpreter, argv has zero length.


The last line, "If no script name was passed to the Python interpreter, argv has zero length." does not appear to be correct, the length of argv is 1 and argv[0] is '':

kamil at zaphod:~$ python2.5
Python 2.5 (release25-maint, Dec  9 2006, 14:35:53)
[GCC 4.1.2 20061115 (prerelease) (Debian 4.1.1-20)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> len(sys.argv)
1
>>> sys.argv[0]
''
>>>


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1705717&group_id=5470

From noreply at sourceforge.net  Mon Apr 23 12:18:56 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Mon, 23 Apr 2007 03:18:56 -0700
Subject: [ python-Bugs-1705365 ] 'nonlocal x' at top level crashes interpreter
Message-ID: <E1HfvdI-0000eS-Vl@sc8-sf-web23.sourceforge.net>

Bugs item #1705365, was opened at 2007-04-23 05:20
Message generated for change (Comment added) made by ncoghlan
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1705365&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Interpreter Core
Group: Python 3000
Status: Open
Resolution: None
Priority: 6
Private: No
Submitted By: John Reese (johnreese)
Assigned to: Nick Coghlan (ncoghlan)
Summary: 'nonlocal x' at top level crashes interpreter

Initial Comment:
A nonlocal declaration at the interpreter top level causes a bus error.  This happens in the p3yk branch at head (r54922).  Reporter is using MacOS 10.4.9 on Intel.  Reporter is perfectly aware that nonlocal doesn't do anything useful at the top level, but feels it should not crash the interpreter.


% gdb python3.0                                          
GNU gdb 6.3.50-20050815 (Apple version gdb-563) (Wed Jul 19 05:10:58 GMT 2006)
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "i386-apple-darwin"...Reading symbols for shared libraries .. done

(gdb) run
Starting program: /Users/jtr/bin/python3.0 
Reading symbols for shared libraries . done
Python 3.0x (p3yk:54922, Apr 22 2007, 12:15:19) 
[GCC 4.0.1 (Apple Computer, Inc. build 5363)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> nonlocal x

Program received signal EXC_BAD_ACCESS, Could not access memory.
Reason: KERN_PROTECTION_FAILURE at address: 0x00000004
PySet_Contains (anyset=0x0, key=0x532640) at Objects/setobject.c:2129
2129            if (!PyAnySet_Check(anyset)) {
(gdb) The program is running.  Exit anyway? (y or n) y


----------------------------------------------------------------------

>Comment By: Nick Coghlan (ncoghlan)
Date: 2007-04-23 20:18

Message:
Logged In: YES 
user_id=1038590
Originator: NO

Fixed in rev 54925 to raise SyntaxError instead of crashing.

The bug appears to have been present in the original nonlocal code - both
the original code and the refactored code checked if a nonlocal variable
was in the set of bound names without first checking that the relevant
pointer was non-NULL.

----------------------------------------------------------------------

Comment By: Georg Brandl (gbrandl)
Date: 2007-04-23 08:23

Message:
Logged In: YES 
user_id=849994
Originator: NO

Nick, that crash is in the symtable code you refactored with the setcomp
patch.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1705365&group_id=5470

From noreply at sourceforge.net  Mon Apr 23 12:56:22 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Mon, 23 Apr 2007 03:56:22 -0700
Subject: [ python-Bugs-1701389 ] utf-16 codec problems with multiple file
	append
Message-ID: <E1HfwDW-0000m6-Da@sc8-sf-web24.sourceforge.net>

Bugs item #1701389, was opened at 2007-04-16 12:05
Message generated for change (Comment added) made by doerwalter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1701389&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Unicode
Group: Python 2.5
Status: Closed
Resolution: Remind
Priority: 5
Private: No
Submitted By: Iceberg Luo (iceberg4ever)
Assigned to: M.-A. Lemburg (lemburg)
Summary: utf-16 codec problems with multiple file append

Initial Comment:
This bug is similar but not exactly the same as bug215974.  (http://sourceforge.net/tracker/?group_id=5470&atid=105470&aid=215974&func=detail)

In my test, even multiple write() within an open()~close() lifespan will not cause the multi BOM phenomena mentioned in bug215974. Maybe it is because bug 215974 was somehow fixed during the past 7 years, although Lemburg classified it as WontFix. 

However, if a file is appended for more than once, by an "codecs.open('file.txt', 'a', 'utf16')", the multi BOM appears.

At the same time, the saying of "(Extra unnecessary) BOM marks are removed from the input stream by the Python UTF-16 codec" in bug215974 is not true even in today, on Python2.4.4 and Python2.5.1c1 on Windows XP.

Iceberg
------------------

PS: Did not find the "File Upload" checkbox mentioned in this web page, so I think I'd better paste the code right here...

import codecs, os

filename = "test.utf-16"
if os.path.exists(filename): os.unlink(filename)  # reset

def myOpen():
  return codecs.open(filename, "a", 'UTF-16')
def readThemBack():
  return list( codecs.open(filename, "r", 'UTF-16') )
def clumsyPatch(raw): # you can read it after your first run of this program
  for line in raw:
    if line[0] in (u'\ufffe', u'\ufeff'): # get rid of the BOMs
      yield line[1:]
    else:
      yield line

fout = myOpen()
fout.write(u"ab\n") # to simplify the problem, I only use ASCII chars here
fout.write(u"cd\n")
fout.close()
print readThemBack()
assert readThemBack() == [ u'ab\n', u'cd\n' ]
assert os.stat(filename).st_size == 14  # Only one BOM in the file

fout = myOpen()
fout.write(u"ef\n")
fout.write(u"gh\n")
fout.close()
print readThemBack()
#print list( clumsyPatch( readThemBack() ) )  # later you can enable this fix
assert readThemBack() == [ u'ab\n', u'cd\n', u'ef\n', u'gh\n' ] # fails here
assert os.stat(filename).st_size == 26  # not to mention here: multi BOM appears


----------------------------------------------------------------------

>Comment By: Walter D?rwald (doerwalter)
Date: 2007-04-23 12:56

Message:
Logged In: YES 
user_id=89016
Originator: NO

But BOMs *may* appear in normal content: Then their meaning is that of
ZERO WIDTH NO-BREAK SPACE (see
http://docs.python.org/lib/encodings-overview.html for more info).


----------------------------------------------------------------------

Comment By: Iceberg Luo (iceberg4ever)
Date: 2007-04-20 05:39

Message:
Logged In: YES 
user_id=1770538
Originator: YES

If such a bug would be fixed, either StreamWriter or StreamReader should
do something.

I can understand Doerwalter that it is somewhat not comfortable for a
StreamWriter to detect whether these is already a BOM at current file
header, especially when operating in append mode. But, IMHO, the
StreamReader should be able to detect multi BOM during its life span and
automatically ignore the non-first one, providing that a BOM is never
supposed to occur in normal content.  Not to mention that such a Reader
seems exist for a while, according to the "(extra unnecessary) BOM marks
are removed
from the input stream by the Python UTF-16 codec" in bug215974
(http://sourceforge.net/tracker/?group_id=5470&atid=105470&aid=215974&func=
detail).

Therefore I don't think a WontFix will be the proper FINAL solution for
this case.

----------------------------------------------------------------------

Comment By: Walter D?rwald (doerwalter)
Date: 2007-04-19 13:30

Message:
Logged In: YES 
user_id=89016
Originator: NO

Closing as "won't fix"

----------------------------------------------------------------------

Comment By: M.-A. Lemburg (lemburg)
Date: 2007-04-19 12:35

Message:
Logged In: YES 
user_id=38388
Originator: NO

I suggest you close this as wont fix.


----------------------------------------------------------------------

Comment By: Walter D?rwald (doerwalter)
Date: 2007-04-19 12:30

Message:
Logged In: YES 
user_id=89016
Originator: NO

append mode is simply not supported for codecs. How would the codec find
out the codec state that was active after the last characters where written
to the file?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1701389&group_id=5470

From noreply at sourceforge.net  Mon Apr 23 13:06:25 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Mon, 23 Apr 2007 04:06:25 -0700
Subject: [ python-Bugs-1697248 ] Property with -> annotation triggers assert
Message-ID: <E1HfwNF-0005wX-Kg@sc8-sf-web23.sourceforge.net>

Bugs item #1697248, was opened at 2007-04-10 09:05
Message generated for change (Comment added) made by ncoghlan
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1697248&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Parser/Compiler
Group: Python 3000
>Status: Closed
>Resolution: Fixed
Priority: 5
Private: No
Submitted By: Guido van Rossum (gvanrossum)
Assigned to: Nobody/Anonymous (nobody)
Summary: Property with -> annotation triggers assert

Initial Comment:
Example:

# x.py
class C:
  @property
  def p(self) -> int:
    return 0

$ ./python -E x.py
python: ../Python/ast.c:724: ast_for_arguments: Assertion `((n)->n_type) == 263 || ((n)->n_type) == 267' failed.
Aborted
$

Removing "@property" or "-> int" from the example avoids the problem.  Adding argument annotations inside the argument list doesn't trigger it.


----------------------------------------------------------------------

>Comment By: Nick Coghlan (ncoghlan)
Date: 2007-04-23 21:06

Message:
Logged In: YES 
user_id=1038590
Originator: NO

Fixed in rev 54926 (the check for whether or not decorators were present
wasn't quite right).

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1697248&group_id=5470

From noreply at sourceforge.net  Mon Apr 23 13:07:10 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Mon, 23 Apr 2007 04:07:10 -0700
Subject: [ python-Bugs-1705365 ] 'nonlocal x' at top level crashes interpreter
Message-ID: <E1HfwNy-000634-0e@sc8-sf-web23.sourceforge.net>

Bugs item #1705365, was opened at 2007-04-23 05:20
Message generated for change (Comment added) made by ncoghlan
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1705365&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Interpreter Core
Group: Python 3000
>Status: Closed
>Resolution: Fixed
Priority: 6
Private: No
Submitted By: John Reese (johnreese)
Assigned to: Nick Coghlan (ncoghlan)
Summary: 'nonlocal x' at top level crashes interpreter

Initial Comment:
A nonlocal declaration at the interpreter top level causes a bus error.  This happens in the p3yk branch at head (r54922).  Reporter is using MacOS 10.4.9 on Intel.  Reporter is perfectly aware that nonlocal doesn't do anything useful at the top level, but feels it should not crash the interpreter.


% gdb python3.0                                          
GNU gdb 6.3.50-20050815 (Apple version gdb-563) (Wed Jul 19 05:10:58 GMT 2006)
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "i386-apple-darwin"...Reading symbols for shared libraries .. done

(gdb) run
Starting program: /Users/jtr/bin/python3.0 
Reading symbols for shared libraries . done
Python 3.0x (p3yk:54922, Apr 22 2007, 12:15:19) 
[GCC 4.0.1 (Apple Computer, Inc. build 5363)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> nonlocal x

Program received signal EXC_BAD_ACCESS, Could not access memory.
Reason: KERN_PROTECTION_FAILURE at address: 0x00000004
PySet_Contains (anyset=0x0, key=0x532640) at Objects/setobject.c:2129
2129            if (!PyAnySet_Check(anyset)) {
(gdb) The program is running.  Exit anyway? (y or n) y


----------------------------------------------------------------------

>Comment By: Nick Coghlan (ncoghlan)
Date: 2007-04-23 21:07

Message:
Logged In: YES 
user_id=1038590
Originator: NO

Fixed in rev 54925 to raise SyntaxError instead of crashing.

The bug appears to have been present in the original nonlocal code - both
the original code and the refactored code checked if a nonlocal variable
was in the set of bound names without first checking that the relevant
pointer was non-NULL.

----------------------------------------------------------------------

Comment By: Nick Coghlan (ncoghlan)
Date: 2007-04-23 20:18

Message:
Logged In: YES 
user_id=1038590
Originator: NO

Fixed in rev 54925 to raise SyntaxError instead of crashing.

The bug appears to have been present in the original nonlocal code - both
the original code and the refactored code checked if a nonlocal variable
was in the set of bound names without first checking that the relevant
pointer was non-NULL.

----------------------------------------------------------------------

Comment By: Georg Brandl (gbrandl)
Date: 2007-04-23 08:23

Message:
Logged In: YES 
user_id=849994
Originator: NO

Nick, that crash is in the symtable code you refactored with the setcomp
patch.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1705365&group_id=5470

From noreply at sourceforge.net  Mon Apr 23 18:09:27 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Mon, 23 Apr 2007 09:09:27 -0700
Subject: [ python-Bugs-1705997 ] pydoc.py has typo.
Message-ID: <E1Hg16U-000130-WF@sc8-sf-web22.sourceforge.net>

Bugs item #1705997, was opened at 2007-04-23 11:09
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1705997&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: charlesmchen (charlesmchen)
Assigned to: Nobody/Anonymous (nobody)
Summary: pydoc.py has typo.

Initial Comment:

pydoc.py has typo.

file: C:\Python25\Lib\pydoc.py
method: locate

quote:
def locate(path, forceload=0):
    """Locate an object by name or dotted path, importing as necessary."""
    parts = [part for part in split(path, '.') if part]
    module, n = None, 0
    while n < len(parts):
        nextmodule = safeimport(join(parts[:n+1], '.'), forceload)
        if nextmodule: module, n = nextmodule, n + 1
        else: break
    if module:
        object = module
        for part in parts[n:]:
            try: object = getattr(object, part)
            except AttributeError: return None
        return object
    else:
        if hasattr(__builtin__, path):
            return getattr(__builtin__, path)

bug:
   I believe that by '__builtin__' you meant '__builtins__' in the last two lines.

        if hasattr(__builtin__, path):
            return getattr(__builtin__, path)

should read:

        if hasattr(__builtins__, path):
            return getattr(__builtins__, path)

>>> sys.platform
'win32'
>>> sys.version
'2.5 (r25:51908, Sep 19 2006, 09:52:17) [MSC v.1310 32 bit (Intel)]'

Thanks, Charles


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1705997&group_id=5470

From noreply at sourceforge.net  Tue Apr 24 02:18:53 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Mon, 23 Apr 2007 17:18:53 -0700
Subject: [ python-Feature Requests-1706256 ] Give Partial the ability to skip
	positionals
Message-ID: <E1Hg8k9-0001QM-Dj@sc8-sf-web21.sourceforge.net>

Feature Requests item #1706256, was opened at 2007-04-23 20:18
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1706256&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Calvin Spealman (ironfroggy)
Assigned to: Nobody/Anonymous (nobody)
Summary: Give Partial the ability to skip positionals

Initial Comment:
There are some situations where you want to skip positional arguments in a use of a partial function. In other words, you want to create a partial that applies positional arguments out of order or without applying values to one or more lower positional arguments. In some cases keyword arguments can be used instead, but this has two obvious drawbacks. Firstly, it causes the caller to rely on the name of a positional in a callee, which breaks encapsulation. Secondly, on the case of the function being applied to being a builtin, it fails completely, as they will not take positional arguments by name at all. I propose a class attribute to the partial type, 'skip', which will be a singleton to pass to a partial object signifying this skipping of positionals. The following example demonstrates.

from functools import partial

def add(a, b):
    return a + b

append_abc = partial(add, partial.skip, "abc")
assert append_abc("xyz") == "xyzabc"

Obviously this example would break if used as partial(add, b="abc") and the maintainer of add changed the positional names to 'first' and 'second' or 'pre' and 'post', which is perfectly reasonable. We do not need to expect the names of our positional arguments are depended upon. It would also break when someone gets smart and replaces the add function with operator.add, of course.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1706256&group_id=5470

From noreply at sourceforge.net  Tue Apr 24 05:45:29 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Mon, 23 Apr 2007 20:45:29 -0700
Subject: [ python-Feature Requests-1706256 ] Give Partial the ability to skip
	positionals
Message-ID: <E1HgBy5-0005iP-K2@sc8-sf-web24.sourceforge.net>

Feature Requests item #1706256, was opened at 2007-04-23 20:18
Message generated for change (Comment added) made by ironfroggy
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1706256&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Calvin Spealman (ironfroggy)
Assigned to: Nobody/Anonymous (nobody)
Summary: Give Partial the ability to skip positionals

Initial Comment:
There are some situations where you want to skip positional arguments in a use of a partial function. In other words, you want to create a partial that applies positional arguments out of order or without applying values to one or more lower positional arguments. In some cases keyword arguments can be used instead, but this has two obvious drawbacks. Firstly, it causes the caller to rely on the name of a positional in a callee, which breaks encapsulation. Secondly, on the case of the function being applied to being a builtin, it fails completely, as they will not take positional arguments by name at all. I propose a class attribute to the partial type, 'skip', which will be a singleton to pass to a partial object signifying this skipping of positionals. The following example demonstrates.

from functools import partial

def add(a, b):
    return a + b

append_abc = partial(add, partial.skip, "abc")
assert append_abc("xyz") == "xyzabc"

Obviously this example would break if used as partial(add, b="abc") and the maintainer of add changed the positional names to 'first' and 'second' or 'pre' and 'post', which is perfectly reasonable. We do not need to expect the names of our positional arguments are depended upon. It would also break when someone gets smart and replaces the add function with operator.add, of course.

----------------------------------------------------------------------

>Comment By: Calvin Spealman (ironfroggy)
Date: 2007-04-23 23:45

Message:
Logged In: YES 
user_id=112166
Originator: YES

File Added: partial_skip.patch

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1706256&group_id=5470

From noreply at sourceforge.net  Tue Apr 24 06:54:39 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Mon, 23 Apr 2007 21:54:39 -0700
Subject: [ python-Bugs-1703110 ] Incorrect example for add_password()
Message-ID: <E1HgD31-0002Wt-Eh@sc8-sf-web21.sourceforge.net>

Bugs item #1703110, was opened at 2007-04-18 09:56
Message generated for change (Comment added) made by nnorwitz
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1703110&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Documentation
Group: Python 2.5
>Status: Closed
>Resolution: Accepted
Priority: 5
Private: No
Submitted By: Bob Kline (bkline)
>Assigned to: Neal Norwitz (nnorwitz)
Summary: Incorrect example for add_password()

Initial Comment:
In the documentation for urllib2, the example [1] for using basic HTTP authentication has the following code for the call to add_password():

auth_handler.add_password('realm', 'host', 'username', 'password')

This actually worked in earlier (pre-2.4.3) versions of urllib2, but even in those older versions, the documentation for HTTPPasswordMgr.add_password() makes it clear that a URI must be passed as the second argument to the method, not a host name.

Even though the behavior in the library method has changed (passing a full URI versions of Python earlier than 2.4.3 didn't work, at least when a non-standard port was specified), the documentation has not changed (the add_password() documentation specifies a URI, even in the pre-2.4.3 versions of Python) and the examples in the documentation for urllib2 still say 'host'.

[1] http://docs.python.org/lib/urllib2-examples.html

----------------------------------------------------------------------

>Comment By: Neal Norwitz (nnorwitz)
Date: 2007-04-23 21:54

Message:
Logged In: YES 
user_id=33168
Originator: NO

Thanks for the patch!

Committed revision 54932.
Committed revision 54933. (2.5)


----------------------------------------------------------------------

Comment By: Bob Kline (bkline)
Date: 2007-04-22 18:38

Message:
Logged In: YES 
user_id=291529
Originator: YES

File Added: bug1703110.patch

----------------------------------------------------------------------

Comment By: Georg Brandl (gbrandl)
Date: 2007-04-22 15:25

Message:
Logged In: YES 
user_id=849994
Originator: NO

The docs are maintained separately, you'll have to look at
Doc/lib/liburllib2.tex.

----------------------------------------------------------------------

Comment By: Bob Kline (bkline)
Date: 2007-04-22 14:00

Message:
Logged In: YES 
user_id=291529
Originator: YES

If I'm not right in assuming that the documentation is generated directly
from urllib2.py let me know and I'll try to find the right separate file in
svn.
File Added: urllib2.py.patch

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2007-04-21 16:49

Message:
Logged In: YES 
user_id=33168
Originator: NO

Bob, can you submit a patch to correct these problems?  Note:  we no
longer support the 2.4 series.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1703110&group_id=5470

From noreply at sourceforge.net  Tue Apr 24 07:41:48 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Mon, 23 Apr 2007 22:41:48 -0700
Subject: [ python-Feature Requests-1706256 ] Give Partial the ability to skip
	positionals
Message-ID: <E1HgDme-0001aR-IA@sc8-sf-web22.sourceforge.net>

Feature Requests item #1706256, was opened at 2007-04-24 02:18
Message generated for change (Comment added) made by loewis
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1706256&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Calvin Spealman (ironfroggy)
Assigned to: Nobody/Anonymous (nobody)
Summary: Give Partial the ability to skip positionals

Initial Comment:
There are some situations where you want to skip positional arguments in a use of a partial function. In other words, you want to create a partial that applies positional arguments out of order or without applying values to one or more lower positional arguments. In some cases keyword arguments can be used instead, but this has two obvious drawbacks. Firstly, it causes the caller to rely on the name of a positional in a callee, which breaks encapsulation. Secondly, on the case of the function being applied to being a builtin, it fails completely, as they will not take positional arguments by name at all. I propose a class attribute to the partial type, 'skip', which will be a singleton to pass to a partial object signifying this skipping of positionals. The following example demonstrates.

from functools import partial

def add(a, b):
    return a + b

append_abc = partial(add, partial.skip, "abc")
assert append_abc("xyz") == "xyzabc"

Obviously this example would break if used as partial(add, b="abc") and the maintainer of add changed the positional names to 'first' and 'second' or 'pre' and 'post', which is perfectly reasonable. We do not need to expect the names of our positional arguments are depended upon. It would also break when someone gets smart and replaces the add function with operator.add, of course.

----------------------------------------------------------------------

>Comment By: Martin v. L?wis (loewis)
Date: 2007-04-24 07:41

Message:
Logged In: YES 
user_id=21627
Originator: NO

I would not call it partial.skip, but partial.unbound (or find yet a
better name that indicates that the argument is not skipped, but instead
will be an argument of the resulting partial function).

----------------------------------------------------------------------

Comment By: Calvin Spealman (ironfroggy)
Date: 2007-04-24 05:45

Message:
Logged In: YES 
user_id=112166
Originator: YES

File Added: partial_skip.patch

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1706256&group_id=5470

From noreply at sourceforge.net  Tue Apr 24 10:26:02 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Tue, 24 Apr 2007 01:26:02 -0700
Subject: [ python-Bugs-1046945 ] improving distutils swig support -
	documentation
Message-ID: <E1HgGLa-0000R7-5w@sc8-sf-web23.sourceforge.net>

Bugs item #1046945, was opened at 2004-10-14 12:38
Message generated for change (Comment added) made by epcylon
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1046945&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Distutils
Group: Python 2.4
Status: Open
Resolution: Accepted
Priority: 5
Private: No
Submitted By: Lars Immisch (larsimmisch)
Assigned to: Nobody/Anonymous (nobody)
Summary: improving distutils swig support - documentation

Initial Comment:
This patch updates the documentation of distutils to
describe the new swig_opts Extension keyword argument
and the new --swig-opts commandline option.

The --swig and --swig-cpp commandline options are not
mentioned.

This patch belongs to patch 1046644 or:

http://sourceforge.net/tracker/index.php?func=detail&aid=1046644&group_id=5470&atid=105470

----------------------------------------------------------------------

Comment By: Morten Lied Johansen (epcylon)
Date: 2007-04-24 10:26

Message:
Logged In: YES 
user_id=574105
Originator: NO

Err... did this patch ever get in? (I've been looking in the docs for this
information, but couldn't find it anywhere)

----------------------------------------------------------------------

Comment By: Fred L. Drake, Jr. (fdrake)
Date: 2004-10-14 16:01

Message:
Logged In: YES 
user_id=3066

This looks fine to go in after the 2.4b1 release; it's too late to 
change the docs for that one (the tarball has already been built). 

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1046945&group_id=5470

From noreply at sourceforge.net  Tue Apr 24 10:49:02 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Tue, 24 Apr 2007 01:49:02 -0700
Subject: [ python-Bugs-1706381 ] SWIG options not being followed in all cases
Message-ID: <E1HgGhq-0003zi-3W@sc8-sf-web21.sourceforge.net>

Bugs item #1706381, was opened at 2007-04-24 10:49
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1706381&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Distutils
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Morten Lied Johansen (epcylon)
Assigned to: Nobody/Anonymous (nobody)
Summary: SWIG options not being followed in all cases

Initial Comment:
I guess this is an undocumented feature, so maybe I shouldn't complain (but the patch to the docs has been in the patch tracker waiting to go in since 2004 it seems).

Default for SWIG is to produce C code, in a file called xxx_wrap.c. If you want C++, you need to specify the -c++ option (in which case you would normally want a file called xxx_wrap.cpp).


You can specify this option in two ways, either on the commandline, like so:

python setup.py build_ext --swig-opts="-c++"

or as an option to your Extention module in setup.py like so:
...
ext_modules = [Extension("_xxx", ["xxx.i","xxx.cpp"], swig_opts=["-c++"])]
...


The latter is much preferable, as it doesn't mean you have to remember to specify the option on the commandline all the time.

When specifying on the commandline, the -c++ is taken into account for creating the name of the output, and you get a file called xxx_wrap.cpp. Specifying the option in setup.py however, is not taken into account, and you get a file called xxx_wrap.c, which contains C++ code and generally doesn't compile (since the compiler assumes it's pure C).

I've attached a simple patch that seems to solve the problem.



----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1706381&group_id=5470

From noreply at sourceforge.net  Tue Apr 24 12:01:12 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Tue, 24 Apr 2007 03:01:12 -0700
Subject: [ python-Bugs-1180193 ] broken pyc files
Message-ID: <E1HgHpg-0004rw-8b@sc8-sf-web23.sourceforge.net>

Bugs item #1180193, was opened at 2005-04-10 15:10
Message generated for change (Comment added) made by zseil
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1180193&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Interpreter Core
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Armin Rigo (arigo)
Assigned to: Nobody/Anonymous (nobody)
Summary: broken pyc files

Initial Comment:
In a number of situations, the .pyc files can become "corrupted" in a subtle way: the co_filename attribute of the code objects it contains become wrong.  This can occur if we move or rename directories, or if we access the same set of files from two different locations (e.g. over NFS).

This corruption doesn't prevent the .pyc files from working, but the interpreter looses the reference to the source file.  It causes trouble in tracebacks, in the inspect module, etc.

A simple fix would be to use the following logic when importing a .py file: if there is a corresponding .pyc file, in addition to checking the timestamp, check the co_filename attribute of the loaded object.  If it doesn't point to the original .py file, discard the code object and ignore the .pyc file.

Alternatively, we could force all co_filenames to point to the .py file when loading the .pyc file.

I'll write a patch for whichever alternative seems better.

----------------------------------------------------------------------

>Comment By: Ziga Seilnacht (zseil)
Date: 2007-04-24 12:01

Message:
Logged In: YES 
user_id=1326842
Originator: NO

Here is a patch that implements arigo's last suggestion.

File Added: update_co_filename.diff

----------------------------------------------------------------------

Comment By: Ziga Seilnacht (zseil)
Date: 2007-04-03 15:46

Message:
Logged In: YES 
user_id=1326842
Originator: NO

Wouldn't your first solution be simpler? Changing all
co_filenames would require either changing various
marhal.c functions, or traversing the code object
returned by import.c/read_compiled_module().

Discarding the compiled code when the file names don't
match would be simpler and only require minor changes
in import.c/load_source_module().


----------------------------------------------------------------------

Comment By: Armin Rigo (arigo)
Date: 2007-04-03 13:31

Message:
Logged In: YES 
user_id=4771
Originator: YES

If you ask me, I think that when the importing
system finds both a .py and a .pyc for a module,
then it should ignore all co_filename and replace
them with the real path of the .py file.  I can't
see any point of not doing so.

There are many other quirks caused by .pyc files
accidentally remaining around, but we cannot fix them
all as long as the .pyc files are at the same time
a cache for performance reason and a redistributable
program format (e.g. if "rm x.py" or "svn up" deletes
a .py file, then the module is still importable via
the .pyc left behind, a great way to oversee the fact
that imports elsewhere in the project need to be
updated).

----------------------------------------------------------------------

Comment By: Ziga Seilnacht (zseil)
Date: 2007-04-03 09:16

Message:
Logged In: YES 
user_id=1326842
Originator: NO

This problem is reported quite often in the tracker,
although it shows up in different places:

http://www.python.org/sf/1666807
http://www.python.org/sf/1051638

I closed those bugs as duplicates of this one.

The logging package is also affected:

http://www.python.org/sf/1669498
http://www.python.org/sf/1633605
http://www.python.org/sf/1616422


----------------------------------------------------------------------

Comment By: Armin Rigo (arigo)
Date: 2007-03-28 15:40

Message:
Logged In: YES 
user_id=4771
Originator: YES

What I called "corruption" is the situation
where both the .py and the .pyc files are
present, but the filename stored in the .pyc
co_filenames is no longer the valid absolute
path of the corresponding .py file, for any
reason (renaming, NFS views, etc.).

This situation causes the tracebacks and the
inspect module to fail to locate the .py file,
which I consider a bug.

----------------------------------------------------------------------

Comment By: Martin v. L?wis (loewis)
Date: 2007-03-28 14:01

Message:
Logged In: YES 
user_id=21627
Originator: NO

I fail to see the corruption. It is quite desirable and normal to only
ship pyc files - that the file name they refer to is actually present is
not a requirement at all.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1180193&group_id=5470

From noreply at sourceforge.net  Tue Apr 24 12:47:21 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Tue, 24 Apr 2007 03:47:21 -0700
Subject: [ python-Feature Requests-1706460 ] access to unicodedata (via
	codepoints or 2-char surrogates)
Message-ID: <E1HgIYL-0006Gj-GC@sc8-sf-web22.sourceforge.net>

Feature Requests item #1706460, was opened at 2007-04-24 12:47
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1706460&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Unicode
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: vbr (vlbrom)
Assigned to: Nobody/Anonymous (nobody)
Summary: access to unicodedata (via codepoints or 2-char surrogates)

Initial Comment:
Currently, most functions of the unicodedata module require the unichr - unicode string of length 1 - as a parameter; for most uses it's ok, but especially while working with characters outside the BMP - (the code point over FFFF) on a narrow python build it could be quite handy, to access the properties of these characters simply using the codepoint or ordinal (since the simple unichr(x) only works for x <= FFFF on a narrow build, hence the other unicode planes are unaccessible this way).

I belive, the unicode database could be allready indexed using some numerical values like codepoints, or isn't it true?

With this improvement, the whole database could be effectively accessible also on narrow python builds, where it isn't possible to pass one-character string for codepoints over FFFF (even if the explicit limitation of unichr is bypassed, eg. by creating an unicode literal u'\Uxxxxxxxx', the resulting string consist of a surrogate pair and has obviously the length 2)

Alternatively, it could be possible, that the respective functions would also accept a two-character string, provided, this sequence can be correcly interpretted as a surrogate-pair representation of some valid unicode codepoint. 

Currently such behaviour (e.g. codepoint access) can be emulated with custom datasets derived from the unicode database, but I belive, that it should be possible to access the allready present data somehow (also on narrow builds), rather than having to duplicate it.



----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1706460&group_id=5470

From noreply at sourceforge.net  Tue Apr 24 17:10:38 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Tue, 24 Apr 2007 08:10:38 -0700
Subject: [ python-Bugs-1705717 ] Error in sys.argv documenation
Message-ID: <E1HgMf8-0007Jg-0U@sc8-sf-web23.sourceforge.net>

Bugs item #1705717, was opened at 2007-04-23 09:31
Message generated for change (Comment added) made by gbrandl
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1705717&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Documentation
Group: Python 2.5
>Status: Closed
>Resolution: Fixed
Priority: 5
Private: No
Submitted By: Kamil Kisiel (kisielk)
Assigned to: Nobody/Anonymous (nobody)
Summary: Error in sys.argv documenation

Initial Comment:
>From the current version on http://docs.python.org/lib/module-sys.html:

argv

The list of command line arguments passed to a Python script. argv[0] is the script name (it is operating system dependent whether this is a full pathname or not). If the command was executed using the -c command line option to the interpreter, argv[0] is set to the string '-c'. If no script name was passed to the Python interpreter, argv has zero length.


The last line, "If no script name was passed to the Python interpreter, argv has zero length." does not appear to be correct, the length of argv is 1 and argv[0] is '':

kamil at zaphod:~$ python2.5
Python 2.5 (release25-maint, Dec  9 2006, 14:35:53)
[GCC 4.1.2 20061115 (prerelease) (Debian 4.1.1-20)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> len(sys.argv)
1
>>> sys.argv[0]
''
>>>


----------------------------------------------------------------------

>Comment By: Georg Brandl (gbrandl)
Date: 2007-04-24 15:10

Message:
Logged In: YES 
user_id=849994
Originator: NO

Thanks for the report, fixed in rev. 54939, 54940 (2.5).

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1705717&group_id=5470

From noreply at sourceforge.net  Tue Apr 24 17:11:33 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Tue, 24 Apr 2007 08:11:33 -0700
Subject: [ python-Bugs-1705997 ] pydoc.py has typo.
Message-ID: <E1HgMg1-0007QD-An@sc8-sf-web23.sourceforge.net>

Bugs item #1705997, was opened at 2007-04-23 16:09
Message generated for change (Comment added) made by gbrandl
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1705997&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: None
>Status: Closed
>Resolution: Invalid
Priority: 5
Private: No
Submitted By: charlesmchen (charlesmchen)
Assigned to: Nobody/Anonymous (nobody)
Summary: pydoc.py has typo.

Initial Comment:

pydoc.py has typo.

file: C:\Python25\Lib\pydoc.py
method: locate

quote:
def locate(path, forceload=0):
    """Locate an object by name or dotted path, importing as necessary."""
    parts = [part for part in split(path, '.') if part]
    module, n = None, 0
    while n < len(parts):
        nextmodule = safeimport(join(parts[:n+1], '.'), forceload)
        if nextmodule: module, n = nextmodule, n + 1
        else: break
    if module:
        object = module
        for part in parts[n:]:
            try: object = getattr(object, part)
            except AttributeError: return None
        return object
    else:
        if hasattr(__builtin__, path):
            return getattr(__builtin__, path)

bug:
   I believe that by '__builtin__' you meant '__builtins__' in the last two lines.

        if hasattr(__builtin__, path):
            return getattr(__builtin__, path)

should read:

        if hasattr(__builtins__, path):
            return getattr(__builtins__, path)

>>> sys.platform
'win32'
>>> sys.version
'2.5 (r25:51908, Sep 19 2006, 09:52:17) [MSC v.1310 32 bit (Intel)]'

Thanks, Charles


----------------------------------------------------------------------

>Comment By: Georg Brandl (gbrandl)
Date: 2007-04-24 15:11

Message:
Logged In: YES 
user_id=849994
Originator: NO

I don't think so. __builtin__ is imported at the top of the module, and
its use is intentional here.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1705997&group_id=5470

From noreply at sourceforge.net  Tue Apr 24 17:27:34 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Tue, 24 Apr 2007 08:27:34 -0700
Subject: [ python-Bugs-1706381 ] SWIG options not being followed in all cases
Message-ID: <E1HgMvW-0001yU-Ct@sc8-sf-web22.sourceforge.net>

Bugs item #1706381, was opened at 2007-04-24 08:49
Message generated for change (Comment added) made by gbrandl
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1706381&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Distutils
Group: Python 2.5
>Status: Closed
>Resolution: Fixed
Priority: 5
Private: No
Submitted By: Morten Lied Johansen (epcylon)
Assigned to: Nobody/Anonymous (nobody)
Summary: SWIG options not being followed in all cases

Initial Comment:
I guess this is an undocumented feature, so maybe I shouldn't complain (but the patch to the docs has been in the patch tracker waiting to go in since 2004 it seems).

Default for SWIG is to produce C code, in a file called xxx_wrap.c. If you want C++, you need to specify the -c++ option (in which case you would normally want a file called xxx_wrap.cpp).


You can specify this option in two ways, either on the commandline, like so:

python setup.py build_ext --swig-opts="-c++"

or as an option to your Extention module in setup.py like so:
...
ext_modules = [Extension("_xxx", ["xxx.i","xxx.cpp"], swig_opts=["-c++"])]
...


The latter is much preferable, as it doesn't mean you have to remember to specify the option on the commandline all the time.

When specifying on the commandline, the -c++ is taken into account for creating the name of the output, and you get a file called xxx_wrap.cpp. Specifying the option in setup.py however, is not taken into account, and you get a file called xxx_wrap.c, which contains C++ code and generally doesn't compile (since the compiler assumes it's pure C).

I've attached a simple patch that seems to solve the problem.



----------------------------------------------------------------------

>Comment By: Georg Brandl (gbrandl)
Date: 2007-04-24 15:27

Message:
Logged In: YES 
user_id=849994
Originator: NO

Thanks for the report, fixed in rev. 54941, 54942 (2.5).

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1706381&group_id=5470

From noreply at sourceforge.net  Tue Apr 24 18:32:37 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Tue, 24 Apr 2007 09:32:37 -0700
Subject: [ python-Bugs-1568240 ] Tix is not included in 2.5 for Windows
Message-ID: <E1HgNwT-0005Dw-Oy@sc8-sf-web22.sourceforge.net>

Bugs item #1568240, was opened at 2006-09-30 12:19
Message generated for change (Comment added) made by tzot
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1568240&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Build
Group: Python 2.5
Status: Open
Resolution: None
Priority: 7
Private: No
Submitted By: Christos Georgiou (tzot)
Assigned to: Martin v. L?wis (loewis)
Summary: Tix is not included in 2.5 for Windows

Initial Comment:
(I hope "Build" is more precise than "Extension
Modules" and "Tkinter" for this specific bug.)

At least the following files are missing from 2.5 for
Windows:

DLLs\tix8184.dll
tcl\tix8184.lib
tcl\tix8.1\*

----------------------------------------------------------------------

>Comment By: Christos Georgiou (tzot)
Date: 2007-04-24 19:32

Message:
Logged In: YES 
user_id=539787
Originator: YES

This bug can be closed. Tix installs and works fine in 2.5.1, thank you
Martin.

----------------------------------------------------------------------

Comment By: Christos Georgiou (tzot)
Date: 2007-01-22 14:13

Message:
Logged In: YES 
user_id=539787
Originator: YES

For me, yes, x86 is sufficient. Hopefully for others too.

----------------------------------------------------------------------

Comment By: Martin v. L?wis (loewis)
Date: 2007-01-20 15:16

Message:
Logged In: YES 
user_id=21627
Originator: NO

It seems that I can provide Tix binaries only for x86, not for AMD64 or
Itanium. Is that sufficient?

----------------------------------------------------------------------

Comment By: Martin v. L?wis (loewis)
Date: 2007-01-03 16:59

Message:
Logged In: YES 
user_id=21627
Originator: NO

Ah, ok. No, assigning this report to Neal or bumping its priority should
not be done.

----------------------------------------------------------------------

Comment By: Christos Georgiou (tzot)
Date: 2007-01-02 12:22

Message:
Logged In: YES 
user_id=539787
Originator: YES

Neal's message is this:
http://mail.python.org/pipermail/python-dev/2006-December/070406.html
and it refers to the 2.5.1 release, not prior to it.

As you see, I refrained from both increasing the priority and assigning it
to Neal, and actually just added a comment to the case with a related
question, since I know you are the one responsible for the windows build
and you already had assigned the bug to you. My adding this comment to the
bug was nothing more or less than the action that felt appropriate, and
still does feel appropriate to me (ie I didn't overstep any limits).

The "we" was just all parties interested, and in this case, the ones I
know are at least you (responsible for the windows build) and I (a user of
Tix on windows).

Happy new year, Martin!

----------------------------------------------------------------------

Comment By: Martin v. L?wis (loewis)
Date: 2006-12-30 00:26

Message:
Logged In: YES 
user_id=21627
Originator: NO

I haven't read Neal's message yet, but I wonder what he could do about
it.

I plan to fix this with 2.5.1, there is absolutely no way to fix this
earlier.

I'm not sure who "we" is who would like to bump the bug, and what
precisely this bumping would do; tzot, please refrain from changing the
priority to higher than 7. These priorities are reserved to the release
manager.

----------------------------------------------------------------------

Comment By: Christos Georgiou (tzot)
Date: 2006-12-27 19:46

Message:
Logged In: YES 
user_id=539787
Originator: YES

Should we bump the bug up and/or assign it to Neal Norwitz as he requested
on Python-Dev?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1568240&group_id=5470

From noreply at sourceforge.net  Tue Apr 24 20:09:55 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Tue, 24 Apr 2007 11:09:55 -0700
Subject: [ python-Bugs-1706815 ] socket.error exceptions not subclass of
	StandardError
Message-ID: <E1HgPSd-0001gY-1J@sc8-sf-web21.sourceforge.net>

Bugs item #1706815, was opened at 2007-04-24 18:09
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1706815&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: John Nagle (nagle)
Assigned to: Nobody/Anonymous (nobody)
Summary: socket.error exceptions not subclass of StandardError

Initial Comment:
The "socket.error" exception is a subclass of Exception, but not of StandardError.  It needs to be placed properly in the exception hierarchy, presumably somewhere under IOError.

Socket errors have some known problems.  See also:

[ 805194 ] Inappropriate error received using socket timeout
[ 1019808 ] wrong socket error returned
[ 1571878 ] Improvements to socket module exceptions
[ 708927 ] socket timeouts produce wrong errors in win32 

Just figuring out what exceptions can be raised from the socket module is tough.  I've seen exceptions derived from "socket.error", exceptions from IOError, and exceptions from the SSL layer, which patches the
sockets module when loaded.  These are non-bug exceptions; that is, the problem is out in the network, external to the program.

Some are retryable, some indicate that a different approach (different port, different protocol) should be tried, and some mean that some named resource doesn't exist.  Programs need to make those distinctions reliably.

The most important distinction with sockets is "external network problem" vs. "local program program".  To resolve this, I suggest a "NetworkException" in the exception hierarchy, with all the things that can go wrong due to conditions external to the local machine under that exception.

I'd suggest the following:

1.  Add "NetworkError" under "IOError" in the exception hierarchy.

2.  Put the existing "socket.error" under "NetworkError". Since "socket.error" needs to be reparented anyway (it's currently a direct descendant of "Exception") this provides a good place for it.

3.  Find any places where the socket module can raise IOError or OSError due to an external network condition, and make them raise something under NetworkError instead.  Code that catches IOError will still work.

4.  Put all errors in the various SSL modules (SSLError, etc.) which can be raised due to external network conditions under "NetworkError"

5.  Move "urllib2.URLError", which is currently under IOError, down a level under NetworkError.

6.  Move the misc. errors from "urllib", like "ContentTooShortError", which are currently under IOError, down a level under NetworkError.

7.  URL translation errors from the IDNA (Unicode URL encoding) module probably should raise an error similar to that for an incorrect URL, rather than raising a UnicodeError.  

Then, programs that catch NetworkError could be sure of catching all network trouble conditions, but not local code bugs. 

With these changes, any exception that's being caught now will still be caught.

I'd suggest doing 1) above immediately, since that's a clear bug, but the others need to be discussed.

             

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1706815&group_id=5470

From noreply at sourceforge.net  Tue Apr 24 20:12:57 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Tue, 24 Apr 2007 11:12:57 -0700
Subject: [ python-Bugs-1706815 ] socket.error exceptions not subclass of
	StandardError
Message-ID: <E1HgPVZ-0008Jt-RR@sc8-sf-web23.sourceforge.net>

Bugs item #1706815, was opened at 2007-04-24 18:09
Message generated for change (Comment added) made by nagle
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1706815&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: John Nagle (nagle)
Assigned to: Nobody/Anonymous (nobody)
Summary: socket.error exceptions not subclass of StandardError

Initial Comment:
The "socket.error" exception is a subclass of Exception, but not of StandardError.  It needs to be placed properly in the exception hierarchy, presumably somewhere under IOError.

Socket errors have some known problems.  See also:

[ 805194 ] Inappropriate error received using socket timeout
[ 1019808 ] wrong socket error returned
[ 1571878 ] Improvements to socket module exceptions
[ 708927 ] socket timeouts produce wrong errors in win32 

Just figuring out what exceptions can be raised from the socket module is tough.  I've seen exceptions derived from "socket.error", exceptions from IOError, and exceptions from the SSL layer, which patches the
sockets module when loaded.  These are non-bug exceptions; that is, the problem is out in the network, external to the program.

Some are retryable, some indicate that a different approach (different port, different protocol) should be tried, and some mean that some named resource doesn't exist.  Programs need to make those distinctions reliably.

The most important distinction with sockets is "external network problem" vs. "local program program".  To resolve this, I suggest a "NetworkException" in the exception hierarchy, with all the things that can go wrong due to conditions external to the local machine under that exception.

I'd suggest the following:

1.  Add "NetworkError" under "IOError" in the exception hierarchy.

2.  Put the existing "socket.error" under "NetworkError". Since "socket.error" needs to be reparented anyway (it's currently a direct descendant of "Exception") this provides a good place for it.

3.  Find any places where the socket module can raise IOError or OSError due to an external network condition, and make them raise something under NetworkError instead.  Code that catches IOError will still work.

4.  Put all errors in the various SSL modules (SSLError, etc.) which can be raised due to external network conditions under "NetworkError"

5.  Move "urllib2.URLError", which is currently under IOError, down a level under NetworkError.

6.  Move the misc. errors from "urllib", like "ContentTooShortError", which are currently under IOError, down a level under NetworkError.

7.  URL translation errors from the IDNA (Unicode URL encoding) module probably should raise an error similar to that for an incorrect URL, rather than raising a UnicodeError.  

Then, programs that catch NetworkError could be sure of catching all network trouble conditions, but not local code bugs. 

With these changes, any exception that's being caught now will still be caught.

I'd suggest doing 1) above immediately, since that's a clear bug, but the others need to be discussed.

             

----------------------------------------------------------------------

>Comment By: John Nagle (nagle)
Date: 2007-04-24 18:12

Message:
Logged In: YES 
user_id=5571
Originator: YES

(See also PEP 352).

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1706815&group_id=5470

From noreply at sourceforge.net  Tue Apr 24 20:53:18 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Tue, 24 Apr 2007 11:53:18 -0700
Subject: [ python-Bugs-1706850 ] tarfile fails in 2.5.1 (NoneType has no ...)
Message-ID: <E1HgQ8c-0000Cn-7f@sc8-sf-web24.sourceforge.net>

Bugs item #1706850, was opened at 2007-04-24 19:53
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1706850&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Thomas Leonard (tal197)
Assigned to: Nobody/Anonymous (nobody)
Summary: tarfile fails in 2.5.1 (NoneType has no ...)

Initial Comment:
In Python 2.5.1's tarfile.py, we have:

def __init__(self, name=None, mode="r", fileobj=None):
        self.name = os.path.abspath(name)

If name is None (e.g. extracting from a stream) we get e.g.:

Traceback (most recent call last):
  File "tarfile.py", line 1168, in open
    _Stream(name, filemode, comptype, fileobj, bufsize))
  File "tarfile.py", line 1047, in __init__
    self.name = os.path.abspath(name)
  File "posixpath.py", line 402, in abspath
    if not isabs(path):
  File "posixpath.py", line 49, in isabs
    return s.startswith('/')
AttributeError: 'NoneType' object has no attribute 'startswith'

It works with Python 2.3.6, 2.4.4 and 2.5.0, but not 2.5.1.

(discovered by Zero Install unit-tests)

Thanks,

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1706850&group_id=5470

From noreply at sourceforge.net  Tue Apr 24 21:18:41 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Tue, 24 Apr 2007 12:18:41 -0700
Subject: [ python-Bugs-1706863 ] Failed to build Python 2.5.1 with sqlite3
Message-ID: <E1HgQXB-0003Du-Af@sc8-sf-web22.sourceforge.net>

Bugs item #1706863, was opened at 2007-04-24 22:18
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1706863&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Build
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Vitaliy Yermolenko (vitalyy2000)
Assigned to: Nobody/Anonymous (nobody)
Summary: Failed to build Python 2.5.1 with sqlite3

Initial Comment:
Failed to build new Python 2.5.1 with default config options, except --prefix, from Python-2.5.1.tar.bz2 on x86_64 (GNU/Linux 2.6.18-1.2798.fc6) with preliminary installed sqlite 3.3.6 - see below:

-- BEGIN --
[skipped]
running build
running build_ext
db.h: found (4, 3) in /usr/include
db lib: using (4, 3) db-4.3
sqlite: found /usr/include/sqlite3.h
/usr/include/sqlite3.h: version 3.3.6
Traceback (most recent call last):
  File "./setup.py", line 1525, in <module>
    main()
  File "./setup.py", line 1520, in main
    'Lib/smtpd.py']
  File "/tmp/070421/Python-2.5.1/Lib/distutils/core.py", line 151, in setup
    dist.run_commands()
  File "/tmp/070421/Python-2.5.1/Lib/distutils/dist.py", line 974, in run_commands
    self.run_command(cmd)
  File "/tmp/070421/Python-2.5.1/Lib/distutils/dist.py", line 994, in run_command
    cmd_obj.run()
  File "/tmp/070421/Python-2.5.1/Lib/distutils/command/build.py", line 112, in run
    self.run_command(cmd_name)
  File "/tmp/070421/Python-2.5.1/Lib/distutils/cmd.py", line 333, in run_command
    self.distribution.run_command(command)
  File "/tmp/070421/Python-2.5.1/Lib/distutils/dist.py", line 994, in run_command
    cmd_obj.run()
  File "/tmp/070421/Python-2.5.1/Lib/distutils/command/build_ext.py", line 290, in run
    self.build_extensions()
  File "./setup.py", line 97, in build_extensions
    self.detect_modules()
  File "./setup.py", line 795, in detect_modules
    sqlite_libdir = [os.path.abspath(os.path.dirname(sqlite_libfile))]
  File "/tmp/070421/Python-2.5.1/Lib/posixpath.py", line 119, in dirname
    return split(p)[0]
  File "/tmp/070421/Python-2.5.1/Lib/posixpath.py", line 77, in split
    i = p.rfind('/') + 1
AttributeError: 'NoneType' object has no attribute 'rfind'
make: *** [sharedmods] Error 1
--- END ---

Workaround: once /usr/include/sqlite3.h was "avoided" (actually just temporarily renamed), the installation was completed successfully.

-+-->
WBR,
Vitaliy Yermolenko.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1706863&group_id=5470

From noreply at sourceforge.net  Tue Apr 24 22:32:00 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Tue, 24 Apr 2007 13:32:00 -0700
Subject: [ python-Bugs-1706850 ] tarfile fails in 2.5.1 (NoneType has no ...)
Message-ID: <E1HgRg8-0002pB-1Z@sc8-sf-web23.sourceforge.net>

Bugs item #1706850, was opened at 2007-04-24 18:53
Message generated for change (Settings changed) made by gbrandl
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1706850&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.5
Status: Open
Resolution: None
>Priority: 7
Private: No
Submitted By: Thomas Leonard (tal197)
>Assigned to: Lars Gust?bel (gustaebel)
Summary: tarfile fails in 2.5.1 (NoneType has no ...)

Initial Comment:
In Python 2.5.1's tarfile.py, we have:

def __init__(self, name=None, mode="r", fileobj=None):
        self.name = os.path.abspath(name)

If name is None (e.g. extracting from a stream) we get e.g.:

Traceback (most recent call last):
  File "tarfile.py", line 1168, in open
    _Stream(name, filemode, comptype, fileobj, bufsize))
  File "tarfile.py", line 1047, in __init__
    self.name = os.path.abspath(name)
  File "posixpath.py", line 402, in abspath
    if not isabs(path):
  File "posixpath.py", line 49, in isabs
    return s.startswith('/')
AttributeError: 'NoneType' object has no attribute 'startswith'

It works with Python 2.3.6, 2.4.4 and 2.5.0, but not 2.5.1.

(discovered by Zero Install unit-tests)

Thanks,

----------------------------------------------------------------------

>Comment By: Georg Brandl (gbrandl)
Date: 2007-04-24 20:31

Message:
Logged In: YES 
user_id=849994
Originator: NO

Lars?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1706850&group_id=5470

From noreply at sourceforge.net  Tue Apr 24 22:44:55 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Tue, 24 Apr 2007 13:44:55 -0700
Subject: [ python-Bugs-1706850 ] tarfile fails in 2.5.1 (NoneType has no ...)
Message-ID: <E1HgRsd-0007is-Ou@sc8-sf-web21.sourceforge.net>

Bugs item #1706850, was opened at 2007-04-24 20:53
Message generated for change (Comment added) made by gustaebel
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1706850&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.5
Status: Open
Resolution: None
Priority: 7
Private: No
Submitted By: Thomas Leonard (tal197)
Assigned to: Lars Gust?bel (gustaebel)
Summary: tarfile fails in 2.5.1 (NoneType has no ...)

Initial Comment:
In Python 2.5.1's tarfile.py, we have:

def __init__(self, name=None, mode="r", fileobj=None):
        self.name = os.path.abspath(name)

If name is None (e.g. extracting from a stream) we get e.g.:

Traceback (most recent call last):
  File "tarfile.py", line 1168, in open
    _Stream(name, filemode, comptype, fileobj, bufsize))
  File "tarfile.py", line 1047, in __init__
    self.name = os.path.abspath(name)
  File "posixpath.py", line 402, in abspath
    if not isabs(path):
  File "posixpath.py", line 49, in isabs
    return s.startswith('/')
AttributeError: 'NoneType' object has no attribute 'startswith'

It works with Python 2.3.6, 2.4.4 and 2.5.0, but not 2.5.1.

(discovered by Zero Install unit-tests)

Thanks,

----------------------------------------------------------------------

>Comment By: Lars Gust?bel (gustaebel)
Date: 2007-04-24 22:44

Message:
Logged In: YES 
user_id=642936
Originator: NO

Apparently this is a duplicate of #1695229. It was fixed in rev. 54889 in
release25-maint, about a week too late for 2.5.1 :-(
I attached a patch with the changes.
File Added: r54889.diff

----------------------------------------------------------------------

Comment By: Georg Brandl (gbrandl)
Date: 2007-04-24 22:31

Message:
Logged In: YES 
user_id=849994
Originator: NO

Lars?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1706850&group_id=5470

From noreply at sourceforge.net  Tue Apr 24 22:56:09 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Tue, 24 Apr 2007 13:56:09 -0700
Subject: [ python-Bugs-1706850 ] tarfile fails in 2.5.1 (NoneType has no ...)
Message-ID: <E1HgS3V-0006wU-AT@sc8-sf-web24.sourceforge.net>

Bugs item #1706850, was opened at 2007-04-24 18:53
Message generated for change (Comment added) made by gbrandl
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1706850&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.5
>Status: Closed
>Resolution: Out of Date
Priority: 7
Private: No
Submitted By: Thomas Leonard (tal197)
Assigned to: Lars Gust?bel (gustaebel)
Summary: tarfile fails in 2.5.1 (NoneType has no ...)

Initial Comment:
In Python 2.5.1's tarfile.py, we have:

def __init__(self, name=None, mode="r", fileobj=None):
        self.name = os.path.abspath(name)

If name is None (e.g. extracting from a stream) we get e.g.:

Traceback (most recent call last):
  File "tarfile.py", line 1168, in open
    _Stream(name, filemode, comptype, fileobj, bufsize))
  File "tarfile.py", line 1047, in __init__
    self.name = os.path.abspath(name)
  File "posixpath.py", line 402, in abspath
    if not isabs(path):
  File "posixpath.py", line 49, in isabs
    return s.startswith('/')
AttributeError: 'NoneType' object has no attribute 'startswith'

It works with Python 2.3.6, 2.4.4 and 2.5.0, but not 2.5.1.

(discovered by Zero Install unit-tests)

Thanks,

----------------------------------------------------------------------

>Comment By: Georg Brandl (gbrandl)
Date: 2007-04-24 20:56

Message:
Logged In: YES 
user_id=849994
Originator: NO

Okay, closing then.

----------------------------------------------------------------------

Comment By: Lars Gust?bel (gustaebel)
Date: 2007-04-24 20:44

Message:
Logged In: YES 
user_id=642936
Originator: NO

Apparently this is a duplicate of #1695229. It was fixed in rev. 54889 in
release25-maint, about a week too late for 2.5.1 :-(
I attached a patch with the changes.
File Added: r54889.diff

----------------------------------------------------------------------

Comment By: Georg Brandl (gbrandl)
Date: 2007-04-24 20:31

Message:
Logged In: YES 
user_id=849994
Originator: NO

Lars?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1706850&group_id=5470

From noreply at sourceforge.net  Tue Apr 24 23:19:56 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Tue, 24 Apr 2007 14:19:56 -0700
Subject: [ python-Bugs-1568240 ] Tix is not included in 2.5 for Windows
Message-ID: <E1HgSQW-0001MY-RO@sc8-sf-web22.sourceforge.net>

Bugs item #1568240, was opened at 2006-09-30 11:19
Message generated for change (Comment added) made by loewis
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1568240&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Build
Group: Python 2.5
>Status: Closed
>Resolution: Fixed
Priority: 7
Private: No
Submitted By: Christos Georgiou (tzot)
Assigned to: Martin v. L?wis (loewis)
Summary: Tix is not included in 2.5 for Windows

Initial Comment:
(I hope "Build" is more precise than "Extension
Modules" and "Tkinter" for this specific bug.)

At least the following files are missing from 2.5 for
Windows:

DLLs\tix8184.dll
tcl\tix8184.lib
tcl\tix8.1\*

----------------------------------------------------------------------

>Comment By: Martin v. L?wis (loewis)
Date: 2007-04-24 23:19

Message:
Logged In: YES 
user_id=21627
Originator: NO

Closing as fixed in 2.5.1.

----------------------------------------------------------------------

Comment By: Christos Georgiou (tzot)
Date: 2007-04-24 18:32

Message:
Logged In: YES 
user_id=539787
Originator: YES

This bug can be closed. Tix installs and works fine in 2.5.1, thank you
Martin.

----------------------------------------------------------------------

Comment By: Christos Georgiou (tzot)
Date: 2007-01-22 13:13

Message:
Logged In: YES 
user_id=539787
Originator: YES

For me, yes, x86 is sufficient. Hopefully for others too.

----------------------------------------------------------------------

Comment By: Martin v. L?wis (loewis)
Date: 2007-01-20 14:16

Message:
Logged In: YES 
user_id=21627
Originator: NO

It seems that I can provide Tix binaries only for x86, not for AMD64 or
Itanium. Is that sufficient?

----------------------------------------------------------------------

Comment By: Martin v. L?wis (loewis)
Date: 2007-01-03 15:59

Message:
Logged In: YES 
user_id=21627
Originator: NO

Ah, ok. No, assigning this report to Neal or bumping its priority should
not be done.

----------------------------------------------------------------------

Comment By: Christos Georgiou (tzot)
Date: 2007-01-02 11:22

Message:
Logged In: YES 
user_id=539787
Originator: YES

Neal's message is this:
http://mail.python.org/pipermail/python-dev/2006-December/070406.html
and it refers to the 2.5.1 release, not prior to it.

As you see, I refrained from both increasing the priority and assigning it
to Neal, and actually just added a comment to the case with a related
question, since I know you are the one responsible for the windows build
and you already had assigned the bug to you. My adding this comment to the
bug was nothing more or less than the action that felt appropriate, and
still does feel appropriate to me (ie I didn't overstep any limits).

The "we" was just all parties interested, and in this case, the ones I
know are at least you (responsible for the windows build) and I (a user of
Tix on windows).

Happy new year, Martin!

----------------------------------------------------------------------

Comment By: Martin v. L?wis (loewis)
Date: 2006-12-29 23:26

Message:
Logged In: YES 
user_id=21627
Originator: NO

I haven't read Neal's message yet, but I wonder what he could do about
it.

I plan to fix this with 2.5.1, there is absolutely no way to fix this
earlier.

I'm not sure who "we" is who would like to bump the bug, and what
precisely this bumping would do; tzot, please refrain from changing the
priority to higher than 7. These priorities are reserved to the release
manager.

----------------------------------------------------------------------

Comment By: Christos Georgiou (tzot)
Date: 2006-12-27 18:46

Message:
Logged In: YES 
user_id=539787
Originator: YES

Should we bump the bug up and/or assign it to Neal Norwitz as he requested
on Python-Dev?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1568240&group_id=5470

From noreply at sourceforge.net  Wed Apr 25 09:04:57 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Wed, 25 Apr 2007 00:04:57 -0700
Subject: [ python-Bugs-1704156 ] TarFile.addfile() throws a struct.error
Message-ID: <E1HgbYf-0003m0-17@sc8-sf-web22.sourceforge.net>

Bugs item #1704156, was opened at 2007-04-20 10:21
Message generated for change (Settings changed) made by gustaebel
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1704156&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.5
>Status: Closed
Resolution: None
Priority: 5
Private: No
Submitted By: K. C. Wong (dvusboy)
Assigned to: Lars Gust?bel (gustaebel)
Summary: TarFile.addfile() throws a struct.error

Initial Comment:
When adding a file to a TarFile instance using addfile(), if the file paths (name and arcname) are unicode strings, then a struct.error will the raised. Python versions prior to 2.5 do not show this behaviour.

Assuming the current directory has a file name 'mac.txt', here is an interactive session that shows the problem:

Python 2.5 (r25:51908, Apr 18 2007, 19:06:57)
[GCC 3.4.6 20060404 (Red Hat 3.4.6-3)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import tarfile
>>> t=tarfile.open('test.tar', 'w')
>>> i=t.gettarinfo(u'mac.txt', u'mac.txt')
>>> t.addfile(i, file(u'mac.txt', 'r'))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.5/tarfile.py", line 1422, in addfile
    self.fileobj.write(tarinfo.tobuf(self.posix))
  File "/usr/lib/python2.5/tarfile.py", line 871, in tobuf
    buf = struct.pack("%ds" % BLOCKSIZE, "".join(parts))
  File "/usr/lib/python2.5/struct.py", line 63, in pack
    return o.pack(*args)
struct.error: argument for 's' must be a string


----------------------------------------------------------------------

Comment By: K. C. Wong (dvusboy)
Date: 2007-04-22 06:13

Message:
Logged In: YES 
user_id=414175
Originator: YES

Much thanks for your effort.

----------------------------------------------------------------------

Comment By: Lars Gust?bel (gustaebel)
Date: 2007-04-21 14:25

Message:
Logged In: YES 
user_id=642936
Originator: NO

I checked in a simple fix to the release25-maint branch (rev. 54908). I
will come up with a more solid approach for 2.6.

----------------------------------------------------------------------

Comment By: K. C. Wong (dvusboy)
Date: 2007-04-21 06:54

Message:
Logged In: YES 
user_id=414175
Originator: YES

I see the work around, and I have already implemented similar workarounds
in my code. However, I have 2 problem with this response:

1. The behaviour changed. As the documentation did not explicitly say
tarfile does not support unicode file paths, and it worked prior to 2.5,
then I would say breaking that behaviour at the least calls for a
documentation update.
2. The error message stamps from struct failing to pack a unicode string.
First of all, I did not grasp the subtle message of it being a unicode
string as opposed to a non-unicode string. You see, I actually did not
expect unicode string in the first place, it was a by-product of TEXT_DATA
from a DOM tree. I can understand why struct.pack() throws (because no
explicit encoding scheme was specified) but it was so cryptic with regard
to tarfile itself, that I had to modify tarfile to track down the reason
for the exception.
In short, I would prefer the owner of tarfile to make an explicit support
or not-supported call on unicode file path, document said decision and make
more reasonable attempt in presenting releavant exceptions.
Thank you for looking into this.

----------------------------------------------------------------------

Comment By: Lars Gust?bel (gustaebel)
Date: 2007-04-20 21:25

Message:
Logged In: YES 
user_id=642936
Originator: NO

tarfile.py was never guaranteed to work correctly with unicode filenames.
The fact that this works with Python < 2.5 is purely accidental.

You can work around this (sticking to your example):

i = t.gettarinfo(u'mac.txt', 'mac.txt')

or:

i = t.gettarinfo('mac.txt')


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1704156&group_id=5470

From noreply at sourceforge.net  Wed Apr 25 09:06:23 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Wed, 25 Apr 2007 00:06:23 -0700
Subject: [ python-Bugs-1704156 ] TarFile.addfile() throws a struct.error
Message-ID: <E1Hgba3-0003yz-2I@sc8-sf-web22.sourceforge.net>

Bugs item #1704156, was opened at 2007-04-20 10:21
Message generated for change (Settings changed) made by gustaebel
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1704156&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.5
Status: Closed
>Resolution: Fixed
Priority: 5
Private: No
Submitted By: K. C. Wong (dvusboy)
Assigned to: Lars Gust?bel (gustaebel)
Summary: TarFile.addfile() throws a struct.error

Initial Comment:
When adding a file to a TarFile instance using addfile(), if the file paths (name and arcname) are unicode strings, then a struct.error will the raised. Python versions prior to 2.5 do not show this behaviour.

Assuming the current directory has a file name 'mac.txt', here is an interactive session that shows the problem:

Python 2.5 (r25:51908, Apr 18 2007, 19:06:57)
[GCC 3.4.6 20060404 (Red Hat 3.4.6-3)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import tarfile
>>> t=tarfile.open('test.tar', 'w')
>>> i=t.gettarinfo(u'mac.txt', u'mac.txt')
>>> t.addfile(i, file(u'mac.txt', 'r'))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.5/tarfile.py", line 1422, in addfile
    self.fileobj.write(tarinfo.tobuf(self.posix))
  File "/usr/lib/python2.5/tarfile.py", line 871, in tobuf
    buf = struct.pack("%ds" % BLOCKSIZE, "".join(parts))
  File "/usr/lib/python2.5/struct.py", line 63, in pack
    return o.pack(*args)
struct.error: argument for 's' must be a string


----------------------------------------------------------------------

Comment By: K. C. Wong (dvusboy)
Date: 2007-04-22 06:13

Message:
Logged In: YES 
user_id=414175
Originator: YES

Much thanks for your effort.

----------------------------------------------------------------------

Comment By: Lars Gust?bel (gustaebel)
Date: 2007-04-21 14:25

Message:
Logged In: YES 
user_id=642936
Originator: NO

I checked in a simple fix to the release25-maint branch (rev. 54908). I
will come up with a more solid approach for 2.6.

----------------------------------------------------------------------

Comment By: K. C. Wong (dvusboy)
Date: 2007-04-21 06:54

Message:
Logged In: YES 
user_id=414175
Originator: YES

I see the work around, and I have already implemented similar workarounds
in my code. However, I have 2 problem with this response:

1. The behaviour changed. As the documentation did not explicitly say
tarfile does not support unicode file paths, and it worked prior to 2.5,
then I would say breaking that behaviour at the least calls for a
documentation update.
2. The error message stamps from struct failing to pack a unicode string.
First of all, I did not grasp the subtle message of it being a unicode
string as opposed to a non-unicode string. You see, I actually did not
expect unicode string in the first place, it was a by-product of TEXT_DATA
from a DOM tree. I can understand why struct.pack() throws (because no
explicit encoding scheme was specified) but it was so cryptic with regard
to tarfile itself, that I had to modify tarfile to track down the reason
for the exception.
In short, I would prefer the owner of tarfile to make an explicit support
or not-supported call on unicode file path, document said decision and make
more reasonable attempt in presenting releavant exceptions.
Thank you for looking into this.

----------------------------------------------------------------------

Comment By: Lars Gust?bel (gustaebel)
Date: 2007-04-20 21:25

Message:
Logged In: YES 
user_id=642936
Originator: NO

tarfile.py was never guaranteed to work correctly with unicode filenames.
The fact that this works with Python < 2.5 is purely accidental.

You can work around this (sticking to your example):

i = t.gettarinfo(u'mac.txt', 'mac.txt')

or:

i = t.gettarinfo('mac.txt')


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1704156&group_id=5470

From noreply at sourceforge.net  Wed Apr 25 11:45:16 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Wed, 25 Apr 2007 02:45:16 -0700
Subject: [ python-Bugs-1707255 ] lost global variables in main memory
	intensive execution
Message-ID: <E1Hge3o-0001dx-Sp@sc8-sf-web23.sourceforge.net>

Bugs item #1707255, was opened at 2007-04-25 11:45
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1707255&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Parser/Compiler
Group: Python 2.4
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Jordi Pujol Ahull? (jpahullo)
Assigned to: Nobody/Anonymous (nobody)
Summary: lost global variables in main memory intensive execution

Initial Comment:
Hello,

I was running a very main memory intensive program in my computer. I looked for similar bugs or troubles in the site and I didn't found any similar to this. I know that the use of global variables is not recommended by software engineering, but for special cases it is very fast to develop/test some ideas.

My problem statement is the following (very simplified and also attached):

########## BEGINNING OF CODE
#test for globals

counter_1 = 0

def modifierfunction():
    global counter_1

    #some code
    counter_1 += 1
    #some other code

def test():
    for i in range(2):
        global counter_1
        counter_1 = 0

        for j in range(10):
            modifierfunction()

        print "COUNTER_1:", counter_1

def test2():
    global counter_1
    for i in range(2):
        counter_1 = 0

        for j in range(10):
            modifierfunction()

        print "COUNTER_1:", counter_1

if __name__ == "__main__":
    test()
    test2()
########## END OF CODE

Globally speaking, it is a global variable, defined at the begining of the python file (counter_1), and it is modified in some functions within it (in this example, modifierfunction). At the end, it appear some tests that make what I need. 

If you try to run this code, it will always show the expected values in the standard out. But, let me to show you my problem.

In the beginning, I have the global statement as in test2. But I found that it only take a corrent value for the first iteration. The others it has always a zero value. I didn't understand anything.

Then, some collegue suggested me to change the place of the global statement (as in test()), and then it worked correctly, as it was expected.

I repeat. The above example works fine. But when this same structure, but with my big problem, very main memory intensive, test2() didn't work correctly.

Thank you for your attention.

Regards,

Jordi

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1707255&group_id=5470

From noreply at sourceforge.net  Wed Apr 25 17:20:40 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Wed, 25 Apr 2007 08:20:40 -0700
Subject: [ python-Bugs-1707497 ] generation errors in PDF-A4 tags for lib.pdf
Message-ID: <E1HgjIO-00037b-QU@sc8-sf-web23.sourceforge.net>

Bugs item #1707497, was opened at 2007-04-25 17:20
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1707497&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Documentation
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Antoine LECA (antoinel)
Assigned to: Nobody/Anonymous (nobody)
Summary: generation errors in PDF-A4 tags for lib.pdf

Initial Comment:
I just downloaded 2.5.1, get the "books" from <URL:http://www.python.org/ftp/python/doc/2.5.1/pdf-a4-2.5.1.tar.bz2> and start reading (using Acrobat 5.01 17/09/2002 Spanish _and_ Foxit Reader 1.3 build 0930 if it matters) and I noticed in lib.pdf some bookmarks (right-hand column, "Marcadores" in my localized interface) incorrectly generated, with traces from the LaTeX sources still apparent:

4.8.5 encodings.utf_8_sig ? UTF-8 codec with BOM signature
13.3 copy_reg ? Register pickle support functions
15.4 dummy_thread ? Drop-in replacement for the thread module
15.5 dummy_threading ? Drop-in replacement for the threading
module
18.4.3 wsgiref.simple_server ? a simple WSGI HTTP server
19.4.1 AU read Objects
19.4.2 AU write Objects
19.5.1 Wave read Objects
19.5.2 Wave write Objects
23.5 test.test_support ? Utility functions for tests
26.2 __builtin__ ? Built-in objects
26.3 __main__ ? Top-level script environment
26.8 __future__ ? Future statement definitions
30.8 py_compile ? Compile Python source files
36.3 _winreg ? Windows registry access

The actual text is correct (the above is a direct copy-and-paste), only the summary bookmarks are wrong; and unfortunately, I cannot copy-and-paste them... The last one shows something like

36.3 unhbox voidb at x penalty @M hskip z at skip unhbox voidb at x .06emvbox {hrule width.3em}discretionary {-}{}{}penalty @M hskip z at skip winreg -- Windows registry access

(at first sight I thought it was semi-crypted, anti-Windows or similar thing :-))

HTH
Antoine

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1707497&group_id=5470

From noreply at sourceforge.net  Wed Apr 25 20:04:34 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Wed, 25 Apr 2007 11:04:34 -0700
Subject: [ python-Bugs-1633941 ] for line in sys.stdin: doesn't notice EOF the
	first time 
Message-ID: <E1Hglr0-0004R7-Et@sc8-sf-web23.sourceforge.net>

Bugs item #1633941, was opened at 2007-01-12 05:34
Message generated for change (Comment added) made by draghuram
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1633941&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Matthias Klose (doko)
Assigned to: Nobody/Anonymous (nobody)
Summary: for line in sys.stdin: doesn't notice EOF the first time 

Initial Comment:
[forwarded from http://bugs.debian.org/315888]

for line in sys.stdin: doesn't notice EOF the first time when reading from tty.

The test program:

    import sys
    for line in sys.stdin:
            print line,
    print "eof"

A sample session:

    liw at esme$ python foo.py
    foo         <--- I pressed Enter and then Ctrl-D
    foo         <--- then this appeared, but not more
    eof         <--- this only came when I pressed Ctrl-D a second time
    liw at esme$

Seems to me that there is some buffering issue where Python needs to
read end-of-file twice to notice it on all levels. Once should be 
enough.



----------------------------------------------------------------------

Comment By: Raghuram Devarakonda (draghuram)
Date: 2007-04-25 14:04

Message:
Logged In: YES 
user_id=984087
Originator: NO


BTW, I opened bug 1643712 for doc change.

----------------------------------------------------------------------

Comment By: Raghuram Devarakonda (draghuram)
Date: 2007-01-24 12:20

Message:
Logged In: YES 
user_id=984087
Originator: NO


I tested two kinds of inputs with iter and noiter verisons. I posted
"noter" code and OP's code is the iter version.

1) For input without newline at all (line1<CTRL-D><CTRL-D><CTRL-D>)
behaves same with both versions.
2) The noiter version prints "eof" with "line1\n<CTRL-D>" while the iter
version requires an additional CTRL-D. This is because iter version uses
read ahead which is implemented using fread() . A simple C program using
fread() behaves exactly same way. 

I tested on Linux but am sure windows behaviour (as posted by 
gagenellina) will have same reasons. Since the issue is with platform's
stdio library, I don't think python should fix anything here. However, it
may be worthwhile to mention something about this in documentation. I will
open a bug for this purpose. 







----------------------------------------------------------------------

Comment By: Raghuram Devarakonda (draghuram)
Date: 2007-01-22 12:45

Message:
Logged In: YES 
user_id=984087
Originator: NO


Ok. This may sound stupid but I couldn't find a way to attach a file to
this bug report. So I am copying the code here:

************
import sys

line = sys.stdin.readline()
while (line):
    print  line,
    line = sys.stdin.readline()

print "eof"
*************


----------------------------------------------------------------------

Comment By: Raghuram Devarakonda (draghuram)
Date: 2007-01-22 12:37

Message:
Logged In: YES 
user_id=984087
Originator: NO


Sorry for my duplicate comment. It was a mistake. On closer examination,
the OP's description does seem to indicate some issue. Please look at
(attached) stdin_noiter.py which uses readline() directly and it does not
have the problem described here. It properly detects EOF on first CTRL-D.
This points to some problem with the iterator function
fileobject.c:file_iternext(). I think that the first CTRL-D might be
getting lost somewhere in the read ahead code (which only comes into
picture with iterator).

----------------------------------------------------------------------

Comment By: Raghuram Devarakonda (draghuram)
Date: 2007-01-22 11:34

Message:
Logged In: YES 
user_id=984087
Originator: NO


I am not entirely sure that this is a bug.

$ cat testfile
line1
line2

$ python foo.py < testfile

This command behaves as expected. Only when the input is from tty, the
above described behaviour happens. That could be because of the terminal
settings where characters may be buffered until a newline is entered.



----------------------------------------------------------------------

Comment By: Raghuram Devarakonda (draghuram)
Date: 2007-01-22 11:34

Message:
Logged In: YES 
user_id=984087
Originator: NO


I am not entirely sure that this is a bug.

$ cat testfile
line1
line2

$ python foo.py < testfile

This command behaves as expected. Only when the input is from tty, the
above described behaviour happens. That could be because of the terminal
settings where characters may be buffered until a newline is entered.



----------------------------------------------------------------------

Comment By: Gabriel Genellina (gagenellina)
Date: 2007-01-13 23:20

Message:
Logged In: YES 
user_id=479790
Originator: NO

Same thing occurs on Windows. Even worse, if the line does not end with
CR, Ctrl-Z (EOF in Windows, equivalent to Ctrl-D) has to be pressed 3
times:

D:\Temp>python foo.py
foo  <--- I pressed Enter
^Z   <--- I pressed Ctrl-Z and then Enter again
foo  <--- this appeared
^Z   <--- I pressed Ctrl-Z and then Enter again
<EOF>

D:\Temp>python foo.py
foo^Z   <--- I pressed Ctrl-Z and then Enter
^Z      <--- cursor stays here; I pressed Ctrl-Z and then Enter again
^Z      <--- cursor stays here; I pressed Ctrl-Z and then Enter again
foo <EOF>


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1633941&group_id=5470

From noreply at sourceforge.net  Wed Apr 25 20:04:56 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Wed, 25 Apr 2007 11:04:56 -0700
Subject: [ python-Bugs-1707607 ] raw text can't end with backslash
Message-ID: <E1HglrM-0006dW-Ct@sc8-sf-web21.sourceforge.net>

Bugs item #1707607, was opened at 2007-04-25 18:04
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1707607&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Interpreter Core
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: lomm (korka)
Assigned to: Nobody/Anonymous (nobody)
Summary: raw text can't end with backslash

Initial Comment:
somehow the interpreter thinks it's an escape for the last quote

print r'abc\'

  File "<stdin>", line 1
    print 'abc\'
               ^
SyntaxError: EOL while scanning single-quoted string



----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1707607&group_id=5470

From noreply at sourceforge.net  Wed Apr 25 21:36:12 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Wed, 25 Apr 2007 12:36:12 -0700
Subject: [ python-Bugs-1707607 ] raw text can't end with backslash
Message-ID: <E1HgnHg-00032R-QY@sc8-sf-web22.sourceforge.net>

Bugs item #1707607, was opened at 2007-04-25 18:04
Message generated for change (Comment added) made by gbrandl
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1707607&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Interpreter Core
Group: Python 2.5
>Status: Closed
>Resolution: Wont Fix
Priority: 5
Private: No
Submitted By: lomm (korka)
Assigned to: Nobody/Anonymous (nobody)
Summary: raw text can't end with backslash

Initial Comment:
somehow the interpreter thinks it's an escape for the last quote

print r'abc\'

  File "<stdin>", line 1
    print 'abc\'
               ^
SyntaxError: EOL while scanning single-quoted string



----------------------------------------------------------------------

>Comment By: Georg Brandl (gbrandl)
Date: 2007-04-25 19:36

Message:
Logged In: YES 
user_id=849994
Originator: NO

This is a tokenizer restriction and not considered a bug. Closing as
"won't fix".

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1707607&group_id=5470

From noreply at sourceforge.net  Wed Apr 25 22:46:41 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Wed, 25 Apr 2007 13:46:41 -0700
Subject: [ python-Feature Requests-1707693 ] Please add .bz2 in encodings_map
	(in the module mimetypes)
Message-ID: <E1HgoNt-0004Bm-CX@sc8-sf-web24.sourceforge.net>

Feature Requests item #1707693, was opened at 2007-04-25 22:46
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1707693&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: elghinn (elghinn)
Assigned to: Nobody/Anonymous (nobody)
Summary: Please add .bz2 in encodings_map (in the module mimetypes)

Initial Comment:
In the module mimetypes, can you add .bz2 in 
    encodings_map = {
        '.gz': 'gzip',
        '.Z': 'compress',
        }
please ?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1707693&group_id=5470

From noreply at sourceforge.net  Thu Apr 26 01:44:05 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Wed, 25 Apr 2007 16:44:05 -0700
Subject: [ python-Bugs-1707768 ] os.path.normpath changes path (chops of
	trailing slash)
Message-ID: <E1Hgr9Z-0001br-Pc@sc8-sf-web24.sourceforge.net>

Bugs item #1707768, was opened at 2007-04-26 01:44
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1707768&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Robert Siemer (siemer)
Assigned to: Nobody/Anonymous (nobody)
Summary: os.path.normpath changes path (chops of trailing slash)

Initial Comment:
Hello everybody!

>>> os.path.normpath('/etc/passwd')
'/etc/passwd'


I don't know any environment at all where

a) '/etc/passwd/'
b) '/etc/passwd'

are treated the same. It clearly does not apply for the path part of http urls (this is left as an exercise for the reader).

But it also does not apply for (e.g.) Linux either:
an open() on path a) return ENOTDIR while it succeeds with b).

(assuming /etc/passwd is a file)

This is definitively not a documentation bug, as "normpath" should normalize a path and not fuck it up.


Robert

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1707768&group_id=5470

From noreply at sourceforge.net  Thu Apr 26 01:47:27 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Wed, 25 Apr 2007 16:47:27 -0700
Subject: [ python-Bugs-1707768 ] os.path.normpath changes path (chops of
	trailing slash)
Message-ID: <E1HgrCp-0006Ts-SK@sc8-sf-web22.sourceforge.net>

Bugs item #1707768, was opened at 2007-04-26 01:44
Message generated for change (Comment added) made by siemer
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1707768&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Robert Siemer (siemer)
Assigned to: Nobody/Anonymous (nobody)
Summary: os.path.normpath changes path (chops of trailing slash)

Initial Comment:
Hello everybody!

>>> os.path.normpath('/etc/passwd')
'/etc/passwd'


I don't know any environment at all where

a) '/etc/passwd/'
b) '/etc/passwd'

are treated the same. It clearly does not apply for the path part of http urls (this is left as an exercise for the reader).

But it also does not apply for (e.g.) Linux either:
an open() on path a) return ENOTDIR while it succeeds with b).

(assuming /etc/passwd is a file)

This is definitively not a documentation bug, as "normpath" should normalize a path and not fuck it up.


Robert

----------------------------------------------------------------------

>Comment By: Robert Siemer (siemer)
Date: 2007-04-26 01:47

Message:
Logged In: YES 
user_id=150699
Originator: YES

A bugreport bug:

The example should read os.path.normpath('/etc/passwd/')...

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1707768&group_id=5470

From noreply at sourceforge.net  Thu Apr 26 03:40:37 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Wed, 25 Apr 2007 18:40:37 -0700
Subject: [ python-Bugs-1672853 ] Error reading files larger than 4GB
Message-ID: <E1HgsyL-0005bw-Vo@sc8-sf-web24.sourceforge.net>

Bugs item #1672853, was opened at 2007-03-03 03:01
Message generated for change (Comment added) made by joearmbruster
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1672853&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Windows
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Case Van Horsen (casevh)
Assigned to: Nobody/Anonymous (nobody)
Summary: Error reading files larger than 4GB

Initial Comment:
When reading test files larger than 4GB, sometimes two lines are merged into a single line. The problem is reproducible on Windows 2K SP4 with both Python 2.4 and 2.5. It does not appear to occur on Linux.

I have attached a test case that creates the problem.



----------------------------------------------------------------------

Comment By: Joseph Armbruster (joearmbruster)
Date: 2007-04-25 21:40

Message:
Logged In: YES 
user_id=1643128
Originator: NO

URL:  http://svn.python.org/projects/python/trunk
Revision: 54922

Note:  Reproduced Error using test application on the following system

OS Name:    Microsoft Windows XP Professional
OS Version: 5.1.2600 Service Pack 2 Build 2600

Try modifying line 13:

fh = open(fname, 'r')
to:
fh = open(fname, 'rb')

and notice the behavior.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1672853&group_id=5470

From noreply at sourceforge.net  Thu Apr 26 04:11:36 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Wed, 25 Apr 2007 19:11:36 -0700
Subject: [ python-Bugs-1672853 ] Error reading files larger than 4GB
Message-ID: <E1HgtSK-0005Gc-Ge@sc8-sf-web22.sourceforge.net>

Bugs item #1672853, was opened at 2007-03-03 03:01
Message generated for change (Comment added) made by joearmbruster
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1672853&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Windows
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Case Van Horsen (casevh)
Assigned to: Nobody/Anonymous (nobody)
Summary: Error reading files larger than 4GB

Initial Comment:
When reading test files larger than 4GB, sometimes two lines are merged into a single line. The problem is reproducible on Windows 2K SP4 with both Python 2.4 and 2.5. It does not appear to occur on Linux.

I have attached a test case that creates the problem.



----------------------------------------------------------------------

Comment By: Joseph Armbruster (joearmbruster)
Date: 2007-04-25 22:11

Message:
Logged In: YES 
user_id=1643128
Originator: NO

josepharmbruster on #python or #python-dev in freenode

----------------------------------------------------------------------

Comment By: Joseph Armbruster (joearmbruster)
Date: 2007-04-25 21:40

Message:
Logged In: YES 
user_id=1643128
Originator: NO

URL:  http://svn.python.org/projects/python/trunk
Revision: 54922

Note:  Reproduced Error using test application on the following system

OS Name:    Microsoft Windows XP Professional
OS Version: 5.1.2600 Service Pack 2 Build 2600

Try modifying line 13:

fh = open(fname, 'r')
to:
fh = open(fname, 'rb')

and notice the behavior.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1672853&group_id=5470

From noreply at sourceforge.net  Thu Apr 26 04:31:01 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Wed, 25 Apr 2007 19:31:01 -0700
Subject: [ python-Feature Requests-1706256 ] Give Partial the ability to skip
	positionals
Message-ID: <E1Hgtl7-0008TF-As@sc8-sf-web23.sourceforge.net>

Feature Requests item #1706256, was opened at 2007-04-23 20:18
Message generated for change (Comment added) made by belopolsky
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1706256&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Calvin Spealman (ironfroggy)
Assigned to: Nobody/Anonymous (nobody)
Summary: Give Partial the ability to skip positionals

Initial Comment:
There are some situations where you want to skip positional arguments in a use of a partial function. In other words, you want to create a partial that applies positional arguments out of order or without applying values to one or more lower positional arguments. In some cases keyword arguments can be used instead, but this has two obvious drawbacks. Firstly, it causes the caller to rely on the name of a positional in a callee, which breaks encapsulation. Secondly, on the case of the function being applied to being a builtin, it fails completely, as they will not take positional arguments by name at all. I propose a class attribute to the partial type, 'skip', which will be a singleton to pass to a partial object signifying this skipping of positionals. The following example demonstrates.

from functools import partial

def add(a, b):
    return a + b

append_abc = partial(add, partial.skip, "abc")
assert append_abc("xyz") == "xyzabc"

Obviously this example would break if used as partial(add, b="abc") and the maintainer of add changed the positional names to 'first' and 'second' or 'pre' and 'post', which is perfectly reasonable. We do not need to expect the names of our positional arguments are depended upon. It would also break when someone gets smart and replaces the add function with operator.add, of course.

----------------------------------------------------------------------

Comment By: Alexander Belopolsky (belopolsky)
Date: 2007-04-25 22:31

Message:
Logged In: YES 
user_id=835142
Originator: NO

Names I've seen used for this purpose elsewhere: slot, arg, missing.

----------------------------------------------------------------------

Comment By: Martin v. L?wis (loewis)
Date: 2007-04-24 01:41

Message:
Logged In: YES 
user_id=21627
Originator: NO

I would not call it partial.skip, but partial.unbound (or find yet a
better name that indicates that the argument is not skipped, but instead
will be an argument of the resulting partial function).

----------------------------------------------------------------------

Comment By: Calvin Spealman (ironfroggy)
Date: 2007-04-23 23:45

Message:
Logged In: YES 
user_id=112166
Originator: YES

File Added: partial_skip.patch

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1706256&group_id=5470

From noreply at sourceforge.net  Thu Apr 26 04:58:14 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Wed, 25 Apr 2007 19:58:14 -0700
Subject: [ python-Feature Requests-1706256 ] Give Partial the ability to skip
	positionals
Message-ID: <E1HguBS-0005Yv-6q@sc8-sf-web24.sourceforge.net>

Feature Requests item #1706256, was opened at 2007-04-23 20:18
Message generated for change (Comment added) made by ironfroggy
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1706256&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Calvin Spealman (ironfroggy)
Assigned to: Nobody/Anonymous (nobody)
Summary: Give Partial the ability to skip positionals

Initial Comment:
There are some situations where you want to skip positional arguments in a use of a partial function. In other words, you want to create a partial that applies positional arguments out of order or without applying values to one or more lower positional arguments. In some cases keyword arguments can be used instead, but this has two obvious drawbacks. Firstly, it causes the caller to rely on the name of a positional in a callee, which breaks encapsulation. Secondly, on the case of the function being applied to being a builtin, it fails completely, as they will not take positional arguments by name at all. I propose a class attribute to the partial type, 'skip', which will be a singleton to pass to a partial object signifying this skipping of positionals. The following example demonstrates.

from functools import partial

def add(a, b):
    return a + b

append_abc = partial(add, partial.skip, "abc")
assert append_abc("xyz") == "xyzabc"

Obviously this example would break if used as partial(add, b="abc") and the maintainer of add changed the positional names to 'first' and 'second' or 'pre' and 'post', which is perfectly reasonable. We do not need to expect the names of our positional arguments are depended upon. It would also break when someone gets smart and replaces the add function with operator.add, of course.

----------------------------------------------------------------------

>Comment By: Calvin Spealman (ironfroggy)
Date: 2007-04-25 22:58

Message:
Logged In: YES 
user_id=112166
Originator: YES

If anyone has a serious argument against the partial.skip name, I would
take partial.unbound as my second choice, but I definitely prefer
partial.skip to it. Third would be partial.latebound. I still can't figure
out how my code broke subclassing of partial, so if anyone can take a look,
I'd appreciate it hugely.

----------------------------------------------------------------------

Comment By: Alexander Belopolsky (belopolsky)
Date: 2007-04-25 22:31

Message:
Logged In: YES 
user_id=835142
Originator: NO

Names I've seen used for this purpose elsewhere: slot, arg, missing.

----------------------------------------------------------------------

Comment By: Martin v. L?wis (loewis)
Date: 2007-04-24 01:41

Message:
Logged In: YES 
user_id=21627
Originator: NO

I would not call it partial.skip, but partial.unbound (or find yet a
better name that indicates that the argument is not skipped, but instead
will be an argument of the resulting partial function).

----------------------------------------------------------------------

Comment By: Calvin Spealman (ironfroggy)
Date: 2007-04-23 23:45

Message:
Logged In: YES 
user_id=112166
Originator: YES

File Added: partial_skip.patch

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1706256&group_id=5470

From noreply at sourceforge.net  Thu Apr 26 05:21:21 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Wed, 25 Apr 2007 20:21:21 -0700
Subject: [ python-Feature Requests-1706256 ] Give Partial the ability to skip
	positionals
Message-ID: <E1HguXp-0004ur-CK@sc8-sf-web21.sourceforge.net>

Feature Requests item #1706256, was opened at 2007-04-23 20:18
Message generated for change (Comment added) made by belopolsky
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1706256&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Calvin Spealman (ironfroggy)
Assigned to: Nobody/Anonymous (nobody)
Summary: Give Partial the ability to skip positionals

Initial Comment:
There are some situations where you want to skip positional arguments in a use of a partial function. In other words, you want to create a partial that applies positional arguments out of order or without applying values to one or more lower positional arguments. In some cases keyword arguments can be used instead, but this has two obvious drawbacks. Firstly, it causes the caller to rely on the name of a positional in a callee, which breaks encapsulation. Secondly, on the case of the function being applied to being a builtin, it fails completely, as they will not take positional arguments by name at all. I propose a class attribute to the partial type, 'skip', which will be a singleton to pass to a partial object signifying this skipping of positionals. The following example demonstrates.

from functools import partial

def add(a, b):
    return a + b

append_abc = partial(add, partial.skip, "abc")
assert append_abc("xyz") == "xyzabc"

Obviously this example would break if used as partial(add, b="abc") and the maintainer of add changed the positional names to 'first' and 'second' or 'pre' and 'post', which is perfectly reasonable. We do not need to expect the names of our positional arguments are depended upon. It would also break when someone gets smart and replaces the add function with operator.add, of course.

----------------------------------------------------------------------

Comment By: Alexander Belopolsky (belopolsky)
Date: 2007-04-25 23:21

Message:
Logged In: YES 
user_id=835142
Originator: NO

The current patch does not compile:

Modules/_functoolsmodule.c: In function 'init_functools':
Modules/_functoolsmodule.c:306: error: too many arguments to function
'PyObject_CallObject'

Once I removed the extra NULL argument, it seems to work fine.  What
exactly is broken?  Can you add unit tests for the new functionality?

----------------------------------------------------------------------

Comment By: Calvin Spealman (ironfroggy)
Date: 2007-04-25 22:58

Message:
Logged In: YES 
user_id=112166
Originator: YES

If anyone has a serious argument against the partial.skip name, I would
take partial.unbound as my second choice, but I definitely prefer
partial.skip to it. Third would be partial.latebound. I still can't figure
out how my code broke subclassing of partial, so if anyone can take a look,
I'd appreciate it hugely.

----------------------------------------------------------------------

Comment By: Alexander Belopolsky (belopolsky)
Date: 2007-04-25 22:31

Message:
Logged In: YES 
user_id=835142
Originator: NO

Names I've seen used for this purpose elsewhere: slot, arg, missing.

----------------------------------------------------------------------

Comment By: Martin v. L?wis (loewis)
Date: 2007-04-24 01:41

Message:
Logged In: YES 
user_id=21627
Originator: NO

I would not call it partial.skip, but partial.unbound (or find yet a
better name that indicates that the argument is not skipped, but instead
will be an argument of the resulting partial function).

----------------------------------------------------------------------

Comment By: Calvin Spealman (ironfroggy)
Date: 2007-04-23 23:45

Message:
Logged In: YES 
user_id=112166
Originator: YES

File Added: partial_skip.patch

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1706256&group_id=5470

From noreply at sourceforge.net  Thu Apr 26 05:34:11 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Wed, 25 Apr 2007 20:34:11 -0700
Subject: [ python-Feature Requests-1706256 ] Give Partial the ability to skip
	positionals
Message-ID: <E1HgukF-0000xx-AZ@sc8-sf-web24.sourceforge.net>

Feature Requests item #1706256, was opened at 2007-04-23 20:18
Message generated for change (Comment added) made by belopolsky
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1706256&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Calvin Spealman (ironfroggy)
Assigned to: Nobody/Anonymous (nobody)
Summary: Give Partial the ability to skip positionals

Initial Comment:
There are some situations where you want to skip positional arguments in a use of a partial function. In other words, you want to create a partial that applies positional arguments out of order or without applying values to one or more lower positional arguments. In some cases keyword arguments can be used instead, but this has two obvious drawbacks. Firstly, it causes the caller to rely on the name of a positional in a callee, which breaks encapsulation. Secondly, on the case of the function being applied to being a builtin, it fails completely, as they will not take positional arguments by name at all. I propose a class attribute to the partial type, 'skip', which will be a singleton to pass to a partial object signifying this skipping of positionals. The following example demonstrates.

from functools import partial

def add(a, b):
    return a + b

append_abc = partial(add, partial.skip, "abc")
assert append_abc("xyz") == "xyzabc"

Obviously this example would break if used as partial(add, b="abc") and the maintainer of add changed the positional names to 'first' and 'second' or 'pre' and 'post', which is perfectly reasonable. We do not need to expect the names of our positional arguments are depended upon. It would also break when someone gets smart and replaces the add function with operator.add, of course.

----------------------------------------------------------------------

Comment By: Alexander Belopolsky (belopolsky)
Date: 2007-04-25 23:34

Message:
Logged In: YES 
user_id=835142
Originator: NO

OK, I've got it.  Your patch breaks test_functools.  This is because you
have blown up partial_type.tp_dict . 

----------------------------------------------------------------------

Comment By: Alexander Belopolsky (belopolsky)
Date: 2007-04-25 23:21

Message:
Logged In: YES 
user_id=835142
Originator: NO

The current patch does not compile:

Modules/_functoolsmodule.c: In function 'init_functools':
Modules/_functoolsmodule.c:306: error: too many arguments to function
'PyObject_CallObject'

Once I removed the extra NULL argument, it seems to work fine.  What
exactly is broken?  Can you add unit tests for the new functionality?

----------------------------------------------------------------------

Comment By: Calvin Spealman (ironfroggy)
Date: 2007-04-25 22:58

Message:
Logged In: YES 
user_id=112166
Originator: YES

If anyone has a serious argument against the partial.skip name, I would
take partial.unbound as my second choice, but I definitely prefer
partial.skip to it. Third would be partial.latebound. I still can't figure
out how my code broke subclassing of partial, so if anyone can take a look,
I'd appreciate it hugely.

----------------------------------------------------------------------

Comment By: Alexander Belopolsky (belopolsky)
Date: 2007-04-25 22:31

Message:
Logged In: YES 
user_id=835142
Originator: NO

Names I've seen used for this purpose elsewhere: slot, arg, missing.

----------------------------------------------------------------------

Comment By: Martin v. L?wis (loewis)
Date: 2007-04-24 01:41

Message:
Logged In: YES 
user_id=21627
Originator: NO

I would not call it partial.skip, but partial.unbound (or find yet a
better name that indicates that the argument is not skipped, but instead
will be an argument of the resulting partial function).

----------------------------------------------------------------------

Comment By: Calvin Spealman (ironfroggy)
Date: 2007-04-23 23:45

Message:
Logged In: YES 
user_id=112166
Originator: YES

File Added: partial_skip.patch

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1706256&group_id=5470

From noreply at sourceforge.net  Thu Apr 26 06:10:45 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Wed, 25 Apr 2007 21:10:45 -0700
Subject: [ python-Feature Requests-1706256 ] Give Partial the ability to skip
	positionals
Message-ID: <E1HgvJd-0001sD-Eu@sc8-sf-web21.sourceforge.net>

Feature Requests item #1706256, was opened at 2007-04-23 20:18
Message generated for change (Comment added) made by ironfroggy
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1706256&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Calvin Spealman (ironfroggy)
Assigned to: Nobody/Anonymous (nobody)
Summary: Give Partial the ability to skip positionals

Initial Comment:
There are some situations where you want to skip positional arguments in a use of a partial function. In other words, you want to create a partial that applies positional arguments out of order or without applying values to one or more lower positional arguments. In some cases keyword arguments can be used instead, but this has two obvious drawbacks. Firstly, it causes the caller to rely on the name of a positional in a callee, which breaks encapsulation. Secondly, on the case of the function being applied to being a builtin, it fails completely, as they will not take positional arguments by name at all. I propose a class attribute to the partial type, 'skip', which will be a singleton to pass to a partial object signifying this skipping of positionals. The following example demonstrates.

from functools import partial

def add(a, b):
    return a + b

append_abc = partial(add, partial.skip, "abc")
assert append_abc("xyz") == "xyzabc"

Obviously this example would break if used as partial(add, b="abc") and the maintainer of add changed the positional names to 'first' and 'second' or 'pre' and 'post', which is perfectly reasonable. We do not need to expect the names of our positional arguments are depended upon. It would also break when someone gets smart and replaces the add function with operator.add, of course.

----------------------------------------------------------------------

>Comment By: Calvin Spealman (ironfroggy)
Date: 2007-04-26 00:10

Message:
Logged In: YES 
user_id=112166
Originator: YES

Hmm, I didn't get such an error here on VC with the extra argument. I'll
change that here, too.

I figured the breaking of the subclassing of partial was related to what I
do with tp_dict, but I don't understand how. How did I blow it up? And,
yes, I will write tests for the new functionality, of course.

----------------------------------------------------------------------

Comment By: Alexander Belopolsky (belopolsky)
Date: 2007-04-25 23:34

Message:
Logged In: YES 
user_id=835142
Originator: NO

OK, I've got it.  Your patch breaks test_functools.  This is because you
have blown up partial_type.tp_dict . 

----------------------------------------------------------------------

Comment By: Alexander Belopolsky (belopolsky)
Date: 2007-04-25 23:21

Message:
Logged In: YES 
user_id=835142
Originator: NO

The current patch does not compile:

Modules/_functoolsmodule.c: In function 'init_functools':
Modules/_functoolsmodule.c:306: error: too many arguments to function
'PyObject_CallObject'

Once I removed the extra NULL argument, it seems to work fine.  What
exactly is broken?  Can you add unit tests for the new functionality?

----------------------------------------------------------------------

Comment By: Calvin Spealman (ironfroggy)
Date: 2007-04-25 22:58

Message:
Logged In: YES 
user_id=112166
Originator: YES

If anyone has a serious argument against the partial.skip name, I would
take partial.unbound as my second choice, but I definitely prefer
partial.skip to it. Third would be partial.latebound. I still can't figure
out how my code broke subclassing of partial, so if anyone can take a look,
I'd appreciate it hugely.

----------------------------------------------------------------------

Comment By: Alexander Belopolsky (belopolsky)
Date: 2007-04-25 22:31

Message:
Logged In: YES 
user_id=835142
Originator: NO

Names I've seen used for this purpose elsewhere: slot, arg, missing.

----------------------------------------------------------------------

Comment By: Martin v. L?wis (loewis)
Date: 2007-04-24 01:41

Message:
Logged In: YES 
user_id=21627
Originator: NO

I would not call it partial.skip, but partial.unbound (or find yet a
better name that indicates that the argument is not skipped, but instead
will be an argument of the resulting partial function).

----------------------------------------------------------------------

Comment By: Calvin Spealman (ironfroggy)
Date: 2007-04-23 23:45

Message:
Logged In: YES 
user_id=112166
Originator: YES

File Added: partial_skip.patch

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1706256&group_id=5470

From noreply at sourceforge.net  Thu Apr 26 06:14:04 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Wed, 25 Apr 2007 21:14:04 -0700
Subject: [ python-Feature Requests-1706256 ] Give Partial the ability to skip
	positionals
Message-ID: <E1HgvMq-00083i-5S@sc8-sf-web23.sourceforge.net>

Feature Requests item #1706256, was opened at 2007-04-23 20:18
Message generated for change (Comment added) made by belopolsky
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1706256&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Calvin Spealman (ironfroggy)
Assigned to: Nobody/Anonymous (nobody)
Summary: Give Partial the ability to skip positionals

Initial Comment:
There are some situations where you want to skip positional arguments in a use of a partial function. In other words, you want to create a partial that applies positional arguments out of order or without applying values to one or more lower positional arguments. In some cases keyword arguments can be used instead, but this has two obvious drawbacks. Firstly, it causes the caller to rely on the name of a positional in a callee, which breaks encapsulation. Secondly, on the case of the function being applied to being a builtin, it fails completely, as they will not take positional arguments by name at all. I propose a class attribute to the partial type, 'skip', which will be a singleton to pass to a partial object signifying this skipping of positionals. The following example demonstrates.

from functools import partial

def add(a, b):
    return a + b

append_abc = partial(add, partial.skip, "abc")
assert append_abc("xyz") == "xyzabc"

Obviously this example would break if used as partial(add, b="abc") and the maintainer of add changed the positional names to 'first' and 'second' or 'pre' and 'post', which is perfectly reasonable. We do not need to expect the names of our positional arguments are depended upon. It would also break when someone gets smart and replaces the add function with operator.add, of course.

----------------------------------------------------------------------

Comment By: Alexander Belopolsky (belopolsky)
Date: 2007-04-26 00:14

Message:
Logged In: YES 
user_id=835142
Originator: NO

If you remove partial_type.tp_dict = PyDict_New(); at line 309, the patch
will pass test_functools.

A few comments:

Design:
1. partial.skip should be an instance of a singleton class (look at
NoneType implementation in object.c)
2. repr(partial.skip) should be 'functools.partial.skip' (easy to
implement once #1 is done)

Implementation:
1.  In the loop over pto->args you know that i < npargs, so you can use
PyTuple_GET_ITEM and there is no need to check for arg==NULL
2. You should check PyTuple_GetItem(args, pull_index) for null and return
with error if too few arguments is supplied.  Better yet, find number of
supplied args outside the loop and raise your own error if pool_index grows
to that number.
3. It looks like you are leaking references. I don't see where you decref
ptoargscopy and arg after concatenation.


----------------------------------------------------------------------

Comment By: Calvin Spealman (ironfroggy)
Date: 2007-04-26 00:10

Message:
Logged In: YES 
user_id=112166
Originator: YES

Hmm, I didn't get such an error here on VC with the extra argument. I'll
change that here, too.

I figured the breaking of the subclassing of partial was related to what I
do with tp_dict, but I don't understand how. How did I blow it up? And,
yes, I will write tests for the new functionality, of course.

----------------------------------------------------------------------

Comment By: Alexander Belopolsky (belopolsky)
Date: 2007-04-25 23:34

Message:
Logged In: YES 
user_id=835142
Originator: NO

OK, I've got it.  Your patch breaks test_functools.  This is because you
have blown up partial_type.tp_dict . 

----------------------------------------------------------------------

Comment By: Alexander Belopolsky (belopolsky)
Date: 2007-04-25 23:21

Message:
Logged In: YES 
user_id=835142
Originator: NO

The current patch does not compile:

Modules/_functoolsmodule.c: In function 'init_functools':
Modules/_functoolsmodule.c:306: error: too many arguments to function
'PyObject_CallObject'

Once I removed the extra NULL argument, it seems to work fine.  What
exactly is broken?  Can you add unit tests for the new functionality?

----------------------------------------------------------------------

Comment By: Calvin Spealman (ironfroggy)
Date: 2007-04-25 22:58

Message:
Logged In: YES 
user_id=112166
Originator: YES

If anyone has a serious argument against the partial.skip name, I would
take partial.unbound as my second choice, but I definitely prefer
partial.skip to it. Third would be partial.latebound. I still can't figure
out how my code broke subclassing of partial, so if anyone can take a look,
I'd appreciate it hugely.

----------------------------------------------------------------------

Comment By: Alexander Belopolsky (belopolsky)
Date: 2007-04-25 22:31

Message:
Logged In: YES 
user_id=835142
Originator: NO

Names I've seen used for this purpose elsewhere: slot, arg, missing.

----------------------------------------------------------------------

Comment By: Martin v. L?wis (loewis)
Date: 2007-04-24 01:41

Message:
Logged In: YES 
user_id=21627
Originator: NO

I would not call it partial.skip, but partial.unbound (or find yet a
better name that indicates that the argument is not skipped, but instead
will be an argument of the resulting partial function).

----------------------------------------------------------------------

Comment By: Calvin Spealman (ironfroggy)
Date: 2007-04-23 23:45

Message:
Logged In: YES 
user_id=112166
Originator: YES

File Added: partial_skip.patch

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1706256&group_id=5470

From noreply at sourceforge.net  Thu Apr 26 08:15:08 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Wed, 25 Apr 2007 23:15:08 -0700
Subject: [ python-Feature Requests-1706256 ] Give Partial the ability to skip
	positionals
Message-ID: <E1HgxG0-00075o-Cv@sc8-sf-web21.sourceforge.net>

Feature Requests item #1706256, was opened at 2007-04-23 19:18
Message generated for change (Comment added) made by rhettinger
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1706256&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Calvin Spealman (ironfroggy)
Assigned to: Nobody/Anonymous (nobody)
Summary: Give Partial the ability to skip positionals

Initial Comment:
There are some situations where you want to skip positional arguments in a use of a partial function. In other words, you want to create a partial that applies positional arguments out of order or without applying values to one or more lower positional arguments. In some cases keyword arguments can be used instead, but this has two obvious drawbacks. Firstly, it causes the caller to rely on the name of a positional in a callee, which breaks encapsulation. Secondly, on the case of the function being applied to being a builtin, it fails completely, as they will not take positional arguments by name at all. I propose a class attribute to the partial type, 'skip', which will be a singleton to pass to a partial object signifying this skipping of positionals. The following example demonstrates.

from functools import partial

def add(a, b):
    return a + b

append_abc = partial(add, partial.skip, "abc")
assert append_abc("xyz") == "xyzabc"

Obviously this example would break if used as partial(add, b="abc") and the maintainer of add changed the positional names to 'first' and 'second' or 'pre' and 'post', which is perfectly reasonable. We do not need to expect the names of our positional arguments are depended upon. It would also break when someone gets smart and replaces the add function with operator.add, of course.

----------------------------------------------------------------------

>Comment By: Raymond Hettinger (rhettinger)
Date: 2007-04-26 01:15

Message:
Logged In: YES 
user_id=80475
Originator: NO

-1 on the concept for this patch.  We're working too hard to avoid simple
uses of lambda or def.  The additonal complexity and impact on readability
isn't work it.

----------------------------------------------------------------------

Comment By: Alexander Belopolsky (belopolsky)
Date: 2007-04-25 23:14

Message:
Logged In: YES 
user_id=835142
Originator: NO

If you remove partial_type.tp_dict = PyDict_New(); at line 309, the patch
will pass test_functools.

A few comments:

Design:
1. partial.skip should be an instance of a singleton class (look at
NoneType implementation in object.c)
2. repr(partial.skip) should be 'functools.partial.skip' (easy to
implement once #1 is done)

Implementation:
1.  In the loop over pto->args you know that i < npargs, so you can use
PyTuple_GET_ITEM and there is no need to check for arg==NULL
2. You should check PyTuple_GetItem(args, pull_index) for null and return
with error if too few arguments is supplied.  Better yet, find number of
supplied args outside the loop and raise your own error if pool_index grows
to that number.
3. It looks like you are leaking references. I don't see where you decref
ptoargscopy and arg after concatenation.


----------------------------------------------------------------------

Comment By: Calvin Spealman (ironfroggy)
Date: 2007-04-25 23:10

Message:
Logged In: YES 
user_id=112166
Originator: YES

Hmm, I didn't get such an error here on VC with the extra argument. I'll
change that here, too.

I figured the breaking of the subclassing of partial was related to what I
do with tp_dict, but I don't understand how. How did I blow it up? And,
yes, I will write tests for the new functionality, of course.

----------------------------------------------------------------------

Comment By: Alexander Belopolsky (belopolsky)
Date: 2007-04-25 22:34

Message:
Logged In: YES 
user_id=835142
Originator: NO

OK, I've got it.  Your patch breaks test_functools.  This is because you
have blown up partial_type.tp_dict . 

----------------------------------------------------------------------

Comment By: Alexander Belopolsky (belopolsky)
Date: 2007-04-25 22:21

Message:
Logged In: YES 
user_id=835142
Originator: NO

The current patch does not compile:

Modules/_functoolsmodule.c: In function 'init_functools':
Modules/_functoolsmodule.c:306: error: too many arguments to function
'PyObject_CallObject'

Once I removed the extra NULL argument, it seems to work fine.  What
exactly is broken?  Can you add unit tests for the new functionality?

----------------------------------------------------------------------

Comment By: Calvin Spealman (ironfroggy)
Date: 2007-04-25 21:58

Message:
Logged In: YES 
user_id=112166
Originator: YES

If anyone has a serious argument against the partial.skip name, I would
take partial.unbound as my second choice, but I definitely prefer
partial.skip to it. Third would be partial.latebound. I still can't figure
out how my code broke subclassing of partial, so if anyone can take a look,
I'd appreciate it hugely.

----------------------------------------------------------------------

Comment By: Alexander Belopolsky (belopolsky)
Date: 2007-04-25 21:31

Message:
Logged In: YES 
user_id=835142
Originator: NO

Names I've seen used for this purpose elsewhere: slot, arg, missing.

----------------------------------------------------------------------

Comment By: Martin v. L?wis (loewis)
Date: 2007-04-24 00:41

Message:
Logged In: YES 
user_id=21627
Originator: NO

I would not call it partial.skip, but partial.unbound (or find yet a
better name that indicates that the argument is not skipped, but instead
will be an argument of the resulting partial function).

----------------------------------------------------------------------

Comment By: Calvin Spealman (ironfroggy)
Date: 2007-04-23 22:45

Message:
Logged In: YES 
user_id=112166
Originator: YES

File Added: partial_skip.patch

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1706256&group_id=5470

From noreply at sourceforge.net  Thu Apr 26 08:16:48 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Wed, 25 Apr 2007 23:16:48 -0700
Subject: [ python-Feature Requests-1705520 ] pyunit should allow __unittest in
	locals to trim stackframes
Message-ID: <E1HgxHc-0007DD-Lt@sc8-sf-web21.sourceforge.net>

Feature Requests item #1705520, was opened at 2007-04-22 22:56
Message generated for change (Settings changed) made by rhettinger
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1705520&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.6
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Robert Collins (rbcollins)
>Assigned to: Steve Purcell (purcell)
Summary: pyunit should allow __unittest in locals to trim stackframes

Initial Comment:
pyunit looks for __unittest = 1 in the globals of functions in an assertion stacktrace. This is used to trim the trace so that unittest implementation methods do  not show up.

It would be great if it looked in the locals of the frame as well, as this would allow subclasses of TestCase that add new assertFOO methods to have them filtered in the same manner.

e.g. (bad example :))
def assertComplexState(self, inputs):
    __unittest = 1
    self.assertEqual('42', inputs[0], 'the input %s is not the right answer' % inputs)



----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1705520&group_id=5470

From noreply at sourceforge.net  Thu Apr 26 08:32:39 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Wed, 25 Apr 2007 23:32:39 -0700
Subject: [ python-Feature Requests-1706256 ] Give Partial the ability to skip
	positionals
Message-ID: <E1HgxWx-0000WH-4s@sc8-sf-web21.sourceforge.net>

Feature Requests item #1706256, was opened at 2007-04-23 20:18
Message generated for change (Comment added) made by belopolsky
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1706256&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Calvin Spealman (ironfroggy)
Assigned to: Nobody/Anonymous (nobody)
Summary: Give Partial the ability to skip positionals

Initial Comment:
There are some situations where you want to skip positional arguments in a use of a partial function. In other words, you want to create a partial that applies positional arguments out of order or without applying values to one or more lower positional arguments. In some cases keyword arguments can be used instead, but this has two obvious drawbacks. Firstly, it causes the caller to rely on the name of a positional in a callee, which breaks encapsulation. Secondly, on the case of the function being applied to being a builtin, it fails completely, as they will not take positional arguments by name at all. I propose a class attribute to the partial type, 'skip', which will be a singleton to pass to a partial object signifying this skipping of positionals. The following example demonstrates.

from functools import partial

def add(a, b):
    return a + b

append_abc = partial(add, partial.skip, "abc")
assert append_abc("xyz") == "xyzabc"

Obviously this example would break if used as partial(add, b="abc") and the maintainer of add changed the positional names to 'first' and 'second' or 'pre' and 'post', which is perfectly reasonable. We do not need to expect the names of our positional arguments are depended upon. It would also break when someone gets smart and replaces the add function with operator.add, of course.

----------------------------------------------------------------------

Comment By: Alexander Belopolsky (belopolsky)
Date: 2007-04-26 02:32

Message:
Logged In: YES 
user_id=835142
Originator: NO

I am actually -1 on the concept as well.  If you go to the trouble of
having a skip singleton in the language, then partial application syntax
should just be call syntax as in add(skip, 2). However this will not be
python anymore.  Other languages that have partial application support use
special syntax such as add(,2) or add(:,2).  Since adding syntax is out of
the question, it is hard to argue that partial(add, partial.skip, 2) is so
much better than lambda x: add(x,2).

----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2007-04-26 02:15

Message:
Logged In: YES 
user_id=80475
Originator: NO

-1 on the concept for this patch.  We're working too hard to avoid simple
uses of lambda or def.  The additonal complexity and impact on readability
isn't work it.

----------------------------------------------------------------------

Comment By: Alexander Belopolsky (belopolsky)
Date: 2007-04-26 00:14

Message:
Logged In: YES 
user_id=835142
Originator: NO

If you remove partial_type.tp_dict = PyDict_New(); at line 309, the patch
will pass test_functools.

A few comments:

Design:
1. partial.skip should be an instance of a singleton class (look at
NoneType implementation in object.c)
2. repr(partial.skip) should be 'functools.partial.skip' (easy to
implement once #1 is done)

Implementation:
1.  In the loop over pto->args you know that i < npargs, so you can use
PyTuple_GET_ITEM and there is no need to check for arg==NULL
2. You should check PyTuple_GetItem(args, pull_index) for null and return
with error if too few arguments is supplied.  Better yet, find number of
supplied args outside the loop and raise your own error if pool_index grows
to that number.
3. It looks like you are leaking references. I don't see where you decref
ptoargscopy and arg after concatenation.


----------------------------------------------------------------------

Comment By: Calvin Spealman (ironfroggy)
Date: 2007-04-26 00:10

Message:
Logged In: YES 
user_id=112166
Originator: YES

Hmm, I didn't get such an error here on VC with the extra argument. I'll
change that here, too.

I figured the breaking of the subclassing of partial was related to what I
do with tp_dict, but I don't understand how. How did I blow it up? And,
yes, I will write tests for the new functionality, of course.

----------------------------------------------------------------------

Comment By: Alexander Belopolsky (belopolsky)
Date: 2007-04-25 23:34

Message:
Logged In: YES 
user_id=835142
Originator: NO

OK, I've got it.  Your patch breaks test_functools.  This is because you
have blown up partial_type.tp_dict . 

----------------------------------------------------------------------

Comment By: Alexander Belopolsky (belopolsky)
Date: 2007-04-25 23:21

Message:
Logged In: YES 
user_id=835142
Originator: NO

The current patch does not compile:

Modules/_functoolsmodule.c: In function 'init_functools':
Modules/_functoolsmodule.c:306: error: too many arguments to function
'PyObject_CallObject'

Once I removed the extra NULL argument, it seems to work fine.  What
exactly is broken?  Can you add unit tests for the new functionality?

----------------------------------------------------------------------

Comment By: Calvin Spealman (ironfroggy)
Date: 2007-04-25 22:58

Message:
Logged In: YES 
user_id=112166
Originator: YES

If anyone has a serious argument against the partial.skip name, I would
take partial.unbound as my second choice, but I definitely prefer
partial.skip to it. Third would be partial.latebound. I still can't figure
out how my code broke subclassing of partial, so if anyone can take a look,
I'd appreciate it hugely.

----------------------------------------------------------------------

Comment By: Alexander Belopolsky (belopolsky)
Date: 2007-04-25 22:31

Message:
Logged In: YES 
user_id=835142
Originator: NO

Names I've seen used for this purpose elsewhere: slot, arg, missing.

----------------------------------------------------------------------

Comment By: Martin v. L?wis (loewis)
Date: 2007-04-24 01:41

Message:
Logged In: YES 
user_id=21627
Originator: NO

I would not call it partial.skip, but partial.unbound (or find yet a
better name that indicates that the argument is not skipped, but instead
will be an argument of the resulting partial function).

----------------------------------------------------------------------

Comment By: Calvin Spealman (ironfroggy)
Date: 2007-04-23 23:45

Message:
Logged In: YES 
user_id=112166
Originator: YES

File Added: partial_skip.patch

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1706256&group_id=5470

From noreply at sourceforge.net  Thu Apr 26 09:50:19 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Thu, 26 Apr 2007 00:50:19 -0700
Subject: [ python-Feature Requests-1706256 ] Give Partial the ability to skip
	positionals
Message-ID: <E1Hgyk7-00077H-L5@sc8-sf-web22.sourceforge.net>

Feature Requests item #1706256, was opened at 2007-04-24 02:18
Message generated for change (Comment added) made by loewis
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1706256&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
>Status: Closed
>Resolution: Rejected
Priority: 5
Private: No
Submitted By: Calvin Spealman (ironfroggy)
Assigned to: Nobody/Anonymous (nobody)
Summary: Give Partial the ability to skip positionals

Initial Comment:
There are some situations where you want to skip positional arguments in a use of a partial function. In other words, you want to create a partial that applies positional arguments out of order or without applying values to one or more lower positional arguments. In some cases keyword arguments can be used instead, but this has two obvious drawbacks. Firstly, it causes the caller to rely on the name of a positional in a callee, which breaks encapsulation. Secondly, on the case of the function being applied to being a builtin, it fails completely, as they will not take positional arguments by name at all. I propose a class attribute to the partial type, 'skip', which will be a singleton to pass to a partial object signifying this skipping of positionals. The following example demonstrates.

from functools import partial

def add(a, b):
    return a + b

append_abc = partial(add, partial.skip, "abc")
assert append_abc("xyz") == "xyzabc"

Obviously this example would break if used as partial(add, b="abc") and the maintainer of add changed the positional names to 'first' and 'second' or 'pre' and 'post', which is perfectly reasonable. We do not need to expect the names of our positional arguments are depended upon. It would also break when someone gets smart and replaces the add function with operator.add, of course.

----------------------------------------------------------------------

>Comment By: Martin v. L?wis (loewis)
Date: 2007-04-26 09:50

Message:
Logged In: YES 
user_id=21627
Originator: NO

Based on this feedback, I'm rejecting the patch. Thanks for proposing it,
anyway.

----------------------------------------------------------------------

Comment By: Alexander Belopolsky (belopolsky)
Date: 2007-04-26 08:32

Message:
Logged In: YES 
user_id=835142
Originator: NO

I am actually -1 on the concept as well.  If you go to the trouble of
having a skip singleton in the language, then partial application syntax
should just be call syntax as in add(skip, 2). However this will not be
python anymore.  Other languages that have partial application support use
special syntax such as add(,2) or add(:,2).  Since adding syntax is out of
the question, it is hard to argue that partial(add, partial.skip, 2) is so
much better than lambda x: add(x,2).

----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2007-04-26 08:15

Message:
Logged In: YES 
user_id=80475
Originator: NO

-1 on the concept for this patch.  We're working too hard to avoid simple
uses of lambda or def.  The additonal complexity and impact on readability
isn't work it.

----------------------------------------------------------------------

Comment By: Alexander Belopolsky (belopolsky)
Date: 2007-04-26 06:14

Message:
Logged In: YES 
user_id=835142
Originator: NO

If you remove partial_type.tp_dict = PyDict_New(); at line 309, the patch
will pass test_functools.

A few comments:

Design:
1. partial.skip should be an instance of a singleton class (look at
NoneType implementation in object.c)
2. repr(partial.skip) should be 'functools.partial.skip' (easy to
implement once #1 is done)

Implementation:
1.  In the loop over pto->args you know that i < npargs, so you can use
PyTuple_GET_ITEM and there is no need to check for arg==NULL
2. You should check PyTuple_GetItem(args, pull_index) for null and return
with error if too few arguments is supplied.  Better yet, find number of
supplied args outside the loop and raise your own error if pool_index grows
to that number.
3. It looks like you are leaking references. I don't see where you decref
ptoargscopy and arg after concatenation.


----------------------------------------------------------------------

Comment By: Calvin Spealman (ironfroggy)
Date: 2007-04-26 06:10

Message:
Logged In: YES 
user_id=112166
Originator: YES

Hmm, I didn't get such an error here on VC with the extra argument. I'll
change that here, too.

I figured the breaking of the subclassing of partial was related to what I
do with tp_dict, but I don't understand how. How did I blow it up? And,
yes, I will write tests for the new functionality, of course.

----------------------------------------------------------------------

Comment By: Alexander Belopolsky (belopolsky)
Date: 2007-04-26 05:34

Message:
Logged In: YES 
user_id=835142
Originator: NO

OK, I've got it.  Your patch breaks test_functools.  This is because you
have blown up partial_type.tp_dict . 

----------------------------------------------------------------------

Comment By: Alexander Belopolsky (belopolsky)
Date: 2007-04-26 05:21

Message:
Logged In: YES 
user_id=835142
Originator: NO

The current patch does not compile:

Modules/_functoolsmodule.c: In function 'init_functools':
Modules/_functoolsmodule.c:306: error: too many arguments to function
'PyObject_CallObject'

Once I removed the extra NULL argument, it seems to work fine.  What
exactly is broken?  Can you add unit tests for the new functionality?

----------------------------------------------------------------------

Comment By: Calvin Spealman (ironfroggy)
Date: 2007-04-26 04:58

Message:
Logged In: YES 
user_id=112166
Originator: YES

If anyone has a serious argument against the partial.skip name, I would
take partial.unbound as my second choice, but I definitely prefer
partial.skip to it. Third would be partial.latebound. I still can't figure
out how my code broke subclassing of partial, so if anyone can take a look,
I'd appreciate it hugely.

----------------------------------------------------------------------

Comment By: Alexander Belopolsky (belopolsky)
Date: 2007-04-26 04:31

Message:
Logged In: YES 
user_id=835142
Originator: NO

Names I've seen used for this purpose elsewhere: slot, arg, missing.

----------------------------------------------------------------------

Comment By: Martin v. L?wis (loewis)
Date: 2007-04-24 07:41

Message:
Logged In: YES 
user_id=21627
Originator: NO

I would not call it partial.skip, but partial.unbound (or find yet a
better name that indicates that the argument is not skipped, but instead
will be an argument of the resulting partial function).

----------------------------------------------------------------------

Comment By: Calvin Spealman (ironfroggy)
Date: 2007-04-24 05:45

Message:
Logged In: YES 
user_id=112166
Originator: YES

File Added: partial_skip.patch

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1706256&group_id=5470

From noreply at sourceforge.net  Thu Apr 26 09:57:10 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Thu, 26 Apr 2007 00:57:10 -0700
Subject: [ python-Feature Requests-1692592 ] Stripping debugging symbols from
	compiled C extensions
Message-ID: <E1Hgyqk-0001WP-Oa@sc8-sf-web21.sourceforge.net>

Feature Requests item #1692592, was opened at 2007-04-02 01:00
Message generated for change (Comment added) made by loewis
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1692592&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Distutils
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Viktor Ferenczi (complex)
Assigned to: Nobody/Anonymous (nobody)
Summary: Stripping debugging symbols from compiled C extensions

Initial Comment:
It would be nice to automatically strip debugging symbols from compiled library files (such as .so files) if a C extension module is not build in debug mode. This could somehow reduce memory footprint and storage requirements for extension modules.

Distutils already does this for cygwin and emx compilers with the following code in cygwinccompiler.py and emxccompiler.py:

# who wants symbols and a many times larger output file
# should explicitly switch the debug mode on
# otherwise we let dllwrap/ld strip the output file
# (On my machine: 10KB < stripped_file < ??100KB
#   unstripped_file = stripped_file + XXX KB
#  ( XXX=254 for a typical python extension))
if not debug:
    extra_preargs.append("-s")

This code should be somehow integrated into base compiler classes such as UnixCCompiler. I've added the following at the beginning of UnixCCompiler.link function:

if not debug:
    if extra_preargs is None:
        extra_preargs = []
    extra_preargs.append("-s")

This works for me with gcc under Linux (Debian Sarge). I does not provide a patch, since this could be the best solution for this.

----------------------------------------------------------------------

>Comment By: Martin v. L?wis (loewis)
Date: 2007-04-26 09:57

Message:
Logged In: YES 
user_id=21627
Originator: NO

Just for the record: stripping symbols does *not* reduce memory footprint.
It does (only) reduce storage requirements.

One problem with implementing the feature is to make it work portably on
all systems. Adding -s to the linker command line is most likely not
portable, i.e. some systems may not support that. Python currently ignores
the concept of stripping entirely (not just for extension modules, but also
for the interpreter itself, partially due to the problem that adding that
further complicates portability. 

On Linux, this problem is commonly solved by distributors performing the
stripping as part of the packaging utilities. E.g. for Debian, just add
dh_strip into the debian/rules file, and the packaging
will strip all binaries according to the Debian policy.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1692592&group_id=5470

From noreply at sourceforge.net  Thu Apr 26 15:55:13 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Thu, 26 Apr 2007 06:55:13 -0700
Subject: [ python-Bugs-1682241 ] Problems with urllib2 read()
Message-ID: <E1Hh4RF-00049Y-NB@sc8-sf-web21.sourceforge.net>

Bugs item #1682241, was opened at 2007-03-16 12:00
Message generated for change (Comment added) made by ironfroggy
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1682241&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Lucas Malor (lucas_malor)
Assigned to: Nobody/Anonymous (nobody)
Summary: Problems with urllib2 read()

Initial Comment:
urllib2 objects opened with urlopen() does not have the method seek() as file objects. So reading only some bytes from opened urls is pratically forbidden.

An example: I tried to open an url and check if it's a gzip file. If IOError is raised I read the file (to do this I applied the #1675951 patch: https://sourceforge.net/tracker/index.php?func=detail&aid=1675951&group_id=5470&atid=305470 )

But after I tried to open the file as gzip, if it's not a gzip file the current position in the urllib object is on the second byte. So read() returns the data from the 3rd to the last byte. You can't check the header of the file before storing it on hd. Well, so what is urlopen() for? If I must store the file by url on hd and reload it, I can use urlretrieve() ...

----------------------------------------------------------------------

Comment By: Calvin Spealman (ironfroggy)
Date: 2007-04-26 09:55

Message:
Logged In: YES 
user_id=112166
Originator: NO

I have to agree that this is not a bug. HTTP responses are strings, not
random access files. Adding a seek would have disastrous performance
penalties. If you think the work around is too complicated, I can't
understand why.

----------------------------------------------------------------------

Comment By: Zacherates (maenpaa)
Date: 2007-03-20 21:39

Message:
Logged In: YES 
user_id=1421845
Originator: NO

> I use the method you wrote, but this must be done manually,
> and I don't know why.
read() is a stream processing method, whereas seek() is a random access
processing method.  HTTP resources are in essence streams so they implement
read() but not seek().  Trying to shoehorn a stream to act like a random
access file has some rather important technical implications.  For example:
what happens when an HTTP resource is larger than available memory and we
try to maintain a full featured seek() implementation?

> so what is urlopen() for?
Fetching a webpage or RSS feed and feeding it to a parser, for example.

StringIO is a class that was designed to implement feature complete,
random access, file-like object behavior that can be wrapped around a
stream.  StringIO can and should be used as an adapter for when you have a
stream that you need random access to.  This allows designers the freedom
to simply implement a good read() implementation and let clients wrap the
output in a StringIO if needed.

If in your application you always want random access and you don't have to
deal with large files:
def my_urlopen(*args, **kwargs):
   return StringIO.StringIO(urllib2.urlopen(*args, **kwargs).read())

Python makes delegation trivially easy.

In essence, urlfiles (the result of urllib2.urlopen()) and regular files
(the result of open()) behave differently because they implement different
interfaces.  If you use the common interface (read), then you can treat
them equally.  If you use the specialized interface (seek, tell, etc.)
you'll have trouble.  The solution is wrap the general objects in a
specialized object that implements the desired interface, StringIO.

----------------------------------------------------------------------

Comment By: Lucas Malor (lucas_malor)
Date: 2007-03-20 04:59

Message:
Logged In: YES 
user_id=1403274
Originator: YES

> If you need to seek, you can wrap the file-like object in a
> StringIO (which is what urllib would have to do internally
> [...] )

I think it's really a bug, or at least a non-pythonic method.
I use the method you wrote, but this must be done manually,
and I don't know why. Actually without this "trick" you can't
handle url and file objects together as they don't work in
the same manner. I think it's not too complicated using the
internal StringIO object in urllib class when I must seek()
or use other file-like methods.

> You can check the type of the response content before you try
> to uncompress it via the Content-Encoding header of the
> response

It's not a generic solution

(thanks anyway for suggested solutions :) )

----------------------------------------------------------------------

Comment By: Zacherates (maenpaa)
Date: 2007-03-19 22:43

Message:
Logged In: YES 
user_id=1421845
Originator: NO

I'd contend that this is not a bug:
 * If you need to seek, you can wrap the file-like object in a StringIO
(which is what urllib would have to do internally, thus incurring the
StringIO overhead for all clients, even those that don't need the
functionality).
 * You can check the type of the response content before you try to
uncompress it via the Content-Encoding header of the response.  The
meta-data is there for a reason.

Check
http://www.diveintopython.org/http_web_services/gzip_compression.html for a
rather complete treatment of your use-case.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1682241&group_id=5470

From noreply at sourceforge.net  Thu Apr 26 21:02:04 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Thu, 26 Apr 2007 12:02:04 -0700
Subject: [ python-Bugs-1708316 ] doctest work with Windows PyReadline
Message-ID: <E1Hh9EC-0005GC-By@sc8-sf-web24.sourceforge.net>

Bugs item #1708316, was opened at 2007-04-26 12:02
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1708316&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: manuelg (manuelg_)
Assigned to: Nobody/Anonymous (nobody)
Summary: doctest work with Windows PyReadline

Initial Comment:
doctest crashes when working with Windows PyReadline (PyReadline is a component of Windows IPython)

PyReadline expects "_SpoofOut" to have an "encoding" attribute

E
======================================================================
ERROR: testDocTest (__main__.TestDocTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test_freecell_solver.py", line 26, in testDocTest
    r = doctest.testmod(freecell_solver)
  File "c:\Python25\Lib\doctest.py", line 1799, in testmod
    runner.run(test)
  File "c:\Python25\Lib\doctest.py", line 1335, in run
    self.debugger = _OutputRedirectingPdb(save_stdout)
  File "c:\Python25\Lib\doctest.py", line 320, in __init__
    pdb.Pdb.__init__(self, stdout=out)
  File "c:\Python25\Lib\pdb.py", line 66, in __init__
    import readline
  File "C:\Python25\Lib\site-packages\readline.py", line 5, in <module>
    from pyreadline import *
  File "C:\Python25\Lib\site-packages\pyreadline\__init__.py", line 10, in <module>
    from rlmain import *
  File "C:\Python25\Lib\site-packages\pyreadline\rlmain.py", line 13, in <module>
    import clipboard,logger,console
  File "C:\Python25\Lib\site-packages\pyreadline\console\__init__.py", line 14,in <module>
    from console import *
  File "C:\Python25\Lib\site-packages\pyreadline\console\console.py", line 118,in <module>
    consolecodepage=sys.stdout.encoding
AttributeError: _SpoofOut instance has no attribute 'encoding'

This is an easy fix with 2 lines of code to doctest.py

right after doctest.py imports "sys", store the "sys.stdout.encoding"

_sys_stdout_encoding = sys.stdout.encoding

Then add this as an attribute "encoding" in the "_SpoofOut" class

# Override some StringIO methods.
class _SpoofOut(StringIO):

    ....
            
    encoding = _sys_stdout_encoding



----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1708316&group_id=5470

From noreply at sourceforge.net  Thu Apr 26 21:18:45 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Thu, 26 Apr 2007 12:18:45 -0700
Subject: [ python-Bugs-1708326 ] find_module doc ambiguity
Message-ID: <E1Hh9UL-00070t-0C@sc8-sf-web24.sourceforge.net>

Bugs item #1708326, was opened at 2007-04-26 13:18
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1708326&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Documentation
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Andrew McNabb (amcnabb)
Assigned to: Nobody/Anonymous (nobody)
Summary: find_module doc ambiguity

Initial Comment:
The doc string for find_module doesn't make it clear that you can do:

stats_path = imp.find_module('scipy/stats')

It makes it sound like you would have to do:

scipy_path = imp.find_module('scipy')[1]
stats_path = imp.find_module('scipy', stats_path)[1]

However, the shorter snippet seems to work just fine.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1708326&group_id=5470

From noreply at sourceforge.net  Thu Apr 26 22:38:42 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Thu, 26 Apr 2007 13:38:42 -0700
Subject: [ python-Bugs-1708362 ] Why not sequential?
Message-ID: <E1HhAji-0004bL-EX@sc8-sf-web23.sourceforge.net>

Bugs item #1708362, was opened at 2007-04-26 22:38
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1708362&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Lucas Malor (lucas_malor)
Assigned to: Nobody/Anonymous (nobody)
Summary: Why not sequential?

Initial Comment:
In my opinion it's not complicated, it's convoluted. I must use two object to handle one data stream.

Furthermore it's a waste of resources. I must copy data to another object. Luckily in my script I download and handle only little files. But what if a python program must handle big files?

If seek() can't be used (an except is raised), urllib could use a sequential access method.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1708362&group_id=5470

From noreply at sourceforge.net  Thu Apr 26 22:39:59 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Thu, 26 Apr 2007 13:39:59 -0700
Subject: [ python-Bugs-1708362 ] Why not sequential?
Message-ID: <E1HhAkx-0004ia-CR@sc8-sf-web23.sourceforge.net>

Bugs item #1708362, was opened at 2007-04-26 22:38
Message generated for change (Settings changed) made by lucas_malor
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1708362&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
>Status: Closed
Resolution: None
Priority: 5
Private: No
Submitted By: Lucas Malor (lucas_malor)
Assigned to: Nobody/Anonymous (nobody)
Summary: Why not sequential?

Initial Comment:
In my opinion it's not complicated, it's convoluted. I must use two object to handle one data stream.

Furthermore it's a waste of resources. I must copy data to another object. Luckily in my script I download and handle only little files. But what if a python program must handle big files?

If seek() can't be used (an except is raised), urllib could use a sequential access method.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1708362&group_id=5470

From noreply at sourceforge.net  Thu Apr 26 22:41:56 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Thu, 26 Apr 2007 13:41:56 -0700
Subject: [ python-Bugs-1682241 ] Problems with urllib2 read()
Message-ID: <E1HhAmq-0004w8-Ny@sc8-sf-web23.sourceforge.net>

Bugs item #1682241, was opened at 2007-03-16 17:00
Message generated for change (Comment added) made by lucas_malor
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1682241&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Lucas Malor (lucas_malor)
Assigned to: Nobody/Anonymous (nobody)
Summary: Problems with urllib2 read()

Initial Comment:
urllib2 objects opened with urlopen() does not have the method seek() as file objects. So reading only some bytes from opened urls is pratically forbidden.

An example: I tried to open an url and check if it's a gzip file. If IOError is raised I read the file (to do this I applied the #1675951 patch: https://sourceforge.net/tracker/index.php?func=detail&aid=1675951&group_id=5470&atid=305470 )

But after I tried to open the file as gzip, if it's not a gzip file the current position in the urllib object is on the second byte. So read() returns the data from the 3rd to the last byte. You can't check the header of the file before storing it on hd. Well, so what is urlopen() for? If I must store the file by url on hd and reload it, I can use urlretrieve() ...

----------------------------------------------------------------------

>Comment By: Lucas Malor (lucas_malor)
Date: 2007-04-26 22:41

Message:
Logged In: YES 
user_id=1403274
Originator: YES

In my opinion it's not complicated, it's convoluted. I must use two
object
to handle one data stream.

Furthermore it's a waste of resources. I must copy data to another
object.
Luckily in my script I download and handle only little files. But what if
a
python program must handle big files?

If seek() can't be used (an except is raised), urllib could use a
sequential access method.

----------------------------------------------------------------------

Comment By: Calvin Spealman (ironfroggy)
Date: 2007-04-26 15:55

Message:
Logged In: YES 
user_id=112166
Originator: NO

I have to agree that this is not a bug. HTTP responses are strings, not
random access files. Adding a seek would have disastrous performance
penalties. If you think the work around is too complicated, I can't
understand why.

----------------------------------------------------------------------

Comment By: Zacherates (maenpaa)
Date: 2007-03-21 02:39

Message:
Logged In: YES 
user_id=1421845
Originator: NO

> I use the method you wrote, but this must be done manually,
> and I don't know why.
read() is a stream processing method, whereas seek() is a random access
processing method.  HTTP resources are in essence streams so they implement
read() but not seek().  Trying to shoehorn a stream to act like a random
access file has some rather important technical implications.  For example:
what happens when an HTTP resource is larger than available memory and we
try to maintain a full featured seek() implementation?

> so what is urlopen() for?
Fetching a webpage or RSS feed and feeding it to a parser, for example.

StringIO is a class that was designed to implement feature complete,
random access, file-like object behavior that can be wrapped around a
stream.  StringIO can and should be used as an adapter for when you have a
stream that you need random access to.  This allows designers the freedom
to simply implement a good read() implementation and let clients wrap the
output in a StringIO if needed.

If in your application you always want random access and you don't have to
deal with large files:
def my_urlopen(*args, **kwargs):
   return StringIO.StringIO(urllib2.urlopen(*args, **kwargs).read())

Python makes delegation trivially easy.

In essence, urlfiles (the result of urllib2.urlopen()) and regular files
(the result of open()) behave differently because they implement different
interfaces.  If you use the common interface (read), then you can treat
them equally.  If you use the specialized interface (seek, tell, etc.)
you'll have trouble.  The solution is wrap the general objects in a
specialized object that implements the desired interface, StringIO.

----------------------------------------------------------------------

Comment By: Lucas Malor (lucas_malor)
Date: 2007-03-20 09:59

Message:
Logged In: YES 
user_id=1403274
Originator: YES

> If you need to seek, you can wrap the file-like object in a
> StringIO (which is what urllib would have to do internally
> [...] )

I think it's really a bug, or at least a non-pythonic method.
I use the method you wrote, but this must be done manually,
and I don't know why. Actually without this "trick" you can't
handle url and file objects together as they don't work in
the same manner. I think it's not too complicated using the
internal StringIO object in urllib class when I must seek()
or use other file-like methods.

> You can check the type of the response content before you try
> to uncompress it via the Content-Encoding header of the
> response

It's not a generic solution

(thanks anyway for suggested solutions :) )

----------------------------------------------------------------------

Comment By: Zacherates (maenpaa)
Date: 2007-03-20 03:43

Message:
Logged In: YES 
user_id=1421845
Originator: NO

I'd contend that this is not a bug:
 * If you need to seek, you can wrap the file-like object in a StringIO
(which is what urllib would have to do internally, thus incurring the
StringIO overhead for all clients, even those that don't need the
functionality).
 * You can check the type of the response content before you try to
uncompress it via the Content-Encoding header of the response.  The
meta-data is there for a reason.

Check
http://www.diveintopython.org/http_web_services/gzip_compression.html for a
rather complete treatment of your use-case.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1682241&group_id=5470

From noreply at sourceforge.net  Thu Apr 26 22:44:19 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Thu, 26 Apr 2007 13:44:19 -0700
Subject: [ python-Bugs-1708362 ] Why not sequential?
Message-ID: <E1HhAp9-0005Fa-OP@sc8-sf-web23.sourceforge.net>

Bugs item #1708362, was opened at 2007-04-26 22:38
Message generated for change (Settings changed) made by lucas_malor
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1708362&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
>Group: Trash
>Status: Deleted
>Resolution: Duplicate
Priority: 5
Private: No
Submitted By: Lucas Malor (lucas_malor)
Assigned to: Nobody/Anonymous (nobody)
Summary: Why not sequential?

Initial Comment:
In my opinion it's not complicated, it's convoluted. I must use two object to handle one data stream.

Furthermore it's a waste of resources. I must copy data to another object. Luckily in my script I download and handle only little files. But what if a python program must handle big files?

If seek() can't be used (an except is raised), urllib could use a sequential access method.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1708362&group_id=5470

From noreply at sourceforge.net  Fri Apr 27 05:36:58 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Thu, 26 Apr 2007 20:36:58 -0700
Subject: [ python-Bugs-1682241 ] Problems with urllib2 read()
Message-ID: <E1HhHGU-0004t4-Sf@sc8-sf-web22.sourceforge.net>

Bugs item #1682241, was opened at 2007-03-16 12:00
Message generated for change (Comment added) made by maenpaa
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1682241&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Lucas Malor (lucas_malor)
Assigned to: Nobody/Anonymous (nobody)
Summary: Problems with urllib2 read()

Initial Comment:
urllib2 objects opened with urlopen() does not have the method seek() as file objects. So reading only some bytes from opened urls is pratically forbidden.

An example: I tried to open an url and check if it's a gzip file. If IOError is raised I read the file (to do this I applied the #1675951 patch: https://sourceforge.net/tracker/index.php?func=detail&aid=1675951&group_id=5470&atid=305470 )

But after I tried to open the file as gzip, if it's not a gzip file the current position in the urllib object is on the second byte. So read() returns the data from the 3rd to the last byte. You can't check the header of the file before storing it on hd. Well, so what is urlopen() for? If I must store the file by url on hd and reload it, I can use urlretrieve() ...

----------------------------------------------------------------------

Comment By: Zacherates (maenpaa)
Date: 2007-04-26 23:36

Message:
Logged In: YES 
user_id=1421845
Originator: NO

> In my opinion it's not complicated, it's convoluted. I must use two
> object to handle one data stream.

seek() is not a stream operation. It is a random access operation
(file-like != stream). If you were only trying to use stream operations
then you wouldn't have these problems.   

Each class provides a separate functionality, urllib gets the file while
StringIO stores it.  The fact that these responsibilities are given to
different classes should not be surprising since the represent separately
useful concepts that abstract different things.  It's not convoluted, it's
good design.  If every class tried to do everything, pretty soon you're
adding solve_my_business_problem_using_SOA() to __builtins__ and nobody
wants that.


> Furthermore it's a waste of resources. I must copy data to another
> object. Luckily in my script I download and handle only little files.
But what if
> a python program must handle big files?

This is exactly why urllib *doesn't* provide seek. Deep down in the
networking library there's a socket with a 8KiB buffer talking to the HTTP
server. No matter how big the file you're getting with urllib, once that
buffer is full the socket starts dropping packets. 

To provide seek(), urllib would need to keep an entire copy of the file
that was retrieved, (or provide mark()/seek(), but those have wildly
different semantics from the seek()s were used to in python, and besides
they're too Java).  This works fine if you're only working with small
files, but you raise a good point: "But what if a python program must
handle big files?".  What about really big files (say a Knoppix DVD ISO)? 
Sure you could use urlretrieve, but what if urlretrive is implemented in
terms of urlopen?

Sure urllib could implement seek (with the same semantics as file.seek())
but that would mean breaking urllib for any resource big enough that you
don't want the whole thing in memory.


>> You can check the type of the response content before you try
>> to uncompress it via the Content-Encoding header of the
>> response

>It's not a generic solution

The point of this suggestion is not that this is the be all and end all
solution, but that code that *needs* seek can probably be rewritten so that
it does not.  Either that or you could implement BufferedReader with the
methods mark() and seek() and wrap the result of urlopen.


----------------------------------------------------------------------

Comment By: Lucas Malor (lucas_malor)
Date: 2007-04-26 16:41

Message:
Logged In: YES 
user_id=1403274
Originator: YES

In my opinion it's not complicated, it's convoluted. I must use two
object
to handle one data stream.

Furthermore it's a waste of resources. I must copy data to another
object.
Luckily in my script I download and handle only little files. But what if
a
python program must handle big files?

If seek() can't be used (an except is raised), urllib could use a
sequential access method.

----------------------------------------------------------------------

Comment By: Calvin Spealman (ironfroggy)
Date: 2007-04-26 09:55

Message:
Logged In: YES 
user_id=112166
Originator: NO

I have to agree that this is not a bug. HTTP responses are strings, not
random access files. Adding a seek would have disastrous performance
penalties. If you think the work around is too complicated, I can't
understand why.

----------------------------------------------------------------------

Comment By: Zacherates (maenpaa)
Date: 2007-03-20 21:39

Message:
Logged In: YES 
user_id=1421845
Originator: NO

> I use the method you wrote, but this must be done manually,
> and I don't know why.
read() is a stream processing method, whereas seek() is a random access
processing method.  HTTP resources are in essence streams so they implement
read() but not seek().  Trying to shoehorn a stream to act like a random
access file has some rather important technical implications.  For example:
what happens when an HTTP resource is larger than available memory and we
try to maintain a full featured seek() implementation?

> so what is urlopen() for?
Fetching a webpage or RSS feed and feeding it to a parser, for example.

StringIO is a class that was designed to implement feature complete,
random access, file-like object behavior that can be wrapped around a
stream.  StringIO can and should be used as an adapter for when you have a
stream that you need random access to.  This allows designers the freedom
to simply implement a good read() implementation and let clients wrap the
output in a StringIO if needed.

If in your application you always want random access and you don't have to
deal with large files:
def my_urlopen(*args, **kwargs):
   return StringIO.StringIO(urllib2.urlopen(*args, **kwargs).read())

Python makes delegation trivially easy.

In essence, urlfiles (the result of urllib2.urlopen()) and regular files
(the result of open()) behave differently because they implement different
interfaces.  If you use the common interface (read), then you can treat
them equally.  If you use the specialized interface (seek, tell, etc.)
you'll have trouble.  The solution is wrap the general objects in a
specialized object that implements the desired interface, StringIO.

----------------------------------------------------------------------

Comment By: Lucas Malor (lucas_malor)
Date: 2007-03-20 04:59

Message:
Logged In: YES 
user_id=1403274
Originator: YES

> If you need to seek, you can wrap the file-like object in a
> StringIO (which is what urllib would have to do internally
> [...] )

I think it's really a bug, or at least a non-pythonic method.
I use the method you wrote, but this must be done manually,
and I don't know why. Actually without this "trick" you can't
handle url and file objects together as they don't work in
the same manner. I think it's not too complicated using the
internal StringIO object in urllib class when I must seek()
or use other file-like methods.

> You can check the type of the response content before you try
> to uncompress it via the Content-Encoding header of the
> response

It's not a generic solution

(thanks anyway for suggested solutions :) )

----------------------------------------------------------------------

Comment By: Zacherates (maenpaa)
Date: 2007-03-19 22:43

Message:
Logged In: YES 
user_id=1421845
Originator: NO

I'd contend that this is not a bug:
 * If you need to seek, you can wrap the file-like object in a StringIO
(which is what urllib would have to do internally, thus incurring the
StringIO overhead for all clients, even those that don't need the
functionality).
 * You can check the type of the response content before you try to
uncompress it via the Content-Encoding header of the response.  The
meta-data is there for a reason.

Check
http://www.diveintopython.org/http_web_services/gzip_compression.html for a
rather complete treatment of your use-case.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1682241&group_id=5470

From noreply at sourceforge.net  Fri Apr 27 09:30:42 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Fri, 27 Apr 2007 00:30:42 -0700
Subject: [ python-Bugs-1708538 ] TimedRotatingFileHandler's doRollover opens
	file in "w" mode
Message-ID: <E1HhKug-000623-Eb@sc8-sf-web24.sourceforge.net>

Bugs item #1708538, was opened at 2007-04-27 09:30
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1708538&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Florian Schmidt (fastflo)
Assigned to: Nobody/Anonymous (nobody)
Summary: TimedRotatingFileHandler's doRollover opens file in "w" mode

Initial Comment:
if TimedRotatingFileHandler does an doRollover() it opens the new file hardcoded with mode "w" whilst the baseclass FileHandler opens it with mode "a" if nothing else is specified in the constructor. it should use the same default behavior as FileHandler and/or give a way to specify another open-mode.

if multiple processes write to the same file, and one does the RollOver all other newly spawned processes write to the end of the new file but the still running first process who did the rollover will write from the beginning of the file - overwriting log entries from the other procs...

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1708538&group_id=5470

From noreply at sourceforge.net  Fri Apr 27 11:26:35 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Fri, 27 Apr 2007 02:26:35 -0700
Subject: [ python-Bugs-1682241 ] Problems with urllib2 read()
Message-ID: <E1HhMip-0004er-HV@sc8-sf-web23.sourceforge.net>

Bugs item #1682241, was opened at 2007-03-16 17:00
Message generated for change (Comment added) made by lucas_malor
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1682241&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Lucas Malor (lucas_malor)
Assigned to: Nobody/Anonymous (nobody)
Summary: Problems with urllib2 read()

Initial Comment:
urllib2 objects opened with urlopen() does not have the method seek() as file objects. So reading only some bytes from opened urls is pratically forbidden.

An example: I tried to open an url and check if it's a gzip file. If IOError is raised I read the file (to do this I applied the #1675951 patch: https://sourceforge.net/tracker/index.php?func=detail&aid=1675951&group_id=5470&atid=305470 )

But after I tried to open the file as gzip, if it's not a gzip file the current position in the urllib object is on the second byte. So read() returns the data from the 3rd to the last byte. You can't check the header of the file before storing it on hd. Well, so what is urlopen() for? If I must store the file by url on hd and reload it, I can use urlretrieve() ...

----------------------------------------------------------------------

>Comment By: Lucas Malor (lucas_malor)
Date: 2007-04-27 11:26

Message:
Logged In: YES 
user_id=1403274
Originator: YES

If you don't want the intere file on your memory, there's two solutions:

----------
import urllib

urlobj = urllib.urlopen("someurl")
header = urlobj.read(1)
# some other operations (no other urlobj.read())

contents = header + urlobj.read()

----------

I don't think it's a --good-- solution, because some other programmers can
do other read() operations and mess all the result.

The other solution is:

----------
def readall(x) :
  url = x.geturl
  x.close()
  try :
    y = urllib.openurl(url)
  except :
    return None
  return y.read()
  

import urllib

urlobj = urllib.urlopen("someurl")
header = urlobj.read(1)
# some other operations

contents = readall(urlobj)

----------

This is still a bad solution (two calls to the server for the same file).

On the contrary I'm pretty sure using a sequencial access this can be done
without doing these workarounds.

(anyway I don't understand a thing: HTTP can't delegate the server to
seek() the file?)

----------------------------------------------------------------------

Comment By: Zacherates (maenpaa)
Date: 2007-04-27 05:36

Message:
Logged In: YES 
user_id=1421845
Originator: NO

> In my opinion it's not complicated, it's convoluted. I must use two
> object to handle one data stream.

seek() is not a stream operation. It is a random access operation
(file-like != stream). If you were only trying to use stream operations
then you wouldn't have these problems.   

Each class provides a separate functionality, urllib gets the file while
StringIO stores it.  The fact that these responsibilities are given to
different classes should not be surprising since the represent separately
useful concepts that abstract different things.  It's not convoluted, it's
good design.  If every class tried to do everything, pretty soon you're
adding solve_my_business_problem_using_SOA() to __builtins__ and nobody
wants that.


> Furthermore it's a waste of resources. I must copy data to another
> object. Luckily in my script I download and handle only little files.
But what if
> a python program must handle big files?

This is exactly why urllib *doesn't* provide seek. Deep down in the
networking library there's a socket with a 8KiB buffer talking to the HTTP
server. No matter how big the file you're getting with urllib, once that
buffer is full the socket starts dropping packets. 

To provide seek(), urllib would need to keep an entire copy of the file
that was retrieved, (or provide mark()/seek(), but those have wildly
different semantics from the seek()s were used to in python, and besides
they're too Java).  This works fine if you're only working with small
files, but you raise a good point: "But what if a python program must
handle big files?".  What about really big files (say a Knoppix DVD ISO)? 
Sure you could use urlretrieve, but what if urlretrive is implemented in
terms of urlopen?

Sure urllib could implement seek (with the same semantics as file.seek())
but that would mean breaking urllib for any resource big enough that you
don't want the whole thing in memory.


>> You can check the type of the response content before you try
>> to uncompress it via the Content-Encoding header of the
>> response

>It's not a generic solution

The point of this suggestion is not that this is the be all and end all
solution, but that code that *needs* seek can probably be rewritten so that
it does not.  Either that or you could implement BufferedReader with the
methods mark() and seek() and wrap the result of urlopen.


----------------------------------------------------------------------

Comment By: Lucas Malor (lucas_malor)
Date: 2007-04-26 22:41

Message:
Logged In: YES 
user_id=1403274
Originator: YES

In my opinion it's not complicated, it's convoluted. I must use two
object
to handle one data stream.

Furthermore it's a waste of resources. I must copy data to another
object.
Luckily in my script I download and handle only little files. But what if
a
python program must handle big files?

If seek() can't be used (an except is raised), urllib could use a
sequential access method.

----------------------------------------------------------------------

Comment By: Calvin Spealman (ironfroggy)
Date: 2007-04-26 15:55

Message:
Logged In: YES 
user_id=112166
Originator: NO

I have to agree that this is not a bug. HTTP responses are strings, not
random access files. Adding a seek would have disastrous performance
penalties. If you think the work around is too complicated, I can't
understand why.

----------------------------------------------------------------------

Comment By: Zacherates (maenpaa)
Date: 2007-03-21 02:39

Message:
Logged In: YES 
user_id=1421845
Originator: NO

> I use the method you wrote, but this must be done manually,
> and I don't know why.
read() is a stream processing method, whereas seek() is a random access
processing method.  HTTP resources are in essence streams so they implement
read() but not seek().  Trying to shoehorn a stream to act like a random
access file has some rather important technical implications.  For example:
what happens when an HTTP resource is larger than available memory and we
try to maintain a full featured seek() implementation?

> so what is urlopen() for?
Fetching a webpage or RSS feed and feeding it to a parser, for example.

StringIO is a class that was designed to implement feature complete,
random access, file-like object behavior that can be wrapped around a
stream.  StringIO can and should be used as an adapter for when you have a
stream that you need random access to.  This allows designers the freedom
to simply implement a good read() implementation and let clients wrap the
output in a StringIO if needed.

If in your application you always want random access and you don't have to
deal with large files:
def my_urlopen(*args, **kwargs):
   return StringIO.StringIO(urllib2.urlopen(*args, **kwargs).read())

Python makes delegation trivially easy.

In essence, urlfiles (the result of urllib2.urlopen()) and regular files
(the result of open()) behave differently because they implement different
interfaces.  If you use the common interface (read), then you can treat
them equally.  If you use the specialized interface (seek, tell, etc.)
you'll have trouble.  The solution is wrap the general objects in a
specialized object that implements the desired interface, StringIO.

----------------------------------------------------------------------

Comment By: Lucas Malor (lucas_malor)
Date: 2007-03-20 09:59

Message:
Logged In: YES 
user_id=1403274
Originator: YES

> If you need to seek, you can wrap the file-like object in a
> StringIO (which is what urllib would have to do internally
> [...] )

I think it's really a bug, or at least a non-pythonic method.
I use the method you wrote, but this must be done manually,
and I don't know why. Actually without this "trick" you can't
handle url and file objects together as they don't work in
the same manner. I think it's not too complicated using the
internal StringIO object in urllib class when I must seek()
or use other file-like methods.

> You can check the type of the response content before you try
> to uncompress it via the Content-Encoding header of the
> response

It's not a generic solution

(thanks anyway for suggested solutions :) )

----------------------------------------------------------------------

Comment By: Zacherates (maenpaa)
Date: 2007-03-20 03:43

Message:
Logged In: YES 
user_id=1421845
Originator: NO

I'd contend that this is not a bug:
 * If you need to seek, you can wrap the file-like object in a StringIO
(which is what urllib would have to do internally, thus incurring the
StringIO overhead for all clients, even those that don't need the
functionality).
 * You can check the type of the response content before you try to
uncompress it via the Content-Encoding header of the response.  The
meta-data is there for a reason.

Check
http://www.diveintopython.org/http_web_services/gzip_compression.html for a
rather complete treatment of your use-case.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1682241&group_id=5470

From noreply at sourceforge.net  Fri Apr 27 13:35:43 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Fri, 27 Apr 2007 04:35:43 -0700
Subject: [ python-Bugs-1708652 ] Exact matching
Message-ID: <E1HhOjn-00073T-Ne@sc8-sf-web23.sourceforge.net>

Bugs item #1708652, was opened at 2007-04-27 11:35
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1708652&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Regular Expressions
Group: Feature Request
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Tom Lynn (tlynn)
Assigned to: Gustavo Niemeyer (niemeyer)
Summary: Exact matching

Initial Comment:
I'd like to see a regexp.exact() method on regexp objects, equivalent to regexp.search(r'\A%s\Z' % pattern, ...), for parsing binary formats.

It's probably not worth disturbing the current library interface for, but maybe in Py3k?


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1708652&group_id=5470

From noreply at sourceforge.net  Fri Apr 27 13:57:44 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Fri, 27 Apr 2007 04:57:44 -0700
Subject: [ python-Bugs-1682241 ] Problems with urllib2 read()
Message-ID: <E1HhP56-0003AX-O5@sc8-sf-web24.sourceforge.net>

Bugs item #1682241, was opened at 2007-03-16 12:00
Message generated for change (Comment added) made by maenpaa
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1682241&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Lucas Malor (lucas_malor)
Assigned to: Nobody/Anonymous (nobody)
Summary: Problems with urllib2 read()

Initial Comment:
urllib2 objects opened with urlopen() does not have the method seek() as file objects. So reading only some bytes from opened urls is pratically forbidden.

An example: I tried to open an url and check if it's a gzip file. If IOError is raised I read the file (to do this I applied the #1675951 patch: https://sourceforge.net/tracker/index.php?func=detail&aid=1675951&group_id=5470&atid=305470 )

But after I tried to open the file as gzip, if it's not a gzip file the current position in the urllib object is on the second byte. So read() returns the data from the 3rd to the last byte. You can't check the header of the file before storing it on hd. Well, so what is urlopen() for? If I must store the file by url on hd and reload it, I can use urlretrieve() ...

----------------------------------------------------------------------

Comment By: Zacherates (maenpaa)
Date: 2007-04-27 07:57

Message:
Logged In: YES 
user_id=1421845
Originator: NO

> import urllib

> urlobj = urllib.urlopen("someurl")
> header = urlobj.read(1)
> # some other operations (no other urlobj.read())

> contents = header + urlobj.read()

This is effectively buffering the output, which is a perfectly acceptable
solution...  although I'd write like this:

import urllib

class BufferedReader(object):
   def __init__(self, fileobj, buffsize = 8192):
   
   def mark(self, maxbytes = 8192):

   def seek(self):

br = BufferedReader(urllib.urlopen())
br.mark()
header = br.read(1)

br.seek()
contents = br.read()

That way you store all bytes that have been read.  Rather than hoping
nobody calls read().


> On the contrary I'm pretty sure using a sequential access this can be
done
> without doing these workarounds.

Right now sequential access is provided without keeping a copy in memory. 
The issue arises when you want random access, however; urlobjs have no
indication as to whether you're going to call seek().  As such, to provide
the method they must assume you will call it.  Therefore, regardless of
whether seek() is actually called or not, a copy must be kept to offer the
*possibility* that it can be called.

You can work around this by offering the degenerate seek() provided by
BufferedReader, but that's functionality that belongs in it's own class
anyway.


> anyway I don't understand a thing: HTTP can't delegate the server to
> seek() the file?

For one thing, its not supported by the standard.  For another, it would
be a waste of server resources, bandwidth and to top it off it would be
really slow... even slower than using StringIO.  HTTP resources are not
simply files served up by httpd, they can also be dynamically generated
content... How is an HTTP server supposed to seek backward and forward in a
page that is programatically generated? Go try an tell web developers that
they need to keep a copy of every page requested indefinitely, in case you
want to send a SEEK request.




HTTP resources are not local.  To treat them as local you must make then
local by putting them in a container, such as StringIO, a buffer or a local
file. It's that simple.  To try and abstract this fact would result in
major performance issues or unreliability, or both.

----------------------------------------------------------------------

Comment By: Lucas Malor (lucas_malor)
Date: 2007-04-27 05:26

Message:
Logged In: YES 
user_id=1403274
Originator: YES

If you don't want the intere file on your memory, there's two solutions:

----------
import urllib

urlobj = urllib.urlopen("someurl")
header = urlobj.read(1)
# some other operations (no other urlobj.read())

contents = header + urlobj.read()

----------

I don't think it's a --good-- solution, because some other programmers can
do other read() operations and mess all the result.

The other solution is:

----------
def readall(x) :
  url = x.geturl
  x.close()
  try :
    y = urllib.openurl(url)
  except :
    return None
  return y.read()
  

import urllib

urlobj = urllib.urlopen("someurl")
header = urlobj.read(1)
# some other operations

contents = readall(urlobj)

----------

This is still a bad solution (two calls to the server for the same file).

On the contrary I'm pretty sure using a sequencial access this can be done
without doing these workarounds.

(anyway I don't understand a thing: HTTP can't delegate the server to
seek() the file?)

----------------------------------------------------------------------

Comment By: Zacherates (maenpaa)
Date: 2007-04-26 23:36

Message:
Logged In: YES 
user_id=1421845
Originator: NO

> In my opinion it's not complicated, it's convoluted. I must use two
> object to handle one data stream.

seek() is not a stream operation. It is a random access operation
(file-like != stream). If you were only trying to use stream operations
then you wouldn't have these problems.   

Each class provides a separate functionality, urllib gets the file while
StringIO stores it.  The fact that these responsibilities are given to
different classes should not be surprising since the represent separately
useful concepts that abstract different things.  It's not convoluted, it's
good design.  If every class tried to do everything, pretty soon you're
adding solve_my_business_problem_using_SOA() to __builtins__ and nobody
wants that.


> Furthermore it's a waste of resources. I must copy data to another
> object. Luckily in my script I download and handle only little files.
But what if
> a python program must handle big files?

This is exactly why urllib *doesn't* provide seek. Deep down in the
networking library there's a socket with a 8KiB buffer talking to the HTTP
server. No matter how big the file you're getting with urllib, once that
buffer is full the socket starts dropping packets. 

To provide seek(), urllib would need to keep an entire copy of the file
that was retrieved, (or provide mark()/seek(), but those have wildly
different semantics from the seek()s were used to in python, and besides
they're too Java).  This works fine if you're only working with small
files, but you raise a good point: "But what if a python program must
handle big files?".  What about really big files (say a Knoppix DVD ISO)? 
Sure you could use urlretrieve, but what if urlretrive is implemented in
terms of urlopen?

Sure urllib could implement seek (with the same semantics as file.seek())
but that would mean breaking urllib for any resource big enough that you
don't want the whole thing in memory.


>> You can check the type of the response content before you try
>> to uncompress it via the Content-Encoding header of the
>> response

>It's not a generic solution

The point of this suggestion is not that this is the be all and end all
solution, but that code that *needs* seek can probably be rewritten so that
it does not.  Either that or you could implement BufferedReader with the
methods mark() and seek() and wrap the result of urlopen.


----------------------------------------------------------------------

Comment By: Lucas Malor (lucas_malor)
Date: 2007-04-26 16:41

Message:
Logged In: YES 
user_id=1403274
Originator: YES

In my opinion it's not complicated, it's convoluted. I must use two
object
to handle one data stream.

Furthermore it's a waste of resources. I must copy data to another
object.
Luckily in my script I download and handle only little files. But what if
a
python program must handle big files?

If seek() can't be used (an except is raised), urllib could use a
sequential access method.

----------------------------------------------------------------------

Comment By: Calvin Spealman (ironfroggy)
Date: 2007-04-26 09:55

Message:
Logged In: YES 
user_id=112166
Originator: NO

I have to agree that this is not a bug. HTTP responses are strings, not
random access files. Adding a seek would have disastrous performance
penalties. If you think the work around is too complicated, I can't
understand why.

----------------------------------------------------------------------

Comment By: Zacherates (maenpaa)
Date: 2007-03-20 21:39

Message:
Logged In: YES 
user_id=1421845
Originator: NO

> I use the method you wrote, but this must be done manually,
> and I don't know why.
read() is a stream processing method, whereas seek() is a random access
processing method.  HTTP resources are in essence streams so they implement
read() but not seek().  Trying to shoehorn a stream to act like a random
access file has some rather important technical implications.  For example:
what happens when an HTTP resource is larger than available memory and we
try to maintain a full featured seek() implementation?

> so what is urlopen() for?
Fetching a webpage or RSS feed and feeding it to a parser, for example.

StringIO is a class that was designed to implement feature complete,
random access, file-like object behavior that can be wrapped around a
stream.  StringIO can and should be used as an adapter for when you have a
stream that you need random access to.  This allows designers the freedom
to simply implement a good read() implementation and let clients wrap the
output in a StringIO if needed.

If in your application you always want random access and you don't have to
deal with large files:
def my_urlopen(*args, **kwargs):
   return StringIO.StringIO(urllib2.urlopen(*args, **kwargs).read())

Python makes delegation trivially easy.

In essence, urlfiles (the result of urllib2.urlopen()) and regular files
(the result of open()) behave differently because they implement different
interfaces.  If you use the common interface (read), then you can treat
them equally.  If you use the specialized interface (seek, tell, etc.)
you'll have trouble.  The solution is wrap the general objects in a
specialized object that implements the desired interface, StringIO.

----------------------------------------------------------------------

Comment By: Lucas Malor (lucas_malor)
Date: 2007-03-20 04:59

Message:
Logged In: YES 
user_id=1403274
Originator: YES

> If you need to seek, you can wrap the file-like object in a
> StringIO (which is what urllib would have to do internally
> [...] )

I think it's really a bug, or at least a non-pythonic method.
I use the method you wrote, but this must be done manually,
and I don't know why. Actually without this "trick" you can't
handle url and file objects together as they don't work in
the same manner. I think it's not too complicated using the
internal StringIO object in urllib class when I must seek()
or use other file-like methods.

> You can check the type of the response content before you try
> to uncompress it via the Content-Encoding header of the
> response

It's not a generic solution

(thanks anyway for suggested solutions :) )

----------------------------------------------------------------------

Comment By: Zacherates (maenpaa)
Date: 2007-03-19 22:43

Message:
Logged In: YES 
user_id=1421845
Originator: NO

I'd contend that this is not a bug:
 * If you need to seek, you can wrap the file-like object in a StringIO
(which is what urllib would have to do internally, thus incurring the
StringIO overhead for all clients, even those that don't need the
functionality).
 * You can check the type of the response content before you try to
uncompress it via the Content-Encoding header of the response.  The
meta-data is there for a reason.

Check
http://www.diveintopython.org/http_web_services/gzip_compression.html for a
rather complete treatment of your use-case.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1682241&group_id=5470

From noreply at sourceforge.net  Fri Apr 27 15:26:33 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Fri, 27 Apr 2007 06:26:33 -0700
Subject: [ python-Bugs-1708710 ] Tutorial error
Message-ID: <E1HhQT3-000781-2O@sc8-sf-web24.sourceforge.net>

Bugs item #1708710, was opened at 2007-04-27 08:26
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1708710&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Documentation
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: john hite (jhite)
Assigned to: Nobody/Anonymous (nobody)
Summary: Tutorial error

Initial Comment:
Section 2.1 of the tutorial, http://docs.python.org/tut/node4.html ,

states

set path=%path%;C:\python24

Shouldn't it be python25?

I know it is just a nit, I expect most people would probably catch this if I managed to.


Best Regards,

John Hite

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1708710&group_id=5470

From noreply at sourceforge.net  Fri Apr 27 21:20:30 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Fri, 27 Apr 2007 12:20:30 -0700
Subject: [ python-Bugs-1708710 ] Tutorial error
Message-ID: <E1HhVza-0003Hw-GM@sc8-sf-web24.sourceforge.net>

Bugs item #1708710, was opened at 2007-04-27 13:26
Message generated for change (Comment added) made by gbrandl
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1708710&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Documentation
Group: None
>Status: Closed
>Resolution: Fixed
Priority: 5
Private: No
Submitted By: john hite (jhite)
Assigned to: Nobody/Anonymous (nobody)
Summary: Tutorial error

Initial Comment:
Section 2.1 of the tutorial, http://docs.python.org/tut/node4.html ,

states

set path=%path%;C:\python24

Shouldn't it be python25?

I know it is just a nit, I expect most people would probably catch this if I managed to.


Best Regards,

John Hite

----------------------------------------------------------------------

>Comment By: Georg Brandl (gbrandl)
Date: 2007-04-27 19:20

Message:
Logged In: YES 
user_id=849994
Originator: NO

Thanks, fixed in rev. 55001, 55002.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1708710&group_id=5470

From noreply at sourceforge.net  Fri Apr 27 23:41:17 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Fri, 27 Apr 2007 14:41:17 -0700
Subject: [ python-Bugs-1651995 ] sgmllib _convert_ref UnicodeDecodeError
	exception, new in 2.
Message-ID: <E1HhYBp-0004Db-G0@sc8-sf-web21.sourceforge.net>

Bugs item #1651995, was opened at 2007-02-04 22:34
Message generated for change (Comment added) made by nagle
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1651995&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: John Nagle (nagle)
Assigned to: Nobody/Anonymous (nobody)
Summary: sgmllib _convert_ref UnicodeDecodeError exception, new in 2.

Initial Comment:
   I'm running a website page through BeautifulSoup.  It parses OK with Python 2.4, but Python 2.5 fails with an exception:

Traceback (most recent call last):
  File "./sitetruth/InfoSitePage.py", line 268, in httpfetch
    self.pagetree = BeautifulSoup.BeautifulSoup(sitetext) # parse into tree form
  File "./sitetruth/BeautifulSoup.py", line 1326, in __init__
    BeautifulStoneSoup.__init__(self, *args, **kwargs)
  File "./sitetruth/BeautifulSoup.py", line 973, in __init__
    self._feed()
  File "./sitetruth/BeautifulSoup.py", line 998, in _feed
    SGMLParser.feed(self, markup or "")
  File "/usr/lib/python2.5/sgmllib.py", line 99, in feed
    self.goahead(0)
  File "/usr/lib/python2.5/sgmllib.py", line 133, in goahead
    k = self.parse_starttag(i)
  File "/usr/lib/python2.5/sgmllib.py", line 291, in parse_starttag
    self.finish_starttag(tag, attrs)
  File "/usr/lib/python2.5/sgmllib.py", line 340, in finish_starttag
    self.handle_starttag(tag, method, attrs)
  File "/usr/lib/python2.5/sgmllib.py", line 376, in handle_starttag
    method(attrs)
  File "./sitetruth/BeautifulSoup.py", line 1416, in start_meta
    self._feed(self.declaredHTMLEncoding)
  File "./sitetruth/BeautifulSoup.py", line 998, in _feed
    SGMLParser.feed(self, markup or "")
  File "/usr/lib/python2.5/sgmllib.py", line 99, in feed
    self.goahead(0)
  File "/usr/lib/python2.5/sgmllib.py", line 133, in goahead
    k = self.parse_starttag(i)
  File "/usr/lib/python2.5/sgmllib.py", line 285, in parse_starttag
    self._convert_ref, attrvalue)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xa7 in position 0: ordinal not in range(128)

    The code that's failing is in "_convert_ref", which is new in Python 2.5. That function wasn't present in 2.4.  I think the code is trying to handle single quotes inside of double quotes in HTML attributes, or something like that.

    To replicate, run

	http://www.bankofamerica.com
or
	http://www.gm.com

through BeautifulSoup.  

Something about this code doesn't like big companies. Web sites of smaller companies are going through OK.

----------------------------------------------------------------------

>Comment By: John Nagle (nagle)
Date: 2007-04-27 21:41

Message:
Logged In: YES 
user_id=5571
Originator: YES

We've been running this fix for several months now, and it seems to work. 
Would someone please check it and put it into the trunk?  Thanks.

----------------------------------------------------------------------

Comment By: John Nagle (nagle)
Date: 2007-02-07 07:57

Message:
Logged In: YES 
user_id=5571
Originator: YES

Found the problem. In sgmllib.py for Python 2.5, in convert_charref, the
code for handling character escapes assumes that ASCII characters have
values up to 255.
But the correct limit is 127, of course.

If a Unicode string is run through SGMLparser, and that string has a
character in an attribute with a value between 128 and 255, which is valid
in Unicode, the
value is passed through as a character with "chr", creating a
one-character invalid ASCII string.  

Then, when the bad string is later converted to Unicode as the output is
assembled, the UnicodeDecodeError exception is raised. 

So the fix is to change 255 to 127 in convert_charref in sgmllib.py,
as shown below.  This forces characters above 127 to be expressed with
escape sequences.  Please patch accordingly.  Thanks.

def convert_charref(self, name):
    """Convert character reference, may be overridden."""
    try:
        n = int(name)
    except ValueError:
        return
    if not 0 <= n <= 127 : # ASCII ends at 127, not 255
        return
    return self.convert_codepoint(n)


----------------------------------------------------------------------

Comment By: wrstl prmpft (wrstlprmpft)
Date: 2007-02-05 07:16

Message:
Logged In: YES 
user_id=801589
Originator: NO

I had a similar problem recently and did not have time to file a
bug-report. Thanks for doing that.

The problem is the code that handles entity and character references in
SGMLParser.parse_starttag. Seems that it is not careful about unicode/str
issues.
(But maybe Beautifulsoup needs to tell it to?)

My quick'n'dirty workaround was to remove the offending char-entity from
the website before feeding it to Beautifulsoup::

  text = text.replace('&#174;', '') # remove rights reserved sign entity

cheers,
stefan


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1651995&group_id=5470

From noreply at sourceforge.net  Sat Apr 28 03:40:58 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Fri, 27 Apr 2007 18:40:58 -0700
Subject: [ python-Bugs-1708538 ] TimedRotatingFileHandler's doRollover opens
	file in "w" mode
Message-ID: <E1Hhbvm-0006qF-2b@sc8-sf-web23.sourceforge.net>

Bugs item #1708538, was opened at 2007-04-27 00:30
Message generated for change (Comment added) made by nnorwitz
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1708538&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Florian Schmidt (fastflo)
>Assigned to: Vinay Sajip (vsajip)
Summary: TimedRotatingFileHandler's doRollover opens file in "w" mode

Initial Comment:
if TimedRotatingFileHandler does an doRollover() it opens the new file hardcoded with mode "w" whilst the baseclass FileHandler opens it with mode "a" if nothing else is specified in the constructor. it should use the same default behavior as FileHandler and/or give a way to specify another open-mode.

if multiple processes write to the same file, and one does the RollOver all other newly spawned processes write to the end of the new file but the still running first process who did the rollover will write from the beginning of the file - overwriting log entries from the other procs...

----------------------------------------------------------------------

>Comment By: Neal Norwitz (nnorwitz)
Date: 2007-04-27 18:40

Message:
Logged In: YES 
user_id=33168
Originator: NO

This sounds like a logging module issue.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1708538&group_id=5470

From noreply at sourceforge.net  Sat Apr 28 14:50:28 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Sat, 28 Apr 2007 05:50:28 -0700
Subject: [ python-Bugs-1708538 ] TimedRotatingFileHandler's doRollover opens
	file in "w" mode
Message-ID: <E1HhmNg-00057w-Bb@sc8-sf-web24.sourceforge.net>

Bugs item #1708538, was opened at 2007-04-27 07:30
Message generated for change (Comment added) made by vsajip
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1708538&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.5
>Status: Closed
>Resolution: Invalid
Priority: 5
Private: No
Submitted By: Florian Schmidt (fastflo)
Assigned to: Vinay Sajip (vsajip)
Summary: TimedRotatingFileHandler's doRollover opens file in "w" mode

Initial Comment:
if TimedRotatingFileHandler does an doRollover() it opens the new file hardcoded with mode "w" whilst the baseclass FileHandler opens it with mode "a" if nothing else is specified in the constructor. it should use the same default behavior as FileHandler and/or give a way to specify another open-mode.

if multiple processes write to the same file, and one does the RollOver all other newly spawned processes write to the end of the new file but the still running first process who did the rollover will write from the beginning of the file - overwriting log entries from the other procs...

----------------------------------------------------------------------

>Comment By: Vinay Sajip (vsajip)
Date: 2007-04-28 12:50

Message:
Logged In: YES 
user_id=308438
Originator: NO

Since there is no interprocess synchronisation in Python, writing to the
same logging file from multiple processes is not supported. If you want
output from multiple processes to end up in the same file, have all your
processes send logging events to a SocketHandler, and set up a separate
socket receiver (which could be in one of the processes) to receive the
events and log to file. A working example is given here:

http://docs.python.org/lib/network-logging.html

Note that even if the system was changed to use 'a' rather than 'w', this
would not eliminate potential problems with e.g. data in unflushed buffers,
etc.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2007-04-28 01:40

Message:
Logged In: YES 
user_id=33168
Originator: NO

This sounds like a logging module issue.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1708538&group_id=5470

From noreply at sourceforge.net  Sat Apr 28 16:52:04 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Sat, 28 Apr 2007 07:52:04 -0700
Subject: [ python-Bugs-1709282 ] test_1686475 fails when pagefile.sys does not
	exist
Message-ID: <E1HhoHM-0006EQ-5f@sc8-sf-web23.sourceforge.net>

Bugs item #1709282, was opened at 2007-04-28 10:52
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1709282&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Windows
Group: Platform-specific
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Calvin Spealman (ironfroggy)
Assigned to: Nobody/Anonymous (nobody)
Summary: test_1686475 fails when pagefile.sys does not exist

Initial Comment:
In older versions of Windows I suppose the file is reliable, but apparently no longer. This patch fixes it, by creating a temporary file as a test fixture, instead of relying on an operating system file.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1709282&group_id=5470

From noreply at sourceforge.net  Sat Apr 28 16:54:52 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Sat, 28 Apr 2007 07:54:52 -0700
Subject: [ python-Bugs-1709284 ] test_1686475 fails because pagefile.sys does
	not exist
Message-ID: <E1HhoK4-0005qf-Q5@sc8-sf-web22.sourceforge.net>

Bugs item #1709284, was opened at 2007-04-28 10:54
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1709284&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Windows
Group: Platform-specific
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Calvin Spealman (ironfroggy)
Assigned to: Nobody/Anonymous (nobody)
Summary: test_1686475 fails because pagefile.sys does not exist

Initial Comment:
The file pagefile.sys is used as a test on stat'ing an open file, because previously we could rely on windows using it. It does not exist in newer versions, so the test fails. The attached patch uses a temporary file instead.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1709284&group_id=5470

From noreply at sourceforge.net  Sat Apr 28 16:56:24 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Sat, 28 Apr 2007 07:56:24 -0700
Subject: [ python-Bugs-1709282 ] test_1686475 fails when pagefile.sys does not
	exist
Message-ID: <E1HhoLY-000641-Ek@sc8-sf-web22.sourceforge.net>

Bugs item #1709282, was opened at 2007-04-28 10:52
Message generated for change (Settings changed) made by ironfroggy
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1709282&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Windows
Group: Platform-specific
Status: Open
>Resolution: Duplicate
Priority: 5
Private: No
Submitted By: Calvin Spealman (ironfroggy)
Assigned to: Nobody/Anonymous (nobody)
Summary: test_1686475 fails when pagefile.sys does not exist

Initial Comment:
In older versions of Windows I suppose the file is reliable, but apparently no longer. This patch fixes it, by creating a temporary file as a test fixture, instead of relying on an operating system file.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1709282&group_id=5470

From noreply at sourceforge.net  Sat Apr 28 16:57:32 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Sat, 28 Apr 2007 07:57:32 -0700
Subject: [ python-Bugs-1709282 ] test_1686475 fails when pagefile.sys does not
	exist
Message-ID: <E1HhoMe-0006Aq-AH@sc8-sf-web22.sourceforge.net>

Bugs item #1709282, was opened at 2007-04-28 10:52
Message generated for change (Settings changed) made by ironfroggy
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1709282&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Windows
Group: Platform-specific
>Status: Deleted
Resolution: Duplicate
Priority: 5
Private: No
Submitted By: Calvin Spealman (ironfroggy)
Assigned to: Nobody/Anonymous (nobody)
Summary: test_1686475 fails when pagefile.sys does not exist

Initial Comment:
In older versions of Windows I suppose the file is reliable, but apparently no longer. This patch fixes it, by creating a temporary file as a test fixture, instead of relying on an operating system file.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1709282&group_id=5470

From noreply at sourceforge.net  Sat Apr 28 16:59:56 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Sat, 28 Apr 2007 07:59:56 -0700
Subject: [ python-Bugs-1707255 ] lost global variables in main memory
	intensive execution
Message-ID: <E1HhoOy-0006On-KW@sc8-sf-web22.sourceforge.net>

Bugs item #1707255, was opened at 2007-04-25 05:45
Message generated for change (Comment added) made by ironfroggy
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1707255&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Parser/Compiler
Group: Python 2.4
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Jordi Pujol Ahull? (jpahullo)
Assigned to: Nobody/Anonymous (nobody)
Summary: lost global variables in main memory intensive execution

Initial Comment:
Hello,

I was running a very main memory intensive program in my computer. I looked for similar bugs or troubles in the site and I didn't found any similar to this. I know that the use of global variables is not recommended by software engineering, but for special cases it is very fast to develop/test some ideas.

My problem statement is the following (very simplified and also attached):

########## BEGINNING OF CODE
#test for globals

counter_1 = 0

def modifierfunction():
    global counter_1

    #some code
    counter_1 += 1
    #some other code

def test():
    for i in range(2):
        global counter_1
        counter_1 = 0

        for j in range(10):
            modifierfunction()

        print "COUNTER_1:", counter_1

def test2():
    global counter_1
    for i in range(2):
        counter_1 = 0

        for j in range(10):
            modifierfunction()

        print "COUNTER_1:", counter_1

if __name__ == "__main__":
    test()
    test2()
########## END OF CODE

Globally speaking, it is a global variable, defined at the begining of the python file (counter_1), and it is modified in some functions within it (in this example, modifierfunction). At the end, it appear some tests that make what I need. 

If you try to run this code, it will always show the expected values in the standard out. But, let me to show you my problem.

In the beginning, I have the global statement as in test2. But I found that it only take a corrent value for the first iteration. The others it has always a zero value. I didn't understand anything.

Then, some collegue suggested me to change the place of the global statement (as in test()), and then it worked correctly, as it was expected.

I repeat. The above example works fine. But when this same structure, but with my big problem, very main memory intensive, test2() didn't work correctly.

Thank you for your attention.

Regards,

Jordi

----------------------------------------------------------------------

Comment By: Calvin Spealman (ironfroggy)
Date: 2007-04-28 10:59

Message:
Logged In: YES 
user_id=112166
Originator: NO

If you really think there is a bug, don't post working code, post code
that actually demonstrates the problem. Something anyone else could use to
reproduce the bug.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1707255&group_id=5470

From noreply at sourceforge.net  Sat Apr 28 20:32:50 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Sat, 28 Apr 2007 11:32:50 -0700
Subject: [ python-Bugs-1709284 ] test_1686475 fails because pagefile.sys does
	not exist
Message-ID: <E1Hhrj0-0004iL-NG@sc8-sf-web23.sourceforge.net>

Bugs item #1709284, was opened at 2007-04-28 10:54
Message generated for change (Comment added) made by ironfroggy
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1709284&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Windows
Group: Platform-specific
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Calvin Spealman (ironfroggy)
Assigned to: Nobody/Anonymous (nobody)
Summary: test_1686475 fails because pagefile.sys does not exist

Initial Comment:
The file pagefile.sys is used as a test on stat'ing an open file, because previously we could rely on windows using it. It does not exist in newer versions, so the test fails. The attached patch uses a temporary file instead.

----------------------------------------------------------------------

>Comment By: Calvin Spealman (ironfroggy)
Date: 2007-04-28 14:32

Message:
Logged In: YES 
user_id=112166
Originator: YES

File Added: os_open_stat.patch

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1709284&group_id=5470

From noreply at sourceforge.net  Sat Apr 28 22:27:21 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Sat, 28 Apr 2007 13:27:21 -0700
Subject: [ python-Bugs-1707768 ] os.path.normpath changes path (chops of
	trailing slash)
Message-ID: <E1HhtVp-0007fe-Ji@sc8-sf-web24.sourceforge.net>

Bugs item #1707768, was opened at 2007-04-26 01:44
Message generated for change (Comment added) made by iszegedi
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1707768&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Robert Siemer (siemer)
Assigned to: Nobody/Anonymous (nobody)
Summary: os.path.normpath changes path (chops of trailing slash)

Initial Comment:
Hello everybody!

>>> os.path.normpath('/etc/passwd')
'/etc/passwd'


I don't know any environment at all where

a) '/etc/passwd/'
b) '/etc/passwd'

are treated the same. It clearly does not apply for the path part of http urls (this is left as an exercise for the reader).

But it also does not apply for (e.g.) Linux either:
an open() on path a) return ENOTDIR while it succeeds with b).

(assuming /etc/passwd is a file)

This is definitively not a documentation bug, as "normpath" should normalize a path and not fuck it up.


Robert

----------------------------------------------------------------------

Comment By: Istvan Szegedi (iszegedi)
Date: 2007-04-28 22:27

Message:
Logged In: YES 
user_id=1772412
Originator: NO


Here is what Posix standard says about pathnames:

"Base Definitions volume of IEEE Std 1003.1-2001, Section 3.266,
Pathname.

A character string that is used to identify a file. In the context of IEEE
Std 1003.1-2001, a pathname consists of, at most, {PATH_MAX} bytes,
including the terminating null byte. It has an optional beginning slash,
followed by zero or more filenames separated by slashes. A pathname may
optionally contain one or more trailing slashes. Multiple successive
slashes are considered to be the same as one slash."

And in the details:

"A pathname that contains at least one non-slash character and that ends
with one or more trailing slashes shall be resolved as if a single dot
character ( '.' ) were appended to the pathname."

So if I am not mistaken, according to the POSIX standard the example that
you gave - '/etc/passwd/' - should be normalized to '/etc/passwd/.' That
does not happen, indeed.

The reason for that is that in posixpath.py file the normpath() function
is using a split('/') function to split up the path into smaller chunks,
skips everything which is empty or '.' and at the end of the normpath()
function it adds slash(es) only to the beginning of the string. 

As a test, I modified the normpath() function in the posixpath.py as
follows:

--- clip ---

def normpath(path):
    """Normalize path, eliminating double slashes, etc."""
    if path == '':
        return '.'
    initial_slashes = path.startswith('/')
    # The next two lines were added by iszegedi
    path = path.rstrip()
    trailing_slash = path.endswith('/')
    # POSIX allows one or two initial slashes, but treats three or more
    # as single slash.
    if (initial_slashes and
        path.startswith('//') and not path.startswith('///')):
        initial_slashes = 2
    comps = path.split('/')
    new_comps = []
    for comp in comps:
        if comp in ('', '.'):
            continue
        if (comp != '..' or (not initial_slashes and not new_comps) or
             (new_comps and new_comps[-1] == '..')):
            new_comps.append(comp)
        elif new_comps:
            new_comps.pop()
    comps = new_comps
    path = '/'.join(comps)
    if initial_slashes:
        path = '/'*initial_slashes + path
    # The next two lines were added by iszegedi
    if trailing_slash:
        path = path + '/.'
    return path or '.'
  
-- clip --

So I added two lines (marked as "added by iszegedi" ) in the beginning to
remove any trailing whitespaces and check whether the path ends with slash.
Then at the end of the function I added another two lines to append '/.' to
the end of the return value if the input path variable ended by slash

This works now fine.

What makes it a bit tricky is that python os module imports different
xxxpath.py module depending on the host operating system. So it imports
different modules for nt, for mac, for os2, for posix, etc.  The solution
above works for posix, but the other modules need to be checked, to.


----------------------------------------------------------------------

Comment By: Robert Siemer (siemer)
Date: 2007-04-26 01:47

Message:
Logged In: YES 
user_id=150699
Originator: YES

A bugreport bug:

The example should read os.path.normpath('/etc/passwd/')...

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1707768&group_id=5470

From noreply at sourceforge.net  Sun Apr 29 09:21:27 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Sun, 29 Apr 2007 00:21:27 -0700
Subject: [ python-Bugs-1709506 ] struct.calcsize() incorrect
Message-ID: <E1Hi3ip-000737-29@sc8-sf-web21.sourceforge.net>

Bugs item #1709506, was opened at 2007-04-29 00:21
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1709506&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: JoelBondurant (joelbondurant)
Assigned to: Nobody/Anonymous (nobody)
Summary: struct.calcsize() incorrect

Initial Comment:
The result of:

import struct
struct.calcsize('cd') #returns 16

appears to be incorrect.  The correct result should be 1 + 8 = 9.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1709506&group_id=5470

From noreply at sourceforge.net  Sun Apr 29 12:38:20 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Sun, 29 Apr 2007 03:38:20 -0700
Subject: [ python-Bugs-1709506 ] struct.calcsize() incorrect
Message-ID: <E1Hi6nM-0003JV-3i@sc8-sf-web21.sourceforge.net>

Bugs item #1709506, was opened at 2007-04-29 09:21
Message generated for change (Comment added) made by loewis
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1709506&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.5
>Status: Closed
>Resolution: Invalid
Priority: 5
Private: No
Submitted By: JoelBondurant (joelbondurant)
Assigned to: Nobody/Anonymous (nobody)
Summary: struct.calcsize() incorrect

Initial Comment:
The result of:

import struct
struct.calcsize('cd') #returns 16

appears to be incorrect.  The correct result should be 1 + 8 = 9.

----------------------------------------------------------------------

>Comment By: Martin v. L?wis (loewis)
Date: 2007-04-29 12:38

Message:
Logged In: YES 
user_id=21627
Originator: NO

No, the result is correct as it stands. If you define

struct Foo{
  char c;
  double d;
};

on your system, and then compute sizeof(struct Foo) using the C compiler,
it will tell you that the struct is 16 bytes in size. Read some text about
alignment and padding.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1709506&group_id=5470

From noreply at sourceforge.net  Sun Apr 29 17:29:59 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Sun, 29 Apr 2007 08:29:59 -0700
Subject: [ python-Bugs-1708538 ] TimedRotatingFileHandler's doRollover opens
	file in "w" mode
Message-ID: <E1HiBLb-0003Q2-PX@sc8-sf-web23.sourceforge.net>

Bugs item #1708538, was opened at 2007-04-27 09:30
Message generated for change (Comment added) made by fastflo
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1708538&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.5
Status: Closed
Resolution: Invalid
Priority: 5
Private: No
Submitted By: Florian Schmidt (fastflo)
Assigned to: Vinay Sajip (vsajip)
Summary: TimedRotatingFileHandler's doRollover opens file in "w" mode

Initial Comment:
if TimedRotatingFileHandler does an doRollover() it opens the new file hardcoded with mode "w" whilst the baseclass FileHandler opens it with mode "a" if nothing else is specified in the constructor. it should use the same default behavior as FileHandler and/or give a way to specify another open-mode.

if multiple processes write to the same file, and one does the RollOver all other newly spawned processes write to the end of the new file but the still running first process who did the rollover will write from the beginning of the file - overwriting log entries from the other procs...

----------------------------------------------------------------------

>Comment By: Florian Schmidt (fastflo)
Date: 2007-04-29 17:29

Message:
Logged In: YES 
user_id=182876
Originator: YES

i think opening the log file with "w" is inconsistent and therefore should
be fixed.

as a comment: i do not need IPC/synchronisation if all processes are
writing in append-mode to the same file. i am aware that sometimes log
messages could get mixed up, but no single one will get completely lost
(just eventually scattert if its a really big one).

for now we overloaded that function in our project. thanks!

----------------------------------------------------------------------

Comment By: Vinay Sajip (vsajip)
Date: 2007-04-28 14:50

Message:
Logged In: YES 
user_id=308438
Originator: NO

Since there is no interprocess synchronisation in Python, writing to the
same logging file from multiple processes is not supported. If you want
output from multiple processes to end up in the same file, have all your
processes send logging events to a SocketHandler, and set up a separate
socket receiver (which could be in one of the processes) to receive the
events and log to file. A working example is given here:

http://docs.python.org/lib/network-logging.html

Note that even if the system was changed to use 'a' rather than 'w', this
would not eliminate potential problems with e.g. data in unflushed buffers,
etc.

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2007-04-28 03:40

Message:
Logged In: YES 
user_id=33168
Originator: NO

This sounds like a logging module issue.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1708538&group_id=5470

From noreply at sourceforge.net  Mon Apr 30 03:54:06 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Sun, 29 Apr 2007 18:54:06 -0700
Subject: [ python-Bugs-1686475 ] os.stat() WindowsError 13 when file in use
Message-ID: <E1HiL5a-00035F-Gu@sc8-sf-web24.sourceforge.net>

Bugs item #1686475, was opened at 2007-03-23 02:31
Message generated for change (Comment added) made by abkhd
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1686475&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.5
Status: Closed
Resolution: Fixed
Priority: 5
Private: No
Submitted By: d roberts (dcroberts)
Assigned to: Martin v. L?wis (loewis)
Summary: os.stat() WindowsError 13 when file in use

Initial Comment:
Using 2.5 Sept 19, 2006 on Windows XP, 32-bit.

os.stat() on an "in use" file (i.e., open by another process) erroneously returns WindowsError [Error 13]. The file stats should be available on an open file, even if you cannot read/write the file itself.

Python 2.4 correctly returns the stats on the file.

----------------
CORRECT in 2.4:
Python 2.4.4 (#71, Oct 18 2006, 08:34:43) [MSC v.1310 32 bit (Intel)] on win32
>>> import os
>>> os.stat('c:\\hiberfil.sys')
(33206, 0L, 2, 1, 0, 0, 804311040L, 1173962381, 1173962381, 1069302780)

-----------------
ERROR in 2.5:
Python 2.5 (r25:51908, Sep 19 2006, 09:52:17) [MSC v.1310 32 bit (Intel)] on win32
>>> import os
>>> os.stat("c:\\hiberfil.sys")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
WindowsError: [Error 13] The process cannot access the file because it is being
used by another process: 'c:\\hiberfil.sys'

----------------------------------------------------------------------

Comment By: A.B., Khalid (abkhd)
Date: 2007-04-30 01:54

Message:
Logged In: YES 
user_id=1079026
Originator: NO

Please see http://www.python.org/sf/1709112 for comment and patch.

----------------------------------------------------------------------

Comment By: Martin v. L?wis (loewis)
Date: 2007-04-04 18:31

Message:
Logged In: YES 
user_id=21627
Originator: NO

I have now committed this patch as r54685 and r54686.

----------------------------------------------------------------------

Comment By: Martin v. L?wis (loewis)
Date: 2007-03-30 06:29

Message:
Logged In: YES 
user_id=21627
Originator: NO

Attached is a patch that makes it fall back to FindFirstFile if
GetFileAttributesEx fail with ERROR_SHARING_VIOLATION.
File Added: stat.diff

----------------------------------------------------------------------

Comment By: d roberts (dcroberts)
Date: 2007-03-23 17:49

Message:
Logged In: YES 
user_id=1750755
Originator: YES

Taking your hint, I just looked at the code in 42230 which uses the Win32
GetFileAttributesEx() function. This function does indeed return an error
for an in-use file. Not sure if this is a "feature" or "bug" in the
underlying Windows function. I did try FindFirstFile(), which is able to
return info on an in-use file. Not sure if this is a reasonable alternative
to GFAE(), but wanted to make folks aware of the possibility. It's probably
slower, since it can handle wild-cards, and returns a handle for subsequent
calls to FindNextFile(). The handle must be closed with FindClose(). I'm
not sure what underlying mechanism FindFirstFile() uses, but I'm assuming
it accesses the "directory" to get its information, rather than trying to
access the file itself. FWIW.

----------------------------------------------------------------------

Comment By: Martin v. L?wis (loewis)
Date: 2007-03-23 10:24

Message:
Logged In: YES 
user_id=21627
Originator: NO

It certainly is; Python is likely opening the file in the wrong mode now.
I'll investigate, although contributions would be welcome.

----------------------------------------------------------------------

Comment By: Georg Brandl (gbrandl)
Date: 2007-03-23 10:15

Message:
Logged In: YES 
user_id=849994
Originator: NO

May this be related to rev. 42230 - "Drop C library for stat/fstat on
Windows." ?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1686475&group_id=5470

From noreply at sourceforge.net  Mon Apr 30 08:48:58 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Sun, 29 Apr 2007 23:48:58 -0700
Subject: [ python-Bugs-1707768 ] os.path.normpath changes path (chops of
	trailing slash)
Message-ID: <E1HiPgw-0000ie-1t@sc8-sf-web24.sourceforge.net>

Bugs item #1707768, was opened at 2007-04-25 23:44
Message generated for change (Comment added) made by josm
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1707768&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Robert Siemer (siemer)
Assigned to: Nobody/Anonymous (nobody)
Summary: os.path.normpath changes path (chops of trailing slash)

Initial Comment:
Hello everybody!

>>> os.path.normpath('/etc/passwd')
'/etc/passwd'


I don't know any environment at all where

a) '/etc/passwd/'
b) '/etc/passwd'

are treated the same. It clearly does not apply for the path part of http urls (this is left as an exercise for the reader).

But it also does not apply for (e.g.) Linux either:
an open() on path a) return ENOTDIR while it succeeds with b).

(assuming /etc/passwd is a file)

This is definitively not a documentation bug, as "normpath" should normalize a path and not fuck it up.


Robert

----------------------------------------------------------------------

Comment By: jos (josm)
Date: 2007-04-30 06:48

Message:
Logged In: YES 
user_id=1776568
Originator: NO

I think we should be careful enough to tackle on this.
iszegedi's patch seems to work correctly,
but XBD's spec itself has some defect.
http://www.opengroup.org/austin/mailarchives/ag-review/msg01722.html

What do you think of the follow behavior?
>>> os.mkdir('dir/')
>>> os.mkdir('dir2/')
>>> os.rmdir(os.path.normpath('dir'))
>>> os.rmdir(os.path.normpath('dir2/'))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OSError: [Errno 22] Invalid argument: 'dir2/.'




----------------------------------------------------------------------

Comment By: Istvan Szegedi (iszegedi)
Date: 2007-04-28 20:27

Message:
Logged In: YES 
user_id=1772412
Originator: NO


Here is what Posix standard says about pathnames:

"Base Definitions volume of IEEE Std 1003.1-2001, Section 3.266,
Pathname.

A character string that is used to identify a file. In the context of IEEE
Std 1003.1-2001, a pathname consists of, at most, {PATH_MAX} bytes,
including the terminating null byte. It has an optional beginning slash,
followed by zero or more filenames separated by slashes. A pathname may
optionally contain one or more trailing slashes. Multiple successive
slashes are considered to be the same as one slash."

And in the details:

"A pathname that contains at least one non-slash character and that ends
with one or more trailing slashes shall be resolved as if a single dot
character ( '.' ) were appended to the pathname."

So if I am not mistaken, according to the POSIX standard the example that
you gave - '/etc/passwd/' - should be normalized to '/etc/passwd/.' That
does not happen, indeed.

The reason for that is that in posixpath.py file the normpath() function
is using a split('/') function to split up the path into smaller chunks,
skips everything which is empty or '.' and at the end of the normpath()
function it adds slash(es) only to the beginning of the string. 

As a test, I modified the normpath() function in the posixpath.py as
follows:

--- clip ---

def normpath(path):
    """Normalize path, eliminating double slashes, etc."""
    if path == '':
        return '.'
    initial_slashes = path.startswith('/')
    # The next two lines were added by iszegedi
    path = path.rstrip()
    trailing_slash = path.endswith('/')
    # POSIX allows one or two initial slashes, but treats three or more
    # as single slash.
    if (initial_slashes and
        path.startswith('//') and not path.startswith('///')):
        initial_slashes = 2
    comps = path.split('/')
    new_comps = []
    for comp in comps:
        if comp in ('', '.'):
            continue
        if (comp != '..' or (not initial_slashes and not new_comps) or
             (new_comps and new_comps[-1] == '..')):
            new_comps.append(comp)
        elif new_comps:
            new_comps.pop()
    comps = new_comps
    path = '/'.join(comps)
    if initial_slashes:
        path = '/'*initial_slashes + path
    # The next two lines were added by iszegedi
    if trailing_slash:
        path = path + '/.'
    return path or '.'
  
-- clip --

So I added two lines (marked as "added by iszegedi" ) in the beginning to
remove any trailing whitespaces and check whether the path ends with slash.
Then at the end of the function I added another two lines to append '/.' to
the end of the return value if the input path variable ended by slash

This works now fine.

What makes it a bit tricky is that python os module imports different
xxxpath.py module depending on the host operating system. So it imports
different modules for nt, for mac, for os2, for posix, etc.  The solution
above works for posix, but the other modules need to be checked, to.


----------------------------------------------------------------------

Comment By: Robert Siemer (siemer)
Date: 2007-04-25 23:47

Message:
Logged In: YES 
user_id=150699
Originator: YES

A bugreport bug:

The example should read os.path.normpath('/etc/passwd/')...

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1707768&group_id=5470

From noreply at sourceforge.net  Mon Apr 30 23:52:17 2007
From: noreply at sourceforge.net (SourceForge.net)
Date: Mon, 30 Apr 2007 14:52:17 -0700
Subject: [ python-Bugs-1710295 ] Tutorial - Section 8.3 - <type 'instance'>
Message-ID: <E1Hidn7-0001Mn-Mg@sc8-sf-web24.sourceforge.net>

Bugs item #1710295, was opened at 2007-04-30 14:52
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1710295&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Documentation
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: elrond79 (elrond79)
Assigned to: Nobody/Anonymous (nobody)
Summary: Tutorial - Section 8.3 - <type 'instance'>

Initial Comment:
In section 8.3 of the Tutorial, after a paragraph saying "But use of .args is discouraged" is a code sample; in this sample, I believe the line

<type 'instance'>

should be changed to 

<type 'exceptions.Exception'>

At least, that's what I get when I input the sample... and it certainly makes more sense.

Python version: 2.5.1
OS: Microsoft Windows XP
    Professional x64 Edition
    Version 2003
    Service Pack 1
(taken from the System Properties / General page...)


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1710295&group_id=5470

From report at bugs.python.org  Thu Apr 12 17:30:44 2007
From: report at bugs.python.org (artleafoll)
Date: Thu, 12 Apr 2007 15:30:44 -0000
Subject: [issue1028] Art
Message-ID: <1176391842.44.0.460179389712.issue1028@psf.upfronthosting.co.za>


New submission from artleafoll:

behavior activity

----------
files: test.html
messages: 51800
nosy: artleafoll
severity: normal
status: open
title: Art
type: behavior

__________________________________
Tracker <report at bugs.python.org>
<http://bugs.python.org/issue1028>
__________________________________
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: test.html
Url: http://mail.python.org/pipermail/python-bugs-list/attachments/20070412/da4b0838/attachment-0001.pot 

From report at bugs.python.org  Thu Apr 12 17:54:46 2007
From: report at bugs.python.org (artleafoll)
Date: Thu, 12 Apr 2007 15:54:46 -0000
Subject: [issue1028] Art
Message-ID: <1176393285.29.0.694947690023.issue1028@psf.upfronthosting.co.za>


Changes by artleafoll:


__________________________________
Tracker <report at bugs.python.org>
<http://bugs.python.org/issue1028>
__________________________________

From report at bugs.python.org  Sun Apr 15 21:43:35 2007
From: report at bugs.python.org (Terry Jan. Reedy)
Date: Sun, 15 Apr 2007 19:43:35 -0000
Subject: [issue1023] Buy Tramadol online
Message-ID: <1176666214.54.0.789611842413.issue1023@psf.upfronthosting.co.za>


Changes by Terry Jan. Reedy:


__________________________________
Tracker <report at bugs.python.org>
<http://bugs.python.org/issue1023>
__________________________________

From report at bugs.python.org  Sat Apr 21 21:22:12 2007
From: report at bugs.python.org (devel)
Date: Sat, 21 Apr 2007 19:22:12 -0000
Subject: [issue1025] Test issue
Message-ID: <1177183331.09.0.779398781511.issue1025@psf.upfronthosting.co.za>


devel added the comment:

Minor test.

----------
nosy: +devel

__________________________________
Tracker <report at bugs.python.org>
<http://bugs.python.org/issue1025>
__________________________________

From shirishag75 at gmail.com  Fri Apr 27 17:09:24 2007
From: shirishag75 at gmail.com (shirish agarwal)
Date: Fri, 27 Apr 2007 15:09:24 -0000
Subject: using urllib to have an upload status bar.
Message-ID: <511f47f50704270809t2b9abf5dgdaf43404496ef924@mail.gmail.com>

Hi all,
     Before I start, please know that I'm no developer, just a user.  We
have a bug-reporting system in ubuntu called apport. Apparently apport uses
a python library called urllib. As of now, apport does a bouncing bar while
uploading which doesn't give the user idea as to how much is being uploaded.
Refer https://bugs.launchpad.net/ubuntu/+source/apport/+bug/91521 . I did
try to see if this issue has been addressed or not but came nowhere. It
would be nice to know if there are some solutions, or is there a roadmap
where this feature would be integrated in the library. Looking forward to
suggestions, additions, flames & constructive criticism of the same. Keep up
the good work. Cheers!
          Shirish Agarwal
  This work is licensed under the Creative Commons NonCommercial Sampling
Plus 1.0 License. To view a copy of this license, visit
http://creativecommons.org/licenses/nc-sampling+/1.0/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-bugs-list/attachments/20070427/479ca291/attachment-0001.html