[ python-Bugs-1501291 ] python/ncurses bug in 2.4.3 with extended ascii

[SourceForge.net]

Bugs item #1501291, was opened at 2006-06-05 16:34
Message generated for change (Comment added) made by unixops1234
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1501291&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Extension Modules
Group: Python 2.4
>Status: Closed
>Resolution: Fixed
Priority: 5
Submitted By: UnixOps (unixops1234)
Assigned to: Nobody/Anonymous (nobody)
Summary: python/ncurses bug in 2.4.3 with extended ascii

Initial Comment:
There is a problem displaying extended ascii characters
in ncurses through python, which does not exist in
versions prior to 2.4.2. I am running RedHat Enterprise
Linux WS 3 with updated patches, using the system
version of ncurses (ncurses-5.3-9.4 and devel). When
building a vanilla python 2.4.3 from source, printing
extended ascii characters in ncurses fails:

$ cat test.py
import curses

screen = curses.initscr()

screen.addstr("\x80")

screen.getch()
curses.endwin()

$ python test.py
Traceback (most recent call last):hon test.py
File "test.py", line 5, in ?
screen.addstr("\x80")
_curses.error: addstr() returned ERR

This should produce a blank ncurses screen, rather than
the addstr() error. I've been able to confirm that it
works with python 2.4.2 and earlier. 

To ensure that ncurses was able to display the
character, I wrote this test C program: 

$ cat test.c
#include <ncurses.h>

int main()
{
initscr();
int rtn = addstr("\x80");
getch(); 
endwin();
printf("The return value was %d.\n",rtn);
return 0;
}

$ gcc test.c -o test -lncurses
$ ./test

This works just fine, so I think the problem lies
somewhere in the python interface to ncurses. Python
can print this character without errors when not using
ncurses. Perhaps ncurses is expecting different
initialization than python is providing.  

I've also tested this on a RedHat WS 4 machine, where
it works just fine. This system is running
ncurses-5.4-13 and ncurses-devel-5.4-13. It seems the
newer release of python has changed something that the
older versions of ncurses are unable to handle.  

Can this be fixed in _cursesmodule.c?

----------------------------------------------------------------------

>Comment By: UnixOps (unixops1234)
Date: 2006-07-31 16:33

Message:
Logged In: YES 
user_id=1534776

akuchling, thanks. Your suggestion worked. I added

import locale
locale.setlocale(locale.LC_ALL, 'en_US')

to the beginning of my test.py script and it correctly
displayed the funky character.

loewis, what I meant is that the C code correctly printed a
strange character instead of crashing with a curses error
message. 

The panelw patch is not needed after all. The test.py script
also works just fine if I build python against ncurses 5.5
instead of 5.4.

Thank you both for your help with this.

----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2006-07-26 14:39

Message:
Logged In: YES 
user_id=21627

unixops1234, can you please explain what "works just fine"
for the C code means? \x80 is not a meaningful character for
the terminal. On my system (Debian with ncurses 5.5, locale
de_DE.UTF-8, konsole), the program prints something, namely
the characters ~@


----------------------------------------------------------------------

Comment By: A.M. Kuchling (akuchling)
Date: 2006-07-26 12:45

Message:
Logged In: YES 
user_id=11375

The ncurses 5.5 changelog, currently at
http://www.gnu.org/software/ncurses/ncurses.html, contains
this entry:

  * use setlocale() to query the program's current locale
rather than using getenv(). This supports applications which
rely upon legacy treatment of 8-bit characters when the
locale is not initialized.

So maybe this is a problem in ncurses 5.4 that can be
avoided if your Python script calls 'locale.setlocale("<some
locale that uses Latin-1>")'.


----------------------------------------------------------------------

Comment By: UnixOps (unixops1234)
Date: 2006-06-08 14:02

Message:
Logged In: YES 
user_id=1534776

I tried recompiling Python with the patch from bug #1464056
that fixes the panelw linking in _curses_panel.so. It now
properly links with libpanelw instead of libpanel, but the
original problem persists. I still get "_curses.error:
addstr() returned ERR" when running the test code.

----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2006-06-08 11:58

Message:
Logged In: YES 
user_id=21627

I doubt it can be fixed in _cursesmodule. Notice that Python
links with ncursesw now, not with ncurses anymore. There is
a bug in 2.4.3 which links, apparently incorrectly, with
panel instead of panelw. Try changing that and see whether
it helps.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1501291&group_id=5470