[Patches] [ python-Patches-1049855 ] PyOS_InputHook inconsistency on Windows

Fri Aug 5 23:07:18 CEST 2005

Patches item #1049855, was opened at 2004-10-19 18:03
Message generated for change (Comment added) made by mdehoon
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1049855&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Core (C code)
Group: Python 2.4
Status: Open
Resolution: None
Priority: 5
Submitted By: Michiel de Hoon (mdehoon)
Assigned to: Nobody/Anonymous (nobody)
Summary: PyOS_InputHook inconsistency on Windows

Initial Comment:
PyOS_InputHook is a pointer to a function to be called
periodically when Python is idle. It is used, for
example, to get messages delivered to graphics windows.
If I compile Python from source (which uses
Modules/readline.c), PyOS_InputHook is called ten times
per second. However, with the Windows-version of
Python, PyOS_InputHook is called only once for each
command typed by the user. It seems that the problem
resides in Parser/myreadline.c (which I presume is used
for the Windows-version of Python):

if (PyOS_InputHook != NULL)
       (void)(PyOS_InputHook)();
...
p = fgets(buf, len, fp);

Python idles at "fgets", but PyOS_InputHook is not
being called while Python is idle.

The attached patch solves this problem by using the
"select" function, basically in the same way as what is
in Module/readline.c. I don't know how to compile
Python for Windows, so I wasn't able to test this patch.

----------------------------------------------------------------------

>Comment By: Michiel de Hoon (mdehoon)
Date: 2005-08-06 06:07

Message:
Logged In: YES 
user_id=488897

One more fix: I noticed that on Windows, the patch causes
the CPU usage to go up to 100%. The reason for this is that
if the users presses a control key (e.g. shift), the
WaitForSingleObject function returns, but _kbhit() returns
false since no character input is waiting in the buffer.
Hence, the loop continues to run, and WaitForSingleObject
will again return because the control key stroke has not
been flushed from the buffer. The fix is in the patch dated
20050803, which is to flush the buffer with
FlushConsoleInputBuffer if a key was hit but no character
input is available.

----------------------------------------------------------------------

Comment By: Michiel de Hoon (mdehoon)
Date: 2005-07-29 11:40

Message:
Logged In: YES 
user_id=488897

I have found a solution for the problem described below
("python -i < inputfile.py" behaves differently). The
solution was to use _isatty to check if stdin is redirected,
and check for keyboard input only if stdin is not
redirected. With the latest version of this patch
(20050728), there are no changes in Python's behaviour on
Windows or Unix (except that PyOS_InputHook is called ten
times per second, which is what this patch intends to
solve). I apologize for the fact that multiple iterations
were needed to converge to the right (I hope) solution for
this bug.

----------------------------------------------------------------------

Comment By: Michiel de Hoon (mdehoon)
Date: 2005-07-16 11:14

Message:
Logged In: YES 
user_id=488897

I have now recompiled Python on Windows to test this patch.
When compiling this patch, I found out that _get_osfhandle
needs to be cast to a HANDLE*. The latest version of this
patch (dated 20050715) includes this cast. I have tested
this patch by running the test suite.
One note: Whereas "python -i inputfile.py" and "python <
inputfile.py" work as expected, "python -i < inputfile.py"
behaves differently with this patch (python waits for the
user to hit enter before proceeding). I am not sure if this
is significant, as I wouldn't know what a user might try to
achieve with "python -i < inputfile.py".
I'd be happy to send a binary for Windows to anybody who
would like to test this patch.

----------------------------------------------------------------------

Comment By: Michiel de Hoon (mdehoon)
Date: 2005-06-11 05:46

Message:
Logged In: YES 
user_id=488897

Thomas Heller sent me this comment:

> The PyOS_StdioReadline function calls my_fgets with a file
> pointer argument.  The my_fgets function in the patch assumes
> that STD_INPUT_HANDLE is the handle to use - is this 
> assumption always correct?

He is right, this assumption is not necessarily correct. I
have made a new patch (labeled 20050610) to solve this
issue. This latest version has been tested on Cygwin, but
not yet on Windows -- I need to dig up a compiler for
Windows first.

----------------------------------------------------------------------

Comment By: Michiel de Hoon (mdehoon)
Date: 2005-05-13 12:57

Message:
Logged In: YES 
user_id=488897

I have rewritten the patch to include Windows support. I
compiled Python on Windows with VC98, and on Linux and Mac
OS X and found no problems with it. The new patch is attached.

----------------------------------------------------------------------

Comment By: Kurt B. Kaiser (kbk)
Date: 2005-05-02 12:40

Message:
Logged In: YES 
user_id=149084

Yeah, that's why I didn't want to check it in, I can't build
on Windows ithe VC 5.

----------------------------------------------------------------------

Comment By: Michiel de Hoon (mdehoon)
Date: 2005-05-02 11:52

Message:
Logged In: YES 
user_id=488897

Thanks for looking at this patch.

Today I found out that it is possible to compile Python
2.4.1 with the older Microsoft Visual Studio 6.0, which I
happen to have in my office. It turned out that the patch
does not work correctly on Windows, due to the fact that the
"select" function doesn't work with stdin on Windows. I will
fix the patch so it'll work on Windows too.

----------------------------------------------------------------------

Comment By: Kurt B. Kaiser (kbk)
Date: 2005-05-01 03:21

Message:
Logged In: YES 
user_id=149084

The revised patch looks ok to me.

----------------------------------------------------------------------

Comment By: Michiel de Hoon (mdehoon)
Date: 2005-03-26 22:33

Message:
Logged In: YES 
user_id=488897

I have now tested this patch. After fixing one error (I had
forgotton to declare one variable), the patch works
correctly. I have uploaded a fixed patch.
Note that this bug occurs not only on Windows, but on any
Python compiled without readline. (which allowed me to test
this patch by compiling Python without readline support).

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1049855&group_id=5470