Sockets and threads

[Alan Kennedy]

Byron Morgan wrote:

>>> If I monitor a socket connection or even jsust listen to a 
>>> socket in a thread, Python's cpu usage expands to use all
>>> available processor time.

> Here are two code samples - sample 1 runs very politely, using
> minmal cpu. Sample 2 causes the cpu usage meter to max out to 100%
> unless there is a sleep cycle in main(), in which case cpu usage
> looks very much like sample 1 (but I need for my apps to do stuff,
> not spend the day sleeping).


> # tnet1.py

> while not msvcrt.kbhit():
>     txt = session.read_until('\n',.5)
>     #seems to make no difference whether a timeout value is used or not.
>     if txt.strip() is not '':
>         print txt.strip()

I think that this is one problem. My understanding is that msvcrt.kbhit() is a
non-blocking call, meaning that there will no pause whether or not a key has
been hit. This while loop will cycle continually until a key is pressed. And if
the code that is running inside the loop does not pause for any reason, then
this will consume all of the CPU cycles.

But, inside your function, you do a "Telnet().read_until()", which is actually
blocking waiting for particular input, i.e. the "\n". This means that the
".read_until" function won't return until it has read a "\n" from the input
stream, or until 0.5 seconds have elapsed. Which means that most often, the
keyboard will only be checked every 0.5 seconds. Which is fairly light load for
a modern machine ;-) Which is why this while loop *seems* to behave properly.

> #tnet2.py
> def main():
>     while not msvcrt.kbhit():
>         pass
>         #sleep(5)
>         #uncomment sleep statement and cpu usage returns to normal.
>     session.close()

And the same while loop again, but this time without a blocking call inside it
to prevent it from chewing up all of the CPU cycles. If you uncomment your
sleep(5), you will find that it runs continually, waking up every five seconds,
checking once if the keyboard was hit and then going back to sleep or exiting.
One keyboard check every five seconds is a pretty light load, so you wouldn't
notice it on your cpu meter. But it won't respond to your input while it is
inside that sleep call.

If you want to check two different sources of information in this way, you have
to make a trade-off between how many CPU cycles you want to consume and how
responsive you want your application to be. Make your while loop pause for a
configurable amount of time, and pause that amount. These loops often have a
format like this:

timeout = 0.100 # 100 milliseconds
while 1:
    # Do a non-blocking check of source 1
    checkSource1()
    # Do a non-blocking check of source 2
    checkSource2()
    # And be nice to the cpu
    sleep(timeout)

On platforms like *nix, there are ways of checking multiple sources at a time,
through a select, but you need to be able to wrap everything as a file
descriptor.

This kind of activity should only be required in a single-threaded situation.
The reason for having multiple threads is that threads can operate
independently, meaning that some can block while others continue executing, i.e.
do multiple things at once, without you having to explicitly manage them.

BTW, another thing I notice (in "tnet2.py") is that you're checking
"msvcrt.kbhit()" in two threads at a time. This is threading on thin ice, since
input sources like keyboards can generally not be shared between two threads
simultaneously consuming the input. However, if you don't actually read any
characters from the keyboard, it should be OK.

HTH,

-- 
alan kennedy
-----------------------------------------------------
check http headers here: http://xhaus.com/headers
email alan:              http://xhaus.com/mailto/alan