Control over Communication Processes

[Benny Shomer]

Dear Folks,

I know this problem has been "chewed up" before, but I couldn't find
any single solid solution from all theh blurb I read.

PROBLEM:
========
I have a web agent module that is used in web robots and spiders.
I __must__ have a firm control over stalled connections, on a time
base, since my program runs a very long and complex operation.
The agent is an inheritting subclass of httplib.

Possible Solutions:
-------------------
1. Spawn the actual reads (after performing httplib.getfile()) as
a thread. Set a self.timer to the beginning of process.
On each readline by the thread, reset the internal class timer, so
that while large quantities of data still flow in, the timer doesn't
tick. Once connection stalls, readlines are not operated and the
timer ticks.
	BUT: I don't have a safe way to terminate the stalled thread.
             That leaves behind effectively a network zombie waiting
             for data that will never come sometimes.
             Even if I use getpid() in the thread and kill it externally
             I have a problem because it kills the spawning main class
             (...and is not safe to do anyway...)

2. Fork a child process, which can safely be killed in the even of
stalled connections.
       BUT:  The child, although it receives the self environment, can
             only read it, but can't modify it. So:
             a. It can't directly send the data back to the spawning
                class (this is solvable through using a temp file, or
                less likely a pipe, because the pipe will stall together
                with the connection)
             b. It has no effective way of resetting the class timer
                so the clock will be ticking even if the child works
                but simply reads large amounts of data (which is my
                case), so it gets killed when it's not supposed to...

Argghhhh...........

Any clever minds out there have descent solutions/offers?

Thanks a zillion,

Benny.


--
----                 E pur si muove!   (Galileo)                 ----
---------------------------------------------------------------------
Benny Shomer, Ph.D.                     Bioinformatics specialist
Tel#     +972 (8)  971-3165             email: bshomer at yahoo.com
Mobile#  +972 (54) 655-868              Fax#  +972 (15154) 655-868
Public PGP Key (0x3610041C) at URL:
http://wwwkeys.nl.pgp.net:11371/pks/lookup?op=index&search=0x3610041C


Sent via Deja.com http://www.deja.com/
Before you buy.