Problem: 'Threads' in Python?

[Francis Avila]

Ralph Sluiters wrote in message ...
>Hi,
>i've got a small problem with my python-script. It is a cgi-script, which
is
>called regulary (e.g. every 5 minutes) and returns a xml-data-structure.
>This script calls a very slow function, with a duration of 10-40 seconds.
To
>avoid delays, i inserted a cache for the data. So, if the script is called,
>it returns the last caculated data-structure and then the function is
called
>again and the new data is stored in the cache. (There is no problem to use
>older but faster data)
>
>My problem is, that the client (A Java program (or browser, command line))
>waits, until the whole script has ended and so the cache is worthless. How
>can I tell the client/browser/... that after the last print line there is
no
>more data and it can proceed? Or how can I tell the python script, that
>everything after the return of the data (the retieval of the new data and
>the storage in a file) can be done in an other thread or in the background?

Wouldn't a better approach be to decouple the cache mechanism from the cgi
script?  Have a long-running Python process act as a memoizing cache and
delegate requests to the slow function.  The cgi scripts then connect to
this cache process (via your favorite IPC mechanism).  If the cache process
has a record of the call/request, it returns the previous value immediately,
and updates its cache in the meantime.  If it doesn't have a record, then it
blocks the cgi script until it gets a result.

How can threading help you if the cgi-process dies after each request unless
you store the value somewhere else?  And if you store the value somewhere,
why not have another process manage that storage?  If it's possible to
output a complete page before the cgi script terminates (I don't know if the
server blocks until the script terminates), then you could do the cache
updating afterwards.  In this case I guess you could use a pickled
dictionary or something as your cache, and you don't need a separate
process.  But even here you wouldn't necessarily use threads.

Threads are up there with regexps: powerful, but avoid as much as possible.
--
Francis Avila