Cancel threads after timeout

[dieter]

hyperboreean <hyperboreean at nerdshack.com> writes:

> Here's the use case I want to implement - I have to generate a report
> from multiple database servers. This report should be generated every 2
> hours. Sometimes it happens that a query on one of the database servers
> takes longer than expected and impedes the generation of this report
> (that's right, the queries are ran sequential). What I am trying to
> achieve is to parallelize the queries on each database server and to be
> able to cancel one of them if it takes longer than X minutes.
> threading.Thread doesn't support this and seems that in
> general programming languages don't implement a way to cancel threads
> from the outside.

And it is difficult in general.

The problem lies in the interaction between Python and (unknown by Python)
C extensions. Many C extensions would not work properly when simply
aborted in between (loose memory, fail to release resources,
leave an inconsistent state). You need something like "try: ... finally: ..."
or "try: ... except: ..." to have the ability to handle asynchronous
aborts - but C lacks such support.

In the light of this difficulty, the Python developpers decided not
to expose the possibility of many threading packages to abort a thread.

I was told that their is an approximative function at the Python
C interface (in Python 2, not sure whether it is also available in
Python 3): a function that allows you register the wish for a thread
aborting. This wish is recognized only when the thread is on
Python (not C) level (thus, not applicable in your case) and it causes
an exception that normally leads to the thread abortion but
plays well with Python's "try: ... finally: ..." and
"try: ... except: ...": thus, the thread can properly clean up.


You best approach is probably to ask the databases in parallel,
wait for some time and simply ignore answers that do not arrive
within that time (without any aborting trial).