Multithreaded class with queues

[Aahz]

In article <mailman.2037.1122027168.10512.python-list at python.org>,
Antal Rutz  <arutz at mimoza.pantel.net> wrote:
>
>I wrote a little class to make multihreading easier. It's based on one
>of aahz's threading example scripts. What it does:

Thanks!

>It spawns up number of CollectorThreads and one ProcessThread. The
>CollectorThreads listen on one queue (inputqueue), read, process the
>data (with colfunc), put the result onto the outputqueue. The
>ProcessThread listens on the outputqueue, reads, processes (with
>prfunc). end. (more details in the attached file)
>
>it seems to work with test functions but when I use a network-intensive
>function (snmp-queries) it just gets slower with maxThreads set to more
>than 1.

Hrm.  There's nothing obviously wrong.  What happens if you use it as a
spider?
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

The way to build large Python applications is to componentize and
loosely-couple the hell out of everything.