Python threading, and processes

[Alex Martelli]

Robin Haswell <rob at digital-crocus.com> wrote:
   ...
> Cheers for that info. The thread's main tasks are getting webpages
> (spidering), the actual amount of processing done in each thread is
> minimal - that's why I'm confused by the CPU usage.

BTW, spidering is a great use case for async (even-driven) programming;
use Twisted (or even build on top of asyncore) and you'll easily
saturate your network bandwidth with modest CPU demands.


Alex