please critique my thread code

[Pau Freixes]

>
> MRAB wrote:
>
> Erm, shurely the bottleneck will be bandwidth not processor/memory?* If it
> isn't then - yes, you run the risk of actually DOSing their servers!


>
> Your mac will run thousands of threads comfortably but your router may not
> handle the thousands of TCP/IP connections you throw at it very well,
> especially if it is a domestic model, and sure as hell sourceforge aren't
> going to want more than a handfull of concurrent connections from you.
>
> Typical sourceforge page ~ 30K
> Project pages to read = 240000
>
> = ~6.8 Gigabytes


The best path for avoid congestion network problem and probably "DOS attack"
it's use a equilibrate number of parallel connection with a number of
threads. One router can handle hundred connections and Mac thread
implementation may be also - with a lot of memory or stack size profiling.
If you run hundred threads you will be fight with OS, Webserver, Router and
network starvation problems.

I believe use a 64 number of threads it's ok, but probably all 64 thread
connections will be performance for the same sourceforge web server -
balancing algorithm - and may be that this week up suspicion to the admin.

May be, you can use httplib [1] for do a persistent connection and reuse
same connection for same  thread.


[1] http://www.python.org/doc/2.4.2/lib/httplib-examples.html



-- 
Pau Freixes
Linux GNU/User
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20080619/affae819/attachment-0001.html>