how to get 20000 html pages content quickly from one server?

JuHui phpbird at gmail.com
Wed Mar 15 11:22:35 EST 2006


Hi
  I want to get 200000 html pages content from one server, you know
urllib.urlopen need construct network connection, it will be very
slowly, how to speed up this function?
 I try to using multi-thread, it speed up, but I want to quickly more,
any idea about it?
Thanks!




More information about the Python-list mailing list