Any python scripts to do parallel downloading?

Carl Banks pavlovevidence at gmail.com
Thu Feb 1 09:14:40 EST 2007


On Jan 31, 3:37 pm, Jean-Paul Calderone <exar... at divmod.com> wrote:
> On 31 Jan 2007 12:24:21 -0800, Carl Banks <pavlovevide... at gmail.com> wrote:
>
> >Michele Simionato wrote:
> >> On Jan 31, 5:23 pm, "Frank Potter" <could.... at gmail.com> wrote:
> >> > I want to find a multithreaded downloading lib in python,
> >> > can someone recommend one for me, please?
> >> > Thanks~
>
> >> Why do you want to use threads for that? Twisted is the
> >> obvious solution for your problem,
>
> >Overkill?  Just to download a few web pages?  You've got to be
> >kidding.
>
> Better "overkill" (whatever that is) than wasting time re-implementing
> the same boring thing over and over for no reason.

"I need to download some web pages in parallel."

"Here's tremendously large and complex framework.  Download, install,
and learn this large and complex framework.  Then you can write your
very simple throwaway script with ease."

Is the twisted solution even shorter?  Doing this with threads I'm
thinking would be on the order of 20 lines of code.


Carl Banks




More information about the Python-list mailing list