Any python scripts to do parallel downloading?

Jean-Paul Calderone exarkun at divmod.com
Thu Feb 1 09:20:37 EST 2007


On 1 Feb 2007 06:14:40 -0800, Carl Banks <pavlovevidence at gmail.com> wrote:
>On Jan 31, 3:37 pm, Jean-Paul Calderone <exar... at divmod.com> wrote:
>> On 31 Jan 2007 12:24:21 -0800, Carl Banks <pavlovevide... at gmail.com> wrote:
>>
>> >Michele Simionato wrote:
>> >> On Jan 31, 5:23 pm, "Frank Potter" <could.... at gmail.com> wrote:
>> >> > I want to find a multithreaded downloading lib in python,
>> >> > can someone recommend one for me, please?
>> >> > Thanks~
>>
>> >> Why do you want to use threads for that? Twisted is the
>> >> obvious solution for your problem,
>>
>> >Overkill?  Just to download a few web pages?  You've got to be
>> >kidding.
>>
>> Better "overkill" (whatever that is) than wasting time re-implementing
>> the same boring thing over and over for no reason.
>
>"I need to download some web pages in parallel."
>
>"Here's tremendously large and complex framework.  Download, install,
>and learn this large and complex framework.  Then you can write your
>very simple throwaway script with ease."
>
>Is the twisted solution even shorter?  Doing this with threads I'm
>thinking would be on the order of 20 lines of code.
>

The /already written/ solution I linked to in my original response was five
lines shorter than that.

Hmm.

Jean-Paul



More information about the Python-list mailing list