Any python scripts to do parallel downloading?

[Jean-Paul Calderone]

On 1 Feb 2007 06:41:56 -0800, Carl Banks <pavlovevidence at gmail.com> wrote:
>On Feb 1, 9:20 am, Jean-Paul Calderone <exar... at divmod.com> wrote:
>> On 1 Feb 2007 06:14:40 -0800, Carl Banks <pavlovevide... at gmail.com> wrote:
>>
>>
>>
>> >On Jan 31, 3:37 pm, Jean-Paul Calderone <exar... at divmod.com> wrote:
>> >> On 31 Jan 2007 12:24:21 -0800, Carl Banks <pavlovevide... at gmail.com> wrote:
>>
>> >> >Michele Simionato wrote:
>> >> >> On Jan 31, 5:23 pm, "Frank Potter" <could.... at gmail.com> wrote:
>> >> >> > I want to find a multithreaded downloading lib in python,
>> >> >> > can someone recommend one for me, please?
>> >> >> > Thanks~
>>
>> >> >> Why do you want to use threads for that? Twisted is the
>> >> >> obvious solution for your problem,
>>
>> >> >Overkill?  Just to download a few web pages?  You've got to be
>> >> >kidding.
>>
>> >> Better "overkill" (whatever that is) than wasting time re-implementing
>> >> the same boring thing over and over for no reason.
>>
>> >"I need to download some web pages in parallel."
>>
>> >"Here's tremendously large and complex framework.  Download, install,
>> >and learn this large and complex framework.  Then you can write your
>> >very simple throwaway script with ease."
>>
>> >Is the twisted solution even shorter?  Doing this with threads I'm
>> >thinking would be on the order of 20 lines of code.
>>
>> The /already written/ solution I linked to in my original response was five
>> lines shorter than that.
>
>And I suppose "re-implementing the same boring thing over and over" is
>ok if it's 15 lines but is too much to bear if it's 20 (irrespective
>of the additional large framework the former requires).
>

It's written.  Copy it and use it.  There's no re-implementation to do.
And if you don't want to _limit_ the number of concurrent connections,
then you don't even need those 15 lines, you need four, half of which
are imports.  I could complain about what a waste of time it is to always
have to import things, but that'd be silly. :)

Jean-Paul