How to batch download files from web page?

Wed May 12 08:49:03 EDT 2004

That's actually pretty easy. Download the list, parse it up to find the links/file names (turning them into links if you have to), shove them up a list. Begin pop'ing from the list and download. you *could* thread that out so you can have a few downloads going at once, but I wouldn't go beyond 5 or 6, tops. Upon successful completion, toss the link away. If you fail, append it to the list for a retry.

--------------------------
Chris Hornberger
Blackrock - 302.797.2318
chris.hornberger at blackrock.com

Card carrying MSDN member since 2004.
No, really. I've got the card to prove it.

-----Original Message-----
From: python-list-bounces+chris.hornberger=blackrock.com at python.org
[mailto:python-list-bounces+chris.hornberger=blackrock.com at python.org]On
Behalf Of sj
Sent: Wednesday, May 12, 2004 1:25 AM
To: python-list at python.org
Subject: How to batch download files from web page?

I wish to download hundreds of files from the University of Iowa sound
archive.  Doing it manually would be a daunting task especially since the
files are each a few mega bytes long.  Is there a standard way of using
Python for such a task?  I have a fair amount of programming experiance but
very little of it relates to networks.  

For those who are intrested the University of Iowa's sound archive may be
found at http://theremin.music.uiowa.edu/MIS.html

Thanks in advance
Steven Jones
-- 
http://mail.python.org/mailman/listinfo/python-list