looking for wget-like module for getching software

[Robert P. J. Day]

On Thu, 16 Dec 2004, Stefan Behnel wrote:

>
> Robert P. J. Day schrieb:
> >   that is, i can just say, "go get file gcc-3.4.2.tar.bz2", and start
> > searching at "ftp://pub.gnu.org/pub/gcc".  i may not know how far down
> > in the directory structure that file is, but wget will happily search
> > recursively until it finds it.
>
> That sounds pretty inefficient and should produce some load on the
> server side. Imagine everyone doing that. Many FTP-servers provide
> files like "ls-R.gz" somewhere, meaning, a list of files and
> directories.

i realize that and, as much as possible, i try to start the process
with what i *know* is the actual directory location.  but
occasionally, if i can't guarantee an exact directory, i just want the
option to start a bit higher up the directory structure.  i wasn't
going to make a habit of it, it's more an absolute last resort, if it
comes down to it.

> And then, there's Google. Why do you have to use FTP? Ask Google for
> "yourfile.tar.bz2 site:the.domain.name". It will very likely return
> a suitable URL for you that you can use for download. Just tried it
> with "site:ftp.gnu.org gcc-3.4.2.tar.bz2" - works. You can even use
> that link and try to replace "http://" with "ftp://" for download.
> Should work for many source repositories.
>
> You can use urllib2 for both querying and download.

and that pretty much answers the question i had.  much thanks.  and
now that i have my answer, i might have to unsubscribe.  you guys sure
are a chatty bunch. :)

rday

p.s.  just kidding.