Speeding up ftp, async/threadpool?

Ville Vainio vvainio at tp.spt.fi
Thu Feb 13 02:43:32 EST 2003


Trying to do a mget-tish thing with ftplib for a large number of small
files, one can see how slow a protocol ftp is. I've thought of
optimizing this with a threadpool (which is easy to implement), but
OTOH it seems that an async ftp client could juggle hundreds of ftp
downloads/uploads at the time, effectively saturating the network. How
come such thing hasn't been implemented yet? I'm thinking of an API
like:

a = asyncftp(connections=200)
a.login("host1.com","myuid","mypasswd")
a.login("host2.com","myuid2","mypasswd2")

a.getfile("host1.com","/home/myuid/foo.txt",target="/tmp/foo.txt")
a.getfile("host2.com","/home/myuid/bar.txt",target="/tmp/bar.txt")

a.go()
a.waitforall()




More information about the Python-list mailing list