urllib2 performance on windows, usb connection

[MRAB]

dq wrote:
 > Martin v. Löwis wrote:
 >>> So does anyone know what the deal is with this?  Why is the same 
code so
 >>> much slower on Windows?  Hope someone can tell me before a holy war
 >>> erupts :-)
 >>
 >> Only the holy war can give an answer here. It certainly has *nothing* to
 >> do with Python; Python calls the operating system functions to read from
 >> the network and write to the disk almost directly. So it must be the
 >> operating system itself that slows it down.
 >>
 >> To investigate further, you might drop the write operating, and measure
 >> only source.read(). If that is slower, then, for some reason, the
 >> network speed is bad on Windows. Maybe you have the network interfaces
 >> misconfigured? Maybe you are using wireless on Windows, but cable on
 >> Linux? Maybe you have some network filtering software running on
 >> Windows? Maybe it's just that Windows sucks?-)
 >>
 >> If the network read speed is fine, but writing slows down, I ask the
 >> same questions. Perhaps you have some virus scanner installed that
 >> filters all write operations? Maybe Windows sucks?
 >>
 >> Regards,
 >> Martin
 >>
 >
 > Thanks for the ideas, Martin.  I ran a couple of experiments to find the
 > culprit, by downloading the same 20 MB file from the same fast server. I
 > compared:
 >
 > 1.  DL to HD vs USB iPod.
 > 2.  AV on-access protection on vs. off
 > 3.  "source. read()" only vs.  "file.write( source.read() )"
 >
 > The culprit is definitely the write speed on the iPod.  That is,
 > everything runs plenty fast (~1 MB/s down) as long as I'm not writing
 > directly to the iPod.  This is kind of odd, because if I copy the file
 > over from the HD to the iPod using windows (drag-n-drop), it takes about
 > a second or two, so about 10 MB/s.
 >
 > So the problem is definitely partially Windows, but it also seems that
 > Python's file.write() function is not without blame.  It's the
 > combination of Windows, iPod and Python's data stream that is slowing me
 > down.
 >
 > I'm not really sure what I can do about this.  I'll experiment a little
 > more and see if there's any way around this bottleneck.  If anyone has
 > run into a problem like this, I'd love to hear about it...
 >
You could try copying the file to the iPod using the command line, or
copying data from disk to iPod in, say, C, anything but Python. This
would allow you to identify whether Python itself has anything to do
with it.