Downloading a file with Python 2

[Peter Otten]

Rahul Sircar wrote:

> I wrote my code for downloading a file 'Metasploitable' using urllib2.But
> it seems to have entered infinite loop.Because the screen is blank.It just
> hangs there.

It "hangs", i. e. doesn't give any feedback while the data is retrieved.

> Please have a look at my code.
> 
> import urllib2
> file = 'metasploitable-linux-2.0.0.zip'
> url='
> https://downloads.sourceforge.net/project/metasploitable/Metasploitable2/
> metasploitable-linux-2.0.0.zip
> '
> response = urllib2.urlopen(url)
> fh=open(file,'w')
> fh.write(response.read())

The line above reads the complete file into memory before it even writes the 
first byte. That's usually fine if the file is small, a few KB, say, but in 
this case (833MB) it's better to read smaller chunks, like in 

CHUNKSIZE = 2 ** 12
total = 0
with open(file, "wb") as f:
    while True:
        chunk = response.read(CHUNKSIZE)
        if not chunk:
            break
        f.write(chunk)
        total += len(chunk)
        print total, "bytes written"


Another option is to use urllib.urlretrieve which also allows to give some 
feedback while downloading:

import sys
import urllib

file = 'metasploitable-linux-2.0.0.zip'
url = (
    "https://downloads.sourceforge.net/project"
    "/metasploitable/Metasploitable2/metasploitable-linux-2.0.0.zip"
)

def progress(block, blocksize, filesize):
    sys.stdout.write("\r%s of %s" % (block * blocksize, filesize))
    sys.stdout.flush()

urllib.urlretrieve(url, filename=file, reporthook=progress)