[Tutor] Fwd: Problem reading large files in binary mode

[Peter Otten]

Mirage Web Studio wrote:

> I am new to python programming.  while trying it out i find that in my
> code file io.read is not reading large files particularly over 1 gb. my
> code is posted below.  i am working on python 3.3 on windows with ntfs
> partition and intel corei3 ram 3gb. the execution always stops saying
> error with py.exe but idle editor and idle window remains open.

 
>      try:
>          #print ("Readding")
>          fcontents=f.read()

In the line above you are reading the whole file into memory.

>          #print ("Read successfully")
>          hashvalue=hashlib.md5(fcontents).hexdigest()
>          #print ("md5 successfully")

Try reading the file in chunks instead:

CHUNKSIZE = 2**20
hash = hashlib.md5()
while True:
    chunk = f.read(CHUNKSIZE)
    if not chunk: 
        break
    hash.update(chunk)
hashvalue = hash.hexdigest()