Python v.s. huge files PROBLEM!!

Tim Peters tim.one at comcast.net
Thu Jul 18 19:17:19 EDT 2002


[Jose Rivera]
> Scenario:
> OS      : WinNT 4.0
> FileName: RESPALDO_MENSUAL_Data.MDF
> Size    : 243,386,941,440 bytes
>
> Problem:
> I want to copy this file to another disk. Both disks have 500 GB of
> free space.
>
> Microsoft problem:
>    You May Not Be Able to Copy Large Files on Computers That Are
> Running Windows NT 4.0 or Windows 2000 (Q259837)
>
> Workaround suggested by Microsoft:
>    Use Backup / Restore utilities
>   Result: They didn't work either... using HP OmniBack
>
> Workaround made by us:
> Make a python program that read and writes to the other file in
> theother disk.
>
> Python Code:
>
> import sys
>
> if len(sys.argv) != 3:
> 	print 'Format:'
> 	print '\t pyCopy.exe SourceFile EndFile'
> else:
> 	fn1=sys.argv[1]
> 	fn2=sys.argv[2]
> 	f1=open(fn1,'rb')
> 	f2=open(fn2,'wb')
> 	data=f1.read(1024*1000)
> 	while data:
> 		f2.write(data)
> 		data=f1.read(1024*1000)
> 	f1.close()
> 	f2.close()
>
> Result:
> IOError: [Errno 22] Invalid argument

Tell us which version of Python you're using, and give a complete traceback.
It's hopeless unless you're using Python 2.2 (2.1 didn't support large files
on Windows).

Looks like 2.2 probably won't help either, though; the KB article you
referenced:

    http://support.microsoft.com/default.aspx?scid=kb;EN-US;q259837

makes it quite clear that this is a fundamental flaw in the OS kernel
support for buffered I/O.  Python also uses buffered I/O.






More information about the Python-list mailing list