creating/modifying sparse files on linux

[Marc 'BlackJack' Rintsch]

In <1124304819.090356.200280 at f14g2000cwb.googlegroups.com>,
draghuram at gmail.com wrote:

> options.size = 6442450944
> options.ranges = ["4096,1024","30000,314572800"]
> fd = open("testfile", "w")
> fd.seek(options.size-1)
> fd.write("a")
> for drange in options.ranges:
>     off = int(drange.split(",")[0])
>     len = int(drange.split(",")[1])
>     print "off =", off, " len =", len
>     fd.seek(off)
>     for x in range(len):
>     fd.write("a")
> 
> fd.close()
> 
> This piece of code takes very long time and in fact I had to kill it as
> the linux system started doing lot of swapping. Am I doing something
> wrong here? Is there a better way to create/modify sparse files?

`range(len)` creates a list of size `len` *in memory* so you are trying to
build a list with 314,572,800 numbers.  That seems to eat up all your RAM
and causes the swapping.

You can use `xrange(len)` instead which uses a constant amount of memory. 
But be prepared to wait some time because now you are writing 314,572,800
characters *one by one* into the file.  It would be faster to write larger
strings in each step.

Ciao,
	Marc 'BlackJack' Rintsch