writing large files quickly

[Fredrik Lundh]

Bengt Richter wrote:

> >> How the heck does that make a 400 MB file that fast? It literally takes
> >> a second or two while every other solution takes at least 2 - 5 minutes.
> >> Awesome... thanks for the tip!!!
> >
> >Because it isn't really writing the zeros.   You can make these
> >files all day long and not run out of disk space, because this
> >kind of file doesn't take very many blocks.   The blocks that
> >were never written are virtual blocks, inasmuch as read() at
> >that location will cause the filesystem to return a block of NULs.
> >
> I wonder if it will also "write" virtual blocks when it gets real
> zero blocks to write from a user, or even with file system copy utils?

I've seen this behaviour on "big iron" Unix systems, in a benchmark that
repeatedly copied data from a memory mapped section to an output file.

but for the general case, I doubt that adding "is this block all zeros" or
"does this block match something we recently wrote to disk" checks will
speed things up, on average...

</F>