Efficient checksum calculating on lagre files

[Nick Craig-Wood]

Christos TZOTZIOY Georgiou <tzot at sil-tec.gr> wrote:
>  On 09 Feb 2005 10:31:22 GMT, rumours say that Nick Craig-Wood
> <nick at craig-wood.com> might have written:
> 
> >But you won't be able to md5sum a file bigger than about 4 Gb if using
> >a 32bit processor (like x86) will you?  (I don't know how the kernel /
> >user space VM split works on windows but on linux 3Gb is the maximum
> >possible size you can mmap.)
> 
>  Indeed... but the context was calculating efficiently checksums for large files
>  to be /served/ by a webserver.  I deduce it's almost certain that the files
>  won't be larger than 3GiB, but ICBW :)

You are certainly right ;-)

However I did want to make the point that while mmap is extremely
attractive for certain things, it does limit you to files < 4 Gb which
is something that people don't always realise.

-- 
Nick Craig-Wood <nick at craig-wood.com> -- http://www.craig-wood.com/nick