[SciPy-dev] Difficulty creating an example for illustrating memmap.flush()

[Charles R Harris]

On Sun, Aug 2, 2009 at 9:21 PM, David Goldsmith <d_l_goldsmith at yahoo.com>wrote:

>
> Hi, David.  Thanks for your (validating) reply. :-)  Well, there's
> something more going on than just the size of the mmap or the size of the
> change to the mmap: I experimented with adding a shape (10,) complex128
> array to shape (10, 10), (10, 10, 10), (10, 10, 10, 10), (10, 10, 10, 10,
> 10) mmaps (and their corresponding "flat" mmaps, shape (100,), etc.), both
> just in the last dimension using [9, 9, 9, 9, :], e.g., and everywhere using
> [:,:,:,:,:] - which worked! - and the file on disc was updated every time
> without ever having to call flush, not once!  So, hopefully, someone who
> knows what's going on will chime in...
>

The data in memmapped files is swapped in and out of memory by the operating
system, so what happens to be in memory at any given time is OS dependent
and also changes with demands for memory from other processes and such. It's
a bunch a behind the scenes machinery and not predictable. Even a flush can
be delayed. My guess is that you need to flush the data if you want it to go
to disk, which might be the case if you have a program that needs to push
out data and you want to minimise losses from power outages and such.
Otherwise the data could just sit in memory and never get written out until
the file is closed.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20090802/47746e81/attachment.html>