Avoiding excessive writes

prometheus235 at gmail.com prometheus235 at gmail.com
Wed Mar 18 22:52:54 EDT 2009


I'm aware that C, and thus Python I suppose, will cache file writes
for efficiency and to avoid excessive disk activity, causing
occasional problems when forgetting to flush.  How exactly this works
is still somewhat of a mystery to me, but a concern because of a
program I'm developing:

The program performs tests on some hardware attached to a serial port
and cycles every few seconds; for the reliability tests I'll do this
anywhere from 10k to 100k times, logging each event.  Sometimes the
file gets targeted to a thumbdrive, and 100k writes likely puts
significant wear on the flash, so what I've done in the past is just
make my own cache of ~500 data points then go and write it all at
once, but is there some way to set up the file operations so it
effectively does this for me?

Does the cache get written simply when an X-byte buffer fills up, and
can I change X?  Does it write after waiting so long?



More information about the Python-list mailing list