File IO question.

[Larry Bates]

You answered your own question:

> I'm guessing the hard drives would fill up eventually
> anyway regardless of the size of the hard drive, the
> crashes just wouldn't occur quite as often.

If this statement is true, compressing the files won't
help either as that is just the same as a larger hard
drive.

But, you could limit the size and number of trace files
and periodically throw away the oldest ones to keep the
trace files at a nearly constant size.  I just can't tell
from your description exactly what is getting written to
the trace file that makes it so large (debugging info?).
Maybe you could only log errors/warnings to the trace
file so it grows more slowly?

Larry Bates
Syscon, Inc.

"J Poirier" <oo1z at yahoo.com> wrote in message
news:mailman.445.1089911565.5135.python-list at python.org...
> Hi All,
>
> I'm hoping that someone might have some pointers or
> examples on how to proceed with a solution to the
> following problem:
>
> A test application, which produces a trace file, is
> being run for very long periods of time. Say 72 hours
> or more.
>
> The application is often running on older PCs that
> have relatively small hard drives in comparison to
> to how big the trace file can become.
>
> The trace files tend to accumulate as they're not
> always deleted once archived to a network database,
> and often times a machine will crash in the middle of
> a
> very long test due to the hard drive filling up.
>
> There are dozens of PCs being used for the tests so
> buying bigger hard drives isn't really feasible. And
> I'm guessing the hard drives would fill up eventually
> anyway regardless of the size of the hard drive, the
> crashes just wouldn't occur quite as often.
>
> The nice thing is that the trace files compress quite
> well.
>
> I messed around with the mmap and file object stuff
> as well as the win32 extensions thinking that I could
> extract and compress the data that was being written
> to
> the trace file, by the application, in chuncks.
>
> Although I was able to get it to work on a contrived
> setup, it didn't work when used with the real
> application.
>
> Any hints on how to get something similar to the above
> to work or recommendations on alternate solutions
> would
> be *greatly* appreciated.
>
> Thanks,
> Joe
>
>
>
>
>
> __________________________________
> Do you Yahoo!?
> Yahoo! Mail - 50x more storage than other providers!
> http://promotions.yahoo.com/new_mail