Testers please

[Scott David Daniels]

Martien Friedeman wrote:
> The size of the 'recording' is a major drawback, it was clearly not meant
> to record the processing of millions of records.
> 
> I need to find a way to specify the area in the code that needs recording.

Use something like zlib and write a compressed log -- especially
with logs you are working with highly compressible data.  The gzip
module can be used to easily write to it on the fly (incrementally).
You'll find with very little effort you'll get very good compression.
Recently I saw a set of standard logs, not designed for any compression,
showing a factor of 9 compression -- that is, I was seeing less than
a single bit required per byte in the original.

-- 
--Scott David Daniels
scott.daniels at acm.org