[spambayes-dev] Unsafe Pickle Saving

[David Abrahams]

on Tue Oct 23 2007, Toby Dickenson <tdickenson-OuhhWZXUDiwPIx3zVcpl1ZqQE7yCjDx5-AT-public.gmane.org> wrote:

> skip at pobox.com wrote:
>
>> 
>>     Dave> Somebody should have already written a file lock library that
>>     Dave> encapsulates it, so maybe we can use that.  And if it doesn't
>>     Dave> exist, it should!
>> 
>> It does exist, in several flavors, as my query to python-dev indicates.
>
> Sounds like a good addition to python, but Im not sure its the solution in
> this case. The cron job probably should tte using a different database
> filename, then atomically swap the new database in place of the
> old. 

It does do that.  However, the caches for dns and images (which are
pickles) are the same in both cases.

> This ensures you dont leave a partial database in place if the
> machine crashes/is shutdown/process is killed during the tte run. It
> also allows procmail to run on the old database concurrently with
> the training run, which I guess a lock file would prevent.
>
> Or am I confused here. Ive just re-read David's original post, and you say
> this the problem is caches rather than the database. Which caches are
> these?

Bingo; see above.

-- 
Dave Abrahams
Boost Consulting
http://www.boost-consulting.com