[spambayes-dev] Strange performance dip
andDBRunRecoveryErrorretreat
Richie Hindle
richie at entrian.com
Fri Jan 2 03:49:55 EST 2004
[Richie]
> I've committed the script as testtools/hammer.py,
[Tony]
> Actually utilities/hammer.py :)
Like I said, it was 2am. 8-)
> Richie - around about how many messages could you do before it crashed? It
> crashes early (~1700) for me with the 1.0a6 reopen-before-closing bug
> (sometimes a DB_RUNRECOVERY, sometimes a ham/spam count) but if I get it to
> always close or save the db before reopening, it just goes and goes - I got
> to 57900 before Python crashed *.
With the bug it would get to a few tens of thousands, same as you.
Without it went to 250,000 before I killed it.
> If I get it to close before reopening, then interrupt (ctrl-c) it at some
> point after its done its first save, and restart it (without it deleting the
> existing db file) it will chug along for a while, and then at some later
> point die with a RUNRECOVERY. Restarting it then will provoke an immediate
> RUNRECOVERY. This suggests (I think) that reopening the db without closing
> it can cause a RUNRECOVERY error at some later point - even several
> reopenings later - rather than immediately like I expected.
Agreed, though I never noticed the "even several reopenings later" part,
probably due to impatience...
[Tony]
> Maybe this is one of the causes for the RUNRECOVERY errors - the user
> doesn't close sb_server properly, so the db isn't properly closed/saved.
> Some time later the error occurs. It would explain why they are less
> frequent with the plug-in, because the plug-in saves the db much more often
> (after every "delete as spam"/"recover from spam" event, and IIRC after any
> incremental training).
[Tim]
> I think you may be on to something here!
Sadly not. sb_server saves the db after ever train as well, out of
paranoia. The page should always say "Training... Saving... Done". If
there's a way of training without saving, maybe that's the problem, but I
don't believe there is...?
--
Richie Hindle
richie at entrian.com
More information about the spambayes-dev
mailing list