Persistent Queue implementations?

[Aahz]

In article <auf9a4$ovp$1 at baldur.whoi.edu>,
Karl A. Krueger <kkrueger at example.edu> wrote:
>Aahz <aahz at pythoncraft.com> wrote:
>>
>> The question really is: what happens to dequeued objects that are not
>> fully processed (due to a system crash, say)?  That's why I was
>> suggesting a full-blown database, so that you can have a two-phase
>> commit.
>
>That's a very good point.  I may ponder it for a bit and come back. :)
>
>I'm not resistant to databases -- I have a mod_python / PostgreSQL app
>currently in maintenance -- but I'd kind of like something faster and
>more transparent here.

If re-processing already completed items isn't a problem, you could have
a separate thread that monitors the queue and checkpoints every X items.
It's less transparent in principle, but if the queue hides the
checkpoint thread from other threads, it doesn't look that way from the
outside.
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

"I disrespectfully agree."  --SJM