persistent fifo queue class

[Diez B. Roggisch]

David Bear schrieb:
> Diez B. Roggisch wrote:
> 
>> David Bear schrieb:
>>> I'm looking to see if there are any examples or prewritting fifo queue
>>> classes. I know this is a broad topic. I'm looking to implement a simple
>>> application where a web server enqueue and pickle using a local socket on
>>> to a 'queue server' -- and then I will have another application dequeue
>>> the pickles again using a local socket.
>> Why don't you use a DB for that? If you want pickles, use a  blob
>> column. But all the rest - a defined protocol, stable server,
>> transactions - you get for free.
>>
>> Diez
> 
> Thanks for the suggestion. I did think of this. Indeed the final destination
> of the data is in a db. However, the postsgresql server is on a separate
> box. It will be connected via a private lan. I was worried that possible
> network disruptions would cause either the web application to hang -- or
> data to just get lost. I was thinking along the lines is a message queue
> architecture, where the web app would push data directly onto a queue --
> and then a worker app would dequeue the data and handle it by sending it to
> the db server or otherwise.

Well, if your DB connection is flaky, data will be corrupted - or lost. 
In any case. What good does it do for you to have the messages then? And 
what if the Harddisk the file sits on is corrupt? And don't forget the 
added complexity of a home-brewn solution with all its possible errors - 
much more likely to cause data loss.


Diez