Message queueing

[Christopher Browne]

Oops! "Noah Spurrier" <noah at noah.org> was seen spray-painting on a wall:
> Message queueing You could also roll your own using Sendmail or
> qmail or some other mail server.  That at least gets you a the
> transport layer. You would then have to add an interface for Python
> or XML-RPC -- and feed mail server error logs back into your system.
> It's easier than it sounds. We did something like this at BlueLight
> (K-Mart) -- of course, maybe that is a bad example ;-) But it
> worked; it was easy to use and fix; and easy to figure out how it
> was working -- unlike a horrible Java wrapped MQ-Series thing they
> also made us use.
>
> Your XML-RPC to Jython/JMS bridge thing sounds elegant compared to
> my idea.  Unless you find a better Message Queue then I think that's
> your best bet.  An XML-RPC interface to JMS almost sounds like a
> product.
>
> It's sad, but the importance of message queues does not seemed to
> have been picked up by the open source community. As far as I can
> tell there are no open source message queues... I don't have a job,
> maybe someone wants to help me write one :-)

The BIG problem with the "rolling via SMTP" is that SMTP doesn't
provide you _any_ kind of end-to-end guarantees on delivery.

SOAP::Lite has transport mechanisms where you can have messages go out
(client side) via SMTP and come in (server side) via POP3; it's kind
of cool stuff, but absent of guarantees of quaility of delivery, it's
of really limited usefulness.  In a web application, it would be very
slick to set up an interface to the accounting system where you'd just
throw transactions "over the wall," not caring in the slightest _when_
they arrive, so long as you are pretty sure that they _will_ arrive.
Nobody really _cares_ when the data goes into the General Ledger so
long as it arrives some time this year.  (That might exaggerate things
slightly, but it's still _true_.)

What's needed is a "tracked mail server," so to speak, where clients
get to use a sort of "FedEx Tracker" to see if their message has made
it to the destination.  (Designing a system to support that sort of
"audit trail" is almost certainly a "necessary and sufficient"
criterion to make this work.)

Maybe what it needs is to build a "really seriously in user space"
server, using one of the SMTP servers as starting points, with a
database of some sort alongside to manage the tracking information.

Or maybe it would be more appropriate to take one of the "Maildir"
libraries and use that to manage a bunch of queues, alongside a
database that does the "tracking" part.

I don't think SMTP is the right protocol for the purpose; that would
have the downside that it might fool people into thinking this ought
to interoperate well with SMTP.  More likely, it makes sense to have a
couple of protocols:

 a) XML-RPC and/or SOAP for plain ordinary clients to use; verbose,
    but who cares;

 b) Perhaps CORBA for messages servers to use to transfer messages
    back and forth; that's definitely much less verbose...

That's a long way of saying "I agree" :-).
-- 
(reverse (concatenate 'string "ac.notelrac.teneerf@" "454aa"))
http://www.cbbrowne.com/info/corbaalternatives.html#MSGQUEUE
And me, with this terrible pain in all the diodes down my left side...
-- Marvin the Paranoid Android