Message queuing

[Noah]

You are right. We did send back confirmations at the application level...
We maintained our own audit trails in mySQL.
So we did most of the work ourselves.
When all is said and I done I suppose we could have
used almost any transport mechanism.
But it did do queuing! You can't deny that!

I've never had need for Quality of Service in the queuing systems
I have used (and QOS was sort of a non-issue at BlueLight.)

SOAP::Lite is Perl. What's the matter with you?
I'd sooner write Java...

That OpenQueue stuff looks interesting. Although it's Java
it does not appear to be a Java only interface.
Last updated nearly two years ago... hmmm... Looks dead.
What else looks fresh?

Yours,
Noah


-----Original Message-----
From: python-list-admin at python.org
[mailto:python-list-admin at python.org]On Behalf Of Christopher Browne
Sent: Thursday, May 02, 2002 7:45 PM
To: python-list at python.org
Subject: Re: Message queueing


Oops! "Noah Spurrier" <noah at noah.org> was seen spray-painting on a wall:
> Message queueing You could also roll your own using Sendmail or
> qmail or some other mail server.  That at least gets you a the
> transport layer. You would then have to add an interface for Python
> or XML-RPC -- and feed mail server error logs back into your system.
> It's easier than it sounds. We did something like this at BlueLight
> (K-Mart) -- of course, maybe that is a bad example ;-) But it
> worked; it was easy to use and fix; and easy to figure out how it
> was working -- unlike a horrible Java wrapped MQ-Series thing they
> also made us use.
>
> Your XML-RPC to Jython/JMS bridge thing sounds elegant compared to
> my idea.  Unless you find a better Message Queue then I think that's
> your best bet.  An XML-RPC interface to JMS almost sounds like a
> product.
>
> It's sad, but the importance of message queues does not seemed to
> have been picked up by the open source community. As far as I can
> tell there are no open source message queues... I don't have a job,
> maybe someone wants to help me write one :-)

The BIG problem with the "rolling via SMTP" is that SMTP doesn't
provide you _any_ kind of end-to-end guarantees on delivery.

SOAP::Lite has transport mechanisms where you can have messages go out
(client side) via SMTP and come in (server side) via POP3; it's kind
of cool stuff, but absent of guarantees of quaility of delivery, it's
of really limited usefulness.  In a web application, it would be very
slick to set up an interface to the accounting system where you'd just
throw transactions "over the wall," not caring in the slightest _when_
they arrive, so long as you are pretty sure that they _will_ arrive.
Nobody really _cares_ when the data goes into the General Ledger so
long as it arrives some time this year.  (That might exaggerate things
slightly, but it's still _true_.)

What's needed is a "tracked mail server," so to speak, where clients
get to use a sort of "FedEx Tracker" to see if their message has made
it to the destination.  (Designing a system to support that sort of
"audit trail" is almost certainly a "necessary and sufficient"
criterion to make this work.)

Maybe what it needs is to build a "really seriously in user space"
server, using one of the SMTP servers as starting points, with a
database of some sort alongside to manage the tracking information.

Or maybe it would be more appropriate to take one of the "Maildir"
libraries and use that to manage a bunch of queues, alongside a
database that does the "tracking" part.

I don't think SMTP is the right protocol for the purpose; that would
have the downside that it might fool people into thinking this ought
to interoperate well with SMTP.  More likely, it makes sense to have a
couple of protocols:

 a) XML-RPC and/or SOAP for plain ordinary clients to use; verbose,
    but who cares;

 b) Perhaps CORBA for messages servers to use to transfer messages
    back and forth; that's definitely much less verbose...

That's a long way of saying "I agree" :-).
-- 
(reverse (concatenate 'string "ac.notelrac.teneerf@" "454aa"))
http://www.cbbrowne.com/info/corbaalternatives.html#MSGQUEUE
And me, with this terrible pain in all the diodes down my left side...
-- Marvin the Paranoid Android