Message queueing

[Etienne Labuschagne]

Hi there,

A Python Queue object won't work as this is an object that lives inside your
program's address space.  I need a queue that is used between different
programs (the programs are not all nescessarily written in Python) to
communicate with each other.  The programs can literally sit in different
countries and the messages they send each other must be guaranteed not get
lost on the way.  I must be able to send a message to another program, even
if that program is currently offline (the message is stored until the
program logs on again).  

This can be achieve by using e-mail as transport (I am actually
investigating that avenue on Holger Krekel's advice).  The reason why
programmers use message queueing software rather than e-mail has probably to
do with performance reasons and I also think that message queue connectors
give a more natural programatic interface than packaging your message in a
mime-encoded e-mail message and checking the inbox, etc.

Regards
Etiénne Labuschagne



-----Original Message-----
From: James J. Besemer [mailto:jb at cascade-sys.com]
Sent: Thursday, May 02, 2002 3:44 PM
To: Etienne Labuschagne
Cc: Python List (E-mail)
Subject: Re: Message queueing



This is just a shot in the dark, but have you looked at the "Queue"
module?  Chapter 7.6 in the libary reference.  It's NOT AT ALL a
window's message queue (if that's what you meant) but it can queue
arbitrary data and if you have a place to shove windows messages in a
multi-threading environment you could possibly include the queue in
front to buffer incoming data.

Below is sample code how I used it to queue incoming speech requests to
the sound card.

Hope it helps.

Regards

--jb

--
James J. Besemer  503-280-0838 voice
http://cascade-sys.com  503-280-0375 fax
mailto:jb at cascade-sys.com




import sys, os
import socket, thread
import Queue
import string
from select import *
# the next two are my lower-level library routines
from speak import *
from playsounds import *

# server port offered on localhost
PORT=50011
HOST=""

# two queues connect three threads

ReaderQ = Queue.Queue()
SpeakerQ = Queue.Queue()

# Server thread accepts socket connects and passes
# them on to the Reader.  The idea is to accept sockets
# as fast as possible (FWIW).  Given the small number of
# messages in this case, the separation between Server and Reader
# is of dubious value.

def Server():
        s = socket.socket( socket.AF_INET, socket.SOCK_STREAM )
        s.bind(( HOST, PORT ))

        speak( "Sound server is ready" )

        while 1:
                s.listen(3)
                conn, addr = s.accept()
                ReaderQ.put(( conn, addr ))

# Reader thread reads up all the data submitted by the client
# and forwards it to the Speaker thread.

def Reader():
        while 1:
                conn, addr = ReaderQ.get()
                text = ""
                while 1:
                        data = conn.recv( 1024 )
                        if not data:
                                break
                        text += data

                if text:
                        SpeakerQ.put(( conn, text ))
                else:
                        conn.close()

# Speaker thread actually causes the speech to be emitted.
# This is ultimately done by an external exe.  Speaker can take several
# seconds to perform each action, so the queue here is essential.
# We send a return code back to the client before closing
# the connection.

def Speaker():
        while 1:
                rc = -1
                conn, data = SpeakerQ.get()
                items = data.split( "\n" )

                for item in items:
                        if not item:
                                continue

                        # if it looks like a filename
                        # or has a "-f" prefix
                        # then play WAV file
                        # else speech to text

                        if item[0] in ['.', '/']:
                                rc = playsounds( item )

                        elif len( item ) > 2 and item[:2] == '-f':
                                rc = playsounds( item[2:].strip())

                        else:
                                rc = speak( item )

                        conn.send( str( rc ) + "\n" )

                conn.close()

# launch the threads
# (Server runs as primary thread)

def main():
        try:
                thread.start_new_thread( Reader, ())
                thread.start_new_thread( Speaker, ())

                Server()
        finally:
                speak( "Sound Server is exiting" )

main()


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20020502/19fd90b1/attachment.html>