[Mailman-Developers] Thread support

[Ellen Spertus]

My students and I have built a prototype system, Javamlm, supporting user 
subscriptions on a per-thread basis using standard MUAs.    I approached 
Barry about adding threads to Mailman, and he kindly encouraged me to do 
so.  I'm now trying to figure out the best way to add the functionality and 
welcome any suggestions.  You can get the full description of our system at 
http://www.mills.edu/ACAD_INFO/MCS/SPERTUS/Papers/lisa2001.pdf.  I'll 
summarize important features here in the hope of getting feedback on the 
best way to fold them into Mailman.

FUNCTIONALITY

A user of mailing list foo creates a new thread with a base name of bar by 
sending email to foo-new-bar@host.  The system appends an integer to bar to 
generate a unique thread name, e.g., bar1.  All subscribers to mailing list 
foo receive the first message of any thread.  The message headers are:
         To: foo-bar1@host
         From: original-sender@wherever
If a user chooses reply, the new message goes to the original sender 
(assuming a decent MUA).  It a user chooses reply-all, the new message 
becomes part of the thread bar1.

When subscribing (or subsequently), each user specifies whether they want 
to be subscribed to new threads by default.  If so, the user gets all 
messages in a new thread (unless they explicitly unsubscribe).  If a user 
chooses NOT to be subscribed to new threads by default, they only get the 
initial message unless they explicitly subscribe.  Each message contains 
(un)subscribe information both as a URL and mailto to make the operations 
as lightweight as possible; e.g.,

         To unsubscribe from this conversation, send email to 
foo-bar1-unsubscribe@javamlm.mills.edu
         or visit 
http://javamlm.mills.edu/scripts/override?listname=foo&conversation=14&preference=0.
         To unsubscribe from foo, send email to 
foo-unsubscribe@javamlm.mills.edu.

Note that thread membership is determined solely by the address to which a 
message is sent, not the message's subject, references, or other headers.

IMPLEMENTATION

We implemented threads by including a threadID in each message's metadata, 
which was stored in a relational database, as was information about each 
thread, subscription, etc.  The following SQL query determines to whom to 
send a message in thread 37:

SELECT email
FROM Subscriber
WHERE deleted = FALSE AND
((Subscriber.preference = 1 AND
   NOT EXISTS
    (SELECT * FROM Override
     WHERE Override.subscriber_id = Subscriber.subscriber_id
     AND Override.thread_id = 37
     AND Override.preference = 0))
  OR
  (Subscriber.preference = 0 AND
   EXISTS
    (SELECT * FROM Override
     WHERE Override.subscriber_id = Subscriber.subscriber_id
     AND Override.thread_id = 37
     AND Override.preference = 1)))

The full schema is in the paper.  I'm including the query here to give you 
an idea of the complexity and to show that list membership is calculated on 
the fly.

The biggest implementation question in my mind is whether I should store 
subscription information in a relational database (so I can make the above 
efficient query) or build on top of the exiting pickles.

I welcome any comments and pointers.

Ellen