[Mailman-Developers] Re: Future of pipermail?

[Bill Bumgarner]

Let me describe a bit about what I envisioned when going down the =
Mailman+WebDAV implementation path.  Maybe it is reasonable, maybe not.

I agree with Chuq, Ken and Barry regarding various concerns with =
Mailman.  That the integration of technologies as a part of the required =
baseline makes the tool *less* flexible and *less* easily accesible;  =
that security is a major concern; and that the future of Mailman should =
be about refining/abstracting what we have, not glomming new bits onto =
it.

With that said, let me discuss some of the issues I faced and some of =
the thinking that resulted.  This is in no particular order and is not a =
criticism of anyone's coding contributions!   My code is *not*Not*NOT* =
perfect and it all should be reviewed, as well!!

- archival of messages is a lot more than just writing the bodies to a =
web server and then generating some kind of automatic TOC/index.  MIME =
based email content is not only a reality, but it is often the default.  =
 Mailman needs to more gracefully handle mime-types;  automatic filters =
would be one solution [very desirable for things like a pure discussion =
list] and automatic decoding/filing/handling of attachments is another =
solution [and very desirable for things like project discussion lists, =
etc].

- message rewriting is a really powerful concept and should be an =
abstract and first class part of the Mailman core processing pipeline.   =
When I implemented the decode-and-file-to-WebDAV solution, the first =
challenge was to build a handler that sat in the delivery pipeline and =
did some basic decode-and-rewrite-based-on-mailing-list-configuration as =
the message passed through the pipe.   Once decoded/rewritten, then =
filing/processing/filtering/etc can all be abstracted *away* from the =
actual decode/rewrite process.

- Python's built in MIME type handling code sucks.   It is great if you =
want to *compose* MIME content, it sucks if you want to rip apart =
existing mime content and have any kind of random access.   As such, I =
turned to mimecntl.py by <some very good developer's name I can't =
remember>.   It was quite useful and enabled me to get the job done, but =
does not have the most obvious of APIs.   Did it improve in 2.0?  I =
don't think so, but have not looked closely.

- the current solution for creating an archive is lacking-- that doesn't =
mean it doesn't work.  It works and it works very well, but it is *not* =
modular, does *not* follow the design patterns of the rest of the tool =
and makes it *very* difficult to slap in replacement models for archival =
(has this changed since, say, may?  If it has, I'll have to retract this =
until I look again).   Before WebDAV or anything else can be considered, =
the actual archival process needs to be abstracted away from the hack =
that is in there now.

- gatewaying messages to an HTTP server as they are processed is an =
extremely powerful addition [and very easy to do].   However, it is =
useful to *also* add the ability to automatically requeue messages for =
blastting at the HTTP server if the server returns certain error codes.  =
This is done in the codebase I posted an ftp:// to earlier.  This allows =
for *extremely* flexible integration of Mailman w/other infrastructure.

- WebDAV does *not* address security.   Nor does it contribute to =
"security issues".   Any web based archive of data is only as secure as =
the Web server that allows access to the content.  Given that the Web is =
now the *standard* way of presenting things like Mailing list archives, =
this is a security issue mailman faces *regardless* of the use of WebDAV =
as the means of storing the archive.   WebDAV is merely an extension to =
the existing HTTP protocol.     However, WebDAV does allow for =
interesting potential security implementations;  URLs that automatically =
time out after X minutes/hours/days are trivial to implement.

- for the archival of plain text messages, WebDAV is overkill [as Chuq =
mentions].  However, as soon as you move to archiving mime attachments, =
it quickly becomes extremely advantageous to archive various properties =
with the archived message pieces.   Example;  if my modified Mailman =
receives a message from a Macintosh with an attachment, it stores the =
Creator/Type information [and other extended headers] as a set of =
properties associated with the Resource on the attachment server.   This =
allows for...

- ....restoring decoded attachments and reencoding back to their =
original state with their original headers is an extremely cool feature. =
  With this [in the modified version of Mailman+WebDAV], it is quite =
easy to send a request to Mailman that causes it to reconstitute a =
single attachment in all of its original encoded glory with its original =
headers based entirely on the decoded information found on the WebDAV =
server.   This message can then be automatically sent to the user and =
will be treated just like the original attachment in the original =
message.

- if we are to manage the complexity associated with the integration of =
numerous technologies, it is only going to happen through well refined =
and highly modular APIs....

b.bum