 tag contents (body of message) right at the NUL although it does
start displaying again at the  tag.

On MSIE 5.5, although the NUL byte isn't displayed, it seems to show
as "==><==" with the rest of the message being displayed just fine.

So I'd have to say at this point that if there's a bug anywhere, it's
in Pipermail's generation of the indexes.  Oddly enough, the subject
index seems fine, but thread, date, and author seem hosed.  I'll
submit a bug report on SF for this.

Other than that, I think Mailman is doing what it should be doing.

-Barry

From bwarsaw@beopen.com  Thu Sep 21 20:46:29 2000
From: bwarsaw@beopen.com (Barry A. Warsaw)
Date: Thu, 21 Sep 2000 15:46:29 -0400 (EDT)
Subject: [Mailman-Developers] Building a boucing detector
References: <39C96137.F84BD172@samba-choro.com.br>
Message-ID: <14794.26005.166676.432115@anthem.concentric.net>

>>>>> "PEN" == Paulo Eduardo Neves  writes:

    PEN> Hi, In my site, I send a lot of emails to users. I'd like to
    PEN> build a software to detect bouncing email, and automatically
    PEN> take some actions, like removing the user from the
    PEN> database. I'm considering using mailman code to do it. Why
    PEN> build, if you can steal?:-)

Absolutely!  And feel free to contribute back any new bounce
dectectors you develop; it's a never ending battle with MTA authors
who feel the need to be clever and unique. ;/

    PEN> After a quick look in the code, here is my approach. I would
    PEN> like very much to hear people more knowledgeable about
    PEN> Mailman code criticize it.

    PEN> There's two spots that handle bouncing email. The files in
    PEN> directory Bouncers/ identify if a message is a
    PEN> bounce. Calling BounceAPI.py with a message instance and a
    PEN> list instance calls the RegisterBounce method in list if the
    PEN> email is a bounce.

    PEN> Bouncer.py is the mix-in class that handle the bounces and
    PEN> calls some methods of its sibiling and subclasses
    PEN> (e.g. IsMember and Save).

    PEN> I'd put a pipe in the alias file to pass it to a simple
    PEN> program that builds the message object, the list object and
    PEN> pass them to BounceAPI.

    PEN> Subclass Bouncer class in Bouncer.py to redefine list objects
    PEN> methods that would be defined in other classes. Some would be
    PEN> dummy methods, others do the real action.

    PEN> Any opinions? Is there any trap?

I think this is more work than you need to do actually.  If you look
at ScanMessages() in BouncerAPI.py, it really does two things:

1) iterate through the list of bounce detectors until one of them
  returns a match

2) calls into the MailList object to "register" the bouncing
  addresses

#2 probably has little use for you outside the context of Mailman, so
if all you're interested in is the bounce detection framework, I'd say
you should just re-implement ScanMessages() to do something more
useful for your application.

Forget the MailList instance; all you really need is a Message object.
I think, but am not positive, that Python's standard rfc822.Message
class should suffice here, although you'll need to be sure that file
object you pass to the constructor is rewindable.  Some sys.stdins are
not, and if that's the case for you, I suggest reading all of
sys.stdin into a StringIO instance and passing /that/ to the
rfc822.Message constructor.

You'll probably need to change the __import__() call in ScanMessages()
because your copy probably won't live in the Mailman.Bouncers package.

The API to the individual bounce detectors is this:

- the module must have a module-global function called process()
- process() must take a single argument, the Message object
- process() returns None (or any false value) to indicate it could not
  find a matching address in the message
- process() returns a list of matching addresses if it does find a
  match

The algorithm in ScanMessages() simply iterates through every bounce
detector until one of them returns a non-false value.  Given that list
of addresses, you can then do whatever is appropriate for your
application.

Hope that helps,
-Barry

From R.Barrett@ftel.co.uk  Fri Sep 22 11:53:28 2000
From: R.Barrett@ftel.co.uk (Richard Barrett)
Date: Fri, 22 Sep 2000 11:53:28 +0100
Subject: [Mailman-Developers] internal mail archiver/htdig integration
Message-ID: 

In commissioning Mailman I needed to provide a per mail list search 
facility which fully honored private list access constraints (i.e. if 
the use cannot read it then do not tell him it even exists in search 
results). In implementing this I have made some changes to the 
mailman-2.0beta5 source. What I have done is hardly rocket-science. 
That said it would be good for me if the work was integrated with the 
standard releases of Mailman so I do not have to keep applying my own 
unique patches to any subsequent releases. I don't know the protocol 
for doing this nor whether other developers think my work is worth 
incorporating into the mainline of Mailman's code. I've outlined the 
changes below and would appreciate some advice. btw: If there's a 
better way to achieve the same objectives then I'm more than happy to 
throw my changes away.

I've integrated Mailman's internal archiver with htdig. Use of htdig 
is conditional on a new Mailman configuration variable being set, the 
use of the internal archiver and a list being subject to archiving. 
If these conditions are met then the relevant list-specific htdig 
conf files and such are created and maintained automatically. If you 
don't want the integration then turning of a single Mailman config 
variable makes this integration completely transparent.

For qualifying lists the htsearch form is embedded in the list's 
archive index page, when that is generated by HyperArch.py, and hence 
is downstream of user authentication for accessing a private archive. 
Access to URL's returned by htsearch are all mediated by an 
additional wrapped cgi script called htdig.py which is similar to 
private.py. Similar in the sense that for private lists it requires 
the presence of a list authentication cookie or it rejects the 
access. For public lists it just responds with the html page 
requested. In the normal course of events the user's browser gets the 
authentication cookie on board while reaching the archive index page, 
to do a search, in the normal fashion. This approach means that 
changing a list from public to private or vice versa does not 
invalidate the htdig search files as access to archive html files in 
either type of list uses a common root in their URLs.

An additional directory called htdig is created under each 
.../mailman/archives/private/ in which the htdig search 
files and configuration file for each list are stored.  An additional 
directory called .../mailman/archives/htdig holds symlinks to the 
list-specific htdig configuration files. .../mailman/archives/htdig 
is itself the target of a symlink in the directory in which htdig is 
configured to find its configuration files. [These symlinks are to 
allow htsearch to locate the list specific htdig conf files.] This 
directory arrangement meets that deleting a list in the normal 
fashion also disposes  of related search stuff without any further 
effort or thought.

The whole is rounded off with a cron initiated script which uses 
rundig to update list-specific search files on a regular basis iff 
each given list archive has had more data added to it since the 
script last ran.

Changes have been made to the following mailman-2.0beta5 source files:

Hyperarch.py

1. extra function to set up list-specific htdig creates list's htdig 
directory and generates list's htdig conf file in that directory. 
Logic added so that this function is called when archiving for the 
list is commenced.
2. added meta tags and and 