[Mailman-Developers] Tweaking Mail->News Gateway

[Michael Heydekamp]

Hi Mailman developers,

we at freexp.de are setting up Mailman and INN and are trying to get the
Mail<->News facility to work.  We are neither familiar with Mailman nor
with Python, just being poor old Pascal programmers. ;)

I have been in touch with Brad Knowles before and he advised me to ask
here.

We have three mailing lists and three corresponding newsgroups, which
shall be gated bidirectional.  It even works but what is puzzling us is
that in Mail->News direction Mailman always creates an own MsgID.  This
screws up threading by references: If a newsgroup user replies to a
posting that has been gated from the mailing list to the newsgroup, then
- from the perspective of the mailing list users - his references point
to a "wrong" MsgID when they receive this followup posting being gated
from the newsgroup to the mailing list.

I found the respective code in Queue/NewsRunner.py and also read the
explanation why Mailmain is creating own MsgIDs (I'll come back to this
later on).  I changed the code as follows:

----------8<----------
    # Our Message-ID format is <mailman.secs.pid.listname at hostname>
    msgid = msg['message-id']
    hackmsgid = True
    if msgid:
        hackmsgid = False
    #     mo = mcre.search(msgid)
    #     if mo:
    #         lname, hname = mo.group('listname', 'hostname')
    #         if lname == mlist.internal_name() and hname == mlist.host_name:
    #             hackmsgid = False
    if hackmsgid:
        del msg['message-id']
        msg['Message-ID'] = Utils.unique_message_id(mlist)
----------8<----------

It is not really clear to me how Python can know where a conditional
statement such as "if msgid:" ends (just by the indention?), but this
piece seems to work: The MsgID is now being left untouched.

This is question #1: Is this code OK?  I'm also not sure if the variable
'msgid' contains a boolean or a string...?


Now to the more important issues:

I understand that there are two main reasons why Mailman wants to create
an own MsgID upon gating Mail->News, one of them being to avoid loops.

Even after I changed the code as above, I still can't see a loop.  I
first thought Mailman might check the MsgID of a posting upon deciding
whether it should be gated from the newsgroup to the mailing list, but
this is obviously not the case.  I've had a look into cron/gate_news.py
but couldn't figure out exactly how Mailman is taking that decision.

This is question #2: How does Mailman know if the mailing list has
already seen a posting or not?


Anyway, the second and more important reason for creating own MsgIDs
seems to be the issue of an XPost to two (or more) Usenet-gated mailing
lists.

This is indeed a problem and I understand the background.  Thus we'd
like to tweak Mailman in a way that it does the following:


1. Say we have the mailing lists ml-1, ml-2 and ml-3 and the
   corresponding newsgroups ng-1, ng-2 and ng-3.  Now a mail comes in,
   being addressed to ml-1 and ml-2 (which thus should be gated to ng-1
   and ng-2).

   In fact, two physical mails will come in, and obviously Mailman does
   now create two physical postings, one addressed to ng-1, the other
   one addressed to ng-2.  This in fact creates the problem of duplicate
   MsgIDs.

   But what Mailman should do is to create *one* physical posting,
   x-posted to ng-1 and ng-2 ("Newsgroups: ng-1,ng-2").  This would mean
   that upon dealing with the mail to ml-1 it would have to check if the
   addresses in the To: and Cc: headers contain more than one of "our"
   mailing lists and if these mailing lists are being gated to "our"
   Newsgroups.  If so, then an x-posting as described shall be created.

   When Mailman now has to deal with the second mail to ml-2 (with the
   same MsgID) it shall *not* create another posting anymore.

   Can this be accomplished (and if yes, how)?


2. Of course, if a mail does not have a MsgID at all, then Mailman
   should still create its own MsgID.


Any hint will be appreciated (and again, please consider that we are
newbees with regards to Mailman and Python).  Even if we would have to
hardcode some strings (of mailing lists and/or newsgroups) this would be
a better solution than no solution.


        Michael