[Mailman-Users] Mailman is no longer gatewaying news...

[Grant Taylor]

On 11/13/08 02:56, Brad Knowles wrote:
> It sounds like you've already done a lot of the right stuff initially -- 
> tracing packets with tcpdump, etc....
> 
> Beyond that, you kind of just have to get stuck into the Python.

'k

> I was going for bi-directionality.
> 
> Using the server interface to accept articles gives you a much easier 
> and more obvious way to handle re-posts and mail/news/mail loops, using 
> the exact same mechanisms that real news servers use.  With this kind of 
> mechanism, you could even use Mailman to act as a robo-moderator for a 
> newsgroup, in addition to acting as the official gateway.
> 
> Using the server interface to send articles allows you to avoid some 
> extra labels in the path, and gives you more control over the other 
> headers, etc....

*nod*

> But no one else was interested, and I didn't have the skills necessary 
> to do the code myself.  So, the simpler newsreader interface was kept.

Sad but *nod*.

Another trick you could do is to do a small install of INN (or the 
likes) with a configured feed and then configure INN its self (let's see 
if I get this wording right) to use batched or program feeds in to 
Mailman.  This way you could let INN handle the duplication its self. 
You could also have Mailman point back to INN as its out going NNTP 
server and let it do the posting to upstream servers.  Just a thought.

> Sorry, I meant "fromusenet".  It's been a while since I hacked on the 
> mmdsr script, and I had forgotten what the proper name was for that 
> particular log.

*nod*

Below are the top three and bottom three lines in the fromusenet log file.

    Oct 15 02:20:03 2008 (3687) comp.mail.sendmail: [20872..42079]
    Oct 15 02:20:04 2008 (3687) nothing new for list comp.mail.sendmail
    Oct 15 02:20:04 2008 (3687) comp.mail.sendmail watermark: 339203
    ...
    Nov 13 12:00:02 2008 (1003) comp.mail.sendmail: [20872..42368]
    Nov 13 12:00:02 2008 (1003) nothing new for list comp.mail.sendmail
    Nov 13 12:00:02 2008 (1003) comp.mail.sendmail watermark: 339203

Greping through the log file, it looks like the "watermark" has not 
changed.  In fact the only thing that I see that has changed is the 
second number (in the square brackets) on the first like of each log entry.

> Unfortunately, you're not having problems in the incoming direction, are 
> you?

Actually, yes.  Incoming (NNTP -> SMTP) is the problem.  Outgoing (SMTP 
-> NNTP) is working just fine.  Or at least it was when I verified it 
the other day.

> I would be curious to know if you're seeing anything related in the 
> error, post, or queuerunner logs.

The queuerunner log file is 29 lines long, (I'll post them if you want 
me to.)  The lines look like they are from when I last (re)started 
Mailman, "... <bla service> qrunner started." type lines.  There are 
some lines that I don't know if they if they are pertinent or not. 
However I think these lines are from where I stopped Mailman.

    Nov 04 10:36:33 2008 (14630) Master watcher caught SIGTERM.  Exiting.
    Nov 04 10:36:37 2008 (14630) Master qrunner detected subprocess exit
    (pid: 14640, sig: None, sts: 15, class: MaildirRunner, slice: 1/1)
    Nov 04 10:36:37 2008 (14630) Master qrunner detected subprocess exit
    (pid: 14636, sig: None, sts: 15, class: NewsRunner, slice: 1/1)
    Nov 04 10:36:40 2008 (14630) Master qrunner detected subprocess exit
    (pid: 14639, sig: None, sts: 15, class: RetryRunner, slice: 1/1)
    Nov 04 10:36:41 2008 (14630) Master qrunner detected subprocess exit
    ...

> The Python code in /usr/local/mailman/cron/gate_news is actually pretty 
> readable.  A quick scan confirms that the watermark is encoded in the 
> list configuration pickle, but without knowing more about the data 
> structures in the list configuration pickles, I couldn't tell you 
> anything more than that.

Understood.  If you will tell me what (and possibly how to get) you want 
and I'll get it for you.

> Note that there have been some significant security fixes as well as a 
> number of other changes since 2.1.7.  See the latest version of the 
> changes at <http://tinyurl.com/6mprx3>.

I'll take a look.

> Upgrading is usually pretty painless.  I've done it several times for 
> the python.org site where all the official mailman-* mailing lists are 
> hosted, as well as all of the other official Python-related mailing lists.

*nod*

> In looking at the list of changes, I did see two or three mentions of 
> things being fixed related to USENET newsgroups and gatewaying, but I 
> don't know the exact details of what all was fixed, and what other 
> things might potentially have been silently fixed while in the process 
> of fixing other things.

*nod*

>  From what I can tell, everything is in the launchpad tracker.
> 
> The path from <https://launchpad.net/mailman> is not that clear, but if 
> you follow that TinyURL I posted above, that will take you straight to 
> the as-of-now current version of the "NEWS" file that should be updated 
> with every major change to the code, and you should be able to follow 
> the links up or down from there to the actual code in question.

Ok.  I'll take a look and let you know.

Thank you for all your effort.



Grant. . . .