[Mailman-Users] Upgrade to version 2.1 - latency on processing incoming messages

[Richard Barrett]

At 16:25 16/06/2003, Ray Gardener wrote:
>Richard,
>
>the logs indicate that posts are being accepted into the mailman in queue
>within a minute from being sent by the MUA.  From the mailman post logs I
>can see the messages being sent by out mailman, after which they arriving in
>the mailing list members mailbox within a minute. So the majority of the
>wait seems to be in the mailman system rather than in the surrounding mail
>systems. (Running "qrunner -r Incoming" invariably causes the  mail to much
>more quickly.) However I do notice that there is a gap between the time the
>message arrives in the in queue and it getting logged in the post log that
>accounts for much of the delay
>  I can see currently, files in the qfiles/in directory that have been there
>for 15 minutes without anything logged for them in either the post or smtp
>logs.
>

The post and smtp log entries for a given message are written after the 
message goes out and give no direct read on the time the local MTA handed 
over the message to MM and it was inserted into the incoming queue; the 
MTA's mail log (/var/log/mail ??, depends on your system) is good for that.

I do not think there is a generic problem for the latest stable 2.1 (MM 
2.1.2). I'm currently looking at a lightly loaded (but quite slow hardware) 
MM server running MM 2.1.2 and it is taking just a few tens of seconds to 
pass messages right through the system.

The MM code that delivers to the outgoing MTA (assuming you are using 
SMTPDirect) does not write to the MM post or smtp logs until delivery to 
the outbound MTA is complete so there is the real possibility that the 
delay in processing is not due to MM per se but brought about by delays it 
is seeing in handing off outgoing mail to the SMTP server.

Is your MM handing outgoing mail to the local MTA or are using SMTP to 
separate outbound mail relay machine? If so, that machine's load may also 
be affecting things.

As an aside, I originally configured the system I am currently looking at 
to use SMTP to the main outgoing mail relay machine for the site. I 
subsequently changed it to send out via the local MTA as MM can be a touch 
aggressive trying to hand off to a sluggish MTA and the main relay machine 
is running a whole slew of virus and spam checking which slowed it at times 
of high traffic. Going via the local MTA got the mail off MM's hands 
quickly and it (Sendmail) was better better behaved in dealing with the 
communication with the main mail relay machine in times of stress. But that 
may not reflect your usage.



>----- Original Message -----
>From: "Richard Barrett" <r.barrett at openinfo.co.uk>
>To: "Gardener, Ray A" <R.A.Gardener at shu.ac.uk>
>Cc: <mailman-users at python.org>
>Sent: Monday, June 16, 2003 3:09 PM
>Subject: Re: [Mailman-Users] Upgrade to version 2.1 - latency on processing
>incoming messages
>
>
> > At 13:32 16/06/2003, Gardener, Ray A wrote:
> > >Hi
> > >
> > >Last week I upgraded to 2.1 - after a few problem things seems to be
> > >almost working fine.
> > >However I notice that mails sent in regularly  take over 20 minutes to be
> > >delivered. The logs directory files for mailman don't show any
> > >problems and the system is lightly loaded.
> >
> > But where is the time delay occurring: between receipt by the local MTA
>and
> > Mailman sending it out, between Mailman handing it off to the outbound MTA
> > and that passing the mail on. What do the headers on "delayed" mail show;
> > also the MTA and Mailman's post/smtp logs.
>
> >
> > If anything the latency should be better with 2.1.x than with 2.0.x
> >
> > Are these posts to lists with many subscribers or do they have some
> > characteristic (large posts, many attachments) that might affect their
> > handling?
>
>
>No this has been reproduced in my test list with two subscribers and small
>(< 3k) messages.
>
>Regards,
>
>Ray
>
>Ray

------------------------------------------------------------------------------
Richard Barrett                                      http://www.openinfo.co.uk