[Mailman-Users] Upgrade to version 2.1 - latency on processing incoming messages

[Richard Barrett]

At 09:16 17/06/2003, Ray Gardener wrote:
>Richard,
>
>Deliveries are now flowing quickly again. I noticed an old and orphaned .pck
>file in the qfiles/in directory and deleted it (the system had been halted
>using reboot earlier in the day which may have caused this).  But I didn't
>see any errors related to this file in the logs or on the console. Next time
>I will see what the qrunner -v option tells me ;-)  BTW Is the in/queue
>processing always FIFO in Mailman 2.1?

 From a quick glance at the code the handling of the queues is FIFO.

The queue files have their date/time as part of their file name and this is 
used to order the queue.

>Regards and thanks,
>
>Ray
>
>
>Is the
>----- Original Message -----
>From: "Richard Barrett" <r.barrett at openinfo.co.uk>
>To: "Ray Gardener" <R.A.Gardener at shu.ac.uk>
>Cc: <mailman-users at python.org>
>Sent: Monday, June 16, 2003 5:31 PM
>Subject: Re: [Mailman-Users] Upgrade to version 2.1 - latency on processing
>incoming messages
>
>
> > At 16:25 16/06/2003, Ray Gardener wrote:
> > >Richard,
> > >
> > >the logs indicate that posts are being accepted into the mailman in queue
> > >within a minute from being sent by the MUA.  From the mailman post logs I
> > >can see the messages being sent by out mailman, after which they arriving
>in
> > >the mailing list members mailbox within a minute. So the majority of the
> > >wait seems to be in the mailman system rather than in the surrounding
>mail
> > >systems. (Running "qrunner -r Incoming" invariably causes the  mail to
>much
> > >more quickly.) However I do notice that there is a gap between the time
>the
> > >message arrives in the in queue and it getting logged in the post log
>that
> > >accounts for much of the delay
> > >  I can see currently, files in the qfiles/in directory that have been
>there
> > >for 15 minutes without anything logged for them in either the post or
>smtp
> > >logs.
> > >
> >
> > The post and smtp log entries for a given message are written after the
> > message goes out and give no direct read on the time the local MTA handed
> > over the message to MM and it was inserted into the incoming queue; the
> > MTA's mail log (/var/log/mail ??, depends on your system) is good for
>that.
> >
> > I do not think there is a generic problem for the latest stable 2.1 (MM
> > 2.1.2). I'm currently looking at a lightly loaded (but quite slow
>hardware)
> > MM server running MM 2.1.2 and it is taking just a few tens of seconds to
> > pass messages right through the system.
> >
> > The MM code that delivers to the outgoing MTA (assuming you are using
> > SMTPDirect) does not write to the MM post or smtp logs until delivery to
> > the outbound MTA is complete so there is the real possibility that the
> > delay in processing is not due to MM per se but brought about by delays it
> > is seeing in handing off outgoing mail to the SMTP server.
> >
> > Is your MM handing outgoing mail to the local MTA or are using SMTP to
> > separate outbound mail relay machine? If so, that machine's load may also
> > be affecting things.
> >
> > As an aside, I originally configured the system I am currently looking at
> > to use SMTP to the main outgoing mail relay machine for the site. I
> > subsequently changed it to send out via the local MTA as MM can be a touch
> > aggressive trying to hand off to a sluggish MTA and the main relay machine
> > is running a whole slew of virus and spam checking which slowed it at
>times
> > of high traffic. Going via the local MTA got the mail off MM's hands
> > quickly and it (Sendmail) was better better behaved in dealing with the
> > communication with the main mail relay machine in times of stress. But
>that
> > may not reflect your usage.
> >
> >
> >
> > >----- Original Message -----
> > >From: "Richard Barrett" <r.barrett at openinfo.co.uk>
> > >To: "Gardener, Ray A" <R.A.Gardener at shu.ac.uk>
> > >Cc: <mailman-users at python.org>
> > >Sent: Monday, June 16, 2003 3:09 PM
> > >Subject: Re: [Mailman-Users] Upgrade to version 2.1 - latency on
>processing
> > >incoming messages
> > >
> > >
> > > > At 13:32 16/06/2003, Gardener, Ray A wrote:
> > > > >Hi
> > > > >
> > > > >Last week I upgraded to 2.1 - after a few problem things seems to be
> > > > >almost working fine.
> > > > >However I notice that mails sent in regularly  take over 20 minutes
>to be
> > > > >delivered. The logs directory files for mailman don't show any
> > > > >problems and the system is lightly loaded.
> > > >
> > > > But where is the time delay occurring: between receipt by the local
>MTA
> > >and
> > > > Mailman sending it out, between Mailman handing it off to the outbound
>MTA
> > > > and that passing the mail on. What do the headers on "delayed" mail
>show;
> > > > also the MTA and Mailman's post/smtp logs.
> > >
> > > >
> > > > If anything the latency should be better with 2.1.x than with 2.0.x
> > > >
> > > > Are these posts to lists with many subscribers or do they have some
> > > > characteristic (large posts, many attachments) that might affect their
> > > > handling?
> > >
> > >
> > >No this has been reproduced in my test list with two subscribers and
>small
> > >(< 3k) messages.
> > >
> > >Regards,
> > >
> > >Ray
> > >
> > >Ray
> >
> > --------------------------------------------------------------------------
>----
> > Richard Barrett
>http://www.openinfo.co.uk
> >
> >

------------------------------------------------------------------------------
Richard Barrett                                      http://www.openinfo.co.uk