[Mailman-Developers] Regarding Handlers/SMTPDirect.py and "chunkify"

[Mark Sapiro]

Stefan Förster wrote:
>
>I don't really know how to start this topic, I'm fairly new to Mailman
>and I don't speak anything but the most basic Python, but I am unsure
>whether the function "def chunkify(recips, chunksize):" which is
>defined in "Mailman/Handlers/SMTPDirect.py" might not have an adverse
>effect on mail delivery performance.
>
>The problem is: At the moment, I'm looking at a specific setup without
>knowing anything about the "historical" development of "chunkify".
>There is a comment which mentions an initial suggestion made by Chuq
>Von Rospach and further improvements made by Barry Warsaw.
>Unfortunately, I was unable to find these original discussion, and
>frankly, without knowing why "chunkify" is implemented the way it is,
>I don't think I'm qualified to discuss my concern (which involves VERP
>delivery using a specific MTAs VERP implementation and a highly I/O
>saturated mail server) on this list.


See
<http://www.python.org/cgi-bin/faqw-mm.py?req=show&file=faq04.012.htp>
for some background.


>What I found out from the archives is that "chunkfiy" has two
>purposes:
>
>1. Make sure that not more than mm_cfg.SMTP_MAX_RCPTS are passed to
>the MTA in a single transaction.
>2. To improve delivery performance by grouping destinations which will
>prevent messages which are addressed to dead/congested destinations
>blocking delivery of messages to working destinations.


1. is correct.

2. is only partially correct. The purpose of grouping is to try to
minimize the number of transactions between the outgoing MTA and the
remote MTA. E.g., if all the aol.com addresses are passed from mailman
to the outgoing MTA in one SMTP transaction, then the MTA is able (if
so configured, etc., ...) to deliver the message to aol.com in one
transaction with the same recipients. It has nothing to do with trying
to prevent blocking, as the chunks are delivered serially anyway.

None of this is relevant if your MTA is VERPing the outgoing message,
because the MTA will send each VERPed message to a single recipient
because each recipient's message has a different envelope from
(separate SMTP MAIL FROM command).


>Is this assumption of mine basically correct? If it is and if I didn't
>miss any important parts, I would like to point out a specific
>scenario in which the way "chunkify" is implemented and the way in
>which it is called based on delivery style (verp, bulk) and the
>setting of mm_cfg.SMTP_MAX_RCPTS might actually worsen overall mail
>delivery performance.
>
>If I am totally wrong and "chunkfiy" serves a totally different
>purpose, please feel free to ignore this posting.


I wouldn't say you are totally wrong, but you are not totally correct.
In any case, if you have 'tuning' suggestions, either in general or
for specific cases, we would be glad to receive them and either
incorporate them in the code or add them to the FAQ as appropriate.

-- 
Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
San Francisco Bay Area, California    better use your sense - B. Dylan