append in IMAP4 from imaplib very slow

[Tim Chase]

On 07/25/12 12:47, Simon Pirschel wrote:
> I'm currently experimenting with IMAP using Python 2.7.3 and
> IMAP4 from imaplib. I noticed the performance to be very bad. I
> read 5000 files from a directory and append them to an IMAP
> INBOX. The hole procedure of reading and appending is taking
> about 210 seconds.
> 
> I set up the exact same code in Perl to check if there is a
> general IMAP server configuration issue, since CPU and I/O isn't
> the problem. The same amount of data on the same IMAP server is
> done after 7.9 seconds using Perl.
> 
> The difference is huge and I tried to narrow the issue down by
> profiling the Python code. The profile results are, 206 seconds
> are spent in calling socket.recv.

While I don't know the ins and outs of the Perl code, it looks like
the Python imaplib.py is doing some pretty straight-forward
send/receive work.  However, IMAP4 does support both pipelining and
multi-append extensions, so if the Perl code takes advantage of the
features if the server offers them, it might be reaping significant
gains over the naïve Python code.

Can you get a debugging dump of the commands being sent each way?
(particularly if you see a bunch of APPEND commands before the reply
acknowledgement, as detailed at [1])

-tkc

[1]
http://www.ietf.org/rfc/rfc3502.txt on page 4