From barry at python.org  Tue Aug 15 13:26:48 2017
From: barry at python.org (Barry Warsaw)
Date: Tue, 15 Aug 2017 10:26:48 -0700
Subject: [Email-SIG] Mailgun email libraries
Message-ID: <A01F43C4-9847-4332-B539-62105BA6EF74@python.org>

Hi folks - apologies for the cross-posting; there is some intersection on the topic so I don?t want to leave anyone out.

This past weekend, I was at PyBay 2017 and attended an excellent talk by Christine Spang, founder and CTO of Nylas, an email syncing platform.  They?ve been using Mailgun?s Flanker library for email address and MIME parsing.  Flanker is Apache 2.0 licensed although unfortunately only Python 2 currently.  Here?s a blog post about it, along with their GitHub repo:

http://blog.mailgun.com/we-just-open-sourced-flanker-our-python-email-address-and-mime-parsing-library/
https://github.com/mailgun/flanker

This is mostly an FYI email since I?ve only had time to read the blog post and briefly skim the docs, but there may be interesting things we can learn from their work.

Cheers,
-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: Message signed with OpenPGP
URL: <http://mail.python.org/pipermail/email-sig/attachments/20170815/5dd82a09/attachment.sig>

From matt at mondoinfo.com  Tue Aug 15 22:35:44 2017
From: matt at mondoinfo.com (Matthew Dixon Cowles)
Date: Tue, 15 Aug 2017 21:35:44 -0500 (CDT)
Subject: [Email-SIG] Mailgun email libraries
In-Reply-To: <A01F43C4-9847-4332-B539-62105BA6EF74@python.org>
References: <A01F43C4-9847-4332-B539-62105BA6EF74@python.org>
Message-ID: <1502849803.99.5827@mint-julep.mondoinfo.com>

[Barry]
> This is mostly an FYI email since I've only had time to read the
> blog post and briefly skim the docs, but there may be interesting
> things we can learn from their work.

I've long been in favor of email module having an RFC 822 address
parser. I've also long been of the opinion that RFC 822 was written
by out-of-work parser writers in the hopes of creating long-term
demand for their services. (See section 3.3, "Lexical Tokens" and
what follows for the origin of that opinion.) That's one reason that
I haven't sat down to write a real one to contribute to the module.

If theirs really does the job correctly, I think it would be a great
idea to steal it.

Regards,
Matt


From rdmurray at bitdance.com  Wed Aug 16 09:20:25 2017
From: rdmurray at bitdance.com (R. David Murray)
Date: Wed, 16 Aug 2017 09:20:25 -0400
Subject: [Email-SIG] Mailgun email libraries
In-Reply-To: <1502849803.99.5827@mint-julep.mondoinfo.com>
References: <A01F43C4-9847-4332-B539-62105BA6EF74@python.org>
 <1502849803.99.5827@mint-julep.mondoinfo.com>
Message-ID: <20170816132025.7D4A01B10015@webabinitio.net>

On Tue, 15 Aug 2017 21:35:44 -0500, Matthew Dixon Cowles <matt at mondoinfo.com> wrote:
> [Barry]
> > This is mostly an FYI email since I've only had time to read the
> > blog post and briefly skim the docs, but there may be interesting
> > things we can learn from their work.
> 
> I've long been in favor of email module having an RFC 822 address
> parser. I've also long been of the opinion that RFC 822 was written

It always did have a minimal one, but it has a much better one now[1]:
email._header_value_parser, which handles more than just address headers.
The code is rather gnarly and needs to be rewritten for efficiency[2].
I would not object to replacing it by something better that someone got
paid to write[3], but that brings up the issue of maintenance...

Note that the new email policies also do one of the other bullet points
from the list in that blog post: the original header is preserved and
as long as it is not changed, the original is emitted on message
serialization.  (Unless you specify to reflow all headers in the
policy; but the default is preserve.)

> by out-of-work parser writers in the hopes of creating long-term
> demand for their services. (See section 3.3, "Lexical Tokens" and

Tell me about it.  On the other hand, once you add "Postel's rule"
processing, which you pretty much have to if you want to handle spam,
you are pretty much required to write a custom parser anyway, no matter
how clean the spec might be.

> what follows for the origin of that opinion.) That's one reason that
> I haven't sat down to write a real one to contribute to the module.
> 
> If theirs really does the job correctly, I think it would be a great
> idea to steal it.

Stealing has costs, but it is worth investigating for sure.

--David

[1] Well, except that unlike the old one the new one doesn't *actually*
handle comments, though the structure is there to support doing so and
they should be simply ignored in the meantime.  But no one has complained
yet that comment support isn't fully implemented.

[2] But before I even think about that I have a patch-in-progress to
finish that rewrites the gnarliest part of the code, which is the header
folding algorithm.

[3] Actually, Blackberry (then RIM) gave me a grant that funded about
half of the development of that parser.  But you get the point :)