The email package and KLEZ mails

[Sheila King]

On 29 May 2002 22:17:23 -0400, pinard at iro.umontreal.ca (François Pinard)
wrote in comp.lang.python in article
<mailman.1022725133.16968.python-list at python.org>:

> [Sheila King]
> > [Anthony Baxter]
> > > [François Pinard]
> 
> > > > In my experience, incorrect MIME structure is one of the numerous
> > > > hints about mail being SPAM.  I do not remember a single false positive.
> 
> > > I wish. I have to deal with end-user email, and trust me, it's not all
> > > spam.
> 
> > I concur with Anthony.  I have written an email filter package using the
> > email module and if you use the strict Parser class included in that
> > module, it does throw away too much good email (because any good mail
> > thrown away is too much).
> 
> Maybe the `email' package is stricter than the various MIME processing
> tools that were in Python 1.5.2 in still exist in more recent versions,
> but I would be tempted to think they are of comparable strictness.  I do
> not really know.

Well, no, the email package appears to be stricter. I've used code that had
the old 1.5.2 parser in it, and it "broke" noticeably with the new email
module, due to the strictness.

> The proverb ways that "alike people get together", it might explain why
> I do not see more problems: most of my correspondents have mailer agents
> which do a fair job at MIME generation.  And when MIME mistakes happens,
> it is usually sufficient to raise the subject with my correspondents,
> who are usually happy to get the problem solved at their end.
> 
> Often (but not necessarily), badly structured messages come from people
> who do not care much.  Otherwise, they would have set up themselves better.
> As I much prefer people who care, from my viewpoint, there is a significant
> correlation between a message being MIME-erroneous and a message not being
> worth much interest.

??? I don't understand the point of this? I had an email message I received
from the razor mailing list that couldn't be parsed by the email module.
Now that is a list for people who care very much about email and preventing
spam. So...?

And, if I get emails that don't parse, I should do what? Change my circle
of email friends? Sorry, but your points above are lost on me.

> > Moreover, as I've mentioned in other posts and email correspondence,
> > if you're writing software for end users, you really can't just
> > tell them: "Oh, all those mails that caused errors...they were just
> > non-RFC compliant. Probably SPAM or virus."
> 
> If you are writing filters for everybody, you are probably right.  When I
> write filters for my friends or for myself, in my experience, careless
> MIME may be filtered out as SPAM, and we do not loose much in practice :-).

I have had a different experience than you, as I've pointed out. Most of it
is spam, but some is not.

> > Secondly, why is it that the three other mail readers I use (Agent,
> > Pegasus, and PocoMail) are all able to parse these messages?  I also
> > agree with the idea that applications must be strict in what they write
> > and liberal in what they accept.
> 
> This is a good principle, but only when kept within reasonable bounds.
> Users should be on the side of being strict, and applications should be on
> the side of being liberal.  Users might suffer uselessly by being overly
> ascetic, applications might miss their goal through unlimited friendliness.

I agree.

> For example, I expect compilers to raise diagnostics and help me at being
> strict, because being overly liberal for a compiler is just not helpful.
> Another example, a sad one, is the messy state of HTML all around us,
> it comes from browsers having been by far too liberal, and for too long.

> If mailer agents are very lenient to MIME mis-formatting, they actively
> prevent progress.  They do not really help it, as they trigger confusion.
> Moreover, by implementing MIME poorly, they throw discredit on a good idea.
> MIME standards are not that hard to read, you know.  It is a mystery to
> me why some mail agents mangle the MIME they generate, or miss to assemble
> it conveniently, in the spirit of the standards, at presentation time.

I agree that there is a point where one can go too far. However, I don't
think that the email module is in any danger of that. I've seen quite a few
articles posted here in this newsgroup from people who are having
difficulty parsing emails with that module. It wouldn't be a bad idea to
make it better able to handle some of the offending emails that it
currently cannot handle.

> > I have written a "smart parser" class that I am using in my email
> > filter. I use this class instead of the Parser class provided with the
> > email module.  I provide the code below for all interested parties.
> > [...]  Code follows the signature. Enjoy,
> 
> I'm saving it for possible later use!  Thanks for providing this...


You are welcome. So far as I can tell from reading a few of the messages on
the mimelib developers list, it looks like others who are also interested
in this "problem" will possibly come up with an even better solution in the
not too distant future.

I'm keeping my fingers crossed.


--
Sheila King
http://www.thinkspot.net/sheila/
http://www.k12groups.org/