[issue2676] email/message.py [Message.get_content_type]: Trivial regex hangs on pathological input

[Antoine Pitrou]

Antoine Pitrou <pitrou at free.fr> added the comment:

Hi Jack,

> Antoine, I looked at your patch and I'm not sure why you applied it
> instead of applying mine (or saying +1 on me applying my patch).
> 
> Yours uses str.partition which I pointed out is sub-optimal (same big-Oh
> but with a larger constant factor) and also adds a function that returns
> two things, one of which is thrown away after having a str.strip
> performed on it.

I added that function so that the header splitting facility is
explicitly exposed as an internal API, as was the case with the regular
expression. I tried to mimick the behaviour of the regex as closely as
possible, which meant returning two things as well :-)

I think the point of the issue is to remove the pathological
(exponential) behaviour when parsing some headers, not to try to squeeze
out the last microseconds out of content-type parsing (which shouldn't
be, IMO, the limiting factor in email handling performance as soon as
it's not super-linear).

That said, I've timed the function against the regular expression and
the former is always faster, even for tiny strings (e.g. "a;b").

Your patch was keeping the regular expression as a module-level constant
while replacing all uses of it with a function, which I found a bit
strange (I don't think people are using paramre from the outside since
it's not documented, it's an internal not public API IMO). I also found
it strange to devote a docstring to the discussion of a performance
detail. But I don't have any strong feeling against it either, so you
can still apply it if you think it's important performance-wise.

Regards

Antoine.

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue2676>
_______________________________________