When to compile regex?

[Tom Culliton]

A common performance trick I use is to not just precompile all my
regular expressions, but to even precompute the search or match method
reference (e.g. - asrch = regex.compile(arg).search).  Something that
even caching can't help with.  With a match or search in an inner loop
this can make a significant difference in your throughput, and it's an
optimization that can actually improve readability.

In article <n8vu4.8332$al3.110406 at newsc.telia.net>,
Fredrik Lundh <effbot at telia.com> wrote:
>Tim Ottinger <tottinge at concentric.net> wrote:
>> >In section 4.2.3 of the Python Library Reference (1.5.2) it says that
>> >using compile is more efficient "when the expresion will be used several
>> >times in a single program". I'm trying to figure out the intended
>> >meaning of this. Does that mean "executed more than once", or "occurs
>> >more than once in the program text"? Specifically, if it only occurs
>> >once, but that is in a loop, is there an advantage to compiling before
>> >entering the loop?
>>
>> It has to be compiled before it's used. If you don't compile it, then
>> it will be compiled at the point of use, as a temporary, and then
>> tossed away later.
>
>better make that:
>
>    ...compiled at the point of use, stored in a cache,
>    possibly tossed away later, if the cache fills up.
>
>in 1.5.2, the cache holds 20 regular expressions.  in 1.6,
>it will probably be much larger.