Python too slow for real world

Tim Peters tim_one at email.msn.com
Tue May 4 00:39:53 EDT 1999


[/F]
> re.py keeps up to 20 patterns in a cache
> (see the _cachecompile function at the
> top of that file).
>
> (however, the cache cleaning code looks
> a bit odd:
>
>     if len(_cache) >= _MAXCACHE:
>         _cache.clear()
>
> maybe an LRU or a
> del _cache[random.choice(_cache.keys())]
> would be better?)

The (deeply!) hidden puzzle here is how to make the cache thread-safe.  You
don't want the overhead of fussing with a lock, and the current mass
deletion doesn't need one.  Wrapping your "del" in a "try: del ... except:
pass" block would work (there may be no keys remaining by the time .keys()
is executed, so random.choice([]) may gripe; and some other thread may have
deleted the same key before you get a chance, so the "del" may gripe too).

If/when re.py is recoded in C, the cache thread-safety problem goes away
(thanks to the global lock).

memory-like-a-threaded-octopus's-ly y'rs  - tim






More information about the Python-list mailing list