string issue or questions about intern

[jepler epler]

On 14 Jul 2000 16:56:47 -0400, Andrew Kuchling
 <akuchlin at mems-exchange.org> wrote:
>Brian Kelley <kelley at bioreason.com> writes:
>> eval('"AAAAA"')
>> eval('"AAAAB"')
>> eval ('"AAAAC"')
>
>This seems a reasonable outcome of the rule that Python always interns
>string constants.  Every string gets interned, and interned strings
>are never freed, so memory is constantly being consumed.  
>
>If you need full Python expressions, there seems no way around using
>eval().  But if you simply need string escapes, it's probably worth
>writing a parser.  (Does such a parser already exist somewhere in the
>standard library?  Anyone know?) 

Well, the problem is pickle.Unpickler.load_string():
    def load_string(self):
        self.append(eval(self.readline()[:-1],
                         {'__builtins__': {}})) # Let's be careful
    dispatch[STRING] = load_string

You can see that unpickling lots of unique strings causes Python to use
progressively more memory.

Jeff