hashing strings to integers

[Adam Funk]

On 2014-05-27, Steven D'Aprano wrote:

> On Tue, 27 May 2014 16:13:46 +0100, Adam Funk wrote:

>> Well, here's the way it works in my mind:
>> 
>>    I can store a set of a zillion strings (or a dict with a zillion
>>    string keys), but every time I test "if new_string in seen_strings",
>>    the computer hashes the new_string using some kind of "short hash",
>>    checks the set for matching buckets (I'm assuming this is how python
>>    tests set membership --- is that right?), 
>
> So far so good. That applies to all objects, not just strings.
>
>
>>    then checks any
>>    hash-matches for string equality.  Testing string equality is slower
>>    than integer equality, and strings (unless they are really short)
>>    take up a lot more memory than long integers.
>
> But presumably you have to keep the string around anyway. It's going to 
> be somewhere, you can't just throw the string away and garbage collect 
> it. The dict doesn't store a copy of the string, it stores a reference to 
> it, and extra references don't cost much.

In the case where I did something like that, I wasn't keeping copies
of the strings in memory after hashing (& otherwise processing them).
I know that putting the strings' pointers in the set is a light memory
load.



[snipping the rest because...]

You've convinced me.  Thanks.



-- 
I heard that Hans Christian Andersen lifted the title for "The Little
Mermaid" off a Red Lobster Menu.                         [Bucky Katt]