Flexible string representation, unicode, typography, ...

[Peter Otten]

wxjmfauth at gmail.com wrote:

> Le dimanche 2 septembre 2012 14:01:18 UTC+2, Serhiy Storchaka a écrit :

>> Hmm, and with locale.strxfrm Python 3.3 20% slower than 3.2.
> 
> With a memory gain = 0 since my text contains non-latin-1 characters!

I can't confirm this. At least users of wide builds will see a decrease in 
memory use:

$ cat strxfrm_getsize.py 
import locale
import sys

print("maxunicode:", sys.maxunicode)
locale.setlocale(locale.LC_ALL, "fr_FR.UTF-8")
words = [
    'noël', 'noir', 'nœud', 'noduleux',
    'noétique', 'noèse', 'noirâtre']
print("total size of original strings:",
      sum(sys.getsizeof(s) for s in words))
print(
    "total size of transformed strings:",
    sum(sys.getsizeof(locale.strxfrm(s)) for s in words))

$ python3.2 strxfrm_getsize.py
maxunicode: 1114111
total size of original strings: 584
total size of transformed strings: 980

$ python3.3 strxfrm_getsize.py
maxunicode: 1114111
total size of original strings: 509
total size of transformed strings: 483

The situation is more complex than you suppose -- you need less dogma and 
more experiments ;)