Bytes indexing returns an int

[wxjmfauth at gmail.com]

Le mercredi 8 janvier 2014 12:05:49 UTC+1, Robin Becker a écrit :
> On 07/01/2014 19:48, Serhiy Storchaka wrote:
> 
> ........
> 
> > data[0] == b'\xE1'[0] works as expected in both Python 2.7 and 3.x.
> 
> >
> 
> >
> 
> I have been porting a lot of python 2 only code to a python2.7 + 3.3 version for 
> 
> a few months now. Bytes indexing was a particular problem. PDF uses quite a lot 
> 
> of single byte indicators so code like
> 
> 
> 
> if text[k] == 'R':
> 
>     .....
> 
> 
> 
> or
> 
> 
> 
> dispatch_dict.get(text[k],error)()
> 
> 
> 
> is much harder to make compatible because of this issue. I think this change was 
> 
> a mistake.
> 
> 
> 
> To get round this I have tried the following class to resurrect the old style 
> 
> behaviour
> 
> 
> 
> if isPy3:
> 
> 	class RLBytes(bytes):
> 
> 		'''simply ensures that B[x] returns a bytes type object and not an int'''
> 
> 		def __getitem__(self,x):
> 
> 			if isinstance(x,int):
> 
> 				return RLBytes([bytes.__getitem__(self,x)])
> 
> 			else:
> 
> 				return RLBytes(bytes.__getitem__(self,x))
> 
> 
> 
> I'm not sure if that covers all possible cases, but it works for my dispatching 
> 
> cases. Unfortunately you can't do simple class assignment to change the 
> 
> behaviour so you have to copy the text.
> 
> 
> 
> I find a lot of the "so glad we got rid of byte strings" fervour a bit silly. 
> 
> Bytes, chars,  words etc etc were around long before unicode. Byte strings could 
> 
> already represent unicode in efficient ways that happened to be useful for 
> 
> western languages. Having two string types is inconvenient and error prone, 
> 
> swapping their labels and making subtle changes is a real pain.
> 
> -- 


--

Byte strings (encoded code points) or native unicode is one
thing.

But on the other side, the problem is elsewhere. These very 
talented ascii narrow minded, unicode illiterate devs only
succeded to produce this (I, really, do not wish to be rude).

>>> import unicodedata
>>> unicodedata.name('ǟ')
'LATIN SMALL LETTER A WITH DIAERESIS AND MACRON'
>>> sys.getsizeof('a')
26
>>> sys.getsizeof('ǟ')
40
>>> timeit.timeit("unicodedata.normalize('NFKD', 'ǟ')", "import unicodedata")
0.8040018888575129
>>> timeit.timeit("unicodedata.normalize('NFKD', 'zzz')", "import unicodedata")
0.3073749330963995
>>> timeit.timeit("unicodedata.normalize('NFKD', 'z')", "import unicodedata")
0.2874013282653962
>>> 
>>> timeit.timeit("len(unicodedata.normalize('NFKD', 'zzz'))", "import unicodedata")
0.3803570633857589
>>> timeit.timeit("len(unicodedata.normalize('NFKD', 'ǟ'))", "import unicodedata")
0.9359970320201683

pdf, typography, linguistic, scripts, ... in mind, in other word the real
*unicode* world.

jmf