[Python-Dev] Python 3.0.1 (io-in-c)

[Antoine Pitrou]

Daniel Stutzbach <daniel <at> stutzbachenterprises.com> writes:
> 
> What kind of input are you using for the Text tests?  I'm kind of surprised
that the conversion to Unicode results in such a dramatic slowdown, if you're
feeding it plain text (characters 0x00 through 0x7f).

It's some arbitrary text composed of 95% ASCII characters and 5% non-ASCII. On
this specific example, utf8 decodes at around 250 MB/s, latin1 at almost 1 GB/s
(on the same machine on which I ran the benchmarks).

You can find the test here:
http://svn.python.org/view/sandbox/trunk/iobench/