[New-bugs-announce] [issue40313] bytes.hex(sep, bytes_per_sep) is many times slower than manually inserting the separators

[Antony Lee]

New submission from Antony Lee <anntzer.lee at gmail.com>:

Consider the following example, linewrapping 10^4 bytes in hex form to 128 characters per line, on Py 3.8.2 (Arch Linux repo package):

    In [1]: import numpy as np, math

    In [2]: data = np.random.randint(0, 256, (100, 100), dtype=np.uint8).tobytes()                  

    In [3]: %timeit data.hex("\n", -64)
    123 µs ± 5.8 µs per loop (mean ± std. dev. of 7 runs, 10000 loops each)

    In [4]: %timeit h = data.hex(); "\n".join([h[n * 128 : (n+1) * 128] for n in range(math.ceil(len(h) / 128))])
    45.4 µs ± 746 ns per loop (mean ± std. dev. of 7 runs, 10000 loops each)

    In [5]: h = data.hex(); "\n".join([h[n * 128 : (n+1) * 128] for n in range(math.ceil(len(h) / 128))]) == data.hex("\n", -64)                                                                       
    Out[5]: True

(the last line checks the validity of the code.)

It appears that a naive manual wrap is nearly 3x faster than the builtin functionality.

----------
components: Library (Lib)
messages: 366678
nosy: Antony.Lee
priority: normal
severity: normal
status: open
title: bytes.hex(sep, bytes_per_sep) is many times slower than manually inserting the separators
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue40313>
_______________________________________