[New-bugs-announce] [issue20879] base64 module of Python 3.4 uses 920 kB of memory

[STINNER Victor]

New submission from STINNER Victor:

According to tracemalloc, "import base64" allocates 920.6 kB of memory. The 3 top locations are:

Lib/base64.py:414: size=420 KiB, count=7226, average=59 B
    _b85chars2 = [(a + b) for a in _b85chars for b in _b85chars]
Lib/base64.py:306: size=420 KiB, count=7226, average=59 B
    _a85chars2 = [(a + b) for a in _a85chars for b in _a85chars]
Lib/base64.py:142: size=59.8 KiB, count=1025, average=60 B
    _b32tab2 = [a + b for a in _b32tab for b in _b32tab]

Compare it to Python 3.3: base64 of Python 3.3 only allocates 10.3 kB. (I installed tracemalloc manually on Python 3.3 to get this number.)

I suggest to initialized these precomputed tables to None, and compute them at the first call to b32encode() / b85encode() / a85encode().

The new Base65 and Ascii85 codecs were added by the issue #17618.

_b32tab2 comes from changeset (1b5ef05d6ced) of issue #17812: "quadratic complexity of base64.b32encode(). Optimize base64.b32encode() and base64.b32decode() (speed up to 3x)." So 3.3 branch is also affected (for the base32 table).

----------
files: base64_mem.py
messages: 213018
nosy: haypo, pitrou, serhiy.storchaka
priority: normal
severity: normal
status: open
title: base64 module of Python 3.4 uses 920 kB of memory
type: performance
versions: Python 3.3, Python 3.4
Added file: http://bugs.python.org/file34325/base64_mem.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue20879>
_______________________________________