[issue24085] large memory overhead when pyc is recompiled

[Jonathan G. Underwood]

Jonathan G. Underwood <jonathan.underwood at gmail.com> added the comment:

Seeing a very similar problem - very high memory useage during byte compilation.

Consider the very simple code in a file:

```
def test_huge():
    try:
        huge = b'\0' * 0x100000000  # this allocates 4GB of memory!
    except MemoryError:
        print('OOM')
```

Running this sequence of commands shows that during byte compilation, 4 GB memory is used. Presumably this is because of the `huge` object - note of course the function isn't actually executed.

```
valgrind --tool=massif python memdemo.py
ms_print massif.out.7591 | less
```

You'll need to replace 7591 with whatever process number valgrind reports.

Is there any hope of fixing this? It's currently a problem for me when running tests on Travis, where the memory limit is 3GB. I had hoped to use a conditional like the above to skip tests that would require more memory than is available. However, the testing is killed before that simply because the byte compilation is causing an OOM.

----------
nosy: +jonathan.underwood

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue24085>
_______________________________________