[issue26415] Out of memory, trying to parse a 35MB dict

[Eryk Sun]

Eryk Sun added the comment:

I don't think this is Windows related. Are you using 32-bit Python? On Linux, if I limit the process address space to 2 gigs, it crashes almost immediately:

    $ ulimit -v 2000000
    $ python-dbg -c 'import crash'
    Segmentation fault

It runs out of memory while parsing the file:

    Program received signal SIGSEGV, Segmentation fault.
    0x000000000048ecb4 in PyObject_Malloc (nbytes=72) at ../Objects/obmalloc.c:932
    932	../Objects/obmalloc.c: No such file or directory.
    (gdb) bt
    #0  0x000000000048ecb4 in PyObject_Malloc (nbytes=72) at ../Objects/obmalloc.c:932
    #1  0x000000000048f8be in _PyObject_DebugMallocApi (id=111 'o', nbytes=40) at ../Objects/obmalloc.c:1469
    #2  0x000000000048fa2b in _PyObject_DebugReallocApi (api=111 'o', p=0x0, nbytes=40) at ../Objects/obmalloc.c:1520
    #3  0x000000000048f83c in _PyObject_DebugRealloc (p=0x0, nbytes=40) at ../Objects/obmalloc.c:1441
    #4  0x0000000000418a02 in PyNode_AddChild (n1=0x7fff85cbffb8, type=318, str=0x0, lineno=1, col_offset=6446977)
        at ../Parser/node.c:98
    #5  0x0000000000418f53 in push (s=0xa6b680, type=318, d=0x8bfc70 <dfas+2480>, newstate=1, lineno=1, col_offset=6446977)
        at ../Parser/parser.c:126
    #6  0x000000000041946c in PyParser_AddToken (ps=0xa6b680, type=262144, str=0x7fff85cba720 "11", lineno=1, 
        col_offset=6446977, expected_ret=0x7fffffffd324) at ../Parser/parser.c:252
    #7  0x0000000000419f19 in parsetok (tok=0xa5f650, g=0x8c0ac0 <_PyParser_Grammar>, start=257, err_ret=0x7fffffffd300, 
        flags=0x7fffffffd2ec) at ../Parser/parsetok.c:198
    #8  0x0000000000419cb6 in PyParser_ParseFileFlagsEx (fp=0xa19b70, filename=0xa53b00 "crash.py", 
        g=0x8c0ac0 <_PyParser_Grammar>, start=257, ps1=0x0, ps2=0x0, err_ret=0x7fffffffd300, flags=0x7fffffffd2ec)
        at ../Parser/parsetok.c:106

----------
components:  -Windows
nosy: +eryksun

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue26415>
_______________________________________