MemoryError vs malloc error

[Dan Stromberg]

Is the whole program pure python?  Sometimes a reference to undefined memory
or a lack of error checking elsewhere in a program, can cause innocuous code
to fail later: http://stromberg.dnsalias.org/~dstromberg/checking-early.html

If the program uses ctypes, or an unusual Python/C API module, I'd check
there first.

Who compiled the Python interpreter and extension modules, using what
compiler?  At what optimization level?

For the sake of comparison, you might try your program on Pypy or Jython, to
see if:
1) ...they also give an error
2) ...they give a more useful reaction to the error condition
...especially if you're not using any ctypes or uncommon Python/C API
modules.

Are you on a 32 bit FreeBSD or 64 bit?  How much RAM does your machine
have?  How much virtual memory (is it physmem + swap or just swap on
FreeBSD?)?  How big is the list you are join'ing?

Is the error deterministically reproducible?  If yes, and it's not a ctypes
or unusual Python/C API, it might be a unique opportunity to improve
CPython.

What version of CPython (?) are you using?

On Fri, Jul 15, 2011 at 10:51 PM, Amit Dev <amitdev at gmail.com> wrote:

> Hi,
>
> I've a long running python process (running on freebsd). Sometimes when it
> uses too much memory it core dumps. I would've expected it to raise
> MemoryError. Normally, when would a python process raise MemoryError and
> when would it fail with malloc error and cores? This is happening in pure
> python code (Eg. if ' '.join(biglist)) etc.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20110716/f2ea174b/attachment-0001.html>