Python randomly exits with Linux OS error -9 or -15

[Dan Stromberg]

You might try running your Python process with:

  strace -f -s 1024 -o /tmp/script.strace python /path/to/script.py

Then you (perhaps with a C programmer) can likely track down what happened
right before the crash by examining the system call tracer near the end of
the file.

http://stromberg.dnsalias.org/~strombrg/debugging-with-syscall-tracers.html

On Mon, Apr 9, 2012 at 3:01 AM, Janis <janis.viksne at gmail.com> wrote:

> Hello!
>
> I have this problem with my script exiting randomly with Linux OS
> status code -9 (most often) or -15 (also sometimes, but much more
> rarely). As far as I understand -9 corresponds to Bad file descriptor
> and -15 Block device required.
>
> 1) Is there a way how I could find out what exactly causes Python
> process to exit?
> 2) What could be the reason of Python exiting with these status code?
>
> The script is a crawler that crawls several web sites to download web
> pages and extract information. Most often it exits after having run
> for 2 hours and having downloaded ~24 000 files. Some specific web
> sites are more affected than others, i.e., there are other instances
> of the script running in parallel that download more pages and
> complete normally. That could be related to the speed each page is
> returned etc.
>
> I have a try-catch block in the root of the script which works very
> well to catch any kind of exceptions, but in these cases either Python
> does not catch the exception or fails to log it into MySQL error log
> table.
>
> I can not use debugger because the code exits after it has run for an
> hour or more. In order to try to catch the exact place I have put
> logging after each line in some places. The logging prints to stdout,
> to a file and logs the line into MySQL DB. This has revealed that code
> may exit upon a random simple lines such as: time.sleep(0.1) - most
> often, f = opener.open(request) - also often, but sometimes also such
> simple statements as list.add('string').
>
> It can also be that the problem occues and then the script exits upon
> attempt to do any output - stdout (i.e. regular print '1'), writing to
> file and logging into MySQL. I have changed either of these to be the
> first ones in the debug log function, but each time the script did not
> fail in between these lines.
>
> The main libraries that are in use: MySQLDB, urllib and urllib2, but
> none of them seems to be the direct cause of the problem, there is no
> direct call of any of these upon exit. I suspect that this could be
> related to some garbadge collecting process.
>
> Versions tried with: Python 2.6, Python 2.7 (and I think it happened
> also in Python 2.5, but I'm not sure)
>
> Thanks,
> Janis
> --
> http://mail.python.org/mailman/listinfo/python-list
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20120409/ba57f963/attachment-0001.html>