[Python-Dev] Fw: Access violation when no memory

[Vladimir Marangozov]

Mark Hammond wrote:
> 
> >From the newsgroup.  Any thoughts?
> 
> Mark.
> 
> "Phil Mayes" <nospam@bitbucket.com> wrote in message
> news:<olD35.82$_%.4481@newsfeed.avtel.net>...
> > The following program progressively allocates all memory:
> >     size = 1 << 20
> >     list = [None,]
> >     while size:
> >         try:
> >             mem = [None] * size
> >             mem[0] = list
> >             list = mem
> >         except:
> >             size = size / 2
> >
> > It fails with an access violation.  (Turn off virtual memory
> > to provoke it.)

I had a closer look at this report. It revealed a buglet in Instance_New
and raises a question on how to handle the possible NULL values pushed
on the stack in ceval.c, at the exception handling code block:

> > 3. the NULL pointer at tb is pushed here:
> >      ceval.c line 1816:    PUSH(tb);
> >      ceval.c line 1817:    PUSH(val);
> >      ceval.c line 1818:    PUSH(exc);

1. In PyInstance_New, when the inst->in_dict allocation fails, there's
   a DECREF(inst) which triggers the instance_dealloc function. The latter
   goes on checking whether __del__ is defined in the instance's dict
   (by calling getattr2) whithout checking first whether inst->in_dict
   is not NULL. I.e. the dealloc_function assumes the instance has been
   initialized successfully. The net effect is a segfault in PyDict_GetItem
   which is called with a NULL dict.

   There are two possible fixes, and I'm not sure which one is better:

   a) Fix the Instance_New code, by freeing the instance on inst->in_dict
      allocation failure (without decref'ing the instance).

   b) Fix the dealloc code, by checking whether in_dict is not NULL
      before proceeding with the __del__ finalization logic.

   Opinions?

2. In ceval.c, there are indeed cases where 'tb' and 'exc' are NULL on
   low-memory conditions. They are pushed on the stack, then popped with
   a segfault.  The only reasonable behavior I get, after some testing,
   is by setting the tb and exc to Py_None if any of them is NULL before
   pushing them onto the stack. The program succeeds or the process is
   killed by the OS (tested on Linux).

   However, I'm not sure that setting tb and/or exc to Py_None, if NULL,
   makes any sense and won't cause side effects.

   Any other suggestions?

-- 
       Vladimir MARANGOZOV          | Vladimir.Marangozov@inrialpes.fr
http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252