[Python-Dev] PyRun_ with file name

["Martin v. Löwis"]

Tim Peters wrote:
> If you understand the problem, and believe it will fix it, +1 (I don't
> think I fully understand the problem, but it gets points from me for
> being a small change).

To fully understand it, you need to spend three days in a locked room
together with the VC debugger :-) The submitter claims that VC7 has
changed the CRT ABI, I believe this is incorrect. Instead, the problem
results from having to CRT copies simultaneously, even though they all
have the very same code sice ages. Here is the full story:

- there are two file structures, struct FILE and struct _FILEX, see
   internal.h. _FILEX is

   typedef struct {
         FILE f;
         CRITICAL_SECTION lock;
   }   _FILEX;

- For historical reasons, the lock couldn't be added to the existing
   FILE objects (stdin, stdout, etc) since compiled code *knew* how
   large struct FILE is, and where therefore _iob[2] (say) is.

- To solve this, MS added a second array of CRITICAL_SECTION (see
   mlock.c), and a helper function _lock to lock one of these.

- Now, _lock_file becomes (see _file.c)

   void __cdecl _lock_file (void *pf)
   {
     if((pf >= (void *)_iob) && (pf <= (void *)(&_iob[_IOB_ENTRIES-1])))
           _lock( _STREAM_LOCKS + (int)((FILE *)pf - _iob) );
     else
           EnterCriticalSection( &(((_FILEX *)pf)->lock) );
   }

   So essentially, if it is a builtin FILE, find its lock elsewhere,
   otherwise, it is a _FILEX, and find its lock inside the FILE itself.

- Suppose we have an embedded Python, and the container is compiled
   with VC7 (or 5, doesn't matter). The host application does

     FILE *f = fopen("some file", "r");
     PyRun_SimpleFile(f);

   This allocates a builtin FILE (say, fileno 5) inside the VC7 CRT, and
   passes it to python23.dll. We do fread, and fread (from the VC6 CRT)
   does _lock_file. This checks whether this is a builtin FILE, which it
   isn't, so the VC6 CRT assumes it is a FILEX, and uses the
   CRTITICAL_SECTION. Of course, this is really the storage of fileno 6
   of the VC7 CRT, which is not an initialized CRTICAL_SECTION.

The end result is that the _lock_file operation crashes. As MSDN
explains, you must not pass FILE* across CRTs. The proposed patch avoids
the problem by passing file names instead.

Regards,
Martin