Race condition when generating .pyc files

[Gabriel Genellina]

En Tue, 07 Oct 2008 12:21:40 -0300, yogamatt1970 at gmail.com  
<yogamatt1970 at gmail.com> escribió:

> I have a large body of Python code which runs on many different (Unix)
> machines concurrently.  Part of the code lives in one place, but most
> of it lives in directories which I find at runtime.  I only have one
> copy of each Python source file and I think I'm hitting a race
> condition where two hosts attempt to import the same module at the
> same time.  My import fails on one of the machines and the following
> exception is thrown:
> EOFError: EOF read where object expected
> My hypothesis is that there's contention between the two (or more)
> hosts when the module's .pyc file is generated.
>
> Possible solutions I see:
> 1) Running a cron job before my code executes which compiles all the
> python source first.

You could make your source directories not writeable by the user under  
those processes run. Then make the cron job at 1) compile all new sources  
(of course *this* process should have write permission). Or better, watch  
those directories for changes and compile when needed.
Use the py_compile module for that.

> 2) Making separate copies of all the .py files for each host running
> the code - I'd rather not do this, it seems like a big pain.
> 3) Inhibiting the generation of .pyc files altogether if that's even
> possible - I saw a PEP for a possible -R flag (http://www.python.org/
> dev/peps/pep-0304/) but I don't think it has been added to a released
> version of Python yet plus I am stuck with Python 2.4 for the time
> being.

Yes, with Python 2.6 (and 3.0) you may use the -B option or set the  
PYTHONDONTWRITEBYTECODE env. var., but 2.4 doesn't support it. And having  
to recompile every module when it is imported may be time consuming.

-- 
Gabriel Genellina