PEP 304 - Controlling generation of bytecode files

[Skip Montanaro]

    >> That outdated .pyc file will get imported.  It's your responsibility
    >> to manage your directory tree properly.  If you think this is a bug,
    >> please file a report on SF (though I don't whoever responds will
    >> consider it a bug).

    Ian> Well, it's somewhat of a bug with the current behavior (IMHO), or
    Ian> at least error-prone, but the bug could be *much* worse if you have
    Ian> pyc files separate from the py files.

Good point.  I will take that into consideration.  Here's a thought off the
top of my head.  (sys.bytecodebase refers to the base of the tree for
writing pyc files and defaults to "/" on Unix.)  To make it a concrete
example, assume that sys.bytecodebase is "/tmp/skip" and consider urllib.py,
located in /usr/lib/python2.3/urllib.py.  The corresponding .pyc files (.pyo
is analogous), can only appear in two places:

    /usr/lib/python2.3/urllib.pyc 

or

    /tmp/skip/usr/lib/python2.3/urllib.pyc 

We always look for the former before the latter.  Suppose the former version
doesn't exist.  We are left with these two files:

    /usr/lib/python2.3/urllib.py
    /tmp/skip/usr/lib/python2.3/urllib.pyc 

While /tmp/skip/usr/lib/python2.3/urllib.pyc is in a different directory
than /usr/lib/python2.3/urllib.py, it's still tied to it.  That is, we need
to make sure it corresponds date-wise with the source file.

Considering the no-source case, suppose we delete
/usr/lib/python2.3/urllib.py, but not
/tmp/skip/usr/lib/python2.3/urllib.pyc.  In this case, we will never even
consider the .pyc file because we never found a corresponding source file in
/usr/lib/python2.3.

To say it another way, I'm *not* proposing that for every directory in
sys.path we add a directory prefixed by the sys.bytecodebase.  That would
make them independent.  The directories in sys.bytecodebase are intimately
associated with the directories listed in sys.path, but completely dependent
on the entries in sys.path.

    Ian> I think the more appropriate behavior would be, at least if
    Ian> bytecodebase was being used, that the .pyc files would only be
    Ian> loaded from that location if the corresponding .py file existed.

Correct.  That is the intention.  I will clarify this in the PEP.

    Ian> I'm not sure if the PEP addressed that either -- if you have a
    Ian> search path with /foo, and a bytecodebase of /pyc, and there's no
    Ian> /foo/bar.py or /pyc/foo/bar.pyc (or /pyc/bar.pyc? either way I
    Ian> guess), then if there's a /foo/bar.pyc will it be found?

Yes.  That's the case today and it won't change tomorrow.  The current
proposal only comes into play when no .pyc file is found in a directory in
sys.path (call it sys.path[i]), but a .py file is.  If that's the case the
directory containing the .py file is prefixed by sys.bytecodebase and an
extra probe is made for a .pyc file.  If none is found, the .py file is
compiled and a .pyc file is written to the directory specified (roughly) by
os.path.join(sys.bytecodebase, sys.path[i]).  If you have no source code,
but an appropriate .pyc file exists within sys.path[i] it will get loaded.

Skip