.pyc files??

[Andrew Dalke]

[on when .pyc files are regenerated from .py files]

The .pyc file contains the modification timestamp
in the file itself.  It's this embedded timestamp
which is used to check if the .py needs to be rebuilt,
and not a comparison of the respective timestamps
for the file.  (Surprised me, I also thought it was the
latter.)

The code is in dist/src/Python/import.c.  See
check_compiled_module, which checks the first long
as the magic number then the second long for the
timestamp to use to compare with the mtime of the
.py file.  Here's a way to test it:

   # Create a simple Python file

% rm spam.py*
% cat > spam.py
1
% python
Python 2.4a2 (#1, Aug 29 2004, 22:30:12)
[GCC 3.3 20030304 (Apple Computer, Inc. build 1495)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
 >>> import spam
 >>>
% ls -l spam.py*
-rw-r--r--  1 dalke  staff   2  2 Sep 15:23 spam.py
-rw-r--r--  1 dalke  staff  96  2 Sep 15:23 spam.pyc
%
    # Let's gets some information about the .pyc

% sum spam.pyc
16387 1 spam.pyc
% od -c spam.pyc
0000000    c 362  \r  \n   ? 217   7   A   c  \0  \0  \0  \0  \0  \0  \0
0000020   \0 001  \0  \0  \0   @  \0  \0  \0   s  \b  \0  \0  \0   d  \0
0000040   \0 001   d 001  \0   S   ( 002  \0  \0  \0   i 001  \0  \0  \0
0000060    N   (  \0  \0  \0  \0   (  \0  \0  \0  \0   (  \0  \0  \0  \0
0000100    (  \0  \0  \0  \0   t 007  \0  \0  \0   s   p   a   m   .   p
0000120    y   t 001  \0  \0  \0   ? 001  \0  \0  \0   s  \0  \0  \0  \0
0000140
%

    # modify the mtime for the .py file

% touch spam.py
% python
Python 2.4a2 (#1, Aug 29 2004, 22:30:12)
[GCC 3.3 20030304 (Apple Computer, Inc. build 1495)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
 >>> import spam
 >>>
% ls -l spam.py*
-rw-r--r--  1 dalke  staff   2  2 Sep 15:23 spam.py
-rw-r--r--  1 dalke  staff  96  2 Sep 15:23 spam.pyc
%

    # any changes?  Looks like it.

% sum spam.pyc
21410 1 spam.pyc
% od -c spam.pyc
0000000    c 362  \r  \n   \ 217   7   A   c  \0  \0  \0  \0  \0  \0  \0
0000020   \0 001  \0  \0  \0   @  \0  \0  \0   s  \b  \0  \0  \0   d  \0
0000040   \0 001   d 001  \0   S   ( 002  \0  \0  \0   i 001  \0  \0  \0
0000060    N   (  \0  \0  \0  \0   (  \0  \0  \0  \0   (  \0  \0  \0  \0
0000100    (  \0  \0  \0  \0   t 007  \0  \0  \0   s   p   a   m   .   p
0000120    y   t 001  \0  \0  \0   ? 001  \0  \0  \0   s  \0  \0  \0  \0
0000140
%

    # the difference was the '?' turned to a '\'  in the 5th byte.
    # That probably indicates a difference of 29 seconds.
    # Now modify the .pyc's mtime

% touch 01010101 spam.pyc
% ls -l spam.py*
-rw-r--r--  1 dalke  staff   2  2 Sep 15:23 spam.py
-rw-r--r--  1 dalke  staff  96  1 Jan  2004 spam.pyc
% python
Python 2.4a2 (#1, Aug 29 2004, 22:30:12)
[GCC 3.3 20030304 (Apple Computer, Inc. build 1495)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
 >>> import spam
 >>>
%

    # Notice, no change to the .pyc file.  Its mtime is ignored
    # and the .pyc is not rebuilt.

% ls -l spam.py*
-rw-r--r--  1 dalke  staff   2  2 Sep 15:23 spam.py
-rw-r--r--  1 dalke  staff  96  1 Jan  2004 spam.pyc
% sum spam.pyc
21410 1 spam.pyc
%

    # Another way to see it is with the -v flag to Python.
    # Again, the mtime for the .pyc is << that of the .py file.
    # See that the .pyc is used without a recompile.

% python -v
   [... many lines removed ...]
Python 2.4a2 (#1, Aug 29 2004, 22:30:12)
[GCC 3.3 20030304 (Apple Computer, Inc. build 1495)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
import readline # dynamically loaded from 
/Library/Frameworks/Python.framework/Versions/2.4/lib/python2.4/lib-dynload/readline.so
 >>> import spam
# spam.pyc matches spam.py
import spam # precompiled from spam.pyc
 >>>
   [... many lines removed ...]
%

    # Now tweak the mtime of the .py file and see that
    # the .pyc is regenerated.

% touch spam.py
% python -v
   [... many lines removed ...]
Python 2.4a2 (#1, Aug 29 2004, 22:30:12)
[GCC 3.3 20030304 (Apple Computer, Inc. build 1495)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
import readline # dynamically loaded from 
/Library/Frameworks/Python.framework/Versions/2.4/lib/python2.4/lib-dynload/readline.so
 >>> import spam
# spam.pyc has bad mtime
import spam # from spam.py
# wrote spam.pyc
 >>>
   [... many lines removed ...]


				Andrew
				dalke at dalkescientific.com