[issue15822] installed python may fail incorrectly trying to rebuild lib2to3 grammar pickles

[Ned Deily]

Ned Deily added the comment:

The problem here is in the "libinstall" Makefile target and that hidden dependency within lib2to3 that, as Tomi notes, causes lib2to3 to try to recreate the pickles anytime it runs if it thinks they are out of date.  The fixes for Issue15645 cause the recipes for that target to create the two grammar pickles in the source Lib directory before files are copied and installed into the destination lib directories. But there is no guarantee that the source files, Grammar.txt and PatternGrammar.txt, are installed into the destination lib before the corresponding pickle files are.  lib2to3/pgen2/driver compares the value of os.path.getmtime() for the txt and pickle files (assuming both exist).  On many (most) file systems, the copies happen fast enough and the resolution of getmtime is coarse enough that the files are created with the same time value so post-installation use of lib2to3 works OK. But on file systems like ext4 with high-resolution time values, it may happen that one or both of the pickle files is copied first and ends up with a slightly older time value than its corresponding txt file.  In that case the first use of lib2to3 will try to recreate the pickle which will most likely fail with a file permission error since the install directories are typically read-only to non-root users.

The attached patch tries to work around around the problem by touching the two pickle files after they and all the other relevant lib files have been installed.

Since any user could run into this problem on the right configuration, I think the fix (assuming reviewers concur with it) should be cherry-picked for 3.3.0rc2.

----------
components: +Installation
keywords: +patch
nosy: +ronaldoussoren
priority: high -> release blocker
stage:  -> patch review
title: Python 3.3 creates lib2to3 grammar in wrong order -> installed python may fail incorrectly trying to rebuild lib2to3 grammar pickles
versions: +Python 2.7, Python 3.2
Added file: http://bugs.python.org/file27084/issue15822_33.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue15822>
_______________________________________