[Python-Dev] [python-committers] Proposed Python 3.0 schedule

[Mark Hammond]

> > * Better support for 2to3 in distutils (specifically, the support in
> > build_py is stale, plus 'build_scripts' and 'install_data' should
> > convert
> > .py files to py3k syntax.)
> 
> Please do create a bug report for that. It sounds like it's easy to
> fix.

Yeah, build_py is fairly easy to fix, but I also needed to extend the
support to build_scripts and install_data.  In addition, some already
reported bugs in 2to3 mean that some files fail to convert, and this breaks
the entire process - so as a result I ended up duplicating lib2to3's
'refactor_items()' but with exceptions being logged and ingored rather than
aborting the process.  Oh - and I deleted the .bak files (a copy of the
sources are converted, not the sources themselves)

Please see bugs 4072 and 4073  - but as mentioned below, the lack of a test
case means I didn't supply a tested patch.

> > An 'example' project that uses py2k syntax and
> > "just works" on py3k using this strategy might be useful here.
> 
> Perhaps pywin32 :-?
> 
> I don't think a demo project would do much good, as it doesn't exercise
> all the issues that may occur.

My idea was that the demo project would simply demonstrate the 2to3 concepts
that such a project could use.  pywin32 isn't a good example as it has a
very non-trivial setup.py and a large set of C extensions (the demo I had in
mind could avoid C extensions completely - C developers will already assume
#ifdef will be their friend, but .py code is the unknown...)

It would basically be a 'distutils demo', could have a single .py module and
a single .py script.  setup.py would support both 2.x and 3.x and would
demonstrate how the source is converted to py3k syntax before it is
installed into the py3k distribution.

It would also provide a useful test case - eg, for the distutils bug above,
I'm not sure how I can (a) demonstrate it is currently broken and (b)
demonstrate a patch corrects the problem.

> > * A standard 'helper script' that allows people to use py3k to
> > execute a py2x syntax script by auto-converting the code.  I've 
> > a 10ish-line script that uses lib2to3 plus exec() to achieve that 
> > result, but a helper in 2to3
> > for this would be nice.  For a concrete use-case, we want to keep our
> > distutils script in py2x syntax, but execute it via py3k.  Its very
> > possible this already exists and I've just missed it...
> 
> For the case of setup.py, I was hoping that it could be written in
> compatible syntax even without needing conversion. That worked fine for
> my Django port. Is that not the case for pywin32?

setup.py catches and examines some exceptions.  Consider the more general
case though - pywin32 has a number of tests all of which will also be
maintained in py2x syntax.  It is extremely convenient to be able to
execute:

% py3k run2.py my_test.py etc

And have 'my_test.py' (which is 2.x syntax) be executed directly by py3k
without doing a full 'setup.py install' or manually invoking 2to3 via a temp
file, etc.  As mentioned, 'run2.py' is quite short and just uses
lib2to3+exec, but I'm not sure everyone will work out how to roll their
own...

Specifically, I believe that a script with similar capabilities could be
installed with py3k in the "scripts" directory and it advertised as a
reasonable way to directly execute your *scripts* which, although py3x
compatible, are being maintained in py2x syntax.  Below is my quick attempt
at such a script, which I promptly stopped looking at as soon as it worked
(ie, I'm not sure if all those options are needed, etc), but it does let me
execute my tests using py3k directly from the source tree.
 
Cheers,

Mark

---
# This is a Python 3.x script to execute a python 2.x script by 2to3'ing it.
import sys
from lib2to3.refactor import RefactoringTool, get_fixers_from_package

fixers = get_fixers_from_package('lib2to3.fixes')
options = dict(doctests_only=False, fix=[], list_fixes=[], 
               print_function=False, verbose=False,
               write=True)
r = RefactoringTool(fixers, options)
script = sys.argv[1]
data = open(script).read()
print("Converting...")
got = r.refactor_string(data, script)
print("Executing...")
# nuke ourselves from argv
del sys.argv[1]
exec(str(got))
---