[Python-Dev] Suggestion: new 3 release with backwards compatibility

[Glyph Lefkowitz]

On Jan 5, 2010, at 2:00 PM, Ian Bicking wrote:

> It's not even that easy -- libraries can't apply patches for Python 3 compatibility as they usually break Python 2 compatibility.  Potentially libraries could apply patches that make a codebase 2to3 ready, but from what I've seen that's more black magic than straight forward updating, as such patches have to trick 2to3 producing the output that is desired.

It seems like this is a problem to be addressed, then.  Let's get the "black magic" to be better specified and documented. <http://code.google.com/p/python-incompatibility/> is an interesting start on this, but it would be better if this work could be put into 2to3 fixers as well.

> The only workable workflow I've seen people propose for maintaining a single codebase with compatibility across both 2 and 3 is to use such tricks, with aliases to suppress some 2to3 updates when they are inappropriate, so that you can run 2to3 on install and have a single canonical Python 2 source.  Python 2.7 won't help much (even though it is trying) as the introduction of non-ambiguous constructions like b"" aren't compatible with previous versions of Python and so can't be used in many libraries (support at least back to Python 2.5 is the norm for most libraries, I think).

No-op constructions like 'bytes("")' could help for older versions of Python, though.  A very, very small runtime shim could provide support for these, if 2to3 could be told about it somehow.

> Also, running 2to3 on installation is kind of annoying, as you get source that isn't itself the canonical source, so to fix bugs you have to look at the installed source and trace it back to the bug in the original source.

Given the way tracebacks are built, i.e. from filenames stored in .pycs rather than based on where the code was actually loaded in the filesystem, couldn't 2to3 could do .pyc rewriting to point at the original source?  Sort of like our own version of the #line directive? :)

Seriously though, I find it hard to believe that this is a big problem.  The 3.x source looks pretty similar to the 2.x source, and it's good to look at both if you're dealing with a 3.x issue.

> I suspect a reasonable workflow might be possible with hg and maybe patch queues, but I don't feel familiar enough with those tools to map that out.

This is almost certainly more of a pain than trying to trick 2to3 into doing the right thing.