The real problem with Python 3 - no business case for conversion (was "I strongly dislike Python 3")

[Paul McGuire]

On Jul 6, 3:30 am, David Cournapeau <courn... at gmail.com> wrote:
> On Tue, Jul 6, 2010 at 4:30 AM, D'Arcy J.M. Cain <da... at druid.net> wrote:
>
> One thing that would be very useful is how to maintain something that
> works on 2.x and 3.x, but not limiting yourself to 2.6. Giving up
> versions below 2.6 is out of the question for most projects with a
> significant userbase IMHO. As such, the idea of running the python 3
> warnings is not so useful IMHO - unless it could be made to work
> better for python 2.x < 2.6, but I am not sure the idea even makes
> sense.
>
This is exactly how I felt about my support for pyparsing, that I was
trying continue to provide support for 2.3 users, up through 3.x
users, with a single code base.  (This would actually have been
possible if I had been willing to introduce a performance penalty for
Python 2 users, but performance is such a critical issue for parsing I
couldn't justify it to myself.)  This meant that I had to constrain my
implementation, while trying to incorporate forward-looking support
features (such as __bool__ and __dir__), which have no effect on older
Python versions, but support additions in newer Pythons.  I just
couldn't get through on the python-dev list that I couldn't just
upgrade my code to 2.6 and then use 2to3 to keep in step across the
2-3 chasm, as this would leave behind my faithful pre-2.6 users.

Here are some of the methods I used:

- No use of sets.  Instead I defined a very simple set simulation
using dict keys, which could be interchanged with set for later
versions.

- No generator expressions, only list comprehensions.

- No use of decorators.  BUT, pyparsing includes a decorator method,
traceParseAction, which can be used by users with later Pythons as
@traceParseAction in their own code.

- No print statements.  As pyparsing is intended to be an internal
module, it does no I/O as part of its function - it only processes a
given string, and returns a data structure.

- Python 2-3 compatible exception syntax.  This may have been my
trickiest step.  The change of syntax for except from

    except ExceptionType, ex:

to:

    except ExceptionType as ex:

is completely forward and backward incompatible.  The workaround is to
rewrite as:

    except ExceptionType:
        ex = sys.exc_info()[0]

which works just fine in 2.x and 3.x.  However, there is a slight
performance penalty in doing this, and pyparsing uses exceptions as
part of its grammar success/failure signalling and backtracking; I've
used this technique everywhere I can get away with it, but there is
one critical spot where I can't use it, so I have to keep 2 code bases
with slight differences between them.

- Implement __bool__, followed by __nonzero__ = __bool__.  This will
give you boolean support for your classes in 2.3-3.1.

- Implement __dir__, which is unused by old Pythons, but supports
customization of dir() output for your own classes.

- Implement __len__, __contains__, __iter__ and __reversed__ for
container classes.

- No ternary expressions.  Not too difficult really, there are several
well-known workarounds for this, either by careful use of and's and
or's, or using the bool-as-int to return the value from
(falseValue,trueValue)[condition].

- Define a version-sensitive portion of your module, to define
synonyms for constants that changed name between versions.  Something
like:

    _PY3K = sys.version_info[0] > 2
    if _PY3K:
        _MAX_INT = sys.maxsize
        basestring = str
        _str2dict = set
        alphas = string.ascii_lowercase + string.ascii_uppercase
    else:
        _MAX_INT = sys.maxint
        range = xrange
        _str2dict = lambda strg : dict( [(c,0) for c in strg] )
        alphas = string.lowercase + string.uppercase

The main body of my code uses range throughout (for example), and with
this definition I get the iterator behavior of xrange regardless of
Python version.


In the end I still have 2 source files, one for Py2 and one for Py3,
but there is only a small and manageable number of differences between
them, and I expect at some point I will move forward to supporting Py3
as my primary target version.  But personally I think this overall
Python 2-3 migration process is moving along at a decent rate, and I
should be able to make my switchover in another 12-18 months.  But in
the meantime, I am still able to support all versions of Python NOW,
and I plan to continue doing so (albeit "support" for 2.x versions
will eventually mean "continue to offer a frozen feature set, with
minimal bug-fixing if any").

I realize that pyparsing is a simple-minded module in comparison to
others: it is pure Python, so it has no issues with C extensions; it
does no I/O, so print-as-statement vs. print-as-function is not an
issue; and it imports few other modules, so the ones it does have not
been dropped in Py3; and overall it is only a few thousand lines of
code.  But I just offer this post as a concrete data point in this
discussion.

-- Paul