Python 2 to 3 conversion - embrace the pain

[Paul Rubin]

Steven D'Aprano <steve+comp.lang.python at pearwood.info> writes:
>> Why did the changes have to be introduced at all?
> For the same reason any improvement and functional update is
> introduced. To make a better language.

There comes a point when you decide that maintaining existing code is
important enough that you have to stop breaking things.  That point is
called "maturity", and it's fine if you decide that your language
doesn't have it.  Haskell for a long time had a motto "avoid success at
all costs", i.e. it had an explicit goal of being a research testbed for
language geeks and its designers wanted to be able to change stuff
without alienating a big user community.  Python claimed

> If we were designing Python from scratch today, here are some of the
> changes we would certainly make: [mostly good changes]

I agree with most of those changes and I'd add some of my own that are
even more radical.  But, they shouldn't be made in dribs and drabs in
the production releases: it's better to make a fork of the language that
does all the changes at the same time.  Think of C++ as a fork of C in
that manner, although it suffered a lot from trying to be a superset.

> If only we could turn back the clock and redesign it with the benefit
> of hindsight, without worrying about backwards compatibility!

We are currently getting the disadvantages of both approaches.  We break
stuff enough to inflict pain on users, without bringing enough benefits
to make them want to accept the pain.

> design fixes all in one lump or spread out of twenty versions, you
> still have to break backwards compatibility, or keep carrying the
> obsolete code forever.

If people are still using it then it's not obsolete.  Code is only
obsolete when the last user is dead.

> Of course, not *all* C code written in the 1980s still works fine,

Can you give an example of some way C changed that broke old code that
was written in accordance with the language manuals?  Of course tons of
C code then and now used implementation-dependent hacks that went
outside the documented behavior, but that doesn't count.  

> but your point is taken. C, like Fortran and Java, have an even
> stricter view of backward compatibility than Python.

Right, they take their deployed base seriously, while Python has been
more like an experimental language by comparison.

> The downside of that is that learning and using C means you have to
> know and learn a whole lot of ugly, dirty cruft, if only so you know
> what it means when you see it in somebody else's code. Long after
> feature X has been made obsolete, people still need to deal with it,
> *even if nobody uses it*.

Can you give an example?  I wouldn't count things like gets, which
aren't as much changes in the language, as recognition that using it was
buggy from the start.

> And they will. We both know that no matter how bad and foolish a
> feature is, no matter how loudly you tell people not to use this
> deprecated, insecure, unsafe feature, until it is actually gone people
> will continue to write new code using it.

I haven't noticed that much in C.  Maybe it's more of an issue with C++.

> You can't please everybody. Some people want Python to change even
> faster, some want Python to stop changing so fast...

I don't notice anyone wanting Python to change faster, in the sense of
breaking existing code more often.  Maybe there are some specific
proposals that I missed.

> Programmers... are like cats in that they have a deep distrust for
> anything too different. "Give me something completely new, only
> exactly the same as the old one."

It's more like "don't break the old thing unless the new thing is so
much better that I want to change over anyway".  Python to Erlang or
Ocaml or Go or Haskell are far more drastic changes than Python 2 to
Python 3, but at the same time possibly more attractive.

> I find it hard to imagine anyone seriously preferring Javascript to
> Python because Python has too many language warts. 

There are only a few things about raw JS that are really warty and hard
to avoid, and Python has different but comparable warts.  There's
admittedly a bunch of crap in JS that shouldn't be used, but that's
relatively harmless as long as you ignore it.  Current JS
implementations are far superior to Python in performance and there's a
ton of JS library code out there, both for browsers and on the server
side, that might or might not have Python counterparts.  JS is also a
compilation target for other languages like Purescript, Ocaml, or even C
(emscripten).  I haven't tried Purescript yet but it looks cool, and it
might be the nerd answer to JS cruft.  It's Haskell-like but its data
and evaluation models translate more directly to JS than Haskell does.

> Nobody is stopping you from migrating all your code to Ocaml if you
> like.

We were talking about new projects, not migration.

Do you know if any of the big Python shops (Google maybe?) are using
Python 3 these days?  Have they migrated Python 2 codebases or are they
using both 2 and 3?  I don't personally know (i.e. in person, not
online) anyone using Python 3.  Everyone is still using Python 2.
E.g. I think Twitter was using Python 2 a year or so ago (don't know
about now).  I believe they still use Python for some things but have
gravitated production systems towards Scala.