Pyhon 2.x or 3.x, which is faster?

[Ben Finney]

BartC <bc at freeuk.com> writes:

> On 07/03/2016 20:47, Chris Angelico wrote:
> > Look! Creating a floating-point value is faster under Python 2 than
> > Python 3. What could be more perfect?
>
> > This is like a microbenchmark in that it doesn't tell you anything
> > about real-world usage.
>
> Microbenchmarks have their uses, when you are concentrating on a
> particular aspect of a language.

The “micro” is not referring to the focus of the operation; that's a
good thing to do and is a way to produce useful, salient results. Choose
a specific aspect and design a test case which focusses as narrowly as
possible on that aspect. Then you can gather data to make meaningful
statements about that aspect in isolation from others.

What is “micro” about the benchmark you showed is that it doesn't gather
enough data to be useful. The test should not just do one statement; it
should do some relevant complete unit of the algorithm. Focussed, but
interesting: a huge amount of data, or a complex data conversion, or
something else that qualifies as “real”.

> Anyway, is reading a jpeg not real world usage? It's a task normally
> done with a compiled language, but can also be a good test for an
> interpreted one.

Try testing on a large battery of highly varied JPEG images, and getting
the aggregate results from that. Or something that is more interesting
(i.e. relevant to the real usage) than one image.

> If you want an interpreted language to take on wider range of tasks,
> then you need to make it faster, and that means measuring how
> efficient it is at executing algorithms itself, not measuring how long
> some library in a different language takes to execute.

That's quite true. The trick is to avoid being *so* focussed that you
are mostly measuring a corner case, instead of a relevant use case.

> > CPython 2.5 and 2.7 are very different. Even 2.7.0 and 2.7.11 are
> > going to differ in performance. And that's without even looking at
> > what core devs would refer to as "other Pythons", which would
> > include IronPython, Jython, PyPy (well, you got that, but you're
> > treating it as an afterthought), MicroPython, Brython, wpython,
> > Nuitka, Cython..... these are *other Pythons*.
>
> What are you suggesting here? That all these Pythons are going to be
> faster or slower than each other?

At specific, highly-focussed operations? Of course. That is going to be
true almost by definition: the implementations are different, so if your
test cases are extremely focussed they will be much more likely to find
variations in the implementations — differences that are not relevant to
which version of the Python source you're coming from.

> I would guess that most of them are going to be roughly the same,
> other than PyPy.

That's the kind of guess which should be subjected to testing. We humans
are *very* bad at estimating those kinds of behaviour in implementation.
This is especially true because we humans also demand baroque,
highly-complex implementations in order to get our speed improvements.

And those complexities of implementation will be a major factor
thwarting our intuitions about how those implementations will perform.
If you think you know how an implementation will perform, the data
indicates you should not trust that instinct.

> If there was a fast version, then I would have heard about it!

The mistake, I think, is in assuming a “fast version” means anything.

Different implementations will be faster at some things, slower at other
things, where “things” can be a very lumpy and convoluted landscape of
complex use cases. Test those assumptions with realistic benchmarks run
under each implementation.

-- 
 \          “When [science] permits us to see the far side of some new |
  `\      horizon, we remember those who prepared the way – seeing for |
_o__)                          them also.” —Carl Sagan, _Cosmos_, 1980 |
Ben Finney