How to make Python run as fast (or faster) than Julia

[Chris Angelico]

On Tue, Feb 27, 2018 at 12:57 PM, bartc <bc at freeuk.com> wrote:
> On 27/02/2018 00:35, Chris Angelico wrote:
>>
>> On Tue, Feb 27, 2018 at 11:17 AM, Steven D'Aprano
>> <steve+comp.lang.python at pearwood.info> wrote:
>>>
>>> On Tue, 27 Feb 2018 02:09:53 +1100, Chris Angelico wrote:
>>>
>>>> You're still reimplementing the C code in Python, which is inefficient.
>>>> Have you considered going back to the *actual algorithm* and
>>>> implementing that idiomatically in Python? I think you'll find that (a)
>>>> the code is more readable, and (b) the run time is much lower.
>>>
>>>
>>> Chris, I think this is incredibly optimistic, if not naive. We're talking
>>> about a PRNG by Marsaglia, so my guess is that the "original algorithm"
>>> *is* the C code. Or possibly Fortran.
>>>
>>> Even if not, even if there is actually a language-neutral algorithm, its
>>> a PRNG which means its going to be filled with bit-twiddling and number-
>>> crunching operations. Pure Python without a JIT is never going to be
>>> competitive with C or Fortran for code like that.
>>>
>>
>> I may have been a little unclear. It's highly unlikely that the run
>> time of the properly-implemented Python code will be lower than the
>> original C or Fortran. But it most certainly CAN be more efficient
>> than the Python reimplementation of the C implementation, which would
>> narrow the gap considerably. Implementing Python idiotically instead
>> of idiomatically gives you suboptimal performance.
>
>
> Nonsense. You shouldn't need to care about such things. An algorithm is an
> algorithm. And the better ones should be easily written in any language.
>
> This particular one, which was of interest because the calculations tended
> to overflow into undesirable bignums, is just a bunch of calculations.
> 'Bit-twiddling'.
>
> Anyway, even this pure Python version can deliver pseudo random numbers at
> some 200,000 per second, while the built-in generator does 450,000 per
> second, so it's not bad going.

The built-in generator is using a completely different algorithm
though, so rate of generation isn't the entire story. How long is the
period of the one you're using? (How long before it loops?) If you
churn the generator to an arbitrary number and then take the next 100
values it generates, are they randomly distributed? Can you
reconstruct the RNG's internal state by watching it generate numbers
for a short while?

Obviously no PRNG is going to be perfect at this, but there are
certainly degrees of quality to be compared.

ChrisA