Without compilation, how to find bugs?

[Steve D'Aprano]

On Fri, 14 Oct 2016 08:04 pm, BartC wrote:

> On 14/10/2016 01:59, sohcahtoa82 at gmail.com wrote:
>> On Thursday, October 13, 2016 at 4:06:36 PM UTC-7, pozz wrote:
> 
>>> Are the things exactly how I understood, or do I miss something in
>>> Python?
>>
>> As others have said, user a linter.
> 
> With Python you're supposed to just be able run any source code
> instantly; how will using a 'lint' tool impact that process? Or is it
> only meant to be used infrequently?

The process is little different between C and Python:

With C: during development, you run the compiler (which includes built-in
static analysis) or stand-alone linter, and optionally any tests you have,
etc. The deployed software rarely if ever includes static analysis.

With Python: during development, you optionally run linters, static
analysis, tests, etc. After deployment, you rarely run static tests,
linting, etc.


>> I'd go a step further and use an actual code editor or IDE that includes
>> some basic static analysis.  Using this example that Skip used:
>>
>> def func2(a, b):
>>     print(a, b)
>>
>> def func1(a):
>>     print(a)
>>
>> func2(1)
>>
>> Any code editor worth using will highlight the ) on the last line and
>> tell you that there's a missing parameter.

That's a matter of opinion.



> How can that work? I thought one of the biggest deals with Python is
> that you can re-bind function names to anything else. So:
> 
> if cond:
>      func2 = 38
> else:
>      func2 = func1
> 
> Then func2(1) can either be perfectly correct, or completely erroneous!

Of course. Unless the compiler can perform full-program analysis and rule
out the possibility that func2(1) has been rebound to something that will
accept a single argument, it cannot reject that code. The design principle
here is that anything which isn't provably wrong should be allowed, in case
the programmer knows better than the compiler.

A corollary of this is that most (all?) existing Python compilers are quite
simple-minded and naive. As has been pointed out, even the most obviously
wrong code:

    x = 1 + "2"

is not rejected until runtime. Why not? You'd have to ask Guido for a
ruling, but I think he would say:

- he's not interested in making static analysis part of the language
specification;

- as far as the CPython reference implementation is concerned, he thinks it
is a waste of time and effort to do such static analysis in the compiler/
interpreter;

- but third-party tools (linters, IDEs, code checkers, etc) or compilers are
welcome to do so if they choose.

(I stress that I don't *know* this is what Guido will say, I'm just
guessing.)

As external tools, linters and IDEs etc aren't constrained by that strict
rule "never reject code that isn't provably wrong". They are permitted to
make a reasonable guess. They are only giving optional warnings which
indicate something which *might* be incorrect.


> (I have my own suspicions that functions in Python are predominantly
> used in a boring, predictable, static manner (so allowing certain
> optimisations - or error checking), 

Indeed. I have a vague memory of somebody actually doing a survey of some
large code base, and finding that (excluding decorators, which technically
perform a rebinding) something like 98% or more of functions and classes
were used in a boring, static manner.

But of course, that's partly a matter of convention and coding styles. Some
code bases are written in a boring, static manner because the corporate
style prohibits dynamic rebinding of functions.


> but I got the impression from some 
> threads here that many apparently do little else in their code but bind
> and rebind function names.)

Heh, everybody loves to talk about the most extreme forms of Python
dynamicism, but hardly anyone uses them often. But they are used, just
enough to rule out banning them.

You may be interested in some work being done by one of the core developers.
One of the features of Python that keeps it relatively slow is that
function calls are resolved at runtime: calling len(x) has to do a runtime
search for the name "len" before calling it. Victor Stinner is working on
run-time optimizations which can detect when it is safe to replace that
run-time search with a compile-time static call.

That's also one of the future optimizations Nuitika is looking at.

And of course PyPy does a similar thing, except as just-in-time compilation
rather than ahead-of-time.



 

-- 
Steve
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.