Returning none

[C.Laurence Gonsalves]

On Sun, 29 Aug 1999 17:32:27 -0400, Tim Peters <tim_one at email.msn.com> wrote:
> Except that Python isn't trying to guess what you mean here -- as
> before, functions returning None by default is not a convenience, it's
> a cheap way to make an error likely when mistakenly using a procedure
> as a function.  It's not a bulletproof approach, but it is *an*
> approach, and mischaracterizing its nature weakens your case.

If no-one is using it with the intent of getting None, then it shouldn't
break any existing code if the behaviour were changed to instead raise
an exception when a non-returning-function-call is used in an
expression.

> I know -- that's why I wasn't <wink>.  Dismissing ideas out of hand as
> all loss and no gain is as unfruitful as embracing suggestions as if
> all gain and no cost.  As PaulP said, in reality it's a balancing act.

In one of my other posts I suggested three or four separate ideas. In
each I listed pros and cons for each. I don't claim that there's a
perfect solution, but I think that all four of these solutions were at
the very least marginally better than the current behaviour.

> > also says that when I go outside the bounds of a list in Python, I
> > should get back None, since in C I would access random stack/heap
> > trash.
> 
> I agree that having an out-of-bounds list reference silently return
> None would also be a way to make it (merely) likely to lead to an
> error.

Yes, but Python doesn't do this. It rises an IndexError. In the same
way, I think attempting to obtain the result from a function that
doesn't return one should raise some sort of exception.

> If there were no costs associated with this, or if the costs were
> thought smaller than the benefits, don't you think Python already
> would?

That isn't a convincing argument.

> In the list case, it cost nothing:  no additional syntax or
> long-winded rules were required, and the interpreter had to check for
> out-of-bounds anyway (else risk having the interpreter itself blow up
> due to a bogus C operation).
> 
> As to benefits, I'm with most of the other respondents:  I can't
> recall this insecurity (it certainly is one!) ever causing a
> significant problem -- and I've been programming in Python even longer
> than Tom Christiansen <wink>.

Perhaps your programming style is different than mine. I know I've
frequently run into situations where I've forgotten to actually return
the result I calculated. Apparently others have as well. I'm also sure
there is a lot of code that ignores return values. There's a way to add
checking for this as well...

And I don't see where the big overhead in this idea comes from.

> > That's why I also suggested some other possible rules that would
> > enable some amount of compile-time checking, so that function
> > writers who do questionable things would be alerted.
> 
> The best hope for this is that it gets folded into the push for
> optional type declarations in Python2; that will need a way to
> indicate function return types (or lack thereof) anyway.

Some amount of runtime checking is necessary though. Because Python is
so dynamic, a lot of the useful checks simply can't be done at compile
time.

> No.  You showed no sign of having considered why it might be the way
> it is, so I was inviting you to do so.  Sarcasm was not the hoped-for
> result.

Maybe I got a bit over-emotional at your response. I really think it
would have been better if you stated why you think it is the way it is,
and what's wrong with my ideas, rather than just flatly saying that "if
that idea were any good it would already be implemented". When you say
something like that, it just tells me that you aren't given what I've
suggested any real consideration at all.

> > In another post I suggested another idea that requires little or no
> > modifications to the grammar, but would just add some new runtime
> > and compile time checks.
> 
> You should study eval_code2 (in Python/ceval.c) if you want to suggest
> implementations.  That would give you a handle on part of the "cost"
> side of the tradeoffs.

I'll check that out.

> > Functions using return (or falling off the end) would return nothing
> > at all (rather than None),
> 
> The interpreter can't do that as currently structured -- a call must
> return a pointer to a PyObject.  You could invent a new internal
> NotARealObject type and return a pointer to an object of that type,
> but then it gets messier.

I haven't taken a look yet. What happens when exceptions are raised?
What value is returned then? And is "NULL" used for anything?

> Currently a CALL_FUNCTION opcode simply pushes the object it gets back
> -- it has *no idea* whether the result will be used later or
> discarded, so doesn't have enough info to know whether to complain.
> So you'll have to make CALL_FUNCTION very much smarter about its
> context.

Jonathan Giddy suggested the idea of making a CALL_PROCEDURE opcode that
would be different from a CALL_FUNCTION opcode.

> But then you have miserable semantic issues to resolve; for example,
> while you clearly must allow f to return nothing given the statement
> 
>     f()
> 
> is it also OK for f() to return nothing in the statement
> 
>     f(), g(), h()
> 
> ?  You must allow that else break existing code, but the result of f()
> *is* used by a later BUILD_TUPLE opcode:  it's not the case that
> "returning nothing" can work here, even if the interpreter were
> structured in a way that such a thing were possible -- BUILD_TUPLE
> requires an object.

This would be illegal, and would raise an exception. Why would you make
a tuple containing the result of a function that doesn't have a result?
These should be three separate statements. Semicolons are just as easy
to type as commas on most keyboards. :-)

> In this example, the tuple itself is later discarded, but there's no
> way for CALL_FUNCTION to know that without prior flow & lifetime
> analysis (Python has neither today), and a rigorous definition of
> which opcodes do and don't count as "real uses".

If the function call is in an expression, that's a "real use". If it's
just a statement, that isn't. One of the other ideas I'd proposed was
the removal of "expression" as a type of statement. Procedure calls
would be statements. There could also be another new type of statement
for calling functions and tossing the result, if you wanted to make it
illegal for a "procedure call" to have a result.

> Simply returning None was a pragmatic compromise that saved a world of
> implementation pain in return for a tiny chance of letting an error go
> undiagnosed forever, and an excellent chance of catching an error but
> later than you might hope for.

<grumble> That sounds like a good reason to remove array bounds checking
as well... someone's going to notice when the program starts acting
weird, but it might be "later than you might hope for".

Seriously, the current behaviour doesn't do any error checking at all.
It returns a goofy value with the hopes that someone will spot the
problem. If that's after you've committed it into a database, or sent
out a whole bunch of emails containing the word 'None', tough bananas.

I really don't see why raising an exception when appropriate would be so
hard.

> > and function definitions would require that the function either
> > always return a value, or never returns a value. This can be very
> > easily checked at compile time.
> 
> While this part is more clearly doable, it's also a probabilistic
> compromise. In your original example, it wouldn't complain (you had no
> returns, so this scheme would let it pass).  Better for optional
> declarations to allow you to say explicitly whether you intend to
> return a result or not, rather than replace one "guessing" scheme with
> another.

You're right, the example I gave wouldn't have any compile time errors
in this particular scheme. It would cause a runtime error though if I
tried to call it and get the result. I know Python is too dynamic to be
able to catch very much at compile time. I'm pretty sure I said in
another message that some amount of compile time checks would be
necessary for any of this to be worth it. Compile time checks alone do
diddly. Which is another reason why adding this to a lint tool probably
won't help much.

One of the other ideas I proposed was to have a separate "proc" keyword
that would be used in place of "def" for procedure definitions.

> It also introduces an ugliness Python doesn't suffer today; e.g.,
> 
> def first_factor(n):
>     for i in xrange(2, n):
>         if n % i == 0:
>             return i
>     complain(ValueError, n, "doesn't have a non-trivial factor")
> 
> def complain(exc, *args):
>     msg = string.join(map(str, args), " ")
>     raise exc(msg)
> 
> There's nothing wrong with either function, but to stop a bogus
> complaint the first would need a
> 
>     return 0   # make the stupid compiler shut up
> 
> line at the end.  To avoid this kind of artificial ugliness in a
> language with exceptions requires something akin to Java's (many)
> pages of "definite assignment" rules.  I happen to *like* Java's rules
> here (they catch many errors at compile-time Python cannot, and almost
> never give me bogus complaints), but without static declarations (or
> global inference coupled with weakened dynamics) Python can't approach
> that.

That's the best argument against the compile-time checks I've seen. The
runtime checks would still be fine though. So how about this: 

- functions return "nothing" (not None) if you fall off the end or say
  "return" without an argument.
- expressions are no longer statements (grammar change)
- there is a new type of "procedure call" statement (grammar change)
- function calls that are in an expression cause an exception to be
  raised if they don't actually return anything (eg: NoResultError)
- the root* function call of a procedure call statement is permitted to
  return nothing at all 
  
* by "root" function call I'm referring to the function call that would
  be the root of the expression tree, if the procedure call statement
  was treated as an expression. This is easy to determine syntactically.

Distinguishing between expression or statement context should be easy
for the compiler. As Jonathan Giddy suggested, it could use different
opcodes for these situations. I haven't yet checked to see how a return
of "nothing" could be indicated internally, but I'll do that soon.

And if you want to prevent people from ignoring return values, you can
add the following:
- the root function call of a procedure call statment *must* return
  nothing at all, or an exception is raised (eg: IgnoredResultError)
- another new type of statment could be used to make function calls and
  discard the result, ala Modula-3. (grammar change)

If you think there is a problem with these ideas, please let me know
why. I'm not trying to start a fight here. I like Python a lot, and
would like to see it improved. If you think it's better the way it
already is, convince me.

BTW, I know that the last two points will break a *lot* of existing
code. They're only necessary if you want to make sure results are never
ignored though, and I'd be interested in knowing if there are any
problems with these ideas idependant of breaking existing code.

-- 
  C. Laurence Gonsalves            "Any sufficiently advanced
  clgonsal at kami.com                 technology is indistinguishable
  http://www.cryogen.com/clgonsal/  from magic." -- Arthur C. Clarke