Whitespace not/required

[dn]

Although many new-comers are intrigued by the compulsory indentation 
rule, I have been surprised to discover that even though whitespace does 
not usually feature as a formatting-specification, nevertheless Python 
sometimes requires an absence of whitespace.

Will this behavior/requirement continue when using (a pre-release 
version featuring) the new parser?


Whitespace has meaning to us - when it comes to indentation defining 
(not merely illustrating) blocks of code. However, the rest of the time, 
it is ignored by Python. (yes, this discussion disdains comments!) For 
example, whitespace is no problem when it comes to defining a list:

month_names = ['Januari', 'Februari', 'Maart',      # These are the
                'April',   'Mei',      'Juni',       # Dutch names...

Similarly, a totally blank line will be equally-totally ignored.

As my eye-sight ages (it's older than my teeth!), I find code easier to 
read when there is more whitespace - and it becomes more difficult to 
parse when whitespace is omitted. For example, I receive regular 
comments/criticisms for writing an argument list with spaces inside the 
parentheses, eg

     def func( arg1, arg2, ):

Whilst there are some 'standards' which decry such practice, I note that 
many IDEs will offer to add such spaces, auto-magically, as a 
config/option. So, apparently I'm not unique - just 'special'?

I don't use 'back-slash continuity' very often (at the expense of more 
parens!), and was intrigued to discover in experiments; that not only is 
a space before the back-slash considered optional, but sometimes a space 
is not necessary at all, eg

 >>> if\
... True: print( 'yes' )
...
yes
 >>> ifTrue: print( 'yes' )
yes

although Python won't let me go-crazy:

 >>> if True andFalse: print( 'yes' )
   File "<stdin>", line 1
     if True andFalse: print( 'yes' )
             ^
SyntaxError: invalid syntax
 >>> if Tr\
... ue: print( 'yes' )
   File "<stdin>", line 2
     if Tr\
ue: print( 'yes' )
     ^
SyntaxError: invalid syntax

The legal statement: <<<2.1.9. Whitespace between tokens
Except at the beginning of a logical line or in string literals, the 
whitespace characters space, tab and formfeed can be used 
interchangeably to separate tokens. Whitespace is needed between two 
tokens only if their concatenation could otherwise be interpreted as a 
different token (e.g., ab is one token, but a b is two tokens).>>>

Thus, (a little later on the same page): <<<One syntactic restriction 
not indicated by these productions is that whitespace is not allowed 
between the stringprefix or bytesprefix and the rest of the literal.>>> 
Thus, it must be expressed as "b'bytes'" and not "b 'bytes'". 
Admittedly, such separation hadn't occurred to me, in much the same way 
that "print ()" doesn't seem correct, but...

When we break the language-elements into "tokens" the 'bytes' and the 
argument-list's parentheses are separate from the preceding "b" or 
function-name, albeit semantically inseparable.

For f-strings/formatted string literals, the most usual form is:

     "{" f_expression ["="] ["!" conversion] [":" format_spec] "}"

Remembering that this is BNF, see the space separating the closing-brace 
from anything preceding it - how else would we separate the components 
to comprehend?

Returning to Python:

 >>> one = 1    # is the loneliest number...
 >>> f'{ one }'
'1'
 >>> f'{ one:03 }'
Traceback (most recent call last):
   File "<stdin>", line 1, in <module>
ValueError: Unknown format code '\x20' for object of type 'int'
 >>> f'{ one:03}'
'001'

Notice the presence/absence of the final space.

 >>> pi = 3.14159   # better to import math
 >>> f'{ pi!r:10 }'
Traceback (most recent call last):
   File "<stdin>", line 1, in <module>
ValueError: Unknown format code '\x20' for object of type 'str'
 >>> f'{ pi!r:10}'
'3.14159   '
 >>> f'{ pi!r }'
   File "<stdin>", line 1
SyntaxError: f-string: expecting '}'
 >>> f'{ pi!r}'
'3.14159'

So, the f-string will work if the braces include only an expression 
surrounded by spaces. However, if one adds a conversion or 
format-specification, that final space becomes a no-no. Eh what!

To be fair, the 'book of words' does say: "A replacement field ends with 
a closing curly bracket '}'.". No mention of whitespace. No mention that 
a replacement field consisting only of an f_expression, will be treated 
differently by allowing a space.

Version 3.8 introduced the "=" short-cut:

 >>> f"{ foo = }" # preserves whitespace
" foo = 'bar'"

Note the comment! Yet, the manual's examples continue:

 >>> line = "The mill's closed"
 >>> f"{line = }"
'line = "The mill\'s closed"'
 >>> f"{line = :20}"
"line = The mill's closed   "

Hey, why does this second example dispense with the braces-internal 
spaces? Sure enough, when I check it for myself:

 >>> line = "The mill's closed"
 >>> f'{line = :20 }'
Traceback (most recent call last):
   File "<stdin>", line 1, in <module>
ValueError: Unknown format code '\x20' for object of type 'str'
 >>> f'{ line = :20}'
" line = The mill's closed   "
 >>> f'{line = :20}'
"line = The mill's closed   "

Should the closing brace be considered part of a conversion or 
format-specification? The space (I'd like to add) cannot be considered 
part of a conversion or format-specification (see BNF, see text of web.ref)!


(when you compare how long it took me to (speed) type this message, and 
how long it took me to figure-out that my 'excess space' was causing a 
fault in an f-string, the time-cost was about the same. Grrr! (the 
relative frustrations-caused, somewhat different!)


Will be interested in explanations, and improvements to understanding.
(but not so much in 'corrections' to my (mis-)behavior - I already pay 
that price!)

If you're operating at the 'bleeding edge', will be interested to hear 
how the 'new parser' handles same.


Web.Ref:
https://docs.python.org/3/reference/lexical_analysis.html
-- 
Regards,
=dn