New PEP: The directive statement

[Remco Gerlich]

I'm not that happy with this PEP. I'll try to explain why. 

First I have to point out an inconsistency in the PEP, then I'll explain my
broader concerns.

For problem a) I think 'from __future__' is better, for b) I think
directives are too general, allow things we want to keep away from the
language, this problem is better solved with, say, a sys.options dict.

Martin von Loewis <loewis at informatik.hu-berlin.de> wrote in comp.lang.python:

(I'll leave some bits that I want to comment on)

(A)
>     In discussion of this PEP, readers commented that there are two
>     kinds of "settable" language features:
> 
>     a) those that are designed to eventually become the only option,
>        at which time specifying use of them is not necessary anymore.
>        The features for which the syntax of the "Back to the
>        __future__" PEP [3] was proposed fall into this category.  This
>        PEP supports declaring such features, and supports phasing out
>        the "old" meaning of constructs whose semantics has changed
>        under the new feature. However, it defines no policy as to what
>        features must be phased out eventually.
> 
>     b) those which are designed to stay optional forever, e.g. if they
>        change some default setting in the interpreter. An example for
>        such settings might be the request to always emit line-number
>        instructions for a certain module; no specific flags of that
>        kind are proposed in this specification.
> 
>     Since a primary goal of this PEP is to support new language
>     constructs without immediately breaking old libraries, special
>     care was taken not to break old libraries by introducing the new
>     syntax.

(B)
> Syntax
> 
>     A directive_statement is a statement of the form
> 
>         directive_statement: 'directive' NAME [atom] [';'] NEWLINE
> 
>     The name in the directive indicates the kind of the directive; it
>     defines whether the optional atom can be present, and whether
>     there are further syntactical or semantical restrictions to the
>     atom. In addition, depending on the name of the directive, certain
>     additional syntactical or semantical restrictions may be placed on
>     the directive (e.g. placement of the directive in the module may be
>     restricted to the top of the module).

(C)
> Backwards Compatibility
> 
>     Introducing 'directive' as a new keyword might cause
>     incompatibilities with existing code. Following the guideline in
>     [1], in the initial implementation of this specification,
>     directive is a new keyword only if it was used in a valid
>     directive_statement (i.e. if it appeared as the first non-string
>     token in a module).


First the inconsistency: the syntax definition in (C) claims that some
restrictions on the placement of directive may be in place, depending on
the directive. However, (D) notes that directives must appear as the first
non-string token in a file, for backwards compatibility. So in practice,
directive can't be used anywhere in a file.


Then the two kinds of language features, from (A). The transitional features
are your a), the features that will have a short transitional period before
they become just another part of the language. This is the part that the
__future__ PEP addresses. Your problem b) is for truly optional features, I
imagine that would be things like optimization, warnings, debugging output
(like your line numbers). These things are currently addressed by command
line options. I don't think these two problems are solved best by a single
solution.

What I miss most in the 'directive' approach is the introspection. Can a
program see which directives are on? List the current directives and their
docstrings? Also, if I can turn on a directive further down a file, how can
I turn it *off*?

Transitional features are transitional for a hopefully short time. They are
provided for people who need to test their existing software against an
upcoming incompatible feature, and for early adopters who want the new
feature now so they can tinker with it. The version in which the feature
will become part of the language proper is already decided (2.2, in the case
of nested scopes).

The 'from __future__ import' idea shows that a feature is being used from a
future version. It's also a realy module with real information, so that a
running program can see what is in there, and can see information like the
version when this will become mandatory and, not least, the feature's
docstring. What is ugly about that is the magical side effect of an import
statement at the first line of a file that has compile time side effects.
But since this is for testing code against upcoming changes and for early
adopters who are usually testers anyway, I don't see that as a big problem.
Besides, using __underscores__ always implies at least a little magic, as
should time machines.

The 'directive' idea still has the magical first line of the file bit, but
it doesn't provide information to the running program about what is turned
on. It's not possible to make a single tool once that removes unneeded
'directive' statements from files, or even prints a list of available
statements. Also, 'transitional' is odd. The feature is *in a transitional
period*, but neither the file nor the feature is transitional (it's either
on or off, not in some transitional state). So I don't even agree on the
clarity point.

And then the other half of the problem, use of the directive statement for
other things. Is this vapor? I can't think of any useful applications at the
moment; for systemwide features we have command line options (not that
that's perfect, but that's not the point here), and directives can only be
used on a file-wide level. I can imagine there would be options you want to
toggle on a smaller level, say, for only part of a file. But you can't
toggle directives, you can't even see what their current setting is, inside
the program.

So I feel the application domain for the second form of directive is very
small, basically compile time options that make sense for single files. I
haven't seen a pressing demand for those.

Besides, a dictionary 'sys.options' could control a lot of options. In
a much nicer and more Pythonic way. In the case of compile time options,
they'd have to be set before the file is compiled, say, before an 'import'
statement. Want to warn for mixed tabs and spaces for a single file, not
using the compile time option? set sys.options["mixedtabs"] to 1, then
import it.

What this means is that you can't have real (non-transitional) options that
change the semantics of the file and distribute the files, because you can't
rely on the option being turned on. This is a good thing! It keeps Python a
single language.

Option directives basically mean "This file is in Python, with directives 2,
4 and 6c turned on." We don't want that. Transitional features have a use,
but only because they'll be mandatory soon.



For problem a) I think 'from __future__' is better, for b) I think
directives are too general, allow things we want to keep away from the
language, this problem is better solved with, say, a sys.options dict.



I could go on, but noone reads this far anyway, I'll wait for the reactions
:-).

-- 
Remco Gerlich