Proper shebang for python3

[Peter J. Holzer]

On 2019-07-20 15:26:46 -0500, Tim Daneliuk wrote:
> On 7/20/19 2:56 PM, Peter J. Holzer wrote:
> > On 2019-07-20 14:11:44 -0500, Tim Daneliuk wrote:
> >> So, no, do NOT encode the hard location - ever.  Always use env to
> >> discover the one that the user has specified.  The only exception is
> >> /bin/sh which - for a variety of reasons - can reliably counted upon.
> >>
> >> We don't need to bikeshed this.  All we need is people who disagree
> >> with this view to spend a year in software packaging, operations,
> >> deployment and DevOps ... then get back to us...
> >
> > After 25 years in software packaging, operations, deployment and DevOps
> > I disagree: A program should not behave differently because of a
> > different path, #!/usr/bin/env is a total no-no.
> >
> > There is a nice way to achieve this: Just use the interpreter of the
> > virtual environment in the shebang.
> > (That requires rewriting the shebang during installation, but that's a
> > minor inconvenience)
> 
> 
> And what happens with programs that have no virtenv equivalent?
> perl, go, ruby, awk, sed, grep, etc. have no simple way to
> get installed virtual short of insisting the everything live in a
> docker container or VM?

Perl is an excellent example: When you install a Perl script which has
been packaged with one of the usual tools (e.g., Makemaker or
Module::Build), the shebang is set to the interpreter used for
installation. The user of the script doesn't have to know what
environment to use. (Perl also has perlbrew which is similar to virtual
environments, but I haven't used that much).

Go is a compiled language: The executables are binaries (no shebang) and
even statically linked. Again, the user doesn't have to care about a
language environment.

awk, grep, sed, etc. are often used in shell scripts, and they were
probably the reason why I first developed a "a program's behaviour must
not depend on the environment /except as documented/" policy: Back then
in the 1990s one usually wanted to have GNU awk, sed, grep etc. in
addition to the system utilities. Having scripts randomly fail or
(worse) produce wrong results depending on the order of /usr/bin and
/usr/local/bin in the user's PATH and the exact contents of
/usr/local/bin wasn't fun (oh, and don't forget cron).


> The fact is that most large compute environments are slow to upgrade
> the OS.  That means core tools also lag considerably behind as well.
> Being able to install newer versions along side the OS' own and then
> use them by default is manifestly necessary.

I agree. But (for me at least) it is important to do that robustly, and
#!/usr/bin/env is not robust. 

> That's why users have the ability to modify $PATH to suit their own
> needs.  All /usr/bin/env does is to tell the interpreter, "honor the
> intent of the spawning shell".

You probably just phrased that badly, but as written this is completely
wrong: /usr/bin/env doesn't tell the interpreter anything. It *chooses*
the interpreter to invoke.


> If you want really big fun, try going into an older CentOS or RedHat instances and, say,
> upgrading system python to python3.

Don't do that. Install python3 somewhere else, e.g. into /usr/local.

> It's super fun.  Yes, in that case, you COULD use a venv.  But there
> are tons of other tools for which this is not an option - gcc,
> autoconf, perl, go awk, sed, bash, ad infinitum, ad nauseum are
> invariably backleveled on production OS instances.  My way fixes that
> problem.

Your way only fixes your problem only if environments where your script
may be called. You may know that /usr/local/bin/foo is a python script
that requires /usr/local/python3.8/bin/python to be in the path before
any other python interpreter and you may rememmber that in every
instance where that script is called directly or indirectly. But do your
colleagues? Especially if they aren't programmers?

And what happens if /usr/local/bin/bar requires
/usr/local/python3.4/bin/python? Do you explicitely set the PATH before
invoking each script?

By putting #!/usr/local/python3.8/bin/python at the top of
/usr/local/bin/foo and #!/usr/local/python3.4/bin/python at the top of
/usr/local/bin/bar I can just invoke both scripts. Even better, my users
can invoke both scripts and they don't even have to know they are
written in Python.


> You may have 25 years at this but I have 40 - does that make me nearly twice
> as right?  Arguments from authority are silly.

No, but it invalidates your notion that anybody with a bit of experience
would obviously use #!/usr/bin/env. I do have a bit more than one year
of experience and I think #!/usr/bin/env is a terrible idea. (I use it
for throwaway scripts which need a venv, but not for anything in
production.)

        hp

-- 
   _  | Peter J. Holzer    | we build much bigger, better disasters now
|_|_) |                    | because we have much more sophisticated
| |   | hjp at hjp.at         | management tools.
__/   | http://www.hjp.at/ | -- Ross Anderson <https://www.edge.org/>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-list/attachments/20190721/82396796/attachment.sig>