Proposal: Ignore all empty entries within $PYTHONPATH

[David A. Wheeler]

All: I propose that all empty entries with $PYTHONPATH be ignored.
This would eliminate some accidents that can lead to security problems.
I thought it'd be easiest to explain by drafting a PEP, so please see
this first draft below.

Thanks! Comments welcome.

--- David A. Wheeler

=============================

PEP: 9999
Title: Ignore all empty entries within ``$PYTHONPATH``
Author: David A. Wheeler <dwheeler at dwheeler.com>
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 27-Aug-2020
Python-Version: 3.7.10
Post-History: 27-Aug-2020


Abstract
========

Currently, common ways to set ``$PYTHONPATH`` also unexpectedly add the
current directory as an entry.  As a result, users have an increased
risk of unintentionally running malicious code.

This PEP proposes that all empty entries within ``$PYTHONPATH`` be ignored,
instead of being treated as the current directory.
In the rare cases where the current directory is intended, "." (or
even better its absolute path) can continue to be used instead.
This small change eliminates an easily-made and subtle mistake that can
lead to a security vulnerability.

Motivation
==========

The essay "Never Run 'python' In Your Downloads Folder" by Glyph
`Never Run`_
points out that the way Python currently implements ``$PYTHONPATH``
easily leads to users unintentionally running malicioous code.
The problem is that "most places that [recommend using] PYTHONPATH
recommend adding things to it like so":

    ``export PYTHONPATH="/new/useful/stuff:$PYTHONPATH"``

As the essay notes, "this idiom has a critical flaw:
the first time it's invoked, if ``$PYTHONPATH`` was previously
either empty or un-set, this then includes an empty string,
which resolves to the current directory..."
As a result, someone who executed this line above will quietly execute
Python libraries in whatever their current directory happens to be,
and that could quickly lead to running malicious code.

It could be argued that this is fine, because the shell's ``$PATH``
does the same thing.
After all, the current documentation for ``$PYTHONPATH`` says that it
augments "the default search path for module files.  The format is
the same as the shell’s PATH: one or more directory pathnames
separated by os.pathsep (e.g. colons on Unix or semicolons on
Windows)."
`Cmdline`_

But this argument ignores a key difference: the shell's ``$PATH`` practically
*always* has an initial non-empty value, while ``$PYTHONPATH`` typically
starts with an *empty* value.
This means that the same patterns that are normally safe with ``$PATH``
(because ``$PATH`` is non-empty) are *dangerous* with ``$PYTHONPATH``
`Hacker News`_.
Once ``$PYTHONPATH`` is set with a dangerous value, it is likely to stay
dangerous after having other values appended.

A far safer approach is for Python to simply skip all empty entries
within ``$PYTHONPATH``.
In this situation, a ``$PYTHONPATH`` with the value "``:spam::eggs:``"
would be treated the same as "``spam:eggs``".
Empty entries are almost never intended.
They are also widely confusing, in part because they are not obvious.
Users who truly want to include the runtime current directory
can use "." instead, but in almost all cases they will want the
absolute path to the current directory anyway.

By ignoring all empty entries, Python users will be quietly protected
from this mistake, making Python just a little easier to use securely.

Setting ``$PYTHONPATH`` is less common (due to virtualenvs), but it's
still in use and is a useful mechanism.
It'd be best to quietly interpret empty entries in ``$PYTHONPATH``
as unintentional mistakes and ignore them, instead of quietly
enabling security vulnerabilities.

Implementing this is trivial. I have created this PEP because this
is technically a behavioral change in a long-present mechanism.


References
==========

.. _Never Run: https://glyph.twistedmatrix.com/2020/08/never-run-python-in-your-downloads-folder.html
 by Glyph
.. _Cmdline: https://docs.python.org/3/using/cmdline.html
.. _Hacker News: https://news.ycombinator.com/item?id=24250418


Copyright
=========

This document is placed in the public domain or under the
CC0-1.0-Universal license, whichever is more permissive.




..
   Local Variables:
   mode: indented-text
   indent-tabs-mode: nil
   sentence-end-double-space: t
   fill-column: 70
   coding: utf-8
   End: