PyWart: "Python's import statement and the history of external dependencies"

[Steven D'Aprano]

Ian Kelly wrote:

> On Thu, Nov 20, 2014 at 8:53 PM, Rick Johnson
> <rantingrickjohnson at gmail.com> wrote:
>> FOR INSTANCE: Let's say i write a module that presents a
>> reusable GUI calendar widget, and then i name the module
>> "calender.py".
>>
>> Then Later, when i try to import *MY* GUI widget named
>> "calendar", i will not get *MY* calendar widget, no, i will
>> get the Python calendar module.
> 
> Because like a fool you created a local module with a totally generic
> name like "calendar" and dumped it into the top-level namespace. 

Firstly, let me say that I am amazed that people have waded through Rick's
self-aggrandising purple prose, insults, demands and utter confusion to
find something worth engaging him on.

Having said that, it's not fair to blame the user for shadowing standard
library modules:

- All users start off as beginners, who may not be aware that this is even a
possibility;

- It's hard to keep track of what modules are in the standard library. Which
of the following is *not* in Python 3.3's std lib? No cheating by looking
them up.)

    os2emxpath, wave, sndheader, statslib, poplist, plist, 
    pickletools, picklelib, path, cgi, cgitb, copylib, xpath

- By default, sys.path is set up so that user modules have priority over std
lib modules. Given a name clash, shadowing in inevitable.


> The 
> Python import system gives you the ability to create packages, so use
> them -- create a package to contain all the widgets you create.


Packages come with their own challenges and complexities, especially if
you're trying to support both Python 2 and 3 from the same code base.
Besides, for simple uses, packages are overkill.


[...]
>> # Contrary to popular belief, sys.path is *NOT* a module,  #
>> # no, it's a global!                                       #
> 
> I really doubt that this is a popular belief.

I'm not aware of anyone who believes that sys.path is a module. If they did,
a simple import would prove that it is not:

py> import sys.path
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: No module named path


But yes, sys.path is not just global, but process-wide global. *All* modules
share the same sys.path.


>> This is another confusing fundamental of Python, it seems *MORE*
>> intuitive that changes to the "import search path" would only
>> affect the *CURRENT* module's "import search path", but that's
>> is not the case!
> 
> This just seems like it would create a maintenance nightmare. Before
> you could import anything, you'd have to figure out what the search
> path is for the particular module you're working and whether it
> happens to include the particular package you want. You find it
> doesn't, so you make a local change to the search path. Later, you
> make the same change to other modules that need to import it. Later
> still, the package moves to a different location in the file system,
> and now you get to go through and update every one of those modules
> with the new directory. Lucky you.

That would be horrible. But here's an alternative which is less horrible and
maybe even useful.

There's still a single process-wide search path, but there's a second
per-module search path which is searched first. By default it's empty.

So a module can define it's own extra search path:

__path__ = ['look/here', 'and/here']
import something

without affecting any other modules.


> And after all that, it would still fail if you happened to want to
> import both "calendar" modules into the same module.

__path__ = []
import calendar
__path__ = ['my/python/modules']
import calendar as mycalendar



-- 
Steven