[Python-3000] Chaning the import machinery; was: Re: [Python-Dev] setuptools in 2.5.

[Walter Dörwald]

Guido van Rossum wrote:

> Sorry, there's so much here that seems poorly thought out that I don't
> know where to start.

Consider it a collection of wild ideas.

> Getting rid of the existing import syntax in favor of the incredibly
> verbose and ugly
> 
>   foo = import("foo")
> 
> just isn't acceptable.

OK, then how about

import foo
import foo from url("file:~guido/python-packages/foo")

How would this work with multiple imports?

import foo, bar from url("file:~guido/python-packages/")

How would it recognize whether url() refers to the module itself or to a
package from which modules are imported?

> Importing from remote URLs is a non-starter from a security POV; and
> using HTTPS would be too slow. For code that's known to reside
> remotely, a better approach is to use setuptools to install that code
> once and for all.

I don't see how that changes anything from a security POV. You have to
trust the source in both cases. Performancewise you are right, it
wouldn't make sense to call such an import in a tight loop.

> How would a module know its own name?

It probably would only have a real name for a standard "import foo".

import foo from url("file:~guido/python-packages/foo")

would create a module with foo.__origin__ ==
url("file:~guido/python-packages/foo")

> How do you deal with packages
> (importing a module from a package implies importing/loading the
> package's __init__.py).

I have no idea. This probably won't work (at least not in the sense that
importing something imports all its parent modules).

But packages seem to be a problem for setuptools too (at least if parts
of the package are distributed separately).

Maybe for

import os.path from url("file:~guido/python-packages/os/path")

url("file:~guido/python-packages/os/path").parent() should return
url("file:~guido/python-packages/os") which then gets imported and
url("file:~guido/python-packages") should return None. But this would
mean that the number of dots on the left side has to match the number of
times calling parent() on the right side returns somehing not None. Not
good. Maybe we should leave the current import syntax alone and add a
new one for importing from files etc..

> I suggest that instead of answering these questions from the
> perspective of the solution you're offering here, you tell us a bit
> more about the use cases that make you think of this solution. What
> are you trying to do? Why are you importing code from a specific file
> instead of configuring sys.path so the file will be found naturally?

The Python files I'm importing define "layout classes" that are used for
generating a bunch of static HTML files (actually JSP files). I.e.
something like

foo.py:
def link(text, href):
   return "<a href='%s' class='foo'>%s</a>" % (href, text)

bar.py:
foo = specialimport("foo.py")

def linklist(links):
   return "<ul>%s</ul>" % "".join("<li>%s</li>" % bar.link(text, href)
for (text, href) in links)

This function linklist() is used for generating the HTML files. The
interactive Python shell is used as a "make shell", so when I change
foo.py in an editor and do
   >>> project.build("install")
in the shell all the HTML files that depend on bar.py (which in turn
depends on foo.py) must be rebuilt.

> I
> suspect that your module-from-a-database use case is actually intended
> as a building block for an import hook.

Maybe, but ACAIK import hooks can't solve the dependency and reload problem.

> I think we ought to redesign the import machinery in such a way that
> you could do things like importing from a specific file or from a
> database, but without changing the import statement syntax -- instead,
> we should change what import actually *does*.

But there has to be a spot where I can actually specify from *where* I
want to import, and IMHO this should be in the import statement, not by
putting some hooks somewhere.

> While we're at it, we
> should also fix the silliness that __import__("foo.bar") imports
> foo.bar but returns foo.

+1

Servus,
   Walter