[Python-Dev] Import redesign (was: Python 1.6 status)

[Gordon McMillan]

[Guido]
> > I think the standard format should be a subclass of zip or jar
> > (which is itself a subclass of zip).  We have already written
> > (at CNRI, as yet unreleased) the necessary Python tools to
> > manipulate zip archives; moreover 3rd party tools are
> > abundantly available, both on Unix and on Windows (as well as
> > in Java).  Zip files also lend themselves to self-extracting
> > archives and similar things, because the file index is at the
> > end, so I think that Greg & Gordon should be happy.

No problem (I created my own formats for relatively minor 
reasons).
 
[JimA]
> Think about multiple packages in multiple zip files.  The zip
> files store file directories.  That means we would need a
> sys.zippath to search the zip files.  I don't want another
> PYTHONPATH phenomenon.

What if sys.path looked like:
 [DirImporter('.'), ZlibImporter('c:/python/stdlib.pyz'), ...]
 
> Greg Stein and I once discussed this (and Gordon I think).  They
> argued that the directories should be flattened.  That is, think
> of all directories which can be reached on PYTHONPATH.  Throw
> away all initial paths.  The resultant archive has *.pyc at the
> top level, as well as package directories only.  The search path
> is "." in every archive file.  No directory information is
> stored, only module names, some with dots.

While I do flat archives (no dots, but that's a different story), 
there's no reason the archive couldn't be structured. Flat 
archives are definitely simpler.
 
[JimA]
> > > I don't like sys.path at all.  It is currently part of the
> > > problem.
[Guido] 
> > Eh?  That's the first thing I hear something bad about it. 
> > Maybe that's because you live on Windows -- on Unix, search
> > paths are ubiquitous.
> 
> On windows, just print sys.path.  It is junk.  A commercial
> distribution has to "just work", and it fails if a second
> installation (by someone else) changes PYTHONPATH to suit their
> app.  I am trying to get to "just works", no excuses, no
> complications.

		Py_Initialize ();
		PyRun_SimpleString ("import sys; del sys.path[1:]");

Yeah, there's a hole there. Fixable if you could do a little pre- 
Py_Initialize twiddling.
 
> > > I suggest that archive files MUST be put into a known
> > > directory.

No way. Hard code a directory? Overwrite someone else's 
Python "standalone"? Write to a C: partition that is 
deliberately sized to hold nothing but Windows? Make 
network installations impossible?
 
> > Why?  Maybe this works on Windows; on Unix this is asking for
> > trouble because it prevents users from augmenting the
> > installation provided by the sysadmin.  Even on newer Windows
> > versions, users without admin perms may not be allowed to add
> > files to that privileged directory.
> 
> It works on Windows because programs install themselves in their
> own subdirectories, and can put files there instead of
> /windows/system32. This holds true for Windows 2000 also.  A
> Unix-style installation to /windows/system32 would (may?) require
> "administrator" privilege.

There's nothing Unix-style about installing to 
/Windows/system32. 'Course *they* have symbolic links that 
actually work...
 
> On Unix you are right.  I didn't think of that because I am the
> Unix sysadmin here, so I can put things where I want.  The
> Windows solution doesn't fit with Unix, because executables go in
> a ./bin directory and putting library files there is a no-no. 
> Hmmmm... This needs more thought.  Anyone else have ideas??

The official Windows solution is stuff in registry about app 
paths and such. Putting the dlls in the exe's directory is a 
workaround which works and is more managable than the 
official solution.
 
> > > We should also have the ability to append archive files to
> > > the executable or a shared library assuming the OS allows
> > > this

That's a handy trick on Windows, but it's got nothing to do 
with Python.

> > Well, the question is really if we want flexibility or archive
> > files. I care more about the flexibility.  If we get a clear
> > vote for archive files, I see no problem with implementing that
> > first.
> 
> I don't like flexibility, I like standardization and simplicity.
> Flexibility just encourages users to do the wrong thing.

I've noticed that the people who think there should only be one 
way to do things never agree on what it is.
 
> Everyone vote please.  I don't have a solid feeling about
> what people want, only what they don't like.

Flexibility. You can put Christian's favorite Einstein quote here 
too.
 
> > > If the Python library is available as an archive, I think
> > > startup will be greatly improved anyway.
> > 
> > Really?  I know about all the system calls it makes, but I
> > don't really see much of a delay -- I have a prompt in well
> > under 0.1 second.
> 
> So do I.  I guess I was just echoing someone else's complaint.

Install some stuff. Deinstall some of it. Repeat (mixing up the 
order) until your registry and hard drive are shattered into tiny 
little fragments. It doesn't take long (there's lots of stuff a 
defragmenter can't touch once it's there).


- Gordon