[Python-Dev] imputil speed (was: .DLL vs .PYD search order)

[Greg Stein]

On Sat, 4 Dec 1999, M.-A. Lemburg wrote:
>...
> > Don't worry Fredrik... I'm with you on this one. I do not believe there is
> > a problem with the speed. Nobody has yet profiled imputil to find out
> > where/how the time is being spent. Nobody has tried to speed it up.
> 
> Sorry, Greg, but that is simply not true. I've spend a few
> days on trying to get more performance out of it and have
> succeeded, but in the end it wasn't enough to convince me
> of the approach.

You sent me your changes... I don't believe that you were aggressive
enough. As I've mentioned before, I think it is quite possible to retain
the general Importer style and get_code() interface, but to shift some
functionality out (to be computed once) to a higher-level mechanism. The
patches that you sent me did not do this, so I'm not surprised that you
hit a wall.

Ack. See? Now I'm getting into discussions about performance and
implementation without truly knowing where the timing is spent. Eyeballing
it, I have an idea, but it would be best too see a profile output. My
mantra is always "90% of the time you're wrong about where 90% of the time
is being spent."

I am unconcerned about performance, but will work on it so that I don't
need to continue this conversation. That burden is on me.

> > Therefore, any claims about its performance are simply FUD.
> 
> BTW, did anybody mention that an import manager  wouldn't
> be able to provide an API which is useable for imputil
> style importers ? I'm not argueing against the possibility
> to use imputil style importers, just against making it the
> sole method of adding wisdom to Python imports.

Since the core will delegate out to Python (note: current working theory),
then it certainly is not the "sole method" (since you can just replace the
Python code). But there must be a default mechanism.

The ihooks stuff was too complicated. imputil seems to be much easier. I'd
love to see a third mechanism.... so I can steal ideas :-)

> The imputil importers could well benefit from a manager
> providing logic to do basic things like importing
> shared libs, checking signatures, downloading modules
> from the web, etc.

For shared libs, yes. For the others: geez... I don't want to see that in
the core infrastructure. Shift that out to specialized Importers. The
infrstructure ought to be teeny and agnostic about how to map a module
name to a module.


Side note to python-dev people: I apologize... I realize that I'm
beginning to get a bit defensive here. I'm going to be at XML '99 until
Friday, so that should give me a breather. When I get back, I'll skip the
talk and do some code.

Cheers,
-g

-- 
Greg Stein, http://www.lyra.org/