(More) Re: Is __import__ known to be slow in windows?

[Joshua Kugler]

John Machin wrote:

> On Dec 1, 2:12 pm, Joshua Kugler <jkug... at bigfoot.com> wrote:
>>             x = __import__(m)
> 
> Have you ever tried print m, x.__file__ here to check that the modules
> are being found where you expect them to be found?

No, I haven't, but I do know for a fact that the only location of the module
found is where I think it is.  There are no other modules on the system (or
in the search path) named <ourprefix>_modulename.

>>         except ImportError, e:
>>             if not e.message.startswith('No module named'):
>>                 raise
> 
> Why are you blindly ignoring the possibly that the module is not
> found? Note that loading a module after it is found is not a zero-cost
> operation.

That was in the original code to silently ignore things like a README file
or a directory that didn't have a __init__.py file.  The code I posted was
from my benchmarking script.  The code in the framework does a listdir()
and goes through that list trying to import.  We'll probably "smarten up"
that code a bit, but for now it works.

>>     x = None
>>
>> Each of those three module names is a directory under /path/to/code with
>> an empty __init_.py.
> Is there anything else in the /path/to/code directory?

A directory and a file that aren't referenced in my test, but are
in the framework's import attempts (see listdir() comment above). 
Regardless, setting sys.path to just one directory speeds up both the
benchmark and the framework.

>> def r():
>>     for m in ['three','module','names']:
>>         x = __import__(m)
> 
> Have you tried print m, x.__file__ here to check that the modules are
> being found where you expect them to be found?

As I said above, no I haven't, but I will just to double check.

On Linux, print x, x.__name__ produces the expected result whether I have
one element in sys.path or all of them. On Windows, same result.

> Call me crazy, but: First experiment, sys.path was ['/path/to/code',
> '', etc etc]. Now it's only ['/path/to/code']. How can that still load
> properly but run faster??

I don't know, but it does.  Right now, we're just importing empty modules. 
Once those modules have imports of their own, my "fix" will probably fall
apart, and we'll be back to square one.

> What directory tree walking?? Should be none 
> if the modules are found in /path/to/code.

I know, that's the crazy thing...there should be no directory tree walking
or other filesystem activity after finding the module.  But setting
sys.path to one element does make a difference; that can be seen.  I don't
know the internal implementation of __import__, so I can't really comment
beyond my results.  Maybe I need to strace a python run.

> Are you sure the modules 
> are always found in /path/to/code? What is in the current directory
> [matched by '' in sys.path]?

'' would be whereever I start the benchmark script (~/bin), or whereever the
framework is started from, probably from the web server's home directory,
or from ~/bin when I'm profiling the framework.

Thanks for your responses.  I still trying to figure this out too. :)

j