[Distutils] short circuiting module lookups

[Tres Seaver]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Noah Gift wrote:
> On Wed, Apr 8, 2009 at 7:44 PM, Marius Gedminas <marius at pov.lt> wrote:
>> On Tue, Apr 07, 2009 at 02:23:50PM -0400, Jim Fulton wrote:
>>> On Apr 7, 2009, at 9:28 AM, P.J. Eby wrote:
>>>> At 11:54 PM 4/7/2009 +1200, Noah Gift wrote:
>>>>> 1.  In the case of entry points for setuptools, it actually recurses
>>>>> into EVERY egg directory in your path, not just the egg you
>>>>> requested, adds them to your sys.path and additionally looks for
>>>>> four files inside of every egg.  On a laptop on local storage, this
>>>>> doesn't matter, but when thousands of machines hit the same filer,
>>>>> with many python processes, bad things happen...
>>>> Install your eggs with --multi-version, and then only the eggs that
>>>> are required for the running script will be added to sys.path or have
>>>> their contents opened.  (Installing them as zip files rather than
>>>> directories may also speed this up.)
>>> My experience on Linux is that installing eggs as Zip files slows
>>> imports.
>> Was that in the presence of NFS?
> 
> yes.

Jim's report, which matches my experience, is not related to NFS:
multiple, zipped bdist_eggs are slower to import / use thant the same
eggs unzipped.

I think the "zip is faster" meme comes from the case of a single
monolithic zip archive (e.g., the whole stdlib, or all the Zope eggs,
in one file).


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJ3JYh+gerLs4ltQ4RAsYKAKDUEqv2+kVmHPnkLjp+mKNKsxv1SgCdF0OC
UjkqmPNfjYz5/ScBjG9ELN0=
=d5+b
-----END PGP SIGNATURE-----