[Python-Dev] requirements for moving __import__ over to importlib?

PJ Eby pje at telecommunity.com
Wed Feb 8 03:27:04 CET 2012


On Tue, Feb 7, 2012 at 5:24 PM, Brett Cannon <brett at python.org> wrote:

>
> On Tue, Feb 7, 2012 at 16:51, PJ Eby <pje at telecommunity.com> wrote:
>
>> On Tue, Feb 7, 2012 at 3:07 PM, Brett Cannon <brett at python.org> wrote:
>>
>>> So, if there is going to be some baseline performance target I need to
>>> hit to make people happy I would prefer to know what that (real-world)
>>> benchmark is and what the performance target is going to be on a non-debug
>>> build. And if people are not worried about the performance then I'm happy
>>> with that as well. =)
>>>
>>
>> One thing I'm a bit worried about is repeated imports, especially ones
>> that are inside frequently-called functions.  In today's versions of
>> Python, this is a performance win for "command-line tool platform" systems
>> like Mercurial and PEAK, where you want to delay importing as long as
>> possible, in case the code that needs the import is never called at all...
>>  but, if it *is* used, you may still need to use it a lot of times.
>>
>> When writing that kind of code, I usually just unconditionally import
>> inside the function, because the C code check for an already-imported
>> module is faster than the Python "if" statement I'd have to clutter up my
>> otherwise-clean function with.
>>
>> So, in addition to the things other people have mentioned as performance
>> targets, I'd like to keep the slowdown factor low for this type of scenario
>> as well.  Specifically, the slowdown shouldn't be so much as to motivate
>> lazy importers like Mercurial and PEAK to need to rewrite in-function
>> imports to do the already-imported check ourselves.  ;-)
>>
>> (Disclaimer: I haven't actually seen Mercurial's delayed/dynamic import
>> code, so I can't say for 100% sure if they'd be affected the same way.)
>>
>
> IOW you want the sys.modules case fast, which I will never be able to
> match compared to C code since that is pure execution with no I/O.
>

Couldn't you just prefix the __import__ function with something like this:

     ...
     try:
          module = sys.modules[name]
     except KeyError:
          # slow code path

(Admittedly, the import lock is still a problem; initially I thought you
could just skip it for this case, but the problem is that another thread
could be in the middle of executing the module.)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20120207/f3c406ee/attachment.html>


More information about the Python-Dev mailing list