[Patches] [ python-Patches-1553314 ] Fix inspect.py 2.5 slowdown

[SourceForge.net]

Patches item #1553314, was opened at 2006-09-06 10:51
Message generated for change (Comment added) made by pje
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1553314&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Library (Lib)
Group: Python 2.5
Status: Open
Resolution: None
Priority: 9
Submitted By: Nick Coghlan (ncoghlan)
Assigned to: Neal Norwitz (nnorwitz)
Summary: Fix inspect.py 2.5 slowdown

Initial Comment:
Every call to inspect.findsource() in Python 2.5 is
calling both os.path.abspath() and os.path.normpath(),
even if the file name is already absolute and
normalized, and the relevant entry is in the module
name cache.

This patch changes getmodule() to first check the
modulesbyfile cache earlier, so these calls are skipped
when the module name for the file is already known.

----------------------------------------------------------------------

>Comment By: Phillip J. Eby (pje)
Date: 2006-09-06 13:49

Message:
Logged In: YES 
user_id=56214

I just looked at the current patch; please don't use
sys.modules.iteritems() in your search loop; it'll break if
touching a module causes another module to be imported. 
This can happen if somebody is using one of the many "lazy
import" tools out there.  Use .items() instead.

Second, there was a reason that the loop used
module.__name__ instead of the sys.modules keys; a module
can be listed under more than one sys.modules key, so the
loop was storing the *canonical* name of the module.  I'm
not 100% positive this is a critical thing, but I'm
suspicious of the idea of just randomly choosing which of
the possible keys should be used.


----------------------------------------------------------------------

Comment By: Nick Coghlan (ncoghlan)
Date: 2006-09-06 13:04

Message:
Logged In: YES 
user_id=1038590

Bah. The linecache version has the same problem as the
previous version - it's the failure case that searches the
whole of sys.modules that is slow, not the success case.

Patch attached which stores a mapping from module names to
module.__file__ values and skips the absolute lookup if the
value of module.__file__ hasn't changed.



----------------------------------------------------------------------

Comment By: Nick Coghlan (ncoghlan)
Date: 2006-09-06 12:48

Message:
Logged In: YES 
user_id=1038590

Low impact approach that simply checks the linecache
directly before doing anything.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1553314&group_id=5470