[issue32797] Tracebacks from Cython modules no longer work

[Erik Bray]

Erik Bray <erik.m.bray at gmail.com> added the comment:

> To do that, I believe it can be made to work in much the same way it did in Python 2 if SageMath were to do the following:
>
> 1. Define a subclass of ExtensionModuleLoader [1] that overrides get_source() to also look for a ".pyx" file adjacent to the extension module. This would also look for any of the file suffixes in SOURCE_SUFFIXES if a .pyx file isn't found.
> 2. Create an instance of FileFinder [3] that uses the custom loader subclass for any of the file suffixes in EXTENSION_SUFFIXES [4]
> 3. Replace the regular file finding hook in sys.path_hooks with  the path_hook method from that new FileFinder instance (by default, there's only one FileFinder hook installed, and it can currently be identified as "obj.__name__ == 'path_hook_for_FileFinder')
> 4. Invalidate importlib's caches, so any future extension module imports will use the custom extension module loader, rather than the default one

This is pretty edifying, because Nick's idea is almost exactly what I did six months ago :)  https://git.sagemath.org/sage.git/diff/?id2=0a674fd488dcd7cb779101d263c10a874a13cf77&id=8b63abe731c510f0de9ef0e3ab9a0bda3669dce1

Turned out to be very non-trivial of course, and I believe it should not have been as complicated as it was.

It also still doesn't solve the problem that Loader.get_source does not support multiple source files, which Cython code may have (a .pyx and a .pxd being a common case).  I'm glad Paul Moore seems to also agree (now that I've actually read the rest of the thread) that the ExtensionLoader.get_source, at the very least, could be made more useful.  Whatever form that takes would be worth extending to other loaders that implement get_source as well...

----------

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue32797>
_______________________________________