[Cython] Slow traceback reconstruction in IPython between 0.15.1 and master

[Stefan Behnel]

Wes McKinney, 26.01.2012 18:56:
> Just wanted to bring this issue to your guys' attention in case you
> knew what was responsible for this:
> 
> https://github.com/ipython/ipython/issues/1317#issuecomment-3652550
> 
> I traced down the problem (with git bisect) to a seemingly innocuous
> commit referenced in that GitHub thread. The issue seemed to only
> present itself in IPython, so likely there was some problem with
> inspecting the Cython frames for giving context around the full
> traceback.

That's not impossible. Traceback frames behave differently in Cython for
two reasons: a) because they are only being constructed after the fact in
an error handling case, not for normal functions, and b) because Cython
doesn't have real code objects for inspection and fakes them in a rather
ad-hoc way. For example, there is currently no function signature
information in them, and line number computation doesn't use the normal
CPython way that matches the byte code position with a compressed line
table. Instead, Cython creates a new code object for a given line on the
fly and just pretends that the function starts there. This usually works
well enough for a traceback, but this kind of differences makes it quite
possible that IPython makes assumptions about inspection here that Cython
doesn't meet.

In another thread ("AddTraceback() slows down generators"), I was proposing
to cache the code object for a given function. That would then imply
providing a (fake) byte code position map as well and would make it easier
to provide static signature information, thus improving the overall
compatibility with the code objects that CPython creates.

Note that it's unclear without further investigation if the problem you ran
into really has to do with these internal details. I'm just raising a
possible explanation here.

Stefan