[issue7173] Cython compiler run crashes CPython 3.1.1 and 3.2

[Stefan Behnel]

Stefan Behnel <scoder at users.sourceforge.net> added the comment:

I'll add a couple of comments about the relevant parts of the code that appears to trigger the crash.

The code runs through the parse tree and applies transformations to it.

1) For node matching, we use a dispatcher (in Visitor.py) that looks up methods for node class names (or the nearest matching class in its MRO hierarchy), e.g. a plain "Node" type would be handled by a "visit_Node" method, whereas an ExprNode(Node) type would match "visit_ExprNode" if it exists and "visit_Node" otherwise. The handler then recursively calls back into the dispatch code to visit its children.

For performance reasons, we sometimes reuse existing methods for this through assignment, e.g. instead of reimplementing the default fallback "visit_Node" all over the place, we assign an existing superclass method to it as in "visit_Node = VisitorTransform.recurse_to_children". This may or may not have an impact on this problem (it shouldn't - normally).

So far, all of this appears to work perfectly fine also in Py3. It's the next step that seems to be required to trigger the crash.

2) To specify assertions in our test suite, we use an XPath-like language for node tests (poor man's Schematron, sort-of). The implementation in TreePath.py (based on "ElementPath" in ElementTree 1.3) makes heavy use of generators to traverse the tree. It is used by the TreeAssertVisitor in TestUtils.py, which is itself a recursive tree visitor (as described above).

So the situation is as follows. Cython traverses the parse tree recursively, several times in a pipeline, dispatching to methods that further recurse into the tree. One of the pipeline steps is the tree assertion visitor, which, on matching functions, applies the generator-based path language to the tree, that tries to find nodes by iterating over a chain of generators.

At some point in this process, somehow, sys.exc_info gets corrupted, and the doctest/unittest framework crashes the runtime with a segfault when trying to print an exception (raised unexpectedly by the "pure" test).

I hope this makes it clearer a) what happens and b) why it's not easy to cut this down into a simple test case. As the previous comments show, it's already hard enough to reproduce it reliably with a reduced test setup.

Please do not hesitate to contact me if you need any further information.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue7173>
_______________________________________