[pytest-dev] Enabling viewing of rewritten AST trees as Python

[Tom Viner]

Hi,

In the process of fixing a recent bug
<https://bitbucket.org/hpk42/pytest/issue/615/valueerror-on-compound-assert-with-percent>
in pytest source rewriting I found a library to take the rewritten AST tree
and generate Python source code from it. Floris mentioned this could be
helpful to pytest developers working on the assertion rewriting code.

The library is called Meta <https://pypi.python.org/pypi/meta>.
(Note, currently only the develop branch on github works successfully on
pytest's rewrite tests.) It allows viewing an AST tree as Python source
with a single call:

import meta
log.debug(meta.asttools.dump_python_source(ast_tree))

The question I have: how can pytest developers (and those debugging
pytest's rewriting of their tests) best get access to the rewritten AST
tree, to be able to view it's Python source representation?

The AST tree is seen in pytest/_pytest/assertion/rewrite.py::rewrite_asserts
<https://bitbucket.org/hpk42/pytest/src/default/_pytest/assertion/rewrite.py?at=default#cl-328>
in
the mod variable:


def rewrite_asserts(mod):    """Rewrite the assert statements in
mod."""    AssertionRewriter().run(mod)


The four options I can think of to gain access to this variable:
- monkey patch rewrite.rewrite_asserts
- edit rewrite_asserts and paste in the logging code above:
    -- temporarily, by each developer, on each occasion required
    -- permanently, but only running when a certain debug flag is active
- expose the rewritten AST via a new hook

monkey patch is my preferred option, because if I can make it work, a
plugin could be written. I just need some help to find an appropriate hook
to do the monkey patching before pytest gets going with the
AssertionRewritingHook and calls rewrite_asserts. Without knowledge of an
early-running hook, I could only get this working by patching
rewrite_asserts then calling pytest.main myself, see
https://gist.github.com/tomviner/13c95cdb1e159028fc0b

Second option, I could explain how to do the temporary edit in a short blog
post. This would get past the pytest.main downside to the gist above. But
telling people to edit random pytest source files seems bad form.

Third option, a pull request on pytest logging the Python source, only if a
debug flag is active and Meta is importable. But is this reasonable just
for debugging purposes?

Alternatively, fourth, a new hook could be added to expose the rewritten
AST tree, and everything else could then be a plugin.

Thoughts? To see the assertion rewriting pytest does, as Python code,
follow the instructions in my gist:
https://gist.github.com/tomviner/13c95cdb1e159028fc0b

Cheers,
Tom
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pytest-dev/attachments/20141109/6553188e/attachment.html>