[Python-Dev] Preprocessing the CPython Source Tree

[Paul Ross]

I have implemented a C preprocessor written in Python which gives some
useful visualisations of source code, particularly macro usage:
https://github.com/paulross/cpip

I have been running this on the CPython source code and it occurs to me
that this might be useful to the python-dev community.

For example the Python dictionary source code is visualised here:
http://cpip.readthedocs.io/en/latest/_static/dictobject.c/
index_dictobject.c_a3f5bfec1ed531371fb1a2bcdcb2e9c2.html I found this
really useful when I was getting a segfault during a dictionary insert from
my C code. The segfault was on this line http://cpip.readthedocs.io/en/
latest/_static/dictobject.c/dictobject.c_a3f5bfec1ed531371fb1a2bcdcb2e9
c2.html#1130 but it is hard to see what is going on with macros inside
macros. If you click on the link on the left end of the line it takes you
to the full expansion of the macros http://cpip.readthedocs.io/en/
latest/_static/dictobject.c/dictobject.c.html#1130 as this is what the
compiler and debugger see.
I could examine these values in GDB and figure out what was going on. I
could also figure what that MAINTAIN_TRACKING macro was doing by looking at
the macros page generated by CPIP: http://cpip.readthedocs.io/en/
latest/_static/dictobject.c/macros_ref.html#_TUFJTlRBSU5fVFJBQ0tJTkdfMA__
and following those links.

I was wondering if it would be valuable to python-dev developers if this
tool was run regularly over the CPython source tree(s). A single source
tree takes about 12 CPU hours to process and generates 8GB of HTML/SVG. If
this is useful then where to host this?

Regards,

Paul Ross
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20171016/0c89be5b/attachment.html>