[pypy-dev] PyPy for analysis?

[William ML Leslie]

Ugh, thanks Gmail (:


On 8 September 2014 01:33, William ML Leslie <william.leslie.ttg at gmail.com>
wrote:

> On 7 September 2014 21:42, Scott West <scott.gregory.west at gmail.com>
> wrote:
>
>> Hello all,
>>
>> I was looking recently into trying to do some simple static analysis of
>> Python programs (to experiment with what is possible), and came across
>> PyPy. Reading some of the documentation it seems that PyPy forms a control
>> flow graph and does some abstract interpretation, which seems to indicate
>> that it could be used for other forms of static analysis!
>>
>
> Static analysis of python programs is not difficult, weather you use ast
> or code objects.  The flow object space really only works on rpython
> though, using it to analyse python code generally is not sensible.
>
> I do think pypy is a great platform for associating axioms with builtin
> functions and bytecodes (and sometimes other weird stuff like exception
> handling, depending on your analysis) which you would otherwise have to
> write by hand.​  If you want to do that, target the llgraph stage of the
> compiler, as there is nothing 'outside' it that you need to rely upon.  You
> can possibly even generate the app-level analyser from the interpreter, but
> in order to do this I had to rewrite the eval loop - I could never figure
> out how to extract semantics from the frame object.
>
> I did at one point have something like a points-to analysis for
> interp-level code working (a region polymorphic effects analysis), but it
> has bitrotten beyond repair.  I would still say pypy is a great place to
> start, however.
>
>
>>
>> I had a couple questions:
>>   1) Would PyPy (seem) to be a good place to start for a general analysis
>> framework? If so, would it be of general interest?
>>   2) If one doesn't care about code generation, can the build time be
>> reduced? I tried just using pyinteractive.py and that _seemed_ to work,
>> though even that compiles a few things to start.
>>
>
> By about 40% if you target llgraph by my last count.  You could avoid
> building the jit, since it should not have any semantic impact.​
>
>
>
>>   3) What is a good way to get into modifying the internals (adding
>> analyses, skipping code gen.)? I have read the chapter of the Open Source
>> Architecture book, and some of the documentation pages.
>> ​​
>>
>>
> 0) ​Write lots of tests.​
>
> ​ The existing tests make for good starting points.​
> ​​
> ​1) ​
> ​Start with a very small interpreter, not the full pypy interpreter.  IIRC
> there was a language called TL which was for testing the JIT, use it as the
> target until everything works.​
>
> 2) Dumping the llgraph to disk somewhere - perhaps into a logic or
> relational database - will make little experiments easier.
>
> ​​
> ​​
>
>> ​​
>> ​​
>>
>> I w
>> ​​
>> ​​
>> ould be most grateful if anyone could provide any comments on these
>> issues, or pointers to other similar works.
>>
>
> ​Unfortunately I have no idea what state my work was in when I left it, I
> don't even have it on my current box; I would like to get back to it at
> some point but Armin suggested I look at better low level languages to
> target and I'm still working in that space.​
>
> --
> William Leslie
>
> Notice:
> Likely much of this email is, by the nature of copyright, covered under
> copyright law.  You absolutely MAY reproduce any part of it in accordance
> with the copyright law of the nation you are reading this in.  Any attempt
> to DENY YOU THOSE RIGHTS would be illegal without prior contractual
> agreement.
>



-- 
William Leslie

Notice:
Likely much of this email is, by the nature of copyright, covered under
copyright law.  You absolutely MAY reproduce any part of it in accordance
with the copyright law of the nation you are reading this in.  Any attempt
to DENY YOU THOSE RIGHTS would be illegal without prior contractual
agreement.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20140908/d084aaf1/attachment.html>