[pypy-svn] r60806 - in pypy/extradoc/talk/sfi2008: . demo
fijal at codespeak.net
fijal at codespeak.net
Mon Jan 5 21:44:45 CET 2009
Author: fijal
Date: Mon Jan 5 21:44:44 2009
New Revision: 60806
Added:
pypy/extradoc/talk/sfi2008/talk.html
Modified:
pypy/extradoc/talk/sfi2008/demo/rainbow-view-jit-graphs
pypy/extradoc/talk/sfi2008/demo/targetrpn.py
pypy/extradoc/talk/sfi2008/talk.txt
Log:
last version including html
Modified: pypy/extradoc/talk/sfi2008/demo/rainbow-view-jit-graphs
==============================================================================
--- pypy/extradoc/talk/sfi2008/demo/rainbow-view-jit-graphs (original)
+++ pypy/extradoc/talk/sfi2008/demo/rainbow-view-jit-graphs Mon Jan 5 21:44:44 2009
@@ -1,5 +1,8 @@
#! /usr/bin/env python
+import sys
+sys.path.insert(0, '/home/fijal/lang/python/pypy-rainbow')
+
import rpn
from pypy.conftest import option
option.view = True
@@ -20,4 +23,4 @@
return rpn.interpret(code, arg)
self.timeshift_from_portal(main, rpn.interpret,
- [1, 100], policy=P_OOPSPEC)
+ [2, 100], policy=P_OOPSPEC)
Modified: pypy/extradoc/talk/sfi2008/demo/targetrpn.py
==============================================================================
--- pypy/extradoc/talk/sfi2008/demo/targetrpn.py (original)
+++ pypy/extradoc/talk/sfi2008/demo/targetrpn.py Mon Jan 5 21:44:44 2009
@@ -12,3 +12,4 @@
def target(*args):
return entry_point, None
+
Added: pypy/extradoc/talk/sfi2008/talk.html
==============================================================================
--- (empty file)
+++ pypy/extradoc/talk/sfi2008/talk.html Mon Jan 5 21:44:44 2009
@@ -0,0 +1,660 @@
+<?xml version="1.0" encoding="utf-8" ?>
+<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+<head>
+<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
+<meta name="generator" content="Docutils 0.4.1: http://docutils.sourceforge.net/" />
+<meta name="version" content="S5 1.1" />
+<title>PyPy</title>
+<meta name="author" content="Maciej Fijalkowski, merlinux GmbH" />
+<meta name="date" content="6th March 2008" />
+<style type="text/css">
+
+/*
+:Author: David Goodger
+:Contact: goodger at users.sourceforge.net
+:Date: $Date: 2005-12-18 01:56:14 +0100 (Sun, 18 Dec 2005) $
+:Revision: $Revision: 4224 $
+:Copyright: This stylesheet has been placed in the public domain.
+
+Default cascading style sheet for the HTML output of Docutils.
+
+See http://docutils.sf.net/docs/howto/html-stylesheets.html for how to
+customize this style sheet.
+*/
+
+/* used to remove borders from tables and images */
+.borderless, table.borderless td, table.borderless th {
+ border: 0 }
+
+table.borderless td, table.borderless th {
+ /* Override padding for "table.docutils td" with "! important".
+ The right padding separates the table cells. */
+ padding: 0 0.5em 0 0 ! important }
+
+.first {
+ /* Override more specific margin styles with "! important". */
+ margin-top: 0 ! important }
+
+.last, .with-subtitle {
+ margin-bottom: 0 ! important }
+
+.hidden {
+ display: none }
+
+a.toc-backref {
+ text-decoration: none ;
+ color: black }
+
+blockquote.epigraph {
+ margin: 2em 5em ; }
+
+dl.docutils dd {
+ margin-bottom: 0.5em }
+
+/* Uncomment (and remove this text!) to get bold-faced definition list terms
+dl.docutils dt {
+ font-weight: bold }
+*/
+
+div.abstract {
+ margin: 2em 5em }
+
+div.abstract p.topic-title {
+ font-weight: bold ;
+ text-align: center }
+
+div.admonition, div.attention, div.caution, div.danger, div.error,
+div.hint, div.important, div.note, div.tip, div.warning {
+ margin: 2em ;
+ border: medium outset ;
+ padding: 1em }
+
+div.admonition p.admonition-title, div.hint p.admonition-title,
+div.important p.admonition-title, div.note p.admonition-title,
+div.tip p.admonition-title {
+ font-weight: bold ;
+ font-family: sans-serif }
+
+div.attention p.admonition-title, div.caution p.admonition-title,
+div.danger p.admonition-title, div.error p.admonition-title,
+div.warning p.admonition-title {
+ color: red ;
+ font-weight: bold ;
+ font-family: sans-serif }
+
+/* Uncomment (and remove this text!) to get reduced vertical space in
+ compound paragraphs.
+div.compound .compound-first, div.compound .compound-middle {
+ margin-bottom: 0.5em }
+
+div.compound .compound-last, div.compound .compound-middle {
+ margin-top: 0.5em }
+*/
+
+div.dedication {
+ margin: 2em 5em ;
+ text-align: center ;
+ font-style: italic }
+
+div.dedication p.topic-title {
+ font-weight: bold ;
+ font-style: normal }
+
+div.figure {
+ margin-left: 2em ;
+ margin-right: 2em }
+
+div.footer, div.header {
+ clear: both;
+ font-size: smaller }
+
+div.line-block {
+ display: block ;
+ margin-top: 1em ;
+ margin-bottom: 1em }
+
+div.line-block div.line-block {
+ margin-top: 0 ;
+ margin-bottom: 0 ;
+ margin-left: 1.5em }
+
+div.sidebar {
+ margin-left: 1em ;
+ border: medium outset ;
+ padding: 1em ;
+ background-color: #ffffee ;
+ width: 40% ;
+ float: right ;
+ clear: right }
+
+div.sidebar p.rubric {
+ font-family: sans-serif ;
+ font-size: medium }
+
+div.system-messages {
+ margin: 5em }
+
+div.system-messages h1 {
+ color: red }
+
+div.system-message {
+ border: medium outset ;
+ padding: 1em }
+
+div.system-message p.system-message-title {
+ color: red ;
+ font-weight: bold }
+
+div.topic {
+ margin: 2em }
+
+h1.section-subtitle, h2.section-subtitle, h3.section-subtitle,
+h4.section-subtitle, h5.section-subtitle, h6.section-subtitle {
+ margin-top: 0.4em }
+
+h1.title {
+ text-align: center }
+
+h2.subtitle {
+ text-align: center }
+
+hr.docutils {
+ width: 75% }
+
+img.align-left {
+ clear: left }
+
+img.align-right {
+ clear: right }
+
+ol.simple, ul.simple {
+ margin-bottom: 1em }
+
+ol.arabic {
+ list-style: decimal }
+
+ol.loweralpha {
+ list-style: lower-alpha }
+
+ol.upperalpha {
+ list-style: upper-alpha }
+
+ol.lowerroman {
+ list-style: lower-roman }
+
+ol.upperroman {
+ list-style: upper-roman }
+
+p.attribution {
+ text-align: right ;
+ margin-left: 50% }
+
+p.caption {
+ font-style: italic }
+
+p.credits {
+ font-style: italic ;
+ font-size: smaller }
+
+p.label {
+ white-space: nowrap }
+
+p.rubric {
+ font-weight: bold ;
+ font-size: larger ;
+ color: maroon ;
+ text-align: center }
+
+p.sidebar-title {
+ font-family: sans-serif ;
+ font-weight: bold ;
+ font-size: larger }
+
+p.sidebar-subtitle {
+ font-family: sans-serif ;
+ font-weight: bold }
+
+p.topic-title {
+ font-weight: bold }
+
+pre.address {
+ margin-bottom: 0 ;
+ margin-top: 0 ;
+ font-family: serif ;
+ font-size: 100% }
+
+pre.literal-block, pre.doctest-block {
+ margin-left: 2em ;
+ margin-right: 2em ;
+ background-color: #eeeeee }
+
+span.classifier {
+ font-family: sans-serif ;
+ font-style: oblique }
+
+span.classifier-delimiter {
+ font-family: sans-serif ;
+ font-weight: bold }
+
+span.interpreted {
+ font-family: sans-serif }
+
+span.option {
+ white-space: nowrap }
+
+span.pre {
+ white-space: pre }
+
+span.problematic {
+ color: red }
+
+span.section-subtitle {
+ /* font-size relative to parent (h1..h6 element) */
+ font-size: 80% }
+
+table.citation {
+ border-left: solid 1px gray;
+ margin-left: 1px }
+
+table.docinfo {
+ margin: 2em 4em }
+
+table.docutils {
+ margin-top: 0.5em ;
+ margin-bottom: 0.5em }
+
+table.footnote {
+ border-left: solid 1px black;
+ margin-left: 1px }
+
+table.docutils td, table.docutils th,
+table.docinfo td, table.docinfo th {
+ padding-left: 0.5em ;
+ padding-right: 0.5em ;
+ vertical-align: top }
+
+table.docutils th.field-name, table.docinfo th.docinfo-name {
+ font-weight: bold ;
+ text-align: left ;
+ white-space: nowrap ;
+ padding-left: 0 }
+
+h1 tt.docutils, h2 tt.docutils, h3 tt.docutils,
+h4 tt.docutils, h5 tt.docutils, h6 tt.docutils {
+ font-size: 100% }
+
+tt.docutils {
+ background-color: #eeeeee }
+
+ul.auto-toc {
+ list-style-type: none }
+
+</style>
+<!-- configuration parameters -->
+<meta name="defaultView" content="slideshow" />
+<meta name="controlVis" content="hidden" />
+<!-- style sheet links -->
+<script src="ui/default/slides.js" type="text/javascript"></script>
+<link rel="stylesheet" href="ui/default/slides.css"
+ type="text/css" media="projection" id="slideProj" />
+<link rel="stylesheet" href="ui/default/outline.css"
+ type="text/css" media="screen" id="outlineStyle" />
+<link rel="stylesheet" href="ui/default/print.css"
+ type="text/css" media="print" id="slidePrint" />
+<link rel="stylesheet" href="ui/default/opera.css"
+ type="text/css" media="projection" id="operaFix" />
+
+<style type="text/css">
+#currentSlide {display: none;}
+</style>
+</head>
+<body>
+<div class="layout">
+<div id="controls"></div>
+<div id="currentSlide"></div>
+<div id="header">
+
+</div>
+<div id="footer">
+<h1>PyPy</h1>
+
+</div>
+</div>
+<div class="presentation">
+<div class="slide" id="slide0">
+<h1 class="title">PyPy</h1>
+<table class="docinfo" frame="void" rules="none">
+<col class="docinfo-name" />
+<col class="docinfo-content" />
+<tbody valign="top">
+<tr><th class="docinfo-name">Author:</th>
+<td>Maciej Fijalkowski, merlinux GmbH</td></tr>
+<tr class="field"><th class="docinfo-name">Title:</th><td class="field-body">PyPy - Automatic Generation of VMs for Dynamic Languages</td>
+</tr>
+<tr><th class="docinfo-name">Date:</th>
+<td>6th March 2008</td></tr>
+</tbody>
+</table>
+
+</div>
+<div class="slide" id="vms-for-dynamic-languages">
+<h1>VMs for dynamic languages</h1>
+<ul class="simple">
+<li>Example - Python</li>
+<li>Coded in a low-level language, such as C (or Java or C#)</li>
+<li>Hard-coded design decisions (garbage collection, threading model)</li>
+<li>Primary example - CPython.<ul>
+<li>refcounting fixed hard to change</li>
+<li>global interpreter lock hard to change</li>
+</ul>
+</li>
+</ul>
+</div>
+<div class="slide" id="ideally-we-would">
+<h1>Ideally, we would...</h1>
+<ul class="simple">
+<li>Use a high-level language</li>
+<li>Use a statically analyzable implementation language</li>
+<li>Have a flexible compiler toolchain to do part of the job for you</li>
+</ul>
+</div>
+<div class="slide" id="generating-vms">
+<h1>Generating VMs</h1>
+<ul class="simple">
+<li>Writing by hand is a lot of trouble</li>
+<li>We want to perform each of the following once:<ul>
+<li>encode the language semantics</li>
+<li>encode language-agnostic design decisions (ie object model)</li>
+<li>encode platform details</li>
+</ul>
+</li>
+</ul>
+</div>
+<div class="slide" id="pypy-motivation">
+<h1>PyPy Motivation</h1>
+<ul class="simple">
+<li>Create highly-flexible compiler toolchain</li>
+<li>Use high-level language for encoding semantics</li>
+<li>Use compiler toolchain to try hard to make
+things such as object model orthogonal to language semantics
+encoding</li>
+</ul>
+</div>
+<div class="slide" id="development-technology">
+<h1>Development technology</h1>
+<ul class="simple">
+<li>We use test driven development extensively</li>
+<li>We've got a test suite which would need ~8 hours to run on a single processor</li>
+<li>Our special testing tool, py.test, little hassle and a lot of features</li>
+<li>Sprint-driven development</li>
+</ul>
+</div>
+<div class="slide" id="general-idea">
+<h1>General idea</h1>
+<img src="overview1.png"/></div>
+<div class="slide" id="even-more">
+<h1>Even more</h1>
+<ul class="simple">
+<li>Take any interpreter, python, javascript, smalltalk...</li>
+<li>Make some decisions (which gc to use...)</li>
+<li>Compile it to your platform (C/POSIX, .NET, ...)</li>
+</ul>
+</div>
+<div class="slide" id="an-example-of-benefit">
+<h1>An example of benefit</h1>
+<ul class="simple">
+<li>Python has complicated semantics</li>
+<li>Python guarantees that it won't segfault on
+a stack exhaustion</li>
+<li>CPython includes some stack checks in the source, but they
+don't catch every case</li>
+<li>We include it automatically so all cases are guaranteed to be covered</li>
+</ul>
+</div>
+<div class="slide" id="example-of-static-analysis">
+<h1>Example of static analysis</h1>
+<ul class="simple">
+<li>we analyze interpreter source, not user code!</li>
+<li>replace all external calls with stubs</li>
+<li>sandboxing - a very small piece of code to trust</li>
+<li>build your own library (in python) which implements
+your own security policy</li>
+<li>no need to worry about third party modules</li>
+</ul>
+</div>
+<div class="slide" id="rpython">
+<h1>RPython</h1>
+<ul class="simple">
+<li>We use RPython, a proper subset of Python to implement
+a Python interpreter</li>
+<li>More static than Python</li>
+<li>Designed for speed, and designed for writing interpreters</li>
+<li>Still high-level, fully analyzable</li>
+<li>User code (executed by the interpreter) is full python (not restricted)</li>
+</ul>
+</div>
+<div class="slide" id="abstract-interpretation">
+<h1>Abstract interpretation</h1>
+<ul class="simple">
+<li>We start by importing the RPython modules on top of CPython</li>
+<li>Next we analyze the bytecode and produce a forest of flow graphs</li>
+<li>We call this the abstract interpretation of bytecode</li>
+<li>The bytecode can be prepared dynamically (meta-programming)</li>
+</ul>
+</div>
+<div class="slide" id="flow-graphs">
+<h1>Flow graphs</h1>
+<ul class="simple">
+<li>Intermediate representation</li>
+<li>Can encode different layers of abstraction</li>
+</ul>
+</div>
+<div class="slide" id="annotation">
+<h1>Annotation</h1>
+<ul class="simple">
+<li>The part which does the type inference over existing
+graphs</li>
+<li>A Powerful but practical type system</li>
+<li>Still very high-level</li>
+</ul>
+</div>
+<div class="slide" id="rtyper">
+<h1>RTyper</h1>
+<ul class="simple">
+<li>Translates high-level graphs into lower-levels</li>
+<li>Two type systems:<ul>
+<li>lltype, for backends that understand pointers,
+structures etc.</li>
+<li>ootype, for backends that have notion of objects</li>
+</ul>
+</li>
+<li>lltype has more low-level details, such as an implementation of strings
+(ootype assumes strings built-in)</li>
+</ul>
+</div>
+<div class="slide" id="translation-aspects">
+<h1>Translation aspects</h1>
+<ul class="simple">
+<li>Optimizations (inlining, malloc removal)</li>
+<li>Different GCs</li>
+<li>...</li>
+</ul>
+</div>
+<div class="slide" id="gc-framework">
+<h1>GC framework</h1>
+<ul class="simple">
+<li>flexible</li>
+<li>implemented as a translation aspect</li>
+<li>gc-specific things, such as write barriers, are inserted automatically</li>
+<li>different root finding strategies</li>
+<li>effective (RPython is faster than C when GC is a bottleneck)</li>
+<li>GCs are written in RPython as well</li>
+<li>Motivation: MMtk</li>
+</ul>
+</div>
+<div class="slide" id="backends">
+<h1>Backends</h1>
+<ul class="simple">
+<li>lltype-based - C, LLVM</li>
+<li>ootype-based - CLI, JVM, ...</li>
+<li>llinterpreter - a virtual backend for running flow graphs
+directly, useful for testing</li>
+<li>.NET bindings, for example, are backend specific</li>
+</ul>
+</div>
+<div class="slide" id="special-translation-aspect">
+<h1>Special translation aspect</h1>
+<ul class="simple">
+<li>generating Just-in-time compilers from interpreters</li>
+</ul>
+</div>
+<div class="slide" id="jit-motivation">
+<h1>JIT - motivation</h1>
+<ul class="simple">
+<li>Interpreters are much easier to write than compilers</li>
+<li>For many languages, one cannot achieve C-level performance
+with static analysis</li>
+<li>Dynamic compilation can produce faster results
+than static compilation (e.g. a good Java VM versus gcj)</li>
+</ul>
+</div>
+<div class="slide" id="traditional-jit-approach">
+<h1>Traditional JIT approach</h1>
+<ul class="simple">
+<li>Written by hand</li>
+<li>Carefully encoded language semantics</li>
+<li>Hard to maintain as language evolves</li>
+<li>We can do better!</li>
+</ul>
+</div>
+<div class="slide" id="pypy-approach-to-jit">
+<h1>PyPy approach to JIT</h1>
+<div>
+<img src="overview2.png"/>
+</div></div>
+<div class="slide" id="jit-basics">
+<h1>JIT - basics</h1>
+<ul class="simple">
+<li>partial evaluation</li>
+<li>automatic generation of a compiler from an interpreter</li>
+<li>an old idea with relatively few practical applications so far</li>
+</ul>
+</div>
+<div class="slide" id="jit-general-idea">
+<h1>JIT - general idea</h1>
+<ul class="simple">
+<li>constant-propagate python bytecode through the interpreter</li>
+<li>may not yield good performance (our experiments show
+about 2x for removing intepretation overhead)</li>
+<li>things such as the types of Python objects are still not known</li>
+</ul>
+</div>
+<div class="slide" id="solution-promotion">
+<h1>Solution: promotion</h1>
+<ul class="simple">
+<li>Enhance partial evaluation to be able to <em>promote</em> run-time
+values into compile-time values</li>
+<li>Implementation-wise this is a generalization of the polymorphic
+in-line caches used in Java VM JITs</li>
+</ul>
+</div>
+<div class="slide" id="concrete-ingredients">
+<h1>Concrete ingredients</h1>
+<ul class="simple">
+<li>Variety of hints to guide partial evaluation</li>
+<li>Promotion on certain paths to achieve
+"static-enough" parts</li>
+<li>Lazy allocation of objects (allocation only happens when the object escapes)</li>
+<li>Use CPU stack and registers as a cache for Python frame
+objects that live in the heap</li>
+</ul>
+</div>
+<div class="slide" id="irrelevant-to-interpreter">
+<h1>Irrelevant to interpreter</h1>
+<ul class="simple">
+<li>These techniques can be applied relatively easily
+to other interpreters, just by adding a few hints</li>
+<li>Or to any other program written in RPython
+(for instance a templating language where the templates are constant)</li>
+</ul>
+</div>
+<div class="slide" id="the-hint-annotator">
+<h1>The hint annotator</h1>
+<ul class="simple">
+<li>This annotator is very similiar to the normal one; it
+adds color propagation</li>
+<li>Green - compile-time, can be constant-folded</li>
+<li>Red - runtime</li>
+<li>Some rules about the color propagation</li>
+</ul>
+</div>
+<div class="slide" id="the-rainbow-interpreter">
+<h1>The rainbow interpreter</h1>
+<ul class="simple">
+<li>Very very experimental, lives on a branch right now</li>
+<li>We dump colored graphs as a bytecode and interpret them</li>
+</ul>
+</div>
+<div class="slide" id="jit-details-backends">
+<h1>JIT details - backends</h1>
+<ul class="simple">
+<li>JIT backends produce assembly code from coloured graphs</li>
+<li>As usual, we plan to have many (PPC, i386, JVM,
+dummy one...)</li>
+<li>They're very rudimentary right now</li>
+<li>It's a relatively pleasant job to write Python that generates
+assembler code</li>
+<li>We would definitely benefit from help from an assembler
+expert</li>
+</ul>
+</div>
+<div class="slide" id="jit-conclusion">
+<h1>JIT conclusion</h1>
+<ul class="simple">
+<li>We're able to run carefully crafted examples ~60x faster
+than CPython</li>
+<li>About the same as gcc -O0</li>
+<li>We can definitely do better, for example by enhancing the backends</li>
+<li>There is a lot of work to be done in JIT area</li>
+</ul>
+</div>
+<div class="slide" id="jit-plans">
+<h1>JIT plans</h1>
+<ul class="simple">
+<li>Faster than C!</li>
+<li>For long-enough running programs that's theoretically
+possible</li>
+<li>We have more information at runtime than one can have
+at compile-time</li>
+</ul>
+</div>
+<div class="slide" id="jit-short-term-plans">
+<h1>JIT short term plans</h1>
+<ul class="simple">
+<li>Compile just hotspots (not everything)</li>
+<li>Extend backends and make them smarter</li>
+<li>Make generating assembly code faster</li>
+<li>Benchmark bottlenecks</li>
+</ul>
+</div>
+<div class="slide" id="it-s-not-only-technical">
+<h1>It's not only technical</h1>
+<ul class="simple">
+<li>EU 6th Framework Programme</li>
+<li>Actively searching for both grants and consultancy contracts</li>
+</ul>
+</div>
+<div class="slide" id="project-future">
+<h1>Project future</h1>
+<ul class="simple">
+<li>More JIT work</li>
+<li>Making PyPy's python interpreter run existing
+applications</li>
+<li>Interfacing with external libraries</li>
+<li>We're very friendly for newcomers</li>
+</ul>
+</div>
+</div>
+</body>
+</html>
Modified: pypy/extradoc/talk/sfi2008/talk.txt
==============================================================================
--- pypy/extradoc/talk/sfi2008/talk.txt (original)
+++ pypy/extradoc/talk/sfi2008/talk.txt Mon Jan 5 21:44:44 2009
@@ -40,7 +40,7 @@
- encode the language semantics
- - encode language-agnostic design decisions (ie threading model)
+ - encode language-agnostic design decisions (ie object model)
- encode platform details
@@ -102,7 +102,7 @@
* replace all external calls with stubs
-* a very small piece of code to trust
+* sandboxing - a very small piece of code to trust
* build your own library (in python) which implements
your own security policy
@@ -268,8 +268,6 @@
* Implementation-wise this is a generalization of the polymorphic
in-line caches used in Java VM JITs
-XXX put slide about virtualizables somewhere
-
Concrete ingredients
====================
@@ -304,8 +302,6 @@
* Some rules about the color propagation
-XXX demo
-
The rainbow interpreter
=====================================
@@ -313,8 +309,6 @@
* We dump colored graphs as a bytecode and interpret them
-XXX demo
-
JIT details - backends
======================
More information about the Pypy-commit
mailing list