From arigo at codespeak.net Mon Jan 2 12:56:20 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Mon, 2 Jan 2006 12:56:20 +0100 (CET) Subject: [pypy-svn] r21623 - pypy/extradoc/talk/chalmers Message-ID: <20060102115620.86C9827B5C@code1.codespeak.net> Author: arigo Date: Mon Jan 2 12:56:07 2006 New Revision: 21623 Added: pypy/extradoc/talk/chalmers/techpaper.pdf (contents, props changed) pypy/extradoc/talk/chalmers/techpaper.sty (contents, props changed) - copied, changed from r21572, pypy/extradoc/talk/22c3/techpaper.sty pypy/extradoc/talk/chalmers/techpaper.tex (contents, props changed) pypy/extradoc/talk/chalmers/techpaper.txt - copied, changed from r21572, pypy/extradoc/talk/22c3/techpaper.txt pypy/extradoc/talk/chalmers/translation-greyscale-small.pdf - copied unchanged from r21572, pypy/extradoc/talk/22c3/translation-greyscale-small.pdf Modified: pypy/extradoc/talk/chalmers/ (props changed) Log: Copied the CCC paper (itself copied from architecture.txt) as the Chalmers Winter Meeting paper. Changed the style, the title and the authors (mentioned all non-trivial svn authors in a footnote, for want of a better idea). Added: pypy/extradoc/talk/chalmers/techpaper.pdf ============================================================================== Binary file. No diff available. Added: pypy/extradoc/talk/chalmers/techpaper.tex ============================================================================== --- (empty file) +++ pypy/extradoc/talk/chalmers/techpaper.tex Mon Jan 2 12:56:07 2006 @@ -0,0 +1,422 @@ +\documentclass[a4paper,11pt,english]{article} +\usepackage{babel} +\usepackage{ae} +\usepackage{aeguill} +\usepackage{shortvrb} +\usepackage[latin1]{inputenc} +\usepackage{tabularx} +\usepackage{longtable} +\setlength{\extrarowheight}{2pt} +\usepackage{amsmath} +\usepackage{graphicx} +\usepackage{color} +\usepackage{multirow} +\usepackage{ifthen} +\usepackage[colorlinks=false,linkcolor=black,urlcolor=black]{hyperref} +\usepackage[DIV12]{typearea} +%% generator Docutils: http://docutils.sourceforge.net/ +\newlength{\admonitionwidth} +\setlength{\admonitionwidth}{0.9\textwidth} +\newlength{\docinfowidth} +\setlength{\docinfowidth}{0.9\textwidth} +\newlength{\locallinewidth} +\newcommand{\optionlistlabel}[1]{\bf #1 \hfill} +\newenvironment{optionlist}[1] +{\begin{list}{} + {\setlength{\labelwidth}{#1} + \setlength{\rightmargin}{1cm} + \setlength{\leftmargin}{\rightmargin} + \addtolength{\leftmargin}{\labelwidth} + \addtolength{\leftmargin}{\labelsep} + \renewcommand{\makelabel}{\optionlistlabel}} +}{\end{list}} +\newlength{\lineblockindentation} +\setlength{\lineblockindentation}{2.5em} +\newenvironment{lineblock}[1] +{\begin{list}{} + {\setlength{\partopsep}{\parskip} + \addtolength{\partopsep}{\baselineskip} + \topsep0pt\itemsep0.15\baselineskip\parsep0pt + \leftmargin#1} + \raggedright} +{\end{list}} +% begin: floats for footnotes tweaking. +\setlength{\floatsep}{0.5em} +\setlength{\textfloatsep}{\fill} +\addtolength{\textfloatsep}{3em} +\renewcommand{\textfraction}{0.5} +\renewcommand{\topfraction}{0.5} +\renewcommand{\bottomfraction}{0.5} +\setcounter{totalnumber}{50} +\setcounter{topnumber}{50} +\setcounter{bottomnumber}{50} +% end floats for footnotes +% some commands, that could be overwritten in the style file. +\newcommand{\rubric}[1]{\subsection*{~\hfill {\it #1} \hfill ~}} +\newcommand{\titlereference}[1]{\textsl{#1}} +% end of "some commands" +\input{techpaper.sty} +\title{PyPy - Architecture Overview} +\hypersetup{ +pdftitle={PyPy - Architecture Overview} +} +\raggedbottom +\begin{document} +\maketitle + + +\setlength{\locallinewidth}{\linewidth} +% command to produce latex file: +% rst2latex.py --use-latex-citations --use-latex-footnotes --use-latex-toc --documentoptions=a4paper,11pt --hyperlink-color=0 techpaper.txt --use-latex-docinfo --stylesheet=techpaper.sty techpaper.tex +\begin{abstract} +PyPy\footnotemark[1] is an implementation of the Python\footnotemark[2] programming language written in +Python itself, flexible and easy to experiment with. Our long-term goals are +to target a large variety of platforms, small and large, by providing a +compiler toolsuite that can produce custom Python versions. Platform, memory +and threading models are to become aspects of the translation process - as +opposed to encoding low level details into the language implementation itself. +Eventually, dynamic optimization techniques - implemented as another +translation aspect - should become robust against language changes. +\footnotetext[1]{ +\href{http://codespeak.net/pypy}{http://codespeak.net/pypy} +} +\footnotetext[2]{ +\href{http://docs.python.org/ref}{http://docs.python.org/ref} +} +\end{abstract} + +%___________________________________________________________________________ + +\hypertarget{pypy-an-implementation-of-python-in-python}{} +\section{PyPy - an implementation of Python in Python} + +It has become a tradition in the development of computer languages to +implement each language in itself. This serves many purposes. By doing so, +you demonstrate the versatility of the language and its applicability for +large projects. Writing compilers and interpreters are among the most +complex endeavours in software development. + +An important aspect of implementing Python in Python is the high level of +abstraction and compactness of the language. This allows an implementation +that is, in some respects, easier to understand and play with than the one +written in C (referred to throughout the PyPy documentation and source as +``CPython''\footnotemark[3]). +\footnotetext[3]{ +\href{http://www.python.org}{http://www.python.org} +} + +Another central idea in PyPy is building the implementation in the form +of a number of independent modules with clearly defined and well tested API's. +This eases reuse and allows experimenting with multiple implementations +of specific features. + +Later in the project we will introduce optimizations, following the +ideas of Psyco\footnotemark[4], a Just in Time Specializer, that should make PyPy +run Python programs faster than CPython. Extensions that increase the +expressive power are also planned. For instance, we will include the +ideas of Stackless\footnotemark[5], which moves the execution frames off the stack into +heap space, allowing for massive parallellism. +\footnotetext[4]{ +\href{http://psyco.sourceforge.net}{http://psyco.sourceforge.net} +} +\footnotetext[5]{ +\href{http://stackless.com}{http://stackless.com} +} + + +%___________________________________________________________________________ + +\hypertarget{pypy-meta-goals}{} +\section{PyPy - Meta Goals} + +PyPy is not only about writing another Python interpreter. +Traditionally, interpreters are written in a target platform language +like C/Posix, Java or C{\#}. Each such interpreter provides a ``mapping'' +from application source code to the target environment. One of the +goals of the ``all-encompassing'' environments, like the .NET framework +and to some extent the Java virtual machine, is to provide standardized +and higher level functionalities to language implementors. This reduces +the burden of having to write and maintain many interpreters or +compilers. + +PyPy is experimenting with a different approach. We are not writing a +Python interpreter for a specific target platform. We have written a +Python interpreter in Python, with as few references as possible to +low-level details. (Because of the nature of Python, this is already +a complicated task, although not as complicated as writing it in - say +- C.) Then we use this as a ``language specification'' and manipulate +it to produce the more traditional interpreters that we want. In the +above sense, we are generating the concrete ``mappings'' of Python into +lower-level target platforms. + +So far (autumn 2005), we have already succeeded in turning this ``language +specification'' into reasonably efficient C-level code that performs +basically the same job as CPython. Memory management is inserted during +this \emph{translation} process. It can be configured to use reference +counting or not; thus we have already achieved two very different +mappings of application Python code over C/Posix. We have +also successfully translated our Python interpreter into LLVM\footnotemark[6] code, +and we are working on targeting higher-level environments like +Java and Squeak. + +In some senses, PyPy project's central component is not its +interpreter implementation, but its configurable translator. +We think it provides a good way to avoid writing \texttt{n * m * o} +interpreters for \texttt{n} dynamic languages and \texttt{m} platforms +with \texttt{o} crucial design decisions. PyPy aims at having any +one of these parameters changeable independently from each other: +\begin{itemize} +\item {} +we can modify or replace the language we interpret and just regenerate +a concrete interpreter for each target; + +\item {} +we can write new translator back-ends to target new platforms; + +\item {} +we can tweak the translation process to produce low-level code based +on different models and tradeoffs. + +\end{itemize} + +By contrast, a standardized target environment - say .NET - +enforces \texttt{m=1} as far as it is concerned. This helps making \texttt{o} a +bit smaller by providing a higher-level base to build upon. Still, +we believe that enforcing the use of one common environment +is not necessary. PyPy's goal is to give weight to this claim - at least +as far as language implementation is concerned - showing an approach +to the \texttt{n * m * o} problem that does not rely on standardization. + +This is the \emph{meta-goal}; a more concrete goal worth mentioning at this +point is that language specifications can be used to generate cool stuff +in addition to traditional interpreters - e.g. Just-In-Time compilers. +\footnotetext[6]{ +\href{http://llvm.cs.uiuc.edu/}{http://llvm.cs.uiuc.edu/} +} + + +%___________________________________________________________________________ + +\hypertarget{higher-level-picture}{} +\section{Higher level picture} + +As you would expect from a project implemented using ideas from the world +of Extreme Programming\footnotemark[7], the architecture of PyPy has evolved over time +and continues to evolve. Nevertheless, the high level architecture is now +clear. There are two independent basic subsystems: the Standard +Interpreter and the Translation Process. +\footnotetext[7]{ +\href{http://www.extremeprogramming.com/}{http://www.extremeprogramming.com/} +} + + +%___________________________________________________________________________ + +\hypertarget{the-standard-interpreter}{} +\subsection{The Standard Interpreter} + +The \emph{standard interpreter} is the subsystem implementing the Python language. +It is divided into two components: +\begin{itemize} +\item {} +the bytecode interpreter which is responsible for interpreting +code objects and implementing bytecodes, + +\item {} +the standard object space which implements creating, accessing and +modifying application level objects. + +\end{itemize} + +Note that the \emph{standard interpreter} can run fine on top of CPython if one +is willing to pay the performance penalty for double-interpretation. + +The \emph{bytecode interpreter} is the part that interprets the compact +bytecode format produced from user Python sources by a preprocessing +phase, the \emph{bytecode compiler}. The bytecode compiler itself is +implemented as a chain of flexible passes (tokenizer, lexer, parser, +abstract syntax tree builder, bytecode generator). The bytecode +interpreter then does its work by delegating all actual manipulation of +user objects to the \emph{object space}. The latter can be thought of as the +library of built-in types. It defines the implementation of the user +objects, like integers and lists, as well as the operations between +them, like addition or truth-value-testing. + +This division between bytecode interpreter and object space is very +important, as it gives a lot of flexibility. It is possible to use +different object spaces to get different behaviours of the Python +objects. Using a special object space is also an important technique +for our translation process. + + +%___________________________________________________________________________ + +\hypertarget{the-translation-process}{} +\subsection{The Translation Process} + +The \emph{translation process} aims at producing a different (low-level) +representation of our standard interpreter. The \emph{translation process} +is done in four steps: +\begin{itemize} +\item {} +producing a \emph{flow graph} representation of the standard interpreter. +A combination of the bytecode interpreter and a \emph{flow object space} +performs \emph{abstract interpretation} to record the flow of objects +and execution throughout a Python program into such a \emph{flow graph}; + +\item {} +the \emph{annotator} which performs type inference on the flow graph; + +\item {} +the \emph{typer} which, based on the type annotations, turns the flow graph +into one using only low-level operations that fit the model of the +target platform; + +\item {} +the \emph{code generator} which translates the resulting flow graph into +another language, currently C, LLVM, Javascript (experimental). + +\end{itemize} + +A more complete description of the phases of this process is out of the scope +of the present introduction. We will only give a short overview in the sequel. +\hypertarget{initialization-time}{}\hypertarget{translation-process-in-more-details}{} + +%___________________________________________________________________________ + +\hypertarget{rpython-the-flow-object-space-and-translation}{} +\section{RPython, the Flow Object Space and translation} + +One of PyPy's now achieved objectives is to enable translation of our +bytecode interpreter and standard object space into a lower-level language. +In order for our translation and type inference mechanisms to work +effectively, we need to restrict the dynamism of our interpreter-level +Python code at some point. However, in the start-up phase, we are +completely free to use all kinds of powerful Python constructs, including +metaclasses and execution of dynamically constructed strings. However, +when the initialization phase finishes, all code objects involved need to +adhere to a more static subset of Python: +Restricted Python, also known as RPython. + +The Flow Object Space then, with the help of our bytecode interpreter, +works through those initialized RPython code objects. The result of +this abstract interpretation is a flow graph: yet another +representation of a Python program, but one which is suitable for +applying translation and type inference techniques. The nodes of the +graph are basic blocks consisting of Object Space operations, flowing +of values, and an exitswitch to one, two or multiple links which connect +each basic block to other basic blocks. + +The flow graphs are fed as input into the Annotator. The Annotator, +given entry point types, infers the types of values that flow through +the program variables. This is the core of the definition of RPython: +RPython code is restricted in such a way that the +Annotator is able to infer consistent types. How much +dynamism we allow in RPython depends on, and is restricted by, the Flow +Object Space and the Annotator implementation. The more we can improve +this translation phase, the more dynamism we can allow. In some cases, +however, it is more feasible and practical to just get rid +of some of the dynamism we use in our interpreter level code. It is +mainly because of this trade-off situation that the definition of +RPython has shifted over time. Although the Annotator is +pretty stable now and able to process the whole of PyPy, the RPython +definition will probably continue to shift marginally as we improve it. + +The newest piece of this puzzle is the +\emph{Typer}, which inputs the high-level types inferred by the Annotator and +uses them to modify the flow graph in-place to replace its operations with +low-level ones, directly manipulating low-level values and data structures. + +The actual low-level code is emitted by ``visiting'' the type-annotated +flow graph after the typer introduced low-level operations. Currently we have +a C-producing backend, and an LLVM-producing backend. The former also +accepts non-annotated or partially-annotated graphs, which allow us to +test it on a larger class of programs than what the Annotator can (or +ever will) fully process. +\begin{figure*}[htbp]\begin{center} +\includegraphics{translation-greyscale-small.pdf} +overview of the translation process +\end{center}\end{figure*} +The complete translation process is described in more detail in the +documentation section on the PyPy homepage\footnotemark[8]. +\footnotetext[8]{ +\href{http://codespeak.net/pypy}{http://codespeak.net/pypy} +} + + +%___________________________________________________________________________ + +\hypertarget{status-of-the-implementation-nov-2005}{} +\section{Status of the implementation (Nov 2005)} + +With the pypy-0.8.0 release we have integrated our Abstract Syntax +Tree (AST) compiler with the rest of PyPy. The compiler gets +translated with the rest to a static self-contained version of the +standard interpreter. Like with 0.7.0 this version is very compliant\footnotemark[9] to CPython 2.4.1 but you cannot run many existing programs on it +yet because we are still missing a number of C-modules like socket or +support for process creation. + +The self-contained PyPy version (single-threaded and using the +Boehm-Demers-Weiser garbage collector\footnotemark[10]) now runs around 10-20 +times slower than CPython, i.e. around 10 times faster than 0.7.0. +This is the result of optimizations, adding short cuts for some common +paths in our interpreter and adding relatively straight forward +optimising transforms to our tool chain, like inlining paired with +simple escape analysis to remove unnecessary heap allocations. We +still have some way to go. However we expect that most of our speed +will come from the Just-In-Time compiler - work which we have barely +started yet. + +With the 0.8.0 release the ``Thunk Object Space'' can also be +translated. This is a module that proxies the Standard Object Space, +adding lazy evaluation features to Python. It is a small scale +show-case for how our whole tool-chain supports flexibility from the +interpreter written in Python to the resulting self-contained +executable. + +Our rather complete and Python 2.4-compliant interpreter consists +of about 30,000-50,000 lines of code (depending on the way you +count code borrowed and adapted from other sources), with +another 14,000 lines of unit tests. If we include the tools, +the parts related to code analysis and generation, and the +standard library modules ported from C, PyPy is now 138,000 +lines of code and 32,000 lines of tests. Refer to +the statistics web page\footnotemark[11] for more detailed information. +\footnotetext[9]{ +\href{http://www.hpl.hp.com/personal/Hans_Boehm/gc/}{http://www.hpl.hp.com/personal/Hans{\_}Boehm/gc/} +} +\footnotetext[10]{ +\href{http://codespeak.net/~hpk/pypy-testresult/}{http://codespeak.net/{\textasciitilde}hpk/pypy-testresult/} +} +\footnotetext[11]{ +\href{http://codespeak.net/~hpk/pypy-stat/}{http://codespeak.net/{\textasciitilde}hpk/pypy-stat/} +} + + +%___________________________________________________________________________ + +\hypertarget{future-work-and-foreseen-possibilities}{} +\section{Future work and foreseen possibilities} + +In 2006, the PyPy project aims to translate the standard Python +Interpreter to a JIT-compiler and also to support massive parallelism +(micro-threads) within the language. These are not trivial tasks +especially if we want to retain and improve the modularity and +flexibility aspects of our implementation - like giving an independent +choice of memory or threading models for translation. Moreover it is +likely that our Javascript and other higher level backends (in +contrast to our current low-level ones) will continue to evolve. + +Apart from optimization-related translation choices PyPy is to enable +new possibilities regarding persistence, security and distribution +issues. We intend to experiment with ortoghonal persistence for +Python objects, i.e. one that doesn't require application objects to +behave in a particular manner. Security-wise we will look at +sandboxing or capabilities based schemes. For distribution we already +experimented with allowing transparent migration of objects between +processes with the help of the existing (and translateable) Thunk +Object Space. In general, all experiments are much easier to conduct +in PyPy and should provide a resulting standalone executable in +a shorter time than traditional approaches. + +\end{document} From arigo at codespeak.net Mon Jan 2 13:04:29 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Mon, 2 Jan 2006 13:04:29 +0100 (CET) Subject: [pypy-svn] r21624 - in pypy/dist/pypy/lib/pyontology: . test Message-ID: <20060102120429.5ABA727B5C@code1.codespeak.net> Author: arigo Date: Mon Jan 2 13:04:25 2006 New Revision: 21624 Modified: pypy/dist/pypy/lib/pyontology/ (props changed) pypy/dist/pypy/lib/pyontology/__init__.py (props changed) pypy/dist/pypy/lib/pyontology/pyontology.py (props changed) pypy/dist/pypy/lib/pyontology/test/ (props changed) pypy/dist/pypy/lib/pyontology/test/test_ontology.py (props changed) Log: fixeol From pedronis at codespeak.net Mon Jan 2 14:28:37 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Mon, 2 Jan 2006 14:28:37 +0100 (CET) Subject: [pypy-svn] r21625 - pypy/extradoc/planning Message-ID: <20060102132837.0B80127B66@code1.codespeak.net> Author: pedronis Date: Mon Jan 2 14:28:36 2006 New Revision: 21625 Modified: pypy/extradoc/planning/phase2_projectactivities.txt Log: OOPSLA real deadline Modified: pypy/extradoc/planning/phase2_projectactivities.txt ============================================================================== --- pypy/extradoc/planning/phase2_projectactivities.txt (original) +++ pypy/extradoc/planning/phase2_projectactivities.txt Mon Jan 2 14:28:36 2006 @@ -136,7 +136,8 @@ Submission date: ? OPEN: OOPSLA 2006 (Samuele, Armin) - Submission date: probably 20th March 2006 + Submission date: 18th March 2006 + http://www.oopsla.org/2006/researchPapers.html OPEN: Calibre conferences From pedronis at codespeak.net Mon Jan 2 14:35:35 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Mon, 2 Jan 2006 14:35:35 +0100 (CET) Subject: [pypy-svn] r21626 - pypy/extradoc/pypy.org Message-ID: <20060102133535.F038A27B5E@code1.codespeak.net> Author: pedronis Date: Mon Jan 2 14:35:34 2006 New Revision: 21626 Modified: pypy/extradoc/pypy.org/consortium.txt pypy/extradoc/pypy.org/news.txt Log: convert to latin-1 Modified: pypy/extradoc/pypy.org/consortium.txt ============================================================================== --- pypy/extradoc/pypy.org/consortium.txt (original) +++ pypy/extradoc/pypy.org/consortium.txt Mon Jan 2 14:35:34 2006 @@ -1,4 +1,4 @@ -???Consortium Partners +Consortium Partners ------------------------------ DFKI http://www.dfki.de @@ -8,17 +8,17 @@ Anders Lehmann [anders.lehmann at dfki de] AB Strakt http://www.strakt.com - Jacob Hall??n [jacob at strakt com] (project manager) + Jacob Hall?n [jacob at strakt com] (project manager) Samuele Pedroni [pedronis at strakt com] (technical board) - Anders Chrigstr??m [ac at strakt com] + Anders Chrigstr?m [ac at strakt com] Change Maker http://www.changemaker.nu - Beatrice D??ring [bea at changemaker nu] (assistant project manager, press contact) + Beatrice D?ring [bea at changemaker nu] (assistant project manager, press contact) merlinux GmbH http://merlinux.de Holger Krekel [krekel at merlinux de] (technical director/board) -Heinrich Heine Universit??t D??sseldorf http://www.uni-duesseldorf.de/ +Heinrich Heine Universit?t D?sseldorf http://www.uni-duesseldorf.de/ Armin Rigo [arigo at tunes org] (technical board) Tismerysoft http://www.stackless.com Modified: pypy/extradoc/pypy.org/news.txt ============================================================================== --- pypy/extradoc/pypy.org/news.txt (original) +++ pypy/extradoc/pypy.org/news.txt Mon Jan 2 14:35:34 2006 @@ -1,4 +1,4 @@ -???PyPy at PyCon +PyPy at PyCon ------------------------------------------ PyPy has three talks accepted at PyCon, which will be held in Dallas, USA @@ -24,12 +24,12 @@ PyPy at the 2 International Calibre conference, Limerick, September 2005 ------------------------------------------------------------------------------ -Beatrice D??ring from the PyPy team participated in the second International +Beatrice D?ring from the PyPy team participated in the second International Calibre conference at the University of Limerick (9th of September). Calibre is a coordination action project, focused on OSS - business models, agile and distributed development. Beatrice presented a talk about the sprint methodology in use in the PyPy project. -Lot??s of useful contacts where made and there are rumours of a sprint +Lots of useful contacts where made and there are rumours of a sprint in Ireland next year, and maybe also in Japan. For more information about Calibre: www.calibre.ie @@ -55,13 +55,13 @@ afterwards. There where also several PyPy talks at the conference, a joint talk by Holger Krekel, Armin Rigo, Carl Friedrich Bolz about translation aspects of PyPy, Holger Krekel about py.test and -Beatrice D??ring about sprint driven development. This sprint had 20 +Beatrice D?ring about sprint driven development. This sprint had 20 participants which is a PyPy record. PyPy at ACCU April 2005 ------------------------------------------------------ -Armin Rigo and Jacob Hall??n held 2 talks at the ACCU conference,about PyPy and sprint driven +Armin Rigo and Jacob Hall?n held 2 talks at the ACCU conference,about PyPy and sprint driven development. PyPy at PyCon in Washington March 2005 @@ -74,7 +74,7 @@ PyPy at the Calibre conference in Paris March 2005 ------------------------------------------------------ -Beatrice D??ring from the PyPy team participated in the Calibre workshop +Beatrice D?ring from the PyPy team participated in the Calibre workshop "Libre software - which business model?". Sprint in Leysin, Switzerland January 2005 From pedronis at codespeak.net Mon Jan 2 14:41:05 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Mon, 2 Jan 2006 14:41:05 +0100 (CET) Subject: [pypy-svn] r21627 - pypy/extradoc/pypy.org Message-ID: <20060102134105.4D27F27B66@code1.codespeak.net> Author: pedronis Date: Mon Jan 2 14:41:04 2006 New Revision: 21627 Modified: pypy/extradoc/pypy.org/consortium.txt Log: adding Michael Modified: pypy/extradoc/pypy.org/consortium.txt ============================================================================== --- pypy/extradoc/pypy.org/consortium.txt (original) +++ pypy/extradoc/pypy.org/consortium.txt Mon Jan 2 14:41:04 2006 @@ -20,6 +20,7 @@ Heinrich Heine Universit?t D?sseldorf http://www.uni-duesseldorf.de/ Armin Rigo [arigo at tunes org] (technical board) + Michael Hudson [mwh at python net] Tismerysoft http://www.stackless.com Christian Tismer [tismer at stackless com] (technical board) From bea at codespeak.net Mon Jan 2 15:43:25 2006 From: bea at codespeak.net (bea at codespeak.net) Date: Mon, 2 Jan 2006 15:43:25 +0100 (CET) Subject: [pypy-svn] r21629 - pypy/extradoc/pypy.org Message-ID: <20060102144325.F1CF527DB4@code1.codespeak.net> Author: bea Date: Mon Jan 2 15:43:24 2006 New Revision: 21629 Modified: pypy/extradoc/pypy.org/news.txt Log: updated news on pypy.org - can someone please check rest format? Modified: pypy/extradoc/pypy.org/news.txt ============================================================================== --- pypy/extradoc/pypy.org/news.txt (original) +++ pypy/extradoc/pypy.org/news.txt Mon Jan 2 15:43:24 2006 @@ -1,6 +1,26 @@ -PyPy at PyCon ------------------------------------------- +PyPy at CCC conference 27-30th of December 2005 +======================================= +PyPy presented two different talks at the 22C3 conference in Berlin, +"PyPy - the new Python implementation on the block" and "OSS, EUfunding +and agile business". For more information about this conference: +http://events.ccc.de/congress/2005/overview.en.html. +The talks can be found on http://codespeak.net/pypy. + +PyPy sprint in Gothenburg and seminar in Brussels +===================================== +PyPy sprinted in Gothenburg during the first week of December - work +with the JIT compiler was a primary target. This work will continue during +the spring of 2006. + +In parallell with the PyPy sprint, some PyPy teammembers visited +the Commission in Brussels and participated in an internal EC seminar +regarding OSS, distributed and agile development on the 8th of December. +This was done in cooperation with the Calibre project, by invitation from the +PyPy Project Officer Mr Dirk Van Royy. +The talk can be found on http://codespeak.net/pypy. +PyPy at PyCon +=========== PyPy has three talks accepted at PyCon, which will be held in Dallas, USA 24-26th of February. For more information about the conference and the pypy talks: http://us.pycon.org/TX2006/HomePage. From pedronis at codespeak.net Mon Jan 2 15:49:16 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Mon, 2 Jan 2006 15:49:16 +0100 (CET) Subject: [pypy-svn] r21630 - pypy/extradoc/pypy.org Message-ID: <20060102144916.A714127B69@code1.codespeak.net> Author: pedronis Date: Mon Jan 2 15:49:14 2006 New Revision: 21630 Modified: pypy/extradoc/pypy.org/news.txt Log: fix ReST ;) Modified: pypy/extradoc/pypy.org/news.txt ============================================================================== --- pypy/extradoc/pypy.org/news.txt (original) +++ pypy/extradoc/pypy.org/news.txt Mon Jan 2 15:49:14 2006 @@ -1,5 +1,5 @@ PyPy at CCC conference 27-30th of December 2005 -======================================= +------------------------------------------------- PyPy presented two different talks at the 22C3 conference in Berlin, "PyPy - the new Python implementation on the block" and "OSS, EUfunding and agile business". For more information about this conference: @@ -7,7 +7,7 @@ The talks can be found on http://codespeak.net/pypy. PyPy sprint in Gothenburg and seminar in Brussels -===================================== +--------------------------------------------------- PyPy sprinted in Gothenburg during the first week of December - work with the JIT compiler was a primary target. This work will continue during the spring of 2006. @@ -20,7 +20,7 @@ The talk can be found on http://codespeak.net/pypy. PyPy at PyCon -=========== +-------------- PyPy has three talks accepted at PyCon, which will be held in Dallas, USA 24-26th of February. For more information about the conference and the pypy talks: http://us.pycon.org/TX2006/HomePage. From pedronis at codespeak.net Mon Jan 2 16:10:09 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Mon, 2 Jan 2006 16:10:09 +0100 (CET) Subject: [pypy-svn] r21631 - pypy/extradoc/sprintinfo/mallorca Message-ID: <20060102151009.B845127B5E@code1.codespeak.net> Author: pedronis Date: Mon Jan 2 16:10:07 2006 New Revision: 21631 Added: pypy/extradoc/sprintinfo/mallorca/sprint-announcement.html (contents, props changed) Log: html version of the annoucement Added: pypy/extradoc/sprintinfo/mallorca/sprint-announcement.html ============================================================================== --- (empty file) +++ pypy/extradoc/sprintinfo/mallorca/sprint-announcement.html Mon Jan 2 16:10:07 2006 @@ -0,0 +1,131 @@ + + + + + + +Palma de Mallorca PyPy Sprint: 23rd - 29th January 2006 + + +
+

Palma de Mallorca PyPy Sprint: 23rd - 29th January 2006

+

The next PyPy sprint is scheduled to take place January 2006 in +Palma De Mallorca, Balearic Isles, Spain. We'll give newcomer-friendly +introductions and the focus will mainly be on current JIT work, garbage +collection, alternative threading models, logic programming and on +improving the interface with external functions. To learn more about the +new Python-in-Python implementation look here:

+
+http://codespeak.net/pypy
+
+

Goals and topics of the sprint

+

In Gothenburg we have made some first forays into the interesting topics +of Just-in-Time compilation. In Mallorca we will continue that +and have the following ideas:

+
+
    +
  • Further work/experimentation toward Just-In-Time Compiler generation, +which was initiated with the Abstract Interpreter started in Gothenburg.
    • +
  • Integrating our garbage collection toolkit with the backends and the +code generation.
    • +
  • Heading into the direction of adding logic programming to PyPy.
    • +
  • Optimization work: our threading implementation is still incredibly +slow, we need to work on that. Furthermore there are still quite +some slow places in the interpreter that could be improved.
    • +
  • getting the socket module to a more complete state (it is +already improved but still far from complete)
    • +
  • generally improving the way we interface with external functions.
    • +
  • whatever participants want to do with PyPy (please send +suggestions to the mailing list before to allow us to plan +and give feedback)
    • +
+
+
+
+

Location & Accomodation

+

The sprint will be held at the Palma University (UIB - Universitat de +les Illes Balears), in their GNU/Linux lab +(http://mnm.uib.es/phpwiki/AulaLinux). We are hosted by the Computer +Science department and Ricardo Galli is our contact person there, +helping with arranging facilities.

+

The University is located 7 km away from the central Palma. Busses to +the University departs from "Plaza de Espa?a" (which is a very central +location in Palma). Take bus 19 to the UIB campus. A ticket for one +urban trip costs 1 euro. You can also buy a card that is valid for 10 +trips and costs 7.51 euros. Information about bus timetables and routes +can be found on:

+
+http://www.a-palma.es
+

A map over the UIB campus are can be found on:

+
+http://www.uib.es/imagenes/planoCampus.html
+

The actual address is: 3r pis de l'Anselm Turmeda which can be found on +the UIB Campus map.

+

At "Plaza de Espa?a" there is a hostel (Hostal Residencia Terminus) +which has been recommended to us. It's cheap (ca 50 euros/double room +with bathroom). Some more links to accomodations (flats, student homes +and hotels):

+

http://www.lodging-in-spain.com/hotel/town/Islas_Baleares,Mallorca,Palma_de_Mallorca,1/

+

http://www.uib.es/fuguib/residencia/english/index.html

+

http://www.homelidays.com/EN-Holidays-Rental/110_Search/SearchList.asp?DESTINATION=Palma%20de%20Mallorca&ADR_PAYS=ES&ADR_ +LOCALISATION=ES%20ISLASBALEARES%20MALLORCA

+

If you want to find a given street, you can search here:

+
+http://www.callejeando.com/Pueblos/pueblo7_1.htm
+

To get to Palma De Mallorca almost all low fare airlines and travel +agencies have cheap tickets to get there. Information about Mallorca and +Palma (maps, tourist information, local transports, recommended air +lines, ferries and much more) can be found on:

+
+http://www.palmademallorca.es/portalPalma/home.jsp
+

Comments on the weather: In January it is cold and wet on Mallorca

+

Average temperature: 8,4 degrees Celsius +Lowest temperature: 2 degrees Celsius +Highest temperature: 14,5 degrees Celsius +Average humidity rate: 77,6 %

+

So more time for coding and less time for sunbathing and beaches ;-)

+
+
+

Exact times

+

The public PyPy sprint is held Monday 23rd - Sunday 29th January 2006. +Hours will be from 10:00 until people have had enough. It's a good idea +to arrive a day before the sprint starts and leave a day later. In the +middle of the sprint there usually is a break day and it's usually ok to +take half-days off if you feel like it.

+

For this particular break day, Thursday, we are invited to the studio of +Gin?s Qui?onero, a local artist and painter. Gin?s have also been the +person helping us getting connections to UIB and providing much +appreciated help regarding accommodation and other logistical +information.

+

For those of you interested - here is his website where there also are +paintings showing his studio:

+
+http://www.hermetex4.com/damnans/
+

For those interested in playing collectable card games, this will also +be an opportunity to get aquainted with V:TES which will be demoed by +Gin?s and Beatrice and Sten D?ring. For more information on this +cardgame - see: http://www.white-wolf.com/vtes/index.php. (The Mallorca +sprint was organized through contacts within the V:TES community).

+
+
+

Network, Food, currency

+

Currency is Euro.

+

Food is available in the UIB Campus area as well as cheap restaurants in +Palma.

+

You normally need a wireless network card to access the network, but we +can provide a wireless/ethernet bridge.

+

230V AC plugs are used in Mallorca.

+
+
+

Registration etc.pp.

+

Please subscribe to the PyPy sprint mailing list, introduce yourself +and post a note that you want to come. Feel free to ask any questions +there! There also is a separate Mallorca people page tracking who is +already thought to come. If you have commit rights on codespeak then +you can modify yourself a checkout of

+
+http://codespeak.net/svn/pypy/extradoc/sprintinfo/mallorca-2006/people.txt
+
+
+ + From pedronis at codespeak.net Mon Jan 2 16:16:37 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Mon, 2 Jan 2006 16:16:37 +0100 (CET) Subject: [pypy-svn] r21632 - pypy/dist/pypy/doc Message-ID: <20060102151637.A2D1127B5E@code1.codespeak.net> Author: pedronis Date: Mon Jan 2 16:16:35 2006 New Revision: 21632 Modified: pypy/dist/pypy/doc/news.txt Log: Mallorca sprint Modified: pypy/dist/pypy/doc/news.txt ============================================================================== --- pypy/dist/pypy/doc/news.txt (original) +++ pypy/dist/pypy/doc/news.txt Mon Jan 2 16:16:35 2006 @@ -9,6 +9,21 @@ .. _Python: http://www.python.org/doc/current/ref/ref.html .. _`more...`: http://codespeak.net/pypy/dist/pypy/doc/architecture.html#mission-statement +Next PyPy Sprint in Palma De Mallorca 23rd - 29th January 2006 +=============================================================== + +The next sprint is organized for end of January in Palma de Mallorca. +Topics will include progressing with the JIT work started in G?tteborg +and Paris, GC and optimisation work, logic programming, and possibly +improving our way to write glue code for C libraries. As usual we will +give newcomer-friendly introductions, and other topics are possible. + +Read more in `the announcement`_, see who is planning to attend +on the `people page`_. + +.. _`the announcement`: http://codespeak.net/pypy/extradoc/sprintinfo/mallorca/sprint-announcement.html +.. _`People page`: http://codespeak.net/svn/pypy/extradoc/sprintinfo/mallorca/people.txt + PyPy Sprint in G?teborg 7th - 11th December 2005 ================================================= From pedronis at codespeak.net Mon Jan 2 16:32:02 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Mon, 2 Jan 2006 16:32:02 +0100 (CET) Subject: [pypy-svn] r21633 - pypy/dist/pypy/doc Message-ID: <20060102153202.0668227B5E@code1.codespeak.net> Author: pedronis Date: Mon Jan 2 16:32:00 2006 New Revision: 21633 Modified: pypy/dist/pypy/doc/news.txt Log: oops Modified: pypy/dist/pypy/doc/news.txt ============================================================================== --- pypy/dist/pypy/doc/news.txt (original) +++ pypy/dist/pypy/doc/news.txt Mon Jan 2 16:32:00 2006 @@ -13,7 +13,7 @@ =============================================================== The next sprint is organized for end of January in Palma de Mallorca. -Topics will include progressing with the JIT work started in G?tteborg +Topics will include progressing with the JIT work started in G?teborg and Paris, GC and optimisation work, logic programming, and possibly improving our way to write glue code for C libraries. As usual we will give newcomer-friendly introductions, and other topics are possible. From bea at codespeak.net Mon Jan 2 16:40:13 2006 From: bea at codespeak.net (bea at codespeak.net) Date: Mon, 2 Jan 2006 16:40:13 +0100 (CET) Subject: [pypy-svn] r21635 - pypy/extradoc/sprintinfo/mallorca Message-ID: <20060102154013.1AD0E27B69@code1.codespeak.net> Author: bea Date: Mon Jan 2 16:40:11 2006 New Revision: 21635 Modified: pypy/extradoc/sprintinfo/mallorca/people.txt Log: travel to mallorca booked (fly from gothenburg to London stanstead and from there to mallorca) - will bring Sten and Simone Modified: pypy/extradoc/sprintinfo/mallorca/people.txt ============================================================================== --- pypy/extradoc/sprintinfo/mallorca/people.txt (original) +++ pypy/extradoc/sprintinfo/mallorca/people.txt Mon Jan 2 16:40:11 2006 @@ -9,7 +9,7 @@ ==================== ============== ===================== Name Arrive/Depart Accomodation ==================== ============== ===================== -Beatrice Duering ? ? +Beatrice Duering 22/1 -29/1 flat? Armin Rigo ? ? Samuele Pedroni ? ? Eric van Riet Paap ? ? From arigo at codespeak.net Mon Jan 2 21:54:57 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Mon, 2 Jan 2006 21:54:57 +0100 (CET) Subject: [pypy-svn] r21653 - in pypy/dist/pypy/jit: . test Message-ID: <20060102205457.892A427B70@code1.codespeak.net> Author: arigo Date: Mon Jan 2 21:54:52 2006 New Revision: 21653 Added: pypy/dist/pypy/jit/llcontainer.py (contents, props changed) pypy/dist/pypy/jit/llvalue.py (contents, props changed) Modified: pypy/dist/pypy/jit/llabstractinterp.py pypy/dist/pypy/jit/test/test_llabstractinterp.py Log: Refactoring of the state stuff in LLAbstractInterp: * split the large llabstractinterp.py in several files. * separate the "live" LLAbstractValues, as used while flowing a block, from their "frozen" version, now LLFrozenValues, used in the saved states between the blocks. Potentially a bit more overhead, but now each version has only the attributes that are relevant to its role. * LLAbstractValue is no longer a base class: all "live" abstract values are instances of LLAbstractValue directly. This gives a saner way to switch a value from virtual to run-time, for example. It should also later allow values that are both run-time and have a description of their content like virtual values. * LLBlockStates and LLAbstractContainers are still the same class for both the frozen and non-frozen version. Maybe at some point we could choose a more compact frozen format, e.g. as a pair (, ). * one test not reimplemented yet. Unclear if the feature it tests ('GraphState.a_return') will really be useful. Modified: pypy/dist/pypy/jit/llabstractinterp.py ============================================================================== --- pypy/dist/pypy/jit/llabstractinterp.py (original) +++ pypy/dist/pypy/jit/llabstractinterp.py Mon Jan 2 21:54:52 2006 @@ -4,346 +4,20 @@ from pypy.objspace.flow.model import checkgraph, c_last_exception from pypy.rpython.lltypesystem import lltype from pypy.translator.simplify import eliminate_empty_blocks, join_blocks - - -def const(value, T=None): - c = Constant(value) - c.concretetype = T or lltype.typeOf(value) - return c - -def newvar(T): - v = Variable() - v.concretetype = T - return v - - -class LLAbstractValue(object): - pass - -class LLConcreteValue(LLAbstractValue): - - def __init__(self, value): - self.value = value - - def __repr__(self): - return '' % (self.value,) - -# def __eq__(self, other): -# return self.__class__ is other.__class__ and self.value == other.value -# -# def __ne__(self, other): -# return not (self == other) -# -# def __hash__(self): -# return hash(self.value) - - def getconcretetype(self): - return lltype.typeOf(self.value) - - def forcevarorconst(self, builder): - return const(self.value) - - def getruntimevars(self, memo): - return [] - - def maybe_get_constant(self): - return const(self.value) - - def with_fresh_variables(self, memo): - return self - - def match(self, other, memo): - return isinstance(other, LLConcreteValue) and self.value == other.value - - -class LLRuntimeValue(LLAbstractValue): - - origin = None # list of LLRuntimeValues attached to a saved state: - # the sources that did or could allow 'self' to be - # computed as a constant - - fixed = False # only meaningful on LLRuntimeValues attached to a - # saved state. Describes what this LLRuntimeValue will - # become in the next block. Set to True by hint() - # to force a Constant to stay a Constant. - - def __init__(self, orig_v): - if isinstance(orig_v, Variable): - self.copy_v = Variable(orig_v) - self.copy_v.concretetype = orig_v.concretetype - elif isinstance(orig_v, Constant): - # we can share the Constant() - self.copy_v = orig_v - self.origin = [] - elif isinstance(orig_v, lltype.LowLevelType): - # hackish interface :-( we accept a type too - self.copy_v = newvar(orig_v) - else: - raise TypeError(repr(orig_v)) - - def __repr__(self): - return '' % (self.copy_v,) - - def getconcretetype(self): - return self.copy_v.concretetype - - def forcevarorconst(self, builder): - return self.copy_v - - def getruntimevars(self, memo): - if memo.key is None: - return [self.copy_v] - else: - c = memo.key.next() - if self.fixed: - assert isinstance(c, Constant), ( - "unexpected Variable %r reaching a fixed input arg" % (c,)) - return [c] - else: - return [None] - - def maybe_get_constant(self): - if isinstance(self.copy_v, Constant): - return self.copy_v - else: - return None - - def with_fresh_variables(self, memo): - # don't use memo.seen here: shared variables must become distinct - c = memo.key and memo.key.next() - if isinstance(c, Constant): # allowed to propagate as a Constant? - result = LLRuntimeValue(c) - else: - result = LLRuntimeValue(self.getconcretetype()) - result.origin = [self] - return result - - def match(self, other, memo): - memo.dependencies.append((self, other)) - return isinstance(other, LLRuntimeValue) - -ll_no_return_value = LLRuntimeValue(const(None, lltype.Void)) - - -class LLVirtualContainer(LLAbstractValue): - - parent = None - parentindex = None - - def __init__(self, T, a_length=None): - assert (a_length is not None) == T._is_varsize() - self.T = T - self.a_length = a_length - self.names = self.getnames() - self.fields = {} - - def getconcretetype(self): - return lltype.Ptr(self.T) - - def maybe_get_constant(self): - return None - - def setparent(self, parent, parentindex): - self.parent = parent - self.parentindex = parentindex - - def topmostparent(self): - obj = self - while obj.parent is not None: - obj = obj.parent - return obj - - def getfield(self, name): - try: - return self.fields[name] - except KeyError: - T = self.fieldtype(name) - if isinstance(T, lltype.ContainerType): - # reading a substructure - if T._is_varsize(): - a_length = self.a_length - else: - a_length = None - a_sub = virtualcontainer(T, a_length) - a_sub.setparent(self, name) - self.fields[name] = a_sub - return a_sub - else: - # no value ever set, return a default - return LLRuntimeValue(const(T._defl())) - - def setfield(self, name, a_value): - self.fields[name] = a_value - - def with_fresh_variables(self, memo): - if self in memo.seen: - return memo.seen[self] # already seen - else: - result = virtualcontainer(self.T, self.a_length) - memo.seen[self] = result - if self.parent is not None: - # build the parent first -- note that - # parent.with_fresh_variables() will pick up 'result' again, - # because it is already in the memo - result.setparent(self.parent.with_fresh_variables(memo), - self.parentindex) - - # cannot keep lazy fields around: the copy is expected to have - # only variables, not constants - for name in self.names: - a = self.getfield(name).with_fresh_variables(memo) - result.fields[name] = a - return result - - def forcevarorconst(self, builder): - v_result = newvar(lltype.Ptr(self.T)) - if self.parent is not None: - v_parent = self.parent.forcevarorconst(builder) - op = SpaceOperation('getsubstruct', [v_parent, - const(self.parentindex, - lltype.Void)], - v_result) - print 'force:', op - builder.residual_operations.append(op) - else: - if self.T._is_varsize(): - op = SpaceOperation('malloc_varsize', [ - const(self.T, lltype.Void), - self.a_length.forcevarorconst(builder)], - v_result) - else: - op = SpaceOperation('malloc', [const(self.T, lltype.Void)], - v_result) - print 'force:', op - builder.residual_operations.append(op) - self.buildcontent(builder, v_result) - self.__class__ = LLRuntimeValue - self.__dict__ = {'copy_v': v_result} - return v_result - - def buildcontent(self, builder, v_target): - # initialize all fields - for name in self.names: - if name in self.fields: - a_value = self.fields[name] - T = self.fieldtype(name) - if isinstance(T, lltype.ContainerType): - # initialize the substructure/subarray - v_subptr = newvar(lltype.Ptr(T)) - op = SpaceOperation('getsubstruct', - [v_target, const(name, lltype.Void)], - v_subptr) - print 'force:', op - builder.residual_operations.append(op) - assert isinstance(a_value, LLVirtualStruct) - a_value.buildcontent(builder, v_subptr) - else: - v_value = a_value.forcevarorconst(builder) - op = self.setop(v_target, name, v_value) - print 'force:', op - builder.residual_operations.append(op) - - def getruntimevars(self, memo): - result = [] - if self not in memo.seen: - memo.seen[self] = True - if self.parent is not None: - result.extend(self.parent.getruntimevars(memo)) - for name in self.names: - result.extend(self.getfield(name).getruntimevars(memo)) - return result - - def match(self, other, memo): - if self.__class__ is not other.__class__: - return False - - if self in memo.self_alias: - return other is memo.self_alias[self] - if other in memo.other_alias: - return self is memo.other_alias[other] - memo.self_alias[self] = other - memo.other_alias[other] = self - - assert self.T == other.T - if self.names != other.names: - return False - if self.a_length is not None: - if not self.a_length.match(other.a_length, memo): - return False - for name in self.names: - a1 = self.getfield(name) - a2 = other.getfield(name) - if not a1.match(a2, memo): - return False - else: - return True - - -class LLVirtualStruct(LLVirtualContainer): - """Stands for a pointer to a malloc'ed structure; the structure is not - malloc'ed so far, but we record which fields have which value. - """ - def __repr__(self): - items = self.fields.items() - items.sort() - flds = ['%s=%r' % item for item in items] - return '' % (self.T._name, ', '.join(flds)) - - def getnames(self): - return self.T._names - - def fieldtype(self, name): - return getattr(self.T, name) - - def setop(self, v_target, name, v_value): - return SpaceOperation('setfield', [v_target, - const(name, lltype.Void), - v_value], - newvar(lltype.Void)) - -class LLVirtualArray(LLVirtualContainer): - """Stands for a pointer to a malloc'ed array; the array is not - malloc'ed so far, but we record which fields have which value -- here - a field is an item, indexed by an integer instead of a string field name. - """ - def __repr__(self): - items = self.fields.items() - items.sort() - flds = ['%s=%r' % item for item in items] - return '' % (', '.join(flds),) - - def getnames(self): - c = self.a_length.maybe_get_constant() - assert c is not None - return range(c.value) - - def fieldtype(self, index): - return self.T.OF - - def setop(self, v_target, name, v_value): - return SpaceOperation('setarrayitem', [v_target, - const(name, lltype.Signed), - v_value], - newvar(lltype.Void)) - -def virtualcontainer(T, a_length=None): - if isinstance(T, lltype.Struct): - cls = LLVirtualStruct - elif isinstance(T, lltype.Array): - cls = LLVirtualArray - else: - raise TypeError("unsupported container type %r" % (T,)) - return cls(T, a_length) +from pypy.jit.llvalue import LLAbstractValue, const, newvar, dupvar +from pypy.jit.llvalue import FlattenMemo, MatchMemo, FreezeMemo, UnfreezeMemo +from pypy.jit.llcontainer import LLAbstractContainer, virtualcontainervalue # ____________________________________________________________ -class LLState(LLAbstractValue): +class LLState(LLAbstractContainer): """Entry state of a block, as a combination of LLAbstractValues for its input arguments. Abstract base class.""" - generalized_by = None - def __init__(self, a_back, args_a, origblock): - self.a_back = a_back + frozen = False # for debugging + + def __init__(self, back, args_a, origblock): + self.back = back self.args_a = args_a self.origblock = origblock self.copyblocks = {} @@ -352,47 +26,53 @@ def key(self): # two LLStates should return different keys if they cannot match(). result = self.localkey() - if self.a_back is not None: - result += self.a_back.key() - return result - - def getruntimevars(self, memo): - if self.a_back is None: - result = [] - else: - result = self.a_back.getruntimevars(memo) - for a in self.args_a: - result.extend(a.getruntimevars(memo)) + if self.back is not None: + result += self.back.key() return result - def maybe_get_constant(self): - return None + def enum_fixed_constants(self, incoming_link_args=None): + assert self.frozen, "enum_fixed_constants(): only for frozen states" + selfvalues = self.flatten() + if incoming_link_args is None: + for a in selfvalues: + yield None # no incoming args provided, so no fixed constants + else: + assert len(incoming_link_args) == len(selfvalues) + for linkarg, fr in zip(incoming_link_args, selfvalues): + if fr.fixed: + assert isinstance(linkarg, Constant), ( + "unexpected Variable %r reaching the fixed input arg %r" + % (linkarg, fr)) + yield linkarg + else: + yield None - def with_fresh_variables(self, memo): - if self.a_back is not None: - new_a_back = self.a_back.with_fresh_variables(memo) - else: - new_a_back = None - new_args_a = [] - for v, a in zip(self.getlivevars(), self.args_a): - a = a.with_fresh_variables(memo) - # try to preserve the name - if isinstance(a, LLRuntimeValue) and isinstance(a.copy_v, Variable): - a.copy_v.rename(v) - new_args_a.append(a) - return self.__class__(new_a_back, new_args_a, *self.localkey()) + def getruntimevars(self): + assert not self.frozen, "getruntimevars(): not for frozen states" + return [a.runtimevar for a in self.flatten()] + + def flatten(self, memo=None): + if memo is None: + memo = FlattenMemo() + if self.back is not None: + self.back.flatten(memo) + for a in self.args_a: + a.flatten(memo) + return memo.result def match(self, other, memo): + assert self.frozen, "match(): 1st state must be frozen" + assert not other.frozen, "match(): 2nd state must not be frozen" assert self.__class__ is other.__class__ if self.localkey() != other.localkey(): return False - if self.a_back is None: - if other.a_back is not None: + if self.back is None: + if other.back is not None: return False else: - if other.a_back is None: + if other.back is None: return False - if not self.a_back.match(other.a_back, memo): + if not self.back.match(other.back, memo): return False for a1, a2 in zip(self.args_a, other.args_a): if not a1.match(a2, memo): @@ -400,6 +80,32 @@ else: return True + def freeze(self, memo): + assert not self.frozen, "freeze(): state already frozen" + if self.back is not None: + new_back = self.back.freeze(memo) + else: + new_back = None + new_args_a = [a.freeze(memo) for a in self.args_a] + result = self.__class__(new_back, new_args_a, *self.localkey()) + result.frozen = True # for debugging + return result + + def unfreeze(self, memo): + assert self.frozen, "unfreeze(): state not frozen" + if self.back is not None: + new_back = self.back.unfreeze(memo) + else: + new_back = None + new_args_a = [] + for v, a in zip(self.getlivevars(), self.args_a): + a = a.unfreeze(memo) + # try to preserve the name + if isinstance(a.runtimevar, Variable): + a.runtimevar.rename(v) + new_args_a.append(a) + return self.__class__(new_back, new_args_a, *self.localkey()) + def getbindings(self): return dict(zip(self.getlivevars(), self.args_a)) @@ -419,9 +125,9 @@ """Block state in the middle of the execution of one instruction (typically a direct_call() that is causing inlining).""" - def __init__(self, a_back, args_a, origblock, origposition): + def __init__(self, back, args_a, origblock, origposition): self.origposition = origposition - super(LLSuspendedBlockState, self).__init__(a_back, args_a, origblock) + super(LLSuspendedBlockState, self).__init__(back, args_a, origblock) def localkey(self): return (self.origblock, self.origposition) @@ -432,6 +138,9 @@ # ____________________________________________________________ +ll_no_return_value = LLAbstractValue(const(None, lltype.Void)) + + class Policy(object): def __init__(self, inlining=False, const_propagate=False, concrete_propagate=True, concrete_args=True): @@ -454,19 +163,17 @@ self.policy = policy def itercopygraphs(self): - return self.graphs + return iter(self.graphs) def eval(self, origgraph, arghints): # 'arghints' maps argument index to a given ll value args_a = [] for i, v in enumerate(origgraph.getargs()): if i in arghints: - if self.policy.concrete_args: - a = LLConcreteValue(arghints[i]) - else: - a = LLRuntimeValue(const(arghints[i])) + a = LLAbstractValue(const(arghints[i])) + a.concrete = self.policy.concrete_args else: - a = LLRuntimeValue(orig_v=v) + a = LLAbstractValue(dupvar(v)) args_a.append(a) graphstate = self.schedule_graph(args_a, origgraph) graphstate.complete() @@ -474,32 +181,32 @@ def schedule_graph(self, args_a, origgraph): inputstate = LLBlockState(None, args_a, origgraph.startblock) - state = self.schedule_getstate(inputstate) + frozenstate = self.schedule_getstate(inputstate) try: - graphstate = self.graphstates[origgraph][state] + graphstate = self.graphstates[origgraph][frozenstate] except KeyError: d = self.graphstates.setdefault(origgraph, {}) graphstate = GraphState(self, origgraph, inputstate, n=len(d)) - d[state] = graphstate - self.pendingstates[graphstate] = state + d[frozenstate] = graphstate + self.pendingstates[graphstate] = frozenstate #print "SCHEDULE_GRAPH", graphstate return graphstate def schedule(self, inputstate): #print "SCHEDULE", args_a, origblock - state = self.schedule_getstate(inputstate) - args_v = inputstate.getruntimevars(VarMemo()) + frozenstate = self.schedule_getstate(inputstate) + args_v = inputstate.getruntimevars() newlink = Link(args_v, None) - self.pendingstates[newlink] = state + self.pendingstates[newlink] = frozenstate return newlink def schedule_getstate(self, inputstate): # NOTA BENE: copyblocks can get shared between different copygraphs! pendingstates = self.blocks.setdefault(inputstate.key(), []) # try to match the input state with an existing one - for i, state in enumerate(pendingstates): + for i, savedstate in enumerate(pendingstates): memo = MatchMemo() - if state.match(inputstate, memo): + if savedstate.match(inputstate, memo): # already matched must_restart = False for statevar, inputvar in memo.dependencies: @@ -517,32 +224,29 @@ # Record this inputstate's variables in the possible origins # of the saved state's variables. for statevar, inputvar in memo.dependencies: - if statevar.origin is not None: - statevar.origin.append(inputvar) - return state + statevar.origin.extend(inputvar.origin) + return savedstate else: - # cache and return this new state - pendingstates.append(inputstate) - return inputstate + # freeze and return this new state + frozenstate = inputstate.freeze(FreezeMemo()) + pendingstates.append(frozenstate) + return frozenstate def hint_needs_constant(self, a): - # Force the given LLRuntimeValue to be a fixed constant. - fix_me = [a] + # Force the given LLAbstractValue to be a fixed constant. + fix_me = list(a.origin) + progress = False while fix_me: - a = fix_me.pop() - assert isinstance(a, LLRuntimeValue) - if a.fixed: + fr = fix_me.pop() + if fr.fixed: continue # already fixed - print 'fixing:', a - a.fixed = True - # If 'a' is already a Constant, we just fixed it and we can - # continue. If it is a Variable, restart the whole process. - if a.origin: - fix_me.extend(a.origin) - elif a.maybe_get_constant() is None: - # a Variable with no recorded origin - raise Exception("hint() failed: cannot trace the variable %r " - "back to a link where it was a constant" % (a,)) + if not fr.fixed: + print 'fixing:', fr + fr.fixed = True + progress = True + fix_me.extend(fr.origin) + if not progress and a.maybe_get_constant() is None: + raise HintError("cannot trace the origin of %r" % (a,)) class GraphState(object): @@ -562,10 +266,10 @@ self.copygraph.exceptblock.inputargs[1])]: if hasattr(orig_v, 'concretetype'): copy_v.concretetype = orig_v.concretetype - # The 'args' attribute is needed by process_constant_input(), + # The 'args' attribute is needed by try_to_complete(), # which looks for it on either a GraphState or a Link - self.args = inputstate.getruntimevars(VarMemo()) - self.a_return = None + self.args = inputstate.getruntimevars() + #self.a_return = None self.state = "before" def settarget(self, block): @@ -601,8 +305,8 @@ print st = state stlist = [] - while st.a_back is not None: - st = st.a_back + while st.back is not None: + st = st.back stlist.append(st) stlist.reverse() for st in stlist: @@ -635,7 +339,7 @@ # that links with different *fixed* constants don't interfere # with each other. - key = tuple(state.getruntimevars(VarMemo(next.args))) + key = tuple(state.enum_fixed_constants(next.args)) try: block = state.copyblocks[key] except KeyError: @@ -677,7 +381,7 @@ # that it is really the one from 'graph' -- by patching # 'graph' if necessary. if len(link.target.inputargs) == 1: - self.a_return = state.args_a[0] + #self.a_return = state.args_a[0] graph.returnblock = link.target elif len(link.target.inputargs) == 2: graph.exceptblock = link.target @@ -713,14 +417,13 @@ # debugging print arglist = [] if key: - for v1, v2, k in zip(state.getruntimevars(VarMemo()), - builder.runningstate.getruntimevars(VarMemo()), + for a1, a2, k in zip(state.flatten(), + builder.runningstate.flatten(), key): if isinstance(k, Constant): - assert v2 == k + arglist.append('%s => %s' % (a1, k)) else: - assert isinstance(v2, Variable) - arglist.append('%s => %s' % (v1, v2)) + arglist.append('%s => %s' % (a1, a2)) print print '--> %s [%s]' % (origblock, ', '.join(arglist)) for op in origblock.operations: @@ -728,11 +431,10 @@ # end of debugging print try: if origblock.operations == (): - if state.a_back is None: + if state.back is None: # copies of return and except blocks are *normal* blocks # currently; they are linked to the official return or - # except block of the copygraph. If needed, - # LLConcreteValues are turned into Constants. + # except block of the copygraph. if len(origblock.inputargs) == 1: target = self.copygraph.returnblock else: @@ -746,7 +448,7 @@ # XXX GENERATE KEEPALIVES HERE if len(origblock.inputargs) == 1: a_result = builder.binding(origblock.inputargs[0]) - builder.runningstate = builder.runningstate.a_back + builder.runningstate = builder.runningstate.back origblock = builder.runningstate.origblock origposition = builder.runningstate.origposition builder.bindings = builder.runningstate.getbindings() @@ -789,7 +491,7 @@ newlinks = [] for origlink in links: args_a = [builder.binding(v) for v in origlink.args] - nextinputstate = LLBlockState(builder.runningstate.a_back, + nextinputstate = LLBlockState(builder.runningstate.back, args_a, origlink.target) newlink = self.interp.schedule(nextinputstate) if newexitswitch is not None: @@ -805,9 +507,11 @@ def __init__(self, interp, initialstate, key): self.interp = interp - memo = VarMemo(key) - self.runningstate = initialstate.with_fresh_variables(memo) - self.newinputargs = self.runningstate.getruntimevars(VarMemo()) + memo = UnfreezeMemo(key) + self.runningstate = initialstate.unfreeze(memo) + assert list(memo.propagateconsts) == [] # all items consumed + self.newinputargs = self.runningstate.getruntimevars() + assert len(self.newinputargs) == len(key) # {Variables-of-origblock: a_value} self.bindings = self.runningstate.getbindings() self.residual_operations = [] @@ -821,7 +525,7 @@ def binding(self, v): if isinstance(v, Constant): - return LLRuntimeValue(orig_v=v) + return LLAbstractValue(v) else: return self.bindings[v] @@ -847,16 +551,16 @@ if v is None: return None # cannot constant-fold concretevalues.append(v.value) - any_concrete = any_concrete or isinstance(a, LLConcreteValue) + any_concrete = any_concrete or a.concrete # can constant-fold print 'fold:', constant_op.__name__, concretevalues concreteresult = constant_op(*concretevalues) + a_result = LLAbstractValue(const(concreteresult)) if any_concrete and self.interp.policy.concrete_propagate: - return LLConcreteValue(concreteresult) + a_result.concrete = True else: - a_result = LLRuntimeValue(const(concreteresult)) self.record_origin(a_result, args_a) - return a_result + return a_result def residual(self, opname, args_a, a_result): v_result = a_result.forcevarorconst(self) @@ -876,21 +580,17 @@ a_result = self.constantfold(constant_op, args_a) if a_result is not None: return a_result - a_result = LLRuntimeValue(op.result) + a_result = LLAbstractValue(dupvar(op.result)) if constant_op: self.record_origin(a_result, args_a) self.residual(op.opname, args_a, a_result) return a_result def record_origin(self, a_result, args_a): - origin = [] + origin = a_result.origin for a in args_a: - if isinstance(a, LLConcreteValue): - continue - if not isinstance(a, LLRuntimeValue) or a.origin is None: - return - origin.extend(a.origin) - a_result.origin = origin + if not a.concrete: + origin.extend(a.origin) # ____________________________________________________________ # Operation handlers @@ -953,7 +653,7 @@ if hints.get('concrete'): # turn this 'a' into a concrete value a.forcevarorconst(self) - if not isinstance(a, LLConcreteValue): + if not a.concrete: self.interp.hint_needs_constant(a) c = a.maybe_get_constant() if c is None: @@ -963,7 +663,8 @@ # 'fixed', so if we restart now, op_hint() should receive # a constant the next time. raise RestartCompleting - a = LLConcreteValue(c.value) + a = LLAbstractValue(c) + a.concrete = True return a def op_direct_call(self, op, *args_a): @@ -993,7 +694,7 @@ alive_a = [] for v in live_variables(origblock, origposition): alive_a.append(self.bindings[v]) - parentstate = LLSuspendedBlockState(self.runningstate.a_back, alive_a, + parentstate = LLSuspendedBlockState(self.runningstate.back, alive_a, origblock, origposition) nextstate = LLBlockState(parentstate, args_a, origgraph.startblock) raise InsertNextLink(self.interp.schedule(nextstate)) @@ -1003,40 +704,42 @@ any_concrete = False for a in args_a: a.forcevarorconst(self) - any_concrete = any_concrete or isinstance(a,LLConcreteValue) + any_concrete = any_concrete or a.concrete if not any_concrete: return None - a_result = LLRuntimeValue(op.result) + a_result = LLAbstractValue(dupvar(op.result)) + a_real_result = a_result graphstate = self.interp.schedule_graph(args_a, origgraph) #print 'SCHEDULE_GRAPH', args_a, '==>', graphstate.copygraph.name if graphstate.state != "during": print 'ENTERING', graphstate.copygraph.name, args_a graphstate.complete() - if (graphstate.a_return is not None and - graphstate.a_return.maybe_get_constant() is not None): - a_result = graphstate.a_return - print 'LEAVING', graphstate.copygraph.name, graphstate.a_return + #if graphstate.a_return is not None: + # a = graphstate.a_return.unfreeze(UnfreezeMemo()) + # if a.maybe_get_constant() is not None: + # a_real_result = a # propagate a constant result + print 'LEAVING', graphstate.copygraph.name#, graphstate.a_return ARGS = [] new_args_a = [] for a in args_a: - if not isinstance(a, LLConcreteValue): + if not a.concrete: ARGS.append(a.getconcretetype()) new_args_a.append(a) args_a = new_args_a TYPE = lltype.FuncType(ARGS, a_result.getconcretetype()) fptr = lltype.functionptr( TYPE, graphstate.copygraph.name, graph=graphstate.copygraph) - a_func = LLRuntimeValue(const(fptr)) + a_func = LLAbstractValue(const(fptr)) self.residual("direct_call", [a_func] + args_a, a_result) - return a_result + return a_real_result def op_getfield(self, op, a_ptr, a_attrname): - if isinstance(a_ptr, LLVirtualStruct): + if a_ptr.content is not None: c_attrname = a_attrname.maybe_get_constant() assert c_attrname is not None - return a_ptr.getfield(c_attrname.value) + return a_ptr.content.getfield(c_attrname.value) constant_op = None T = a_ptr.getconcretetype().TO if T._hints.get('immutable', False): @@ -1044,23 +747,24 @@ return self.residualize(op, [a_ptr, a_attrname], constant_op) def op_getsubstruct(self, op, a_ptr, a_attrname): - if isinstance(a_ptr, LLVirtualContainer): + if a_ptr.content is not None: c_attrname = a_attrname.maybe_get_constant() assert c_attrname is not None - # this should return new LLVirtualContainer as well - return a_ptr.getfield(c_attrname.value) + # the difference with op_getfield() is only that the following + # line always returns a LLAbstractValue with content != None + return a_ptr.content.getfield(c_attrname.value) return self.residualize(op, [a_ptr, a_attrname], getattr) def op_getarraysize(self, op, a_ptr): - if isinstance(a_ptr, LLVirtualArray): - return a_ptr.a_length + if a_ptr.content is not None: + return LLAbstractValue(const(a_ptr.content.length)) return self.residualize(op, [a_ptr], len) def op_getarrayitem(self, op, a_ptr, a_index): - if isinstance(a_ptr, LLVirtualArray): + if a_ptr.content is not None: c_index = a_index.maybe_get_constant() if c_index is not None: - return a_ptr.getfield(c_index.value) + return a_ptr.content.getfield(c_index.value) constant_op = None T = a_ptr.getconcretetype().TO if T._hints.get('immutable', False): @@ -1070,56 +774,60 @@ def op_malloc(self, op, a_T): c_T = a_T.maybe_get_constant() assert c_T is not None - return LLVirtualStruct(c_T.value) + return virtualcontainervalue(c_T.value) def op_malloc_varsize(self, op, a_T, a_size): - if a_size.maybe_get_constant() is not None: + c_size = a_size.maybe_get_constant() + if c_size is not None: c_T = a_T.maybe_get_constant() assert c_T is not None - return virtualcontainer(c_T.value, a_length=a_size) + return virtualcontainervalue(c_T.value, c_size.value) return self.residualize(op, [a_T, a_size]) def op_setfield(self, op, a_ptr, a_attrname, a_value): - if isinstance(a_ptr, LLVirtualStruct): + if a_ptr.content is not None: c_attrname = a_attrname.maybe_get_constant() assert c_attrname is not None - a_ptr.setfield(c_attrname.value, a_value) + a_ptr.content.setfield(c_attrname.value, a_value) return ll_no_return_value return self.residualize(op, [a_ptr, a_attrname, a_value]) def op_setarrayitem(self, op, a_ptr, a_index, a_value): - if isinstance(a_ptr, LLVirtualArray): + if a_ptr.content is not None: c_index = a_index.maybe_get_constant() if c_index is not None: - a_ptr.setfield(c_index.value, a_value) + a_ptr.content.setfield(c_index.value, a_value) return ll_no_return_value return self.residualize(op, [a_ptr, a_index, a_value]) def op_cast_pointer(self, op, a_ptr): - if isinstance(a_ptr, LLVirtualStruct): + if a_ptr.content is not None: down_or_up = lltype.castable(op.result.concretetype, - a_ptr.getconcretetype()) + a_ptr.content.getconcretetype()) + # the following works because if a structure is virtual, then + # all its parent and inlined substructures are also virtual a = a_ptr if down_or_up >= 0: for n in range(down_or_up): - a = a.getfield(a.T._names[0]) + a = a.content.getfield(a.content.T._names[0]) else: for n in range(-down_or_up): - a = a.parent + a = a.content.a_parent return a def constant_op(ptr): return lltype.cast_pointer(op.result.concretetype, ptr) return self.residualize(op, [a_ptr], constant_op) def op_keepalive(self, op, a_ptr): - if isinstance(a_ptr, LLVirtualStruct): - for v in a_ptr.getruntimevars(VarMemo()): - if isinstance(v, Variable) and not v.concretetype._is_atomic(): - op = SpaceOperation('keepalive', [v], newvar(lltype.Void)) - print 'virtual:', op - self.residual_operations.append(op) - return ll_no_return_value - return self.residualize(op, [a_ptr]) + memo = FlattenMemo() + a_ptr.flatten(memo) + for a in memo.result: + v = a.runtimevar + if isinstance(v, Variable) and not v.concretetype._is_atomic(): + op = SpaceOperation('keepalive', [v], newvar(lltype.Void)) + print 'virtual:', op + self.residual_operations.append(op) + return ll_no_return_value class InsertNextLink(Exception): @@ -1129,19 +837,8 @@ class RestartCompleting(Exception): pass -class MatchMemo(object): - def __init__(self): - self.dependencies = [] - self.self_alias = {} - self.other_alias = {} - -class VarMemo(object): - def __init__(self, key=None): - self.seen = {} - if key is not None: - self.key = iter(key) - else: - self.key = None +class HintError(Exception): + pass def live_variables(block, position): Added: pypy/dist/pypy/jit/llcontainer.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/jit/llcontainer.py Mon Jan 2 21:54:52 2006 @@ -0,0 +1,232 @@ +from pypy.objspace.flow.model import Variable, Constant, SpaceOperation +from pypy.rpython.lltypesystem import lltype +from pypy.jit.llvalue import LLAbstractValue, newvar, const + + +class LLAbstractContainer(object): + """Abstract base class for abstract containers. + LLAbstractValues that are pointers may point to an LLAbstractContainer. + """ + # all methods are placeholders meant to be overridden + + def getconcretetype(self): + raise NotImplementedError("%r object has no concrete type" % ( + self.__class__.__name__,)) + + def flatten(self, memo): + pass + + def match(self, live, memo): + return self == live + + def freeze(self, memo): + return self + + def unfreeze(self, memo): + return self + + +class LLVirtualContainer(LLAbstractContainer): + + a_parent = None + parentindex = None + + def __init__(self, T, length=None): + assert (length is not None) == T._is_varsize() + self.T = T + self.length = length + self.names = self.getnames() + self.fields = {} + + def getconcretetype(self): + return lltype.Ptr(self.T) + + def getfield(self, name): + assert name in self.names # XXX slow + return self.fields[name] + + def setfield(self, name, a_value): + assert name in self.names # XXX slow + self.fields[name] = a_value + + def build_runtime_container(self, builder): + v_result = newvar(lltype.Ptr(self.T)) + if self.a_parent is not None: + v_parent = self.a_parent.build_runtime_container(builder) + op = SpaceOperation('getsubstruct', [v_parent, + const(self.parentindex, + lltype.Void)], + v_result) + print 'force:', op + builder.residual_operations.append(op) + else: + if self.T._is_varsize(): + op = SpaceOperation('malloc_varsize', [ + const(self.T, lltype.Void), + const(self.length, lltype.Signed)], + v_result) + else: + op = SpaceOperation('malloc', [const(self.T, lltype.Void)], + v_result) + print 'force:', op + builder.residual_operations.append(op) + self.buildcontent(builder, v_result) + return v_result + + def buildcontent(self, builder, v_target): + # initialize all fields + for name in self.names: + if name in self.fields: + a_value = self.fields[name] + T = self.fieldtype(name) + if isinstance(T, lltype.ContainerType): + # initialize the substructure/subarray + v_subptr = newvar(lltype.Ptr(T)) + op = SpaceOperation('getsubstruct', + [v_target, const(name, lltype.Void)], + v_subptr) + print 'force:', op + builder.residual_operations.append(op) + assert isinstance(a_value.content, LLVirtualContainer) + a_value.content.buildcontent(builder, v_subptr) + else: + v_value = a_value.forcevarorconst(builder) + op = self.setop(v_target, name, v_value) + print 'force:', op + builder.residual_operations.append(op) + + def flatten(self, memo): + assert self not in memo.seen; memo.seen[self] = True # debugging only + if self.a_parent is not None: + # self.a_parent.flatten() will not enumerate 'self' again, + # because should already be in the memo + self.a_parent.flatten(memo) + for name in self.names: + a_value = self.fields[name] + a_value.flatten(memo) + + def match(self, live, memo): + if self.__class__ is not live.__class__: + return False + + if self in memo.self_alias: + return live is memo.self_alias[self] + if live in memo.live_alias: + return self is memo.live_alias[live] + memo.self_alias[self] = live + memo.live_alias[live] = self + + assert self.T == live.T + if self.length != live.length: + return False + assert self.names == live.names + for name in self.names: + a1 = self.fields[name] + a2 = live.fields[name] + if not a1.match(a2, memo): + return False + else: + return True + + def freeze(self, memo): + result = self.__class__(self.T, self.length) + if self.a_parent is not None: + result.a_parent = self.a_parent.freeze(memo) + result.parentindex = self.parentindex + for name in self.names: + a_frozen = self.fields[name].freeze(memo) + result.fields[name] = a_frozen + return result + + def unfreeze(self, memo): + result = self.__class__(self.T, self.length) + if self.a_parent is not None: + result.a_parent = self.a_parent.unfreeze(memo) + result.parentindex = self.parentindex + for name in self.names: + a = self.fields[name].unfreeze(memo) + result.fields[name] = a + return result + + +class LLVirtualStruct(LLVirtualContainer): + """Stands for a pointer to a malloc'ed structure; the structure is not + malloc'ed so far, but we record which fields have which value. + """ + def __str__(self): + return 'struct %s' % (self.T._name,) + + def __repr__(self): + items = self.fields.items() + items.sort() + flds = ['%s=%r' % item for item in items] + return '' % (self.T._name, ', '.join(flds)) + + def getnames(self): + return self.T._names + + def fieldtype(self, name): + return getattr(self.T, name) + + def setop(self, v_target, name, v_value): + return SpaceOperation('setfield', [v_target, + const(name, lltype.Void), + v_value], + newvar(lltype.Void)) + + +class LLVirtualArray(LLVirtualContainer): + """Stands for a pointer to a malloc'ed array; the array is not + malloc'ed so far, but we record which fields have which value -- here + a field is an item, indexed by an integer instead of a string field name. + """ + def __str__(self): + return 'array[%d]' % (self.length,) + + def __repr__(self): + items = self.fields.items() + items.sort() + flds = ['%s=%r' % item for item in items] + return '' % (', '.join(flds),) + + def getnames(self): + return range(self.length) + + def fieldtype(self, index): + return self.T.OF + + def setop(self, v_target, name, v_value): + return SpaceOperation('setarrayitem', [v_target, + const(name, lltype.Signed), + v_value], + newvar(lltype.Void)) + + +def virtualcontainervalue(T, length=None): + """Build and return a LLAbstractValue() corresponding to a + freshly allocated virtual container. + """ + if isinstance(T, lltype.Struct): + cls = LLVirtualStruct + elif isinstance(T, lltype.Array): + cls = LLVirtualArray + else: + raise TypeError("unsupported container type %r" % (T,)) + content = cls(T, length) + a_result = LLAbstractValue(content=content) + # initialize fields to zero and allocate inlined substructures + for name in content.names: + a_fld = make_default(content.fieldtype(name), length, a_result, name) + content.fields[name] = a_fld + return a_result + +def make_default(T, length, a_parent, parentindex): + if isinstance(T, lltype.ContainerType): + # inlined substructure/subarray + a = virtualcontainervalue(T, length) + a.content.a_parent = a_parent + a.content.parentindex = parentindex + else: + # primitive initialized to zero + a = LLAbstractValue(const(T._defl())) + return a Added: pypy/dist/pypy/jit/llvalue.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/jit/llvalue.py Mon Jan 2 21:54:52 2006 @@ -0,0 +1,238 @@ +from pypy.objspace.flow.model import Variable, Constant +from pypy.rpython.lltypesystem import lltype + + +class LLAbstractValue(object): + """An abstract value, propagated through the blocks of the low-level + flow graphs and recording what the value will concretely be at run-time. + """ + # Instances of LLAbstractValue are either runtime values (where they + # stand for a Variable or Constant in the generated flow graph) or + # virtual values (where they stand for a pointer to a structure or + # array that is not yet built in memory). See __repr__() for how the + # various combinations of 'runtimevar' and 'content' encode the various + # states. + + concrete = False # concrete constants propagate eagerly + + def __init__(self, runtimevar=None, content=None): + self.runtimevar = runtimevar # None or a Variable or a Constant + self.content = content # None or an LLAbstractContainer + self.origin = [] # list of frozen values: the sources that did or + # could allow 'self' to be computed as a constant + + def __repr__(self): + if self.runtimevar is None: + if self.content is None: + return '' + else: + return '' % (self.content,) + else: + # runtime value -- display more precisely if it's a + # Variable or a Constant + if isinstance(self.runtimevar, Variable): + kind = 'runtime' + elif self.concrete: + kind = 'concrete' + else: + kind = 'constant' + return '<%s %s>' % (kind, self.runtimevar) + + def freeze(self, memo): + # turn a run-time value into a frozen value + if self.runtimevar is not None: + if self.concrete: + return LLFrozenConcreteValue(self) + else: + # don't use 'memo' here: for now, shared LLAbstractValues + # need to become distinct LLFrozenRuntimeValues. + return LLFrozenRuntimeValue(self) + else: + # virtual container: preserve sharing + if self in memo.seen: + return memo.seen[self] # already seen + else: + result = LLFrozenVirtualValue() + memo.seen[self] = result + result.fz_content = self.content.freeze(memo) + return result + + def getconcretetype(self): + if self.runtimevar is not None: + return self.runtimevar.concretetype + else: + return lltype.Ptr(self.content.T) + + def forcevarorconst(self, builder): + if self.runtimevar is None: + self.runtimevar = self.content.build_runtime_container(builder) + self.content = None + return self.runtimevar + + def maybe_get_constant(self): + if isinstance(self.runtimevar, Constant): + return self.runtimevar + else: + return None + + def flatten(self, memo): + """Recursively flatten the LLAbstractValue into a list of run-time + LLAbstractValues. + """ + if self.runtimevar is not None: + if not self.concrete: # skip concrete values, they don't need + # to be present in the residual graph at all + memo.result.append(self) + else: + if self not in memo.seen: + memo.seen[self] = True + self.content.flatten(memo) + +# ____________________________________________________________ + +class LLFrozenValue(object): + """An abstract value frozen in a saved state. + """ + # When the end of a block is reached, the LLAbstractValues are + # frozen by creating LLFrozenValues of the same shape. In the + # frozen values, the Variable is forgotten, because it was only + # relevant in the finished block. + + +class LLFrozenConcreteValue(LLFrozenValue): + + def __init__(self, a_source): + self.a_source = a_source + + def flatten(self, memo): + """Recursively flatten the frozen value into a list of frozen values. + """ + pass + + def unfreeze(self, memo): + """Create an un-frozen copy of a frozen value recursively, + creating fresh Variables. + """ + return self.a_source # we can keep the same LLAbstractValue around + + def match(self, a_value, memo): + """Check if a frozen state is a suitable match for a live state + reaching the same point. This checks if freezing the live 'a_value' + would basically result in the same state as 'self'. + """ + return (a_value.concrete and + a_value.runtimevar == self.a_source.runtimevar) + + +class LLFrozenRuntimeValue(LLFrozenValue): + + fixed = False # Describes what this frozen value will become in + # the next block. Set to True by hint() to force a + # Constant to stay a Constant. + + def __init__(self, a_value): + # get the necessary information from the a_value + self.concretetype = a_value.runtimevar.concretetype + self.origin = a_value.origin + + def flatten(self, memo): + """Recursively flatten the frozen value into a list of frozen values. + """ + memo.result.append(self) + + def unfreeze(self, memo): + """Create an un-frozen copy of a frozen value recursively, + creating fresh Variables. + """ + # no need to worry about sharing here: LLFrozenRuntimeValues are + # never shared + propagateconst = memo.propagateconsts.next() + if isinstance(propagateconst, Constant): + v = propagateconst # allowed to propagate as a Constant + assert v.concretetype == self.concretetype + else: + v = newvar(self.concretetype) + result = LLAbstractValue(v) + result.origin.append(self) + return result + + def match(self, a_value, memo): + """Check if a frozen state is a suitable match for a live state + reaching the same point. This checks if freezing the live 'a_value' + would basically result in the same state as 'self'. + """ + # any two non-concrete run-time values match + memo.dependencies.append((self, a_value)) + return a_value.runtimevar is not None and not a_value.concrete + + +class LLFrozenVirtualValue(LLFrozenValue): + # fz_content must be initialized to a frozen LLAbstractContainer + + def flatten(self, memo): + """Recursively flatten the frozen value into a list of frozen values. + """ + if self not in memo.seen: + memo.seen[self] = True + self.fz_content.flatten(memo) + + def unfreeze(self, memo): + """Create an un-frozen copy of a frozen value recursively, + creating fresh Variables. + """ + # virtual container: preserve sharing + if self in memo.seen: + return memo.seen[self] # already seen + else: + a = LLAbstractValue() + memo.seen[self] = a + a.content = self.fz_content.unfreeze(memo) + return a + + def match(self, a_value, memo): + """Check if a frozen state is a suitable match for a live state + reaching the same point. This checks if freezing the live 'a_value' + would basically result in the same state as 'self'. + """ + # check virtual values recursively + return (a_value.content is not None and + self.fz_content.match(a_value.content, memo)) + +# ____________________________________________________________ + +class FlattenMemo: + def __init__(self): + self.seen = {} + self.result = [] + +class MatchMemo: + def __init__(self): + self.dependencies = [] + self.self_alias = {} + self.live_alias = {} + +class FreezeMemo: + def __init__(self): + self.seen = {} + +class UnfreezeMemo: + def __init__(self, propagateconsts): + self.propagateconsts = iter(propagateconsts) + self.seen = {} + +# ____________________________________________________________ + +def const(value, T=None): + c = Constant(value) + c.concretetype = T or lltype.typeOf(value) + return c + +def newvar(T): + v = Variable() + v.concretetype = T + return v + +def dupvar(v): + v1 = Variable(v) + v1.concretetype = v.concretetype + return v1 Modified: pypy/dist/pypy/jit/test/test_llabstractinterp.py ============================================================================== --- pypy/dist/pypy/jit/test/test_llabstractinterp.py (original) +++ pypy/dist/pypy/jit/test/test_llabstractinterp.py Mon Jan 2 21:54:52 2006 @@ -1,3 +1,4 @@ +import py from pypy.translator.translator import TranslationContext from pypy.rpython.annlowlevel import annotate_lowlevel_helper from pypy.rpython.lltypesystem import lltype @@ -192,6 +193,8 @@ assert insns == {} def test_recursive_call(): + py.test.skip("reimplement or remove the test: " + "non-inlined calls with constant results") def ll_factorial(k): if k <= 1: return 1 From arigo at codespeak.net Mon Jan 2 22:02:52 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Mon, 2 Jan 2006 22:02:52 +0100 (CET) Subject: [pypy-svn] r21654 - pypy/extradoc/talk Message-ID: <20060102210252.14CF127B55@code1.codespeak.net> Author: arigo Date: Mon Jan 2 22:02:49 2006 New Revision: 21654 Modified: pypy/extradoc/talk/conference-attendance.txt Log: Mention the Winter Meeting of Chalmers. Modified: pypy/extradoc/talk/conference-attendance.txt ============================================================================== --- pypy/extradoc/talk/conference-attendance.txt (original) +++ pypy/extradoc/talk/conference-attendance.txt Mon Jan 2 22:02:49 2006 @@ -25,6 +25,21 @@ Both talks were accepted into the CCC programme! +Winter Meeting of Chalmers +-------------------------- + +This is an internal meeting of the Computer Science and Engineering -- +Chalmers University of Technology and Goteborg University. + +Time & Location: 11th-13th January 2006, Alingsas. + +Armin submitted a technical implementation talk (mostly the same paper +as for the CCC conference, based on architecture.txt). In principle no +submissions are rejected. + +See http://www.cs.chalmers.se/wm/. + + Solution Linux 2006 ------------------- From pedronis at codespeak.net Mon Jan 2 22:48:28 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Mon, 2 Jan 2006 22:48:28 +0100 (CET) Subject: [pypy-svn] r21659 - pypy/extradoc/planning Message-ID: <20060102214828.B0F9027B5C@code1.codespeak.net> Author: pedronis Date: Mon Jan 2 22:48:27 2006 New Revision: 21659 Modified: pypy/extradoc/planning/phase2_projectactivities.txt Log: remove stale, inactual information about EU review/reports from this document Modified: pypy/extradoc/planning/phase2_projectactivities.txt ============================================================================== --- pypy/extradoc/planning/phase2_projectactivities.txt (original) +++ pypy/extradoc/planning/phase2_projectactivities.txt Mon Jan 2 22:48:27 2006 @@ -71,8 +71,6 @@ plans: - * (10 th January eu review workshop) - * 23-29th January: CS Dep. of the Palma de Mallorca University/CS Dep. * of the Barcelona University. Bea check dates, Bruxelles/Louvain-la-neuve, @@ -171,76 +169,3 @@ - WP05: left over work for Impara (when they get accepted)? - -EU-related report planning -------------------------------------------------- - -People (primary) and co-reviewers (secondary) and completion dates: -(lead partner coordinating, technical board evaluating overall consistency) - -DO1.1 Create QA plan for the project -Primary/Secondary: Bea/Holger -Completion date: 21th Oct 2005 - -D04.1 Partial Python Impl: -Primary/Secondary: Jacob / Christian -Completion date: 28th Oct 2005 - -D04.2 Complete Python Impl: -Primary/Secondary: Jacob / Christian -Completion date: 4th Nov 2005 - -D04.3 Parser report: -Primary/Secondary: ludovic,adrien / Arre -Completion date: 11th Nov 2005 - -D04.4 Release PyPy-research-tool: -Primary/Secondary: Samuele / Ludovic -Completion date: 11th Nov 2005 - -D05.1 Publish on translating a very-high-level description: -Primary/Secondary: Armin / Michael -Completion date: 4th Nov 2005 - -D05.2 A compiled, self-contained version of PyPy: -Primary/Secondary: dfki/Anders L. / Jacob -Completion date: 28th Oct 2005 - -D05.3 Publish on implementation with translation aspects: -Primary/Secondary: Carl / Armin, Christian -Completion date: TBA - -D05.4 Publish on encapsulating low level language aspects: -Primary/Secondary: Armin / holger -Completion date: TBA - -D14.1 Report about Milestone/Phase 1: -Primary/Secondary: Bea/Stephan -Completion date:21th October 2005 - -Review planning: - -Roadmap: - -* 31th October 2005: all partners check completeness of timesheets Dec 2004-Oct 2005 - (Stephan wp1, Jacob wp3,4,5, Bea wp14, all partners) - -* 5th December 2005: all partners submit timesheets for November 2005 - (Stephan wp1, Jacob wp3,4,5, Bea wp14, all partners) - -* 9th December 2005: Periodic management/activity reports updated with all data but costs - (Jacob//Bea) - -* 15th December 2005: submit reports to EC/reviewers - (Stephan) - -* 16th December 2005: all partners submit FORM C:s to DFKI (or communicate a new - completion date for this!) - (Stephan, all partners) - -* 10th January 2006: review in Brussels - (Stephan) - -* 15th of January 2006: submit FORM C:s to EC/PO and possible updated periodic management/ - activity reports - (Stephan//Jacob,Bea From arigo at codespeak.net Tue Jan 3 10:03:29 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 3 Jan 2006 10:03:29 +0100 (CET) Subject: [pypy-svn] r21669 - pypy/extradoc/talk/chalmers Message-ID: <20060103090329.2C4DE27B55@code1.codespeak.net> Author: arigo Date: Tue Jan 3 10:03:25 2006 New Revision: 21669 Modified: pypy/extradoc/talk/chalmers/techpaper.pdf pypy/extradoc/talk/chalmers/techpaper.tex pypy/extradoc/talk/chalmers/techpaper.txt Log: Changing the title. Modified: pypy/extradoc/talk/chalmers/techpaper.pdf ============================================================================== Binary files. No diff available. Modified: pypy/extradoc/talk/chalmers/techpaper.tex ============================================================================== --- pypy/extradoc/talk/chalmers/techpaper.tex (original) +++ pypy/extradoc/talk/chalmers/techpaper.tex Tue Jan 3 10:03:25 2006 @@ -56,9 +56,9 @@ \newcommand{\titlereference}[1]{\textsl{#1}} % end of "some commands" \input{techpaper.sty} -\title{PyPy - Architecture Overview} +\title{PyPy: Dynamic Optimizations for Your Favorite Language} \hypersetup{ -pdftitle={PyPy - Architecture Overview} +pdftitle={PyPy: Dynamic Optimizations for Your Favorite Language} } \raggedbottom \begin{document} Modified: pypy/extradoc/talk/chalmers/techpaper.txt ============================================================================== --- pypy/extradoc/talk/chalmers/techpaper.txt (original) +++ pypy/extradoc/talk/chalmers/techpaper.txt Tue Jan 3 10:03:25 2006 @@ -1,6 +1,6 @@ -=============================================== -PyPy - Architecture Overview -=============================================== +====================================================== +PyPy: Dynamic Optimizations for Your Favorite Language +====================================================== .. command to produce latex file: .. rst2latex.py --use-latex-citations --use-latex-footnotes --use-latex-toc --documentoptions=a4paper,11pt --hyperlink-color=0 techpaper.txt --use-latex-docinfo --stylesheet=techpaper.sty techpaper.tex From bea at codespeak.net Tue Jan 3 16:43:55 2006 From: bea at codespeak.net (bea at codespeak.net) Date: Tue, 3 Jan 2006 16:43:55 +0100 (CET) Subject: [pypy-svn] r21674 - pypy/extradoc/planning Message-ID: <20060103154355.C857127B40@code1.codespeak.net> Author: bea Date: Tue Jan 3 16:43:54 2006 New Revision: 21674 Added: pypy/extradoc/planning/sprint_planning.txt Log: new file - summing up all planned sprints for the second year. We seem to have a "hole" between April and July, October and November sprint location still open.... Added: pypy/extradoc/planning/sprint_planning.txt ============================================================================== --- (empty file) +++ pypy/extradoc/planning/sprint_planning.txt Tue Jan 3 16:43:54 2006 @@ -0,0 +1,69 @@ +Author: Beatrice D?ring +20060103 + +Sprint planning PyPy +---------------------- + +January/Palma de Mallorca +-------------------------- + +Time & location: 23-29th January 2006, CS Dep. of the Palma de +Mallorca University/GNU Linux lab. + +Nature: open + +Primary organizer: Bea + +February/PyCon (USA) +--------------------- + +Time & location: 27 Feb - 1st March 2006 PyCon +(24-26 Feb, 2006 Dallas, Texas, + +Nature: postsprint + +Primary organizer: Holger and Michael + +April/Tokyo +------------- + +Time & location: 23rd to 29th of April, 2006, host: +Yutaka Niibe, National Institute of AIST, Japan (FSIJ) +Akihabara, Tokyo, venue for 32 people. + +Nature: open, dissemination + +Primary organizer: Bea + + +July/Europython +------------------ + +Time & location: 3rd-5th of July, 2005, CERN Switzerland +(sprints at 30th of June-2nd of July and 6th of July to 9th of July) + +Nature: presprint/postsprint? + +Primary organizer: Michael H, Holger) + +August/University of Limerick +------------------------------- + +Time & location: 21th to 27th of August, 2006 at University +of Limerick, Ireland + +Nature: open, dissemination (workshop with Calibre?) + +Primary organizer: Bea + + +October/? +---------------------------- + +November/? +---------------------------- + + +Other general possibilities: +Romania, Switzerland (Leysin,Bern), +Bristol, Uganda (Nkozi), Bruxelles From bea at codespeak.net Tue Jan 3 16:45:12 2006 From: bea at codespeak.net (bea at codespeak.net) Date: Tue, 3 Jan 2006 16:45:12 +0100 (CET) Subject: [pypy-svn] r21675 - pypy/extradoc/planning Message-ID: <20060103154512.C0CDF27B44@code1.codespeak.net> Author: bea Date: Tue Jan 3 16:45:11 2006 New Revision: 21675 Added: pypy/extradoc/planning/conference-planning.txt Log: new file (replacing conference attendance and phase2_projectactivities) - primary file for conferences that we submit papers to but also need to attend Added: pypy/extradoc/planning/conference-planning.txt ============================================================================== --- (empty file) +++ pypy/extradoc/planning/conference-planning.txt Tue Jan 3 16:45:11 2006 @@ -0,0 +1,149 @@ + +Conferences Planning for PyPy talks +--------------------------------------- + +CCC 2005 +--------------- + +Chaos Communication Conference 2005, a 2000-3000 people +conference focusing on all things hacking (including +cultural events). Non-Python specific Audience. + +Call for Papers: http://www.ccc.de/congress/2005/cfp.html +DEADLINE: 31st September 2005 + +see http://codespeak.net/pypy/extradoc/talk/22c3 for +proposal preparations! + +Time & Location: 27th-30th December, Berlin. + +holger submitted two proposals with help of Armin, Carl +and Bea. One very much regarding the technical implementation +and one regarding business/open-source +agile-development/pypy-sprint experiences. + +Both talks were accepted into the CCC programme! + + +Winter Meeting of Chalmers +-------------------------- + +This is an internal meeting of the Computer Science and Engineering -- +Chalmers University of Technology and Goteborg University. + +Time & Location: 11th-13th January 2006, Alingsas. + +Armin submitted a technical implementation talk (mostly the same paper +as for the CCC conference, based on architecture.txt). In principle no +submissions are rejected. + +See http://www.cs.chalmers.se/wm/. + + +Solution Linux 2006 +------------------- + +Logilab will present PyPy in the "free software models" track. + +Time and Location: 31st jan to feb 2nd, Paris. + + +Pycon 2006 +--------------- + +DEADLINE: 31st October 2005 + +Annual US Python conference, strong Python audience. + +Call for Papers: http://www.python.org/pycon/2006/cfp + +Time & location: 24-26 Feburary, 2006, Addison, Texas (near Dallas) + +Out of the pypy group four talks were submitted. +Acceptance pending (15th November ASFAIK). +The 3 PyPy talks where accepted into the PyCon programme! + +Due: sprint planning, possibly inviting Ka Ping (regarding E-lang) + +FOSDEM 2006 +------------ +The sixth meeting of Free and Open Source European Developers meeting. +Time & location: 25th-26th of February, 2006, Brussels + +A conference to attend - maybe for those not participating at PyCon? + +http://www.fosdem.org/2006/ + +No papers submitted - attendance? + +Calibre workshop +----------------- +CALIBRE Workshop on Distributed Development, Open Source & Industry +- Opportunities, Suspicions, & Strategies + +Time & location: 3 of March 2006 Sk?vde Sweden + +www.calibre.ie + +No papers submitted - Bea will attend. + +ACCU 2006 +---------- + +DEADLINE: 10th of February (early bird) + +Software development - C++, Agile methods and Python. + +Time & location: 19th -22nd of April, Oxford, UK + +http://www.accu.org/conference/ + +No papers submitted - attendance - Michael H, Jacob? + +XP 2006 +----------- + +eXtreme programming and agile processes in software engineering. + +Time & location: 17-22th June 2006 Oulu, Finland + +Call for papers: +Submission date: 13th of January 2006 +(Bea plan to submit an experience report) + +http://virtual.vtt.fi/virtual/xp2006/sivu1.html + +Agile 2006 +----------- + +DEADLINE: 23 June 2005 (end of early bird registration) + +Annual US agile conference (the main agile alliance conference) - last year were +sold out. + +Call for papers: +http://www.agile2006.org./ +- 31 January for research papers +- 28 February for experience reports + +Time and location: 23-28th of July 2006, Minneapolis, Minnesota, USA + +Change Maker (Bea) will plan to submit: +- experience report +- research paper + +OOPSLA 2006 +------------ +DEADLINE: registration starts 15th of June (no deadline known) + +International conference on Objecto Oriented programming, Systems +languages and applications. + +Time & location: 22nd-26th of October 2006, Portland, Oregon + +Call for papers: +Submission deadline: 18th of March + +http://www.oopsla.org/2006/ + +Samuele and Armin plan to submit paper and attend. \ No newline at end of file From bea at codespeak.net Tue Jan 3 16:46:18 2006 From: bea at codespeak.net (bea at codespeak.net) Date: Tue, 3 Jan 2006 16:46:18 +0100 (CET) Subject: [pypy-svn] r21676 - pypy/extradoc/talk Message-ID: <20060103154618.7F1E027B45@code1.codespeak.net> Author: bea Date: Tue Jan 3 16:46:17 2006 New Revision: 21676 Removed: pypy/extradoc/talk/conference-attendance.txt Log: removed conference_attendance - it has been replaced by conference_planning in the planning dir From pedronis at codespeak.net Tue Jan 3 22:32:36 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Tue, 3 Jan 2006 22:32:36 +0100 (CET) Subject: [pypy-svn] r21679 - pypy/extradoc/planning Message-ID: <20060103213236.4564927B40@code1.codespeak.net> Author: pedronis Date: Tue Jan 3 22:32:33 2006 New Revision: 21679 Added: pypy/extradoc/planning/sprint-planning.txt - copied unchanged from r21678, pypy/extradoc/planning/sprint_planning.txt Removed: pypy/extradoc/planning/sprint_planning.txt Log: consistent naming From hpk at codespeak.net Tue Jan 3 22:35:22 2006 From: hpk at codespeak.net (hpk at codespeak.net) Date: Tue, 3 Jan 2006 22:35:22 +0100 (CET) Subject: [pypy-svn] r21680 - in pypy/extradoc: sprintinfo/paris talk/22c3 talk/pypy_euworkshop_2005-12-08 Message-ID: <20060103213522.97A8E27B3E@code1.codespeak.net> Author: hpk Date: Tue Jan 3 22:35:21 2006 New Revision: 21680 Modified: pypy/extradoc/sprintinfo/paris/paris-report.txt pypy/extradoc/talk/22c3/slides-agility.txt pypy/extradoc/talk/pypy_euworkshop_2005-12-08/holger_part2_OSSdev.txt pypy/extradoc/talk/pypy_euworkshop_2005-12-08/pypy_alastair_part1_intro.txt Log: various ReST fixes in extradoc Modified: pypy/extradoc/sprintinfo/paris/paris-report.txt ============================================================================== --- pypy/extradoc/sprintinfo/paris/paris-report.txt (original) +++ pypy/extradoc/sprintinfo/paris/paris-report.txt Tue Jan 3 22:35:21 2006 @@ -27,6 +27,7 @@ Nicholas Chauvat ----------------------------------------------------------------------------------------------------------------------------------------- + The largest PyPy sprint yet (in terms of attendants) where done in the offices of Logilab in Paris 2005-10-16-2005-10-16. @@ -40,6 +41,7 @@ devise a low-level model variation for this case Annotator related tasks: + - support the notion of separate/external functions (and classes/PBCs) in preparation for separate compilations @@ -85,12 +87,14 @@ - (DONE) genc: producing multiple .h/.c files tracking Python code origin Larger whole projects: + - Javascript frontend - support for developing C extensions for CPython with RPython code - writing a sort of flow object space for the llinterpreter to experiment with JIT work ---------------------------------------------------------------------------------------------------------------------------------------- + Activities during the sprint - day-to-day: Monday-Wednesday: Modified: pypy/extradoc/talk/22c3/slides-agility.txt ============================================================================== --- pypy/extradoc/talk/22c3/slides-agility.txt (original) +++ pypy/extradoc/talk/22c3/slides-agility.txt Tue Jan 3 22:35:21 2006 @@ -336,3 +336,4 @@ .. |bullet| unicode:: U+02022 .. footer:: Bea During, Holger Krekel |bullet| 22C3 |bullet| 29th December 2005 + Modified: pypy/extradoc/talk/pypy_euworkshop_2005-12-08/holger_part2_OSSdev.txt ============================================================================== --- pypy/extradoc/talk/pypy_euworkshop_2005-12-08/holger_part2_OSSdev.txt (original) +++ pypy/extradoc/talk/pypy_euworkshop_2005-12-08/holger_part2_OSSdev.txt Tue Jan 3 22:35:21 2006 @@ -8,6 +8,7 @@ 1. personal background + - worked in gaming companies, banks and car companies for several years - studied computer science @@ -24,10 +25,10 @@ 3. technical production factors - - automated test driven development - - specific expertise/special interest - - version control (Subversion) - - releases +- automated test driven development +- specific expertise/special interest +- version control (Subversion) +- releases 4. Typical aspects of the Python community? @@ -38,50 +39,51 @@ 5. PyPy: the vision - - founders came from the Python community - - "sprints" were the inital factor - - what is PyPy/Python - one of the five most used programming - languages today - - grass root approach +- founders came from the Python community +- "sprints" were the inital factor +- what is PyPy/Python - one of the five most used programming + languages today +- grass root approach 6. OSS and EU funding: PyPy as a case study - - driven by partially EU funded and non-EU funded parties - - focus on avoiding friction and turning PyPy into a long - term project - - IBM or Sun have done similarly challenging projects - in more time and with more funding - - yet not found completely satisfying "funding" - interactions with communities. +- driven by partially EU funded and non-EU funded parties +- focus on avoiding friction and turning PyPy into a long + term project +- IBM or Sun have done similarly challenging projects + in more time and with more funding +- yet not found completely satisfying "funding" + interactions with communities. 7. PyPy technical status - - three public releases in 2005, well received by the - community - - core deliverables fullfilled - - contributors add different directions + +- three public releases in 2005, well received by the + community +- core deliverables fullfilled +- contributors add different directions 8. PyPy: It's all about communication ... - - pypy-sync meetings, 30 minutes IRC - - day-to-day IRC discussions - - "this week in PyPy" - - mailing lists: pypy-svn/eu-tracking tracks code and document changes - - around 20000 visitors per month on website - - lots of blogs and pypy-dev (developer/researcher list) - - 300-500 people across the world following the project +- pypy-sync meetings, 30 minutes IRC +- day-to-day IRC discussions +- "this week in PyPy" +- mailing lists: pypy-svn/eu-tracking tracks code and document changes +- around 20000 visitors per month on website +- lots of blogs and pypy-dev (developer/researcher list) +- 300-500 people across the world following the project 9. all good and well but the main thing are: sprints - - one-week intense work-meetings with one break day - - EU and non-EU researchers/developers get together - - daily planning sessions - - pair programming - - evolving and adapting to more attendants - - organisational/management tasks happen also on sprints +- one-week intense work-meetings with one break day +- EU and non-EU researchers/developers get together +- daily planning sessions +- pair programming +- evolving and adapting to more attendants +- organisational/management tasks happen also on sprints 10. next - - tackling research and technical goals (challenging!) - - mid-term EU review planned for 20th january - - looking into adjusting some work planning - - increased dissemination, attending conferences (movie features?) - - start talking to and interact with commercial stakeholders +- tackling research and technical goals (challenging!) +- mid-term EU review planned for 20th january +- looking into adjusting some work planning +- increased dissemination, attending conferences (movie features?) +- start talking to and interact with commercial stakeholders Modified: pypy/extradoc/talk/pypy_euworkshop_2005-12-08/pypy_alastair_part1_intro.txt ============================================================================== --- pypy/extradoc/talk/pypy_euworkshop_2005-12-08/pypy_alastair_part1_intro.txt (original) +++ pypy/extradoc/talk/pypy_euworkshop_2005-12-08/pypy_alastair_part1_intro.txt Tue Jan 3 22:35:21 2006 @@ -208,4 +208,3 @@ - Tensions between large commercial developers and outsiders. ---------------------------------------------------------------------------- \ No newline at end of file From pedronis at codespeak.net Tue Jan 3 22:37:52 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Tue, 3 Jan 2006 22:37:52 +0100 (CET) Subject: [pypy-svn] r21681 - pypy/extradoc/planning Message-ID: <20060103213752.B03F027B47@code1.codespeak.net> Author: pedronis Date: Tue Jan 3 22:37:50 2006 New Revision: 21681 Added: pypy/extradoc/planning/phase2_projectactivities_051015.txt - copied unchanged from r21678, pypy/extradoc/planning/phase2_projectactivities.txt Removed: pypy/extradoc/planning/phase2_projectactivities.txt Log: attach a date to the filename to underline that this give at-one-point-in-time picture. From ericvrp at codespeak.net Wed Jan 4 11:47:54 2006 From: ericvrp at codespeak.net (ericvrp at codespeak.net) Date: Wed, 4 Jan 2006 11:47:54 +0100 (CET) Subject: [pypy-svn] r21691 - in pypy/dist/pypy/translator: goal llvm Message-ID: <20060104104754.542A027B3E@code1.codespeak.net> Author: ericvrp Date: Wed Jan 4 11:47:52 2006 New Revision: 21691 Modified: pypy/dist/pypy/translator/goal/bench-cronjob.py pypy/dist/pypy/translator/goal/bench-unix.py pypy/dist/pypy/translator/llvm/build_llvm_module.py Log: Modified nightly benchmark to generate four executables with genllvm. * llvm1.6 with -march=c * llvm1.6 with -march=x86 * llvm1.7 with -march=c * llvm1.7 with -march=x86 Added some llvm optimization option as suggested by Chris Lattner (mr. llvm) Modified: pypy/dist/pypy/translator/goal/bench-cronjob.py ============================================================================== --- pypy/dist/pypy/translator/goal/bench-cronjob.py (original) +++ pypy/dist/pypy/translator/goal/bench-cronjob.py Wed Jan 4 11:47:52 2006 @@ -1,9 +1,10 @@ #! /usr/bin/env python +import autopath +import py import time, os, sys, stat homedir = os.getenv('HOME') -os.putenv('PATH','~/bin:/usr/local/bin:/usr/bin:/bin:/opt/bin:/usr/i686-pc-linux-gnu/gcc-bin/3.3.6') def update_pypy(): os.chdir(homedir + '/projects/pypy-dist') @@ -15,6 +16,80 @@ os.system('make clean 2>&1') os.system('make -j3 tools-only 2>&1') +def compile_llvm_variants(revision): + tmpdir = py.std.tempfile.gettempdir() + '/usession-' + os.environ['USER'] + '/' + + cmd = 'cp %sentry_point.ll pypy/translator/goal/entry_point-%s.ll' % (tmpdir, revision) + print cmd + os.system(cmd) + + #this assumes entry_point.bc is created with llvm 1.6 and pypy-llvm- is already created with the llvm c backend + b = "pypy/translator/goal/pypy-llvm-%s" % revision + cmd = "mv %s %s-llvm1_6_c" % (b, b) + print cmd + os.system(cmd) + + # + b = "%spypy-llvm-%s-llvm1_6_x86" % (tmpdir, revision) + + cmd = "llc %sentry_point.bc -f -o %s.s" % (tmpdir, b) + print cmd + os.system(cmd) + + cmd = "as %s.s -o %s.o" % (b, b) + print cmd + os.system(cmd) + + cmd = "gcc %s.o -static -lgc -lm -lpthread -pipe -o %s" % (b, b) + print cmd + os.system(cmd) + + cmd = "cp %s pypy/translator/goal" % b + print cmd + os.system(cmd) + + + # + b = "%spypy-llvm-%s-llvm1_7_c" % (tmpdir, revision) + + cmd = '~/bin/llvm-as < %sentry_point.ll | ~/bin/opt -verify -lowersetjmp -funcresolve -raiseallocs -simplifycfg -mem2reg -globalopt -globaldce -ipconstprop -deadargelim -instcombine -simplifycfg -basiccg -prune-eh -inline -simplify-libcalls -basiccg -argpromotion -raise -tailduplicate -simplifycfg -scalarrepl -instcombine -break-crit-edges -condprop -tailcallelim -simplifycfg -reassociate -loopsimplify -licm -instcombine -indvars -loop-unroll -instcombine -load-vn -gcse -sccp -instcombine -break-crit-edges -condprop -dse -mergereturn -adce -simplifycfg -deadtypeelim -constmerge -verify -globalopt -constmerge -ipsccp -deadargelim -inline -instcombine -scalarrepl -globalsmodref-aa -licm -load-vn -gcse -instcombine -simplifycfg -globaldce -f -o %sentry_point_llvm1_7.bc' % (tmpdir, tmpdir) + print cmd + os.system(cmd) + + cmd = "~/bin/llc %sentry_point_llvm1_7.bc -march=c -f -o %s.c" % (tmpdir, b) + print cmd + os.system(cmd) + + cmd = "gcc %s.c -O2 -fomit-frame-pointer -static -lgc -lm -lpthread -pipe -o %s" % (b, b) + print cmd + os.system(cmd) + + cmd = "cp %s pypy/translator/goal" % b + print cmd + os.system(cmd) + + + # + b = "%spypy-llvm-%s-llvm1_7_x86" % (tmpdir, revision) + os.system('~/bin/llc -version 2>&1') + + cmd = "~/bin/llc %sentry_point_llvm1_7.bc -f -o %s.s" % (tmpdir, b) + print cmd + os.system(cmd) + + cmd = "as %s.s -o %s.o" % (b, b) + print cmd + os.system(cmd) + + cmd = "gcc %s.o -static -lgc -lm -lpthread -pipe -o %s" % (b, b) + print cmd + os.system(cmd) + + cmd = "cp %s pypy/translator/goal" % b + print cmd + os.system(cmd) + + def compile(backend): os.chdir(homedir + '/projects/pypy-dist') os.system('rm `find . -name *.pyc`') @@ -23,7 +98,7 @@ os.system('rm *') os.chdir(homedir + '/projects/pypy-dist/pypy/translator/goal') - os.system('python translate_pypy.py --backend=%(backend)s --text --batch targetpypystandalone 2>&1' % locals()) + os.system('/usr/local/bin/python translate_pypy.py --backend=%(backend)s --text --batch targetpypystandalone.py 2>&1' % locals()) os.chdir(homedir + '/projects/pypy-dist') try: @@ -39,11 +114,14 @@ os.chmod(realname, stat.S_IRWXU) os.unlink(basename) + if backend == 'llvm': + compile_llvm_variants(revision) + def benchmark(): os.system('cat /proc/cpuinfo') os.system('free') os.chdir(homedir + '/projects/pypy-dist/pypy/translator/goal') - os.system('python bench-unix.py 2>&1 | tee benchmark.txt' % locals()) + os.system('/usr/local/bin/python bench-unix.py 2>&1 | tee benchmark.txt' % locals()) os.system('echo "
"    >  benchmark.html')
     os.system('cat benchmark.txt           >> benchmark.html')
     os.system('echo "
" >> benchmark.html') Modified: pypy/dist/pypy/translator/goal/bench-unix.py ============================================================================== --- pypy/dist/pypy/translator/goal/bench-unix.py (original) +++ pypy/dist/pypy/translator/goal/bench-unix.py Wed Jan 4 11:47:52 2006 @@ -24,12 +24,12 @@ pipe = os.popen(cmd + ' 2>&1') return pipe.read() -def run_pystone(executable='python', n=0): +def run_pystone(executable='/usr/local/bin/python', n=0): argstr = PYSTONE_CMD % (str(n) and n or '') txt = run_cmd('%s -c "%s"' % (executable, argstr)) return get_result(txt, PYSTONE_PATTERN) -def run_richards(executable='python', n=10): +def run_richards(executable='/usr/local/bin/python', n=10): argstr = RICHARDS_CMD % n txt = run_cmd('%s -c "%s"' % (executable, argstr)) return get_result(txt, RICHARDS_PATTERN) Modified: pypy/dist/pypy/translator/llvm/build_llvm_module.py ============================================================================== --- pypy/dist/pypy/translator/llvm/build_llvm_module.py (original) +++ pypy/dist/pypy/translator/llvm/build_llvm_module.py Wed Jan 4 11:47:52 2006 @@ -32,10 +32,7 @@ ])) flags = os.popen("gccas /dev/null -o /dev/null -debug-pass=Arguments 2>&1").read()[17:-1].split() - -#if int(os.popen("opt --help 2>&1").read().find('-heap2stack')) >= 0: -# flags.insert(flags.index("-inline")+1, "-heap2stack -debug") - +flags += "-globalopt -constmerge -ipsccp -deadargelim -inline -instcombine -scalarrepl -globalsmodref-aa -licm -load-vn -gcse -instcombine -simplifycfg -globaldce".split() OPTIMIZATION_SWITCHES = " ".join(flags) def compile_module(module, source_files, object_files, library_files): From ale at codespeak.net Wed Jan 4 14:10:41 2006 From: ale at codespeak.net (ale at codespeak.net) Date: Wed, 4 Jan 2006 14:10:41 +0100 (CET) Subject: [pypy-svn] r21692 - in pypy/dist/pypy/lib/pyontology: . test Message-ID: <20060104131041.29B7827B42@code1.codespeak.net> Author: ale Date: Wed Jan 4 14:10:40 2006 New Revision: 21692 Modified: pypy/dist/pypy/lib/pyontology/pyontology.py pypy/dist/pypy/lib/pyontology/test/test_ontology.py Log: Addedsupport for FunctionalProperties, rearranged the constraint classes, added tests Modified: pypy/dist/pypy/lib/pyontology/pyontology.py ============================================================================== --- pypy/dist/pypy/lib/pyontology/pyontology.py (original) +++ pypy/dist/pypy/lib/pyontology/pyontology.py Wed Jan 4 14:10:40 2006 @@ -79,7 +79,6 @@ def setValues(self, values): for k,v in values: - print "####",k,v self._dict.setdefault(k,[]) if type(v) == list: self._dict[k] = v @@ -118,10 +117,16 @@ pass -class FunctionalProperty: +class FunctionalProperty(Property): - def __init__(self): - pass + def __init__(self, name='', values=[], bases = []): + Property.__init__(self, name, values, bases) + self.constraint = FunctionalCardinality(name, 1) + +class InverseFunctionalProperty(Property): + + def __init__(self, name='', values=[], bases = []): + Property.__init__(self, name, values, bases) class DataRange: @@ -141,7 +146,7 @@ ## 'DatatypeProperty' : DatatypeProperty, ## 'DeprecatedClass' : DeprecatedClass, ## 'DeprecatedProperty' : DeprecatedProperty, -## 'FunctionalProperty' : FunctionalProperty, + 'FunctionalProperty' : FunctionalProperty, ## 'InverseFunctionalProperty' : InverseFunctionalProperty, ## 'Nothing' : Nothing, ## 'ObjectProperty' : ObjectProperty, @@ -185,7 +190,12 @@ avar = self.make_var(fd, s) else: res = [o] - avar = self.make_var(fd, s, p) + avar = self.make_var(Property, p) + # Set the values of the property p to o + sub = self.make_var(fd, s) + obj = self.make_var(fd, o) + res = self.variables[avar].getValues() + self.variables[avar].setValues(res + [(sub, obj)]) if self.variables.get(avar) and type(self.variables[avar]) == fd: self.variables[avar] = fd(list(self.variables[avar].getValues()) + res) else: @@ -250,7 +260,7 @@ res.append(a) var = '.'.join([str(a.replace('-','_')) for a in res]) if not var in self.variables.keys(): - self.variables[var] = cls(name=var) + self.variables[var] = cls(var) return var def find_prop(self, s): @@ -300,6 +310,8 @@ else: # var is a builtin class cls =builtin_voc[var.split('#')[-1]](name=svar) + if hasattr(cls, 'constraint'): + self.constraints.append(cls.constraint) vals = self.variables[svar].getValues() cls.setValues(vals) self.variables[svar] = cls @@ -387,7 +399,6 @@ if not k in vals: vals.append((k,v)) self.variables[svar].setValues(vals) - print "->",vals, self.variables[svar] cons = DomainConstraint(svar, avar) self.constraints.append(cons) @@ -467,21 +478,28 @@ # ----------------- Helper classes ---------------- -class MaxCardinality(AbstractConstraint): - """Contraint: all values must be distinct""" +class OwlConstraint(AbstractConstraint): - def __init__(self, variable, cardinality): + def __init__(self, variable): AbstractConstraint.__init__(self, [variable]) - # worst case complexity - self.__cost = 1 #len(variables) * (len(variables) - 1) / 2 - self.cardinality = cardinality + self.__cost = 1 def __repr__(self): - return '' % (str(self._variables[0]),self.cardinality) + return '<%s %s,%i>' % (self.__class__.__name__, str(self._variables[0]),self.cardinality) def estimateCost(self, domains): return self.__cost + +class MaxCardinality(OwlConstraint): + """Contraint: all values must be distinct""" + + def __init__(self, variable, cardinality): + OwlConstraint.__init__(self, [variable]) + # worst case complexity + self.__cost = 1 #len(variables) * (len(variables) - 1) / 2 + self.cardinality = cardinality + def narrow(self, domains): """narrowing algorithm for the constraint""" if len(domains[self._variables[0]]) > self.cardinality: @@ -491,9 +509,6 @@ class MinCardinality(MaxCardinality): - def __repr__(self): - return '' % (str(self._variables[0]),self.cardinality) - def narrow(self, domains): """narrowing algorithm for the constraint""" @@ -504,9 +519,6 @@ class Cardinality(MaxCardinality): - def __repr__(self): - return '' % (str(self._variables[0]),self.cardinality) - def narrow(self, domains): """narrowing algorithm for the constraint""" @@ -527,12 +539,10 @@ #res[dom] = 1 return res -class SubClassConstraint(AbstractConstraint): +class SubClassConstraint(OwlConstraint): def __init__(self, variable, cls_or_restriction): - AbstractConstraint.__init__(self, [variable]) - # worst case complexity - self.__cost = 1 #len(variables) * (len(variables) - 1) / 2 + OwlConstraint.__init__(self, variable) self.super = cls_or_restriction self.variable = variable @@ -544,12 +554,10 @@ vals = get_values(subdom, domains, 'getValues') superdom.values += [val for val in vals if val not in superdom.values] -class DisjointClassConstraint(AbstractConstraint): +class DisjointClassConstraint(OwlConstraint): def __init__(self, variable, cls_or_restriction): - AbstractConstraint.__init__(self, [variable]) - # worst case complexity - self.__cost = 1 #len(variables) * (len(variables) - 1) / 2 + OwlConstraint.__init__(self, [variable]) self.super = cls_or_restriction self.variable = variable @@ -562,14 +570,12 @@ vals2 = get_values(variable, domains, 'getValues') for i in vals1: if i in vals2: - raise ConsistencyError + raise ConsistencyFailure() -class ComplementClassConstraint(AbstractConstraint): +class ComplementClassConstraint(OwlConstraint): def __init__(self, variable, cls_or_restriction): - AbstractConstraint.__init__(self, [variable]) - # worst case complexity - self.__cost = 1 #len(variables) * (len(variables) - 1) / 2 + OwlConstraint.__init__(self, variable) self.super = cls_or_restriction self.variable = variable @@ -577,12 +583,10 @@ subdom = domains[self.variable] superdom = domains[self.super] -class RangeConstraint(AbstractConstraint): +class RangeConstraint(OwlConstraint): def __init__(self, variable, cls_or_restriction): - AbstractConstraint.__init__(self, [variable]) - # worst case complexity - self.__cost = 1 #len(variables) * (len(variables) - 1) / 2 + OwlConstraint.__init__(self, variable) self.super = cls_or_restriction self.variable = variable @@ -598,12 +602,10 @@ res.append((k,v)) subdom.removeValues(res) -class DomainConstraint(AbstractConstraint): +class DomainConstraint(OwlConstraint): def __init__(self, variable, cls_or_restriction): - AbstractConstraint.__init__(self, [variable]) - # worst case complexity - self.__cost = 1 #len(variables) * (len(variables) - 1) / 2 + OwlConstraint.__init__(self, variable) self.super = cls_or_restriction self.variable = variable @@ -615,16 +617,12 @@ for k,val in get_values(subdom, domains, 'getValues'): if not k in vals and k != superdom: res.append((k,val)) - print "res",res,vals,superdom subdom.removeValues(res) - print "---",subdom - -class SubPropertyConstraint(AbstractConstraint): + +class SubPropertyConstraint(OwlConstraint): def __init__(self, variable, cls_or_restriction): - AbstractConstraint.__init__(self, [variable]) - # worst case complexity - self.__cost = 1 #len(variables) * (len(variables) - 1) / 2 + OwlConstraint.__init__(self, variable) self.super = cls_or_restriction self.variable = variable @@ -634,14 +632,30 @@ vals = get_values(superdom, domains, 'getValues') for val in subdom.getValues(): if not val in vals: - raise ConsistencyError("Value not in prescribed range") + vals.append(val) + superdom.setValues(vals) -class EquivalentPropertyConstraint(AbstractConstraint): +class FunctionalCardinality(OwlConstraint): + """Contraint: all values must be distinct""" + + def __init__(self, variable, cardinality): + OwlConstraint.__init__(self, variable) + self.cardinality = cardinality + self.variable = variable + + def narrow(self, domains): + """narrowing algorithm for the constraint""" + domain = domains[self.variable].getValues() + for cls, val in domain: + if len(val) != self.cardinality: + raise ConsistencyFailure("Maxcardinality exceeded") + else: + return 0 + +class EquivalentPropertyConstraint(OwlConstraint): def __init__(self, variable, cls_or_restriction): - AbstractConstraint.__init__(self, [variable]) - # worst case complexity - self.__cost = 1 #len(variables) * (len(variables) - 1) / 2 + OwlConstraint.__init__(self, variable) self.super = cls_or_restriction self.variable = variable @@ -651,5 +665,4 @@ vals = get_values(superdom, domains, 'getValues') for val in subdom.getValues(): if not val in vals: - raise ConsistencyError("Value not in prescribed range") - \ No newline at end of file + raise ConsistencyFailure("Value not in prescribed range") Modified: pypy/dist/pypy/lib/pyontology/test/test_ontology.py ============================================================================== --- pypy/dist/pypy/lib/pyontology/test/test_ontology.py (original) +++ pypy/dist/pypy/lib/pyontology/test/test_ontology.py Wed Jan 4 14:10:40 2006 @@ -1,4 +1,5 @@ # tests for the Ontology class +import py try: import logilab.constraint @@ -182,4 +183,31 @@ O.type(b, None, obj) O.subPropertyOf(sub, None, b) assert len(O.constraints) ==1 - assert O.variables['a_'].getValues() in O.variables['b_'].getValues() \ No newline at end of file + O.variables['a_'].setValues([('individ_',42)]) + O.consistency() + for val in O.variables['a_'].getValues(): + assert val in O.variables['b_'].getValues() + +def test_functionalproperty(): + + O = Ontology() + #Make functional property + sub = URIRef('p') + obj = URIRef(namespaces['owl']+'#FunctionalProperty') + O.type(sub, None, obj) + #Make class + sub = URIRef('c') + obj = URIRef(namespaces['owl']+'#Class') + O.type(sub, None, obj) + #Make individual with a value of the property + sub = URIRef('individ') + obj = URIRef('c') + O.type(sub, None, obj) + O.variables['p_'].setValues([('individ_',42)]) + assert len(O.constraints) == 2 + #add another valueof the property + O.variables['p_'].setValues([('individ_',42),('individ_',43)]) + #check that consistency raises + print O.constraints + print O.variables + py.test.raises(ConsistencyFailure, O.consistency) \ No newline at end of file From ale at codespeak.net Wed Jan 4 15:39:30 2006 From: ale at codespeak.net (ale at codespeak.net) Date: Wed, 4 Jan 2006 15:39:30 +0100 (CET) Subject: [pypy-svn] r21693 - in pypy/dist/pypy/lib/pyontology: . test Message-ID: <20060104143930.A494F27B5A@code1.codespeak.net> Author: ale Date: Wed Jan 4 15:39:29 2006 New Revision: 21693 Modified: pypy/dist/pypy/lib/pyontology/pyontology.py pypy/dist/pypy/lib/pyontology/test/test_ontology.py Log: Added support for TransitiveProperty + test Modified: pypy/dist/pypy/lib/pyontology/pyontology.py ============================================================================== --- pypy/dist/pypy/lib/pyontology/pyontology.py (original) +++ pypy/dist/pypy/lib/pyontology/pyontology.py Wed Jan 4 15:39:29 2006 @@ -127,6 +127,13 @@ def __init__(self, name='', values=[], bases = []): Property.__init__(self, name, values, bases) + self.constraint = InverseFunctionalCardinality(name, 1) + +class TransitiveProperty(Property): + + def __init__(self, name='', values=[], bases = []): + Property.__init__(self, name, values, bases) + self.constraint = TransitiveConstraint(name) class DataRange: @@ -147,14 +154,14 @@ ## 'DeprecatedClass' : DeprecatedClass, ## 'DeprecatedProperty' : DeprecatedProperty, 'FunctionalProperty' : FunctionalProperty, -## 'InverseFunctionalProperty' : InverseFunctionalProperty, + 'InverseFunctionalProperty' : InverseFunctionalProperty, ## 'Nothing' : Nothing, ## 'ObjectProperty' : ObjectProperty, ## 'Ontology' : Ontology, ## 'OntologyProperty' : OntologyProperty, 'Restriction' : Restriction, ## 'SymmetricProperty' : SymmetricProperty, -## 'TransitiveProperty' : TransitiveProperty + 'TransitiveProperty' : TransitiveProperty } class Ontology(Graph): @@ -482,10 +489,11 @@ def __init__(self, variable): AbstractConstraint.__init__(self, [variable]) + self.variable = variable self.__cost = 1 def __repr__(self): - return '<%s %s,%i>' % (self.__class__.__name__, str(self._variables[0]),self.cardinality) + return '<%s %s>' % (self.__class__.__name__, str(self._variables[0])) def estimateCost(self, domains): return self.__cost @@ -495,9 +503,8 @@ """Contraint: all values must be distinct""" def __init__(self, variable, cardinality): - OwlConstraint.__init__(self, [variable]) - # worst case complexity - self.__cost = 1 #len(variables) * (len(variables) - 1) / 2 + OwlConstraint.__init__(self, variable) + self.__cost = 1 self.cardinality = cardinality def narrow(self, domains): @@ -635,15 +642,25 @@ vals.append(val) superdom.setValues(vals) -class FunctionalCardinality(OwlConstraint): - """Contraint: all values must be distinct""" +class EquivalentPropertyConstraint(OwlConstraint): - def __init__(self, variable, cardinality): + def __init__(self, variable, cls_or_restriction): OwlConstraint.__init__(self, variable) - self.cardinality = cardinality + self.super = cls_or_restriction self.variable = variable def narrow(self, domains): + subdom = domains[self.variable] + superdom = domains[self.super] + vals = get_values(superdom, domains, 'getValues') + for val in subdom.getValues(): + if not val in vals: + raise ConsistencyFailure("Value not in prescribed range") + +class FunctionalCardinality(MaxCardinality): + """Contraint: all values must be distinct""" + + def narrow(self, domains): """narrowing algorithm for the constraint""" domain = domains[self.variable].getValues() for cls, val in domain: @@ -652,17 +669,32 @@ else: return 0 -class EquivalentPropertyConstraint(OwlConstraint): +class InverseFunctionalCardinality(MaxCardinality): + """Contraint: all values must be distinct""" - def __init__(self, variable, cls_or_restriction): - OwlConstraint.__init__(self, variable) - self.super = cls_or_restriction - self.variable = variable + def narrow(self, domains): + """narrowing algorithm for the constraint""" + domain = domains[self.variable].getValues() + vals = {} + for cls, val in domain: + for v in val: + if vals.has_key(v): + raise ConsistencyFailure("Maxcardinality exceeded") + else: + vals[v] = 1 + else: + return 0 + +class TransitiveConstraint(OwlConstraint): + """Contraint: all values must be distinct""" def narrow(self, domains): - subdom = domains[self.variable] - superdom = domains[self.super] - vals = get_values(superdom, domains, 'getValues') - for val in subdom.getValues(): - if not val in vals: - raise ConsistencyFailure("Value not in prescribed range") + """narrowing algorithm for the constraint""" + domain = domains[self.variable].getValues() + domain_dict = dict( domain ) + for cls, val in domain: + for v in val: + if v in domain_dict: + val.extend(domain_dict[v]) + + Modified: pypy/dist/pypy/lib/pyontology/test/test_ontology.py ============================================================================== --- pypy/dist/pypy/lib/pyontology/test/test_ontology.py (original) +++ pypy/dist/pypy/lib/pyontology/test/test_ontology.py Wed Jan 4 15:39:29 2006 @@ -208,6 +208,52 @@ #add another valueof the property O.variables['p_'].setValues([('individ_',42),('individ_',43)]) #check that consistency raises - print O.constraints - print O.variables - py.test.raises(ConsistencyFailure, O.consistency) \ No newline at end of file + py.test.raises(ConsistencyFailure, O.consistency) + +def test_inversefunctionalproperty(): + + O = Ontology() + #Make functional property + sub = URIRef('p') + obj = URIRef(namespaces['owl']+'#InverseFunctionalProperty') + O.type(sub, None, obj) + #Make class + sub = URIRef('c') + obj = URIRef(namespaces['owl']+'#Class') + O.type(sub, None, obj) + #Make individual with a value of the property + sub = URIRef('individ') + obj = URIRef('c') + O.type(sub, None, obj) + O.variables['p_'].setValues([('individ_',42)]) + assert len(O.constraints) == 2 + #add another individual with the same value for the property + sub = URIRef('individ2') + obj = URIRef('c') + O.type(sub, None, obj) + O.variables['p_'].setValues([('individ2_',42)]) + #check that consistency raises + py.test.raises(ConsistencyFailure, O.consistency) + +def test_Transitiveproperty(): + + O = Ontology() + #Make functional property + sub = URIRef('subRegionOf') + obj = URIRef(namespaces['owl']+'#TransitiveProperty') + O.type(sub, None, obj) + #Make class + sub = URIRef('c') + obj = URIRef(namespaces['owl']+'#Class') + O.type(sub, None, obj) + #Make individual with a value of the property + sub = URIRef('Italy') + obj = URIRef('c') + O.type(sub, None, obj) + sub = URIRef('Tuscanny') + O.type(sub, None, obj) + sub = URIRef('Chianti') + O.type(sub, None, obj) + O.variables['subRegionOf_'].setValues([('Italy_','Tuscanny_'),('Tuscanny_','Chianti_')]) + O.consistency() + assert ('Italy_', ['Tuscanny_', 'Chianti_']) in O.variables['subRegionOf_'].getValues() \ No newline at end of file From ale at codespeak.net Wed Jan 4 16:55:57 2006 From: ale at codespeak.net (ale at codespeak.net) Date: Wed, 4 Jan 2006 16:55:57 +0100 (CET) Subject: [pypy-svn] r21695 - in pypy/dist/pypy/lib/pyontology: . test Message-ID: <20060104155557.6EEE527B46@code1.codespeak.net> Author: ale Date: Wed Jan 4 16:55:56 2006 New Revision: 21695 Modified: pypy/dist/pypy/lib/pyontology/pyontology.py pypy/dist/pypy/lib/pyontology/test/test_ontology.py Log: Added support for SymmetricProperty and test Modified: pypy/dist/pypy/lib/pyontology/pyontology.py ============================================================================== --- pypy/dist/pypy/lib/pyontology/pyontology.py (original) +++ pypy/dist/pypy/lib/pyontology/pyontology.py Wed Jan 4 16:55:56 2006 @@ -104,8 +104,7 @@ class Thing: - def __init__(self): - pass + pass class AllDifferent(ClassDomain): # A special class whose members are distinct @@ -135,6 +134,12 @@ Property.__init__(self, name, values, bases) self.constraint = TransitiveConstraint(name) +class SymmetricProperty(Property): + + def __init__(self, name='', values=[], bases = []): + Property.__init__(self, name, values, bases) + self.constraint = SymmetricConstraint(name) + class DataRange: def __init__(self): @@ -150,7 +155,7 @@ 'AllDifferent' : AllDifferent , ## 'AnnotationProperty' : AnnotationProperty, ## 'DataRange' : DataRange, -## 'DatatypeProperty' : DatatypeProperty, + 'DatatypeProperty' : DatatypeProperty, ## 'DeprecatedClass' : DeprecatedClass, ## 'DeprecatedProperty' : DeprecatedProperty, 'FunctionalProperty' : FunctionalProperty, @@ -160,7 +165,7 @@ ## 'Ontology' : Ontology, ## 'OntologyProperty' : OntologyProperty, 'Restriction' : Restriction, -## 'SymmetricProperty' : SymmetricProperty, + 'SymmetricProperty' : SymmetricProperty, 'TransitiveProperty' : TransitiveProperty } @@ -311,7 +316,7 @@ (var in [URIRef(namespaces['owl']+'#'+x) for x in builtin_voc])): # var is not one of the builtin classes avar = self.make_var(ClassDomain, var) - self.variables[svar].values += self.variables[avar].values + self.variables[svar].setValues(self.variables[avar].getValues()) constrain = BinaryExpression([svar, avar],"%s in %s" %(svar, avar)) self.constraints.append(constrain) else: @@ -696,5 +701,18 @@ for v in val: if v in domain_dict: val.extend(domain_dict[v]) + domain_dict[cls] = val + domains[self.variable].setValues(domain_dict.items()) +class SymmetricConstraint(OwlConstraint): + """Contraint: all values must be distinct""" + def narrow(self, domains): + """narrowing algorithm for the constraint""" + domain = domains[self.variable].getValues() + domain_dict = dict( domain ) + for cls, val in domain: + for v in val: + domain_dict.setdefault(v, []) + domain_dict[v].append(cls) + domains[self.variable].setValues(domain_dict.items()) Modified: pypy/dist/pypy/lib/pyontology/test/test_ontology.py ============================================================================== --- pypy/dist/pypy/lib/pyontology/test/test_ontology.py (original) +++ pypy/dist/pypy/lib/pyontology/test/test_ontology.py Wed Jan 4 16:55:56 2006 @@ -256,4 +256,25 @@ O.type(sub, None, obj) O.variables['subRegionOf_'].setValues([('Italy_','Tuscanny_'),('Tuscanny_','Chianti_')]) O.consistency() - assert ('Italy_', ['Tuscanny_', 'Chianti_']) in O.variables['subRegionOf_'].getValues() \ No newline at end of file + assert ('Italy_', ['Tuscanny_', 'Chianti_']) in O.variables['subRegionOf_'].getValues() + +def test_symmetricproperty(): + + O = Ontology() + #Make functional property + sub = URIRef('friend') + obj = URIRef(namespaces['owl']+'#SymmetricProperty') + O.type(sub, None, obj) + #Make class + sub = URIRef('c') + obj = URIRef(namespaces['owl']+'#Class') + O.type(sub, None, obj) + #Make individual with a value of the property + sub = URIRef('Bob') + obj = URIRef('c') + O.type(sub, None, obj) + sub = URIRef('Alice') + O.type(sub, None, obj) + O.variables['friend_'].setValues([('Bob_','Alice_')]) + O.consistency() + assert ('Alice_', ['Bob_']) in O.variables['friend_'].getValues() \ No newline at end of file From tismer at codespeak.net Wed Jan 4 20:09:34 2006 From: tismer at codespeak.net (tismer at codespeak.net) Date: Wed, 4 Jan 2006 20:09:34 +0100 (CET) Subject: [pypy-svn] r21696 - pypy/extradoc/sprintinfo/mallorca Message-ID: <20060104190934.7B62827B42@code1.codespeak.net> Author: tismer Date: Wed Jan 4 20:09:33 2006 New Revision: 21696 Modified: pypy/extradoc/sprintinfo/mallorca/people.txt (props changed) Log: eol-style:native can please everybody do this automatically? From tismer at codespeak.net Wed Jan 4 20:11:21 2006 From: tismer at codespeak.net (tismer at codespeak.net) Date: Wed, 4 Jan 2006 20:11:21 +0100 (CET) Subject: [pypy-svn] r21697 - pypy/extradoc/sprintinfo/mallorca Message-ID: <20060104191121.3165827B46@code1.codespeak.net> Author: tismer Date: Wed Jan 4 20:11:20 2006 New Revision: 21697 Modified: pypy/extradoc/sprintinfo/mallorca/people.txt Log: update for the tismerysoft gang Modified: pypy/extradoc/sprintinfo/mallorca/people.txt ============================================================================== --- pypy/extradoc/sprintinfo/mallorca/people.txt (original) +++ pypy/extradoc/sprintinfo/mallorca/people.txt Wed Jan 4 20:11:20 2006 @@ -16,7 +16,9 @@ Michael Hudson ? ? Carl Friedrich Bolz ? ? Anders Chrigstroem ? ? -Christian Tismer ? ? +Christian Tismer 22/1 - 30/1 ? +Gerald Klix 22/1 - 30/1 ? +Stephan Diehl 22/1 - 28/1 ? Jacob Hallen ? ? Holger Krekel ? ? ==================== ============== ===================== From ale at codespeak.net Thu Jan 5 13:05:09 2006 From: ale at codespeak.net (ale at codespeak.net) Date: Thu, 5 Jan 2006 13:05:09 +0100 (CET) Subject: [pypy-svn] r21710 - pypy/extradoc/sprintinfo/mallorca Message-ID: <20060105120509.D1AF327B55@code1.codespeak.net> Author: ale Date: Thu Jan 5 13:05:08 2006 New Revision: 21710 Modified: pypy/extradoc/sprintinfo/mallorca/people.txt Log: my info Modified: pypy/extradoc/sprintinfo/mallorca/people.txt ============================================================================== --- pypy/extradoc/sprintinfo/mallorca/people.txt (original) +++ pypy/extradoc/sprintinfo/mallorca/people.txt Thu Jan 5 13:05:08 2006 @@ -20,6 +20,7 @@ Gerald Klix 22/1 - 30/1 ? Stephan Diehl 22/1 - 28/1 ? Jacob Hallen ? ? +Anders Lehmann 22/1 -29/1 Hote Almudainal Holger Krekel ? ? ==================== ============== ===================== From ale at codespeak.net Thu Jan 5 13:06:30 2006 From: ale at codespeak.net (ale at codespeak.net) Date: Thu, 5 Jan 2006 13:06:30 +0100 (CET) Subject: [pypy-svn] r21711 - pypy/extradoc/sprintinfo/mallorca Message-ID: <20060105120630.C498C27B47@code1.codespeak.net> Author: ale Date: Thu Jan 5 13:06:30 2006 New Revision: 21711 Modified: pypy/extradoc/sprintinfo/mallorca/people.txt Log: typo Modified: pypy/extradoc/sprintinfo/mallorca/people.txt ============================================================================== --- pypy/extradoc/sprintinfo/mallorca/people.txt (original) +++ pypy/extradoc/sprintinfo/mallorca/people.txt Thu Jan 5 13:06:30 2006 @@ -20,7 +20,7 @@ Gerald Klix 22/1 - 30/1 ? Stephan Diehl 22/1 - 28/1 ? Jacob Hallen ? ? -Anders Lehmann 22/1 -29/1 Hote Almudainal +Anders Lehmann 22/1 -29/1 Hotel Almudainal Holger Krekel ? ? ==================== ============== ===================== From pedronis at codespeak.net Thu Jan 5 14:11:36 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Thu, 5 Jan 2006 14:11:36 +0100 (CET) Subject: [pypy-svn] r21715 - pypy/extradoc/minute Message-ID: <20060105131136.03A3027B43@code1.codespeak.net> Author: pedronis Date: Thu Jan 5 14:11:35 2006 New Revision: 21715 Added: pypy/extradoc/minute/pypy-sync-05-01-2006.txt (contents, props changed) - copied, changed from r21709, pypy/extradoc/minute/pypy-sync-11-10-2005.txt Log: pypy-sync minutes From ale at codespeak.net Thu Jan 5 14:35:50 2006 From: ale at codespeak.net (ale at codespeak.net) Date: Thu, 5 Jan 2006 14:35:50 +0100 (CET) Subject: [pypy-svn] r21719 - in pypy/dist/pypy/lib/pyontology: . test Message-ID: <20060105133550.10E3F27B62@code1.codespeak.net> Author: ale Date: Thu Jan 5 14:35:49 2006 New Revision: 21719 Modified: pypy/dist/pypy/lib/pyontology/pyontology.py pypy/dist/pypy/lib/pyontology/test/test_ontology.py Log: A number of corrections and refactorings. Tests changed accordingly Modified: pypy/dist/pypy/lib/pyontology/pyontology.py ============================================================================== --- pypy/dist/pypy/lib/pyontology/pyontology.py (original) +++ pypy/dist/pypy/lib/pyontology/pyontology.py Thu Jan 5 14:35:49 2006 @@ -65,7 +65,10 @@ def setValues(self, values): self.values = values - + +class List(ClassDomain): + pass + class Property(ClassDomain): # Property contains the relationship between a class instance and a value # - a pair. To accomodate global assertions like 'range' and 'domain' anonymous @@ -75,7 +78,7 @@ self._dict = {} def getValues(self): - return tuple(self._dict.items()) + return self._dict.items() def setValues(self, values): for k,v in values: @@ -98,12 +101,13 @@ pass -class DataTypeProperty(Property): - +class DatatypeProperty(Property): pass -class Thing: +class Thing(ClassDomain): + pass +class DataRange(ClassDomain): pass class AllDifferent(ClassDomain): @@ -111,7 +115,7 @@ # Syntactic sugar pass -class Nothing: +class Nothing(ClassDomain): pass @@ -140,11 +144,6 @@ Property.__init__(self, name, values, bases) self.constraint = SymmetricConstraint(name) -class DataRange: - - def __init__(self): - pass - class Restriction(ClassDomain): pass @@ -154,8 +153,8 @@ 'ObjectProperty' : ObjectProperty, 'AllDifferent' : AllDifferent , ## 'AnnotationProperty' : AnnotationProperty, -## 'DataRange' : DataRange, - 'DatatypeProperty' : DatatypeProperty, + 'DataRange' : DataRange, + 'DatatypeProperty' : DatatypeProperty, ## 'DeprecatedClass' : DeprecatedClass, ## 'DeprecatedProperty' : DeprecatedProperty, 'FunctionalProperty' : FunctionalProperty, @@ -166,7 +165,8 @@ ## 'OntologyProperty' : OntologyProperty, 'Restriction' : Restriction, 'SymmetricProperty' : SymmetricProperty, - 'TransitiveProperty' : TransitiveProperty + 'TransitiveProperty' : TransitiveProperty, + 'List' : List } class Ontology(Graph): @@ -191,11 +191,10 @@ else: ns ='' func = p - - if ns in namespaces.items(): + if ns in namespaces.values(): pred = getattr(self, func) res = pred(s, p, o) - if res == None: + if not res : continue if type(res) != list : res = [res] @@ -207,7 +206,7 @@ sub = self.make_var(fd, s) obj = self.make_var(fd, o) res = self.variables[avar].getValues() - self.variables[avar].setValues(res + [(sub, obj)]) + self.variables[avar].setValues(res + ((sub, obj),)) if self.variables.get(avar) and type(self.variables[avar]) == fd: self.variables[avar] = fd(list(self.variables[avar].getValues()) + res) else: @@ -232,15 +231,14 @@ rep = Repository(self.variables.keys(), self.variables, self.constraints) return Solver().solve(rep, verbose) - def consistency(self): + def consistency(self, verbose=0): rep = Repository(self.variables.keys(), self.variables, self.constraints) - rep.consistency() + rep.consistency(verbose) def get_list(self, subject): res = [] first = list(self.objects(subject, rdf_first)) assert len(first) == 1 - self.seen[self.make_var(fd, subject, p)]= 1 if type(first[0]) == URIRef: var = self.make_var(fd, first[0]) if var not in self.variables.keys(): @@ -248,7 +246,6 @@ res += first rest = list(self.objects(subject, rdf_rest)) - self.seen[self.make_var(fd, subject, p)]= 1 if "#nil" in rest[0] : return res else: @@ -311,16 +308,19 @@ #---------------- Implementation ---------------- def type(self, s, p, var): - svar = self.make_var(ClassDomain, s) - if (type(var) == URIRef and not - (var in [URIRef(namespaces['owl']+'#'+x) for x in builtin_voc])): + + #if (type(var) == URIRef and not + if not (var in [URIRef(namespaces['owl']+'#'+x) for x in builtin_voc]): # var is not one of the builtin classes + svar = self.make_var(ClassDomain, s) avar = self.make_var(ClassDomain, var) self.variables[svar].setValues(self.variables[avar].getValues()) constrain = BinaryExpression([svar, avar],"%s in %s" %(svar, avar)) self.constraints.append(constrain) - else: + else: # type(s) != BNode: # var is a builtin class + svar = self.make_var(ClassDomain, s) + print "===",var cls =builtin_voc[var.split('#')[-1]](name=svar) if hasattr(cls, 'constraint'): self.constraints.append(cls.constraint) @@ -431,7 +431,7 @@ # TODO: implement this pass -#---Label---#000000#FFFFFF------------------------------------------------------ +#---Property restrictions------------------------------------------------------ def maxCardinality(self, s, p, var): """ Len of finite domain of the property shall be less than or equal to var""" @@ -512,6 +512,9 @@ self.__cost = 1 self.cardinality = cardinality + def __repr__(self): + return '<%s %s %i>' % (self.__class__.__name__, str(self._variables[0]), self.cardinality) + def narrow(self, domains): """narrowing algorithm for the constraint""" if len(domains[self._variables[0]]) > self.cardinality: @@ -525,7 +528,7 @@ """narrowing algorithm for the constraint""" if len(domains[self._variables[0]]) < self.cardinality: - raise ConsistencyFailure() + raise ConsistencyFailure("MinCardinality not accomplished") else: return 1 @@ -535,7 +538,7 @@ """narrowing algorithm for the constraint""" if len(domains[self._variables[0]]) != self.cardinality: - raise ConsistencyFailure() + raise ConsistencyFailure("Cardinality constraint not met") else: return 1 @@ -558,6 +561,9 @@ self.super = cls_or_restriction self.variable = variable + def __repr__(self): + return '<%s %s %s>' % (self.__class__.__name__, str(self._variables[0]), self.super) + def narrow(self, domains): subdom = domains[self.variable] superdom = domains[self.super] @@ -566,12 +572,7 @@ vals = get_values(subdom, domains, 'getValues') superdom.values += [val for val in vals if val not in superdom.values] -class DisjointClassConstraint(OwlConstraint): - - def __init__(self, variable, cls_or_restriction): - OwlConstraint.__init__(self, [variable]) - self.super = cls_or_restriction - self.variable = variable +class DisjointClassConstraint(SubClassConstraint): def narrow(self, domains): subdom = domains[self.variable] @@ -584,23 +585,13 @@ if i in vals2: raise ConsistencyFailure() -class ComplementClassConstraint(OwlConstraint): - - def __init__(self, variable, cls_or_restriction): - OwlConstraint.__init__(self, variable) - self.super = cls_or_restriction - self.variable = variable +class ComplementClassConstraint(SubClassConstraint): def narrow(self, domains): subdom = domains[self.variable] superdom = domains[self.super] -class RangeConstraint(OwlConstraint): - - def __init__(self, variable, cls_or_restriction): - OwlConstraint.__init__(self, variable) - self.super = cls_or_restriction - self.variable = variable +class RangeConstraint(SubClassConstraint): def narrow(self, domains): subdom = domains[self.variable] @@ -614,12 +605,7 @@ res.append((k,v)) subdom.removeValues(res) -class DomainConstraint(OwlConstraint): - - def __init__(self, variable, cls_or_restriction): - OwlConstraint.__init__(self, variable) - self.super = cls_or_restriction - self.variable = variable +class DomainConstraint(SubClassConstraint): def narrow(self, domains): subdom = domains[self.variable] @@ -631,12 +617,7 @@ res.append((k,val)) subdom.removeValues(res) -class SubPropertyConstraint(OwlConstraint): - - def __init__(self, variable, cls_or_restriction): - OwlConstraint.__init__(self, variable) - self.super = cls_or_restriction - self.variable = variable +class SubPropertyConstraint(SubClassConstraint): def narrow(self, domains): subdom = domains[self.variable] @@ -647,12 +628,7 @@ vals.append(val) superdom.setValues(vals) -class EquivalentPropertyConstraint(OwlConstraint): - - def __init__(self, variable, cls_or_restriction): - OwlConstraint.__init__(self, variable) - self.super = cls_or_restriction - self.variable = variable +class EquivalentPropertyConstraint(SubClassConstraint): def narrow(self, domains): subdom = domains[self.variable] Modified: pypy/dist/pypy/lib/pyontology/test/test_ontology.py ============================================================================== --- pypy/dist/pypy/lib/pyontology/test/test_ontology.py (original) +++ pypy/dist/pypy/lib/pyontology/test/test_ontology.py Thu Jan 5 14:35:49 2006 @@ -123,7 +123,7 @@ O.type(sub, pred , obj) assert len(O.constraints) == 1 O.constraints[0].narrow(O.variables) - assert O.variables['a_'].getValues() == ((None,[1,2,3,4]),) + assert O.variables['a_'].getValues() == [(None,[1,2,3,4]),] def test_merge(): O = Ontology() @@ -140,7 +140,7 @@ O.type(sub, pred , obj) assert len(O.constraints) == 2 O.consistency() - assert O.variables['a_'].getValues() == ((None, [3,4]),) + assert O.variables['a_'].getValues() == [(None, [3,4]),] def test_domain(): O = Ontology() @@ -154,7 +154,7 @@ O.type(sub, pred , obj) assert len(O.constraints) == 1 O.constraints[0].narrow(O.variables) - assert O.variables['a_'].getValues() == ((O.variables['b_'], [None]),) + assert O.variables['a_'].getValues() == [(O.variables['b_'], [None]),] def test_domain_merge(): O = Ontology() @@ -172,7 +172,7 @@ assert len(O.constraints) == 2 for con in O.constraints: con.narrow(O.variables) - assert O.variables['a_'].getValues() ==() #O.variables['b_'] + assert O.variables['a_'].getValues() ==[] #O.variables['b_'] def test_subproperty(): O = Ontology() @@ -277,4 +277,30 @@ O.type(sub, None, obj) O.variables['friend_'].setValues([('Bob_','Alice_')]) O.consistency() - assert ('Alice_', ['Bob_']) in O.variables['friend_'].getValues() \ No newline at end of file + assert ('Alice_', ['Bob_']) in O.variables['friend_'].getValues() + +def no_test_maxcardinality(): + + O = Ontology() + #Make functional property + sub = URIRef('friend') + obj = URIRef(namespaces['owl']+'#ObjectProperty') + O.type(sub, None, obj) + #Make class + sub = URIRef('c') + obj = URIRef(namespaces['owl']+'#Class') + O.type(sub, None, obj) + #Make individual with a cardinality restriction of the property + sub = URIRef('Bob') + obj = URIRef('c') + O.type(sub, None, obj) + sub = BNode('anon') + obj = URIRef(namespaces['owl']+'#Restriction') + O.type(sub, None, obj) + O.onProperty(sub, None, URIRef('friend')) + O.maxCardinality(sub, None, 1) + O.variables['friend_'].setValues([('Bob_',['Alice_','Finn_'])]) + O.consistency(verbose=5) + print O.variables + print O.constraints + assert not '_anon' in O.variables \ No newline at end of file From pedronis at codespeak.net Thu Jan 5 22:00:45 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Thu, 5 Jan 2006 22:00:45 +0100 (CET) Subject: [pypy-svn] r21725 - in pypy/dist/pypy/translator: . test Message-ID: <20060105210045.85D5E27B46@code1.codespeak.net> Author: pedronis Date: Thu Jan 5 22:00:44 2006 New Revision: 21725 Modified: pypy/dist/pypy/translator/interactive.py pypy/dist/pypy/translator/test/test_interactive.py Log: some progress on interactive.py Modified: pypy/dist/pypy/translator/interactive.py ============================================================================== --- pypy/dist/pypy/translator/interactive.py (original) +++ pypy/dist/pypy/translator/interactive.py Thu Jan 5 22:00:44 2006 @@ -41,13 +41,17 @@ self.update_options(argtypes, kwds) + FREEZE = { + 'annotate': ['debug'], + 'rtype': ['insist'], + } + def driver_event(self, kind, goal, func): if kind == 'pre': print goal self.ensure_setup() elif kind == 'post': - if 'goal' == 'annotate': # xxx use a table instead - self.frozen_options['debug'] = True + self.frozen_options.update(dict.fromkeys(self.FREEZE[goal], True)) def ensure_setup(self, argtypes=None, policy=None): if not self.driver_setup: @@ -60,9 +64,9 @@ else: # check consistency if argtypes is not None and argtypes != self.ann_argtypes: - raise Exception("xxx") + raise Exception("incosistent argtype supplied") if policy is not None and policy != self.ann_policy: - raise Exception("xxx") + raise Exception("incosistent annotation polish supplied") def update_options(self, argtypes, kwds): if argtypes or kwds.get('policy'): @@ -70,15 +74,23 @@ for optname, value in kwds: if optname in self.frozen_options: if getattr(self.driver.options, optname) != value: - raise Exception("xxx") + raise Exception("incosistent option supplied: %s" % optname) else: setattr(self.driver.options, optname, value) self.frozen_options[optname] = True + # backend independent + def annotate(self, argtypes=None, **kwds): self.update_options(argtypes, kwds) return self.driver.annotate() + def rtype(self, argtypes=None, **kwds): + self.update_options(argtypes, kwds) + return self.driver.rtype() + + # backend depedent xxx + def source(self, argtypes, **kwds): backend = self.ensure_backend() self.update_options(argtypes, kwds) Modified: pypy/dist/pypy/translator/test/test_interactive.py ============================================================================== --- pypy/dist/pypy/translator/test/test_interactive.py (original) +++ pypy/dist/pypy/translator/test/test_interactive.py Thu Jan 5 22:00:44 2006 @@ -10,6 +10,10 @@ s = t.annotate([int, int]) assert s.knowntype == int + t = Translation(f, [int, int]) + s = t.annotate() + assert s.knowntype == int + t = Translation(f) s = t.annotate([int, int]) assert s.knowntype == int @@ -18,3 +22,19 @@ py.test.raises(Exception, "t.annotate([int, float])") +def test_simple_rtype(): + + def f(x,y): + return x+y + + t = Translation(f, [int, int]) + s = t.annotate() + t.rtype() + + t = Translation(f) + s = t.annotate([int, int]) + t.rtype() + + t = Translation(f, [int, int]) + t.annotate() + py.test.raises(Exception, "t.rtype([int, int],debug=False)") From pedronis at codespeak.net Thu Jan 5 22:04:11 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Thu, 5 Jan 2006 22:04:11 +0100 (CET) Subject: [pypy-svn] r21726 - pypy/dist/pypy/translator Message-ID: <20060105210411.37D8B27B5A@code1.codespeak.net> Author: pedronis Date: Thu Jan 5 22:04:10 2006 New Revision: 21726 Modified: pypy/dist/pypy/translator/interactive.py Log: clearer naming Modified: pypy/dist/pypy/translator/interactive.py ============================================================================== --- pypy/dist/pypy/translator/interactive.py (original) +++ pypy/dist/pypy/translator/interactive.py Thu Jan 5 22:04:10 2006 @@ -41,7 +41,7 @@ self.update_options(argtypes, kwds) - FREEZE = { + GOAL_USES_OPTS = { 'annotate': ['debug'], 'rtype': ['insist'], } @@ -51,7 +51,8 @@ print goal self.ensure_setup() elif kind == 'post': - self.frozen_options.update(dict.fromkeys(self.FREEZE[goal], True)) + used_opts = dict.fromkeys(self.GOAL_USES_OPTS[goal], True) + self.frozen_options.update(used_opts) def ensure_setup(self, argtypes=None, policy=None): if not self.driver_setup: From cfbolz at codespeak.net Thu Jan 5 23:48:48 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Thu, 5 Jan 2006 23:48:48 +0100 (CET) Subject: [pypy-svn] r21730 - pypy/dist/pypy/doc Message-ID: <20060105224848.D99D927B46@code1.codespeak.net> Author: cfbolz Date: Thu Jan 5 23:48:47 2006 New Revision: 21730 Added: pypy/dist/pypy/doc/index-report.txt Modified: pypy/dist/pypy/doc/index.txt Log: index page for eu-reports Added: pypy/dist/pypy/doc/index-report.txt ============================================================================== --- (empty file) +++ pypy/dist/pypy/doc/index-report.txt Thu Jan 5 23:48:47 2006 @@ -0,0 +1,48 @@ +============================================ +PyPy - Overview over the EU-reports +============================================ + +We have written several reports for the EU of which some are also quite useful +as documentation. The ones not so useful as documentation mostly contain text +that can also be found in the regular documentation section. All these +documents are not approved by the European Union and therefore only +preliminary. + +`D04.1 Partial Python Implementation`_ contains details about the 0.6 release. +All the content can be found in the regular documentation section. + +`D04.2 Complete Python Implementation`_ contains details about the 0.7 release. +All the content can be found in the regular documentation section. + +`D04.3 Parser and Bytecode Compiler`_ describes our parser and bytecode compiler. + +`D04.4 PyPy as a Research Tool`_ contains details about the 0.8 release. +All the content can be found in the regular documentation section. + +`D05.1 Compiling Dynamic Language Implementations`_ is a paper that describes +the translation process, especially the flow object space and the annotator in +detail. + +`D05.2 A Compiled Version of PyPy`_ contains more details about the 0.7 release. +All the content can be found in the regular documentation section. + +`D05.3 Implementation with Translation Aspects`_ +describes how our approach hides away a lot of low level details. + +`D05.4 Encapsulating Low Level Aspects`_ describes how we weave different +properties into our interpreter during the translation process. + +`D14.1 Report about Milestone/Phase 1`_ describes what happened in the PyPy +project during the first year of EU funding (December 2004 - December 2005) + + +.. _`D04.1 Partial Python Implementation`: http://codespeak.net/~cfbolz/eu-reports/D04.1_Partial_Python_Implementation_on_top_of_CPython.pdf +.. _`D04.2 Complete Python Implementation`: http://codespeak.net/~cfbolz/eu-reports/D04.2_Complete_Python_Implementation_on_top_of_CPython.pdf +.. _`D04.3 Parser and Bytecode Compiler`: http://codespeak.net/~cfbolz/eu-reports/D04.3_Report_about_the_parser_and_bytecode_compiler.pdf +.. _`D04.4 PyPy as a Research Tool`: http://codespeak.net/~cfbolz/eu-reports/D04.4_Release_PyPy_as_a_research_tool.pdf +.. _`D05.1 Compiling Dynamic Language Implementations`: http://codespeak.net/~cfbolz/eu-reports/D05.1_Publish_on_translating_a_very-high-level_description.pdf +.. _`D05.2 A Compiled Version of PyPy`: http://codespeak.net/~cfbolz/eu-reports/D05.2_A_compiled,_self-contained_version_of_PyPy.pdf +.. _`D05.3 Implementation with Translation Aspects`: http://codespeak.net/~cfbolz/eu-reports/D05.3_Publish_on_implementation_with_translation_aspects.pdf +.. _`D05.4 Encapsulating Low Level Aspects`: http://codespeak.net/~cfbolz/eu-reports/D05.4_Publish_on_encapsulating_low_level_language_aspects.pdf +.. _`D14.1 Report about Milestone/Phase 1`: http://codespeak.net/~cfbolz/eu-reports/D14.1_Report_about_Milestone_Phase_1.pdf + Modified: pypy/dist/pypy/doc/index.txt ============================================================================== --- pypy/dist/pypy/doc/index.txt (original) +++ pypy/dist/pypy/doc/index.txt Thu Jan 5 23:48:47 2006 @@ -22,15 +22,21 @@ `translation`_ offers the beginnings of documentation about our low level code generator backends. +`EU reports`_ is a page that contains links to the +preliminary reports that we submitted to the European Union. + `dynamic-language translation`_ is a paper that describes the translation process, especially the flow object space -and the annotator in detail. +and the annotator in detail. This document is also part +of the `EU reports`_. `low-level encapsulation`_ describes how our approach hides -away a lot of low level details. +away a lot of low level details. This document is also part +of the `EU reports`_. `translation aspects`_ describes how we weave different -properties into our interpreter during the translation process. +properties into our interpreter during the translation +process. This document is also part of the `EU reports`_. `parser`_ contains the beginnings of documentation about the parser (and the compiler at some point). @@ -69,6 +75,7 @@ .. _`getting started`: getting-started.html .. _`theory`: theory.html .. _`bytecode interpreter`: interpreter.html +.. _`EU reports`: index-report.html .. _`garbage collection`: garbage_collection.html .. _`directory reference`: From cfbolz at codespeak.net Fri Jan 6 14:11:17 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Fri, 6 Jan 2006 14:11:17 +0100 (CET) Subject: [pypy-svn] r21737 - pypy/dist/pypy/doc Message-ID: <20060106131117.A8C2627B55@code1.codespeak.net> Author: cfbolz Date: Fri Jan 6 14:11:14 2006 New Revision: 21737 Modified: pypy/dist/pypy/doc/news.txt Log: news item about reports Modified: pypy/dist/pypy/doc/news.txt ============================================================================== --- pypy/dist/pypy/doc/news.txt (original) +++ pypy/dist/pypy/doc/news.txt Fri Jan 6 14:11:14 2006 @@ -9,6 +9,17 @@ .. _Python: http://www.python.org/doc/current/ref/ref.html .. _`more...`: http://codespeak.net/pypy/dist/pypy/doc/architecture.html#mission-statement +Preliminary EU reports released +=============================== + +After many hours of writing and typo-hunting we finally finished the +`EU-reports`_. They contain most of the material found on our regular +documentation page but also a lot of new material not covered there. Note that +all these documents are not approved by the European Union and therefore only +preliminary. *(01/06/2006)* + +.. _`EU-reports`: http://codespeak.net/pypy/dist/pypy/doc/index-report.html + Next PyPy Sprint in Palma De Mallorca 23rd - 29th January 2006 =============================================================== From cfbolz at codespeak.net Fri Jan 6 14:24:14 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Fri, 6 Jan 2006 14:24:14 +0100 (CET) Subject: [pypy-svn] r21738 - pypy/dist/pypy/doc Message-ID: <20060106132414.42B4227B55@code1.codespeak.net> Author: cfbolz Date: Fri Jan 6 14:24:12 2006 New Revision: 21738 Modified: pypy/dist/pypy/doc/news.txt Log: some reformulation (thanks michael) Modified: pypy/dist/pypy/doc/news.txt ============================================================================== --- pypy/dist/pypy/doc/news.txt (original) +++ pypy/dist/pypy/doc/news.txt Fri Jan 6 14:24:12 2006 @@ -13,12 +13,12 @@ =============================== After many hours of writing and typo-hunting we finally finished the -`EU-reports`_. They contain most of the material found on our regular +`reports for the EU`_. They contain most of the material found on our regular documentation page but also a lot of new material not covered there. Note that all these documents are not approved by the European Union and therefore only preliminary. *(01/06/2006)* -.. _`EU-reports`: http://codespeak.net/pypy/dist/pypy/doc/index-report.html +.. _`reports for the EU`: http://codespeak.net/pypy/dist/pypy/doc/index-report.html Next PyPy Sprint in Palma De Mallorca 23rd - 29th January 2006 =============================================================== From ale at codespeak.net Fri Jan 6 15:48:58 2006 From: ale at codespeak.net (ale at codespeak.net) Date: Fri, 6 Jan 2006 15:48:58 +0100 (CET) Subject: [pypy-svn] r21739 - in pypy/dist/pypy/lib/pyontology: . test Message-ID: <20060106144858.AB0D827B55@code1.codespeak.net> Author: ale Date: Fri Jan 6 15:48:57 2006 New Revision: 21739 Modified: pypy/dist/pypy/lib/pyontology/pyontology.py pypy/dist/pypy/lib/pyontology/test/test_ontology.py Log: Lots of small changes and refactorings, I probably should commit more often Modified: pypy/dist/pypy/lib/pyontology/pyontology.py ============================================================================== --- pypy/dist/pypy/lib/pyontology/pyontology.py (original) +++ pypy/dist/pypy/lib/pyontology/pyontology.py Fri Jan 6 15:48:57 2006 @@ -1,4 +1,4 @@ -from rdflib import Graph, URIRef, BNode +from rdflib import Graph, URIRef, BNode, Literal from logilab.constraint import Repository, Solver from logilab.constraint.fd import Equals, AllDistinct, BinaryExpression, Expression from logilab.constraint.fd import FiniteDomain as fd @@ -71,23 +71,25 @@ class Property(ClassDomain): # Property contains the relationship between a class instance and a value - # - a pair. To accomodate global assertions like 'range' and 'domain' anonymous - # pairs are allowed, ie None as key + # - a pair. To accomodate global assertions like 'range' and 'domain' attributes + # for range and domain must be filled in by rdfs:range and rdfs:domain + def __init__(self, name='', values=[], bases = []): ClassDomain.__init__(self, name, values, bases) - self._dict = {} + self._dict = Linkeddict() + self.range = [] + self.domain = [] def getValues(self): - return self._dict.items() + items = self._dict.items() + res = [] + for k,v in items: + for i in v: + res.append((k,i)) + return res def setValues(self, values): - for k,v in values: - self._dict.setdefault(k,[]) - if type(v) == list: - self._dict[k] = v - else: - if not v in self._dict[k]: - self._dict[k].append(v) + self._dict = Linkeddict(values) def removeValues(self, values): for k,v in values: @@ -124,13 +126,13 @@ def __init__(self, name='', values=[], bases = []): Property.__init__(self, name, values, bases) - self.constraint = FunctionalCardinality(name, 1) + self.constraint = FunctionalCardinality(name) class InverseFunctionalProperty(Property): def __init__(self, name='', values=[], bases = []): Property.__init__(self, name, values, bases) - self.constraint = InverseFunctionalCardinality(name, 1) + self.constraint = InverseFunctionalCardinality(name) class TransitiveProperty(Property): @@ -192,25 +194,22 @@ ns ='' func = p if ns in namespaces.values(): + #predicate is one of builtin OWL predicates pred = getattr(self, func) res = pred(s, p, o) if not res : continue if type(res) != list : res = [res] - avar = self.make_var(fd, s) + avar = self.make_var(ClassDomain, s) else: res = [o] avar = self.make_var(Property, p) # Set the values of the property p to o - sub = self.make_var(fd, s) - obj = self.make_var(fd, o) + sub = self.make_var(ClassDomain, s) + obj = self.make_var(Thing, o) res = self.variables[avar].getValues() - self.variables[avar].setValues(res + ((sub, obj),)) - if self.variables.get(avar) and type(self.variables[avar]) == fd: - self.variables[avar] = fd(list(self.variables[avar].getValues()) + res) - else: - self.variables[avar] = fd(res) + self.variables[avar].setValues(res + [(sub, obj)]) def merge_constraints(self): # Make the intersection of multiple rdfs:range constraints on the same variable @@ -240,9 +239,7 @@ first = list(self.objects(subject, rdf_first)) assert len(first) == 1 if type(first[0]) == URIRef: - var = self.make_var(fd, first[0]) - if var not in self.variables.keys(): - self.variables[var] = ClassDomain(var) + var = self.make_var(ClassDomain, first[0]) res += first rest = list(self.objects(subject, rdf_rest)) @@ -252,24 +249,25 @@ res += self.get_list(rest[0]) return res - def make_var(self, cls=fd, *args): - res = [] - for a in args: - if type(a) == URIRef: - if a.find('#') != -1: - ns,name = a.split('#') - else: - ns,name = a,'' - if ns not in uris.keys(): - uris[ns] = ns.split('/')[-1] - namespaces[uris[ns]] = ns - mangle_name = uris[ns] + '_' + name - res.append(mangle_name) + def make_var(self, cls=fd, a=''): + if type(a) == URIRef: + if a.find('#') != -1: + ns,name = a.split('#') else: - res.append(a) - var = '.'.join([str(a.replace('-','_')) for a in res]) + ns,name = a,'' + if ns not in uris.keys(): + uris[ns] = ns.split('/')[-1] + a = uris[ns] + '_' + name + var = str(a.replace('-','_')) + if not cls: + return var if not var in self.variables.keys(): self.variables[var] = cls(var) + elif type(self.variables[var]) != cls: + olddom = self.variables[var] + newdom = cls(var) + newdom.setValues(olddom.getValues()) + self.variables[var] = newdom return var def find_prop(self, s): @@ -284,7 +282,7 @@ if type(s) == BNode: pr = list( self.subjects(p,s) ) if len(pr) == 0: - return + pr = list( self.subjects(r,s) ) return pr[0] else: return s @@ -296,37 +294,25 @@ def find_property(self, s): prop = self.find_prop(s) - cls = self.find_cls(s) - if cls : - avar = self.make_var(ClassDomain, cls, prop) - else: - avar = self.make_var(ClassDomain, prop) - if not self.variables.get(avar): - self.variables[avar] = ClassDomain(avar) + avar = self.make_var(Property, prop) return avar #---------------- Implementation ---------------- def type(self, s, p, var): - - #if (type(var) == URIRef and not - if not (var in [URIRef(namespaces['owl']+'#'+x) for x in builtin_voc]): + if not (var in [URIRef(namespaces[ns]+'#'+x) for x in builtin_voc for ns in namespaces]): # var is not one of the builtin classes - svar = self.make_var(ClassDomain, s) avar = self.make_var(ClassDomain, var) - self.variables[svar].setValues(self.variables[avar].getValues()) + svar = self.make_var(self.variables[avar].__class__, s) constrain = BinaryExpression([svar, avar],"%s in %s" %(svar, avar)) self.constraints.append(constrain) - else: # type(s) != BNode: + else: # var is a builtin class - svar = self.make_var(ClassDomain, s) - print "===",var - cls =builtin_voc[var.split('#')[-1]](name=svar) + cls =builtin_voc[var.split('#')[-1]] + svar = self.make_var(cls, s) + cls = self.variables[svar] if hasattr(cls, 'constraint'): self.constraints.append(cls.constraint) - vals = self.variables[svar].getValues() - cls.setValues(vals) - self.variables[svar] = cls def first(self, s, p, var): pass @@ -334,14 +320,17 @@ def rest(self, s, p, var): pass + def onProperty(self, s, p, var): + pass + #---Class Axioms---#000000#FFFFFF----------------------------------------------- def subClassOf(self, s, p, var): # s is a subclass of var means that the # class extension of s is a subset of the # class extension of var. - avar = self.make_var(ClassDomain, var) - svar = self.make_var(ClassDomain, s) + avar = self.make_var(None, var) + svar = self.make_var(None, s) cons = SubClassConstraint( svar, avar) self.constraints.append(cons) @@ -350,21 +339,22 @@ self.subClassOf(var, p, s) def disjointWith(self, s, p, var): - avar = self.make_var(ClassDomain, var) - svar = self.make_var(ClassDomain, s) + avar = self.make_var(None, var) + svar = self.make_var(None, s) constrain = DisjointClassConstraint(svar, avar) self.constraints.append(constrain) def complementOf(self, s, p, var): # add constraint of not var # TODO: implementthis for OWL DL - avar = self.make_var(ClassDomain, var) - svar = self.make_var(ClassDomain, s) +## avar = self.make_var(ClassDomain, var) +## svar = self.make_var(ClassDomain, s) + pass def oneOf(self, s, p, var): res = self.get_list(var) - prop = self.find_uriref(s) - avar = self.make_var(fd, prop) + #prop = self.find_uriref(s) + avar = self.make_var(None, s) if self.variables.get(avar) and type(self.variables[avar]) == fd: self.variables[avar] = fd(list(self.variables[avar].getValues()) + res) else: @@ -388,13 +378,6 @@ def range(self, s, p, var): avar = self.make_var(ClassDomain, var) svar = self.make_var(Property, s) - vals = get_values(self.variables[avar], self.variables) - for k,v in self.variables[svar].getValues(): - for x in v: - if not x in vals: - vals.append(x) - vals =[(None,val) for val in vals] - self.variables[svar].setValues(vals) cons = RangeConstraint(svar, avar) self.constraints.append(cons) @@ -424,37 +407,42 @@ def equivalentProperty(self, s, p, var): avar = self.make_var(Property, var) svar = self.make_var(Property, s) - cons = EquivalentConstraint( svar, avar) + cons = EquivalentPropertyConstraint( svar, avar) self.constraints.append(cons) def inverseOf(self, s, p, var): - # TODO: implement this - pass + avar = self.make_var(Property, var) + svar = self.make_var(Property, s) + con = InverseofConstraint(svar, avar) + self.constraints.append(con) #---Property restrictions------------------------------------------------------ def maxCardinality(self, s, p, var): """ Len of finite domain of the property shall be less than or equal to var""" avar = self.find_property(s) - constrain = MaxCardinality(avar,int(var)) + cls =self.make_var(None, self.find_cls(s)) + constrain = MaxCardinality(avar, cls, int(var)) self.constraints.append(constrain) def minCardinality(self, s, p, var): """ Len of finite domain of the property shall be greater than or equal to var""" avar = self.find_property(s) - constrain = MinCardinality(avar,int(var)) + cls =self.make_var(None, self.find_cls(s)) + constrain = MinCardinality(avar, cls, int(var)) self.constraints.append(constrain) def cardinality(self, s, p, var): """ Len of finite domain of the property shall be equal to var""" avar = self.find_property(s) + cls =self.make_var(None, self.find_cls(s)) # Check if var is an int, else find the int buried in the structure - constrain = Cardinality(avar,int(var)) + constrain = Cardinality(avar, cls, int(var)) self.constraints.append(constrain) def differentFrom(self, s, p, var): s_var = self.make_var(ClassDomain, s) - var_var = self.make_var(fd, var) + var_var = self.make_var(Thing, var) constrain = BinaryExpression([s_var, var_var],"%s != %s" %(s_var, var_var)) self.constraints.append(constrain) @@ -464,14 +452,12 @@ return res def sameAs(self, s, p, var): - constrain = BinaryExpression([self.make_var(ClassDomain, s), self.make_var(ClassDomain, var)], - "%s == %s" %(self.make_var(ClassDomain, s), self.make_var(ClassDomain, var))) + s_var = self.make_var(None, s) + var_var = self.make_var(None, var) + constrain = BinaryExpression([s_var, var_var], + "%s == %s" %(s_var, var_var)) self.constraints.append(constrain) - def onProperty(self, s, p, var): - # TODO: implement this - pass - def hasValue(self, s, p, var): # TODO: implement this pass @@ -485,7 +471,7 @@ pass def imports(self, s, p, var): - # TODO: implement this + # PP TODO: implement this pass # ----------------- Helper classes ---------------- @@ -507,17 +493,19 @@ class MaxCardinality(OwlConstraint): """Contraint: all values must be distinct""" - def __init__(self, variable, cardinality): + def __init__(self, variable, cls, cardinality): OwlConstraint.__init__(self, variable) self.__cost = 1 self.cardinality = cardinality + self.cls = cls def __repr__(self): return '<%s %s %i>' % (self.__class__.__name__, str(self._variables[0]), self.cardinality) def narrow(self, domains): """narrowing algorithm for the constraint""" - if len(domains[self._variables[0]]) > self.cardinality: + prop = Linkeddict(domains[self.variable].getValues()) + if len(prop[self.cls]) > self.cardinality: raise ConsistencyFailure("Maxcardinality exceeded") else: return 1 @@ -526,8 +514,8 @@ def narrow(self, domains): """narrowing algorithm for the constraint""" - - if len(domains[self._variables[0]]) < self.cardinality: + prop = Linkeddict(domains[self.variable].getValues()) + if len(prop[self.cls]) < self.cardinality: raise ConsistencyFailure("MinCardinality not accomplished") else: return 1 @@ -536,8 +524,8 @@ def narrow(self, domains): """narrowing algorithm for the constraint""" - - if len(domains[self._variables[0]]) != self.cardinality: + prop = Linkeddict(domains[self.variable].getValues()) + if len(prop[self.cls]) != self.cardinality: raise ConsistencyFailure("Cardinality constraint not met") else: return 1 @@ -545,12 +533,14 @@ def get_values(dom, domains, attr = 'getValues'): res = [] + if type(dom) == Literal: + return [dom] for val in getattr(dom, attr)(): res.append(val) if type(val) == tuple: val = val[0] if val in domains.keys(): - res.extend(get_values(val, domains, attr)) + res.extend(get_values(domains[val], domains, attr)) #res[dom] = 1 return res @@ -558,15 +548,15 @@ def __init__(self, variable, cls_or_restriction): OwlConstraint.__init__(self, variable) - self.super = cls_or_restriction + self.object = cls_or_restriction self.variable = variable def __repr__(self): - return '<%s %s %s>' % (self.__class__.__name__, str(self._variables[0]), self.super) + return '<%s %s %s>' % (self.__class__.__name__, str(self._variables[0]), self.object) def narrow(self, domains): subdom = domains[self.variable] - superdom = domains[self.super] + superdom = domains[self.object] bases = get_values(superdom, domains, 'getBases') subdom.bases += [bas for bas in bases if bas not in subdom.bases] vals = get_values(subdom, domains, 'getValues') @@ -576,11 +566,11 @@ def narrow(self, domains): subdom = domains[self.variable] - superdom = domains[self.super] + superdom = domains[self.object] bases = get_values(superdom, domains, 'getBases') subdom.bases += [bas for bas in bases if bas not in subdom.bases] vals1 = get_values(superdom, domains, 'getValues') - vals2 = get_values(variable, domains, 'getValues') + vals2 = get_values(subdom, domains, 'getValues') for i in vals1: if i in vals2: raise ConsistencyFailure() @@ -589,39 +579,46 @@ def narrow(self, domains): subdom = domains[self.variable] - superdom = domains[self.super] + superdom = domains[self.object] class RangeConstraint(SubClassConstraint): def narrow(self, domains): - subdom = domains[self.variable] - superdom = domains[self.super] - vals = get_values(superdom, domains, 'getValues') - res = [] - svals = get_values(subdom, domains, 'getValues') - for k,val in svals: - for v in val: - if not v in vals: - res.append((k,v)) - subdom.removeValues(res) + propdom = domains[self.variable] + rangedom = domains[self.object] + newrange = get_values(rangedom, domains, 'getValues') + range = [] + prop = Linkeddict(propdom.getValues()) + oldrange = propdom.range#get(None) + if oldrange: + for v in oldrange: + if v in newrange: + range.append(v) + else: + range = newrange + propdom.range = range +# propdom.setValues(prop.items()) + for pval in sum(prop.values(),[]): + if pval not in range: + raise ConsistencyFailure("Value %r not in range %r"%(pval,range)) class DomainConstraint(SubClassConstraint): def narrow(self, domains): - subdom = domains[self.variable] - superdom = domains[self.super] - vals = get_values(superdom, domains, 'getValues') + propdom = domains[self.variable] + domaindom = domains[self.object] + vals = get_values(domaindom, domains, 'getValues') res = [] - for k,val in get_values(subdom, domains, 'getValues'): - if not k in vals and k != superdom: + for k,val in get_values(propdom, domains, 'getValues'): + if not k in vals and k != domaindom: res.append((k,val)) - subdom.removeValues(res) + propdom.removeValues(res) class SubPropertyConstraint(SubClassConstraint): def narrow(self, domains): subdom = domains[self.variable] - superdom = domains[self.super] + superdom = domains[self.object] vals = get_values(superdom, domains, 'getValues') for val in subdom.getValues(): if not val in vals: @@ -632,25 +629,26 @@ def narrow(self, domains): subdom = domains[self.variable] - superdom = domains[self.super] + superdom = domains[self.object] vals = get_values(superdom, domains, 'getValues') for val in subdom.getValues(): if not val in vals: raise ConsistencyFailure("Value not in prescribed range") -class FunctionalCardinality(MaxCardinality): +class FunctionalCardinality(OwlConstraint): """Contraint: all values must be distinct""" def narrow(self, domains): """narrowing algorithm for the constraint""" domain = domains[self.variable].getValues() - for cls, val in domain: - if len(val) != self.cardinality: + domain_dict = Linkeddict(domain) + for cls, val in domain_dict.items(): + if len(val) != 1: raise ConsistencyFailure("Maxcardinality exceeded") else: return 0 -class InverseFunctionalCardinality(MaxCardinality): +class InverseFunctionalCardinality(OwlConstraint): """Contraint: all values must be distinct""" def narrow(self, domains): @@ -658,27 +656,36 @@ domain = domains[self.variable].getValues() vals = {} for cls, val in domain: - for v in val: - if vals.has_key(v): - raise ConsistencyFailure("Maxcardinality exceeded") - else: - vals[v] = 1 + if vals.has_key(val): + raise ConsistencyFailure("Maxcardinality exceeded") + else: + vals[val] = 1 else: return 0 +class Linkeddict(dict): + def __init__(self, values=()): + for k,v in values: + dict.setdefault(self,k,[]) + if type(v) == list: + dict.__setitem__(self, k, v) + else: + if not v in dict.__getitem__(self,k): + dict.__getitem__(self,k).append(v) +## dict.__getitem__(self,k).append(v) + class TransitiveConstraint(OwlConstraint): """Contraint: all values must be distinct""" def narrow(self, domains): """narrowing algorithm for the constraint""" domain = domains[self.variable].getValues() - domain_dict = dict( domain ) + domain_dict = Linkeddict(domain) for cls, val in domain: - for v in val: - if v in domain_dict: - val.extend(domain_dict[v]) - domain_dict[cls] = val - domains[self.variable].setValues(domain_dict.items()) + if val in domain_dict: + for v in domain_dict[val]: + domain.append((cls,v)) + domains[self.variable].setValues(domain) class SymmetricConstraint(OwlConstraint): """Contraint: all values must be distinct""" @@ -686,9 +693,24 @@ def narrow(self, domains): """narrowing algorithm for the constraint""" domain = domains[self.variable].getValues() - domain_dict = dict( domain ) for cls, val in domain: - for v in val: - domain_dict.setdefault(v, []) - domain_dict[v].append(cls) - domains[self.variable].setValues(domain_dict.items()) + if not (val, cls) in domain: + domain.append((val,cls)) + domains[self.variable].setValues(domain) + +class InverseofConstraint(SubClassConstraint): + """Contraint: all values must be distinct""" + + def narrow(self, domains): + """narrowing algorithm for the constraint""" + obj_domain = domains[self.object].getValues() + sub_domain = domains[self.variable].getValues() + res = [] + for cls, val in obj_domain: + if not (val,cls) in sub_domain: + raise ConsistencyFailure("Inverseof failed") + for cls, val in sub_domain: + if not (val,cls) in obj_domain: + raise ConsistencyFailure("Inverseof failed") +## res.append((val, cls)) +## domains[self.variable].setValues(res) Modified: pypy/dist/pypy/lib/pyontology/test/test_ontology.py ============================================================================== --- pypy/dist/pypy/lib/pyontology/test/test_ontology.py (original) +++ pypy/dist/pypy/lib/pyontology/test/test_ontology.py Fri Jan 6 15:48:57 2006 @@ -17,7 +17,7 @@ name = O.make_var(ClassDomain, var) cod = name+' = 1' exec cod - assert O.make_var(ClassDomain, var) in locals() + assert O.make_var(None, var) in locals() assert isinstance(O.variables[name], ClassDomain) def test_subClassof(): @@ -25,10 +25,11 @@ a = URIRef(u'http://www.w3.org/2002/03owlt/unionOf/premises004#A') b = URIRef(u'http://www.w3.org/2002/03owlt/unionOf/premises004#B') c = URIRef(u'http://www.w3.org/2002/03owlt/unionOf/premises004#C') - O.subClassOf(b, None, a) - O.subClassOf(c, None, b) A = O.make_var(ClassDomain, a) C = O.make_var(ClassDomain, c) + C = O.make_var(ClassDomain, b) + O.subClassOf(b, None, a) + O.subClassOf(c, None, b) for con in O.constraints: con.narrow(O.variables) assert len(O.variables) == 3 @@ -100,8 +101,10 @@ pred = URIRef('type') obj = URIRef('o') O = Ontology() + O.make_var(ClassDomain, obj) O.type(sub, pred , obj) - assert O.variables[O.make_var(ClassDomain, sub)].__class__ == ClassDomain + + assert O.variables[O.make_var(None, sub)].__class__ == ClassDomain def test_ObjectProperty(): sub = URIRef('a') @@ -109,7 +112,7 @@ obj = URIRef(namespaces['owl']+'#ObjectProperty') O = Ontology() O.type(sub, pred , obj) - assert O.variables[O.make_var(ClassDomain, sub)].__class__ == ObjectProperty + assert O.variables[O.make_var(None, sub)].__class__ == ObjectProperty def test_range(): O = Ontology() @@ -123,7 +126,7 @@ O.type(sub, pred , obj) assert len(O.constraints) == 1 O.constraints[0].narrow(O.variables) - assert O.variables['a_'].getValues() == [(None,[1,2,3,4]),] + assert O.variables['a_'].range == [1,2,3,4] def test_merge(): O = Ontology() @@ -140,7 +143,7 @@ O.type(sub, pred , obj) assert len(O.constraints) == 2 O.consistency() - assert O.variables['a_'].getValues() == [(None, [3,4]),] + assert O.variables['a_'].range == [ 3,4] def test_domain(): O = Ontology() @@ -154,7 +157,7 @@ O.type(sub, pred , obj) assert len(O.constraints) == 1 O.constraints[0].narrow(O.variables) - assert O.variables['a_'].getValues() == [(O.variables['b_'], [None]),] + assert O.variables['a_'].getValues() == [(O.variables['b_'], None),] def test_domain_merge(): O = Ontology() @@ -165,9 +168,8 @@ obj = URIRef('c') O.variables['c_'] = ClassDomain('c') O.domain(sub, None , obj) - pred = URIRef('type') obj = URIRef(namespaces['owl']+'#ObjectProperty') - O.type(sub, pred , obj) + O.type(sub, None , obj) assert len(O.constraints) == 2 for con in O.constraints: @@ -204,7 +206,7 @@ obj = URIRef('c') O.type(sub, None, obj) O.variables['p_'].setValues([('individ_',42)]) - assert len(O.constraints) == 2 + #assert len(O.constraints) == 2 #add another valueof the property O.variables['p_'].setValues([('individ_',42),('individ_',43)]) #check that consistency raises @@ -226,12 +228,12 @@ obj = URIRef('c') O.type(sub, None, obj) O.variables['p_'].setValues([('individ_',42)]) - assert len(O.constraints) == 2 + #assert len(O.constraints) == 2 #add another individual with the same value for the property sub = URIRef('individ2') obj = URIRef('c') O.type(sub, None, obj) - O.variables['p_'].setValues([('individ2_',42)]) + O.variables['p_'].setValues([('individ_',42),('individ2_',42)]) #check that consistency raises py.test.raises(ConsistencyFailure, O.consistency) @@ -256,7 +258,7 @@ O.type(sub, None, obj) O.variables['subRegionOf_'].setValues([('Italy_','Tuscanny_'),('Tuscanny_','Chianti_')]) O.consistency() - assert ('Italy_', ['Tuscanny_', 'Chianti_']) in O.variables['subRegionOf_'].getValues() + assert ('Italy_', 'Chianti_') in O.variables['subRegionOf_'].getValues() def test_symmetricproperty(): @@ -265,6 +267,7 @@ sub = URIRef('friend') obj = URIRef(namespaces['owl']+'#SymmetricProperty') O.type(sub, None, obj) + assert O.variables[O.make_var(None, sub)].__class__.__name__=='SymmetricProperty' #Make class sub = URIRef('c') obj = URIRef(namespaces['owl']+'#Class') @@ -276,8 +279,9 @@ sub = URIRef('Alice') O.type(sub, None, obj) O.variables['friend_'].setValues([('Bob_','Alice_')]) + print O.variables['friend_']._dict O.consistency() - assert ('Alice_', ['Bob_']) in O.variables['friend_'].getValues() + assert ('Alice_', 'Bob_') in O.variables['friend_'].getValues() def no_test_maxcardinality(): @@ -301,6 +305,4 @@ O.maxCardinality(sub, None, 1) O.variables['friend_'].setValues([('Bob_',['Alice_','Finn_'])]) O.consistency(verbose=5) - print O.variables - print O.constraints assert not '_anon' in O.variables \ No newline at end of file From ale at codespeak.net Fri Jan 6 18:49:41 2006 From: ale at codespeak.net (ale at codespeak.net) Date: Fri, 6 Jan 2006 18:49:41 +0100 (CET) Subject: [pypy-svn] r21749 - in pypy/dist/pypy/lib/pyontology: . test Message-ID: <20060106174941.5AF7727B62@code1.codespeak.net> Author: ale Date: Fri Jan 6 18:49:40 2006 New Revision: 21749 Modified: pypy/dist/pypy/lib/pyontology/pyontology.py pypy/dist/pypy/lib/pyontology/test/test_ontology.py Log: Added test for inverseOf. Changed logic of rdfs:domain to be similar to rdfs:range, and changed test accordingly Modified: pypy/dist/pypy/lib/pyontology/pyontology.py ============================================================================== --- pypy/dist/pypy/lib/pyontology/pyontology.py (original) +++ pypy/dist/pypy/lib/pyontology/pyontology.py Fri Jan 6 18:49:40 2006 @@ -385,15 +385,6 @@ # The classes that has this property (s) must belong to the class extension of var avar = self.make_var(ClassDomain, var) svar = self.make_var(Property, s) - assert isinstance(self.variables[svar], Property) - assert isinstance(self.variables[avar], ClassDomain) - vals = get_values(self.variables[avar], self.variables) - if len(vals) == 0 : - vals = [(self.variables[avar], None)] - for k,v in self.variables[svar].getValues(): - if not k in vals: - vals.append((k,v)) - self.variables[svar].setValues(vals) cons = DomainConstraint(svar, avar) self.constraints.append(cons) @@ -541,7 +532,6 @@ val = val[0] if val in domains.keys(): res.extend(get_values(domains[val], domains, attr)) - #res[dom] = 1 return res class SubClassConstraint(OwlConstraint): @@ -587,9 +577,9 @@ propdom = domains[self.variable] rangedom = domains[self.object] newrange = get_values(rangedom, domains, 'getValues') + print rangedom range = [] - prop = Linkeddict(propdom.getValues()) - oldrange = propdom.range#get(None) + oldrange = propdom.range if oldrange: for v in oldrange: if v in newrange: @@ -597,7 +587,7 @@ else: range = newrange propdom.range = range -# propdom.setValues(prop.items()) + prop = Linkeddict(propdom.getValues()) for pval in sum(prop.values(),[]): if pval not in range: raise ConsistencyFailure("Value %r not in range %r"%(pval,range)) @@ -607,12 +597,20 @@ def narrow(self, domains): propdom = domains[self.variable] domaindom = domains[self.object] - vals = get_values(domaindom, domains, 'getValues') - res = [] - for k,val in get_values(propdom, domains, 'getValues'): - if not k in vals and k != domaindom: - res.append((k,val)) - propdom.removeValues(res) + newdomain = get_values(domaindom, domains, 'getValues') +[self.object] + domain = [] + olddomain = propdom.domain + if olddomain: + for v in olddomain: + if v in newdomain: + domain.append(v) + else: + domain = newdomain + propdom.domain = domain + prop = Linkeddict(propdom.getValues()) + for pval in prop.keys(): + if pval not in domain: + raise ConsistencyFailure("Value %r not in range %r"%(pval, domain)) class SubPropertyConstraint(SubClassConstraint): Modified: pypy/dist/pypy/lib/pyontology/test/test_ontology.py ============================================================================== --- pypy/dist/pypy/lib/pyontology/test/test_ontology.py (original) +++ pypy/dist/pypy/lib/pyontology/test/test_ontology.py Fri Jan 6 18:49:40 2006 @@ -157,7 +157,7 @@ O.type(sub, pred , obj) assert len(O.constraints) == 1 O.constraints[0].narrow(O.variables) - assert O.variables['a_'].getValues() == [(O.variables['b_'], None),] + assert O.variables['a_'].domain == ['b_'] def test_domain_merge(): O = Ontology() @@ -283,26 +283,30 @@ O.consistency() assert ('Alice_', 'Bob_') in O.variables['friend_'].getValues() -def no_test_maxcardinality(): +def test_inverseof(): O = Ontology() - #Make functional property - sub = URIRef('friend') + own = URIRef('owner') obj = URIRef(namespaces['owl']+'#ObjectProperty') - O.type(sub, None, obj) + O.type(own, None, obj) + owned = URIRef('ownedby') + obj = URIRef(namespaces['owl']+'#ObjectProperty') + O.type(owned, None, obj) + O.inverseOf(own, None, owned) #Make class sub = URIRef('c') obj = URIRef(namespaces['owl']+'#Class') O.type(sub, None, obj) - #Make individual with a cardinality restriction of the property + #Make individual with a property value sub = URIRef('Bob') obj = URIRef('c') O.type(sub, None, obj) - sub = BNode('anon') - obj = URIRef(namespaces['owl']+'#Restriction') + sub = URIRef('Fiat') + obj = URIRef('car') O.type(sub, None, obj) - O.onProperty(sub, None, URIRef('friend')) - O.maxCardinality(sub, None, 1) - O.variables['friend_'].setValues([('Bob_',['Alice_','Finn_'])]) - O.consistency(verbose=5) + O.variables['owner_'].setValues([('Bob_','Fiat_')]) + py.test.raises(ConsistencyFailure, O.consistency) + O.variables['ownedby_'].setValues([('Fiat_','Bob_')]) + print O.variables + O.consistency() assert not '_anon' in O.variables \ No newline at end of file From ale at codespeak.net Fri Jan 6 18:50:52 2006 From: ale at codespeak.net (ale at codespeak.net) Date: Fri, 6 Jan 2006 18:50:52 +0100 (CET) Subject: [pypy-svn] r21751 - pypy/dist/pypy/lib/pyontology/test Message-ID: <20060106175052.135AA27B68@code1.codespeak.net> Author: ale Date: Fri Jan 6 18:50:51 2006 New Revision: 21751 Modified: pypy/dist/pypy/lib/pyontology/test/test_ontology.py Log: Oups, forgot to save - removing prints Modified: pypy/dist/pypy/lib/pyontology/test/test_ontology.py ============================================================================== --- pypy/dist/pypy/lib/pyontology/test/test_ontology.py (original) +++ pypy/dist/pypy/lib/pyontology/test/test_ontology.py Fri Jan 6 18:50:51 2006 @@ -279,7 +279,6 @@ sub = URIRef('Alice') O.type(sub, None, obj) O.variables['friend_'].setValues([('Bob_','Alice_')]) - print O.variables['friend_']._dict O.consistency() assert ('Alice_', 'Bob_') in O.variables['friend_'].getValues() @@ -307,6 +306,5 @@ O.variables['owner_'].setValues([('Bob_','Fiat_')]) py.test.raises(ConsistencyFailure, O.consistency) O.variables['ownedby_'].setValues([('Fiat_','Bob_')]) - print O.variables O.consistency() assert not '_anon' in O.variables \ No newline at end of file From rxe at codespeak.net Fri Jan 6 19:31:25 2006 From: rxe at codespeak.net (rxe at codespeak.net) Date: Fri, 6 Jan 2006 19:31:25 +0100 (CET) Subject: [pypy-svn] r21756 - pypy/dist/pypy/translator/llvm Message-ID: <20060106183125.31BB527B62@code1.codespeak.net> Author: rxe Date: Fri Jan 6 19:31:20 2006 New Revision: 21756 Modified: pypy/dist/pypy/translator/llvm/arraynode.py pypy/dist/pypy/translator/llvm/codewriter.py pypy/dist/pypy/translator/llvm/exception.py pypy/dist/pypy/translator/llvm/extfuncnode.py pypy/dist/pypy/translator/llvm/funcnode.py pypy/dist/pypy/translator/llvm/gc.py pypy/dist/pypy/translator/llvm/genllvm.py pypy/dist/pypy/translator/llvm/node.py pypy/dist/pypy/translator/llvm/opaquenode.py pypy/dist/pypy/translator/llvm/opwriter.py pypy/dist/pypy/translator/llvm/structnode.py Log: Finish off the boring codewriter/opwriter refactors from before Xmas. Need to fix up the policies now. Modified: pypy/dist/pypy/translator/llvm/arraynode.py ============================================================================== --- pypy/dist/pypy/translator/llvm/arraynode.py (original) +++ pypy/dist/pypy/translator/llvm/arraynode.py Fri Jan 6 19:31:20 2006 @@ -116,9 +116,9 @@ def get_ref(self): typeval = self.db.repr_type(lltype.typeOf(self.value)) - ref = "cast (%s* %s to %s*)" % (self.get_typerepr(), - self.ref, - typeval) + ref = "cast(%s* %s to %s*)" % (self.get_typerepr(), + self.ref, + typeval) p, c = lltype.parentlink(self.value) assert p is None, "child arrays are NOT needed by rtyper" Modified: pypy/dist/pypy/translator/llvm/codewriter.py ============================================================================== --- pypy/dist/pypy/translator/llvm/codewriter.py (original) +++ pypy/dist/pypy/translator/llvm/codewriter.py Fri Jan 6 19:31:20 2006 @@ -131,20 +131,14 @@ self._indent("%(targetvar)s = cast %(fromtype)s " "%(fromvar)s to %(targettype)s" % locals()) - # XXX refactor - should only be one getelementptr - def raw_getelementptr(self, targetvar, type, typevar, *indices): - word = self.word_repr + def getelementptr(self, targetvar, type, typevar, indices, getptr=True): + # XXX comment getptr + if getptr: + indices = [(self.word_repr, 0)] + list(indices) res = "%(targetvar)s = getelementptr %(type)s %(typevar)s, " % locals() res += ", ".join(["%s %s" % (t, i) for t, i in indices]) self._indent(res) - def getelementptr(self, targetvar, type, typevar, *indices): - word = self.word_repr - res = "%(targetvar)s = getelementptr " \ - "%(type)s %(typevar)s, %(word)s 0, " % locals() - res += ", ".join(["%s %s" % (t, i) for t, i in indices]) - self._indent(res) - def load(self, target, targettype, ptr): self._indent("%(target)s = load %(targettype)s* %(ptr)s" % locals()) @@ -155,22 +149,21 @@ def unwind(self): self._indent("unwind") - def call(self, targetvar, returntype, functionref, argrefs, argtypes, + def call(self, targetvar, returntype, functionref, argtypes, argrefs, tail=DEFAULT_TAIL, cconv=DEFAULT_CCONV): tail = self._resolvetail(tail, cconv) args = ", ".join(["%s %s" % item for item in zip(argtypes, argrefs)]) if returntype == 'void': - self._indent("%scall %s void %s(%s)" % (tail, - cconv, - functionref, - args)) + return_str = '' else: - self._indent("%s = %scall %s %s %s(%s)" % (targetvar, - tail, - cconv, - returntype, - functionref, - args)) - + return_str = '%s = ' % targetvar + + self._indent("%s%scall %s %s %s(%s)" % (return_str, + tail, + cconv, + returntype, + functionref, + args)) + Modified: pypy/dist/pypy/translator/llvm/exception.py ============================================================================== --- pypy/dist/pypy/translator/llvm/exception.py (original) +++ pypy/dist/pypy/translator/llvm/exception.py Fri Jan 6 19:31:20 2006 @@ -1,4 +1,5 @@ -from pypy.objspace.flow.model import c_last_exception +from pypy.objspace.flow.model import Variable, c_last_exception + from pypy.translator.llvm.codewriter import DEFAULT_CCONV from pypy.translator.llvm.backendopt.exception import create_exception_handling @@ -26,6 +27,7 @@ ''' % (RINGBUFFER_ENTRY_MAXSIZE, RINGBUGGER_OVERSIZE, RINGBUGGER_SIZE-1) def __init__(self, db): + self.db = db raise Exception, 'ExceptionPolicy should not be used directly' def transform(self, translator, graph=None): @@ -55,7 +57,6 @@ return ' 0' return 'null' - def _nonoderesult(self, node): returntype, name, dummy = node.getdecl_parts() noresult = self._noresult(returntype) @@ -184,6 +185,12 @@ def llc_options(self): return '-enable-correct-eh-support' + + +def repr_if_variable(db, arg): + if isinstance(arg, Variable): + return db.repr_arg(arg) + class ExplicitExceptionPolicy(ExceptionPolicy): """ uses issubclass() and last_exception tests after each call """ def __init__(self, db): @@ -227,12 +234,21 @@ for graph in translator.flowgraphs.itervalues(): create_exception_handling(translator, graph) - def invoke(self, codewriter, targetvar, returntype, functionref, - argrefs, argtypes, label, except_label): + argrefs, argtypes, + node, block): # XXX Unsure of these being passed in assert functionref != '%keepalive' + # at least one label and one exception label + assert len(block.exits) >= 2 + link = block.exits[0] + assert link.exitcase is None + + none_label = node.block_to_name[link.target] + block_label = node.block_to_name[block] + exc_label = block_label + '_exception_handling' + tmp = '%%invoke.tmp.%d' % self.invoke_count exc = '%%invoke.exc.%d' % self.invoke_count self.invoke_count += 1 @@ -240,10 +256,94 @@ # XXX Hardcoded type... type_ = "%RPYTHON_EXCEPTION_VTABLE*" - codewriter.call(targetvar, returntype, functionref, argrefs, argtypes) + codewriter.call(targetvar, returntype, functionref, argtypes, argrefs) codewriter.load(tmp, type_, "%last_exception_type") codewriter.binaryop("seteq", exc, type_, tmp, "null") - codewriter.br(exc, except_label, label) + codewriter.br(exc, exc_label, none_label) + + # write exception handling blocks + + e = self.db.translator.rtyper.getexceptiondata() + ll_exception_match = self.db.repr_value(e.fn_exception_match._obj) + lltype_of_exception_type = self.db.repr_type(e.lltype_of_exception_type) + lltype_of_exception_value = self.db.repr_type(e.lltype_of_exception_value) + + # start with the exception handling block + # * load the last exception type + # * check it with call to ll_exception_match() + # * branch to to correct block? + + codewriter.label(exc_label) + + catch_all = False + found_blocks_info = [] + last_exception_type = None + + # XXX tmp - debugging info + + # block_label = "block28" + # exc_label = "block28_exception_handling" + # ll_exception_match = function for catching exception + # lltype_of_exception_type, lltype_of_exception_value = generic + # catch_all = ??? + # found_blocks_info = list of found block data to write those blocks + # last_exception_type = Load exception pointer once for handle and not found blocks + + # link = iteration thru rest of links in block + # etype = node for exception + # current_exception_type = repr for node etype + # target = label of the destination block + # exc_found_label = label of intermediate exc found block + # last_exc_type_var = ???? + # last_exc_value_var = ??? + + for link in block.exits[1:]: + assert issubclass(link.exitcase, Exception) + + # information for found blocks + target = node.block_to_name[link.target] + exc_found_label = block_label + '_exception_found_branchto_' + target + link_exc_type = repr_if_variable(self.db, link.last_exception) + link_exc_value = repr_if_variable(self.db, link.last_exc_value) + found_blocks_info.append((exc_found_label, target, + link_exc_type, link_exc_value)) + + # XXX fix database to handle this case + etype = self.db.obj2node[link.llexitcase._obj] + current_exception_type = etype.get_ref() + not_this_exception_label = block_label + '_not_exception_' + etype.ref[1:] + + # catch specific exception (class) type + + # load pointer only once + if not last_exception_type: + last_exception_type = self.db.repr_tmpvar() + codewriter.load(last_exception_type, + lltype_of_exception_type, + '%last_exception_type') + codewriter.newline() + + ll_issubclass_cond = self.db.repr_tmpvar() + + codewriter.call(ll_issubclass_cond, + 'bool', + ll_exception_match, + [lltype_of_exception_type, lltype_of_exception_type], + [last_exception_type, current_exception_type]) + + codewriter.br(ll_issubclass_cond, + not_this_exception_label, + exc_found_label) + + codewriter.label(not_this_exception_label) + + if not catch_all: + self.reraise(node, codewriter) + + self.fetch_exceptions(codewriter, + found_blocks_info, + lltype_of_exception_type, + lltype_of_exception_value) def write_exceptblock(self, funcnode, codewriter, block): """ Raises an exception - called from FuncNode """ Modified: pypy/dist/pypy/translator/llvm/extfuncnode.py ============================================================================== --- pypy/dist/pypy/translator/llvm/extfuncnode.py (original) +++ pypy/dist/pypy/translator/llvm/extfuncnode.py Fri Jan 6 19:31:20 2006 @@ -93,7 +93,7 @@ # call codewriter.call(returnval, expected_rettype, self.ref, - expected_argrefs, expected_argtypes) + expected_argtypes, expected_argrefs) if self.wrapper.rettype: # cast to desired return type Modified: pypy/dist/pypy/translator/llvm/funcnode.py ============================================================================== --- pypy/dist/pypy/translator/llvm/funcnode.py (original) +++ pypy/dist/pypy/translator/llvm/funcnode.py Fri Jan 6 19:31:20 2006 @@ -41,17 +41,10 @@ self.ref = self.make_ref('%pypy_', value.graph.name) self.graph = value.graph - self.db.exceptionpolicy.transform(self.db.translator, - self.graph) - - remove_exception_mallocs(self.db.translator, self.graph, self.ref) - #XXX experimental #from pypy.translator.llvm.backendopt.mergemallocs import merge_mallocs #merge_mallocs(self.db.translator, self.graph, self.ref) - remove_double_links(self.db.translator, self.graph) - def __str__(self): return "" %(self.ref,) @@ -75,7 +68,13 @@ traverse(visit, self.graph) # ______________________________________________________________________ - # main entry points from genllvm + # main entry points from genllvm + + def post_setup_transform(self): + self.db.exceptionpolicy.transform(self.db.translator, self.graph) + remove_exception_mallocs(self.db.translator, self.graph, self.ref) + remove_double_links(self.db.translator, self.graph) + def writedecl(self, codewriter): codewriter.declare(self.getdecl()) @@ -196,20 +195,24 @@ block.exitswitch.concretetype) def write_block_operations(self, codewriter, block): - opwriter = OpWriter(self.db, codewriter, self, block) - + # XXX We dont need multiple of these + opwriter = OpWriter(self.db, codewriter) + + invoke_op = None + block_ops = block.operations if block.exitswitch == c_last_exception: - invoke_prefix = 'invoke:' - # could raise an exception and should therefor have a function - # implementation that can be invoked by the llvm-code. - op = block.operations[len(block.operations) - 1] - assert not op.opname.startswith(invoke_prefix) - op.opname = invoke_prefix + op.opname + invoke_op = block.operations[-1] + block_ops = block.operations[:-1] # emit operations - for op in block.operations: + for op in block_ops: opwriter.write_operation(op) + if invoke_op: + # could raise an exception and should therefore have a function + # implementation that can be invoked by the llvm-code. + opwriter.write_invoke_op(invoke_op, self, block) + # ______________________________________________________________________ # actual block writers Modified: pypy/dist/pypy/translator/llvm/gc.py ============================================================================== --- pypy/dist/pypy/translator/llvm/gc.py (original) +++ pypy/dist/pypy/translator/llvm/gc.py Fri Jan 6 19:31:20 2006 @@ -41,9 +41,9 @@ elemindices = list(indices_to_array) elemindices += [("uint", 1), (lentype, "%actuallen")] - codewriter.getelementptr("%size", ref + "*", "null", *elemindices) + codewriter.getelementptr("%size", ref + "*", "null", elemindices) codewriter.cast("%usize", elemtype + "*", "%size", uword) - self.malloc(codewriter, "%ptr", "sbyte", "%usize", atomic=atomic) + self.malloc(codewriter, "%ptr", "sbyte*", "%usize", atomic=atomic) codewriter.cast("%result", "sbyte*", "%ptr", ref + "*") indices_to_arraylength = tuple(indices_to_array) + (("uint", 0),) @@ -51,21 +51,21 @@ # the following accesses the length field of the array codewriter.getelementptr("%arraylength", ref + "*", "%result", - *indices_to_arraylength) + indices_to_arraylength) codewriter.store(lentype, "%len", "%arraylength") #if is_str: # indices_to_hash = (("uint", 0),) # codewriter.getelementptr("%ptrhash", ref + "*", # "%result", - # *indices_to_hash) + # indices_to_hash) # codewriter.store("int", "0", "%ptrhash") #if ARRAY is STR.chars: # codewriter.getelementptr("%ptrendofchar", ref + "*", # "%result", - # *elemindices) + # elemindices) # codewriter.store(elemtype, "0", "%ptrendofchar") codewriter.ret(ref + "*", "%result") @@ -122,10 +122,10 @@ cnt = '.%d' % self.n_malloced atomic = is_atomic and '_atomic' or '' t = ''' -%%malloc_Size%(cnt)s = getelementptr %(type_)s* null, %(uword)s %(s)s -%%malloc_SizeU%(cnt)s = cast %(type_)s* %%malloc_Size%(cnt)s to %(uword)s +%%malloc_Size%(cnt)s = getelementptr %(type_)s null, %(uword)s %(s)s +%%malloc_SizeU%(cnt)s = cast %(type_)s %%malloc_Size%(cnt)s to %(uword)s %%malloc_Ptr%(cnt)s = call fastcc sbyte* %%pypy_malloc%(atomic)s(%(uword)s %%malloc_SizeU%(cnt)s) -%(targetvar)s = cast sbyte* %%malloc_Ptr%(cnt)s to %(type_)s* +%(targetvar)s = cast sbyte* %%malloc_Ptr%(cnt)s to %(type_)s ''' % locals() if is_atomic: Modified: pypy/dist/pypy/translator/llvm/genllvm.py ============================================================================== --- pypy/dist/pypy/translator/llvm/genllvm.py (original) +++ pypy/dist/pypy/translator/llvm/genllvm.py Fri Jan 6 19:31:20 2006 @@ -86,6 +86,9 @@ self.db.setup_all() self._checkpoint('setup_all externs') + for node in self.db.getnodes(): + node.post_setup_transform() + self._print_node_stats() # create ll file from c code @@ -290,7 +293,12 @@ if optimize: backend_optimizations(t, ssa_form=False) else: - backend_optimizations(t, ssa_form=False, mallocs=False, inline_threshold=0, merge_if_blocks_to_switch=False, propagate=False) + backend_optimizations(t, ssa_form=False, + inline_threshold=0, + mallocs=False, + merge_if_blocks_to_switch=False, + propagate=False) + # note: this is without policy transforms if view: t.view() Modified: pypy/dist/pypy/translator/llvm/node.py ============================================================================== --- pypy/dist/pypy/translator/llvm/node.py (original) +++ pypy/dist/pypy/translator/llvm/node.py Fri Jan 6 19:31:20 2006 @@ -26,6 +26,11 @@ pass # __________________ before "implementation" ____________________ + + + def post_setup_transform(self): + pass + def writedatatypedecl(self, codewriter): """ write out declare names of data types (structs/arrays/function pointers) Modified: pypy/dist/pypy/translator/llvm/opaquenode.py ============================================================================== --- pypy/dist/pypy/translator/llvm/opaquenode.py (original) +++ pypy/dist/pypy/translator/llvm/opaquenode.py Fri Jan 6 19:31:20 2006 @@ -82,5 +82,5 @@ codewriter.call(self.db.repr_tmpvar(), "sbyte*", "%RPyOpaque_LLVM_SETUP_ThreadLock", - argrefs, argtypes) + argtypes, argrefs) # XXX Check result Modified: pypy/dist/pypy/translator/llvm/opwriter.py ============================================================================== --- pypy/dist/pypy/translator/llvm/opwriter.py (original) +++ pypy/dist/pypy/translator/llvm/opwriter.py Fri Jan 6 19:31:20 2006 @@ -1,349 +1,278 @@ -from pypy.objspace.flow.model import Constant, Variable +from pypy.objspace.flow.model import Constant from pypy.rpython.lltypesystem import lltype from pypy.translator.llvm.log import log log = log.opwriter -def repr_if_variable(db, arg): - if isinstance(arg, Variable): - return db.repr_arg(arg) - -class OpWriter(object): - binary_operations = {'int_mul': 'mul', - 'int_add': 'add', - 'int_sub': 'sub', - 'int_floordiv': 'div', - 'int_mod': 'rem', - 'int_and': 'and', - 'int_or': 'or', - 'int_xor': 'xor', - 'int_lt': 'setlt', - 'int_le': 'setle', - 'int_eq': 'seteq', - 'int_ne': 'setne', - 'int_ge': 'setge', - 'int_gt': 'setgt', - - 'llong_mul': 'mul', - 'llong_add': 'add', - 'llong_sub': 'sub', - 'llong_floordiv': 'div', - 'llong_mod': 'rem', - 'llong_and': 'and', - 'llong_or': 'or', - 'llong_xor': 'xor', - 'llong_lt': 'setlt', - 'llong_le': 'setle', - 'llong_eq': 'seteq', - 'llong_ne': 'setne', - 'llong_ge': 'setge', - 'llong_gt': 'setgt', - - 'uint_mul': 'mul', - 'uint_add': 'add', - 'uint_sub': 'sub', - 'uint_floordiv': 'div', - 'uint_mod': 'rem', - 'uint_and': 'and', - 'uint_or': 'or', - 'uint_xor': 'xor', - 'uint_lt': 'setlt', - 'uint_le': 'setle', - 'uint_eq': 'seteq', - 'uint_ne': 'setne', - 'uint_ge': 'setge', - 'uint_gt': 'setgt', - - 'unichar_lt': 'setlt', - 'unichar_le': 'setle', - 'unichar_eq': 'seteq', - 'unichar_ne': 'setne', - 'unichar_ge': 'setge', - 'unichar_gt': 'setgt', - - 'float_mul': 'mul', - 'float_add': 'add', - 'float_sub': 'sub', - 'float_truediv': 'div', - 'float_lt': 'setlt', - 'float_le': 'setle', - 'float_eq': 'seteq', - 'float_ne': 'setne', - 'float_ge': 'setge', - 'float_gt': 'setgt', - - 'ptr_eq': 'seteq', - 'ptr_ne': 'setne', - } - - shift_operations = {'int_lshift': 'shl', - 'int_rshift': 'shr', +class OpRepr(object): + __slots__ = "db op retref rettype argrefs argtypes".split() + def __init__(self, op, db): + self.db = db + self.op = op + self.argrefs = db.repr_arg_multi(op.args) + self.argtypes = db.repr_arg_type_multi(op.args) + self.retref = db.repr_arg(op.result) + self.rettype = db.repr_arg_type(op.result) + +class OpReprCall(OpRepr): + __slots__ = "db op retref rettype argrefs argtypes".split() + def __init__(self, op, db): + super(OpReprCall, self).__init__(op, db) + self.argrefs = [aref for arg, aref in zip(op.args, self.argrefs) + if arg.concretetype is not lltype.Void] + self.argtypes = [atype for arg, atype in zip(op.args, self.argtypes) + if arg.concretetype is not lltype.Void] + if self.db.is_function_ptr(self.op.result): + self.rettype = "%s (%s)*" % (self.rettype, + ", ".join(self.argtypes[1:])) + +class OpReprInvoke(OpReprCall): + __slots__ = "db op retref rettype argrefs argtypes functionref".split() + def __init__(self, op, db): + super(OpReprInvoke, self).__init__(op, db) + + if op.opname == 'direct_call': + self.functionref = self.argrefs[0] + self.argrefs = self.argrefs[1:] + self.argtypes = self.argtypes[1:] + else: + self.functionref = '%pypyop_' + op.opname - 'uint_lshift': 'shl', - 'uint_rshift': 'shr', - - 'llong_lshift': 'shl', - 'llong_rshift': 'shr', +class OpWriter(object): + + binary_operations = { + 'float_mul' : 'mul', + 'float_add' : 'add', + 'float_sub' : 'sub', + 'float_truediv' : 'div', + + 'ptr_eq' : 'seteq', + 'ptr_ne' : 'setne' } + + # generic numeric ops + for tt in 'int llong uint'.split(): + for oo in 'mul add sub and or xor'.split(): + binary_operations['%s_%s' % (tt, oo)] = oo + binary_operations['%s_floordiv' % tt] = 'div' + binary_operations['%s_mod' % tt] = 'rem' + + # comparison ops + for tt in 'int llong uint unichar float'.split(): + for oo in 'lt le eq ne ge gt'.split(): + binary_operations['%s_%s' % (tt, oo)] = 'set%s' % oo + + + shift_operations = {'int_lshift': 'shl', + 'int_rshift': 'shr', + + 'uint_lshift': 'shl', + 'uint_rshift': 'shr', + + 'llong_lshift': 'shl', + 'llong_rshift': 'shr', } + char_operations = {'char_lt': 'setlt', + 'char_le': 'setle', + 'char_eq': 'seteq', + 'char_ne': 'setne', + 'char_ge': 'setge', + 'char_gt': 'setgt'} - char_operations = {'char_lt': 'setlt', - 'char_le': 'setle', - 'char_eq': 'seteq', - 'char_ne': 'setne', - 'char_ge': 'setge', - 'char_gt': 'setgt'} - - def __init__(self, db, codewriter, node, block): + def __init__(self, db, codewriter): self.db = db self.codewriter = codewriter - self.node = node - self.block = block + def _tmp(self, count=1): + if count == 1: + return self.db.repr_tmpvar() + else: + return [self.db.repr_tmpvar() for ii in range(count)] + def write_operation(self, op): - invoke = op.opname.startswith('invoke:') - if invoke: - self.invoke(op) + if op.opname == "direct_call": + opr = OpReprCall(op, self.db) else: - if op.opname in self.binary_operations: - self.binaryop(op) - elif op.opname in self.shift_operations: - self.shiftop(op) - elif op.opname in self.char_operations: - self.char_binaryop(op) - elif op.opname.startswith('cast_') or op.opname.startswith('truncate_'): - if op.opname == 'cast_char_to_int': - self.cast_char_to_int(op) - else: - self.cast_primitive(op) + opr = OpRepr(op, self.db) + + if op.opname in self.binary_operations: + self.binaryop(opr) + elif op.opname in self.shift_operations: + self.shiftop(opr) + elif op.opname in self.char_operations: + self.char_binaryop(opr) + elif op.opname.startswith('cast_') or op.opname.startswith('truncate_'): + if op.opname == 'cast_char_to_int': + self.cast_char_to_int(opr) else: - meth = getattr(self, op.opname, None) - if not meth: - raise Exception, "operation %s not found" % op.opname - meth(op) - - def _generic_pow(self, op, onestr): - mult_type = self.db.repr_arg_type(op.args[0]) - mult_val = self.db.repr_arg(op.args[0]) - last_val = mult_val + self.cast_primitive(opr) + else: + meth = getattr(self, op.opname, None) + if not meth: + raise Exception, "operation %s not found" % op.opname + meth(opr) + + def write_invoke_op(self, op, node, block): + opr = OpReprInvoke(op, self.db) + ep = self.db.exceptionpolicy + ep.invoke(self.codewriter, opr.retref, opr.rettype, opr.functionref, + opr.argrefs, opr.argtypes, node, block) + + def _generic_pow(self, opr, onestr): + + # XXX This broken as... will only work for constants try: value = "NO VALUE" - value = op.args[1].value + value = opr.op.args[1].value operand = int(value) except Exception, exc: msg = 'XXX: Error: _generic_pow: Variable '\ '%s - failed to convert to int %s' % (value, str(exc)) self.codewriter.comment(msg) raise Exception(msg) + + mult_type = opr.argtypes[0] + mult_val = opr.argrefs[0] + last_val = mult_val if operand < 1: res_val = onestr else: res_val = mult_val for ii in range(operand - 1): - res_val = self.db.repr_tmpvar() - self.codewriter.binaryop("mul", - res_val, - mult_type, - last_val, - mult_val) + res_val = self._tmp() + self.codewriter.binaryop("mul", res_val, mult_type, + last_val, mult_val) last_val = res_val - targetvar = self.db.repr_arg(op.result) - self.codewriter.cast(targetvar, mult_type, res_val, mult_type) + self.codewriter.cast(opr.retref, mult_type, res_val, mult_type) - def _skipped(self, op): - self.codewriter.comment('***Skipping operation %s()' % (op.opname,)) + def _skipped(self, opr): + self.codewriter.comment('***Skipping operation %s()' % opr.op.opname) keepalive = _skipped - def int_abs(self, op): - functionref = '%pypyop_' + op.opname - self.codewriter.call(self.db.repr_arg(op.result), - self.db.repr_arg_type(op.result), - functionref, - [self.db.repr_arg(op.args[0])], - [self.db.repr_arg_type(op.args[0])]) + def int_abs(self, opr): + assert len(opr.argrefs) == 1 + functionref = '%pypyop_' + opr.op.opname + self.codewriter.call(opr.retref, opr.rettype, functionref, + opr.argtypes, opr.argrefs) + float_abs = int_abs llong_abs = int_abs - def int_pow(self, op): - self._generic_pow(op, "1") + def int_pow(self, opr): + self._generic_pow(opr, "1") uint_pow = int_pow - def float_pow(self, op): - self._generic_pow(op, "1.0") + def float_pow(self, opr): + self._generic_pow(opr, "1.0") + + def _generic_neg(self, opr, zerostr): + self.codewriter.binaryop("sub", opr.retref, opr.argtypes[0], + zerostr, opr.argrefs[0]) - def _generic_neg(self, op, zerostr): - self.codewriter.binaryop("sub", - self.db.repr_arg(op.result), - self.db.repr_arg_type(op.args[0]), - zerostr, - self.db.repr_arg(op.args[0]), - ) - def int_neg(self, op): - self._generic_neg(op, "0") + def int_neg(self, opr): + self._generic_neg(opr, "0") - #this is really generated, don't know why + # this is really generated, don't know why # XXX rxe: Surely that cant be right? uint_neg = int_neg - def float_neg(self, op): - self._generic_neg(op, "0.0") - - def bool_not(self, op): - self.codewriter.binaryop("xor", - self.db.repr_arg(op.result), - self.db.repr_arg_type(op.args[0]), - self.db.repr_arg(op.args[0]), - "true") - - def int_invert(self, op): - self.codewriter.binaryop("xor", - self.db.repr_arg(op.result), - self.db.repr_arg_type(op.args[0]), - self.db.repr_arg(op.args[0]), - -1) - - def uint_invert(self, op): - self.codewriter.binaryop("xor", - self.db.repr_arg(op.result), - self.db.repr_arg_type(op.args[0]), - self.db.repr_arg(op.args[0]), - str((1L<<32) - 1)) - - def binaryop(self, op): - name = self.binary_operations[op.opname] - assert len(op.args) == 2 - self.codewriter.binaryop(name, - self.db.repr_arg(op.result), - self.db.repr_arg_type(op.args[0]), - self.db.repr_arg(op.args[0]), - self.db.repr_arg(op.args[1])) - - def char_binaryop(self, op): - name = self.char_operations[op.opname] - assert len(op.args) == 2 - res = self.db.repr_arg(op.result) - c1 = self.db.repr_tmpvar() - c2 = self.db.repr_tmpvar() - self.codewriter.cast(c1, "sbyte", self.db.repr_arg(op.args[0]), "ubyte") - self.codewriter.cast(c2, "sbyte", self.db.repr_arg(op.args[1]), "ubyte") - self.codewriter.binaryop(name, res, "ubyte", c1, c2) + def float_neg(self, opr): + self._generic_neg(opr, "0.0") + def bool_not(self, opr): + self.codewriter.binaryop("xor", opr.retref, opr.argtypes[0], + opr.argrefs[0], "true") + + def int_invert(self, opr): + self.codewriter.binaryop("xor", opr.retref, opr.argtypes[0], + opr.argrefs[0], -1) + + def uint_invert(self, opr): + self.codewriter.binaryop("xor", opr.retref, opr.argtypes[0], + opr.argrefs[0], str((1L<<32) - 1)) + + def binaryop(self, opr): + assert len(opr.argrefs) == 2 + name = self.binary_operations[opr.op.opname] + self.codewriter.binaryop(name, opr.retref, opr.argtypes[0], + opr.argrefs[0], opr.argrefs[1]) + + def char_binaryop(self, opr): + assert len(opr.argrefs) == 2 + name = self.char_operations[opr.op.opname] + c1, c2 = self._tmp(2) + self.codewriter.cast(c1, "sbyte", opr.argrefs[0], "ubyte") + self.codewriter.cast(c2, "sbyte", opr.argrefs[1], "ubyte") + self.codewriter.binaryop(name, opr.retref, "ubyte", c1, c2) - def shiftop(self, op): + def shiftop(self, opr): + op = opr.op name = self.shift_operations[op.opname] - assert len(op.args) == 2 + if isinstance(op.args[1], Constant): - tmpvar = self.db.repr_arg(op.args[1]) + var = opr.argrefs[1] else: - tmpvar = self.db.repr_tmpvar() - self.codewriter.cast(tmpvar, self.db.repr_arg_type(op.args[1]), self.db.repr_arg(op.args[1]), 'ubyte') - self.codewriter.shiftop(name, - self.db.repr_arg(op.result), - self.db.repr_arg_type(op.args[0]), - self.db.repr_arg(op.args[0]), - tmpvar) + var = self._tmp() + self.codewriter.cast(var, opr.argtypes[1], opr.argrefs[1], 'ubyte') + + self.codewriter.shiftop(name, opr.retref, opr.argtypes[0], + opr.argrefs[0], var) - def cast_char_to_int(self, op): + def cast_char_to_int(self, opr): " works for all casts " - assert len(op.args) == 1 - targetvar = self.db.repr_arg(op.result) - targettype = self.db.repr_arg_type(op.result) - fromvar = self.db.repr_arg(op.args[0]) - fromtype = self.db.repr_arg_type(op.args[0]) - intermediate = self.db.repr_tmpvar() - self.codewriter.cast(intermediate, fromtype, fromvar, "ubyte") - self.codewriter.cast(targetvar, "ubyte", intermediate, targettype) + assert len(opr.argrefs) == 1 + intermediate = self._tmp() + self.codewriter.cast(intermediate, opr.argtypes[0], + opr.argrefs[0], "ubyte") + self.codewriter.cast(opr.retref, "ubyte", intermediate, opr.rettype) - def cast_primitive(self, op): + def cast_primitive(self, opr): " works for all casts " - assert len(op.args) == 1 - targetvar = self.db.repr_arg(op.result) - targettype = self.db.repr_arg_type(op.result) - fromvar = self.db.repr_arg(op.args[0]) - fromtype = self.db.repr_arg_type(op.args[0]) - self.codewriter.cast(targetvar, fromtype, fromvar, targettype) + assert len(opr.argrefs) == 1 + self.codewriter.cast(opr.retref, opr.argtypes[0], + opr.argrefs[0], opr.rettype) same_as = cast_primitive - def int_is_true(self, op): - self.codewriter.binaryop("setne", - self.db.repr_arg(op.result), - self.db.repr_arg_type(op.args[0]), - self.db.repr_arg(op.args[0]), - "0") + def int_is_true(self, opr): + self.codewriter.binaryop("setne", opr.retref, opr.argtypes[0], + opr.argrefs[0], "0") uint_is_true = int_is_true llong_is_true = int_is_true - def float_is_true(self, op): - self.codewriter.binaryop("setne", - self.db.repr_arg(op.result), - self.db.repr_arg_type(op.args[0]), - self.db.repr_arg(op.args[0]), - "0.0") - - def ptr_nonzero(self, op): - self.codewriter.binaryop("setne", - self.db.repr_arg(op.result), - self.db.repr_arg_type(op.args[0]), - self.db.repr_arg(op.args[0]), - "null") - - def ptr_iszero(self, op): - self.codewriter.binaryop("seteq", - self.db.repr_arg(op.result), - self.db.repr_arg_type(op.args[0]), - self.db.repr_arg(op.args[0]), - "null") - - def direct_call(self, op): - op_args = [arg for arg in op.args - if arg.concretetype is not lltype.Void] - assert len(op_args) >= 1 - targetvar = self.db.repr_arg(op.result) - returntype = self.db.repr_arg_type(op.result) - functionref = self.db.repr_arg(op_args[0]) - argrefs = self.db.repr_arg_multi(op_args[1:]) - argtypes = self.db.repr_arg_type_multi(op_args[1:]) - if self.db.is_function_ptr(op.result): - returntype = "%s (%s)*" % (returntype, ", ".join(argtypes)) - self.codewriter.call(targetvar, returntype, - functionref, argrefs, argtypes) - - def malloc_exception(self, op): - arg_type = op.args[0].value - targetvar = self.db.repr_arg(op.result) - type_ = self.db.repr_type(arg_type) - tmpvar1 = self.db.repr_tmpvar() - tmpvar2 = self.db.repr_tmpvar() - tmpvar3 = self.db.repr_tmpvar() - - ptr_type = type_ + '*' - self.codewriter.raw_getelementptr(tmpvar1, ptr_type, "null", ("int", 1)) - self.codewriter.cast(tmpvar2, ptr_type, tmpvar1, 'uint') - self.codewriter.call(tmpvar3, 'sbyte*', '%malloc_exception', [tmpvar2], ['uint']) - self.codewriter.cast(targetvar, 'sbyte*', tmpvar3, type_+'*') - - def malloc(self, op): - arg_type = op.args[0].value - targetvar = self.db.repr_arg(op.result) - type_ = self.db.repr_type(arg_type) - gp = self.db.gcpolicy - gp.malloc(self.codewriter, targetvar, type_, - atomic=arg_type._is_atomic()) + def float_is_true(self, opr): + self.codewriter.binaryop("setne", opr.retref, opr.argtypes[0], + opr.argrefs[0], "0.0") + + def ptr_nonzero(self, opr): + self.codewriter.binaryop("setne", opr.retref, opr.argtypes[0], + opr.argrefs[0], "null") + + def ptr_iszero(self, opr): + self.codewriter.binaryop("seteq", opr.retref, opr.argtypes[0], + opr.argrefs[0], "null") + + def direct_call(self, opr): + self.codewriter.call(opr.retref, opr.rettype, opr.argrefs[0], + opr.argtypes[1:], opr.argrefs[1:]) + + def malloc(self, opr): + arg_type = opr.op.args[0].value + self.db.gcpolicy.malloc(self.codewriter, opr.retref, opr.rettype, + atomic=arg_type._is_atomic()) + + def malloc_varsize(self, opr): - def malloc_varsize(self, op): - arg_type = op.args[0].value + # XXX transformation + arg_type = opr.op.args[0].value if isinstance(arg_type, lltype.Array) and arg_type.OF is lltype.Void: # This is a backend decision to NOT represent a void array with # anything and save space - therefore not varsized anymore - self.malloc(op) + self.malloc(opr) return - - targetvar = self.db.repr_arg(op.result) - type_ = self.db.repr_type(arg_type) + "*" - type_cons = self.db.repr_constructor(arg_type) - argrefs = self.db.repr_arg_multi(op.args[1:]) - argtypes = self.db.repr_arg_type_multi(op.args[1:]) - self.codewriter.call(targetvar, type_, type_cons, argrefs, argtypes) + + self.codewriter.call(opr.retref, opr.rettype, + self.db.repr_constructor(arg_type), + opr.argtypes[1:], opr.argrefs[1:]) def _getindexhelper(self, name, struct): assert name in list(struct._names) @@ -355,186 +284,139 @@ index = -1 return index - def getfield(self, op): - tmpvar = self.db.repr_tmpvar() - struct, structtype = self.db.repr_argwithtype(op.args[0]) - index = self._getindexhelper(op.args[1].value, op.args[0].concretetype.TO) - targetvar = self.db.repr_arg(op.result) - targettype = self.db.repr_arg_type(op.result) - if targettype != "void": + def getfield(self, opr): + op = opr.op + if opr.rettype != "void": + index = self._getindexhelper(op.args[1].value, + op.args[0].concretetype.TO) assert index != -1 - self.codewriter.getelementptr(tmpvar, structtype, struct, - ("uint", index)) - self.codewriter.load(targetvar, targettype, tmpvar) + tmpvar = self._tmp() + self.codewriter.getelementptr(tmpvar, opr.argtypes[0], + opr.argrefs[0], [("uint", index)]) + self.codewriter.load(opr.retref, opr.rettype, tmpvar) else: - self._skipped(op) + self._skipped(opr) - def getsubstruct(self, op): - struct, structtype = self.db.repr_argwithtype(op.args[0]) - index = self._getindexhelper(op.args[1].value, op.args[0].concretetype.TO) - targetvar = self.db.repr_arg(op.result) - targettype = self.db.repr_arg_type(op.result) - assert targettype != "void" - self.codewriter.getelementptr(targetvar, structtype, - struct, ("uint", index)) + def getsubstruct(self, opr): + index = self._getindexhelper(opr.op.args[1].value, + opr.op.args[0].concretetype.TO) + assert opr.rettype != "void" + self.codewriter.getelementptr(opr.retref, opr.argtypes[0], + opr.argrefs[0], [("uint", index)]) - def setfield(self, op): - tmpvar = self.db.repr_tmpvar() - struct, structtype = self.db.repr_argwithtype(op.args[0]) - index = self._getindexhelper(op.args[1].value, op.args[0].concretetype.TO) - valuevar, valuetype = self.db.repr_argwithtype(op.args[2]) - if valuetype != "void": - #Structure types require uint constants! - #see: http://llvm.cs.uiuc.edu/docs/LangRef.html#i_getelementptr - self.codewriter.getelementptr(tmpvar, structtype, struct, - ("uint", index)) - self.codewriter.store(valuetype, valuevar, tmpvar) + def setfield(self, opr): + op = opr.op + if opr.argtypes[2] != "void": + tmpvar = self._tmp() + index = self._getindexhelper(op.args[1].value, + op.args[0].concretetype.TO) + self.codewriter.getelementptr(tmpvar, opr.argtypes[0], + opr.argrefs[0], [("uint", index)]) + self.codewriter.store(opr.argtypes[2], opr.argrefs[2], tmpvar) + else: - self._skipped(op) + self._skipped(opr) - def getarrayitem(self, op): - array, arraytype = self.db.repr_argwithtype(op.args[0]) - index = self.db.repr_arg(op.args[1]) - indextype = self.db.repr_arg_type(op.args[1]) - tmpvar = self.db.repr_tmpvar() - targetvar = self.db.repr_arg(op.result) - targettype = self.db.repr_arg_type(op.result) - if targettype != "void": + def getarrayitem(self, opr): + if opr.rettype != "void": + arraytype, indextype = opr.argtypes + array, index = opr.argrefs + tmpvar = self._tmp() self.codewriter.getelementptr(tmpvar, arraytype, array, - ("uint", 1), (indextype, index)) - self.codewriter.load(targetvar, targettype, tmpvar) + [("uint", 1), (indextype, index)]) + self.codewriter.load(opr.retref, opr.rettype, tmpvar) else: - self._skipped(op) - - def getarraysubstruct(self, op): - array, arraytype = self.db.repr_argwithtype(op.args[0]) - index = self.db.repr_arg(op.args[1]) - indextype = self.db.repr_arg_type(op.args[1]) - targetvar = self.db.repr_arg(op.result) - self.codewriter.getelementptr(targetvar, arraytype, array, - ("uint", 1), (indextype, index)) - - def setarrayitem(self, op): - array, arraytype = self.db.repr_argwithtype(op.args[0]) - index = self.db.repr_arg(op.args[1]) - indextype = self.db.repr_arg_type(op.args[1]) + self._skipped(opr) - tmpvar = self.db.repr_tmpvar() - - valuevar = self.db.repr_arg(op.args[2]) - valuetype = self.db.repr_arg_type(op.args[2]) + def getarraysubstruct(self, opr): + arraytype, indextype = opr.argtypes + array, index = opr.argrefs + self.codewriter.getelementptr(opr.retref, arraytype, array, + [("uint", 1), (indextype, index)]) + + def setarrayitem(self, opr): + array, index, valuevar = opr.argrefs + arraytype, indextype, valuetype = opr.argtypes + tmpvar = self._tmp() if valuetype != "void": self.codewriter.getelementptr(tmpvar, arraytype, array, - ("uint", 1), (indextype, index)) + [("uint", 1), (indextype, index)]) self.codewriter.store(valuetype, valuevar, tmpvar) else: - self._skipped(op) + self._skipped(opr) - def getarraysize(self, op): - array, arraytype = self.db.repr_argwithtype(op.args[0]) - tmpvar = self.db.repr_tmpvar() - self.codewriter.getelementptr(tmpvar, arraytype, array, ("uint", 0)) - targetvar = self.db.repr_arg(op.result) - targettype = self.db.repr_arg_type(op.result) - self.codewriter.load(targetvar, targettype, tmpvar) - - def adr_delta(self, op): - tmp = self.db.repr_tmpvar - addr1, addr2 = tmp(), tmp() - arg1, argtype1 = self.db.repr_argwithtype(op.args[0]) - arg2, argtype2 = self.db.repr_argwithtype(op.args[1]) - self.codewriter.cast(addr1, argtype1, arg1, "int") - self.codewriter.cast(addr2, argtype2, arg2, "int") - targetvar = self.db.repr_arg(op.result) - targettype = self.db.repr_arg_type(op.result) - self.codewriter.binaryop("sub", - targetvar, targettype, - addr1, addr2,) - - def _op_adr_generic(self, op, llvm_op): - tmp = self.db.repr_tmpvar - addr, res = tmp(), tmp() - arg, argtype = self.db.repr_argwithtype(op.args[0]) - self.codewriter.cast(addr, argtype, arg, "int") - arg2, argtype2 = self.db.repr_argwithtype(op.args[1]) - self.codewriter.binaryop(llvm_op, - res, "int", - addr, arg2) - targetvar = self.db.repr_arg(op.result) - targettype = self.db.repr_arg_type(op.result) - self.codewriter.cast(targetvar, "int", res, targettype) - - def adr_add(self, op): - self._op_adr_generic(op, "add") - - def adr_sub(self, op): - self._op_adr_generic(op, "sub") - - def _op_adr_comparison_generic(self, op, llvm_op): - tmp = self.db.repr_tmpvar - addr1, addr2 = tmp(), tmp() - arg1, argtype1 = self.db.repr_argwithtype(op.args[0]) - arg2, argtype2 = self.db.repr_argwithtype(op.args[1]) - self.codewriter.cast(addr1, argtype1, arg1, "int") - self.codewriter.cast(addr2, argtype2, arg2, "int") - targetvar = self.db.repr_arg(op.result) - targettype = self.db.repr_arg_type(op.result) - assert targettype == "bool" - self.codewriter.binaryop(llvm_op, - targetvar, "int", - addr1, addr2) - - def adr_eq(self, op): - self._op_adr_comparison_generic(op, "seteq") - - def adr_ne(self, op): - self._op_adr_comparison_generic(op, "setne") - - def adr_le(self, op): - self._op_adr_comparison_generic(op, "setle") - - def adr_gt(self, op): - self._op_adr_comparison_generic(op, "setgt") - - def adr_lt(self, op): - self._op_adr_comparison_generic(op, "setlt") - - def adr_ge(self, op): - self._op_adr_comparison_generic(op, "setge") - - def raw_malloc(self, op): - # XXX ignore raise as not last op - targetvar = self.db.repr_arg(op.result) - targettype = self.db.repr_arg_type(op.result) - argrefs = self.db.repr_arg_multi(op.args) - argtypes = self.db.repr_arg_type_multi(op.args) - self.codewriter.call(targetvar, targettype, "%raw_malloc", - argrefs, argtypes) - def raw_free(self, op): - targetvar = self.db.repr_arg(op.result) - targettype = self.db.repr_arg_type(op.result) - argrefs = self.db.repr_arg_multi(op.args) - argtypes = self.db.repr_arg_type_multi(op.args) - self.codewriter.call(targetvar, targettype, "%raw_free", - argrefs, argtypes) - - def raw_memcopy(self, op): - targetvar = self.db.repr_arg(op.result) - targettype = self.db.repr_arg_type(op.result) - argrefs = self.db.repr_arg_multi(op.args) - argtypes = self.db.repr_arg_type_multi(op.args) - self.codewriter.call(targetvar, targettype, "%raw_memcopy", - argrefs, argtypes) + def getarraysize(self, opr): + tmpvar = self._tmp() + self.codewriter.getelementptr(tmpvar, opr.argtypes[0], + opr.argrefs[0], [("uint", 0)]) + self.codewriter.load(opr.retref, opr.rettype, tmpvar) - def raw_store(self, op): - tmp = self.db.repr_tmpvar + + # XXX use machine word size + + def adr_delta(self, opr): + addr1, addr2 = self._tmp(2) + self.codewriter.cast(addr1, opr.argtypes[0], opr.argrefs[0], "int") + self.codewriter.cast(addr2, opr.argtypes[1], opr.argrefs[1], "int") + self.codewriter.binaryop("sub", opr.retref, opr.rettype, addr1, addr2) + + def _op_adr_generic(self, opr, llvm_op): + addr, res = self._tmp(2) + self.codewriter.cast(addr, opr.argtypes[0], opr.argrefs[0], "int") + self.codewriter.binaryop(llvm_op, res, "int", addr, opr.argrefs[1]) + self.codewriter.cast(opr.retref, "int", res, opr.rettype) + + def adr_add(self, opr): + self._op_adr_generic(opr, "add") + + def adr_sub(self, opr): + self._op_adr_generic(opr, "sub") + + def _op_adr_cmp(self, opr, llvm_op): + addr1, addr2 = self._tmp(2) + self.codewriter.cast(addr1, opr.argtypes[0], opr.argrefs[0], "int") + self.codewriter.cast(addr2, opr.argtypes[1], opr.argrefs[1], "int") + assert opr.rettype == "bool" + self.codewriter.binaryop(llvm_op, opr.retref, "int", addr1, addr2) + + def adr_eq(self, opr): + self._op_adr_cmp(opr, "seteq") + + def adr_ne(self, opr): + self._op_adr_cmp(opr, "setne") + + def adr_le(self, opr): + self._op_adr_cmp(opr, "setle") + + def adr_gt(self, opr): + self._op_adr_cmp(opr, "setgt") + + def adr_lt(self, opr): + self._op_adr_cmp(opr, "setlt") + + def adr_ge(self, opr): + self._op_adr_cmp(opr, "setge") + + # XXX Not sure any of this makes sense - maybe seperate policy for + # different flavours of mallocs? Well it depend on what happens the GC + # developments + def raw_malloc(self, opr): + self.codewriter.call(opr.retref, opr.rettype, "%raw_malloc", + opr.argtypes, opr.argrefs) + def raw_free(self, opr): + self.codewriter.call(opr.retref, opr.rettype, "%raw_free", + opr.argtypes, opr.argrefs) + + def raw_memcopy(self, opr): + self.codewriter.call(opr.retref, opr.rettype, "%raw_memcopy", + opr.argtypes, opr.argrefs) - (arg_addr, arg_dummy, - arg_incr, arg_value) = self.db.repr_arg_multi(op.args) + def raw_store(self, opr): + arg_addr, arg_dummy, arg_incr, arg_value = opr.argrefs (argtype_addr, argtype_dummy, - argtype_incr, argtype_value) = self.db.repr_arg_type_multi(op.args) + argtype_incr, argtype_value) = opr.argtypes - cast_addr = tmp() + cast_addr = self._tmp() addr_type = argtype_value + "*" # cast to the correct type before arithmetic/storing @@ -542,168 +424,54 @@ # pointer arithmetic if arg_incr: - incr_addr = tmp() - self.codewriter.raw_getelementptr(incr_addr, - addr_type, - cast_addr, - ("int", arg_incr)) + incr_addr = self._tmp() + self.codewriter.getelementptr(incr_addr, + addr_type, + cast_addr, + [("int", arg_incr)], + getptr=False) cast_addr = incr_addr self.codewriter.store(argtype_value, arg_value, cast_addr) - def raw_load(self, op): - tmp = self.db.repr_tmpvar - - arg_addr, arg_dummy, arg_incr = self.db.repr_arg_multi(op.args) - argtype_addr, argtype_dummy, argtype_incr = \ - self.db.repr_arg_type_multi(op.args) - targetvar = self.db.repr_arg(op.result) - targettype = self.db.repr_arg_type(op.result) + def raw_load(self, opr): + arg_addr, arg_dummy, arg_incr = opr.argrefs + argtype_addr, argtype_dummy, argtype_incr = opr.argtypes - cast_addr = tmp() - addr_type = targettype + "*" + cast_addr = self._tmp() + addr_type = opr.rettype + "*" # cast to the correct type before arithmetic/loading self.codewriter.cast(cast_addr, argtype_addr, arg_addr, addr_type) + # pointer arithmetic if arg_incr: - incr_addr = tmp() - self.codewriter.raw_getelementptr(incr_addr, - addr_type, - cast_addr, - ("int", arg_incr)) + incr_addr = self._tmp() + self.codewriter.getelementptr(incr_addr, + addr_type, + cast_addr, + [("int", arg_incr)], + getptr=False) cast_addr = incr_addr - self.codewriter.load(targetvar, targettype, cast_addr) + self.codewriter.load(opr.retref, opr.rettype, cast_addr) # ______________________________________________________________________ - # exception specific + # + # XXX exception specific - move to policy? - def last_exception_type_ptr(self, op): + def malloc_exception(self, opr): + tmpvar1, tmpvar2, tmpvar3 = self._tmp(3) + cw = self.codewriter + cw.getelementptr(tmpvar1, opr.rettype, "null", [("int", 1)], getptr=False) + cw.cast(tmpvar2, opr.rettype, tmpvar1, 'uint') + cw.call(tmpvar3, 'sbyte*', '%malloc_exception', ['uint'], [tmpvar2]) + cw.cast(opr.retref, 'sbyte*', tmpvar3, opr.rettype) + + def last_exception_type_ptr(self, opr): + op = opr.op e = self.db.translator.rtyper.getexceptiondata() self.codewriter.load('%' + str(op.result), self.db.repr_type(e.lltype_of_exception_type), '%last_exception_type') - def invoke(self, op): - ep = self.db.exceptionpolicy - - op_args = [arg for arg in op.args - if arg.concretetype is not lltype.Void] - - if op.opname == 'invoke:direct_call': - functionref = self.db.repr_arg(op_args[0]) - - else: - # operation - provided by genexterns - opname = op.opname.split(':', 1)[1] - op_args = ['%pypyop_' + opname] + op_args - functionref = op_args[0] - - assert len(op_args) >= 1 - - # at least one label and one exception label - assert len(self.block.exits) >= 2 - - link = self.block.exits[0] - assert link.exitcase is None - - targetvar = self.db.repr_arg(op.result) - returntype = self.db.repr_arg_type(op.result) - argrefs = self.db.repr_arg_multi(op_args[1:]) - argtypes = self.db.repr_arg_type_multi(op_args[1:]) - - none_label = self.node.block_to_name[link.target] - block_label = self.node.block_to_name[self.block] - exc_label = block_label + '_exception_handling' - - # use longhand form - if self.db.is_function_ptr(op.result): - returntype = "%s (%s)*" % (returntype, ", ".join(argtypes)) - - ep.invoke(self.codewriter, targetvar, returntype, functionref, - argrefs, argtypes, none_label, exc_label) - - # write exception handling blocks - - e = self.db.translator.rtyper.getexceptiondata() - ll_exception_match = self.db.repr_value(e.fn_exception_match._obj) - lltype_of_exception_type = self.db.repr_type(e.lltype_of_exception_type) - lltype_of_exception_value = self.db.repr_type(e.lltype_of_exception_value) - - # start with the exception handling block - # * load the last exception type - # * check it with call to ll_exception_match() - # * branch to to correct block? - - self.codewriter.label(exc_label) - - catch_all = False - found_blocks_info = [] - last_exception_type = None - - # XXX tmp - debugging info - - # block_label = "block28" - # exc_label = "block28_exception_handling" - # ll_exception_match = function for catching exception - # lltype_of_exception_type, lltype_of_exception_value = generic - # catch_all = ??? - # found_blocks_info = list of found block data to write those blocks - # last_exception_type = Load exception pointer once for handle and not found blocks - - # link = iteration thru rest of links in block - # etype = node for exception - # current_exception_type = repr for node etype - # target = label of the destination block - # exc_found_label = label of intermediate exc found block - # last_exc_type_var = ???? - # last_exc_value_var = ??? - - for link in self.block.exits[1:]: - assert issubclass(link.exitcase, Exception) - - # information for found blocks - target = self.node.block_to_name[link.target] - exc_found_label = block_label + '_exception_found_branchto_' + target - link_exc_type = repr_if_variable(self.db, link.last_exception) - link_exc_value = repr_if_variable(self.db, link.last_exc_value) - found_blocks_info.append((exc_found_label, target, - link_exc_type, link_exc_value)) - - # XXX fix database to handle this case - etype = self.db.obj2node[link.llexitcase._obj] - current_exception_type = etype.get_ref() - not_this_exception_label = block_label + '_not_exception_' + etype.ref[1:] - - # catch specific exception (class) type - - # load pointer only once - if not last_exception_type: - last_exception_type = self.db.repr_tmpvar() - self.codewriter.load(last_exception_type, - lltype_of_exception_type, - '%last_exception_type') - self.codewriter.newline() - - ll_issubclass_cond = self.db.repr_tmpvar() - - self.codewriter.call(ll_issubclass_cond, - 'bool', - ll_exception_match, - [last_exception_type, current_exception_type], - [lltype_of_exception_type, lltype_of_exception_type]) - - self.codewriter.br(ll_issubclass_cond, - not_this_exception_label, - exc_found_label) - - self.codewriter.label(not_this_exception_label) - - if not catch_all: - ep.reraise(self.node, self.codewriter) - - ep.fetch_exceptions(self.codewriter, - found_blocks_info, - lltype_of_exception_type, - lltype_of_exception_value) Modified: pypy/dist/pypy/translator/llvm/structnode.py ============================================================================== --- pypy/dist/pypy/translator/llvm/structnode.py (original) +++ pypy/dist/pypy/translator/llvm/structnode.py Fri Jan 6 19:31:20 2006 @@ -201,25 +201,25 @@ super(StructVarsizeNode, self).setup() def get_typerepr(self): - try: - return self._get_typerepr_cache - except: - # last type is a special case and need to be worked out recursively - types = self._get_types[:-1] - types_repr = [self.db.repr_type(T) for name, T in types] - types_repr.append(self._get_lastnode().get_typerepr()) - result = "{%s}" % ", ".join(types_repr) - self._get_typerepr_cache = result - return result + try: + return self._get_typerepr_cache + except: + # last type is a special case and need to be worked out recursively + types = self._get_types[:-1] + types_repr = [self.db.repr_type(T) for name, T in types] + types_repr.append(self._get_lastnode().get_typerepr()) + result = "{%s}" % ", ".join(types_repr) + self._get_typerepr_cache = result + return result def get_ref(self): if self._get_ref_cache: return self._get_ref_cache ref = super(StructVarsizeNode, self).get_ref() typeval = self.db.repr_type(lltype.typeOf(self.value)) - ref = "cast (%s* %s to %s*)" % (self.get_typerepr(), - ref, - typeval) + ref = "cast(%s* %s to %s*)" % (self.get_typerepr(), + ref, + typeval) self._get_ref_cache = ref return ref @@ -229,6 +229,6 @@ p, c = lltype.parentlink(self.value) assert p is None, "child arrays are NOT needed by rtyper" fromptr = "%s*" % self.get_typerepr() - refptr = "getelementptr (%s %s, int 0)" % (fromptr, ref) + refptr = "getelementptr(%s %s, int 0)" % (fromptr, ref) ref = "cast(%s %s to %s)" % (fromptr, refptr, toptr) return ref From rxe at codespeak.net Fri Jan 6 20:58:40 2006 From: rxe at codespeak.net (rxe at codespeak.net) Date: Fri, 6 Jan 2006 20:58:40 +0100 (CET) Subject: [pypy-svn] r21757 - in pypy/dist/pypy/translator/llvm: . module Message-ID: <20060106195840.83BC627B62@code1.codespeak.net> Author: rxe Date: Fri Jan 6 20:58:38 2006 New Revision: 21757 Modified: pypy/dist/pypy/translator/llvm/exception.py pypy/dist/pypy/translator/llvm/genllvm.py pypy/dist/pypy/translator/llvm/module/support.py Log: shuffle code around - all llvm/c code is now in module Modified: pypy/dist/pypy/translator/llvm/exception.py ============================================================================== --- pypy/dist/pypy/translator/llvm/exception.py (original) +++ pypy/dist/pypy/translator/llvm/exception.py Fri Jan 6 20:58:38 2006 @@ -2,30 +2,14 @@ from pypy.translator.llvm.codewriter import DEFAULT_CCONV from pypy.translator.llvm.backendopt.exception import create_exception_handling +from pypy.translator.llvm.module.excsupport import \ + ringbuffer_decl, ringbuffer_code, invokeunwind_code, explicit_code -class ExceptionPolicy: - RINGBUGGER_SIZE = 8192 - RINGBUFFER_ENTRY_MAXSIZE = 16 - RINGBUGGER_OVERSIZE = RINGBUGGER_SIZE + RINGBUFFER_ENTRY_MAXSIZE - RINGBUFFER_LLVMCODE = ''' -internal fastcc sbyte* %%malloc_exception(uint %%nbytes) { - %%cond = setle uint %%nbytes, %d - br bool %%cond, label %%then, label %%else - -then: - %%tmp.3 = load uint* %%exception_ringbuffer_index - %%tmp.4 = getelementptr [%d x sbyte]* %%exception_ringbuffer, int 0, uint %%tmp.3 - %%tmp.6 = add uint %%tmp.3, %%nbytes - %%tmp.7 = and uint %%tmp.6, %d - store uint %%tmp.7, uint* %%exception_ringbuffer_index - ret sbyte* %%tmp.4 - -else: - %%tmp.8 = call ccc sbyte* %%pypy_malloc(uint %%nbytes) - ret sbyte* %%tmp.8 -} -''' % (RINGBUFFER_ENTRY_MAXSIZE, RINGBUGGER_OVERSIZE, RINGBUGGER_SIZE-1) +def repr_if_variable(db, arg): + if isinstance(arg, Variable): + return db.repr_arg(arg) +class ExceptionPolicy: def __init__(self, db): self.db = db raise Exception, 'ExceptionPolicy should not be used directly' @@ -33,6 +17,16 @@ def transform(self, translator, graph=None): return + def update_phi_data(self, funcnode, entrylinks, block, blocknames): + """ Exceptions handling code introduces intermediate blocks for + exception handling cases, hence we modify our input phi data + accordingly. """ + for ii, link in enumerate(entrylinks): + if (link.prevblock.exitswitch == c_last_exception and + link.prevblock.exits[0].target != block): + blocknames[ii] += '_exception_found_branchto_' + blocknames[ii] += funcnode.block_to_name[block] + def _noresult(self, returntype): r = returntype.strip() if r == 'void': @@ -54,7 +48,7 @@ elif r in 'float double'.split(): return '0.0' elif r in 'ubyte sbyte ushort short uint int ulong long'.split(): - return ' 0' + return '0' return 'null' def _nonoderesult(self, node): @@ -75,16 +69,6 @@ return exceptionpolicy new = staticmethod(new) - def update_phi_data(self, funcnode, entrylinks, block, blocknames): - """ Exceptions handling code introduces intermediate blocks for - exception handling cases, hence we modify our input phi data - accordingly. """ - for ii, link in enumerate(entrylinks): - if (link.prevblock.exitswitch == c_last_exception and - link.prevblock.exits[0].target != block): - blocknames[ii] += '_exception_found_branchto_' - blocknames[ii] += funcnode.block_to_name[block] - class NoneExceptionPolicy(ExceptionPolicy): """ XXX untested """ @@ -98,41 +82,34 @@ def __init__(self): pass - def llvmcode(self, entrynode): - returntype, entrypointname = entrynode.getdecl().split('%', 1) + def llvm_declcode(self): + return '' + + def llvm_implcode(self, entrynode): + returntype, entrypointname = entrynode.getdecl().split('%', 1) noresult = self._noresult(returntype) cconv = DEFAULT_CCONV - return ''' -ccc %(returntype)s%%__entrypoint__%(entrypointname)s { - %%result = invoke %(cconv)s %(returntype)s%%%(entrypointname)s to label %%no_exception except label %%exception - -no_exception: - store %%RPYTHON_EXCEPTION_VTABLE* null, %%RPYTHON_EXCEPTION_VTABLE** %%last_exception_type - ret %(returntype)s %%result - -exception: - ret %(noresult)s -} - -ccc int %%__entrypoint__raised_LLVMException() { - %%tmp = load %%RPYTHON_EXCEPTION_VTABLE** %%last_exception_type - %%result = cast %%RPYTHON_EXCEPTION_VTABLE* %%tmp to int - ret int %%result -} - -internal fastcc void %%unwind() { - unwind -} -''' % locals() + self.RINGBUFFER_LLVMCODE + return invokeunwind_code % locals() + ringbuffer_code def invoke(self, codewriter, targetvar, tail_, cconv, returntype, functionref, args, label, except_label): labels = 'to label %%%s except label %%%s' % (label, except_label) if returntype == 'void': - codewriter._indent('%sinvoke %s void %s(%s) %s' % (tail_, cconv, functionref, args, labels)) + #XXX + codewriter._indent('%sinvoke %s void %s(%s) %s' % (tail_, + cconv, + functionref, + args, + labels)) else: - codewriter._indent('%s = %sinvoke %s %s %s(%s) %s' % (targetvar, tail_, cconv, returntype, functionref, args, labels)) + codewriter._indent('%s = %sinvoke %s %s %s(%s) %s' % (targetvar, + tail_, + cconv, + returntype, + functionref, + args, + labels)) def _is_raise_new_exception(self, db, graph, block): from pypy.objspace.flow.model import mkentrymap @@ -142,8 +119,11 @@ inputargs = db.repr_arg_multi(block.inputargs) for i, arg in enumerate(inputargs): names = db.repr_arg_multi([link.args[i] for link in entrylinks]) - for name in names: #These tests-by-name are a bit yikes, but I don't see a better way right now - if not name.startswith('%last_exception_') and not name.startswith('%last_exc_value_'): + # these tests-by-name are a bit yikes, but I don't see a better way + # right now + for name in names: + if (not name.startswith('%last_exception_') and + not name.startswith('%last_exc_value_')): is_raise_new = True return is_raise_new @@ -163,19 +143,24 @@ codewriter.store(inputargtypes[1], inputargs[1], '%last_exception_value') else: codewriter.comment('reraise last exception') - #Reraising last_exception. - #Which is already stored in the global variables. - #So nothing needs to happen here! + # Reraising last_exception. + # Which is already stored in the global variables. + # So nothing needs to happen here! codewriter.unwind() - def fetch_exceptions(self, codewriter, exc_found_labels, lltype_of_exception_type, lltype_of_exception_value): + def fetch_exceptions(self, codewriter, exc_found_labels, + lltype_of_exception_type, lltype_of_exception_value): for label, target, last_exc_type_var, last_exc_value_var in exc_found_labels: codewriter.label(label) if last_exc_type_var: - codewriter.load(last_exc_type_var, lltype_of_exception_type, '%last_exception_type') + codewriter.load(last_exc_type_var, + lltype_of_exception_type, + '%last_exception_type') if last_exc_value_var: - codewriter.load(last_exc_value_var, lltype_of_exception_value, '%last_exception_value') + codewriter.load(last_exc_value_var, + lltype_of_exception_value, + '%last_exception_value') codewriter.br_uncond(target) def reraise(self, funcnode, codewriter): @@ -185,48 +170,21 @@ def llc_options(self): return '-enable-correct-eh-support' - - -def repr_if_variable(db, arg): - if isinstance(arg, Variable): - return db.repr_arg(arg) - class ExplicitExceptionPolicy(ExceptionPolicy): """ uses issubclass() and last_exception tests after each call """ def __init__(self, db): self.db = db self.invoke_count = 0 - def llvmcode(self, entrynode): + def llvm_declcode(self): + return ringbuffer_decl + + def llvm_implcode(self, entrynode): returntype, entrypointname = entrynode.getdecl().split('%', 1) noresult = self._noresult(returntype) cconv = DEFAULT_CCONV - return ''' -ccc %(returntype)s%%__entrypoint__%(entrypointname)s { - store %%RPYTHON_EXCEPTION_VTABLE* null, %%RPYTHON_EXCEPTION_VTABLE** %%last_exception_type - %%result = call %(cconv)s %(returntype)s%%%(entrypointname)s - %%tmp = load %%RPYTHON_EXCEPTION_VTABLE** %%last_exception_type - %%exc = seteq %%RPYTHON_EXCEPTION_VTABLE* %%tmp, null - br bool %%exc, label %%no_exception, label %%exception - -no_exception: - ret %(returntype)s %%result - -exception: - ret %(noresult)s -} - -ccc int %%__entrypoint__raised_LLVMException() { - %%tmp = load %%RPYTHON_EXCEPTION_VTABLE** %%last_exception_type - %%result = cast %%RPYTHON_EXCEPTION_VTABLE* %%tmp to int - ret int %%result -} - -internal fastcc void %%unwind() { - ret void -} -''' % locals() + self.RINGBUFFER_LLVMCODE - + return explicit_code % locals() + ringbuffer_code + def transform(self, translator, graph=None): if graph: create_exception_handling(translator, graph) Modified: pypy/dist/pypy/translator/llvm/genllvm.py ============================================================================== --- pypy/dist/pypy/translator/llvm/genllvm.py (original) +++ pypy/dist/pypy/translator/llvm/genllvm.py Fri Jan 6 20:58:38 2006 @@ -111,6 +111,10 @@ self.write_extern_decls(codewriter) self._checkpoint('write externs type declarations') + # write exception declarations + ep = self.db.exceptionpolicy + codewriter.write_lines(ep.llvm_declcode()) + # write node type declarations for typ_decl in self.db.getnodes(): typ_decl.writedatatypedecl(codewriter) @@ -148,7 +152,7 @@ # write exception implementaions ep = self.db.exceptionpolicy - codewriter.write_lines(ep.llvmcode(self.entrynode)) + codewriter.write_lines(ep.llvm_implcode(self.entrynode)) # write all node implementations for typ_decl in self.db.getnodes(): Modified: pypy/dist/pypy/translator/llvm/module/support.py ============================================================================== --- pypy/dist/pypy/translator/llvm/module/support.py (original) +++ pypy/dist/pypy/translator/llvm/module/support.py Fri Jan 6 20:58:38 2006 @@ -3,11 +3,6 @@ %last_exception_type = internal global %RPYTHON_EXCEPTION_VTABLE* null %last_exception_value = internal global %RPYTHON_EXCEPTION* null -; XXXX This in a translation option - move to exception.py! -;8208=8192+16 in the next line because the last one (16 bytes maxsize) might start at 8190 for instance. -%exception_ringbuffer = internal global [8208 x sbyte] zeroinitializer -%exception_ringbuffer_index = internal global uint 0 - declare ccc uint %strlen(sbyte*) declare ccc void %llvm.memset(sbyte*, ubyte, uint, uint) declare ccc void %llvm.memcpy(sbyte*, sbyte*, uint, uint) From rxe at codespeak.net Fri Jan 6 21:57:24 2006 From: rxe at codespeak.net (rxe at codespeak.net) Date: Fri, 6 Jan 2006 21:57:24 +0100 (CET) Subject: [pypy-svn] r21760 - pypy/dist/pypy/translator/llvm/module Message-ID: <20060106205724.5FB0027B62@code1.codespeak.net> Author: rxe Date: Fri Jan 6 21:57:23 2006 New Revision: 21760 Added: pypy/dist/pypy/translator/llvm/module/excsupport.py (contents, props changed) Log: Ooops forgot to checkin. Added: pypy/dist/pypy/translator/llvm/module/excsupport.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/llvm/module/excsupport.py Fri Jan 6 21:57:23 2006 @@ -0,0 +1,79 @@ +RINGBUFFER_SIZE = 8192 +RINGBUFFER_ENTRY_MAXSIZE = 16 +RINGBUFFER_OVERSIZE = RINGBUFFER_SIZE + RINGBUFFER_ENTRY_MAXSIZE + +ringbuffer_decl = """ +; 8208=8192+16 in the next line because the last one (16 bytes maxsize) might +; start at 8190 for instance. [RINGBUFFER_SIZE + RINGBUFFER_ENTRY_MAXSIZE] + +%%exception_ringbuffer = internal global [%s x sbyte] zeroinitializer +%%exception_ringbuffer_index = internal global uint 0 +""" % (RINGBUFFER_SIZE + RINGBUFFER_ENTRY_MAXSIZE) + +ringbuffer_code = ''' +internal fastcc sbyte* %%malloc_exception(uint %%nbytes) { + %%cond = setle uint %%nbytes, %d + br bool %%cond, label %%then, label %%else + +then: + %%tmp.3 = load uint* %%exception_ringbuffer_index + %%tmp.4 = getelementptr [%d x sbyte]* %%exception_ringbuffer, int 0, uint %%tmp.3 + %%tmp.6 = add uint %%tmp.3, %%nbytes + %%tmp.7 = and uint %%tmp.6, %d + store uint %%tmp.7, uint* %%exception_ringbuffer_index + ret sbyte* %%tmp.4 + +else: + %%tmp.8 = call ccc sbyte* %%pypy_malloc(uint %%nbytes) + ret sbyte* %%tmp.8 +} +''' % (RINGBUFFER_ENTRY_MAXSIZE, RINGBUFFER_OVERSIZE, RINGBUFFER_SIZE - 1) + +invokeunwind_code = ''' +ccc %(returntype)s%%__entrypoint__%(entrypointname)s { + %%result = invoke %(cconv)s %(returntype)s%%%(entrypointname)s to label %%no_exception except label %%exception + +no_exception: + store %%RPYTHON_EXCEPTION_VTABLE* null, %%RPYTHON_EXCEPTION_VTABLE** %%last_exception_type + ret %(returntype)s %%result + +exception: + ret %(noresult)s +} + +ccc int %%__entrypoint__raised_LLVMException() { + %%tmp = load %%RPYTHON_EXCEPTION_VTABLE** %%last_exception_type + %%result = cast %%RPYTHON_EXCEPTION_VTABLE* %%tmp to int + ret int %%result +} + +internal fastcc void %%unwind() { + unwind +} +''' + +explicit_code = ''' +ccc %(returntype)s%%__entrypoint__%(entrypointname)s { + store %%RPYTHON_EXCEPTION_VTABLE* null, %%RPYTHON_EXCEPTION_VTABLE** %%last_exception_type + %%result = call %(cconv)s %(returntype)s%%%(entrypointname)s + %%tmp = load %%RPYTHON_EXCEPTION_VTABLE** %%last_exception_type + %%exc = seteq %%RPYTHON_EXCEPTION_VTABLE* %%tmp, null + br bool %%exc, label %%no_exception, label %%exception + +no_exception: + ret %(returntype)s %%result + +exception: + ret %(noresult)s +} + +ccc int %%__entrypoint__raised_LLVMException() { + %%tmp = load %%RPYTHON_EXCEPTION_VTABLE** %%last_exception_type + %%result = cast %%RPYTHON_EXCEPTION_VTABLE* %%tmp to int + ret int %%result +} + +internal fastcc void %%unwind() { + ret void +} +''' From cfbolz at codespeak.net Fri Jan 6 22:05:19 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Fri, 6 Jan 2006 22:05:19 +0100 (CET) Subject: [pypy-svn] r21761 - in pypy/dist/pypy: jit rpython translator translator/backendopt translator/backendopt/test translator/c translator/c/test translator/js translator/llvm/backendopt translator/locality Message-ID: <20060106210519.9153527B62@code1.codespeak.net> Author: cfbolz Date: Fri Jan 6 22:05:13 2006 New Revision: 21761 Modified: pypy/dist/pypy/jit/llabstractinterp.py pypy/dist/pypy/rpython/llinterp.py pypy/dist/pypy/rpython/rpbc.py pypy/dist/pypy/rpython/rptr.py pypy/dist/pypy/translator/backendopt/inline.py pypy/dist/pypy/translator/backendopt/propagate.py pypy/dist/pypy/translator/backendopt/removenoops.py pypy/dist/pypy/translator/backendopt/test/test_malloc.py pypy/dist/pypy/translator/c/funcgen.py pypy/dist/pypy/translator/c/gc.py pypy/dist/pypy/translator/c/test/test_annotated.py pypy/dist/pypy/translator/js/opwriter.py pypy/dist/pypy/translator/llvm/backendopt/exception.py pypy/dist/pypy/translator/locality/calltree.py pypy/dist/pypy/translator/simplify.py Log: introduce a new ll operation: indirect_call. It is used if the function that is being called is a Variable. This makes attaching of the possibly called functions to the operation (which I will be doing next) much cleaner, as I can abuse an argument for it. Modified: pypy/dist/pypy/jit/llabstractinterp.py ============================================================================== --- pypy/dist/pypy/jit/llabstractinterp.py (original) +++ pypy/dist/pypy/jit/llabstractinterp.py Fri Jan 6 22:05:13 2006 @@ -312,9 +312,9 @@ for st in stlist: op = st.origblock.operations[st.origposition] if op.opname == 'direct_call': + v = v.value + elif op.opname == 'indirect_call': v = op.args[0] - if isinstance(v, Constant): - v = v.value else: v = '?' print 'In %r:' % (v,) @@ -673,6 +673,15 @@ a_result = self.residualize(op, args_a) return a_result + def op_indirect_call(self, op, *args_a): + # XXX not really sure what the right thing to do is: + # right now there is no test that produces indirect_call + # the slight ugliness involved is, that it is necessary to + # change the operation from an indirect_call to a direct_call + # when the first argument of the indirect_call is discovered to be + # constant + assert 0, "XXX" + def handle_call(self, op, a_func, *args_a): v_func = a_func.maybe_get_constant() if v_func is None: Modified: pypy/dist/pypy/rpython/llinterp.py ============================================================================== --- pypy/dist/pypy/rpython/llinterp.py (original) +++ pypy/dist/pypy/rpython/llinterp.py Fri Jan 6 22:05:13 2006 @@ -224,6 +224,11 @@ def eval_operation(self, operation): log.operation("considering", operation) ophandler = self.getoperationhandler(operation.opname) + # XXX slighly unnice but an important safety check + if operation.opname == 'direct_call': + assert isinstance(operation.args[0], Constant) + elif operation.opname == 'indirect_call': + assert isinstance(operation.args[0], Variable) vals = [self.getval(x) for x in operation.args] # if these special cases pile up, do something better here if operation.opname in ['cast_pointer', 'ooupcast', 'oodowncast']: @@ -314,6 +319,8 @@ frame = self.__class__(graph, args, self.llinterpreter, self) return frame.eval() + op_indirect_call = op_direct_call # XXX for now + def op_malloc(self, obj): if self.llinterpreter.gc is not None: args = self.llinterpreter.gc.get_arg_malloc(obj) Modified: pypy/dist/pypy/rpython/rpbc.py ============================================================================== --- pypy/dist/pypy/rpython/rpbc.py (original) +++ pypy/dist/pypy/rpython/rpbc.py Fri Jan 6 22:05:13 2006 @@ -312,7 +312,10 @@ vlist += callparse.callparse(self.rtyper, anygraph, hop, opname) rresult = callparse.getrresult(self.rtyper, anygraph) hop.exception_is_here() - v = hop.genop('direct_call', vlist, resulttype = rresult) + if isinstance(vlist[0], Constant): + v = hop.genop('direct_call', vlist, resulttype = rresult) + else: + v = hop.genop('indirect_call', vlist, resulttype = rresult) return hop.llops.convertvar(v, rresult, hop.r_result) class __extend__(pairtype(FunctionsPBCRepr, FunctionsPBCRepr)): Modified: pypy/dist/pypy/rpython/rptr.py ============================================================================== --- pypy/dist/pypy/rpython/rptr.py (original) +++ pypy/dist/pypy/rpython/rptr.py Fri Jan 6 22:05:13 2006 @@ -62,8 +62,11 @@ if isinstance(vlist[0], flowmodel.Constant): if hasattr(vlist[0].value, 'graph'): hop.llops.record_extra_call(vlist[0].value.graph) + opname = 'direct_call' + else: + opname = 'indirect_call' hop.exception_is_here() - return hop.genop('direct_call', vlist, + return hop.genop(opname, vlist, resulttype = self.lowleveltype.TO.RESULT) Modified: pypy/dist/pypy/translator/backendopt/inline.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/inline.py (original) +++ pypy/dist/pypy/translator/backendopt/inline.py Fri Jan 6 22:05:13 2006 @@ -34,8 +34,7 @@ def visit(block): if isinstance(block, Block): for i, op in enumerate(block.operations): - if not (op.opname == "direct_call" and - isinstance(op.args[0], Constant)): + if not op.opname == "direct_call": continue funcobj = op.args[0].value._obj graph = getattr(funcobj, 'graph', None) @@ -335,8 +334,7 @@ def build_call_graph(node): if isinstance(node, Block): for op in node.operations: - if (op.opname == "direct_call" and - isinstance(op.args[0], Constant)): + if op.opname == "direct_call": funcobj = op.args[0].value._obj graph = getattr(funcobj, 'graph', None) if graph is not None: Modified: pypy/dist/pypy/translator/backendopt/propagate.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/propagate.py (original) +++ pypy/dist/pypy/translator/backendopt/propagate.py Fri Jan 6 22:05:13 2006 @@ -118,7 +118,8 @@ _op = """getarrayitem setarrayitem malloc malloc_varsize flavored_malloc flavored_free getfield setfield getsubstruct getarraysubstruct getarraysize raw_malloc raw_free raw_memcopy raw_load - raw_store direct_call cast_pointer cast_ptr_to_int""".split() + raw_store direct_call indirect_call cast_pointer + cast_ptr_to_int""".split() from pypy.objspace.flow.operation import FunctionByName _op += FunctionByName.keys() #operations with PyObjects are dangerous cannot_constant_fold = {} Modified: pypy/dist/pypy/translator/backendopt/removenoops.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/removenoops.py (original) +++ pypy/dist/pypy/translator/backendopt/removenoops.py Fri Jan 6 22:05:13 2006 @@ -56,7 +56,7 @@ graph.startblock.inputargs = args for block in graph.iterblocks(): for op in block.operations: - if op.opname == 'direct_call': + if op.opname in ('direct_call', 'indirect_call'): args = [arg for arg in op.args if arg.concretetype is not Void] op.args = args Modified: pypy/dist/pypy/translator/backendopt/test/test_malloc.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/test/test_malloc.py (original) +++ pypy/dist/pypy/translator/backendopt/test/test_malloc.py Fri Jan 6 22:05:13 2006 @@ -13,10 +13,10 @@ for op in node.operations: if op.opname == 'malloc': count1 += 1 - if op.opname == 'direct_call': + if op.opname in ('direct_call', 'indirect_call'): count2 += 1 assert count1 == 0 # number of mallocs left - assert count2 == 0 # number of direct_calls left + assert count2 == 0 # number of calls left def check(fn, signature, args, expected_result, must_be_removed=True): t = TranslationContext() Modified: pypy/dist/pypy/translator/c/funcgen.py ============================================================================== --- pypy/dist/pypy/translator/c/funcgen.py (original) +++ pypy/dist/pypy/translator/c/funcgen.py Fri Jan 6 22:05:13 2006 @@ -434,6 +434,8 @@ line = '%s\n%s' % (line, self.check_directcall_result(op, err)) return line + OP_INDIRECT_CALL = OP_DIRECT_CALL # XXX for now + def check_directcall_result(self, op, err): return 'if (RPyExceptionOccurred())\n\tFAIL(%s);' % err Modified: pypy/dist/pypy/translator/c/gc.py ============================================================================== --- pypy/dist/pypy/translator/c/gc.py (original) +++ pypy/dist/pypy/translator/c/gc.py Fri Jan 6 22:05:13 2006 @@ -99,7 +99,7 @@ return 'pypy_DecRf_%s(%s);' % (defnode.barename, expr) def push_alive_op_result(self, opname, expr, T): - if opname !='direct_call' and T != PyObjPtr: + if opname not in ('direct_call', 'indirect_call') and T != PyObjPtr: return self.push_alive(expr, T) return '' Modified: pypy/dist/pypy/translator/c/test/test_annotated.py ============================================================================== --- pypy/dist/pypy/translator/c/test/test_annotated.py (original) +++ pypy/dist/pypy/translator/c/test/test_annotated.py Fri Jan 6 22:05:13 2006 @@ -266,4 +266,19 @@ def f(i=r_longlong): return int(i) fn = self.getcompiled(f) - assert f(0) == 0 + assert fn(0) == 0 + + def test_function_ptr(self): + def f1(): + return 1 + def f2(): + return 2 + def g(i=int): + if i: + f = f1 + else: + f = f2 + return f() + fn = self.getcompiled(g) + assert fn(0) == 2 + assert fn(1) == 1 Modified: pypy/dist/pypy/translator/js/opwriter.py ============================================================================== --- pypy/dist/pypy/translator/js/opwriter.py (original) +++ pypy/dist/pypy/translator/js/opwriter.py Fri Jan 6 22:05:13 2006 @@ -228,6 +228,8 @@ argrefs = self.db.repr_arg_multi(op_args[1:]) self.codewriter.call(targetvar, functionref, argrefs) + indirect_call = direct_call # XXX for now + def invoke(self, op): op_args = [arg for arg in op.args if arg.concretetype is not lltype.Void] Modified: pypy/dist/pypy/translator/llvm/backendopt/exception.py ============================================================================== --- pypy/dist/pypy/translator/llvm/backendopt/exception.py (original) +++ pypy/dist/pypy/translator/llvm/backendopt/exception.py Fri Jan 6 22:05:13 2006 @@ -8,7 +8,8 @@ n_calls = n_calls_patched = 0 def create_exception_handling(translator, graph): - """After an exception in a direct_call, that is not catched by an explicit + """After an exception in a direct_call (or indirect_call), that is not caught + by an explicit except statement, we need to reraise the exception. So after this direct_call we need to test if an exception had occurred. If so, we return from the current graph with an unused value (false/0/0.0/null). @@ -23,7 +24,7 @@ last_operation -= 1 for i in range(last_operation, -1, -1): op = block.operations[i] - if op.opname != 'direct_call': + if op.opname not in ('direct_call', 'indirect_call'): continue n_calls += 1 called_can_raise = True #XXX maybe we even want a list of possible exceptions Modified: pypy/dist/pypy/translator/locality/calltree.py ============================================================================== --- pypy/dist/pypy/translator/locality/calltree.py (original) +++ pypy/dist/pypy/translator/locality/calltree.py Fri Jan 6 22:05:13 2006 @@ -68,21 +68,20 @@ for op in block.operations: if op.opname == 'direct_call': fnarg = op.args[0] - if isinstance(fnarg, Constant): - fnptr = fnarg.value - fn = fnptr._obj - graph = fn.graph - try: - callednode = self.graphs2nodes[graph] - except KeyError: - s = "No node found for graph %s" % graph.name - log.calltree.findCallees(s) - continue - else: - res.append(callednode) - else: - s = "Node %s calls Variable %s" % (node.name, fnarg) + fnptr = fnarg.value + fn = fnptr._obj + graph = fn.graph + try: + callednode = self.graphs2nodes[graph] + except KeyError: + s = "No node found for graph %s" % graph.name log.calltree.findCallees(s) + continue + else: + res.append(callednode) + elif op.opname == 'indirect_call': + s = "Node %s calls Variable %s" % (node.name, op.args[0]) + log.calltree.findCallees(s) return res def simulate(self): Modified: pypy/dist/pypy/translator/simplify.py ============================================================================== --- pypy/dist/pypy/translator/simplify.py (original) +++ pypy/dist/pypy/translator/simplify.py Fri Jan 6 22:05:13 2006 @@ -378,13 +378,13 @@ if op.opname in lloperations_with_side_effects: raise HasSideEffects if op.opname == "direct_call": - if isinstance(op.args[0], Variable): - raise HasSideEffects g = get_graph(op.args[0], translator) if g is None: raise HasSideEffects if not has_no_side_effects(translator, g, seen + [graph]): raise HasSideEffects + elif op.opname == "indirect_call": + raise HasSideEffects traverse(visit, graph) except HasSideEffects: return False From cfbolz at codespeak.net Fri Jan 6 22:18:28 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Fri, 6 Jan 2006 22:18:28 +0100 (CET) Subject: [pypy-svn] r21763 - pypy/dist/pypy/translator/llvm Message-ID: <20060106211828.A613527B62@code1.codespeak.net> Author: cfbolz Date: Fri Jan 6 22:18:28 2006 New Revision: 21763 Modified: pypy/dist/pypy/translator/llvm/opwriter.py Log: make llvm work after the split of direct_call into direct_call and indirect_call Modified: pypy/dist/pypy/translator/llvm/opwriter.py ============================================================================== --- pypy/dist/pypy/translator/llvm/opwriter.py (original) +++ pypy/dist/pypy/translator/llvm/opwriter.py Fri Jan 6 22:18:28 2006 @@ -30,7 +30,7 @@ def __init__(self, op, db): super(OpReprInvoke, self).__init__(op, db) - if op.opname == 'direct_call': + if op.opname in ('direct_call', 'indirect_call'): self.functionref = self.argrefs[0] self.argrefs = self.argrefs[1:] self.argtypes = self.argtypes[1:] @@ -89,7 +89,7 @@ return [self.db.repr_tmpvar() for ii in range(count)] def write_operation(self, op): - if op.opname == "direct_call": + if op.opname in ("direct_call", 'indirect_call'): opr = OpReprCall(op, self.db) else: opr = OpRepr(op, self.db) @@ -255,6 +255,8 @@ self.codewriter.call(opr.retref, opr.rettype, opr.argrefs[0], opr.argtypes[1:], opr.argrefs[1:]) + indirect_call = direct_call # XXX for now + def malloc(self, opr): arg_type = opr.op.args[0].value self.db.gcpolicy.malloc(self.codewriter, opr.retref, opr.rettype, From ericvrp at codespeak.net Fri Jan 6 23:30:37 2006 From: ericvrp at codespeak.net (ericvrp at codespeak.net) Date: Fri, 6 Jan 2006 23:30:37 +0100 (CET) Subject: [pypy-svn] r21764 - in pypy/dist/pypy/translator/js: . src test Message-ID: <20060106223037.65E7F27B62@code1.codespeak.net> Author: ericvrp Date: Fri Jan 6 23:30:35 2006 New Revision: 21764 Modified: pypy/dist/pypy/translator/js/codewriter.py pypy/dist/pypy/translator/js/opwriter.py pypy/dist/pypy/translator/js/src/ll_stackless.js pypy/dist/pypy/translator/js/src/stack.js pypy/dist/pypy/translator/js/test/runtest.py Log: * Decreased footprint of stackless javascript. (an unwind is now just one line of javascipt (calling a function) the resume is also just one line (calling eval())) * Renamed some functionnames, proper fix required later * Disabled if->switch transformation because there is no support here yet * Small fix for the indirect calls that raises an exception. (TODO: write a test for this!) Modified: pypy/dist/pypy/translator/js/codewriter.py ============================================================================== --- pypy/dist/pypy/translator/js/codewriter.py (original) +++ pypy/dist/pypy/translator/js/codewriter.py Fri Jan 6 23:30:35 2006 @@ -164,12 +164,7 @@ if self.js.stackless: # XXX and this funcnode has resumepoints self.append("if (slp_frame_stack_top) {") self.indent_more() - self.append('var t = slp_frame_stack_top.vars') - for i, k in enumerate(self._usedvars.keys()): - self.append('%-19s = t[%d]' % (k, i)) - self.append('%-19s = slp_frame_stack_top.resume_blocknum' % 'block') - self.append('eval(slp_frame_stack_top.targetvar + " = slp_return_value")') - self.append('slp_frame_stack_top = null') + self.append('eval(slp_frame_stack_top.resumecode)') self.indent_less() self.append('}') @@ -207,10 +202,11 @@ self.append("slp_stack_depth--") selfdecl = self.decl.split('(')[0] usedvars = ', '.join(self._usedvars.keys()) + usedvarnames = '"' + '", "'.join(self._usedvars.keys()) + '"' self.append('if (slp_frame_stack_bottom) {') self.indent_more() - self.append('slp_new_frame("%s", %s, %d, new Array(%s))' % - (targetvar, selfdecl, self._resume_blocknum, usedvars)) + self.append('slp_new_frame("%s", %s, %d, new Array(%s), new Array(%s))' % + (targetvar, selfdecl, self._resume_blocknum, usedvarnames, usedvars)) if specialreturnvalue: self.append("return " + specialreturnvalue) else: Modified: pypy/dist/pypy/translator/js/opwriter.py ============================================================================== --- pypy/dist/pypy/translator/js/opwriter.py (original) +++ pypy/dist/pypy/translator/js/opwriter.py Fri Jan 6 23:30:35 2006 @@ -234,7 +234,7 @@ op_args = [arg for arg in op.args if arg.concretetype is not lltype.Void] - if op.opname == 'invoke:direct_call': + if op.opname in ('invoke:direct_call', 'invoke:indirect_call'): functionref = self.db.repr_arg(op_args[0]) else: #operation opname = op.opname.split(':',1)[1] Modified: pypy/dist/pypy/translator/js/src/ll_stackless.js ============================================================================== --- pypy/dist/pypy/translator/js/src/ll_stackless.js (original) +++ pypy/dist/pypy/translator/js/src/ll_stackless.js Fri Jan 6 23:30:35 2006 @@ -32,7 +32,7 @@ } return result; } -ll_stackless_stack_frames_depth__ = ll_stackless_stack_frames_depth; +ll_stackless_stack_frames_depth___ = ll_stackless_stack_frames_depth; // @@ -50,30 +50,29 @@ } return result; } -ll_stack_too_big__ = ll_stack_too_big; +ll_stack_too_big___ = ll_stack_too_big; -function slp_new_frame(targetvar, func, resume_blocknum, vars) { - //LOG("slp_new_frame("+targetvar+","+function_name(func)+","+resume_blocknum+","+vars.toSource()+")"); +function slp_new_frame(targetvar, func, resume_blocknum, varnames, vars) { LOG("slp_new_frame("+function_name(func)+")"); - var f = new Object(); - f.func = func; - f.targetvar = targetvar; - f.resume_blocknum = resume_blocknum; - f.vars = vars; - f.f_back = null; + var f = new Object({func:func, vars:vars, f_back:null}); + var s = "block=" + resume_blocknum + ";"; + for (var i = 0;i < vars.length;i++) { + if (varnames[i] == targetvar) { + s += targetvar + "=slp_return_value;" + } else { + s += varnames[i] + "=slp_frame_stack_top.vars[" + i + "];"; + } + } + s += "slp_frame_stack_top=null;"; + f.resumecode = s slp_frame_stack_bottom.f_back = f; // push below bottom, to keep stack slp_frame_stack_bottom = f; // correctly sorted after unwind + LOG(f.resumecode); } function slp_new_frame_simple(func) { LOG("slp_new_frame_simple("+function_name(func)+")"); - var f = new Object(); - f.func = func; - f.targetvar = undefined; - f.resume_blocknum = undefined; - f.vars = undefined; - f.f_back = null; - return f; // note: the non-simple version returns nothing + return new Object({func:func, f_back:null}); // note: the non-simple version returns nothing } function ll_stack_unwind() { @@ -83,10 +82,10 @@ } else { slp_frame_stack_top = slp_frame_stack_bottom = slp_new_frame_simple(ll_stack_unwind); } - LOG('slp_frame_stack_top='+slp_frame_stack_top + ', slp_frame_stack_bottom='+slp_frame_stack_bottom) + LOG('slp_frame_stack_top='+slp_frame_stack_top + ', slp_frame_stack_bottom='+slp_frame_stack_bottom); return slp_return_value; } -ll_stack_unwind__ = ll_stack_unwind; +ll_stack_unwind___ = ll_stack_unwind; function slp_return_current_frame_to_caller() { LOG("slp_return_current_frame_to_caller"); @@ -102,7 +101,7 @@ LOG("slp_end_of_yielding_function"); if (!slp_frame_stack_top) log('slp_end_of_yielding_function !slp_frame_stack_top'); // can only resume from slp_return_current_frame_to_caller() if (!slp_return_value) log('slp_end_of_yielding_function !slp_return_value'); - LOG('slp_return_value is going to ' + function_name(slp_return_value.func)) + LOG('slp_return_value is going to ' + function_name(slp_return_value.func)); slp_frame_stack_top = slp_return_value; return null; } Modified: pypy/dist/pypy/translator/js/src/stack.js ============================================================================== --- pypy/dist/pypy/translator/js/src/stack.js (original) +++ pypy/dist/pypy/translator/js/src/stack.js Fri Jan 6 23:30:35 2006 @@ -14,11 +14,11 @@ } return false; } -ll_stack_too_big__ = ll_stack_too_big; +ll_stack_too_big___ = ll_stack_too_big; function ll_stack_unwind() { throw "Recursion limit exceeded"; } -ll_stack_unwind__ = ll_stack_unwind; +ll_stack_unwind___ = ll_stack_unwind; // End of helpers Modified: pypy/dist/pypy/translator/js/test/runtest.py ============================================================================== --- pypy/dist/pypy/translator/js/test/runtest.py (original) +++ pypy/dist/pypy/translator/js/test/runtest.py Fri Jan 6 23:30:35 2006 @@ -24,7 +24,7 @@ t = Translator(function) a = t.annotate(annotation) t.specialize() - t.backend_optimizations(inline_threshold=0, mallocs=False) + t.backend_optimizations(inline_threshold=0, mallocs=False, merge_if_blocks_to_switch=False) #t.backend_optimizations() if view: t.view() From cfbolz at codespeak.net Fri Jan 6 23:46:18 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Fri, 6 Jan 2006 23:46:18 +0100 (CET) Subject: [pypy-svn] r21765 - pypy/dist/pypy/rpython Message-ID: <20060106224618.9AB5727B62@code1.codespeak.net> Author: cfbolz Date: Fri Jan 6 23:46:16 2006 New Revision: 21765 Modified: pypy/dist/pypy/rpython/llinterp.py pypy/dist/pypy/rpython/rpbc.py pypy/dist/pypy/rpython/rptr.py Log: add a new last argument to indirect_calls. it is a Constant of a list of graphs that can be called by that call. Does not work in all cases, the most notable is rbuiltin.ll_instantiate Modified: pypy/dist/pypy/rpython/llinterp.py ============================================================================== --- pypy/dist/pypy/rpython/llinterp.py (original) +++ pypy/dist/pypy/rpython/llinterp.py Fri Jan 6 23:46:16 2006 @@ -319,7 +319,16 @@ frame = self.__class__(graph, args, self.llinterpreter, self) return frame.eval() - op_indirect_call = op_direct_call # XXX for now + def op_indirect_call(self, f, *args): + graphs = args[-1] + args = args[:-1] + if graphs is not None: + obj = self.llinterpreter.typer.type_system.deref(f) + if hasattr(obj, 'graph'): + assert obj.graph in graphs + else: + print "this should ideally not happen", f, graphs, args + return self.op_direct_call(f, *args) def op_malloc(self, obj): if self.llinterpreter.gc is not None: Modified: pypy/dist/pypy/rpython/rpbc.py ============================================================================== --- pypy/dist/pypy/rpython/rpbc.py (original) +++ pypy/dist/pypy/rpython/rpbc.py Fri Jan 6 23:46:16 2006 @@ -315,6 +315,10 @@ if isinstance(vlist[0], Constant): v = hop.genop('direct_call', vlist, resulttype = rresult) else: + print row_of_graphs + c = Constant(row_of_graphs.values()) + c.concretetype = Void + vlist.append(c) v = hop.genop('indirect_call', vlist, resulttype = rresult) return hop.llops.convertvar(v, rresult, hop.r_result) Modified: pypy/dist/pypy/rpython/rptr.py ============================================================================== --- pypy/dist/pypy/rpython/rptr.py (original) +++ pypy/dist/pypy/rpython/rptr.py Fri Jan 6 23:46:16 2006 @@ -65,6 +65,9 @@ opname = 'direct_call' else: opname = 'indirect_call' + c = flowmodel.Constant(None) + c.concretetype = Void + vlist.append(c) hop.exception_is_here() return hop.genop(opname, vlist, resulttype = self.lowleveltype.TO.RESULT) From rxe at codespeak.net Sat Jan 7 00:04:25 2006 From: rxe at codespeak.net (rxe at codespeak.net) Date: Sat, 7 Jan 2006 00:04:25 +0100 (CET) Subject: [pypy-svn] r21766 - in pypy/dist/pypy/translator/llvm: . test Message-ID: <20060106230425.206D227B62@code1.codespeak.net> Author: rxe Date: Sat Jan 7 00:04:23 2006 New Revision: 21766 Modified: pypy/dist/pypy/translator/llvm/opwriter.py pypy/dist/pypy/translator/llvm/test/test_genllvm.py Log: indirect_call() - remove XXX and add a test. Modified: pypy/dist/pypy/translator/llvm/opwriter.py ============================================================================== --- pypy/dist/pypy/translator/llvm/opwriter.py (original) +++ pypy/dist/pypy/translator/llvm/opwriter.py Sat Jan 7 00:04:23 2006 @@ -255,7 +255,7 @@ self.codewriter.call(opr.retref, opr.rettype, opr.argrefs[0], opr.argtypes[1:], opr.argrefs[1:]) - indirect_call = direct_call # XXX for now + indirect_call = direct_call def malloc(self, opr): arg_type = opr.op.args[0].value Modified: pypy/dist/pypy/translator/llvm/test/test_genllvm.py ============================================================================== --- pypy/dist/pypy/translator/llvm/test/test_genllvm.py (original) +++ pypy/dist/pypy/translator/llvm/test/test_genllvm.py Sat Jan 7 00:04:23 2006 @@ -45,6 +45,28 @@ assert f(0) == invokepointer(0) assert f(1) == invokepointer(1) +def test_invoke_function_pointer2(): + def f1(x): + if x>0: + raise Exception("bla") + return 10 + def f2(x): + return 5 + + l = [f1, f2] + + def invokepointer(i,j): + try: + return l[i](i) + except: + return 123 + + f = compile_function(invokepointer, [int, int]) + assert f(0, 0) == invokepointer(0, 0) + assert f(0, 1) == invokepointer(0, 1) + assert f(1, 0) == invokepointer(1, 0) + assert f(1, 1) == invokepointer(1, 1) + def test_simple_branching(): def simple5(b): if b: From cfbolz at codespeak.net Sat Jan 7 00:05:18 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Sat, 7 Jan 2006 00:05:18 +0100 (CET) Subject: [pypy-svn] r21767 - pypy/dist/pypy/rpython Message-ID: <20060106230518.B27CF27B68@code1.codespeak.net> Author: cfbolz Date: Sat Jan 7 00:05:17 2006 New Revision: 21767 Modified: pypy/dist/pypy/rpython/rpbc.py pypy/dist/pypy/rpython/rptr.py Log: use inputconst instead of constructing the Constant by hand -- thanks Samuele Modified: pypy/dist/pypy/rpython/rpbc.py ============================================================================== --- pypy/dist/pypy/rpython/rpbc.py (original) +++ pypy/dist/pypy/rpython/rpbc.py Sat Jan 7 00:05:17 2006 @@ -315,10 +315,7 @@ if isinstance(vlist[0], Constant): v = hop.genop('direct_call', vlist, resulttype = rresult) else: - print row_of_graphs - c = Constant(row_of_graphs.values()) - c.concretetype = Void - vlist.append(c) + vlist.append(hop.inputconst(Void, row_of_graphs.values())) v = hop.genop('indirect_call', vlist, resulttype = rresult) return hop.llops.convertvar(v, rresult, hop.r_result) Modified: pypy/dist/pypy/rpython/rptr.py ============================================================================== --- pypy/dist/pypy/rpython/rptr.py (original) +++ pypy/dist/pypy/rpython/rptr.py Sat Jan 7 00:05:17 2006 @@ -65,9 +65,7 @@ opname = 'direct_call' else: opname = 'indirect_call' - c = flowmodel.Constant(None) - c.concretetype = Void - vlist.append(c) + vlist.append(hop.inputconst(Void, None)) hop.exception_is_here() return hop.genop(opname, vlist, resulttype = self.lowleveltype.TO.RESULT) From arigo at codespeak.net Sat Jan 7 10:54:58 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sat, 7 Jan 2006 10:54:58 +0100 (CET) Subject: [pypy-svn] r21775 - pypy/dist/pypy/doc/discussion Message-ID: <20060107095458.9945B27DBF@code1.codespeak.net> Author: arigo Date: Sat Jan 7 10:54:56 2006 New Revision: 21775 Modified: pypy/dist/pypy/doc/discussion/somepbc-refactoring-plan.txt Log: The SomePBC refactoring has been finished some time ago. Mark it in this file, which we'll remove once we've moved some of its content as documentation. Modified: pypy/dist/pypy/doc/discussion/somepbc-refactoring-plan.txt ============================================================================== --- pypy/dist/pypy/doc/discussion/somepbc-refactoring-plan.txt (original) +++ pypy/dist/pypy/doc/discussion/somepbc-refactoring-plan.txt Sat Jan 7 10:54:56 2006 @@ -91,22 +91,7 @@ Status ====== -* the branch is here - -* pypy/annotation/desc.py looks good. - -* translator.py is heading in the right direction by having been split - in two classes: a TranslationContext that is just a bag of graphs, and - an obsolete Translator subclass that provides all the interactive - methods (that should eventually be done via the translator/goal/driver - code, but probably not as part of the same refactoring). - -* the annotator has now "synthetic classes", i.e. ClassDefs that - point back to the ClassDesc abstracting the user class, instead - of pointing directly back to the user class. It's needed to reimplement - class specialization. All annotator tests pass at the moment. - -* we started looking in the RTyper (but it's still completely broken). +Done, branch merged. RTyping PBCs of functions From ale at codespeak.net Sat Jan 7 13:32:34 2006 From: ale at codespeak.net (ale at codespeak.net) Date: Sat, 7 Jan 2006 13:32:34 +0100 (CET) Subject: [pypy-svn] r21785 - in pypy/dist/pypy/lib/pyontology: . test Message-ID: <20060107123234.0D7E027DC3@code1.codespeak.net> Author: ale Date: Sat Jan 7 13:32:33 2006 New Revision: 21785 Modified: pypy/dist/pypy/lib/pyontology/pyontology.py pypy/dist/pypy/lib/pyontology/test/test_ontology.py Log: Removed an unused parameter and changed the tests accordingly Modified: pypy/dist/pypy/lib/pyontology/pyontology.py ============================================================================== --- pypy/dist/pypy/lib/pyontology/pyontology.py (original) +++ pypy/dist/pypy/lib/pyontology/pyontology.py Sat Jan 7 13:32:33 2006 @@ -196,7 +196,7 @@ if ns in namespaces.values(): #predicate is one of builtin OWL predicates pred = getattr(self, func) - res = pred(s, p, o) + res = pred(s, o) if not res : continue if type(res) != list : @@ -211,20 +211,6 @@ res = self.variables[avar].getValues() self.variables[avar].setValues(res + [(sub, obj)]) - def merge_constraints(self): - # Make the intersection of multiple rdfs:range constraints on the same variable - cons_dict = {} - new_cons =[] - for con in self.constraints: - if isinstance(con, RangeConstraint): - cons_dict.setdefault(con.variable, []) - cons_dict[con.variable].append(con) - else: - new_cons.append(con) - for k,v in cons_dict.items(): - for con in v: - pass - def solve(self,verbose=0): #self.merge_constraints() rep = Repository(self.variables.keys(), self.variables, self.constraints) @@ -299,7 +285,7 @@ #---------------- Implementation ---------------- - def type(self, s, p, var): + def type(self, s, var): if not (var in [URIRef(namespaces[ns]+'#'+x) for x in builtin_voc for ns in namespaces]): # var is not one of the builtin classes avar = self.make_var(ClassDomain, var) @@ -314,18 +300,18 @@ if hasattr(cls, 'constraint'): self.constraints.append(cls.constraint) - def first(self, s, p, var): + def first(self, s, var): pass - def rest(self, s, p, var): + def rest(self, s, var): pass - def onProperty(self, s, p, var): + def onProperty(self, s, var): pass #---Class Axioms---#000000#FFFFFF----------------------------------------------- - def subClassOf(self, s, p, var): + def subClassOf(self, s, var): # s is a subclass of var means that the # class extension of s is a subset of the # class extension of var. @@ -334,24 +320,24 @@ cons = SubClassConstraint( svar, avar) self.constraints.append(cons) - def equivalentClass(self, s, p, var): - self.subClassOf(s, p, var) - self.subClassOf(var, p, s) + def equivalentClass(self, s, var): + self.subClassOf(s, var) + self.subClassOf(var, s) - def disjointWith(self, s, p, var): + def disjointWith(self, s, var): avar = self.make_var(None, var) svar = self.make_var(None, s) constrain = DisjointClassConstraint(svar, avar) self.constraints.append(constrain) - def complementOf(self, s, p, var): + def complementOf(self, s, var): # add constraint of not var # TODO: implementthis for OWL DL ## avar = self.make_var(ClassDomain, var) ## svar = self.make_var(ClassDomain, s) pass - def oneOf(self, s, p, var): + def oneOf(self, s, var): res = self.get_list(var) #prop = self.find_uriref(s) avar = self.make_var(None, s) @@ -360,11 +346,11 @@ else: self.variables[avar] = fd(res) - def unionOf(self,s, p, var): + def unionOf(self,s, var): res = self.get_list(var) return res #There might be doubles (but fd takes care of that) - def intersectionOf(self, s, p, var): + def intersectionOf(self, s, var): res = self.get_list(var) result = {}.fromkeys(res[0]) for el in res: @@ -375,33 +361,33 @@ #---Property Axioms---#000000#FFFFFF-------------------------------------------- - def range(self, s, p, var): + def range(self, s, var): avar = self.make_var(ClassDomain, var) svar = self.make_var(Property, s) cons = RangeConstraint(svar, avar) self.constraints.append(cons) - def domain(self, s, p, var): + def domain(self, s, var): # The classes that has this property (s) must belong to the class extension of var avar = self.make_var(ClassDomain, var) svar = self.make_var(Property, s) cons = DomainConstraint(svar, avar) self.constraints.append(cons) - def subPropertyOf(self, s, p, var): + def subPropertyOf(self, s, var): # s is a subproperty of var avar = self.make_var(Property, var) svar = self.make_var(Property, s) cons = SubPropertyConstraint( svar, avar) self.constraints.append(cons) - def equivalentProperty(self, s, p, var): + def equivalentProperty(self, s, var): avar = self.make_var(Property, var) svar = self.make_var(Property, s) cons = EquivalentPropertyConstraint( svar, avar) self.constraints.append(cons) - def inverseOf(self, s, p, var): + def inverseOf(self, s, var): avar = self.make_var(Property, var) svar = self.make_var(Property, s) con = InverseofConstraint(svar, avar) @@ -409,21 +395,21 @@ #---Property restrictions------------------------------------------------------ - def maxCardinality(self, s, p, var): + def maxCardinality(self, s, var): """ Len of finite domain of the property shall be less than or equal to var""" avar = self.find_property(s) cls =self.make_var(None, self.find_cls(s)) constrain = MaxCardinality(avar, cls, int(var)) self.constraints.append(constrain) - def minCardinality(self, s, p, var): + def minCardinality(self, s, var): """ Len of finite domain of the property shall be greater than or equal to var""" avar = self.find_property(s) cls =self.make_var(None, self.find_cls(s)) constrain = MinCardinality(avar, cls, int(var)) self.constraints.append(constrain) - def cardinality(self, s, p, var): + def cardinality(self, s, var): """ Len of finite domain of the property shall be equal to var""" avar = self.find_property(s) cls =self.make_var(None, self.find_cls(s)) @@ -431,37 +417,43 @@ constrain = Cardinality(avar, cls, int(var)) self.constraints.append(constrain) - def differentFrom(self, s, p, var): + def differentFrom(self, s, var): s_var = self.make_var(ClassDomain, s) var_var = self.make_var(Thing, var) constrain = BinaryExpression([s_var, var_var],"%s != %s" %(s_var, var_var)) self.constraints.append(constrain) - def distinctMembers(self, s, p, var): + def distinctMembers(self, s, var): res = self.get_list(var) self.constraints.append(AllDistinct([self.make_var(ClassDomain, y) for y in res])) return res - def sameAs(self, s, p, var): + def sameAs(self, s, var): s_var = self.make_var(None, s) var_var = self.make_var(None, var) constrain = BinaryExpression([s_var, var_var], "%s == %s" %(s_var, var_var)) self.constraints.append(constrain) - def hasValue(self, s, p, var): - # TODO: implement this - pass - - def allValuesFrom(self, s, p, var): + def hasValue(self, s, var): + if type(var) != URIRef: + value = var + else: + value = self.make_var(None, var) + prop = self.find_prop(s) + cls = self.make_var(None, self.find_cls(s)) + con = HasvalueConstraint(prop, cls, value) + self.constraints.append(con) + + def allValuesFrom(self, s, var): # TODO: implement this pass - def someValuesFrom(self, s, p, var): + def someValuesFrom(self, s, var): # TODO: implement this pass - def imports(self, s, p, var): + def imports(self, s, var): # PP TODO: implement this pass @@ -712,3 +704,6 @@ raise ConsistencyFailure("Inverseof failed") ## res.append((val, cls)) ## domains[self.variable].setValues(res) + +class HasvalueConstraint: + pass \ No newline at end of file Modified: pypy/dist/pypy/lib/pyontology/test/test_ontology.py ============================================================================== --- pypy/dist/pypy/lib/pyontology/test/test_ontology.py (original) +++ pypy/dist/pypy/lib/pyontology/test/test_ontology.py Sat Jan 7 13:32:33 2006 @@ -28,8 +28,8 @@ A = O.make_var(ClassDomain, a) C = O.make_var(ClassDomain, c) C = O.make_var(ClassDomain, b) - O.subClassOf(b, None, a) - O.subClassOf(c, None, b) + O.subClassOf(b, a) + O.subClassOf(c, b) for con in O.constraints: con.narrow(O.variables) assert len(O.variables) == 3 @@ -90,8 +90,8 @@ b = URIRef('B') c = URIRef('C') O = Ontology() - O.equivalentClass(c, None, a) - O.equivalentClass(c, None, b) + O.equivalentClass(c, a) + O.equivalentClass(c, b) A = O.make_var(ClassDomain, a) B = O.make_var(ClassDomain, b) assert O.variables[A].values == O.variables[B].values @@ -102,7 +102,7 @@ obj = URIRef('o') O = Ontology() O.make_var(ClassDomain, obj) - O.type(sub, pred , obj) + O.type(sub, obj) assert O.variables[O.make_var(None, sub)].__class__ == ClassDomain @@ -111,7 +111,7 @@ pred = URIRef('type') obj = URIRef(namespaces['owl']+'#ObjectProperty') O = Ontology() - O.type(sub, pred , obj) + O.type(sub, obj) assert O.variables[O.make_var(None, sub)].__class__ == ObjectProperty def test_range(): @@ -119,11 +119,11 @@ sub = URIRef('a') obj = URIRef('b') O.variables['b_'] = fd([1,2,3,4]) - O.range(sub, None , obj) + O.range(sub, obj) sub = URIRef('a') pred = URIRef('type') obj = URIRef(namespaces['owl']+'#ObjectProperty') - O.type(sub, pred , obj) + O.type(sub, obj) assert len(O.constraints) == 1 O.constraints[0].narrow(O.variables) assert O.variables['a_'].range == [1,2,3,4] @@ -133,14 +133,14 @@ sub = URIRef('a') obj = URIRef('b') O.variables['b_'] = fd([1,2,3,4]) - O.range(sub, None , obj) + O.range(sub, obj) obj = URIRef('c') O.variables['c_'] = fd([3,4,5,6]) - O.range(sub, None , obj) + O.range(sub, obj) sub = URIRef('a') pred = URIRef('type') obj = URIRef(namespaces['owl']+'#ObjectProperty') - O.type(sub, pred , obj) + O.type(sub, obj) assert len(O.constraints) == 2 O.consistency() assert O.variables['a_'].range == [ 3,4] @@ -150,11 +150,11 @@ sub = URIRef('a') obj = URIRef('b') O.variables['b_'] = ClassDomain('b') - O.domain(sub, None , obj) + O.domain(sub, obj) sub = URIRef('a') pred = URIRef('type') obj = URIRef(namespaces['owl']+'#ObjectProperty') - O.type(sub, pred , obj) + O.type(sub, obj) assert len(O.constraints) == 1 O.constraints[0].narrow(O.variables) assert O.variables['a_'].domain == ['b_'] @@ -164,12 +164,12 @@ sub = URIRef('a') obj = URIRef('b') O.variables['b_'] = ClassDomain('b') - O.domain(sub, None , obj) + O.domain(sub, obj) obj = URIRef('c') O.variables['c_'] = ClassDomain('c') - O.domain(sub, None , obj) + O.domain(sub, obj) obj = URIRef(namespaces['owl']+'#ObjectProperty') - O.type(sub, None , obj) + O.type(sub, obj) assert len(O.constraints) == 2 for con in O.constraints: @@ -180,10 +180,10 @@ O = Ontology() sub = URIRef('a') obj = URIRef(namespaces['owl']+'#ObjectProperty') - O.type(sub, None, obj) + O.type(sub, obj) b = URIRef('b') - O.type(b, None, obj) - O.subPropertyOf(sub, None, b) + O.type(b, obj) + O.subPropertyOf(sub, b) assert len(O.constraints) ==1 O.variables['a_'].setValues([('individ_',42)]) O.consistency() @@ -196,15 +196,15 @@ #Make functional property sub = URIRef('p') obj = URIRef(namespaces['owl']+'#FunctionalProperty') - O.type(sub, None, obj) + O.type(sub, obj) #Make class sub = URIRef('c') obj = URIRef(namespaces['owl']+'#Class') - O.type(sub, None, obj) + O.type(sub, obj) #Make individual with a value of the property sub = URIRef('individ') obj = URIRef('c') - O.type(sub, None, obj) + O.type(sub, obj) O.variables['p_'].setValues([('individ_',42)]) #assert len(O.constraints) == 2 #add another valueof the property @@ -218,21 +218,21 @@ #Make functional property sub = URIRef('p') obj = URIRef(namespaces['owl']+'#InverseFunctionalProperty') - O.type(sub, None, obj) + O.type(sub, obj) #Make class sub = URIRef('c') obj = URIRef(namespaces['owl']+'#Class') - O.type(sub, None, obj) + O.type(sub, obj) #Make individual with a value of the property sub = URIRef('individ') obj = URIRef('c') - O.type(sub, None, obj) + O.type(sub, obj) O.variables['p_'].setValues([('individ_',42)]) #assert len(O.constraints) == 2 #add another individual with the same value for the property sub = URIRef('individ2') obj = URIRef('c') - O.type(sub, None, obj) + O.type(sub, obj) O.variables['p_'].setValues([('individ_',42),('individ2_',42)]) #check that consistency raises py.test.raises(ConsistencyFailure, O.consistency) @@ -243,19 +243,19 @@ #Make functional property sub = URIRef('subRegionOf') obj = URIRef(namespaces['owl']+'#TransitiveProperty') - O.type(sub, None, obj) + O.type(sub, obj) #Make class sub = URIRef('c') obj = URIRef(namespaces['owl']+'#Class') - O.type(sub, None, obj) + O.type(sub, obj) #Make individual with a value of the property sub = URIRef('Italy') obj = URIRef('c') - O.type(sub, None, obj) + O.type(sub, obj) sub = URIRef('Tuscanny') - O.type(sub, None, obj) + O.type(sub, obj) sub = URIRef('Chianti') - O.type(sub, None, obj) + O.type(sub, obj) O.variables['subRegionOf_'].setValues([('Italy_','Tuscanny_'),('Tuscanny_','Chianti_')]) O.consistency() assert ('Italy_', 'Chianti_') in O.variables['subRegionOf_'].getValues() @@ -266,45 +266,54 @@ #Make functional property sub = URIRef('friend') obj = URIRef(namespaces['owl']+'#SymmetricProperty') - O.type(sub, None, obj) + O.type(sub, obj) assert O.variables[O.make_var(None, sub)].__class__.__name__=='SymmetricProperty' #Make class sub = URIRef('c') obj = URIRef(namespaces['owl']+'#Class') - O.type(sub, None, obj) + O.type(sub, obj) #Make individual with a value of the property sub = URIRef('Bob') obj = URIRef('c') - O.type(sub, None, obj) + O.type(sub, obj) sub = URIRef('Alice') - O.type(sub, None, obj) + O.type(sub, obj) O.variables['friend_'].setValues([('Bob_','Alice_')]) O.consistency() assert ('Alice_', 'Bob_') in O.variables['friend_'].getValues() def test_inverseof(): - O = Ontology() own = URIRef('owner') obj = URIRef(namespaces['owl']+'#ObjectProperty') - O.type(own, None, obj) + O.type(own, obj) owned = URIRef('ownedby') obj = URIRef(namespaces['owl']+'#ObjectProperty') - O.type(owned, None, obj) - O.inverseOf(own, None, owned) + O.type(owned, obj) + O.inverseOf(own, owned) #Make class sub = URIRef('c') obj = URIRef(namespaces['owl']+'#Class') - O.type(sub, None, obj) + O.type(sub, obj) #Make individual with a property value sub = URIRef('Bob') obj = URIRef('c') - O.type(sub, None, obj) + O.type(sub, obj) sub = URIRef('Fiat') obj = URIRef('car') - O.type(sub, None, obj) + O.type(sub, obj) O.variables['owner_'].setValues([('Bob_','Fiat_')]) py.test.raises(ConsistencyFailure, O.consistency) O.variables['ownedby_'].setValues([('Fiat_','Bob_')]) O.consistency() - assert not '_anon' in O.variables \ No newline at end of file + +def no_test_hasvalue(): + O = Ontology() + own = URIRef('owner') + obj = URIRef(namespaces['owl']+'#ObjectProperty') + O.type(own, obj) + O.hasValue(own, 42) + py.test.raises(ConsistencyFailure, O.consistency) + #Make class + O.variables['owner_'].setValues([('Fiat_', 42)]) + O.consistency() From cfbolz at codespeak.net Sat Jan 7 15:13:59 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Sat, 7 Jan 2006 15:13:59 +0100 (CET) Subject: [pypy-svn] r21787 - in pypy/dist/pypy/translator/backendopt: . test Message-ID: <20060107141359.B850F27DC3@code1.codespeak.net> Author: cfbolz Date: Sat Jan 7 15:13:58 2006 New Revision: 21787 Modified: pypy/dist/pypy/translator/backendopt/inline.py pypy/dist/pypy/translator/backendopt/test/test_inline.py Log: fix inlining after indirect_call refactoring (thanks samuele for the hint). Now collect_called_graphs tries hard to return all graphs of a function (which is not actually used in inlining but potentially nice). Test for the bug. Modified: pypy/dist/pypy/translator/backendopt/inline.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/inline.py (original) +++ pypy/dist/pypy/translator/backendopt/inline.py Sat Jan 7 15:13:58 2006 @@ -1,6 +1,6 @@ import sys from pypy.translator.simplify import eliminate_empty_blocks, join_blocks -from pypy.translator.simplify import remove_identical_vars +from pypy.translator.simplify import remove_identical_vars, get_graph from pypy.translator.unsimplify import copyvar, split_block from pypy.objspace.flow.model import Variable, Constant, Block, Link from pypy.objspace.flow.model import SpaceOperation, c_last_exception @@ -18,16 +18,24 @@ pass -def collect_called_functions(graph): - funcs = {} - def visit(obj): - if not isinstance(obj, Block): - return - for op in obj.operations: +def collect_called_graphs(graph, translator): + graphs_or_something = {} + for block in graph.iterblocks(): + for op in block.operations: if op.opname == "direct_call": - funcs[op.args[0]] = True - traverse(visit, graph) - return funcs + graph = get_graph(op.args[0], translator) + if graph is not None: + graphs_or_something[graph] = True + else: + graphs_or_something[op.args[0]] = True + if op.opname == "indirect_call": + graphs = op.args[-1].value + if graphs is None: + graphs_or_something[op.args[0]] = True + else: + for graph in graphs: + graphs_or_something[graph] = True + return graphs_or_something def find_callsites(graph, calling_what): callsites = [] @@ -81,7 +89,7 @@ if (block.exitswitch == c_last_exception and index_operation == len(block.operations) - 1): exception_guarded = True - if len(collect_called_functions(graph_to_inline)) != 0: + if len(collect_called_graphs(graph_to_inline, translator)) != 0: raise CannotInline("can't handle exceptions yet") entrymap = mkentrymap(graph_to_inline) beforeblock = block @@ -277,6 +285,7 @@ 'cast_pointer': 0, 'keepalive': 0, 'direct_call': 2, # guess + 'indirect_call': 2, # guess 'yield_current_frame_to_caller': sys.maxint, # XXX bit extreme } Modified: pypy/dist/pypy/translator/backendopt/test/test_inline.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/test/test_inline.py (original) +++ pypy/dist/pypy/translator/backendopt/test/test_inline.py Sat Jan 7 15:13:58 2006 @@ -5,7 +5,7 @@ from pypy.objspace.flow.model import last_exception, checkgraph from pypy.translator.backendopt.inline import inline_function, CannotInline from pypy.translator.backendopt.inline import auto_inlining -from pypy.translator.backendopt.inline import collect_called_functions +from pypy.translator.backendopt.inline import collect_called_graphs from pypy.translator.backendopt.inline import measure_median_execution_cost from pypy.translator.translator import TranslationContext, graphof from pypy.rpython.llinterp import LLInterpreter @@ -277,7 +277,7 @@ return -1 eval_func, t = check_auto_inlining(f, [int], threshold=10) f_graph = graphof(t, f) - assert len(collect_called_functions(f_graph)) == 0 + assert len(collect_called_graphs(f_graph, t)) == 0 result = eval_func([10]) assert result == 45 @@ -357,3 +357,27 @@ graph = t.buildflowgraph(f) res = measure_median_execution_cost(graph) assert res == 19 + +def test_indirect_call_with_exception(): + class MyExc(Exception): + pass + def x1(): + return 1 + def x2(): + return 2 + def x3(x): + if x: + f = x1 + else: + f = x2 + return f() + def x4(): + try: + x3(0) + x3(1) + except MyExc: + return 0 + return 1 + assert x4() == 1 + py.test.raises(CannotInline, check_inline, x3, x4, []) + From ale at codespeak.net Sat Jan 7 18:27:07 2006 From: ale at codespeak.net (ale at codespeak.net) Date: Sat, 7 Jan 2006 18:27:07 +0100 (CET) Subject: [pypy-svn] r21790 - in pypy/dist/pypy/lib/pyontology: . test Message-ID: <20060107172707.054D727DBE@code1.codespeak.net> Author: ale Date: Sat Jan 7 18:27:06 2006 New Revision: 21790 Modified: pypy/dist/pypy/lib/pyontology/pyontology.py pypy/dist/pypy/lib/pyontology/test/test_ontology.py Log: Added support for rdf:List using rdf:first and rdf:rest Modified: pypy/dist/pypy/lib/pyontology/pyontology.py ============================================================================== --- pypy/dist/pypy/lib/pyontology/pyontology.py (original) +++ pypy/dist/pypy/lib/pyontology/pyontology.py Sat Jan 7 18:27:06 2006 @@ -67,8 +67,10 @@ self.values = values class List(ClassDomain): - pass - + def __init__(self, name='', values=[], bases = []): + ClassDomain.__init__(self, name, values, bases) + self.constraint = ListConstraint(name) + class Property(ClassDomain): # Property contains the relationship between a class instance and a value # - a pair. To accomodate global assertions like 'range' and 'domain' attributes @@ -217,8 +219,8 @@ return Solver().solve(rep, verbose) def consistency(self, verbose=0): - rep = Repository(self.variables.keys(), self.variables, self.constraints) - rep.consistency(verbose) + self.rep = Repository(self.variables.keys(), self.variables, self.constraints[:]) + self.rep.consistency(verbose) def get_list(self, subject): res = [] @@ -244,7 +246,9 @@ if ns not in uris.keys(): uris[ns] = ns.split('/')[-1] a = uris[ns] + '_' + name - var = str(a.replace('-','_')) + var = str(a.replace('-','_')) + else: + var = a if not cls: return var if not var in self.variables.keys(): @@ -301,10 +305,23 @@ self.constraints.append(cls.constraint) def first(self, s, var): - pass + avar = self.make_var(None, var) + svar = self.make_var(List, s) + vals = [] + vals += self.variables[svar].getValues() + vals.insert(0, avar) + self.variables[svar].setValues(vals) def rest(self, s, var): - pass + if var == URIRef(namespaces['rdf']+'#nil'): + return + else: + avar = self.make_var(List, var) + svar = self.make_var(List, s) + vals = [] + vals += self.variables[svar].getValues() + vals.append( avar) + self.variables[svar].setValues(vals) def onProperty(self, s, var): pass @@ -473,12 +490,12 @@ return self.__cost -class MaxCardinality(OwlConstraint): +class MaxCardinality(AbstractConstraint): """Contraint: all values must be distinct""" def __init__(self, variable, cls, cardinality): - OwlConstraint.__init__(self, variable) - self.__cost = 1 + AbstractConstraint.__init__(self, [variable, cls]) + self.__cost = 2 self.cardinality = cardinality self.cls = cls @@ -526,10 +543,10 @@ res.extend(get_values(domains[val], domains, attr)) return res -class SubClassConstraint(OwlConstraint): +class SubClassConstraint(AbstractConstraint): def __init__(self, variable, cls_or_restriction): - OwlConstraint.__init__(self, variable) + AbstractConstraint.__init__(self, [variable, cls_or_restriction]) self.object = cls_or_restriction self.variable = variable @@ -569,7 +586,6 @@ propdom = domains[self.variable] rangedom = domains[self.object] newrange = get_values(rangedom, domains, 'getValues') - print rangedom range = [] oldrange = propdom.range if oldrange: @@ -702,8 +718,28 @@ for cls, val in sub_domain: if not (val,cls) in obj_domain: raise ConsistencyFailure("Inverseof failed") -## res.append((val, cls)) -## domains[self.variable].setValues(res) + +class ListConstraint(AbstractConstraint): + """Contraint: all values must be distinct""" + def __init__(self, variable): + AbstractConstraint.__init__(self, [variable]) + self.variable = variable + self.__cost = 10 + + def estimateCost(self, domains): + return self.__cost + + def narrow(self, domains): + """narrowing algorithm for the constraint""" + vals =[] + vals += domains[self.variable].getValues() + while True: + if vals[-1] in domains.keys() and isinstance(domains[vals[-1]], List): + vals = vals[:-1] + domains[vals[-1]].getValues() + else: + break + domains[self.variable].setValues(vals) + return 1 class HasvalueConstraint: - pass \ No newline at end of file + pass Modified: pypy/dist/pypy/lib/pyontology/test/test_ontology.py ============================================================================== --- pypy/dist/pypy/lib/pyontology/test/test_ontology.py (original) +++ pypy/dist/pypy/lib/pyontology/test/test_ontology.py Sat Jan 7 18:27:06 2006 @@ -317,3 +317,18 @@ #Make class O.variables['owner_'].setValues([('Fiat_', 42)]) O.consistency() + +def test_List(): + O = Ontology() + own = URIRef('favlist') + obj = URIRef(namespaces['rdf']+'#List') + O.type(own, obj) + O.first(own, 0) + O.rest(own, URIRef('1')) + O.first( URIRef('1'), 1) + O.rest( URIRef('1'), URIRef('2')) + O.first( URIRef('2'), 2) + O.rest( URIRef('2'), URIRef(namespaces['rdf']+'#nil')) + assert len(O.constraints) == 1 + O.consistency(5) + assert O.rep._domains['favlist_'].getValues() == [0,1,2] \ No newline at end of file From cfbolz at codespeak.net Sat Jan 7 19:14:05 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Sat, 7 Jan 2006 19:14:05 +0100 (CET) Subject: [pypy-svn] r21792 - pypy/dist/pypy/rpython Message-ID: <20060107181405.D44AC27DC9@code1.codespeak.net> Author: cfbolz Date: Sat Jan 7 19:14:04 2006 New Revision: 21792 Modified: pypy/dist/pypy/rpython/rpbc.py Log: pseudo-checkin to test whether poor braintone is working again Modified: pypy/dist/pypy/rpython/rpbc.py ============================================================================== --- pypy/dist/pypy/rpython/rpbc.py (original) +++ pypy/dist/pypy/rpython/rpbc.py Sat Jan 7 19:14:04 2006 @@ -575,3 +575,4 @@ for cdef1 in classdef.getmro(): for attrname in cdef1.attrs: yield cdef1, attrname + From ericvrp at codespeak.net Sat Jan 7 20:18:08 2006 From: ericvrp at codespeak.net (ericvrp at codespeak.net) Date: Sat, 7 Jan 2006 20:18:08 +0100 (CET) Subject: [pypy-svn] r21794 - in pypy/dist/pypy/translator/js: . test Message-ID: <20060107191808.9664527B60@code1.codespeak.net> Author: ericvrp Date: Sat Jan 7 20:17:51 2006 New Revision: 21794 Modified: pypy/dist/pypy/translator/js/codewriter.py pypy/dist/pypy/translator/js/funcnode.py pypy/dist/pypy/translator/js/test/runtest.py Log: Added support for block with multiple exits Modified: pypy/dist/pypy/translator/js/codewriter.py ============================================================================== --- pypy/dist/pypy/translator/js/codewriter.py (original) +++ pypy/dist/pypy/translator/js/codewriter.py Sat Jan 7 20:17:51 2006 @@ -134,6 +134,15 @@ self.append('}') self.skip_closeblock() + def switch(self, cond, defaultdest, value_labels): + if not defaultdest: + raise TypeError('switches must have a default case.') + labels = ','.join(["'%s':%s" % (value, label) for value, label in value_labels]) + #XXX for performance this Object should become global + self.append("if (!(block = {%s}[%s])) block = %s" % (labels, cond, defaultdest)) + self._goto_block('block') + self.skip_closeblock() + def openfunc(self, decl, funcnode, blocks): self.decl = decl self.funcnode = funcnode Modified: pypy/dist/pypy/translator/js/funcnode.py ============================================================================== --- pypy/dist/pypy/translator/js/funcnode.py (original) +++ pypy/dist/pypy/translator/js/funcnode.py Sat Jan 7 20:17:51 2006 @@ -83,14 +83,40 @@ def write_block_branches(self, codewriter, block): if block.exitswitch == c_last_exception: return + if len(block.exits) == 1: codewriter.br_uncond(self.blockindex[block.exits[0].target], block.exits[0]) - elif len(block.exits) == 2: - cond = self.db.repr_arg(block.exitswitch) + return + + cond = self.db.repr_arg(block.exitswitch) + if block.exitswitch.concretetype == lltype.Bool: + assert len(block.exits) == 2 codewriter.br(cond, self.blockindex[block.exits[0].target], block.exits[0], self.blockindex[block.exits[1].target], block.exits[1]) + elif block.exitswitch.concretetype in \ + (lltype.Signed, lltype.Unsigned, lltype.SignedLongLong, + lltype.UnsignedLongLong, lltype.Char, lltype.UniChar): + defaultlink = None + value_labels = [] + for link in block.exits: + if link.exitcase is 'default': + defaultlink = link + continue + + exitcase = link.llexitcase + if block.exitswitch.concretetype in [lltype.Char, lltype.UniChar]: + exitcase = ord(exitcase) + value_labels.append( (exitcase, + self.blockindex[link.target]) ) + + codewriter.switch(cond, self.blockindex[defaultlink.target], value_labels) + + else: + raise BranchException("exitswitch type '%s' not supported" % + block.exitswitch.concretetype) + def write_block_operations(self, codewriter, block): opwriter = OpWriter(self.db, codewriter, self, block) if block.exitswitch == c_last_exception: Modified: pypy/dist/pypy/translator/js/test/runtest.py ============================================================================== --- pypy/dist/pypy/translator/js/test/runtest.py (original) +++ pypy/dist/pypy/translator/js/test/runtest.py Sat Jan 7 20:17:51 2006 @@ -24,7 +24,7 @@ t = Translator(function) a = t.annotate(annotation) t.specialize() - t.backend_optimizations(inline_threshold=0, mallocs=False, merge_if_blocks_to_switch=False) + t.backend_optimizations(inline_threshold=0, mallocs=False) #t.backend_optimizations() if view: t.view() From ericvrp at codespeak.net Sat Jan 7 20:47:20 2006 From: ericvrp at codespeak.net (ericvrp at codespeak.net) Date: Sat, 7 Jan 2006 20:47:20 +0100 (CET) Subject: [pypy-svn] r21798 - in pypy/dist/pypy/translator/js: . test Message-ID: <20060107194720.88C4427B66@code1.codespeak.net> Author: ericvrp Date: Sat Jan 7 20:47:19 2006 New Revision: 21798 Added: pypy/dist/pypy/translator/js/test/test_merge_if_blocks.py Modified: pypy/dist/pypy/translator/js/codewriter.py Log: genjs tests for if->switch support Modified: pypy/dist/pypy/translator/js/codewriter.py ============================================================================== --- pypy/dist/pypy/translator/js/codewriter.py (original) +++ pypy/dist/pypy/translator/js/codewriter.py Sat Jan 7 20:47:19 2006 @@ -140,8 +140,6 @@ labels = ','.join(["'%s':%s" % (value, label) for value, label in value_labels]) #XXX for performance this Object should become global self.append("if (!(block = {%s}[%s])) block = %s" % (labels, cond, defaultdest)) - self._goto_block('block') - self.skip_closeblock() def openfunc(self, decl, funcnode, blocks): self.decl = decl Added: pypy/dist/pypy/translator/js/test/test_merge_if_blocks.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/js/test/test_merge_if_blocks.py Sat Jan 7 20:47:19 2006 @@ -0,0 +1,83 @@ +import py + +from pypy.translator.js.test.runtest import compile_function + +import sys + +def test_merge_if_blocks_simple(): + def merge_if_blocks_simple(i): + if i == 5: + return 1005 + else: + return 2222 + simple = compile_function(merge_if_blocks_simple, [int]) + for i in range(-20,20): + assert simple(i) == merge_if_blocks_simple(i) + +def test_merge_if_blocks_basic(): + py.skip("merge_if_block failing because link exit values are not used") + def merge_if_blocks_basic(i): + if i == 5: + return 1005 + elif i == 8: + return 1008 + return 2222 + basic = compile_function(merge_if_blocks_basic , [int], view=True) + for i in range(-20,20): + assert basic(i) == merge_if_blocks_basic(i) + +def test_merge_if_blocks_chr(): + py.skip("merge_if_block failing because link exit values are not used") + def merge_if_blocks_chr(i): + c = chr(i) + if c == '\x05': + return 1005 + elif c == '!': + return 1008 + return 2222 + basic = compile_function(merge_if_blocks_chr , [int]) + for i in range(0, 50): + assert basic(i) == merge_if_blocks_chr(i) + +def test_merge_if_blocks_uni(): + py.skip("merge_if_block failing because link exit values are not used") + def merge_if_blocks_uni(i): + c = unichr(i) + if c == u'\x05': + return 1005 + elif c == u'!': + return 1008 + return 2222 + basic = compile_function(merge_if_blocks_uni , [int]) + for i in range(0, 50): + assert basic(i) == merge_if_blocks_uni(i) + + +def test_merge_if_blocks_many(): + py.skip("merge_if_block failing because link exit values are not used") + def merge_if_blocks_many(i): + if i == 0: + return 1000 + elif i == 1: + return 1001 + elif i == 2: + return 1002 + elif i == 3: + return 1003 + elif i == 4: + return 1004 + elif i == 5: + return 1005 + elif i == 6: + return 1006 + elif i == 7: + return 1007 + elif i == 8: + return 1008 + elif i == 9: + return 1009 + else: + return 2222 + many = compile_function(merge_if_blocks_many , [int]) + for i in range(-20,20): + assert many(i) == merge_if_blocks_many(i) From ericvrp at codespeak.net Sat Jan 7 20:56:39 2006 From: ericvrp at codespeak.net (ericvrp at codespeak.net) Date: Sat, 7 Jan 2006 20:56:39 +0100 (CET) Subject: [pypy-svn] r21799 - pypy/extradoc/sprintinfo/mallorca Message-ID: <20060107195639.C4BDC27B68@code1.codespeak.net> Author: ericvrp Date: Sat Jan 7 20:56:37 2006 New Revision: 21799 Modified: pypy/extradoc/sprintinfo/mallorca/people.txt Log: My Mallorca dates Modified: pypy/extradoc/sprintinfo/mallorca/people.txt ============================================================================== --- pypy/extradoc/sprintinfo/mallorca/people.txt (original) +++ pypy/extradoc/sprintinfo/mallorca/people.txt Sat Jan 7 20:56:37 2006 @@ -12,7 +12,7 @@ Beatrice Duering 22/1 -29/1 flat? Armin Rigo ? ? Samuele Pedroni ? ? -Eric van Riet Paap ? ? +Eric van Riet Paap 22/1 - 28/1 casa M. Michael Hudson ? ? Carl Friedrich Bolz ? ? Anders Chrigstroem ? ? From cfbolz at codespeak.net Sat Jan 7 22:50:18 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Sat, 7 Jan 2006 22:50:18 +0100 (CET) Subject: [pypy-svn] r21802 - pypy/dist/pypy/translator/backendopt Message-ID: <20060107215018.0E6A327B68@code1.codespeak.net> Author: cfbolz Date: Sat Jan 7 22:50:17 2006 New Revision: 21802 Modified: pypy/dist/pypy/translator/backendopt/inline.py Log: Intermediate step in my feeble attempt to refactor The Inlining Blob (tm). I don't dare to shuffle around more without checking in because afterwards it might not work at all anymore. Modified: pypy/dist/pypy/translator/backendopt/inline.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/inline.py (original) +++ pypy/dist/pypy/translator/backendopt/inline.py Sat Jan 7 22:50:17 2006 @@ -82,62 +82,60 @@ return None return ops[-4].args[2].value -def _inline_function(translator, graph, block, index_operation): - op = block.operations[index_operation] - graph_to_inline = op.args[0].value._obj.graph - exception_guarded = False - if (block.exitswitch == c_last_exception and - index_operation == len(block.operations) - 1): - exception_guarded = True - if len(collect_called_graphs(graph_to_inline, translator)) != 0: - raise CannotInline("can't handle exceptions yet") - entrymap = mkentrymap(graph_to_inline) - beforeblock = block - afterblock = split_block(translator, graph, block, index_operation) - assert afterblock.operations[0] is op - #vars that need to be passed through the blocks of the inlined function - passon_vars = {beforeblock: [arg for arg in beforeblock.exits[0].args - if isinstance(arg, Variable)]} - copied_blocks = {} - varmap = {} - def get_new_name(var): + + +class Inliner(object): + def __init__(self, translator, graph, block, index_operation): + self.translator = translator + self.graph = graph + self.varmap = {} + self.beforeblock = block + self.copied_blocks = {} + self.do_inline(block, index_operation) + + def get_new_name(self, var): if var is None: return None if isinstance(var, Constant): return var - if var not in varmap: - varmap[var] = copyvar(translator, var) - return varmap[var] - def get_new_passon_var_names(block): - result = [copyvar(translator, var) for var in passon_vars[beforeblock]] - passon_vars[block] = result + if var not in self.varmap: + self.varmap[var] = copyvar(self.translator, var) + return self.varmap[var] + + def get_new_passon_var_names(self, block): + result = [copyvar(self.translator, var) for var in self.passon_vars[self.beforeblock]] + self.passon_vars[block] = result return result - def copy_operation(op): - args = [get_new_name(arg) for arg in op.args] - return SpaceOperation(op.opname, args, get_new_name(op.result)) - def copy_block(block): - if block in copied_blocks: + + def copy_operation(self, op): + args = [self.get_new_name(arg) for arg in op.args] + return SpaceOperation(op.opname, args, self.get_new_name(op.result)) + + def copy_block(self, block): + if block in self.copied_blocks: "already there" - return copied_blocks[block] - args = ([get_new_name(var) for var in block.inputargs] + - get_new_passon_var_names(block)) + return self.copied_blocks[block] + args = ([self.get_new_name(var) for var in block.inputargs] + + self.get_new_passon_var_names(block)) newblock = Block(args) - copied_blocks[block] = newblock - newblock.operations = [copy_operation(op) for op in block.operations] - newblock.exits = [copy_link(link, block) for link in block.exits] - newblock.exitswitch = get_new_name(block.exitswitch) + self.copied_blocks[block] = newblock + newblock.operations = [self.copy_operation(op) for op in block.operations] + newblock.exits = [self.copy_link(link, block) for link in block.exits] + newblock.exitswitch = self.get_new_name(block.exitswitch) newblock.exc_handler = block.exc_handler return newblock - def copy_link(link, prevblock): - newargs = [get_new_name(a) for a in link.args] + passon_vars[prevblock] - newlink = Link(newargs, copy_block(link.target), link.exitcase) - newlink.prevblock = copy_block(link.prevblock) - newlink.last_exception = get_new_name(link.last_exception) - newlink.last_exc_value = get_new_name(link.last_exc_value) + + def copy_link(self, link, prevblock): + newargs = [self.get_new_name(a) for a in link.args] + self.passon_vars[prevblock] + newlink = Link(newargs, self.copy_block(link.target), link.exitcase) + newlink.prevblock = self.copy_block(link.prevblock) + newlink.last_exception = self.get_new_name(link.last_exception) + newlink.last_exc_value = self.get_new_name(link.last_exc_value) if hasattr(link, 'llexitcase'): newlink.llexitcase = link.llexitcase return newlink - def generate_keepalive(vars): + + def generate_keepalive(self, vars): keepalive_ops = [] for v in vars: if isinstance(v, Constant): @@ -149,133 +147,155 @@ keepalive_ops.append(SpaceOperation('keepalive', [v], v_keepalive)) return keepalive_ops - linktoinlined = beforeblock.exits[0] - assert linktoinlined.target is afterblock - copiedstartblock = copy_block(graph_to_inline.startblock) - copiedstartblock.isstartblock = False - #find args passed to startblock of inlined function - passon_args = [] - for arg in op.args[1:]: - if isinstance(arg, Constant): - passon_args.append(arg) - else: - index = afterblock.inputargs.index(arg) - passon_args.append(linktoinlined.args[index]) - passon_args += passon_vars[beforeblock] - #rewire blocks - linktoinlined.target = copiedstartblock - linktoinlined.args = passon_args - afterblock.inputargs = [op.result] + afterblock.inputargs - afterblock.operations = generate_keepalive(afterblock.inputargs) + afterblock.operations[1:] - if graph_to_inline.returnblock in entrymap: - copiedreturnblock = copied_blocks[graph_to_inline.returnblock] - linkfrominlined = Link([copiedreturnblock.inputargs[0]] + passon_vars[graph_to_inline.returnblock], afterblock) - linkfrominlined.prevblock = copiedreturnblock - copiedreturnblock.exitswitch = None - copiedreturnblock.exits = [linkfrominlined] - assert copiedreturnblock.exits[0].target == afterblock - if graph_to_inline.exceptblock in entrymap: - #let links to exceptblock of the graph to inline go to graphs exceptblock - copiedexceptblock = copied_blocks[graph_to_inline.exceptblock] - if not exception_guarded: - # find all copied links that go to copiedexceptblock - for link in entrymap[graph_to_inline.exceptblock]: - copiedblock = copied_blocks[link.prevblock] - for copiedlink in copiedblock.exits: - if copiedlink.target is copiedexceptblock: - copiedlink.args = copiedlink.args[:2] - copiedlink.target = graph.exceptblock - for a1, a2 in zip(copiedlink.args, - graph.exceptblock.inputargs): - if hasattr(a2, 'concretetype'): - assert a1.concretetype == a2.concretetype - else: - # if graph.exceptblock was never used before - a2.concretetype = a1.concretetype - else: - def find_args_in_exceptional_case(link, block, etype, evalue): - linkargs = [] - for arg in link.args: - if arg == link.last_exception: - linkargs.append(etype) - elif arg == link.last_exc_value: - linkargs.append(evalue) - elif isinstance(arg, Constant): - linkargs.append(arg) - else: - index = afterblock.inputargs.index(arg) - linkargs.append(passon_vars[block][index - 1]) - return linkargs - exc_match = Constant( - translator.rtyper.getexceptiondata().fn_exception_match) - exc_match.concretetype = typeOf(exc_match.value) - #try to match the exceptions for simple cases - for link in entrymap[graph_to_inline.exceptblock]: - copiedblock = copied_blocks[link.prevblock] - copiedlink = copiedblock.exits[0] - eclass = _find_exception_type(copiedblock) - #print copiedblock.operations - if eclass is None: - continue - etype = copiedlink.args[0] - evalue = copiedlink.args[1] - for exceptionlink in afterblock.exits[1:]: - if exc_match.value(eclass, exceptionlink.llexitcase): - copiedblock.operations += generate_keepalive( - passon_vars[link.prevblock]) - copiedlink.target = exceptionlink.target - linkargs = find_args_in_exceptional_case(exceptionlink, - link.prevblock, - etype, evalue) - copiedlink.args = linkargs - break - #XXXXX don't look: insert blocks that do exception matching - #for the cases where direct matching did not work - blocks = [] - for i, link in enumerate(afterblock.exits[1:]): - etype = copyvar(translator, copiedexceptblock.inputargs[0]) - evalue = copyvar(translator, copiedexceptblock.inputargs[1]) - block = Block([etype, evalue] + get_new_passon_var_names(link.target)) - res = Variable() - res.concretetype = Bool - translator.annotator.bindings[res] = annmodel.SomeBool() - cexitcase = Constant(link.llexitcase) - cexitcase.concretetype = typeOf(cexitcase.value) - args = [exc_match, etype, cexitcase] - block.operations.append(SpaceOperation("direct_call", args, res)) - block.exitswitch = res - linkargs = find_args_in_exceptional_case(link, link.target, - etype, evalue) - l = Link(linkargs, link.target) - l.prevblock = block - l.exitcase = True - l.llexitcase = True - block.exits.append(l) - if i > 0: - l = Link(blocks[-1].inputargs, block) - l.prevblock = blocks[-1] - l.exitcase = False - l.llexitcase = False - blocks[-1].exits.insert(0, l) - blocks.append(block) - blocks[-1].exits = blocks[-1].exits[:1] - blocks[-1].operations = [] - blocks[-1].exitswitch = None - blocks[-1].exits[0].exitcase = None - del blocks[-1].exits[0].llexitcase - linkargs = copiedexceptblock.inputargs - copiedexceptblock.closeblock(Link(linkargs, blocks[0])) - copiedexceptblock.operations += generate_keepalive(linkargs) - if exception_guarded: - assert afterblock.exits[0].exitcase is None - afterblock.exits = [afterblock.exits[0]] - afterblock.exitswitch = None - #cleaning up -- makes sense to be here, because I insert quite - #some empty blocks and blocks that can be joined - checkgraph(graph) - eliminate_empty_blocks(graph) - join_blocks(graph) - remove_identical_vars(graph) + def find_args_in_exceptional_case(self, link, block, etype, evalue, afterblock): + linkargs = [] + for arg in link.args: + if arg == link.last_exception: + linkargs.append(etype) + elif arg == link.last_exc_value: + linkargs.append(evalue) + elif isinstance(arg, Constant): + linkargs.append(arg) + else: + index = afterblock.inputargs.index(arg) + linkargs.append(self.passon_vars[block][index - 1]) + return linkargs + + def do_inline(self, block, index_operation): + # original + op = block.operations[index_operation] + graph_to_inline = op.args[0].value._obj.graph + exception_guarded = False + if (block.exitswitch == c_last_exception and + index_operation == len(block.operations) - 1): + exception_guarded = True + if len(collect_called_graphs(graph_to_inline, self.translator)) != 0: + raise CannotInline("can't handle exceptions yet") + entrymap = mkentrymap(graph_to_inline) + afterblock = split_block(self.translator, self.graph, block, index_operation) + assert afterblock.operations[0] is op + #vars that need to be passed through the blocks of the inlined function + self.passon_vars = { + self.beforeblock: [arg for arg in self.beforeblock.exits[0].args + if isinstance(arg, Variable)]} + + linktoinlined = self.beforeblock.exits[0] + assert linktoinlined.target is afterblock + copiedstartblock = self.copy_block(graph_to_inline.startblock) + copiedstartblock.isstartblock = False + #find args passed to startblock of inlined function + passon_args = [] + for arg in op.args[1:]: + if isinstance(arg, Constant): + passon_args.append(arg) + else: + index = afterblock.inputargs.index(arg) + passon_args.append(linktoinlined.args[index]) + passon_args += self.passon_vars[self.beforeblock] + #rewire blocks + linktoinlined.target = copiedstartblock + linktoinlined.args = passon_args + afterblock.inputargs = [op.result] + afterblock.inputargs + afterblock.operations = self.generate_keepalive(afterblock.inputargs) + afterblock.operations[1:] + if graph_to_inline.returnblock in entrymap: + copiedreturnblock = self.copied_blocks[graph_to_inline.returnblock] # XXX + linkfrominlined = Link([copiedreturnblock.inputargs[0]] + self.passon_vars[graph_to_inline.returnblock], afterblock) + linkfrominlined.prevblock = copiedreturnblock + copiedreturnblock.exitswitch = None + copiedreturnblock.exits = [linkfrominlined] + assert copiedreturnblock.exits[0].target == afterblock + if graph_to_inline.exceptblock in entrymap: + #let links to exceptblock of the graph to inline go to graphs exceptblock + copiedexceptblock = self.copied_blocks[graph_to_inline.exceptblock] #XXX + if not exception_guarded: + # find all copied links that go to copiedexceptblock + for link in entrymap[graph_to_inline.exceptblock]: + copiedblock = self.copied_blocks[link.prevblock] + for copiedlink in copiedblock.exits: + if copiedlink.target is copiedexceptblock: + copiedlink.args = copiedlink.args[:2] + copiedlink.target = self.graph.exceptblock + for a1, a2 in zip(copiedlink.args, + self.graph.exceptblock.inputargs): + if hasattr(a2, 'concretetype'): + assert a1.concretetype == a2.concretetype + else: + # if self.graph.exceptblock was never used before + a2.concretetype = a1.concretetype + else: + exc_match = Constant( + self.translator.rtyper.getexceptiondata().fn_exception_match) + exc_match.concretetype = typeOf(exc_match.value) + #try to match the exceptions for simple cases + for link in entrymap[graph_to_inline.exceptblock]: + copiedblock = self.copied_blocks[link.prevblock] + copiedlink = copiedblock.exits[0] + eclass = _find_exception_type(copiedblock) + #print copiedblock.operations + if eclass is None: + continue + etype = copiedlink.args[0] + evalue = copiedlink.args[1] + for exceptionlink in afterblock.exits[1:]: + if exc_match.value(eclass, exceptionlink.llexitcase): + copiedblock.operations += self.generate_keepalive( + self.passon_vars[link.prevblock]) + copiedlink.target = exceptionlink.target + linkargs = self.find_args_in_exceptional_case(exceptionlink, + link.prevblock, + etype, evalue, afterblock) + copiedlink.args = linkargs + break + #XXXXX don't look: insert blocks that do exception matching + #for the cases where direct matching did not work + blocks = [] + for i, link in enumerate(afterblock.exits[1:]): + etype = copyvar(self.translator, copiedexceptblock.inputargs[0]) + evalue = copyvar(self.translator, copiedexceptblock.inputargs[1]) + block = Block([etype, evalue] + self.get_new_passon_var_names(link.target)) + res = Variable() + res.concretetype = Bool + self.translator.annotator.bindings[res] = annmodel.SomeBool() + cexitcase = Constant(link.llexitcase) + cexitcase.concretetype = typeOf(cexitcase.value) + args = [exc_match, etype, cexitcase] + block.operations.append(SpaceOperation("direct_call", args, res)) + block.exitswitch = res + linkargs = self.find_args_in_exceptional_case(link, link.target, + etype, evalue, afterblock) + l = Link(linkargs, link.target) + l.prevblock = block + l.exitcase = True + l.llexitcase = True + block.exits.append(l) + if i > 0: + l = Link(blocks[-1].inputargs, block) + l.prevblock = blocks[-1] + l.exitcase = False + l.llexitcase = False + blocks[-1].exits.insert(0, l) + blocks.append(block) + blocks[-1].exits = blocks[-1].exits[:1] + blocks[-1].operations = [] + blocks[-1].exitswitch = None + blocks[-1].exits[0].exitcase = None + del blocks[-1].exits[0].llexitcase + linkargs = copiedexceptblock.inputargs + copiedexceptblock.closeblock(Link(linkargs, blocks[0])) + copiedexceptblock.operations += self.generate_keepalive(linkargs) + if exception_guarded: + assert afterblock.exits[0].exitcase is None + afterblock.exits = [afterblock.exits[0]] + afterblock.exitswitch = None + #cleaning up -- makes sense to be here, because I insert quite + #some empty blocks and blocks that can be joined + checkgraph(self.graph) + eliminate_empty_blocks(self.graph) + join_blocks(self.graph) + remove_identical_vars(self.graph) + + +_inline_function = Inliner # ____________________________________________________________ # From cfbolz at codespeak.net Sat Jan 7 23:37:33 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Sat, 7 Jan 2006 23:37:33 +0100 (CET) Subject: [pypy-svn] r21803 - pypy/dist/pypy/translator/backendopt Message-ID: <20060107223733.C1ADE27DB9@code1.codespeak.net> Author: cfbolz Date: Sat Jan 7 23:37:32 2006 New Revision: 21803 Modified: pypy/dist/pypy/translator/backendopt/inline.py Log: some more reshuffling. slowly the code is at least cut up into more handy slices :-) Modified: pypy/dist/pypy/translator/backendopt/inline.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/inline.py (original) +++ pypy/dist/pypy/translator/backendopt/inline.py Sat Jan 7 23:37:32 2006 @@ -91,7 +91,17 @@ self.varmap = {} self.beforeblock = block self.copied_blocks = {} + self.op = block.operations[index_operation] + self.graph_to_inline = self.op.args[0].value._obj.graph + self.exception_guarded = False + if (block.exitswitch == c_last_exception and + index_operation == len(block.operations) - 1): + self.exception_guarded = True + if len(collect_called_graphs(self.graph_to_inline, self.translator)) != 0: + raise CannotInline("can't handle exceptions yet") + self.entrymap = mkentrymap(self.graph_to_inline) self.do_inline(block, index_operation) + self.cleanup() def get_new_name(self, var): if var is None: @@ -161,19 +171,104 @@ linkargs.append(self.passon_vars[block][index - 1]) return linkargs + def rewire_returnblock(self, afterblock): + copiedreturnblock = self.copied_blocks[self.graph_to_inline.returnblock] # XXX + linkfrominlined = Link([copiedreturnblock.inputargs[0]] + self.passon_vars[self.graph_to_inline.returnblock], afterblock) + linkfrominlined.prevblock = copiedreturnblock + copiedreturnblock.exitswitch = None + copiedreturnblock.exits = [linkfrominlined] + assert copiedreturnblock.exits[0].target == afterblock + + def rewire_exceptblock(self, afterblock): + #let links to exceptblock of the graph to inline go to graphs exceptblock + copiedexceptblock = self.copied_blocks[self.graph_to_inline.exceptblock] #XXX + if not self.exception_guarded: + self.rewire_exceptblock_no_guard(afterblock, copiedexceptblock) + else: + self.rewire_exceptblock_with_guard(afterblock, copiedexceptblock) + + def rewire_exceptblock_no_guard(self, afterblock, copiedexceptblock): + # find all copied links that go to copiedexceptblock + for link in self.entrymap[self.graph_to_inline.exceptblock]: + copiedblock = self.copied_blocks[link.prevblock] + for copiedlink in copiedblock.exits: + if copiedlink.target is copiedexceptblock: + copiedlink.args = copiedlink.args[:2] + copiedlink.target = self.graph.exceptblock + for a1, a2 in zip(copiedlink.args, + self.graph.exceptblock.inputargs): + if hasattr(a2, 'concretetype'): + assert a1.concretetype == a2.concretetype + else: + # if self.graph.exceptblock was never used before + a2.concretetype = a1.concretetype + + def rewire_exceptblock_with_guard(self, afterblock, copiedexceptblock): + exc_match = Constant( + self.translator.rtyper.getexceptiondata().fn_exception_match) + exc_match.concretetype = typeOf(exc_match.value) + #try to match the exceptions for simple cases + for link in self.entrymap[self.graph_to_inline.exceptblock]: + copiedblock = self.copied_blocks[link.prevblock] + copiedlink = copiedblock.exits[0] + eclass = _find_exception_type(copiedblock) + #print copiedblock.operations + if eclass is None: + continue + etype = copiedlink.args[0] + evalue = copiedlink.args[1] + for exceptionlink in afterblock.exits[1:]: + if exc_match.value(eclass, exceptionlink.llexitcase): + copiedblock.operations += self.generate_keepalive( + self.passon_vars[link.prevblock]) + copiedlink.target = exceptionlink.target + linkargs = self.find_args_in_exceptional_case( + exceptionlink, link.prevblock, etype, evalue, afterblock) + copiedlink.args = linkargs + break + #XXXXX don't look: insert blocks that do exception matching + #for the cases where direct matching did not work + blocks = [] + for i, link in enumerate(afterblock.exits[1:]): + etype = copyvar(self.translator, copiedexceptblock.inputargs[0]) + evalue = copyvar(self.translator, copiedexceptblock.inputargs[1]) + block = Block([etype, evalue] + self.get_new_passon_var_names(link.target)) + res = Variable() + res.concretetype = Bool + self.translator.annotator.bindings[res] = annmodel.SomeBool() + cexitcase = Constant(link.llexitcase) + cexitcase.concretetype = typeOf(cexitcase.value) + args = [exc_match, etype, cexitcase] + block.operations.append(SpaceOperation("direct_call", args, res)) + block.exitswitch = res + linkargs = self.find_args_in_exceptional_case(link, link.target, + etype, evalue, afterblock) + l = Link(linkargs, link.target) + l.prevblock = block + l.exitcase = True + l.llexitcase = True + block.exits.append(l) + if i > 0: + l = Link(blocks[-1].inputargs, block) + l.prevblock = blocks[-1] + l.exitcase = False + l.llexitcase = False + blocks[-1].exits.insert(0, l) + blocks.append(block) + blocks[-1].exits = blocks[-1].exits[:1] + blocks[-1].operations = [] + blocks[-1].exitswitch = None + blocks[-1].exits[0].exitcase = None + del blocks[-1].exits[0].llexitcase + linkargs = copiedexceptblock.inputargs + copiedexceptblock.closeblock(Link(linkargs, blocks[0])) + copiedexceptblock.operations += self.generate_keepalive(linkargs) + + def do_inline(self, block, index_operation): # original - op = block.operations[index_operation] - graph_to_inline = op.args[0].value._obj.graph - exception_guarded = False - if (block.exitswitch == c_last_exception and - index_operation == len(block.operations) - 1): - exception_guarded = True - if len(collect_called_graphs(graph_to_inline, self.translator)) != 0: - raise CannotInline("can't handle exceptions yet") - entrymap = mkentrymap(graph_to_inline) afterblock = split_block(self.translator, self.graph, block, index_operation) - assert afterblock.operations[0] is op + assert afterblock.operations[0] is self.op #vars that need to be passed through the blocks of the inlined function self.passon_vars = { self.beforeblock: [arg for arg in self.beforeblock.exits[0].args @@ -181,11 +276,11 @@ linktoinlined = self.beforeblock.exits[0] assert linktoinlined.target is afterblock - copiedstartblock = self.copy_block(graph_to_inline.startblock) + copiedstartblock = self.copy_block(self.graph_to_inline.startblock) copiedstartblock.isstartblock = False #find args passed to startblock of inlined function passon_args = [] - for arg in op.args[1:]: + for arg in self.op.args[1:]: if isinstance(arg, Constant): passon_args.append(arg) else: @@ -195,100 +290,21 @@ #rewire blocks linktoinlined.target = copiedstartblock linktoinlined.args = passon_args - afterblock.inputargs = [op.result] + afterblock.inputargs + afterblock.inputargs = [self.op.result] + afterblock.inputargs afterblock.operations = self.generate_keepalive(afterblock.inputargs) + afterblock.operations[1:] - if graph_to_inline.returnblock in entrymap: - copiedreturnblock = self.copied_blocks[graph_to_inline.returnblock] # XXX - linkfrominlined = Link([copiedreturnblock.inputargs[0]] + self.passon_vars[graph_to_inline.returnblock], afterblock) - linkfrominlined.prevblock = copiedreturnblock - copiedreturnblock.exitswitch = None - copiedreturnblock.exits = [linkfrominlined] - assert copiedreturnblock.exits[0].target == afterblock - if graph_to_inline.exceptblock in entrymap: - #let links to exceptblock of the graph to inline go to graphs exceptblock - copiedexceptblock = self.copied_blocks[graph_to_inline.exceptblock] #XXX - if not exception_guarded: - # find all copied links that go to copiedexceptblock - for link in entrymap[graph_to_inline.exceptblock]: - copiedblock = self.copied_blocks[link.prevblock] - for copiedlink in copiedblock.exits: - if copiedlink.target is copiedexceptblock: - copiedlink.args = copiedlink.args[:2] - copiedlink.target = self.graph.exceptblock - for a1, a2 in zip(copiedlink.args, - self.graph.exceptblock.inputargs): - if hasattr(a2, 'concretetype'): - assert a1.concretetype == a2.concretetype - else: - # if self.graph.exceptblock was never used before - a2.concretetype = a1.concretetype - else: - exc_match = Constant( - self.translator.rtyper.getexceptiondata().fn_exception_match) - exc_match.concretetype = typeOf(exc_match.value) - #try to match the exceptions for simple cases - for link in entrymap[graph_to_inline.exceptblock]: - copiedblock = self.copied_blocks[link.prevblock] - copiedlink = copiedblock.exits[0] - eclass = _find_exception_type(copiedblock) - #print copiedblock.operations - if eclass is None: - continue - etype = copiedlink.args[0] - evalue = copiedlink.args[1] - for exceptionlink in afterblock.exits[1:]: - if exc_match.value(eclass, exceptionlink.llexitcase): - copiedblock.operations += self.generate_keepalive( - self.passon_vars[link.prevblock]) - copiedlink.target = exceptionlink.target - linkargs = self.find_args_in_exceptional_case(exceptionlink, - link.prevblock, - etype, evalue, afterblock) - copiedlink.args = linkargs - break - #XXXXX don't look: insert blocks that do exception matching - #for the cases where direct matching did not work - blocks = [] - for i, link in enumerate(afterblock.exits[1:]): - etype = copyvar(self.translator, copiedexceptblock.inputargs[0]) - evalue = copyvar(self.translator, copiedexceptblock.inputargs[1]) - block = Block([etype, evalue] + self.get_new_passon_var_names(link.target)) - res = Variable() - res.concretetype = Bool - self.translator.annotator.bindings[res] = annmodel.SomeBool() - cexitcase = Constant(link.llexitcase) - cexitcase.concretetype = typeOf(cexitcase.value) - args = [exc_match, etype, cexitcase] - block.operations.append(SpaceOperation("direct_call", args, res)) - block.exitswitch = res - linkargs = self.find_args_in_exceptional_case(link, link.target, - etype, evalue, afterblock) - l = Link(linkargs, link.target) - l.prevblock = block - l.exitcase = True - l.llexitcase = True - block.exits.append(l) - if i > 0: - l = Link(blocks[-1].inputargs, block) - l.prevblock = blocks[-1] - l.exitcase = False - l.llexitcase = False - blocks[-1].exits.insert(0, l) - blocks.append(block) - blocks[-1].exits = blocks[-1].exits[:1] - blocks[-1].operations = [] - blocks[-1].exitswitch = None - blocks[-1].exits[0].exitcase = None - del blocks[-1].exits[0].llexitcase - linkargs = copiedexceptblock.inputargs - copiedexceptblock.closeblock(Link(linkargs, blocks[0])) - copiedexceptblock.operations += self.generate_keepalive(linkargs) - if exception_guarded: + if self.graph_to_inline.returnblock in self.entrymap: + self.rewire_returnblock(afterblock) + if self.graph_to_inline.exceptblock in self.entrymap: + self.rewire_exceptblock(afterblock) + if self.exception_guarded: assert afterblock.exits[0].exitcase is None afterblock.exits = [afterblock.exits[0]] afterblock.exitswitch = None - #cleaning up -- makes sense to be here, because I insert quite - #some empty blocks and blocks that can be joined + + def cleanup(self): + """ cleaning up -- makes sense to be done after inlining, because the + inliner inserted quite some empty blocks and blocks that can be + joined. """ checkgraph(self.graph) eliminate_empty_blocks(self.graph) join_blocks(self.graph) From cfbolz at codespeak.net Sun Jan 8 00:26:18 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Sun, 8 Jan 2006 00:26:18 +0100 (CET) Subject: [pypy-svn] r21804 - pypy/dist/pypy/translator/backendopt Message-ID: <20060107232618.E481027DB9@code1.codespeak.net> Author: cfbolz Date: Sun Jan 8 00:26:17 2006 New Revision: 21804 Modified: pypy/dist/pypy/translator/backendopt/inline.py Log: * split up rewire_exceptblock_with_guard into now more or less manageable chunks * clarify usage of _find_exception_type a bit (thanks samuele) Modified: pypy/dist/pypy/translator/backendopt/inline.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/inline.py (original) +++ pypy/dist/pypy/translator/backendopt/inline.py Sun Jan 8 00:26:17 2006 @@ -79,9 +79,8 @@ block.exits[0].args[1] != ops[-1].result or not isinstance(ops[-4].args[1], Constant) or ops[-4].args[1].value != "typeptr"): - return None - return ops[-4].args[2].value - + return None, None + return ops[-4].args[2].value, block.exits[0] class Inliner(object): @@ -185,7 +184,11 @@ if not self.exception_guarded: self.rewire_exceptblock_no_guard(afterblock, copiedexceptblock) else: + # first try to match exceptions using a very simple heuristic self.rewire_exceptblock_with_guard(afterblock, copiedexceptblock) + # generate blocks that do generic matching for cases when the + # heuristic did not work + self.generic_exception_matching(afterblock, copiedexceptblock) def rewire_exceptblock_no_guard(self, afterblock, copiedexceptblock): # find all copied links that go to copiedexceptblock @@ -204,21 +207,19 @@ a2.concretetype = a1.concretetype def rewire_exceptblock_with_guard(self, afterblock, copiedexceptblock): - exc_match = Constant( - self.translator.rtyper.getexceptiondata().fn_exception_match) - exc_match.concretetype = typeOf(exc_match.value) - #try to match the exceptions for simple cases + # this rewiring does not always succeed. in the cases where it doesn't + # there will be generic code inserted + exc_match = self.translator.rtyper.getexceptiondata().fn_exception_match for link in self.entrymap[self.graph_to_inline.exceptblock]: copiedblock = self.copied_blocks[link.prevblock] - copiedlink = copiedblock.exits[0] - eclass = _find_exception_type(copiedblock) + eclass, copiedlink = _find_exception_type(copiedblock) #print copiedblock.operations if eclass is None: continue etype = copiedlink.args[0] evalue = copiedlink.args[1] for exceptionlink in afterblock.exits[1:]: - if exc_match.value(eclass, exceptionlink.llexitcase): + if exc_match(eclass, exceptionlink.llexitcase): copiedblock.operations += self.generate_keepalive( self.passon_vars[link.prevblock]) copiedlink.target = exceptionlink.target @@ -226,8 +227,13 @@ exceptionlink, link.prevblock, etype, evalue, afterblock) copiedlink.args = linkargs break + + def generic_exception_matching(self, afterblock, copiedexceptblock): #XXXXX don't look: insert blocks that do exception matching #for the cases where direct matching did not work + exc_match = Constant( + self.translator.rtyper.getexceptiondata().fn_exception_match) + exc_match.concretetype = typeOf(exc_match.value) blocks = [] for i, link in enumerate(afterblock.exits[1:]): etype = copyvar(self.translator, copiedexceptblock.inputargs[0]) From cfbolz at codespeak.net Sun Jan 8 01:27:28 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Sun, 8 Jan 2006 01:27:28 +0100 (CET) Subject: [pypy-svn] r21805 - pypy/dist/pypy/translator/backendopt Message-ID: <20060108002728.EC02327DB9@code1.codespeak.net> Author: cfbolz Date: Sun Jan 8 01:27:27 2006 New Revision: 21805 Modified: pypy/dist/pypy/translator/backendopt/inline.py Log: * consistently use copy_blocks everywhere (instead of fishing around in copied_blocks) * disentangle the mess around passon_vars: now there is only a method passon_vars that creates a copy of original_passon_vars for each given cachekey. Modified: pypy/dist/pypy/translator/backendopt/inline.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/inline.py (original) +++ pypy/dist/pypy/translator/backendopt/inline.py Sun Jan 8 01:27:27 2006 @@ -89,7 +89,7 @@ self.graph = graph self.varmap = {} self.beforeblock = block - self.copied_blocks = {} + self._copied_blocks = {} self.op = block.operations[index_operation] self.graph_to_inline = self.op.args[0].value._obj.graph self.exception_guarded = False @@ -98,6 +98,7 @@ self.exception_guarded = True if len(collect_called_graphs(self.graph_to_inline, self.translator)) != 0: raise CannotInline("can't handle exceptions yet") + self._passon_vars = {} self.entrymap = mkentrymap(self.graph_to_inline) self.do_inline(block, index_operation) self.cleanup() @@ -111,9 +112,12 @@ self.varmap[var] = copyvar(self.translator, var) return self.varmap[var] - def get_new_passon_var_names(self, block): - result = [copyvar(self.translator, var) for var in self.passon_vars[self.beforeblock]] - self.passon_vars[block] = result + def passon_vars(self, cache_key): + if cache_key in self._passon_vars: + return self._passon_vars[cache_key] + result = [copyvar(self.translator, var) + for var in self.original_passon_vars] + self._passon_vars[cache_key] = result return result def copy_operation(self, op): @@ -121,13 +125,12 @@ return SpaceOperation(op.opname, args, self.get_new_name(op.result)) def copy_block(self, block): - if block in self.copied_blocks: - "already there" - return self.copied_blocks[block] + if block in self._copied_blocks: + return self._copied_blocks[block] args = ([self.get_new_name(var) for var in block.inputargs] + - self.get_new_passon_var_names(block)) + self.passon_vars(block)) newblock = Block(args) - self.copied_blocks[block] = newblock + self._copied_blocks[block] = newblock newblock.operations = [self.copy_operation(op) for op in block.operations] newblock.exits = [self.copy_link(link, block) for link in block.exits] newblock.exitswitch = self.get_new_name(block.exitswitch) @@ -135,7 +138,7 @@ return newblock def copy_link(self, link, prevblock): - newargs = [self.get_new_name(a) for a in link.args] + self.passon_vars[prevblock] + newargs = [self.get_new_name(a) for a in link.args] + self.passon_vars(prevblock) newlink = Link(newargs, self.copy_block(link.target), link.exitcase) newlink.prevblock = self.copy_block(link.prevblock) newlink.last_exception = self.get_new_name(link.last_exception) @@ -156,7 +159,7 @@ keepalive_ops.append(SpaceOperation('keepalive', [v], v_keepalive)) return keepalive_ops - def find_args_in_exceptional_case(self, link, block, etype, evalue, afterblock): + def find_args_in_exceptional_case(self, link, block, etype, evalue, afterblock, passon_vars): linkargs = [] for arg in link.args: if arg == link.last_exception: @@ -167,12 +170,14 @@ linkargs.append(arg) else: index = afterblock.inputargs.index(arg) - linkargs.append(self.passon_vars[block][index - 1]) + linkargs.append(passon_vars[index - 1]) return linkargs def rewire_returnblock(self, afterblock): - copiedreturnblock = self.copied_blocks[self.graph_to_inline.returnblock] # XXX - linkfrominlined = Link([copiedreturnblock.inputargs[0]] + self.passon_vars[self.graph_to_inline.returnblock], afterblock) + copiedreturnblock = self.copy_block(self.graph_to_inline.returnblock) + linkargs = ([copiedreturnblock.inputargs[0]] + + self.passon_vars(self.graph_to_inline.returnblock)) + linkfrominlined = Link(linkargs, afterblock) linkfrominlined.prevblock = copiedreturnblock copiedreturnblock.exitswitch = None copiedreturnblock.exits = [linkfrominlined] @@ -180,7 +185,7 @@ def rewire_exceptblock(self, afterblock): #let links to exceptblock of the graph to inline go to graphs exceptblock - copiedexceptblock = self.copied_blocks[self.graph_to_inline.exceptblock] #XXX + copiedexceptblock = self.copy_block(self.graph_to_inline.exceptblock) if not self.exception_guarded: self.rewire_exceptblock_no_guard(afterblock, copiedexceptblock) else: @@ -193,7 +198,7 @@ def rewire_exceptblock_no_guard(self, afterblock, copiedexceptblock): # find all copied links that go to copiedexceptblock for link in self.entrymap[self.graph_to_inline.exceptblock]: - copiedblock = self.copied_blocks[link.prevblock] + copiedblock = self.copy_block(link.prevblock) for copiedlink in copiedblock.exits: if copiedlink.target is copiedexceptblock: copiedlink.args = copiedlink.args[:2] @@ -211,7 +216,7 @@ # there will be generic code inserted exc_match = self.translator.rtyper.getexceptiondata().fn_exception_match for link in self.entrymap[self.graph_to_inline.exceptblock]: - copiedblock = self.copied_blocks[link.prevblock] + copiedblock = self.copy_block(link.prevblock) eclass, copiedlink = _find_exception_type(copiedblock) #print copiedblock.operations if eclass is None: @@ -220,11 +225,11 @@ evalue = copiedlink.args[1] for exceptionlink in afterblock.exits[1:]: if exc_match(eclass, exceptionlink.llexitcase): - copiedblock.operations += self.generate_keepalive( - self.passon_vars[link.prevblock]) + passon_vars = self.passon_vars(link.prevblock) + copiedblock.operations += self.generate_keepalive(passon_vars) copiedlink.target = exceptionlink.target linkargs = self.find_args_in_exceptional_case( - exceptionlink, link.prevblock, etype, evalue, afterblock) + exceptionlink, link.prevblock, etype, evalue, afterblock, passon_vars) copiedlink.args = linkargs break @@ -238,7 +243,8 @@ for i, link in enumerate(afterblock.exits[1:]): etype = copyvar(self.translator, copiedexceptblock.inputargs[0]) evalue = copyvar(self.translator, copiedexceptblock.inputargs[1]) - block = Block([etype, evalue] + self.get_new_passon_var_names(link.target)) + passon_vars = self.passon_vars(i) + block = Block([etype, evalue] + passon_vars) res = Variable() res.concretetype = Bool self.translator.annotator.bindings[res] = annmodel.SomeBool() @@ -248,7 +254,8 @@ block.operations.append(SpaceOperation("direct_call", args, res)) block.exitswitch = res linkargs = self.find_args_in_exceptional_case(link, link.target, - etype, evalue, afterblock) + etype, evalue, afterblock, + passon_vars) l = Link(linkargs, link.target) l.prevblock = block l.exitcase = True @@ -272,14 +279,16 @@ def do_inline(self, block, index_operation): - # original afterblock = split_block(self.translator, self.graph, block, index_operation) + # these variables have to be passed along all the links in the inlined + # graph because the original function needs them in the blocks after + # the inlined function + # for every inserted block we need a new copy of these variables, + # this copy is created with the method passon_vars + self.original_passon_vars = [arg for arg in self.beforeblock.exits[0].args + if isinstance(arg, Variable)] assert afterblock.operations[0] is self.op #vars that need to be passed through the blocks of the inlined function - self.passon_vars = { - self.beforeblock: [arg for arg in self.beforeblock.exits[0].args - if isinstance(arg, Variable)]} - linktoinlined = self.beforeblock.exits[0] assert linktoinlined.target is afterblock copiedstartblock = self.copy_block(self.graph_to_inline.startblock) @@ -292,7 +301,7 @@ else: index = afterblock.inputargs.index(arg) passon_args.append(linktoinlined.args[index]) - passon_args += self.passon_vars[self.beforeblock] + passon_args += self.original_passon_vars #rewire blocks linktoinlined.target = copiedstartblock linktoinlined.args = passon_args From ale at codespeak.net Sun Jan 8 13:41:33 2006 From: ale at codespeak.net (ale at codespeak.net) Date: Sun, 8 Jan 2006 13:41:33 +0100 (CET) Subject: [pypy-svn] r21813 - in pypy/dist/pypy/lib/pyontology: . test Message-ID: <20060108124133.9A21A27DB9@code1.codespeak.net> Author: ale Date: Sun Jan 8 13:41:32 2006 New Revision: 21813 Modified: pypy/dist/pypy/lib/pyontology/pyontology.py pypy/dist/pypy/lib/pyontology/test/test_ontology.py Log: Added support for restrictions with a test Modified: pypy/dist/pypy/lib/pyontology/pyontology.py ============================================================================== --- pypy/dist/pypy/lib/pyontology/pyontology.py (original) +++ pypy/dist/pypy/lib/pyontology/pyontology.py Sun Jan 8 13:41:32 2006 @@ -149,8 +149,22 @@ self.constraint = SymmetricConstraint(name) class Restriction(ClassDomain): - pass - + """ A owl:restriction is an anonymous class that links a class to a restriction on a property + The restriction is only applied to the property in the conntext of the specific task. In order + to construct a constraint to check the restriction three things are thus needed : + 1. The property to which the restriction applies - this comes from the onProperty tripple. + the property is saved in the Restriction class' property attribute + 2. The restriction itself. This comes from one of the property restrictions triples (oneOf, + maxCardinality ....). It adds a constraint class + 3. The class in which context the restriction should be applied. This comes from subClassOf, + The class is saved in the restrictions clsattribute + """ + def __init__(self, name='', values=[], bases = []): + ClassDomain.__init__(self, name, values, bases) + self.constraint = RestrictionConstraint(name) + self.property = None + self.cls = None + builtin_voc = { 'Thing' : Thing, 'Class' : ClassDomain, @@ -298,8 +312,10 @@ self.constraints.append(constrain) else: # var is a builtin class + svar = self.make_var(None, s) cls =builtin_voc[var.split('#')[-1]] - svar = self.make_var(cls, s) + if not (self.variables.has_key(svar) and isinstance(self.variables, cls)): + svar = self.make_var(cls, s) cls = self.variables[svar] if hasattr(cls, 'constraint'): self.constraints.append(cls.constraint) @@ -324,7 +340,9 @@ self.variables[svar].setValues(vals) def onProperty(self, s, var): - pass + svar =self.make_var(Restriction, s) + avar =self.make_var(Property, var) + self.variables[svar].property = avar #---Class Axioms---#000000#FFFFFF----------------------------------------------- @@ -333,7 +351,7 @@ # class extension of s is a subset of the # class extension of var. avar = self.make_var(None, var) - svar = self.make_var(None, s) + svar = self.make_var(ClassDomain, s) cons = SubClassConstraint( svar, avar) self.constraints.append(cons) @@ -355,13 +373,10 @@ pass def oneOf(self, s, var): - res = self.get_list(var) - #prop = self.find_uriref(s) - avar = self.make_var(None, s) - if self.variables.get(avar) and type(self.variables[avar]) == fd: - self.variables[avar] = fd(list(self.variables[avar].getValues()) + res) - else: - self.variables[avar] = fd(res) + avar = self.make_var(List, var) + svar = self.make_var(None, s) + cons = OneofPropertyConstraint(svar, avar) + self.constraints.append(cons) def unionOf(self,s, var): res = self.get_list(var) @@ -414,9 +429,9 @@ def maxCardinality(self, s, var): """ Len of finite domain of the property shall be less than or equal to var""" - avar = self.find_property(s) - cls =self.make_var(None, self.find_cls(s)) - constrain = MaxCardinality(avar, cls, int(var)) + svar =self.make_var(Restriction, s) + avar =self.make_var(None, var) + constrain = MaxCardinality(avar, cls, var) self.constraints.append(constrain) def minCardinality(self, s, var): @@ -496,6 +511,7 @@ def __init__(self, variable, cls, cardinality): AbstractConstraint.__init__(self, [variable, cls]) self.__cost = 2 + self.variable = variable self.cardinality = cardinality self.cls = cls @@ -506,7 +522,7 @@ """narrowing algorithm for the constraint""" prop = Linkeddict(domains[self.variable].getValues()) if len(prop[self.cls]) > self.cardinality: - raise ConsistencyFailure("Maxcardinality exceeded") + raise ConsistencyFailure("Maxcardinality of %i exceeded by the value %i" %(self.cardinality,len(prop[self.cls]))) else: return 1 @@ -516,7 +532,7 @@ """narrowing algorithm for the constraint""" prop = Linkeddict(domains[self.variable].getValues()) if len(prop[self.cls]) < self.cardinality: - raise ConsistencyFailure("MinCardinality not accomplished") + raise ConsistencyFailure("MinCardinalityof %i exceeded by the value %i" %(self.cardinality,len(prop[self.cls]))) else: return 1 @@ -526,7 +542,7 @@ """narrowing algorithm for the constraint""" prop = Linkeddict(domains[self.variable].getValues()) if len(prop[self.cls]) != self.cardinality: - raise ConsistencyFailure("Cardinality constraint not met") + raise ConsistencyFailure("Cardinality of %i exceeded by the value %i" %(self.cardinality,len(prop[self.cls]))) else: return 1 @@ -556,6 +572,8 @@ def narrow(self, domains): subdom = domains[self.variable] superdom = domains[self.object] + if isinstance(superdom, Restriction): + superdom.cls = self.variable bases = get_values(superdom, domains, 'getBases') subdom.bases += [bas for bas in bases if bas not in subdom.bases] vals = get_values(subdom, domains, 'getValues') @@ -741,5 +759,41 @@ domains[self.variable].setValues(vals) return 1 +class RestrictionConstraint(AbstractConstraint): + def __init__(self, variable): + AbstractConstraint.__init__(self, [variable]) + self.variable = variable + + def estimateCost(self, domains): + return 90 + + def narrow(self, domains): + prop = domains[self.variable].property + cls = domains[self.variable].cls + props = domains[prop].getValues() + props.append((cls, None)) + domains[prop].setValues(props) + +class OneofPropertyConstraint(AbstractConstraint): + + def __init__(self, variable, List): + AbstractConstraint.__init__(self, [variable, List]) + self.variable = variable + self.List = List + + def estimateCost(self, domains): + return 100 + + def narrow(self, domains): + property = domains[self.variable].property + cls = domains[self.variable].cls + val = domains[self.List].getValues() + prop = Linkeddict(domains[property].getValues()) + for v in prop[cls]: + if not v in val: + raise ConsistencyFailure( + "The value of the property %s in the class %s is not oneof %r" + %(property, cls, val)) + class HasvalueConstraint: pass Modified: pypy/dist/pypy/lib/pyontology/test/test_ontology.py ============================================================================== --- pypy/dist/pypy/lib/pyontology/test/test_ontology.py (original) +++ pypy/dist/pypy/lib/pyontology/test/test_ontology.py Sun Jan 8 13:41:32 2006 @@ -331,4 +331,30 @@ O.rest( URIRef('2'), URIRef(namespaces['rdf']+'#nil')) assert len(O.constraints) == 1 O.consistency(5) - assert O.rep._domains['favlist_'].getValues() == [0,1,2] \ No newline at end of file + assert O.rep._domains['favlist_'].getValues() == [0,1,2] + +def test_oneofrestriction(): + O = Ontology() + restrict = BNode('anon') + obj = URIRef(namespaces['owl']+'#Restriction') + O.type(restrict, obj) + p = URIRef('p') + O.onProperty(restrict,p) + obj = URIRef(namespaces['owl']+'#ObjectProperty') + O.type(p, obj) + p = URIRef('favlist') + O.oneOf(restrict, p) + own = URIRef('favlist') + obj = URIRef(namespaces['rdf']+'#List') + O.type(own, obj) + O.first(own, 0) + O.rest(own, URIRef('1')) + O.first( URIRef('1'), 1) + O.rest( URIRef('1'), URIRef('2')) + O.first( URIRef('2'), 2) + O.rest( URIRef('2'), URIRef(namespaces['rdf']+'#nil')) + own = URIRef('class') + obj = URIRef(namespaces['rdf']+'#Class') + O.type(own, obj) + O.subClassOf(own,restrict) + py.test.raises(ConsistencyFailure, O.consistency) From ale at codespeak.net Sun Jan 8 13:54:45 2006 From: ale at codespeak.net (ale at codespeak.net) Date: Sun, 8 Jan 2006 13:54:45 +0100 (CET) Subject: [pypy-svn] r21814 - in pypy/dist/pypy/lib/pyontology: . test Message-ID: <20060108125445.ED59A27DB9@code1.codespeak.net> Author: ale Date: Sun Jan 8 13:54:44 2006 New Revision: 21814 Modified: pypy/dist/pypy/lib/pyontology/pyontology.py pypy/dist/pypy/lib/pyontology/test/test_ontology.py Log: Added a test for class enumeration using oneOf Modified: pypy/dist/pypy/lib/pyontology/pyontology.py ============================================================================== --- pypy/dist/pypy/lib/pyontology/pyontology.py (original) +++ pypy/dist/pypy/lib/pyontology/pyontology.py Sun Jan 8 13:54:44 2006 @@ -785,15 +785,18 @@ return 100 def narrow(self, domains): - property = domains[self.variable].property - cls = domains[self.variable].cls val = domains[self.List].getValues() - prop = Linkeddict(domains[property].getValues()) - for v in prop[cls]: - if not v in val: - raise ConsistencyFailure( - "The value of the property %s in the class %s is not oneof %r" - %(property, cls, val)) - + if isinstance(domains[self.variable],Restriction): + property = domains[self.variable].property + cls = domains[self.variable].cls + prop = Linkeddict(domains[property].getValues()) + for v in prop[cls]: + if not v in val: + raise ConsistencyFailure( + "The value of the property %s in the class %s is not oneof %r" + %(property, cls, val)) + else: + domains[self.variable].setValues(val) + return 1 class HasvalueConstraint: pass Modified: pypy/dist/pypy/lib/pyontology/test/test_ontology.py ============================================================================== --- pypy/dist/pypy/lib/pyontology/test/test_ontology.py (original) +++ pypy/dist/pypy/lib/pyontology/test/test_ontology.py Sun Jan 8 13:54:44 2006 @@ -358,3 +358,20 @@ O.type(own, obj) O.subClassOf(own,restrict) py.test.raises(ConsistencyFailure, O.consistency) + +def test_oneofclassenumeration(): + O = Ontology() + restrict = BNode('anon') + own = URIRef('favlist') + obj = URIRef(namespaces['rdf']+'#List') + O.type(own, obj) + O.first(own, URIRef('first')) + O.rest(own, URIRef('1')) + O.first( URIRef('1'), URIRef('second')) + O.rest( URIRef('1'), URIRef('2')) + O.first( URIRef('2'), URIRef('third')) + O.rest( URIRef('2'), URIRef(namespaces['rdf']+'#nil')) + O.oneOf(restrict, own) + O.type(restrict, namespaces['owl']+'#Class') + O.consistency(4) + assert len(O.rep._domains[restrict].getValues()) == 3 \ No newline at end of file From ale at codespeak.net Sun Jan 8 14:58:22 2006 From: ale at codespeak.net (ale at codespeak.net) Date: Sun, 8 Jan 2006 14:58:22 +0100 (CET) Subject: [pypy-svn] r21815 - in pypy/dist/pypy/lib/pyontology: . test Message-ID: <20060108135822.0D50427DB9@code1.codespeak.net> Author: ale Date: Sun Jan 8 14:58:22 2006 New Revision: 21815 Modified: pypy/dist/pypy/lib/pyontology/pyontology.py pypy/dist/pypy/lib/pyontology/test/test_ontology.py Log: Added a test for oneof with a DataRange as object Modified: pypy/dist/pypy/lib/pyontology/pyontology.py ============================================================================== --- pypy/dist/pypy/lib/pyontology/pyontology.py (original) +++ pypy/dist/pypy/lib/pyontology/pyontology.py Sun Jan 8 14:58:22 2006 @@ -233,7 +233,7 @@ return Solver().solve(rep, verbose) def consistency(self, verbose=0): - self.rep = Repository(self.variables.keys(), self.variables, self.constraints[:]) + self.rep = Repository(self.variables.keys(), self.variables, self.constraints) self.rep.consistency(verbose) def get_list(self, subject): @@ -267,11 +267,11 @@ return var if not var in self.variables.keys(): self.variables[var] = cls(var) - elif type(self.variables[var]) != cls: - olddom = self.variables[var] - newdom = cls(var) - newdom.setValues(olddom.getValues()) - self.variables[var] = newdom +## elif type(self.variables[var]) != cls: +## olddom = self.variables[var] +## newdom = cls(var) +## newdom.setValues(olddom.getValues()) +## self.variables[var] = newdom return var def find_prop(self, s): @@ -374,7 +374,7 @@ def oneOf(self, s, var): avar = self.make_var(List, var) - svar = self.make_var(None, s) + svar = self.make_var(ClassDomain, s) cons = OneofPropertyConstraint(svar, avar) self.constraints.append(cons) @@ -430,15 +430,15 @@ def maxCardinality(self, s, var): """ Len of finite domain of the property shall be less than or equal to var""" svar =self.make_var(Restriction, s) - avar =self.make_var(None, var) - constrain = MaxCardinality(avar, cls, var) + #avar =self.make_var(None, var) + constrain = MaxCardinality(svar, None, int(var)) self.constraints.append(constrain) def minCardinality(self, s, var): """ Len of finite domain of the property shall be greater than or equal to var""" avar = self.find_property(s) cls =self.make_var(None, self.find_cls(s)) - constrain = MinCardinality(avar, cls, int(var)) + constrain = MinCardinality(avar, None, int(var)) self.constraints.append(constrain) def cardinality(self, s, var): @@ -446,7 +446,7 @@ avar = self.find_property(s) cls =self.make_var(None, self.find_cls(s)) # Check if var is an int, else find the int buried in the structure - constrain = Cardinality(avar, cls, int(var)) + constrain = Cardinality(avar, None, int(var)) self.constraints.append(constrain) def differentFrom(self, s, var): @@ -600,6 +600,9 @@ class RangeConstraint(SubClassConstraint): + def estimateCost(self, domains): + return 200 + def narrow(self, domains): propdom = domains[self.variable] rangedom = domains[self.object] @@ -798,5 +801,6 @@ else: domains[self.variable].setValues(val) return 1 + class HasvalueConstraint: pass Modified: pypy/dist/pypy/lib/pyontology/test/test_ontology.py ============================================================================== --- pypy/dist/pypy/lib/pyontology/test/test_ontology.py (original) +++ pypy/dist/pypy/lib/pyontology/test/test_ontology.py Sun Jan 8 14:58:22 2006 @@ -374,4 +374,21 @@ O.oneOf(restrict, own) O.type(restrict, namespaces['owl']+'#Class') O.consistency(4) + assert len(O.rep._domains[restrict].getValues()) == 3 + +def test_oneofdatarange(): + O = Ontology() + restrict = BNode('anon') + own = URIRef('favlist') + obj = URIRef(namespaces['rdf']+'#List') + O.type(own, obj) + O.first(own, URIRef('first')) + O.rest(own, URIRef('1')) + O.first( URIRef('1'), URIRef('second')) + O.rest( URIRef('1'), URIRef('2')) + O.first( URIRef('2'), URIRef('third')) + O.rest( URIRef('2'), URIRef(namespaces['rdf']+'#nil')) + O.oneOf(restrict, own) + O.type(restrict, namespaces['owl']+'#DataRange') + O.consistency(4) assert len(O.rep._domains[restrict].getValues()) == 3 \ No newline at end of file From ale at codespeak.net Sun Jan 8 15:30:55 2006 From: ale at codespeak.net (ale at codespeak.net) Date: Sun, 8 Jan 2006 15:30:55 +0100 (CET) Subject: [pypy-svn] r21816 - in pypy/dist/pypy/lib/pyontology: . test Message-ID: <20060108143055.8D43027B66@code1.codespeak.net> Author: ale Date: Sun Jan 8 15:30:54 2006 New Revision: 21816 Modified: pypy/dist/pypy/lib/pyontology/pyontology.py pypy/dist/pypy/lib/pyontology/test/test_ontology.py Log: Added support for SomeValuesFrom and a test Modified: pypy/dist/pypy/lib/pyontology/pyontology.py ============================================================================== --- pypy/dist/pypy/lib/pyontology/pyontology.py (original) +++ pypy/dist/pypy/lib/pyontology/pyontology.py Sun Jan 8 15:30:54 2006 @@ -482,8 +482,10 @@ pass def someValuesFrom(self, s, var): - # TODO: implement this - pass + svar = self.make_var(Restriction, s) + avar = self.make_var(None, var) + constrain = SomeValueConstraint(svar, avar) + self.constraints.append(constrain) def imports(self, s, var): # PP TODO: implement this @@ -802,5 +804,29 @@ domains[self.variable].setValues(val) return 1 +class SomeValueConstraint(AbstractConstraint): + + def __init__(self, variable, List): + AbstractConstraint.__init__(self, [variable, List]) + self.variable = variable + self.List = List + + def estimateCost(self, domains): + return 100 + + def narrow(self, domains): + print self.variable,self.List + val = domains[self.List].getValues() + property = domains[self.variable].property + cls = domains[self.variable].cls + prop = Linkeddict(domains[property].getValues()) + for v in prop[cls]: + if v in val: + break + else: + raise ConsistencyFailure( + "The value of the property %s in the class %s has no values from %r" + %(property, cls, val)) + class HasvalueConstraint: pass Modified: pypy/dist/pypy/lib/pyontology/test/test_ontology.py ============================================================================== --- pypy/dist/pypy/lib/pyontology/test/test_ontology.py (original) +++ pypy/dist/pypy/lib/pyontology/test/test_ontology.py Sun Jan 8 15:30:54 2006 @@ -391,4 +391,33 @@ O.oneOf(restrict, own) O.type(restrict, namespaces['owl']+'#DataRange') O.consistency(4) - assert len(O.rep._domains[restrict].getValues()) == 3 \ No newline at end of file + assert len(O.rep._domains[restrict].getValues()) == 3 + +def test_somevaluesfrom_datarange(): + O = Ontology() + datarange = BNode('anon') + own = URIRef('favlist') + obj = URIRef(namespaces['rdf']+'#List') + O.type(own, obj) + O.first(own, URIRef('first')) + O.rest(own, URIRef('1')) + O.first( URIRef('1'), URIRef('second')) + O.rest( URIRef('1'), URIRef('2')) + O.first( URIRef('2'), URIRef('third')) + O.rest( URIRef('2'), URIRef(namespaces['rdf']+'#nil')) + O.oneOf(datarange, own) + O.type(datarange, namespaces['owl']+'#DataRange') + restrict = BNode('anon1') + obj = URIRef(namespaces['owl']+'#Restriction') + O.type(restrict, obj) + p = URIRef('p') + O.onProperty(restrict,p) + obj = URIRef(namespaces['owl']+'#ObjectProperty') + O.type(p, obj) + O.someValuesFrom(restrict, datarange) + cls = URIRef('class') + obj = URIRef(namespaces['owl']+'#Class') + O.type(cls, obj) + O.subClassOf(cls,restrict) + O.variables['p_'].setValues([(cls,1)]) + py.test.raises(ConsistencyFailure, O.consistency) From ale at codespeak.net Sun Jan 8 16:04:19 2006 From: ale at codespeak.net (ale at codespeak.net) Date: Sun, 8 Jan 2006 16:04:19 +0100 (CET) Subject: [pypy-svn] r21818 - in pypy/dist/pypy/lib/pyontology: . test Message-ID: <20060108150419.639D527B66@code1.codespeak.net> Author: ale Date: Sun Jan 8 16:04:18 2006 New Revision: 21818 Modified: pypy/dist/pypy/lib/pyontology/pyontology.py pypy/dist/pypy/lib/pyontology/test/test_ontology.py Log: Added support for hasValue and allValuesFrom, with tests Modified: pypy/dist/pypy/lib/pyontology/pyontology.py ============================================================================== --- pypy/dist/pypy/lib/pyontology/pyontology.py (original) +++ pypy/dist/pypy/lib/pyontology/pyontology.py Sun Jan 8 16:04:18 2006 @@ -468,18 +468,16 @@ self.constraints.append(constrain) def hasValue(self, s, var): - if type(var) != URIRef: - value = var - else: - value = self.make_var(None, var) - prop = self.find_prop(s) - cls = self.make_var(None, self.find_cls(s)) - con = HasvalueConstraint(prop, cls, value) - self.constraints.append(con) + svar = self.make_var(Restriction, s) + avar = self.make_var(None, var) + constrain = HasvalueConstraint(svar, avar) + self.constraints.append(constrain) def allValuesFrom(self, s, var): - # TODO: implement this - pass + svar = self.make_var(Restriction, s) + avar = self.make_var(None, var) + constrain = AllValueConstraint(svar, avar) + self.constraints.append(constrain) def someValuesFrom(self, s, var): svar = self.make_var(Restriction, s) @@ -815,7 +813,6 @@ return 100 def narrow(self, domains): - print self.variable,self.List val = domains[self.List].getValues() property = domains[self.variable].property cls = domains[self.variable].cls @@ -828,5 +825,46 @@ "The value of the property %s in the class %s has no values from %r" %(property, cls, val)) -class HasvalueConstraint: - pass +class AllValueConstraint(AbstractConstraint): + + def __init__(self, variable, List): + AbstractConstraint.__init__(self, [variable, List]) + self.variable = variable + self.List = List + + def estimateCost(self, domains): + return 100 + + def narrow(self, domains): + val = domains[self.List].getValues() + property = domains[self.variable].property + cls = domains[self.variable].cls + prop = Linkeddict(domains[property].getValues()) + for v in prop[cls]: + if not v in val: + raise ConsistencyFailure( + "The value of the property %s in the class %s has a value not from %r" + %(property, cls, val)) + +class HasvalueConstraint(AbstractConstraint): + def __init__(self, variable, List): + AbstractConstraint.__init__(self, [variable]) + self.variable = variable + self.List = List + + def estimateCost(self, domains): + return 100 + + def narrow(self, domains): + val = self.List #domains[self.List].getValues() + property = domains[self.variable].property + cls = domains[self.variable].cls + prop = Linkeddict(domains[property].getValues()) + for v in prop[cls]: + if v == val: + break + else: + raise ConsistencyFailure( + "The value of the property %s in the class %s has a value not from %r" + %(property, cls, val)) + Modified: pypy/dist/pypy/lib/pyontology/test/test_ontology.py ============================================================================== --- pypy/dist/pypy/lib/pyontology/test/test_ontology.py (original) +++ pypy/dist/pypy/lib/pyontology/test/test_ontology.py Sun Jan 8 16:04:18 2006 @@ -307,16 +307,22 @@ O.variables['ownedby_'].setValues([('Fiat_','Bob_')]) O.consistency() -def no_test_hasvalue(): +def test_hasvalue(): O = Ontology() - own = URIRef('owner') + restrict = BNode('anon1') + obj = URIRef(namespaces['owl']+'#Restriction') + O.type(restrict, obj) + p = URIRef('p') + O.onProperty(restrict,p) obj = URIRef(namespaces['owl']+'#ObjectProperty') - O.type(own, obj) - O.hasValue(own, 42) + O.type(p, obj) + O.hasValue(restrict, 2) + cls = URIRef('class') + obj = URIRef(namespaces['owl']+'#Class') + O.type(cls, obj) + O.subClassOf(cls,restrict) + O.variables['p_'].setValues([(cls,1)]) py.test.raises(ConsistencyFailure, O.consistency) - #Make class - O.variables['owner_'].setValues([('Fiat_', 42)]) - O.consistency() def test_List(): O = Ontology() @@ -421,3 +427,32 @@ O.subClassOf(cls,restrict) O.variables['p_'].setValues([(cls,1)]) py.test.raises(ConsistencyFailure, O.consistency) + +def test_allvaluesfrom_datarange(): + O = Ontology() + datarange = BNode('anon') + own = URIRef('favlist') + obj = URIRef(namespaces['rdf']+'#List') + O.type(own, obj) + O.first(own, URIRef('first')) + O.rest(own, URIRef('1')) + O.first( URIRef('1'), URIRef('second')) + O.rest( URIRef('1'), URIRef('2')) + O.first( URIRef('2'), URIRef('third')) + O.rest( URIRef('2'), URIRef(namespaces['rdf']+'#nil')) + O.oneOf(datarange, own) + O.type(datarange, namespaces['owl']+'#DataRange') + restrict = BNode('anon1') + obj = URIRef(namespaces['owl']+'#Restriction') + O.type(restrict, obj) + p = URIRef('p') + O.onProperty(restrict,p) + obj = URIRef(namespaces['owl']+'#ObjectProperty') + O.type(p, obj) + O.allValuesFrom(restrict, datarange) + cls = URIRef('class') + obj = URIRef(namespaces['owl']+'#Class') + O.type(cls, obj) + O.subClassOf(cls,restrict) + O.variables['p_'].setValues([(cls,1)]) + py.test.raises(ConsistencyFailure, O.consistency) From arigo at codespeak.net Sun Jan 8 16:21:47 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sun, 8 Jan 2006 16:21:47 +0100 (CET) Subject: [pypy-svn] r21819 - pypy/extradoc/sprintinfo/mallorca Message-ID: <20060108152147.D9B8227B5E@code1.codespeak.net> Author: arigo Date: Sun Jan 8 16:21:47 2006 New Revision: 21819 Modified: pypy/extradoc/sprintinfo/mallorca/people.txt Log: my sprint dates (end date not fixed yet) Modified: pypy/extradoc/sprintinfo/mallorca/people.txt ============================================================================== --- pypy/extradoc/sprintinfo/mallorca/people.txt (original) +++ pypy/extradoc/sprintinfo/mallorca/people.txt Sun Jan 8 16:21:47 2006 @@ -10,7 +10,7 @@ Name Arrive/Depart Accomodation ==================== ============== ===================== Beatrice Duering 22/1 -29/1 flat? -Armin Rigo ? ? +Armin Rigo 21/1 - 30?/1 ? Samuele Pedroni ? ? Eric van Riet Paap 22/1 - 28/1 casa M. Michael Hudson ? ? From pedronis at codespeak.net Sun Jan 8 23:22:20 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Sun, 8 Jan 2006 23:22:20 +0100 (CET) Subject: [pypy-svn] r21825 - in pypy/dist/pypy/translator: . goal test Message-ID: <20060108222220.587C627B5A@code1.codespeak.net> Author: pedronis Date: Sun Jan 8 23:22:18 2006 New Revision: 21825 Modified: pypy/dist/pypy/translator/goal/driver.py pypy/dist/pypy/translator/interactive.py pypy/dist/pypy/translator/test/test_interactive.py Log: a bit more progress Modified: pypy/dist/pypy/translator/goal/driver.py ============================================================================== --- pypy/dist/pypy/translator/goal/driver.py (original) +++ pypy/dist/pypy/translator/goal/driver.py Sun Jan 8 23:22:18 2006 @@ -47,7 +47,7 @@ self.log = log if options is None: - options = DEFAULT_OPTIONS + options = DEFAULT_OPTIONS self.options = options self.done = {} @@ -341,7 +341,7 @@ if args is None: args = [] if options is None: - options = DEFAULT_OPTIONS.copy() + options = DEFAULT_OPTIONS driver = TranslationDriver(options, default_goal, disable) Modified: pypy/dist/pypy/translator/interactive.py ============================================================================== --- pypy/dist/pypy/translator/interactive.py (original) +++ pypy/dist/pypy/translator/interactive.py Sun Jan 8 23:22:18 2006 @@ -3,7 +3,7 @@ from pypy.translator.translator import TranslationContext from pypy.translator.goal import driver -DEFAULT_OPTIONS = optparse.Values(defaults={ +DEFAULT_OPTIONS = { 'gc': 'ref', 'thread': False, # influences GC policy @@ -15,8 +15,10 @@ 'backend': None, 'lowmem': False, - 'fork_before': None -}) + 'fork_before': None, + + 'merge_if_blocks': True +} class Translation(object): @@ -27,8 +29,9 @@ graph = self.context.buildflowgraph(entry_point) self.context._prebuilt_graphs[entry_point] = graph - self.driver = driver.TranslationDriver(DEFAULT_OPTIONS) - + self.driver = driver.TranslationDriver( + optparse.Values(defaults=DEFAULT_OPTIONS)) + # hook into driver events driver_own_event = self.driver._event def _event(kind, goal, func): @@ -44,6 +47,10 @@ GOAL_USES_OPTS = { 'annotate': ['debug'], 'rtype': ['insist'], + 'backendopt': ['merge_if_blocks'], + 'database_c': ['gc', 'stackless'], + 'source_llvm': ['gc', 'stackless'], + 'source_c': [], } def driver_event(self, kind, goal, func): @@ -72,7 +79,7 @@ def update_options(self, argtypes, kwds): if argtypes or kwds.get('policy'): self.ensure_setup(argtypes, kwds.get('policy')) - for optname, value in kwds: + for optname, value in kwds.iteritems(): if optname in self.frozen_options: if getattr(self.driver.options, optname) != value: raise Exception("incosistent option supplied: %s" % optname) @@ -80,6 +87,13 @@ setattr(self.driver.options, optname, value) self.frozen_options[optname] = True + def ensure_backend(self, backend=None): + if backend is not None: + self.update_options(None, {'backend': backend}) + if self.driver.options.backend is None: + raise Exception("a backend should have been specified at this point") + return self.driver.options.backend + # backend independent def annotate(self, argtypes=None, **kwds): @@ -90,15 +104,32 @@ self.update_options(argtypes, kwds) return self.driver.rtype() - # backend depedent xxx + # backend depedent + # xxx finish + # xxx how to skip a goal? + + def backendopt(self, argtypes=None, **kwds): + self.update_options(argtypes, kwds) + self.ensure_backend() + self.driver.backendopt() - def source(self, argtypes, **kwds): + def backendopt_c(self, argtypes=None, **kwds): + self.update_options(argtypes, kwds) + self.ensure_backend('c') + self.driver.backendopt() + + def source(self, argtypes=None, **kwds): backend = self.ensure_backend() self.update_options(argtypes, kwds) getattr(self.driver, 'source_'+backend)() - def source_c(self, argtypes, **kwds): + def source_c(self, argtypes=None, **kwds): self.ensure_backend('c') self.update_options(argtypes, kwds) self.driver.source_c() + + def source_llvm(self, argtypes=None, **kwds): + self.ensure_backend('llvm') + self.update_options(argtypes, kwds) + self.driver.source_c() Modified: pypy/dist/pypy/translator/test/test_interactive.py ============================================================================== --- pypy/dist/pypy/translator/test/test_interactive.py (original) +++ pypy/dist/pypy/translator/test/test_interactive.py Sun Jan 8 23:22:18 2006 @@ -38,3 +38,36 @@ t = Translation(f, [int, int]) t.annotate() py.test.raises(Exception, "t.rtype([int, int],debug=False)") + +def test_simple_backendopt(): + def f(x, y): + return x,y + + t = Translation(f, [int, int], backend='c') + t.backendopt() + + t = Translation(f, [int, int]) + t.backendopt_c() + + t = Translation(f, [int, int]) + py.test.raises(Exception, "t.backendopt()") + +def test_simple_source(): + def f(x, y): + return x,y + + t = Translation(f, backend='c') + t.annotate([int, int]) + t.source() + + t = Translation(f, [int, int]) + t.source_c() + + # this doesn't work, there is to be a bug in llvm backend code + #t = Translation(f, [int, int], backend='llvm') + #t.source() + #t = Translation(f, [int, int]) + #t.source_llvm() + + t = Translation(f, [int, int]) + py.test.raises(Exception, "t.source()") From cfbolz at codespeak.net Mon Jan 9 17:23:13 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Mon, 9 Jan 2006 17:23:13 +0100 (CET) Subject: [pypy-svn] r21840 - in pypy/dist/pypy/translator/c: . test Message-ID: <20060109162313.E400627B45@code1.codespeak.net> Author: cfbolz Date: Mon Jan 9 17:23:12 2006 New Revision: 21840 Modified: pypy/dist/pypy/translator/c/funcgen.py pypy/dist/pypy/translator/c/test/test_lladdresses.py Log: implement flavored_malloc and flavored_free in genc, so far only with the "raw" flavor. Modified: pypy/dist/pypy/translator/c/funcgen.py ============================================================================== --- pypy/dist/pypy/translator/c/funcgen.py (original) +++ pypy/dist/pypy/translator/c/funcgen.py Mon Jan 9 17:23:12 2006 @@ -434,7 +434,9 @@ line = '%s\n%s' % (line, self.check_directcall_result(op, err)) return line - OP_INDIRECT_CALL = OP_DIRECT_CALL # XXX for now + # the following works since the extra arguments that indirect_call has + # is of type Void, which is removed by OP_DIRECT_CALL + OP_INDIRECT_CALL = OP_DIRECT_CALL def check_directcall_result(self, op, err): return 'if (RPyExceptionOccurred())\n\tFAIL(%s);' % err @@ -558,6 +560,27 @@ result += '\n%s->%s = %s;' % (eresult, lenfld, elength) return result + def OP_FLAVORED_MALLOC(self, op, err): + print op + TYPE = self.lltypemap(op.result).TO + typename = self.db.gettype(TYPE) + eresult = self.expr(op.result) + esize = 'sizeof(%s)' % cdecl(typename, '') + flavor = op.args[0].value + if flavor == "raw": + return "OP_RAW_MALLOC(%s, %s, %s);" % (esize, eresult, err) + else: + raise NotImplementedError + + def OP_FLAVORED_FREE(self, op, err): + flavor = op.args[0].value + if flavor == "raw": + return "OP_RAW_FREE(%s, %s, %s)" % (self.expr(op.args[1]), + self.expr(op.result), + err) + else: + raise NotImplementedError + def OP_CAST_POINTER(self, op, err): TYPE = self.lltypemap(op.result) typename = self.db.gettype(TYPE) Modified: pypy/dist/pypy/translator/c/test/test_lladdresses.py ============================================================================== --- pypy/dist/pypy/translator/c/test/test_lladdresses.py (original) +++ pypy/dist/pypy/translator/c/test/test_lladdresses.py Mon Jan 9 17:23:12 2006 @@ -1,6 +1,7 @@ from pypy.rpython.memory.lladdress import * from pypy.annotation.model import SomeAddress, SomeChar from pypy.translator.c.test.test_genc import compile +from pypy.rpython.objectmodel import free_non_gc_object def test_null(): def f(): @@ -86,3 +87,16 @@ fc = compile(f, []) res = fc() assert res == int('011100' * 2, 2) + +def test_flavored_malloc(): + class A(object): + _alloc_flavor_ = "raw" + def __init__(self, val): + self.val = val + def f(x): + a = A(x + 1) + result = a.val + free_non_gc_object(a) + return result + fn = compile(f, [int]) + assert fn(1) == 2 From cfbolz at codespeak.net Mon Jan 9 17:42:48 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Mon, 9 Jan 2006 17:42:48 +0100 (CET) Subject: [pypy-svn] r21843 - pypy/dist/pypy/translator/c Message-ID: <20060109164248.EB66927B55@code1.codespeak.net> Author: cfbolz Date: Mon Jan 9 17:42:47 2006 New Revision: 21843 Modified: pypy/dist/pypy/translator/c/funcgen.py Log: remove debug print Modified: pypy/dist/pypy/translator/c/funcgen.py ============================================================================== --- pypy/dist/pypy/translator/c/funcgen.py (original) +++ pypy/dist/pypy/translator/c/funcgen.py Mon Jan 9 17:42:47 2006 @@ -561,7 +561,6 @@ return result def OP_FLAVORED_MALLOC(self, op, err): - print op TYPE = self.lltypemap(op.result).TO typename = self.db.gettype(TYPE) eresult = self.expr(op.result) From ale at codespeak.net Mon Jan 9 18:33:12 2006 From: ale at codespeak.net (ale at codespeak.net) Date: Mon, 9 Jan 2006 18:33:12 +0100 (CET) Subject: [pypy-svn] r21844 - pypy/dist/pypy/lib/pyontology Message-ID: <20060109173312.2717227B55@code1.codespeak.net> Author: ale Date: Mon Jan 9 18:33:11 2006 New Revision: 21844 Modified: pypy/dist/pypy/lib/pyontology/pyontology.py Log: Make the recognition of builin classes more robust. Modified: pypy/dist/pypy/lib/pyontology/pyontology.py ============================================================================== --- pypy/dist/pypy/lib/pyontology/pyontology.py (original) +++ pypy/dist/pypy/lib/pyontology/pyontology.py Mon Jan 9 18:33:11 2006 @@ -20,6 +20,9 @@ rdf_rest = URIRef(u'http://www.w3.org/1999/02/22-rdf-syntax-ns#rest') rdf_first = URIRef(u'http://www.w3.org/1999/02/22-rdf-syntax-ns#first') +def getUriref(ns, obj): + return URIRef(namespaces[ns]+'#'+obj) + class ClassDomain(AbstractDomain): # Class domain is intended as a (abstract/virtual) domain for implementing @@ -166,25 +169,24 @@ self.cls = None builtin_voc = { - 'Thing' : Thing, - 'Class' : ClassDomain, - 'ObjectProperty' : ObjectProperty, - 'AllDifferent' : AllDifferent , -## 'AnnotationProperty' : AnnotationProperty, - 'DataRange' : DataRange, - 'DatatypeProperty' : DatatypeProperty, -## 'DeprecatedClass' : DeprecatedClass, -## 'DeprecatedProperty' : DeprecatedProperty, - 'FunctionalProperty' : FunctionalProperty, - 'InverseFunctionalProperty' : InverseFunctionalProperty, -## 'Nothing' : Nothing, -## 'ObjectProperty' : ObjectProperty, -## 'Ontology' : Ontology, -## 'OntologyProperty' : OntologyProperty, - 'Restriction' : Restriction, - 'SymmetricProperty' : SymmetricProperty, - 'TransitiveProperty' : TransitiveProperty, - 'List' : List + getUriref('owl', 'Thing') : Thing, + getUriref('owl', 'Class') : ClassDomain, + getUriref('owl', 'ObjectProperty') : ObjectProperty, + getUriref('owl', 'AllDifferent') : AllDifferent , +## getUriref('owl', 'AnnotationProperty') : AnnotationProperty, + getUriref('owl', 'DataRange') : DataRange, + getUriref('owl', 'DatatypeProperty') : DatatypeProperty, +## getUriref('owl', 'DeprecatedClass') : DeprecatedClass, +## getUriref('owl', 'DeprecatedProperty') : DeprecatedProperty, + getUriref('owl', 'FunctionalProperty') : FunctionalProperty, + getUriref('owl', 'InverseFunctionalProperty') : InverseFunctionalProperty, + getUriref('owl', 'Nothing') : Nothing, +## getUriref('owl', 'Ontology') : Ontology, +## getUriref('owl', 'OntologyProperty') : OntologyProperty, + getUriref('owl', 'Restriction') : Restriction, + getUriref('owl', 'SymmetricProperty') : SymmetricProperty, + getUriref('owl', 'TransitiveProperty') : TransitiveProperty, + getUriref('rdf', 'List') : List } class Ontology(Graph): @@ -210,7 +212,7 @@ ns ='' func = p if ns in namespaces.values(): - #predicate is one of builtin OWL predicates + #predicate is one of builtin OWL or rdf predicates pred = getattr(self, func) res = pred(s, o) if not res : @@ -267,44 +269,12 @@ return var if not var in self.variables.keys(): self.variables[var] = cls(var) -## elif type(self.variables[var]) != cls: -## olddom = self.variables[var] -## newdom = cls(var) -## newdom.setValues(olddom.getValues()) -## self.variables[var] = newdom return var - def find_prop(self, s): - p = URIRef(u'http://www.w3.org/2002/07/owl#onProperty') - pr = list(self.objects(s,p)) - assert len(pr) == 1 - return pr[0] - - def find_cls(self, s): - p = URIRef(u'http://www.w3.org/1999/02/22-rdf-syntax-ns#type') - r = URIRef(u'http://www.w3.org/2000/01/rdf-schema#subClassOf') - if type(s) == BNode: - pr = list( self.subjects(p,s) ) - if len(pr) == 0: - pr = list( self.subjects(r,s) ) - return pr[0] - else: - return s - - def find_uriref(self, s): - while type(s) == BNode: - s = list(self.subjects(None,s))[0] - return s - - def find_property(self, s): - prop = self.find_prop(s) - avar = self.make_var(Property, prop) - return avar - #---------------- Implementation ---------------- def type(self, s, var): - if not (var in [URIRef(namespaces[ns]+'#'+x) for x in builtin_voc for ns in namespaces]): + if not var in builtin_voc : # var is not one of the builtin classes avar = self.make_var(ClassDomain, var) svar = self.make_var(self.variables[avar].__class__, s) @@ -313,8 +283,8 @@ else: # var is a builtin class svar = self.make_var(None, s) - cls =builtin_voc[var.split('#')[-1]] - if not (self.variables.has_key(svar) and isinstance(self.variables, cls)): + cls =builtin_voc[var] + if not (self.variables.has_key(svar) and isinstance(self.variables[svar], cls)): svar = self.make_var(cls, s) cls = self.variables[svar] if hasattr(cls, 'constraint'): @@ -430,7 +400,6 @@ def maxCardinality(self, s, var): """ Len of finite domain of the property shall be less than or equal to var""" svar =self.make_var(Restriction, s) - #avar =self.make_var(None, var) constrain = MaxCardinality(svar, None, int(var)) self.constraints.append(constrain) @@ -509,20 +478,25 @@ """Contraint: all values must be distinct""" def __init__(self, variable, cls, cardinality): - AbstractConstraint.__init__(self, [variable, cls]) + AbstractConstraint.__init__(self, [variable]) self.__cost = 2 self.variable = variable self.cardinality = cardinality - self.cls = cls + #self.cls = cls def __repr__(self): return '<%s %s %i>' % (self.__class__.__name__, str(self._variables[0]), self.cardinality) + def estimateCost(self, domains): + return 80 + def narrow(self, domains): """narrowing algorithm for the constraint""" - prop = Linkeddict(domains[self.variable].getValues()) - if len(prop[self.cls]) > self.cardinality: - raise ConsistencyFailure("Maxcardinality of %i exceeded by the value %i" %(self.cardinality,len(prop[self.cls]))) + prop = domains[self.variable].property + props = Linkeddict(domains[prop].getValues()) + cls = domains[self.variable].cls + if len(props[cls]) > self.cardinality: + raise ConsistencyFailure("Maxcardinality of %i exceeded by the value %i" %(self.cardinality,len(props[cls]))) else: return 1 @@ -530,9 +504,11 @@ def narrow(self, domains): """narrowing algorithm for the constraint""" - prop = Linkeddict(domains[self.variable].getValues()) - if len(prop[self.cls]) < self.cardinality: - raise ConsistencyFailure("MinCardinalityof %i exceeded by the value %i" %(self.cardinality,len(prop[self.cls]))) + prop = domains[self.variable].property + props = Linkeddict(domains[prop].getValues()) + cls = domains[self.variable].cls + if len(props[cls]) < self.cardinality: + raise ConsistencyFailure("MinCardinality of %i not achieved by the value %i" %(self.cardinality,len(props[cls]))) else: return 1 @@ -540,9 +516,11 @@ def narrow(self, domains): """narrowing algorithm for the constraint""" - prop = Linkeddict(domains[self.variable].getValues()) - if len(prop[self.cls]) != self.cardinality: - raise ConsistencyFailure("Cardinality of %i exceeded by the value %i" %(self.cardinality,len(prop[self.cls]))) + prop = domains[self.variable].property + props = Linkeddict(domains[prop].getValues()) + cls = domains[self.variable].cls + if len(props[cls]) != self.cardinality: + raise ConsistencyFailure("Cardinality of %i exceeded by the value %i" %(self.cardinality,len(props[cls]))) else: return 1 @@ -606,7 +584,7 @@ def narrow(self, domains): propdom = domains[self.variable] rangedom = domains[self.object] - newrange = get_values(rangedom, domains, 'getValues') + newrange = rangedom.getValues() range = [] oldrange = propdom.range if oldrange: From pedronis at codespeak.net Mon Jan 9 20:55:04 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Mon, 9 Jan 2006 20:55:04 +0100 (CET) Subject: [pypy-svn] r21845 - in pypy/dist/pypy/translator: . test Message-ID: <20060109195504.22BD627B58@code1.codespeak.net> Author: pedronis Date: Mon Jan 9 20:55:02 2006 New Revision: 21845 Modified: pypy/dist/pypy/translator/interactive.py pypy/dist/pypy/translator/test/test_interactive.py Log: enforce gc='boehm' for LLVM backend Modified: pypy/dist/pypy/translator/interactive.py ============================================================================== --- pypy/dist/pypy/translator/interactive.py (original) +++ pypy/dist/pypy/translator/interactive.py Mon Jan 9 20:55:02 2006 @@ -92,7 +92,10 @@ self.update_options(None, {'backend': backend}) if self.driver.options.backend is None: raise Exception("a backend should have been specified at this point") - return self.driver.options.backend + backend = self.driver.options.backend + if backend == 'llvm': + self.update_options(None, {'gc': 'boehm'}) + return backend # backend independent Modified: pypy/dist/pypy/translator/test/test_interactive.py ============================================================================== --- pypy/dist/pypy/translator/test/test_interactive.py (original) +++ pypy/dist/pypy/translator/test/test_interactive.py Mon Jan 9 20:55:02 2006 @@ -63,11 +63,20 @@ t = Translation(f, [int, int]) t.source_c() - # this doesn't work, there is to be a bug in llvm backend code - #t = Translation(f, [int, int], backend='llvm') - #t.source() - #t = Translation(f, [int, int]) - #t.source_llvm() - t = Translation(f, [int, int]) py.test.raises(Exception, "t.source()") + +def test_simple_source_llvm(): + from pypy.translator.llvm.test.runtest import llvm_test + llvm_test() + + def f(x,y): + return x+y + + + t = Translation(f, [int, int], backend='llvm') + t.source(gc='boehm') + + t = Translation(f, [int, int]) + t.source_llvm() + From pedronis at codespeak.net Mon Jan 9 21:39:37 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Mon, 9 Jan 2006 21:39:37 +0100 (CET) Subject: [pypy-svn] r21846 - in pypy/dist/pypy/translator: . goal test Message-ID: <20060109203937.F1BBB27B45@code1.codespeak.net> Author: pedronis Date: Mon Jan 9 21:39:35 2006 New Revision: 21846 Modified: pypy/dist/pypy/translator/goal/driver.py pypy/dist/pypy/translator/interactive.py pypy/dist/pypy/translator/test/test_interactive.py Log: - added Translation.disable([goal...]) to be able to specify goals/steps to skip (with tests) - some fixes - interactive.py completed with compile goals Modified: pypy/dist/pypy/translator/goal/driver.py ============================================================================== --- pypy/dist/pypy/translator/goal/driver.py (original) +++ pypy/dist/pypy/translator/goal/driver.py Mon Jan 9 21:39:35 2006 @@ -52,15 +52,11 @@ self.done = {} - maybe_skip = [] - if disable: - for goal in self.backend_select_goals(disable): - maybe_skip.extend(self._depending_on_closure(goal)) - self.maybe_skip = dict.fromkeys(maybe_skip).keys() + self.disable(disable) if default_goal: default_goal, = self.backend_select_goals([default_goal]) - if default_goal in self.maybe_skip: + if default_goal in self._maybe_skip(): default_goal = None self.default_goal = default_goal @@ -91,6 +87,17 @@ l.append(goal) return l + def disable(self, to_disable): + self._disabled = to_disable + + def _maybe_skip(self): + maybe_skip = [] + if self._disabled: + for goal in self.backend_select_goals(self._disabled): + maybe_skip.extend(self._depending_on_closure(goal)) + return dict.fromkeys(maybe_skip).keys() + + def setup(self, entry_point, inputtypes, policy=None, extra={}, empty_translator=None): standalone = inputtypes is None self.standalone = standalone @@ -333,7 +340,7 @@ elif isinstance(goals, str): goals = [goals] goals = self.backend_select_goals(goals) - return self._execute(goals, task_skip = self.maybe_skip) + return self._execute(goals, task_skip = self._maybe_skip()) def from_targetspec(targetspec_dic, options=None, args=None, empty_translator=None, disable=[], Modified: pypy/dist/pypy/translator/interactive.py ============================================================================== --- pypy/dist/pypy/translator/interactive.py (original) +++ pypy/dist/pypy/translator/interactive.py Mon Jan 9 21:39:35 2006 @@ -51,11 +51,13 @@ 'database_c': ['gc', 'stackless'], 'source_llvm': ['gc', 'stackless'], 'source_c': [], + 'compile_c': [], + 'compile_llvm': [], } def driver_event(self, kind, goal, func): if kind == 'pre': - print goal + #print goal self.ensure_setup() elif kind == 'post': used_opts = dict.fromkeys(self.GOAL_USES_OPTS[goal], True) @@ -97,6 +99,10 @@ self.update_options(None, {'gc': 'boehm'}) return backend + # disable some goals (steps) + def disable(self, to_disable): + self.driver.disable(to_disable) + # backend independent def annotate(self, argtypes=None, **kwds): @@ -108,8 +114,6 @@ return self.driver.rtype() # backend depedent - # xxx finish - # xxx how to skip a goal? def backendopt(self, argtypes=None, **kwds): self.update_options(argtypes, kwds) @@ -122,17 +126,35 @@ self.driver.backendopt() def source(self, argtypes=None, **kwds): - backend = self.ensure_backend() self.update_options(argtypes, kwds) + backend = self.ensure_backend() getattr(self.driver, 'source_'+backend)() def source_c(self, argtypes=None, **kwds): - self.ensure_backend('c') self.update_options(argtypes, kwds) + self.ensure_backend('c') self.driver.source_c() def source_llvm(self, argtypes=None, **kwds): + self.update_options(argtypes, kwds) self.ensure_backend('llvm') + self.driver.source_llvm() + + def compile(self, argtypes=None, **kwds): self.update_options(argtypes, kwds) - self.driver.source_c() - + backend = self.ensure_backend() + getattr(self.driver, 'compile_'+backend)() + return self.driver.c_entryp + + def compile_c(self, argtypes=None, **kwds): + self.update_options(argtypes, kwds) + self.ensure_backend('c') + self.driver.compile_c() + return self.driver.c_entryp + + def compile_llvm(self, argtypes=None, **kwds): + self.update_options(argtypes, kwds) + self.ensure_backend('llvm') + self.driver.compile_llvm() + return self.driver.c_entryp + Modified: pypy/dist/pypy/translator/test/test_interactive.py ============================================================================== --- pypy/dist/pypy/translator/test/test_interactive.py (original) +++ pypy/dist/pypy/translator/test/test_interactive.py Mon Jan 9 21:39:35 2006 @@ -59,9 +59,11 @@ t = Translation(f, backend='c') t.annotate([int, int]) t.source() + assert 'source_c' in t.driver.done t = Translation(f, [int, int]) t.source_c() + assert 'source_c' in t.driver.done t = Translation(f, [int, int]) py.test.raises(Exception, "t.source()") @@ -73,10 +75,51 @@ def f(x,y): return x+y - t = Translation(f, [int, int], backend='llvm') t.source(gc='boehm') + assert 'source_llvm' in t.driver.done t = Translation(f, [int, int]) t.source_llvm() + assert 'source_llvm' in t.driver.done +def test_disable_logic(): + + def f(x,y): + return x+y + + t = Translation(f, [int, int]) + t.disable(['backendopt']) + t.source_c() + + assert 'backendopt' not in t.driver.done + + t = Translation(f, [int, int]) + t.disable(['annotate']) + t.source_c() + + assert 'annotate' not in t.driver.done and 'rtype' not in t.driver.done + + t = Translation(f, [int, int]) + t.disable(['rtype']) + t.source_c() + + assert 'annotate' in t.driver.done + assert 'rtype' not in t.driver.done and 'backendopt' not in t.driver.done + +def test_simple_compile_c(): + def f(x,y): + return x+y + + t = Translation(f, [int, int]) + t.source(backend='c') + t_f = t.compile() + + res = t_f(2,3) + assert res == 5 + + t = Translation(f, [int, int]) + t_f = t.compile_c() + + res = t_f(2,3) + assert res == 5 From pedronis at codespeak.net Mon Jan 9 21:55:38 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Mon, 9 Jan 2006 21:55:38 +0100 (CET) Subject: [pypy-svn] r21847 - in pypy/dist/pypy/translator: . goal Message-ID: <20060109205538.BA8EB27B45@code1.codespeak.net> Author: pedronis Date: Mon Jan 9 21:55:36 2006 New Revision: 21847 Added: pypy/dist/pypy/translator/driver.py - copied unchanged from r21846, pypy/dist/pypy/translator/goal/driver.py Removed: pypy/dist/pypy/translator/goal/driver.py Modified: pypy/dist/pypy/translator/goal/translate_pypy.py pypy/dist/pypy/translator/interactive.py Log: move driver.py to translator now it used both by interactive.py and translate_pypy.py Modified: pypy/dist/pypy/translator/goal/translate_pypy.py ============================================================================== --- pypy/dist/pypy/translator/goal/translate_pypy.py (original) +++ pypy/dist/pypy/translator/goal/translate_pypy.py Mon Jan 9 21:55:36 2006 @@ -256,7 +256,7 @@ targetspec_dic, options, args = parse_options_and_load_target() from pypy.translator import translator - from pypy.translator.goal import driver + from pypy.translator import driver from pypy.translator.tool.pdbplus import PdbPlusShow t = translator.TranslationContext() Modified: pypy/dist/pypy/translator/interactive.py ============================================================================== --- pypy/dist/pypy/translator/interactive.py (original) +++ pypy/dist/pypy/translator/interactive.py Mon Jan 9 21:55:36 2006 @@ -1,7 +1,7 @@ import optparse from pypy.translator.translator import TranslationContext -from pypy.translator.goal import driver +from pypy.translator import driver DEFAULT_OPTIONS = { 'gc': 'ref', From cfbolz at codespeak.net Mon Jan 9 22:04:36 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Mon, 9 Jan 2006 22:04:36 +0100 (CET) Subject: [pypy-svn] r21848 - pypy/dist/pypy/translator/backendopt/test Message-ID: <20060109210436.847B527B46@code1.codespeak.net> Author: cfbolz Date: Mon Jan 9 22:04:35 2006 New Revision: 21848 Modified: pypy/dist/pypy/translator/backendopt/test/test_malloc.py Log: ouch. bad me. don't use the Translator class anymore (not even in tests) Modified: pypy/dist/pypy/translator/backendopt/test/test_malloc.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/test/test_malloc.py (original) +++ pypy/dist/pypy/translator/backendopt/test/test_malloc.py Mon Jan 9 22:04:35 2006 @@ -1,7 +1,8 @@ import py from pypy.translator.backendopt.malloc import remove_simple_mallocs from pypy.translator.backendopt.inline import inline_function -from pypy.translator.translator import TranslationContext, Translator, graphof +from pypy.translator.backendopt.all import backend_optimizations +from pypy.translator.translator import TranslationContext, graphof from pypy.objspace.flow.model import checkgraph, flatten, Block from pypy.rpython.llinterp import LLInterpreter @@ -132,11 +133,11 @@ os.write(1, str(delcalls[0]) + "\n") i += 1 return 1 - t = Translator(f) + t = TranslationContext() t.buildannotator().build_types(f, [int]) t.buildrtyper().specialize() graph = graphof(t, f) - t.backend_optimizations() + backend_optimizations(t) op = graph.startblock.exits[0].target.exits[1].target.operations[0] assert op.opname == "malloc" From pedronis at codespeak.net Mon Jan 9 22:06:33 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Mon, 9 Jan 2006 22:06:33 +0100 (CET) Subject: [pypy-svn] r21849 - pypy/dist/pypy/translator/c/test Message-ID: <20060109210633.0C17327B54@code1.codespeak.net> Author: pedronis Date: Mon Jan 9 22:06:32 2006 New Revision: 21849 Modified: pypy/dist/pypy/translator/c/test/test_ext__socket.py Log: translator.Translator should not be used anymore! is going away. Use TranslationContext instead Modified: pypy/dist/pypy/translator/c/test/test_ext__socket.py ============================================================================== --- pypy/dist/pypy/translator/c/test/test_ext__socket.py (original) +++ pypy/dist/pypy/translator/c/test/test_ext__socket.py Mon Jan 9 22:06:32 2006 @@ -2,7 +2,7 @@ import py import _socket, thread from pypy.translator.c.test.test_genc import compile -from pypy.translator.translator import Translator +from pypy.translator.translator import TranslationContext from pypy.module._socket.test import echoserver @@ -73,9 +73,9 @@ from pypy.module._socket.rpython import rsocket def does_stuff(): return rsocket.newsocket(_socket.AF_INET, _socket.SOCK_STREAM, 0) - t = Translator(does_stuff) - a = t.annotate([]) - assert a.gettype(t.graphs[0].getreturnvar()) == int + t = TranslationContext() + s = t.buildannotator().build_types(does_stuff, []) + assert s.knowntype == int def test_newsocket(): from pypy.module._socket.rpython import rsocket From cfbolz at codespeak.net Mon Jan 9 22:08:50 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Mon, 9 Jan 2006 22:08:50 +0100 (CET) Subject: [pypy-svn] r21850 - in pypy/dist/pypy/rpython: memory/test ootypesystem/test test Message-ID: <20060109210850.6AA5C27B55@code1.codespeak.net> Author: cfbolz Date: Mon Jan 9 22:08:47 2006 New Revision: 21850 Modified: pypy/dist/pypy/rpython/memory/test/test_llinterpsim.py pypy/dist/pypy/rpython/ootypesystem/test/test_ooclean.py pypy/dist/pypy/rpython/test/test_rlist.py pypy/dist/pypy/rpython/test/test_robject.py pypy/dist/pypy/rpython/test/test_rpbc.py pypy/dist/pypy/rpython/test/test_rrange.py pypy/dist/pypy/rpython/test/test_rstr.py pypy/dist/pypy/rpython/test/test_rtuple.py Log: remove unused Translator imports Modified: pypy/dist/pypy/rpython/memory/test/test_llinterpsim.py ============================================================================== --- pypy/dist/pypy/rpython/memory/test/test_llinterpsim.py (original) +++ pypy/dist/pypy/rpython/memory/test/test_llinterpsim.py Mon Jan 9 22:08:47 2006 @@ -3,7 +3,6 @@ from pypy.rpython.lltypesystem.lltype import typeOf, Ptr, PyObject from pypy.rpython.rtyper import RPythonTyper from pypy.rpython.llinterp import LLInterpreter, LLException,log -from pypy.translator.translator import Translator from pypy.rpython.rlist import * from pypy.rpython.rint import signed_repr from pypy.rpython import rstr Modified: pypy/dist/pypy/rpython/ootypesystem/test/test_ooclean.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/test/test_ooclean.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/test/test_ooclean.py Mon Jan 9 22:08:47 2006 @@ -1,4 +1,3 @@ -from pypy.translator.translator import Translator from pypy.rpython.lltypesystem import lltype from pypy.rpython.ootypesystem import ootype from pypy.rpython.test.test_llinterp import get_interpreter Modified: pypy/dist/pypy/rpython/test/test_rlist.py ============================================================================== --- pypy/dist/pypy/rpython/test/test_rlist.py (original) +++ pypy/dist/pypy/rpython/test/test_rlist.py Mon Jan 9 22:08:47 2006 @@ -1,5 +1,4 @@ import sys -from pypy.translator.translator import Translator from pypy.rpython.lltypesystem.lltype import * from pypy.rpython.rtyper import RPythonTyper from pypy.rpython.rlist import * Modified: pypy/dist/pypy/rpython/test/test_robject.py ============================================================================== --- pypy/dist/pypy/rpython/test/test_robject.py (original) +++ pypy/dist/pypy/rpython/test/test_robject.py Mon Jan 9 22:08:47 2006 @@ -1,4 +1,3 @@ -from pypy.translator.translator import Translator from pypy.rpython.lltypesystem.lltype import * from pypy.rpython.test.test_llinterp import interpret Modified: pypy/dist/pypy/rpython/test/test_rpbc.py ============================================================================== --- pypy/dist/pypy/rpython/test/test_rpbc.py (original) +++ pypy/dist/pypy/rpython/test/test_rpbc.py Mon Jan 9 22:08:47 2006 @@ -1,4 +1,3 @@ -from pypy.translator.translator import Translator from pypy.rpython.lltypesystem.lltype import * from pypy.rpython.rtyper import RPythonTyper from pypy.rpython.test.test_llinterp import interpret Modified: pypy/dist/pypy/rpython/test/test_rrange.py ============================================================================== --- pypy/dist/pypy/rpython/test/test_rrange.py (original) +++ pypy/dist/pypy/rpython/test/test_rrange.py Mon Jan 9 22:08:47 2006 @@ -1,4 +1,3 @@ -from pypy.translator.translator import Translator from pypy.rpython.rrange import * from pypy.rpython.test.test_llinterp import interpret from pypy.rpython.rarithmetic import intmask Modified: pypy/dist/pypy/rpython/test/test_rstr.py ============================================================================== --- pypy/dist/pypy/rpython/test/test_rstr.py (original) +++ pypy/dist/pypy/rpython/test/test_rstr.py Mon Jan 9 22:08:47 2006 @@ -1,5 +1,4 @@ import random -from pypy.translator.translator import Translator from pypy.rpython.lltypesystem.lltype import * from pypy.rpython.rstr import parse_fmt_string, ll_find, ll_rfind, STR from pypy.rpython.rtyper import RPythonTyper, TyperError Modified: pypy/dist/pypy/rpython/test/test_rtuple.py ============================================================================== --- pypy/dist/pypy/rpython/test/test_rtuple.py (original) +++ pypy/dist/pypy/rpython/test/test_rtuple.py Mon Jan 9 22:08:47 2006 @@ -1,4 +1,3 @@ -from pypy.translator.translator import Translator from pypy.rpython.lltypesystem.lltype import * from pypy.rpython.rtuple import * from pypy.rpython.rint import signed_repr From cfbolz at codespeak.net Mon Jan 9 22:11:33 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Mon, 9 Jan 2006 22:11:33 +0100 (CET) Subject: [pypy-svn] r21851 - pypy/dist/pypy/translator/llvm/test Message-ID: <20060109211133.330DB27B54@code1.codespeak.net> Author: cfbolz Date: Mon Jan 9 22:11:31 2006 New Revision: 21851 Modified: pypy/dist/pypy/translator/llvm/test/test_class.py pypy/dist/pypy/translator/llvm/test/test_genllvm1.py pypy/dist/pypy/translator/llvm/test/test_seq.py Log: more unused Translator imports Modified: pypy/dist/pypy/translator/llvm/test/test_class.py ============================================================================== --- pypy/dist/pypy/translator/llvm/test/test_class.py (original) +++ pypy/dist/pypy/translator/llvm/test/test_class.py Mon Jan 9 22:11:31 2006 @@ -1,7 +1,6 @@ from __future__ import division import py -from pypy.translator.translator import Translator from pypy.objspace.flow.model import Constant, Variable from pypy.translator.llvm.test.runtest import compile_function from pypy.translator.llvm.test import llvmsnippet Modified: pypy/dist/pypy/translator/llvm/test/test_genllvm1.py ============================================================================== --- pypy/dist/pypy/translator/llvm/test/test_genllvm1.py (original) +++ pypy/dist/pypy/translator/llvm/test/test_genllvm1.py Mon Jan 9 22:11:31 2006 @@ -3,7 +3,6 @@ import py -from pypy.translator.translator import Translator from pypy.objspace.flow.model import Constant, Variable from pypy.translator.llvm.test.runtest import compile_function from pypy.translator.llvm.test import llvmsnippet Modified: pypy/dist/pypy/translator/llvm/test/test_seq.py ============================================================================== --- pypy/dist/pypy/translator/llvm/test/test_seq.py (original) +++ pypy/dist/pypy/translator/llvm/test/test_seq.py Mon Jan 9 22:11:31 2006 @@ -1,7 +1,6 @@ from __future__ import division import py -from pypy.translator.translator import Translator from pypy.objspace.flow.model import Constant, Variable from pypy.translator.llvm.test.runtest import compile_function from pypy.translator.llvm.test import llvmsnippet From cfbolz at codespeak.net Mon Jan 9 22:17:42 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Mon, 9 Jan 2006 22:17:42 +0100 (CET) Subject: [pypy-svn] r21852 - pypy/dist/pypy/translator/llvm/demo Message-ID: <20060109211742.E952A27B54@code1.codespeak.net> Author: cfbolz Date: Mon Jan 9 22:17:41 2006 New Revision: 21852 Modified: pypy/dist/pypy/translator/llvm/demo/run.py Log: another unused one Modified: pypy/dist/pypy/translator/llvm/demo/run.py ============================================================================== --- pypy/dist/pypy/translator/llvm/demo/run.py (original) +++ pypy/dist/pypy/translator/llvm/demo/run.py Mon Jan 9 22:17:41 2006 @@ -6,7 +6,6 @@ from pypy.annotation.listdef import ListDef from pypy.translator.llvm.genllvm import genllvm_compile -from pypy.translator.translator import Translator def p(): print 'Running on top of CPython:' From cfbolz at codespeak.net Tue Jan 10 01:19:15 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Tue, 10 Jan 2006 01:19:15 +0100 (CET) Subject: [pypy-svn] r21854 - pypy/dist/pypy/translator Message-ID: <20060110001915.7473A27B46@code1.codespeak.net> Author: cfbolz Date: Tue Jan 10 01:19:14 2006 New Revision: 21854 Modified: pypy/dist/pypy/translator/simplify.py Log: huh? did this ever work? Modified: pypy/dist/pypy/translator/simplify.py ============================================================================== --- pypy/dist/pypy/translator/simplify.py (original) +++ pypy/dist/pypy/translator/simplify.py Tue Jan 10 01:19:14 2006 @@ -11,6 +11,7 @@ from pypy.objspace.flow.model import checkgraph, traverse, mkentrymap def get_graph(arg, translator): + from pypy.translator.translator import graphof if isinstance(arg, Variable): return None f = arg.value @@ -22,8 +23,7 @@ #external function calls don't have a real graph if getattr(callable, "suggested_primitive", False): return None - if callable in translator.flowgraphs: - return translator.flowgraphs[callable] + return graphof(translator, callable) except AttributeError, KeyError: pass try: From ericvrp at codespeak.net Tue Jan 10 09:12:33 2006 From: ericvrp at codespeak.net (ericvrp at codespeak.net) Date: Tue, 10 Jan 2006 09:12:33 +0100 (CET) Subject: [pypy-svn] r21863 - in pypy/dist/pypy/translator/llvm: . test Message-ID: <20060110081233.E65F727B4E@code1.codespeak.net> Author: ericvrp Date: Tue Jan 10 09:12:32 2006 New Revision: 21863 Modified: pypy/dist/pypy/translator/llvm/opwriter.py pypy/dist/pypy/translator/llvm/test/test_lladdresses.py Log: WIP: flavored_malloc support Modified: pypy/dist/pypy/translator/llvm/opwriter.py ============================================================================== --- pypy/dist/pypy/translator/llvm/opwriter.py (original) +++ pypy/dist/pypy/translator/llvm/opwriter.py Tue Jan 10 09:12:32 2006 @@ -254,7 +254,8 @@ def direct_call(self, opr): self.codewriter.call(opr.retref, opr.rettype, opr.argrefs[0], opr.argtypes[1:], opr.argrefs[1:]) - + # the following works since the extra arguments that indirect_call has + # is of type Void, which is removed by direct_call indirect_call = direct_call def malloc(self, opr): @@ -276,6 +277,29 @@ self.db.repr_constructor(arg_type), opr.argtypes[1:], opr.argrefs[1:]) + def flavored_malloc(self, opr): + #TYPE = self.lltypemap(opr.op.result).TO + #typename = self.db.gettype(TYPE) + #eresult = self.expr(opr.op.result) + #esize = 'sizeof(%s)' % cdecl(typename, '') + flavor = opr.op.args[0].value + if flavor == "raw": + self.codewriter.call(opr.retref, opr.rettype, "%raw_malloc", + opr.argtypes[1:], opr.argrefs[1:]) + #return "OP_RAW_MALLOC(%s, %s);" % (esize, eresult) + else: + raise NotImplementedError + + def flavored_free(self, opr): + flavor = opr.op.args[0].value + if flavor == "raw": + self.codewriter.call(opr.retref, opr.rettype, "%raw_free", + opr.argtypes[1:], opr.argrefs[1:]) + #return "OP_RAW_FREE(%s, %s)" % (self.expr(opr.op.args[1]), + # self.expr(opr.op.result)) + else: + raise NotImplementedError + def _getindexhelper(self, name, struct): assert name in list(struct._names) Modified: pypy/dist/pypy/translator/llvm/test/test_lladdresses.py ============================================================================== --- pypy/dist/pypy/translator/llvm/test/test_lladdresses.py (original) +++ pypy/dist/pypy/translator/llvm/test/test_lladdresses.py Tue Jan 10 09:12:32 2006 @@ -1,6 +1,7 @@ from pypy.rpython.memory.lladdress import * from pypy.annotation.model import SomeAddress, SomeChar from pypy.translator.llvm.test.runtest import compile_function as compile +from pypy.rpython.objectmodel import free_non_gc_object import py def test_null(): @@ -101,3 +102,17 @@ fc = compile(f, []) res = fc() assert res == int('011100' * 2, 2) + +def test_flavored_malloc(): + py.test.skip("not supported yet") + class A(object): + _alloc_flavor_ = "raw" + def __init__(self, val): + self.val = val + def f(x): + a = A(x + 1) + result = a.val + free_non_gc_object(a) + return result + fn = compile(f, [int]) + assert fn(1) == 2 From ericvrp at codespeak.net Tue Jan 10 10:19:16 2006 From: ericvrp at codespeak.net (ericvrp at codespeak.net) Date: Tue, 10 Jan 2006 10:19:16 +0100 (CET) Subject: [pypy-svn] r21864 - in pypy/dist/pypy/translator/llvm: . test Message-ID: <20060110091916.53E7B27B4E@code1.codespeak.net> Author: ericvrp Date: Tue Jan 10 10:19:14 2006 New Revision: 21864 Modified: pypy/dist/pypy/translator/llvm/codewriter.py pypy/dist/pypy/translator/llvm/opwriter.py pypy/dist/pypy/translator/llvm/test/test_lladdresses.py Log: support raw flavored_malloc Modified: pypy/dist/pypy/translator/llvm/codewriter.py ============================================================================== --- pypy/dist/pypy/translator/llvm/codewriter.py (original) +++ pypy/dist/pypy/translator/llvm/codewriter.py Tue Jan 10 10:19:14 2006 @@ -166,4 +166,9 @@ returntype, functionref, args)) - + + def malloc(self, targetvar, vartype): + self._indent("%s = malloc %s" % (targetvar, vartype)) + + def free(self, vartype, varref): + self._indent("free %s %s" % (vartype, varref)) Modified: pypy/dist/pypy/translator/llvm/opwriter.py ============================================================================== --- pypy/dist/pypy/translator/llvm/opwriter.py (original) +++ pypy/dist/pypy/translator/llvm/opwriter.py Tue Jan 10 10:19:14 2006 @@ -278,25 +278,17 @@ opr.argtypes[1:], opr.argrefs[1:]) def flavored_malloc(self, opr): - #TYPE = self.lltypemap(opr.op.result).TO - #typename = self.db.gettype(TYPE) - #eresult = self.expr(opr.op.result) - #esize = 'sizeof(%s)' % cdecl(typename, '') flavor = opr.op.args[0].value + type_ = opr.rettype[:-1] #XXX stripping of * if flavor == "raw": - self.codewriter.call(opr.retref, opr.rettype, "%raw_malloc", - opr.argtypes[1:], opr.argrefs[1:]) - #return "OP_RAW_MALLOC(%s, %s);" % (esize, eresult) + self.codewriter.malloc(opr.retref, type_) else: raise NotImplementedError def flavored_free(self, opr): flavor = opr.op.args[0].value if flavor == "raw": - self.codewriter.call(opr.retref, opr.rettype, "%raw_free", - opr.argtypes[1:], opr.argrefs[1:]) - #return "OP_RAW_FREE(%s, %s)" % (self.expr(opr.op.args[1]), - # self.expr(opr.op.result)) + self.codewriter.free(opr.argtypes[1], opr.argrefs[1]) else: raise NotImplementedError Modified: pypy/dist/pypy/translator/llvm/test/test_lladdresses.py ============================================================================== --- pypy/dist/pypy/translator/llvm/test/test_lladdresses.py (original) +++ pypy/dist/pypy/translator/llvm/test/test_lladdresses.py Tue Jan 10 10:19:14 2006 @@ -104,7 +104,6 @@ assert res == int('011100' * 2, 2) def test_flavored_malloc(): - py.test.skip("not supported yet") class A(object): _alloc_flavor_ = "raw" def __init__(self, val): From ericvrp at codespeak.net Tue Jan 10 12:01:01 2006 From: ericvrp at codespeak.net (ericvrp at codespeak.net) Date: Tue, 10 Jan 2006 12:01:01 +0100 (CET) Subject: [pypy-svn] r21866 - in pypy/dist/pypy/translator/llvm: . test Message-ID: <20060110110101.1CEDB27B54@code1.codespeak.net> Author: ericvrp Date: Tue Jan 10 12:00:59 2006 New Revision: 21866 Modified: pypy/dist/pypy/translator/llvm/codewriter.py pypy/dist/pypy/translator/llvm/opwriter.py pypy/dist/pypy/translator/llvm/test/test_lladdresses.py Log: Added genllvm support for allocating data on the stack with flavored_malloc. Explicit flavored_free operations on stack allocated data is skipped. Includes testcase. Modified: pypy/dist/pypy/translator/llvm/codewriter.py ============================================================================== --- pypy/dist/pypy/translator/llvm/codewriter.py (original) +++ pypy/dist/pypy/translator/llvm/codewriter.py Tue Jan 10 12:00:59 2006 @@ -167,6 +167,9 @@ functionref, args)) + def alloca(self, targetvar, vartype): + self._indent("%s = alloca %s" % (targetvar, vartype)) + def malloc(self, targetvar, vartype): self._indent("%s = malloc %s" % (targetvar, vartype)) Modified: pypy/dist/pypy/translator/llvm/opwriter.py ============================================================================== --- pypy/dist/pypy/translator/llvm/opwriter.py (original) +++ pypy/dist/pypy/translator/llvm/opwriter.py Tue Jan 10 12:00:59 2006 @@ -282,6 +282,8 @@ type_ = opr.rettype[:-1] #XXX stripping of * if flavor == "raw": self.codewriter.malloc(opr.retref, type_) + elif flavor == "alloca": + self.codewriter.alloca(opr.retref, type_) else: raise NotImplementedError @@ -289,6 +291,8 @@ flavor = opr.op.args[0].value if flavor == "raw": self.codewriter.free(opr.argtypes[1], opr.argrefs[1]) + elif flavor == "alloca": + self.codewriter.comment('***Skipping free of stack allocated data') else: raise NotImplementedError Modified: pypy/dist/pypy/translator/llvm/test/test_lladdresses.py ============================================================================== --- pypy/dist/pypy/translator/llvm/test/test_lladdresses.py (original) +++ pypy/dist/pypy/translator/llvm/test/test_lladdresses.py Tue Jan 10 12:00:59 2006 @@ -103,7 +103,7 @@ res = fc() assert res == int('011100' * 2, 2) -def test_flavored_malloc(): +def test_flavored_malloci_raw(): class A(object): _alloc_flavor_ = "raw" def __init__(self, val): @@ -115,3 +115,16 @@ return result fn = compile(f, [int]) assert fn(1) == 2 + +def test_flavored_malloc_alloca(): + class A(object): + _alloc_flavor_ = "alloca" + def __init__(self, val): + self.val = val + def f(x): + a = A(x + 1) + result = a.val + free_non_gc_object(a) + return result + fn = compile(f, [int]) + assert fn(1) == 2 From ericvrp at codespeak.net Tue Jan 10 12:05:53 2006 From: ericvrp at codespeak.net (ericvrp at codespeak.net) Date: Tue, 10 Jan 2006 12:05:53 +0100 (CET) Subject: [pypy-svn] r21867 - in pypy/dist/pypy/translator/llvm: . test Message-ID: <20060110110553.0F96427B55@code1.codespeak.net> Author: ericvrp Date: Tue Jan 10 12:05:53 2006 New Revision: 21867 Modified: pypy/dist/pypy/translator/llvm/opwriter.py pypy/dist/pypy/translator/llvm/test/test_lladdresses.py Log: malloc flavor "alloca" renamed to "stack" Modified: pypy/dist/pypy/translator/llvm/opwriter.py ============================================================================== --- pypy/dist/pypy/translator/llvm/opwriter.py (original) +++ pypy/dist/pypy/translator/llvm/opwriter.py Tue Jan 10 12:05:53 2006 @@ -282,7 +282,7 @@ type_ = opr.rettype[:-1] #XXX stripping of * if flavor == "raw": self.codewriter.malloc(opr.retref, type_) - elif flavor == "alloca": + elif flavor == "stack": self.codewriter.alloca(opr.retref, type_) else: raise NotImplementedError @@ -291,7 +291,7 @@ flavor = opr.op.args[0].value if flavor == "raw": self.codewriter.free(opr.argtypes[1], opr.argrefs[1]) - elif flavor == "alloca": + elif flavor == "stack": self.codewriter.comment('***Skipping free of stack allocated data') else: raise NotImplementedError Modified: pypy/dist/pypy/translator/llvm/test/test_lladdresses.py ============================================================================== --- pypy/dist/pypy/translator/llvm/test/test_lladdresses.py (original) +++ pypy/dist/pypy/translator/llvm/test/test_lladdresses.py Tue Jan 10 12:05:53 2006 @@ -118,7 +118,7 @@ def test_flavored_malloc_alloca(): class A(object): - _alloc_flavor_ = "alloca" + _alloc_flavor_ = "stack" def __init__(self, val): self.val = val def f(x): From cfbolz at codespeak.net Tue Jan 10 12:16:17 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Tue, 10 Jan 2006 12:16:17 +0100 (CET) Subject: [pypy-svn] r21868 - in pypy/dist/pypy/translator: . test Message-ID: <20060110111617.1EF9427B55@code1.codespeak.net> Author: cfbolz Date: Tue Jan 10 12:16:16 2006 New Revision: 21868 Modified: pypy/dist/pypy/translator/simplify.py pypy/dist/pypy/translator/test/test_simplify.py Log: another attempt to fix get_graph, this time with tests :-( Modified: pypy/dist/pypy/translator/simplify.py ============================================================================== --- pypy/dist/pypy/translator/simplify.py (original) +++ pypy/dist/pypy/translator/simplify.py Tue Jan 10 12:16:16 2006 @@ -20,16 +20,20 @@ return None try: callable = f._obj._callable - #external function calls don't have a real graph + # external function calls don't have a real graph if getattr(callable, "suggested_primitive", False): return None - return graphof(translator, callable) - except AttributeError, KeyError: - pass + except (AttributeError, KeyError, AssertionError): + return None try: return f._obj.graph except AttributeError: return None + try: + callable = f._obj._callable + return graphof(translator, callable) + except (AttributeError, KeyError, AssertionError): + return None # ____________________________________________________________ Modified: pypy/dist/pypy/translator/test/test_simplify.py ============================================================================== --- pypy/dist/pypy/translator/test/test_simplify.py (original) +++ pypy/dist/pypy/translator/test/test_simplify.py Tue Jan 10 12:16:16 2006 @@ -1,12 +1,14 @@ from pypy.translator.translator import TranslationContext, graphof from pypy.translator.backendopt.all import backend_optimizations +from pypy.translator.simplify import get_graph from pypy.objspace.flow.model import traverse, Block -def translate(func, argtypes): +def translate(func, argtypes, backend_optimize=True): t = TranslationContext() t.buildannotator().build_types(func, argtypes) t.buildrtyper().specialize() - backend_optimizations(t) + if backend_optimize: + backend_optimizations(t) return graphof(t, func), t def test_remove_direct_call_without_side_effects(): @@ -94,3 +96,34 @@ graph = TranslationContext().buildflowgraph(g) for block in graph.iterblocks(): assert len(block.inputargs) <= 2 # at most 'pc' and 'code' + +def test_get_graph(): + import os + def list_basic_ops(i, j): + l = [1,2,3] + l.insert(0, 42) + del l[1] + l.append(i) + listlen = len(l) + l.extend(l) + del l[listlen:] + l += [5,6] + l[1] = i + return l[j] + def external_function(): + return os.system("ls") + graph, t = translate(list_basic_ops, [int, int], False) + for block in graph.iterblocks(): + for op in block.operations: + if op.opname == "direct_call": + print op + graph = get_graph(op.args[0], t) + assert graph is not None + graph, t = translate(external_function, [], False) + for block in graph.iterblocks(): + for op in block.operations: + if op.opname == "direct_call": + print op + graph = get_graph(op.args[0], t) + assert graph is None + From ericvrp at codespeak.net Tue Jan 10 12:25:56 2006 From: ericvrp at codespeak.net (ericvrp at codespeak.net) Date: Tue, 10 Jan 2006 12:25:56 +0100 (CET) Subject: [pypy-svn] r21869 - pypy/dist/pypy/translator/js/test Message-ID: <20060110112556.4473827B58@code1.codespeak.net> Author: ericvrp Date: Tue Jan 10 12:25:55 2006 New Revision: 21869 Modified: pypy/dist/pypy/translator/js/test/runtest.py pypy/dist/pypy/translator/js/test/test_genllvm1.py pypy/dist/pypy/translator/js/test/test_merge_if_blocks.py pypy/dist/pypy/translator/js/test/test_seq.py Log: genjs: Translator -> TranslationContext Modified: pypy/dist/pypy/translator/js/test/runtest.py ============================================================================== --- pypy/dist/pypy/translator/js/test/runtest.py (original) +++ pypy/dist/pypy/translator/js/test/runtest.py Tue Jan 10 12:25:55 2006 @@ -1,5 +1,6 @@ import py, os -from pypy.translator.translator import Translator +from pypy.translator.translator import TranslationContext +from pypy.translator.backendopt.all import backend_optimizations from pypy.translator.js.js import JS from pypy.translator.js.test.browsertest import jstest from pypy.translator.js import conftest @@ -21,11 +22,11 @@ if not use_browsertest and not _CLI_is_on_path(): py.test.skip('Javascript CLI (js) not found') - t = Translator(function) - a = t.annotate(annotation) - t.specialize() - t.backend_optimizations(inline_threshold=0, mallocs=False) - #t.backend_optimizations() + t = TranslationContext() + t.buildannotator().build_types(function, annotation) + t.buildrtyper().specialize() + backend_optimizations(t, inline_threshold=0, mallocs=False) + #backend_optimizations(t) if view: t.view() self.js = JS(t, function, stackless) Modified: pypy/dist/pypy/translator/js/test/test_genllvm1.py ============================================================================== --- pypy/dist/pypy/translator/js/test/test_genllvm1.py (original) +++ pypy/dist/pypy/translator/js/test/test_genllvm1.py Tue Jan 10 12:25:55 2006 @@ -3,7 +3,6 @@ import py -from pypy.translator.translator import Translator from pypy.objspace.flow.model import Constant, Variable from pypy.translator.js.test.runtest import compile_function from pypy.translator.llvm.test import llvmsnippet Modified: pypy/dist/pypy/translator/js/test/test_merge_if_blocks.py ============================================================================== --- pypy/dist/pypy/translator/js/test/test_merge_if_blocks.py (original) +++ pypy/dist/pypy/translator/js/test/test_merge_if_blocks.py Tue Jan 10 12:25:55 2006 @@ -15,7 +15,7 @@ assert simple(i) == merge_if_blocks_simple(i) def test_merge_if_blocks_basic(): - py.skip("merge_if_block failing because link exit values are not used") + py.test.skip("merge_if_block failing because link exit values are not used") def merge_if_blocks_basic(i): if i == 5: return 1005 @@ -27,7 +27,7 @@ assert basic(i) == merge_if_blocks_basic(i) def test_merge_if_blocks_chr(): - py.skip("merge_if_block failing because link exit values are not used") + py.test.skip("merge_if_block failing because link exit values are not used") def merge_if_blocks_chr(i): c = chr(i) if c == '\x05': @@ -40,7 +40,7 @@ assert basic(i) == merge_if_blocks_chr(i) def test_merge_if_blocks_uni(): - py.skip("merge_if_block failing because link exit values are not used") + py.test.skip("merge_if_block failing because link exit values are not used") def merge_if_blocks_uni(i): c = unichr(i) if c == u'\x05': @@ -54,7 +54,7 @@ def test_merge_if_blocks_many(): - py.skip("merge_if_block failing because link exit values are not used") + py.test.skip("merge_if_block failing because link exit values are not used") def merge_if_blocks_many(i): if i == 0: return 1000 Modified: pypy/dist/pypy/translator/js/test/test_seq.py ============================================================================== --- pypy/dist/pypy/translator/js/test/test_seq.py (original) +++ pypy/dist/pypy/translator/js/test/test_seq.py Tue Jan 10 12:25:55 2006 @@ -1,7 +1,6 @@ from __future__ import division import py -from pypy.translator.translator import Translator from pypy.objspace.flow.model import Constant, Variable from pypy.translator.js.test.runtest import compile_function from pypy.translator.llvm.test import llvmsnippet From cfbolz at codespeak.net Tue Jan 10 12:38:19 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Tue, 10 Jan 2006 12:38:19 +0100 (CET) Subject: [pypy-svn] r21870 - pypy/dist/pypy/jit Message-ID: <20060110113819.37B8727B58@code1.codespeak.net> Author: cfbolz Date: Tue Jan 10 12:38:18 2006 New Revision: 21870 Modified: pypy/dist/pypy/jit/llabstractinterp.py Log: it seems I broke the jit tests when doing the direct_call refactoring. I am sorry. Modified: pypy/dist/pypy/jit/llabstractinterp.py ============================================================================== --- pypy/dist/pypy/jit/llabstractinterp.py (original) +++ pypy/dist/pypy/jit/llabstractinterp.py Tue Jan 10 12:38:18 2006 @@ -312,7 +312,7 @@ for st in stlist: op = st.origblock.operations[st.origposition] if op.opname == 'direct_call': - v = v.value + v = op.args[0].value elif op.opname == 'indirect_call': v = op.args[0] else: From cfbolz at codespeak.net Tue Jan 10 14:12:10 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Tue, 10 Jan 2006 14:12:10 +0100 (CET) Subject: [pypy-svn] r21874 - in pypy/dist/pypy: annotation annotation/test module/_socket/rpython/test module/thread/rpython/test rpython rpython/memory rpython/memory/test rpython/ootypesystem/test rpython/test translator translator/goal translator/pyrex translator/test translator/tool Message-ID: <20060110131210.4209B27B5A@code1.codespeak.net> Author: cfbolz Date: Tue Jan 10 14:11:57 2006 New Revision: 21874 Added: pypy/dist/pypy/annotation/annrpython.py - copied unchanged from r21864, pypy/dist/pypy/translator/annrpython.py pypy/dist/pypy/annotation/test/test_annmm.py - copied, changed from r21868, pypy/dist/pypy/translator/test/test_annmm.py pypy/dist/pypy/annotation/test/test_annrpython.py - copied, changed from r21864, pypy/dist/pypy/translator/test/test_annrpython.py pypy/dist/pypy/annotation/test/test_annsimplifyrpython.py - copied, changed from r21864, pypy/dist/pypy/translator/test/test_annsimplifyrpython.py pypy/dist/pypy/translator/goal/ann_override.py - copied unchanged from r21868, pypy/dist/pypy/translator/ann_override.py Removed: pypy/dist/pypy/translator/ann_override.py pypy/dist/pypy/translator/annrpython.py pypy/dist/pypy/translator/test/test_annmm.py pypy/dist/pypy/translator/test/test_annrpython.py pypy/dist/pypy/translator/test/test_annsimplifyrpython.py Modified: pypy/dist/pypy/module/_socket/rpython/test/test_ll__socket.py pypy/dist/pypy/module/thread/rpython/test/test_ll_thread.py pypy/dist/pypy/rpython/memory/gcwrapper.py pypy/dist/pypy/rpython/memory/test/test_address.py pypy/dist/pypy/rpython/memory/test/test_gc.py pypy/dist/pypy/rpython/ootypesystem/test/test_ooann.py pypy/dist/pypy/rpython/rtyper.py pypy/dist/pypy/rpython/test/test_llann.py pypy/dist/pypy/rpython/test/test_nongc.py pypy/dist/pypy/rpython/test/test_rpbc.py pypy/dist/pypy/rpython/test/test_rptr.py pypy/dist/pypy/rpython/test/test_rtyper.py pypy/dist/pypy/translator/gencl.py pypy/dist/pypy/translator/goal/targetmultiplespaces.py pypy/dist/pypy/translator/goal/targetpypymain.py pypy/dist/pypy/translator/goal/targetpypystandalone.py pypy/dist/pypy/translator/goal/targetthunkstandalone.py pypy/dist/pypy/translator/pyrex/genpyrex.py pypy/dist/pypy/translator/tool/tracer.py pypy/dist/pypy/translator/translator.py Log: move all the files that are relevant to the annotator from pypy/translator to pypy/annotation: * annrpython.py * test/test_annrpython.py * test/test_annsimplifyrpython.py * test/test_annmm.py moved pypy/translator/ann_override to pypy/translator/goal. It does not belong to annotation as it is PyPy specific. Fixed all the references to the moved files. Modified: pypy/dist/pypy/module/_socket/rpython/test/test_ll__socket.py ============================================================================== --- pypy/dist/pypy/module/_socket/rpython/test/test_ll__socket.py (original) +++ pypy/dist/pypy/module/_socket/rpython/test/test_ll__socket.py Tue Jan 10 14:11:57 2006 @@ -2,7 +2,7 @@ import _socket import pypy.module._socket.rpython.exttable from pypy.module._socket.rpython.ll__socket import * -from pypy.translator.annrpython import RPythonAnnotator +from pypy.annotation.annrpython import RPythonAnnotator from pypy.rpython.test.test_llinterp import interpret from pypy.rpython.module.support import from_rstr from pypy.rpython.module.support import to_opaque_object, from_opaque_object Modified: pypy/dist/pypy/module/thread/rpython/test/test_ll_thread.py ============================================================================== --- pypy/dist/pypy/module/thread/rpython/test/test_ll_thread.py (original) +++ pypy/dist/pypy/module/thread/rpython/test/test_ll_thread.py Tue Jan 10 14:11:57 2006 @@ -1,7 +1,7 @@ import thread import pypy.module.thread.rpython.exttable # for declare()/declaretype() from pypy.module.thread.rpython.ll_thread import * -from pypy.translator.annrpython import RPythonAnnotator +from pypy.annotation.annrpython import RPythonAnnotator from pypy.rpython.test.test_llinterp import interpret Modified: pypy/dist/pypy/rpython/memory/gcwrapper.py ============================================================================== --- pypy/dist/pypy/rpython/memory/gcwrapper.py (original) +++ pypy/dist/pypy/rpython/memory/gcwrapper.py Tue Jan 10 14:11:57 2006 @@ -1,4 +1,4 @@ -from pypy.translator.annrpython import RPythonAnnotator +from pypy.annotation.annrpython import RPythonAnnotator from pypy.rpython.rtyper import RPythonTyper from pypy.rpython.lltypesystem import lltype from pypy.rpython.memory.support import AddressLinkedList, INT_SIZE Modified: pypy/dist/pypy/rpython/memory/test/test_address.py ============================================================================== --- pypy/dist/pypy/rpython/memory/test/test_address.py (original) +++ pypy/dist/pypy/rpython/memory/test/test_address.py Tue Jan 10 14:11:57 2006 @@ -2,7 +2,7 @@ import sys from pypy.annotation import model as annmodel -from pypy.translator.annrpython import RPythonAnnotator +from pypy.annotation.annrpython import RPythonAnnotator from pypy.translator.translator import graphof from pypy.objspace.flow import FlowObjSpace from pypy.rpython.rtyper import RPythonTyper Modified: pypy/dist/pypy/rpython/memory/test/test_gc.py ============================================================================== --- pypy/dist/pypy/rpython/memory/test/test_gc.py (original) +++ pypy/dist/pypy/rpython/memory/test/test_gc.py Tue Jan 10 14:11:57 2006 @@ -2,7 +2,7 @@ import sys from pypy.annotation import model as annmodel -from pypy.translator.annrpython import RPythonAnnotator +from pypy.annotation.annrpython import RPythonAnnotator from pypy.rpython.rtyper import RPythonTyper from pypy.rpython.memory.gc import GCError, MarkSweepGC, SemiSpaceGC from pypy.rpython.memory.gc import DeferredRefcountingGC, DummyGC Modified: pypy/dist/pypy/rpython/ootypesystem/test/test_ooann.py ============================================================================== --- pypy/dist/pypy/rpython/ootypesystem/test/test_ooann.py (original) +++ pypy/dist/pypy/rpython/ootypesystem/test/test_ooann.py Tue Jan 10 14:11:57 2006 @@ -1,7 +1,7 @@ from pypy.rpython.ootypesystem.ootype import * from pypy.annotation import model as annmodel from pypy.objspace.flow import FlowObjSpace -from pypy.translator.annrpython import RPythonAnnotator +from pypy.annotation.annrpython import RPythonAnnotator def test_simple_new(): Modified: pypy/dist/pypy/rpython/rtyper.py ============================================================================== --- pypy/dist/pypy/rpython/rtyper.py (original) +++ pypy/dist/pypy/rpython/rtyper.py Tue Jan 10 14:11:57 2006 @@ -7,7 +7,7 @@ operations that are collected in the 'llops' list defined here. When necessary, conversions are inserted. -This logic borrows a bit from pypy.translator.annrpython, without the fixpoint +This logic borrows a bit from pypy.annotation.annrpython, without the fixpoint computation part. """ Modified: pypy/dist/pypy/rpython/test/test_llann.py ============================================================================== --- pypy/dist/pypy/rpython/test/test_llann.py (original) +++ pypy/dist/pypy/rpython/test/test_llann.py Tue Jan 10 14:11:57 2006 @@ -21,7 +21,7 @@ def setup_class(cls): cls.space = FlowObjSpace() - from pypy.translator.annrpython import RPythonAnnotator + from pypy.annotation.annrpython import RPythonAnnotator def annotate(self, ll_function, argtypes): self.a = self.RPythonAnnotator() Modified: pypy/dist/pypy/rpython/test/test_nongc.py ============================================================================== --- pypy/dist/pypy/rpython/test/test_nongc.py (original) +++ pypy/dist/pypy/rpython/test/test_nongc.py Tue Jan 10 14:11:57 2006 @@ -1,7 +1,7 @@ import py from pypy.annotation import model as annmodel -from pypy.translator.annrpython import RPythonAnnotator +from pypy.annotation.annrpython import RPythonAnnotator from pypy.rpython.rtyper import RPythonTyper from pypy.rpython.objectmodel import free_non_gc_object from pypy.rpython.test.test_llinterp import interpret Modified: pypy/dist/pypy/rpython/test/test_rpbc.py ============================================================================== --- pypy/dist/pypy/rpython/test/test_rpbc.py (original) +++ pypy/dist/pypy/rpython/test/test_rpbc.py Tue Jan 10 14:11:57 2006 @@ -877,7 +877,7 @@ def f(a,b): return a + b from pypy.translator import translator - from pypy.translator import annrpython + from pypy.annotation import annrpython a = annrpython.RPythonAnnotator() from pypy.annotation import model as annmodel @@ -915,7 +915,7 @@ return a + b def f2(a,b): return a - b - from pypy.translator import annrpython + from pypy.annotation import annrpython a = annrpython.RPythonAnnotator() from pypy.annotation import model as annmodel @@ -968,7 +968,7 @@ def f(a): return A(a) - from pypy.translator import annrpython + from pypy.annotation import annrpython a = annrpython.RPythonAnnotator() from pypy.annotation import model as annmodel @@ -1017,7 +1017,7 @@ def f(self, a): return A(a) - from pypy.translator import annrpython + from pypy.annotation import annrpython a = annrpython.RPythonAnnotator() from pypy.annotation import model as annmodel @@ -1077,7 +1077,7 @@ def f(self, a): return A(a) - from pypy.translator import annrpython + from pypy.annotation import annrpython a = annrpython.RPythonAnnotator() from pypy.annotation import model as annmodel Modified: pypy/dist/pypy/rpython/test/test_rptr.py ============================================================================== --- pypy/dist/pypy/rpython/test/test_rptr.py (original) +++ pypy/dist/pypy/rpython/test/test_rptr.py Tue Jan 10 14:11:57 2006 @@ -1,4 +1,4 @@ -from pypy.translator.annrpython import RPythonAnnotator +from pypy.annotation.annrpython import RPythonAnnotator from pypy.rpython.annlowlevel import annotate_lowlevel_helper, LowLevelAnnotatorPolicy from pypy.rpython.lltypesystem.lltype import * from pypy.rpython.rtyper import RPythonTyper Modified: pypy/dist/pypy/rpython/test/test_rtyper.py ============================================================================== --- pypy/dist/pypy/rpython/test/test_rtyper.py (original) +++ pypy/dist/pypy/rpython/test/test_rtyper.py Tue Jan 10 14:11:57 2006 @@ -1,7 +1,7 @@ from pypy.annotation import model as annmodel from pypy.objspace.flow.model import Constant from pypy.translator.translator import TranslationContext, graphof -from pypy.translator import annrpython +from pypy.annotation import annrpython from pypy.rpython.lltypesystem.lltype import * from pypy.rpython.test.test_llinterp import interpret from pypy.rpython.rtyper import RPythonTyper Modified: pypy/dist/pypy/translator/gencl.py ============================================================================== --- pypy/dist/pypy/translator/gencl.py (original) +++ pypy/dist/pypy/translator/gencl.py Tue Jan 10 14:11:57 2006 @@ -1,5 +1,5 @@ from pypy.objspace.flow.model import Constant, Variable, last_exception -from pypy.translator.annrpython import RPythonAnnotator +from pypy.annotation.annrpython import RPythonAnnotator from pypy.translator.simplify import simplify_graph from pypy.translator.transform import transform_graph, default_extra_passes, transform_slice Modified: pypy/dist/pypy/translator/goal/targetmultiplespaces.py ============================================================================== --- pypy/dist/pypy/translator/goal/targetmultiplespaces.py (original) +++ pypy/dist/pypy/translator/goal/targetmultiplespaces.py Tue Jan 10 14:11:57 2006 @@ -7,7 +7,7 @@ from pypy.annotation.listdef import ListDef from pypy.interpreter import gateway from pypy.interpreter.error import OperationError -from pypy.translator.ann_override import PyPyAnnotatorPolicy +from pypy.translator.goal.ann_override import PyPyAnnotatorPolicy # WARNING: this requires the annotator. # There is no easy way to build all caches manually, Modified: pypy/dist/pypy/translator/goal/targetpypymain.py ============================================================================== --- pypy/dist/pypy/translator/goal/targetpypymain.py (original) +++ pypy/dist/pypy/translator/goal/targetpypymain.py Tue Jan 10 14:11:57 2006 @@ -7,7 +7,7 @@ from pypy.annotation.listdef import ListDef from pypy.interpreter import gateway from pypy.interpreter.error import OperationError -from pypy.translator.ann_override import PyPyAnnotatorPolicy +from pypy.translator.goal.ann_override import PyPyAnnotatorPolicy # WARNING: this requires the annotator. # There is no easy way to build all caches manually, Modified: pypy/dist/pypy/translator/goal/targetpypystandalone.py ============================================================================== --- pypy/dist/pypy/translator/goal/targetpypystandalone.py (original) +++ pypy/dist/pypy/translator/goal/targetpypystandalone.py Tue Jan 10 14:11:57 2006 @@ -7,7 +7,7 @@ from pypy.annotation.listdef import ListDef from pypy.interpreter import gateway from pypy.interpreter.error import OperationError -from pypy.translator.ann_override import PyPyAnnotatorPolicy +from pypy.translator.goal.ann_override import PyPyAnnotatorPolicy # WARNING: this requires the annotator. # There is no easy way to build all caches manually, Modified: pypy/dist/pypy/translator/goal/targetthunkstandalone.py ============================================================================== --- pypy/dist/pypy/translator/goal/targetthunkstandalone.py (original) +++ pypy/dist/pypy/translator/goal/targetthunkstandalone.py Tue Jan 10 14:11:57 2006 @@ -7,7 +7,7 @@ from pypy.annotation.listdef import ListDef from pypy.interpreter import gateway from pypy.interpreter.error import OperationError -from pypy.translator.ann_override import PyPyAnnotatorPolicy +from pypy.translator.goal.ann_override import PyPyAnnotatorPolicy # WARNING: this requires the annotator. # There is no easy way to build all caches manually, Modified: pypy/dist/pypy/translator/pyrex/genpyrex.py ============================================================================== --- pypy/dist/pypy/translator/pyrex/genpyrex.py (original) +++ pypy/dist/pypy/translator/pyrex/genpyrex.py Tue Jan 10 14:11:57 2006 @@ -6,7 +6,7 @@ from pypy.interpreter.argument import Arguments from pypy.objspace.flow.model import Variable, Constant from pypy.objspace.flow.model import mkentrymap, last_exception -from pypy.translator.annrpython import RPythonAnnotator +from pypy.annotation.annrpython import RPythonAnnotator from pypy.annotation.model import SomePBC from pypy.annotation.description import MethodDesc from pypy.annotation.classdef import ClassDef Modified: pypy/dist/pypy/translator/tool/tracer.py ============================================================================== --- pypy/dist/pypy/translator/tool/tracer.py (original) +++ pypy/dist/pypy/translator/tool/tracer.py Tue Jan 10 14:11:57 2006 @@ -1,7 +1,7 @@ import types, sys from pypy.annotation.model import SomeValue, debugname from pypy.annotation.annset import AnnotationSet -from pypy.translator.annrpython import RPythonAnnotator +from pypy.annotation.annrpython import RPythonAnnotator indent1 = [''] Modified: pypy/dist/pypy/translator/translator.py ============================================================================== --- pypy/dist/pypy/translator/translator.py (original) +++ pypy/dist/pypy/translator/translator.py Tue Jan 10 14:11:57 2006 @@ -62,7 +62,7 @@ def buildannotator(self, policy=None): if self.annotator is not None: raise ValueError("we already have an annotator") - from pypy.translator.annrpython import RPythonAnnotator + from pypy.annotation.annrpython import RPythonAnnotator self.annotator = RPythonAnnotator(self, policy=policy) return self.annotator @@ -244,7 +244,7 @@ if input_arg_types is None: ann = self.annotator else: - from pypy.translator.annrpython import RPythonAnnotator + from pypy.annotation.annrpython import RPythonAnnotator ann = RPythonAnnotator(self) if func is None: codes = [self.generatecode1(gencls, input_arg_types, From ac at codespeak.net Tue Jan 10 15:21:18 2006 From: ac at codespeak.net (ac at codespeak.net) Date: Tue, 10 Jan 2006 15:21:18 +0100 (CET) Subject: [pypy-svn] r21881 - pypy/extradoc/sprintinfo/mallorca Message-ID: <20060110142118.8985127B58@code1.codespeak.net> Author: ac Date: Tue Jan 10 15:21:18 2006 New Revision: 21881 Modified: pypy/extradoc/sprintinfo/mallorca/people.txt Log: Dates for Me and Samuele. Modified: pypy/extradoc/sprintinfo/mallorca/people.txt ============================================================================== --- pypy/extradoc/sprintinfo/mallorca/people.txt (original) +++ pypy/extradoc/sprintinfo/mallorca/people.txt Tue Jan 10 15:21:18 2006 @@ -11,11 +11,11 @@ ==================== ============== ===================== Beatrice Duering 22/1 -29/1 flat? Armin Rigo 21/1 - 30?/1 ? -Samuele Pedroni ? ? +Samuele Pedroni 22/1 - 30/1 Hotel Colon Eric van Riet Paap 22/1 - 28/1 casa M. Michael Hudson ? ? Carl Friedrich Bolz ? ? -Anders Chrigstroem ? ? +Anders Chrigstroem 22/1 - 30/1 Hotel Colon Christian Tismer 22/1 - 30/1 ? Gerald Klix 22/1 - 30/1 ? Stephan Diehl 22/1 - 28/1 ? From pedronis at codespeak.net Tue Jan 10 15:53:39 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Tue, 10 Jan 2006 15:53:39 +0100 (CET) Subject: [pypy-svn] r21884 - in pypy/dist/pypy: annotation annotation/test rpython rpython/test translator/c/test Message-ID: <20060110145339.6712027B55@code1.codespeak.net> Author: pedronis Date: Tue Jan 10 15:53:36 2006 New Revision: 21884 Modified: pypy/dist/pypy/annotation/builtin.py pypy/dist/pypy/annotation/test/test_annrpython.py pypy/dist/pypy/rpython/objectmodel.py pypy/dist/pypy/rpython/rbuiltin.py pypy/dist/pypy/rpython/test/test_objectmodel.py pypy/dist/pypy/translator/c/test/test_backendoptimized.py Log: remove cast_object<->int, they are dangerous and of unclear purpose right now Modified: pypy/dist/pypy/annotation/builtin.py ============================================================================== --- pypy/dist/pypy/annotation/builtin.py (original) +++ pypy/dist/pypy/annotation/builtin.py Tue Jan 10 15:53:36 2006 @@ -270,15 +270,6 @@ def rarith_intmask(s_obj): return SomeInteger() -def robjmodel_cast_obj_to_int(s_instance): # XXX GC behavior - return SomeInteger() - -def robjmodel_cast_int_to_obj(s_int, s_clspbc): # XXX GC behavior - assert len(s_clspbc.descriptions) == 1 - desc = s_clspbc.descriptions.keys()[0] - cdef = desc.getuniqueclassdef() - return SomeInstance(cdef) - def robjmodel_instantiate(s_clspbc): assert isinstance(s_clspbc, SomePBC) clsdef = None @@ -362,8 +353,6 @@ ##BUILTIN_ANALYZERS[pypy.rpython.rarithmetic.ovfcheck] = rarith_ovfcheck ##BUILTIN_ANALYZERS[pypy.rpython.rarithmetic.ovfcheck_lshift] = rarith_ovfcheck_lshift BUILTIN_ANALYZERS[pypy.rpython.rarithmetic.intmask] = rarith_intmask -BUILTIN_ANALYZERS[pypy.rpython.objectmodel.cast_object_to_int] = robjmodel_cast_obj_to_int -BUILTIN_ANALYZERS[pypy.rpython.objectmodel.cast_int_to_object] = robjmodel_cast_int_to_obj BUILTIN_ANALYZERS[pypy.rpython.objectmodel.instantiate] = robjmodel_instantiate BUILTIN_ANALYZERS[pypy.rpython.objectmodel.we_are_translated] = ( robjmodel_we_are_translated) Modified: pypy/dist/pypy/annotation/test/test_annrpython.py ============================================================================== --- pypy/dist/pypy/annotation/test/test_annrpython.py (original) +++ pypy/dist/pypy/annotation/test/test_annrpython.py Tue Jan 10 15:53:36 2006 @@ -1944,21 +1944,6 @@ assert s.knowntype == int graph = tgraphof(t, A.__del__.im_func) assert graph.startblock in a.annotated - - def test_casttoandfromint(self): - class A(object): - pass - def f(): - a = A() - return objectmodel.cast_object_to_int(a) - def g(i): - return objectmodel.cast_int_to_object(i, A) - a = self.RPythonAnnotator() - s = a.build_types(f, []) - assert isinstance(s, annmodel.SomeInteger) - s = a.build_types(g, [annmodel.SomeInteger()]) - assert isinstance(s, annmodel.SomeInstance) - assert s.classdef.classdesc.pyobj is A def g(n): Modified: pypy/dist/pypy/rpython/objectmodel.py ============================================================================== --- pypy/dist/pypy/rpython/objectmodel.py (original) +++ pypy/dist/pypy/rpython/objectmodel.py Tue Jan 10 15:53:36 2006 @@ -37,29 +37,7 @@ obj.__class__ = FREED_OBJECT -# support for cast from object to int and back - -__int_to_weakref = {} - -def cast_object_to_int(obj): - i = id(obj) - if i not in __int_to_weakref: - __int_to_weakref[i] = weakref.ref(obj) - return i - -def cast_int_to_object(i, expected_class): - # only ints are valid that are the result of cast_object_to_int - if i not in __int_to_weakref: - raise ValueError("not a valid object") - obj = __int_to_weakref[i]() - if obj is not None: - if type(obj) != expected_class: - raise ValueError("class of obj != expected_class") - return obj - else: - return FREED_OBJECT() - -# __ invoke XXX this doesn't seem completely the right place for this +# __ hlinvoke XXX this doesn't seem completely the right place for this def hlinvoke(repr, llcallable, *args): raise TypeError, "hlinvoke is meant to be rtyped and not called direclty" Modified: pypy/dist/pypy/rpython/rbuiltin.py ============================================================================== --- pypy/dist/pypy/rpython/rbuiltin.py (original) +++ pypy/dist/pypy/rpython/rbuiltin.py Tue Jan 10 15:53:36 2006 @@ -301,18 +301,6 @@ return hop.genop('runtime_type_info', vlist, resulttype = rptr.PtrRepr(lltype.Ptr(lltype.RuntimeTypeInfo))) -def rtype_cast_object_to_int(hop): - assert isinstance(hop.args_r[0], rclass.InstanceRepr) - vlist = hop.inputargs(hop.args_r[0]) - return hop.genop('cast_ptr_to_int', vlist, - resulttype = rint.signed_repr) - -def rtype_cast_int_to_object(hop): - assert isinstance(hop.args_r[0], rint.IntegerRepr) - vlist = [hop.inputarg(rint.signed_repr, arg=0)] - return hop.genop('cast_int_to_ptr', vlist, - resulttype = hop.r_result.lowleveltype) - BUILTIN_TYPER[lltype.malloc] = rtype_malloc BUILTIN_TYPER[lltype.cast_pointer] = rtype_cast_pointer BUILTIN_TYPER[lltype.cast_ptr_to_int] = rtype_cast_ptr_to_int @@ -326,8 +314,6 @@ BUILTIN_TYPER[rarithmetic.r_ulonglong] = rtype_r_ulonglong BUILTIN_TYPER[objectmodel.r_dict] = rtype_r_dict BUILTIN_TYPER[objectmodel.we_are_translated] = rtype_we_are_translated -BUILTIN_TYPER[objectmodel.cast_object_to_int] = rtype_cast_object_to_int -BUILTIN_TYPER[objectmodel.cast_int_to_object] = rtype_cast_int_to_object BUILTIN_TYPER[rstack.yield_current_frame_to_caller] = ( rtype_yield_current_frame_to_caller) Modified: pypy/dist/pypy/rpython/test/test_objectmodel.py ============================================================================== --- pypy/dist/pypy/rpython/test/test_objectmodel.py (original) +++ pypy/dist/pypy/rpython/test/test_objectmodel.py Tue Jan 10 15:53:36 2006 @@ -11,39 +11,6 @@ res = interpret(fn, []) assert res is True -def test_casttoandfromint_on_cpython(): - class A(object): - pass - class B(object): - pass - a = A() - b = B() - i1 = cast_object_to_int(a) - i2 = cast_object_to_int(b) - assert cast_int_to_object(i1, A) is a - assert cast_int_to_object(i2, B) is b - a = None - b = None - a = cast_int_to_object(i1, A) - py.test.raises(RuntimeError, "a.b") - -def test_casttoandfromint(): - class A(object): - pass - def f(): - a = A() - return cast_object_to_int(a) - def g(): - a = A() - i = cast_object_to_int(a) - return cast_object_to_int(cast_int_to_object(i, A)) == i - res = interpret(f, []) - # cannot really check anything about 'res' here - - # XXX humpf: there is no sane way to implement cast_ptr_to_int - # without going for the same hacks as in robjectmodel.cast_XXX_to_XXX - py.test.raises(AssertionError, interpret, g, []) - def strange_key_eq(key1, key2): return key1[0] == key2[0] # only the 1st character is relevant def strange_key_hash(key): Modified: pypy/dist/pypy/translator/c/test/test_backendoptimized.py ============================================================================== --- pypy/dist/pypy/translator/c/test/test_backendoptimized.py (original) +++ pypy/dist/pypy/translator/c/test/test_backendoptimized.py Tue Jan 10 15:53:36 2006 @@ -77,25 +77,6 @@ res = fn() assert res == 42 - def test_casttoandfromint(self): - class A(object): - pass - def f(): - a = A() - return objectmodel.cast_object_to_int(a) - def g(): - a = A() - i = objectmodel.cast_object_to_int(a) - return objectmodel.cast_object_to_int( - objectmodel.cast_int_to_object(i, A)) == i - fn = self.getcompiled(f) - res = fn() - # cannot really test anything about 'res' here - gn = self.getcompiled(g) - res = gn() - assert res - - class TestTypedOptimizedSwitchTestCase: From cfbolz at codespeak.net Tue Jan 10 15:58:06 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Tue, 10 Jan 2006 15:58:06 +0100 (CET) Subject: [pypy-svn] r21885 - pypy/dist/pypy/doc Message-ID: <20060110145806.C771927B55@code1.codespeak.net> Author: cfbolz Date: Tue Jan 10 15:58:04 2006 New Revision: 21885 Modified: pypy/dist/pypy/doc/_ref.txt pypy/dist/pypy/doc/getting-started.txt Log: doc changes because of the file moval Modified: pypy/dist/pypy/doc/_ref.txt ============================================================================== --- pypy/dist/pypy/doc/_ref.txt (original) +++ pypy/dist/pypy/doc/_ref.txt Tue Jan 10 15:58:04 2006 @@ -3,6 +3,7 @@ .. _`lib-python/2.4.1/dis.py`: ../../lib-python/2.4.1/dis.py .. _`annotation/`: .. _`pypy/annotation`: ../../pypy/annotation +.. _`pypy/annotation/annrpython.py`: ../../pypy/annotation/annrpython.py .. _`annotation/binaryop.py`: ../../pypy/annotation/binaryop.py .. _`pypy/annotation/builtin.py`: ../../pypy/annotation/builtin.py .. _`pypy/annotation/model.py`: ../../pypy/annotation/model.py @@ -67,7 +68,6 @@ .. _`tool/tb_server/`: ../../pypy/tool/tb_server .. _`pypy/translator`: .. _`translator/`: ../../pypy/translator -.. _`pypy/translator/annrpython.py`: ../../pypy/translator/annrpython.py .. _`translator/c/`: ../../pypy/translator/c .. _`pypy/translator/c/extfunc.py`: ../../pypy/translator/c/extfunc.py .. _`pypy/translator/c/src/`: ../../pypy/translator/c/src Modified: pypy/dist/pypy/doc/getting-started.txt ============================================================================== --- pypy/dist/pypy/doc/getting-started.txt (original) +++ pypy/dist/pypy/doc/getting-started.txt Tue Jan 10 15:58:04 2006 @@ -556,7 +556,7 @@ * `pypy/annotation`_ contains the data model for the type annotation that can be inferred about a graph. The graph "walker" that uses this is in - `pypy/translator/annrpython.py`_. + `pypy/annotation/annrpython.py`_. * `pypy/rpython`_ contains the code of the RPython typer. The typer transforms annotated flow graphs in a way that makes them very similar to C code so From hpk at codespeak.net Tue Jan 10 16:11:56 2006 From: hpk at codespeak.net (hpk at codespeak.net) Date: Tue, 10 Jan 2006 16:11:56 +0100 (CET) Subject: [pypy-svn] r21887 - pypy/extradoc/sprintinfo/mallorca Message-ID: <20060110151156.6B80C27B45@code1.codespeak.net> Author: hpk Date: Tue Jan 10 16:11:48 2006 New Revision: 21887 Modified: pypy/extradoc/sprintinfo/mallorca/sprint-announcement.txt Log: corrected the people link in the announcement Modified: pypy/extradoc/sprintinfo/mallorca/sprint-announcement.txt ============================================================================== --- pypy/extradoc/sprintinfo/mallorca/sprint-announcement.txt (original) +++ pypy/extradoc/sprintinfo/mallorca/sprint-announcement.txt Tue Jan 10 16:11:48 2006 @@ -144,7 +144,7 @@ already thought to come. If you have commit rights on codespeak then you can modify yourself a checkout of - http://codespeak.net/svn/pypy/extradoc/sprintinfo/mallorca-2006/people.txt + http://codespeak.net/svn/pypy/extradoc/sprintinfo/mallorca/people.txt .. _`PyPy sprint mailing list`: http://codespeak.net/mailman/listinfo/pypy-sprint -.. _`Mallorca people`: http://codespeak.net/pypy/extradoc/sprintinfo/mallorca-2006/people.html +.. _`Mallorca people`: http://codespeak.net/pypy/extradoc/sprintinfo/mallorca/people.html From arigo at codespeak.net Tue Jan 10 17:02:18 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 10 Jan 2006 17:02:18 +0100 (CET) Subject: [pypy-svn] r21889 - pypy/dist/pypy/translator/tool Message-ID: <20060110160218.BE57027B55@code1.codespeak.net> Author: arigo Date: Tue Jan 10 17:02:17 2006 New Revision: 21889 Modified: pypy/dist/pypy/translator/tool/graphpage.py Log: Quick hack: TranslationContext.view() with no graph computed so far was meant to try to get the graph of the entry point, if any. Modified: pypy/dist/pypy/translator/tool/graphpage.py ============================================================================== --- pypy/dist/pypy/translator/tool/graphpage.py (original) +++ pypy/dist/pypy/translator/tool/graphpage.py Tue Jan 10 17:02:17 2006 @@ -123,6 +123,9 @@ graphs += graphsof(translator, func) else: graphs = self.translator.graphs + if not graphs and hasattr(translator, 'entrypoint'): + graphs = list(graphs) + graphs += graphsof(translator, translator.entrypoint) gs = [(graph.name, graph) for graph in graphs] if self.annotator and self.annotator.blocked_graphs: for block, was_annotated in self.annotator.annotated.items(): From pedronis at codespeak.net Tue Jan 10 17:22:13 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Tue, 10 Jan 2006 17:22:13 +0100 (CET) Subject: [pypy-svn] r21891 - pypy/dist/pypy/translator Message-ID: <20060110162213.7199227B55@code1.codespeak.net> Author: pedronis Date: Tue Jan 10 17:22:12 2006 New Revision: 21891 Modified: pypy/dist/pypy/translator/interactive.py Log: added view/viewcg methods delegating to the wrapped TranslationContext. Modified: pypy/dist/pypy/translator/interactive.py ============================================================================== --- pypy/dist/pypy/translator/interactive.py (original) +++ pypy/dist/pypy/translator/interactive.py Tue Jan 10 17:22:12 2006 @@ -55,6 +55,12 @@ 'compile_llvm': [], } + def view(self): + self.context.view() + + def viewcg(self): + self.context.viewcg() + def driver_event(self, kind, goal, func): if kind == 'pre': #print goal From arigo at codespeak.net Tue Jan 10 17:24:40 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 10 Jan 2006 17:24:40 +0100 (CET) Subject: [pypy-svn] r21892 - in pypy/extradoc/talk/chalmers: . image Message-ID: <20060110162440.2B2FE27B58@code1.codespeak.net> Author: arigo Date: Tue Jan 10 17:24:36 2006 New Revision: 21892 Added: pypy/extradoc/talk/chalmers/image/ pypy/extradoc/talk/chalmers/image/compiler.dot pypy/extradoc/talk/chalmers/image/interpreter.dot pypy/extradoc/talk/chalmers/image/stackless_informal.png (contents, props changed) pypy/extradoc/talk/chalmers/image/translation-overview.png - copied unchanged from r21878, pypy/extradoc/talk/22c3/translation-overview.png pypy/extradoc/talk/chalmers/py-web1.png (contents, props changed) pypy/extradoc/talk/chalmers/talk-wm2006.txt (contents, props changed) Log: Draft of the Chalmers Winter Meeting talk. (ReST/S5) Added: pypy/extradoc/talk/chalmers/image/compiler.dot ============================================================================== --- (empty file) +++ pypy/extradoc/talk/chalmers/image/compiler.dot Tue Jan 10 17:24:36 2006 @@ -0,0 +1,29 @@ +digraph itp_graph { + node [shape="octagon", color="black", fillcolor="white", style="filled"]; + subgraph itp { + interpret_ [shape="box", label="def interpret(code, args):\l while more bytecodes:\l next_instr = ...\l if next_instr == \"add\":\l add(args)\l elif next_instr == \"sub\":\l sub(args)\l", fillcolor="green"]; +interpret_ -> interpret__1 +interpret__1 [shape="octagon", label="more bytecodes?", color="black", fillcolor="white", style="filled"]; +edge [label="No", style="dotted", color="black", dir="forward", weight="5"]; +interpret__1 -> interpret__2 +edge [label="Yes", style="dotted", color="black", dir="forward", weight="5"]; +interpret__1 -> interpret__3 +interpret__2 [shape="box", label="return", color="black", fillcolor="green", style="filled"]; +interpret__3 [shape="octagon", label="next_instr = ...\l\nnext_instr == 'add'?", color="black", fillcolor="white", style="filled"]; +edge [label="No", style="dotted", color="black", dir="forward", weight="5"]; +interpret__3 -> interpret__4 +edge [label="Yes", style="dotted", color="black", dir="forward", weight="5"]; +interpret__3 -> interpret__5 +interpret__4 [shape="octagon", label="next_instr == 'sub' ?", color="black", fillcolor="white", style="filled"]; +edge [label="No", style="dotted", color="black", dir="forward", weight="5"]; +interpret__4 -> interpret__1 +edge [label="Yes", style="dotted", color="black", dir="forward", weight="5"]; +interpret__4 -> interpret__6 +interpret__6 [shape="box", label="generate sub(args)", color="black", fillcolor="yellow", style="filled"]; +edge [label="", style="solid", color="black", dir="forward", weight="5"]; +interpret__6 -> interpret__1 +interpret__5 [shape="box", label="generate add(args)", color="black", fillcolor="yellow", style="filled"]; +edge [label="", style="solid", color="black", dir="forward", weight="5"]; +interpret__5 -> interpret__1 + } +} Added: pypy/extradoc/talk/chalmers/image/interpreter.dot ============================================================================== --- (empty file) +++ pypy/extradoc/talk/chalmers/image/interpreter.dot Tue Jan 10 17:24:36 2006 @@ -0,0 +1,29 @@ +digraph itp_graph { + node [shape="octagon", color="black", fillcolor="white", style="filled"]; + subgraph itp { + interpret_ [shape="box", label="def interpret(code, args):\l while more bytecodes:\l next_instr = ...\l if next_instr == \"add\":\l add(args)\l elif next_instr == \"sub\":\l sub(args)\l", fillcolor="green"]; +interpret_ -> interpret__1 +interpret__1 [shape="octagon", label="more bytecodes?", color="black", fillcolor="white", style="filled"]; +edge [label="No", style="dotted", color="black", dir="forward", weight="5"]; +interpret__1 -> interpret__2 +edge [label="Yes", style="dotted", color="black", dir="forward", weight="5"]; +interpret__1 -> interpret__3 +interpret__2 [shape="box", label="return", color="black", fillcolor="green", style="filled"]; +interpret__3 [shape="octagon", label="next_instr = ...\l\nnext_instr == 'add'?", color="black", fillcolor="white", style="filled"]; +edge [label="No", style="dotted", color="black", dir="forward", weight="5"]; +interpret__3 -> interpret__4 +edge [label="Yes", style="dotted", color="black", dir="forward", weight="5"]; +interpret__3 -> interpret__5 +interpret__4 [shape="octagon", label="next_instr == 'sub' ?", color="black", fillcolor="white", style="filled"]; +edge [label="No", style="dotted", color="black", dir="forward", weight="5"]; +interpret__4 -> interpret__1 +edge [label="Yes", style="dotted", color="black", dir="forward", weight="5"]; +interpret__4 -> interpret__6 +interpret__6 [shape="box", label="sub(args)", color="black", fillcolor="white", style="filled"]; +edge [label="", style="solid", color="black", dir="forward", weight="5"]; +interpret__6 -> interpret__1 +interpret__5 [shape="box", label="add(args)", color="black", fillcolor="white", style="filled"]; +edge [label="", style="solid", color="black", dir="forward", weight="5"]; +interpret__5 -> interpret__1 + } +} Added: pypy/extradoc/talk/chalmers/image/stackless_informal.png ============================================================================== Binary file. No diff available. Added: pypy/extradoc/talk/chalmers/py-web1.png ============================================================================== Binary file. No diff available. Added: pypy/extradoc/talk/chalmers/talk-wm2006.txt ============================================================================== --- (empty file) +++ pypy/extradoc/talk/chalmers/talk-wm2006.txt Tue Jan 10 17:24:36 2006 @@ -0,0 +1,139 @@ +.. include:: + +====================================================== + PyPy +====================================================== + +Dynamic optimizations for your favorite language +###################################################### + +:Author: Armin Rigo +:URL: http://codespeak.net/pypy + +.. image:: py-web1.png + :align: right + + +Python Implementation Facts +=========================== + +- Parser/Compiler produces bytecode +- Virtual Machine interprets bytecode +- strongly dynamically typed +- large library of built-in types +- complicated semantics and object model + +Python Implementations +=========================== + +- CPython: main Python version +- Jython: compiles to Java Bytecode +- IronPython (MS): compiles to .NET's CLR +- PyPy: self-contained - self-translating - flexible + +PyPy Project Facts +======================= + +- started 2002 as a grass-root effort +- aims: flexibility, research, speed +- test-driven development +- received EU-funding from end 2004 on +- 350 subscribers to pypy-dev, 150.000 LOCs, 20.000 visitors per month, +- MIT license + +Overview +========================== + +- Interpreter implements Python language in Python itself +- parts implemented in a restricted subset: RPython +- "static enough" for full-program type inference +- but no low-level details +- suitable for translation to many platforms + +Translation Overview +========================= + +.. image:: translation-overview.png + :align: center + +Translation Aspects +==================== + +- can target very different platforms +- implementation decisions at translation time (GC, threads_, calling conventions...) +- most other language implementations do "fixed" decisions +- translation aspects are weaved into the produced code +- independent from language semantics (python interpreter) + +Translation Aspects (2) +========================= + +- the execution stragegy can be tweaked at translation time too +- if the input RPython program is an Interpreter_... +- ...the output low-level program can be a Compiler_ + +Partial Evaluation +==================== + +- Partial Evaluation: simplify a program for partially known inputs +- in theory, can turn an interpreter into a compiler +- many practical difficulties + +Psyco +==================== + +- Psyco_: hand-written prototype for CPython +- makes CPython run faster, transparently +- produces x86 machine code at run-time +- a "generating extension": generates specialized versions of the CPython Interpreter + +PyPy JIT Overview +==================== + +- translation aspect + +:input: + PyPy Interpreter written in RPython + +:prev output: + an Interpreter -- like CPython + +:new output: + a Compiler -- like Psyco + + +Recipe for a JIT +========================== + +- JIT part 1: high level, language dependent +- JIT part 2: platform-specific code generation and optimizations +- separation of concerns + +PyPy JIT part 1 +========================== + +- generated as translation aspect from PyPy Interpreter +- stays in sync with the evolution of the language +- same technique works for other languages + +PyPy JIT part 2 +========================= + +- no cross-platform code generation toolkit +- needs to be manually rewritten for each platform +- orthogonal to the source language +- can reuse existing code generation and optimization projects + +Conclusion +========================= + +XXX + + +.. _Psyco: http://psyco.sourceforge.net +.. _threads: image/stackless_informal.png +.. _Interpreter: image/interpreter.png +.. _Compiler: image/compiler.png + +.. |bullet| unicode:: U+02022 +.. footer:: Armin Rigo |bullet| Winter Meeting |bullet| 12th January 2006 From cfbolz at codespeak.net Tue Jan 10 17:25:09 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Tue, 10 Jan 2006 17:25:09 +0100 (CET) Subject: [pypy-svn] r21893 - in pypy/dist/pypy/translator/backendopt: . test Message-ID: <20060110162509.1DB0427B58@code1.codespeak.net> Author: cfbolz Date: Tue Jan 10 17:25:03 2006 New Revision: 21893 Added: pypy/dist/pypy/translator/backendopt/escape.py pypy/dist/pypy/translator/backendopt/test/test_escape.py Log: first attempt at a simple inter-procedural escape analysis. many operations are still missing but works for very basic cases. Added: pypy/dist/pypy/translator/backendopt/escape.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/backendopt/escape.py Tue Jan 10 17:25:03 2006 @@ -0,0 +1,283 @@ +from pypy.annotation.model import setunion +from pypy.objspace.flow.model import Variable, Constant +from pypy.rpython.lltypesystem import lltype +from pypy.translator.simplify import get_graph + +class CreationPoint(object): + def __init__(self, creation_method="?"): + self.changes = False + self.escapes = False + self.creation_method = creation_method + if creation_method == "constant": + self.changes = True + self.escapes = True + self.malloced = False + + def __repr__(self): + return ("CreationPoint(<%s>, %s, esc=%s, cha=%s)" % + (id(self), self.creation_method, self.escapes, self.changes)) + +class VarState(object): + def __init__(self, crep=None): + self.creation_points = {} + if crep is not None: + self.creation_points[crep] = True + + def contains(self, other): + for crep in other.creation_points: + if crep not in self.creation_points: + return False + return True + + def merge(self, other): + newstate = VarState() + creation_points = setunion(self.creation_points, other.creation_points) + newstate = VarState() + newstate.creation_points = creation_points + return newstate + + def setescapes(self): + changed = [] + for crep in self.creation_points: + if not crep.escapes: + changed.append(crep) + crep.escapes = True + return changed + + def setchanges(self): + changed = [] + for crep in self.creation_points: + if not crep.changes: + changed.append(crep) + crep.changes = True + return changed + + def does_escape(self): + for crep in self.creation_points: + if crep.escapes: + return True + return False + + def does_change(self): + for crep in self.creation_points: + if crep.changes: + return True + return False + + def __repr__(self): + crepsrepr = (", ".join([repr(crep) for crep in self.creation_points]), ) + return "VarState({%s})" % crepsrepr + +class AbstractDataFlowInterpreter(object): + def __init__(self, translation_context): + self.translation_context = translation_context + self.scheduled = {} # block: graph containing it + self.varstates = {} # var-or-const: state + self.creationpoints = {} # var: creationpoint + self.constant_cps = {} # const: creationpoint + self.dependencies = {} # creationpoint: {block: graph containing it} + self.functionargs = {} # graph: list of state of args + + def getstate(self, var_or_const): + if not isonheap(var_or_const): + return None + if var_or_const in self.varstates: + return self.varstates[var_or_const] + if isinstance(var_or_const, Variable): + varstate = VarState() + else: + if var_or_const not in self.constant_cps: + crep = CreationPoint("constant") + self.constant_cps[var_or_const] = crep + else: + crep = self.constant_cps[var_or_const] + varstate = VarState(crep) + self.varstates[var_or_const] = varstate + return varstate + + def getstates(self, varorconstlist): + return [self.getstate(var) for var in varorconstlist] + + def setstate(self, var, state): + self.varstates[var] = state + + def get_creationpoint(self, var, method="?"): + if var in self.creationpoints: + return self.creationpoints[var] + crep = CreationPoint(method) + self.creationpoints[var] = crep + return crep + + def schedule_function(self, graph): + print "scheduling function:", graph.name + startblock = graph.startblock + if graph in self.functionargs: + args = self.functionargs[graph] + else: + args = [] + for var in startblock.inputargs: + if not isonheap(var): + varstate = None + else: + crep = self.get_creationpoint(var, "arg") + varstate = VarState(crep) + self.setstate(var, varstate) + args.append(varstate) + self.scheduled[startblock] = graph + resultstate = self.getstate(graph.returnblock.inputargs[0]) + return resultstate, args + + def flow_block(self, block, graph): + print "flowing in block %s of function %s" % (block, graph.name) + if block is graph.returnblock: + if isonheap(block.inputargs[0]): + changed = self.getstate(block.inputargs[0]).setescapes() + self.handle_changed(changed) + return + if block is graph.exceptblock: + if isonheap(block.inputargs[0]): + changed = self.getstate(block.inputargs[0]).setescapes() + self.handle_changed(changed) + if isonheap(block.inputargs[0]): + changed = self.gestate(block.inputargs[1]).setescapes() + self.handle_changed(changed) + return + self.curr_block = block + self.curr_graph = graph + print "inputargs", self.getstates(block.inputargs) + for op in block.operations: + self.flow_operation(op) + print "checking exits..." + for exit in block.exits: + print "exit", exit + args = self.getstates(exit.args) + targetargs = self.getstates(exit.target.inputargs) + print " newargs", args + print " targetargs", targetargs + if multicontains(targetargs, args): + print " not necessary" + continue + else: + print " scheduling for flowin" + for prevstate, origstate, var in zip(args, targetargs, + exit.target.inputargs): + if not isonheap(var): + continue + newstate = prevstate.merge(origstate) + self.setstate(var, newstate) + print " args", self.getstates(exit.target.inputargs) + self.scheduled[exit.target] = graph + + def flow_operation(self, op): + print "handling", op + args = self.getstates(op.args) + print "args:", args + opimpl = getattr(self, op.opname, None) + if opimpl is None: + if isonheap(op.result) or filter(None, args): + raise NotImplementedError("can't handle %s" % (op.opname, )) + print "assuming that '%s' is irrelevant" % op + return + res = opimpl(op, *args) + self.setstate(op.result, res) + + def complete(self): + while self.scheduled: + block = self.scheduled.iterkeys().next() + graph = self.scheduled.pop(block) + self.flow_block(block, graph) + + def handle_changed(self, changed): + for crep in changed: + if crep not in self.dependencies: + continue + for block, graph in self.dependencies[crep].iteritems(): + self.scheduled[block] = graph + + def register_block_dependency(self, state, block=None, graph=None): + if block is None: + block = self.curr_block + graph = self.curr_graph + for crep in state.creation_points: + self.dependencies.setdefault(crep, {})[block] = graph + + def register_state_dependency(self, state1, state2): + "state1 depends on state2: if state2 does escape/change, so does state1" + escapes = state2.does_escape() + if escapes and not state1.does_escape(): + changed = state1.setescapes() + self.handle_changed(changed) + changes = state2.does_change() + if changes and not state1: + changed = state1.setchanges() + self.handle_changed(changed) + self.register_block_dependency(state2) + + # _____________________________________________________________________ + # operation implementations + + def malloc(self, op, type): + return VarState(self.get_creationpoint(op.result, "malloc")) + + def cast_pointer(self, op, state): + return state + + def setfield(self, op, objstate, fieldname, valuestate): + changed = objstate.setchanges() + if valuestate is not None: + # be pessimistic for now: + # everything that gets stored into a structure escapes and changes + self.handle_changed(changed) + changed = valuestate.setchanges() + self.handle_changed(changed) + changed = valuestate.setescapes() + self.handle_changed(changed) + return None + + def getfield(self, op, objstate, fieldname): + if isonheap(op.result): + # assume that getfield creates a new value + return VarState(self.get_creationpoint(op.result, "getfield")) + + def direct_call(self, op, function, *args): + graph = get_graph(op.args[0], self.translation_context) + result, funcargs = self.schedule_function(graph) + assert len(args) == len(funcargs) + for localarg, funcarg in zip(args, funcargs): + if localarg is None: + assert funcarg is None + continue + self.register_state_dependency(localarg, funcarg) + if isonheap(op.result): + # assume that a call creates a new value + return VarState(self.get_creationpoint(op.result, "direct_call")) + + def indirect_call(self, op, function, *args): + graphs = op.args[-1].value + args = args[:-1] + for graph in graphs: + result, funcargs = self.schedule_function(graph) + assert len(args) == len(funcargs) + for localarg, funcarg in zip(args, funcargs): + if localarg is None: + assert funcarg is None + continue + self.register_state_dependency(localarg, funcarg) + if isonheap(op.result): + # assume that a call creates a new value + return VarState(self.get_creationpoint(op.result, "indirect_call")) + + def ptr_iszero(self, op, ptr): + return None + +def isonheap(var_or_const): + return isinstance(var_or_const.concretetype, lltype.Ptr) + +def multicontains(l1, l2): + assert len(l1) == len(l2) + for a, b in zip(l1, l2): + if a is None: + assert b is None + elif not a.contains(b): + return False + return True Added: pypy/dist/pypy/translator/backendopt/test/test_escape.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/backendopt/test/test_escape.py Tue Jan 10 17:25:03 2006 @@ -0,0 +1,247 @@ +from pypy.translator.translator import TranslationContext, graphof +from pypy.translator.backendopt.escape import AbstractDataFlowInterpreter + +def build_adi(function, types): + t = TranslationContext() + t.buildannotator().build_types(function, types) + t.buildrtyper().specialize() + adi = AbstractDataFlowInterpreter(t) + graph = graphof(t, function) + adi.schedule_function(graph) + adi.complete() + return t, adi, graph + +def test_simple(): + class A(object): + pass + def f(): + a = A() + a.x = 1 + return a.x + t, adi, graph = build_adi(f, []) + avar = graph.startblock.operations[0].result + state = adi.getstate(avar) + assert len(state.creation_points) == 1 + crep = state.creation_points.keys()[0] + assert crep.changes + assert not crep.escapes + +def test_branch(): + class T: + pass + def fn2(x, y): + t = T() + t.x = x + t.y = y + if x > 0: + return t.x + t.y + else: + return t.x - t.y + t, adi, graph = build_adi(fn2, [int, int]) + tvar = graph.startblock.operations[0].result + state = adi.getstate(tvar) + assert len(state.creation_points) == 1 + crep = state.creation_points.keys()[0] + assert crep.changes + assert not crep.escapes + +def test_loop(): + class A(object): + pass + def f(): + a = A() + i = 0 + while i < 3: + a.x = i + a = A() + i += 1 + return a.x + t, adi, graph = build_adi(f, []) + avar = graph.startblock.operations[0].result + state = adi.getstate(avar) + assert len(state.creation_points) == 1 + crep = state.creation_points.keys()[0] + assert crep.changes + assert not crep.escapes + avarinloop = graph.startblock.exits[0].target.inputargs[1] + #t.view() + state1 = adi.getstate(avarinloop) + assert crep in state1.creation_points + assert len(state1.creation_points) == 2 + +def test_global(): + class A(object): + pass + globala = A() + def f(): + a = A() + a.next = None + globala.next = a + t, adi, graph = build_adi(f, []) + avar = graph.startblock.operations[0].result + state = adi.getstate(avar) + assert len(state.creation_points) == 1 + crep = state.creation_points.keys()[0] + assert crep.changes + assert crep.escapes + +def test_classattrs(): + class A: + attr = 666 + class B(A): + attr = 42 + def fn5(): + b = B() + return b.attr + t, adi, graph = build_adi(fn5, []) + bvar = graph.startblock.operations[0].result + state = adi.getstate(bvar) + assert len(state.creation_points) == 1 + crep = state.creation_points.keys()[0] + assert crep.changes + assert not crep.escapes + +def test_aliasing(): + class A: + pass + def fn6(n): + a1 = A() + a1.x = 5 + a2 = A() + a2.x = 6 + if n > 0: + a = a1 + else: + a = a2 + a.x = 12 + return a1.x + t, adi, graph = build_adi(fn6, [int]) + avar = graph.startblock.exits[0].target.inputargs[1] + state = adi.getstate(avar) + assert len(state.creation_points) == 2 + for crep in state.creation_points.keys(): + assert crep.changes + assert not crep.escapes + +def test_call(): + class A(object): + pass + globala = A() + def g(b): + return b.i + 2 + def f(): + a = A() + a.i = 2 + return g(a) + t, adi, graph = build_adi(f, []) + g_graph = graphof(t, g) + bvar = g_graph.startblock.inputargs[0] + bstate = adi.getstate(bvar) + assert len(bstate.creation_points) == 1 + bcrep = bstate.creation_points.keys()[0] + assert not bcrep.changes + assert not bcrep.escapes + avar = graph.startblock.operations[0].result + astate = adi.getstate(avar) + assert len(astate.creation_points) == 1 + acrep = astate.creation_points.keys()[0] + assert acrep.changes + assert not acrep.escapes + +def test_substruct(): + class A(object): + pass + class B(object): + pass + def g(a, b): + a.b = b + a.b.x = 1 + return a.b + def f(): + a0 = A() + b0 = B() + return g(a0, b0).x + t, adi, graph = build_adi(f, []) + g_graph = graphof(t, g) + a0var = graph.startblock.operations[0].result + b0var = graph.startblock.operations[3].result + a0state = adi.getstate(a0var) + b0state = adi.getstate(b0var) + assert len(a0state.creation_points) == 1 + a0crep = a0state.creation_points.keys()[0] + assert not a0crep.escapes + assert a0crep.changes + assert len(b0state.creation_points) == 1 + b0crep = b0state.creation_points.keys()[0] + assert b0crep.escapes + assert b0crep.changes + +def test_multiple_calls(): + class A(object): + pass + def h(a, b): + a.x = 1 + return b + def g(a, b): + return h(b, a) + def f(): + a1 = A() + a2 = A() + a3 = A() + a4 = h(a1, a2) + a5 = g(a3, a4) + t, adi, graph = build_adi(f, []) + a1var = graph.startblock.operations[0].result + a2var = graph.startblock.operations[3].result + a3var = graph.startblock.operations[6].result + a1state = adi.getstate(a1var) + a2state = adi.getstate(a2var) + a3state = adi.getstate(a3var) + assert len(a1state.creation_points) == 1 + assert len(a2state.creation_points) == 1 + assert len(a3state.creation_points) == 1 + a1crep = a1state.creation_points.keys()[0] + a2crep = a2state.creation_points.keys()[0] + a3crep = a3state.creation_points.keys()[0] + assert a1crep.changes and a2crep.changes and a3crep.changes + assert not a1crep.escapes and a2crep.escapes and a3crep.escapes + +def test_indirect_call(): + class A(object): + pass + def f1(a): + return a.x + def f2(a): + return a.x + 1 + def g1(a): + return a + def g2(a): + return None + def f(i): + a1 = A() + a2 = A() + a1.x = 1 + a2.x = 2 + if i: + f = f1 + g = g1 + else: + f = f2 + g = g2 + x = f(a1) + a0 = g(a2) + if a0 is not None: + return x + else: + return 42 + t, adi, graph = build_adi(f, [int]) + a1var = graph.startblock.operations[0].result + a2var = graph.startblock.operations[3].result + a1state = adi.getstate(a1var) + a2state = adi.getstate(a2var) + assert len(a1state.creation_points) == 1 + assert len(a2state.creation_points) == 1 + a1crep = a1state.creation_points.keys()[0] + a2crep = a2state.creation_points.keys()[0] + assert a1crep.changes and a2crep.changes + assert not a1crep.escapes and a2crep.escapes From arigo at codespeak.net Tue Jan 10 17:26:46 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 10 Jan 2006 17:26:46 +0100 (CET) Subject: [pypy-svn] r21894 - pypy/extradoc/talk/chalmers/image Message-ID: <20060110162646.9486527B55@code1.codespeak.net> Author: arigo Date: Tue Jan 10 17:26:45 2006 New Revision: 21894 Added: pypy/extradoc/talk/chalmers/image/compiler.png (contents, props changed) pypy/extradoc/talk/chalmers/image/interpreter.png (contents, props changed) Log: Checking in PNG versions. (I didn't try the ..graphviz directive in S5...) Added: pypy/extradoc/talk/chalmers/image/compiler.png ============================================================================== Binary file. No diff available. Added: pypy/extradoc/talk/chalmers/image/interpreter.png ============================================================================== Binary file. No diff available. From arigo at codespeak.net Tue Jan 10 17:29:18 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 10 Jan 2006 17:29:18 +0100 (CET) Subject: [pypy-svn] r21895 - pypy/extradoc/talk/chalmers/image Message-ID: <20060110162918.7754127B45@code1.codespeak.net> Author: arigo Date: Tue Jan 10 17:29:17 2006 New Revision: 21895 Modified: pypy/extradoc/talk/chalmers/image/compiler.dot pypy/extradoc/talk/chalmers/image/compiler.png pypy/extradoc/talk/chalmers/image/interpreter.dot pypy/extradoc/talk/chalmers/image/interpreter.png Log: Dotted lines are too hard to see. Modified: pypy/extradoc/talk/chalmers/image/compiler.dot ============================================================================== --- pypy/extradoc/talk/chalmers/image/compiler.dot (original) +++ pypy/extradoc/talk/chalmers/image/compiler.dot Tue Jan 10 17:29:17 2006 @@ -4,20 +4,20 @@ interpret_ [shape="box", label="def interpret(code, args):\l while more bytecodes:\l next_instr = ...\l if next_instr == \"add\":\l add(args)\l elif next_instr == \"sub\":\l sub(args)\l", fillcolor="green"]; interpret_ -> interpret__1 interpret__1 [shape="octagon", label="more bytecodes?", color="black", fillcolor="white", style="filled"]; -edge [label="No", style="dotted", color="black", dir="forward", weight="5"]; +edge [label="No", color="black", dir="forward", weight="5"]; interpret__1 -> interpret__2 -edge [label="Yes", style="dotted", color="black", dir="forward", weight="5"]; +edge [label="Yes", color="black", dir="forward", weight="5"]; interpret__1 -> interpret__3 interpret__2 [shape="box", label="return", color="black", fillcolor="green", style="filled"]; interpret__3 [shape="octagon", label="next_instr = ...\l\nnext_instr == 'add'?", color="black", fillcolor="white", style="filled"]; -edge [label="No", style="dotted", color="black", dir="forward", weight="5"]; +edge [label="No", color="black", dir="forward", weight="5"]; interpret__3 -> interpret__4 -edge [label="Yes", style="dotted", color="black", dir="forward", weight="5"]; +edge [label="Yes", color="black", dir="forward", weight="5"]; interpret__3 -> interpret__5 interpret__4 [shape="octagon", label="next_instr == 'sub' ?", color="black", fillcolor="white", style="filled"]; -edge [label="No", style="dotted", color="black", dir="forward", weight="5"]; +edge [label="No", color="black", dir="forward", weight="5"]; interpret__4 -> interpret__1 -edge [label="Yes", style="dotted", color="black", dir="forward", weight="5"]; +edge [label="Yes", color="black", dir="forward", weight="5"]; interpret__4 -> interpret__6 interpret__6 [shape="box", label="generate sub(args)", color="black", fillcolor="yellow", style="filled"]; edge [label="", style="solid", color="black", dir="forward", weight="5"]; Modified: pypy/extradoc/talk/chalmers/image/compiler.png ============================================================================== Binary files. No diff available. Modified: pypy/extradoc/talk/chalmers/image/interpreter.dot ============================================================================== --- pypy/extradoc/talk/chalmers/image/interpreter.dot (original) +++ pypy/extradoc/talk/chalmers/image/interpreter.dot Tue Jan 10 17:29:17 2006 @@ -4,20 +4,20 @@ interpret_ [shape="box", label="def interpret(code, args):\l while more bytecodes:\l next_instr = ...\l if next_instr == \"add\":\l add(args)\l elif next_instr == \"sub\":\l sub(args)\l", fillcolor="green"]; interpret_ -> interpret__1 interpret__1 [shape="octagon", label="more bytecodes?", color="black", fillcolor="white", style="filled"]; -edge [label="No", style="dotted", color="black", dir="forward", weight="5"]; +edge [label="No", color="black", dir="forward", weight="5"]; interpret__1 -> interpret__2 -edge [label="Yes", style="dotted", color="black", dir="forward", weight="5"]; +edge [label="Yes", color="black", dir="forward", weight="5"]; interpret__1 -> interpret__3 interpret__2 [shape="box", label="return", color="black", fillcolor="green", style="filled"]; interpret__3 [shape="octagon", label="next_instr = ...\l\nnext_instr == 'add'?", color="black", fillcolor="white", style="filled"]; -edge [label="No", style="dotted", color="black", dir="forward", weight="5"]; +edge [label="No", color="black", dir="forward", weight="5"]; interpret__3 -> interpret__4 -edge [label="Yes", style="dotted", color="black", dir="forward", weight="5"]; +edge [label="Yes", color="black", dir="forward", weight="5"]; interpret__3 -> interpret__5 interpret__4 [shape="octagon", label="next_instr == 'sub' ?", color="black", fillcolor="white", style="filled"]; -edge [label="No", style="dotted", color="black", dir="forward", weight="5"]; +edge [label="No", color="black", dir="forward", weight="5"]; interpret__4 -> interpret__1 -edge [label="Yes", style="dotted", color="black", dir="forward", weight="5"]; +edge [label="Yes", color="black", dir="forward", weight="5"]; interpret__4 -> interpret__6 interpret__6 [shape="box", label="sub(args)", color="black", fillcolor="white", style="filled"]; edge [label="", style="solid", color="black", dir="forward", weight="5"]; Modified: pypy/extradoc/talk/chalmers/image/interpreter.png ============================================================================== Binary files. No diff available. From arigo at codespeak.net Tue Jan 10 18:35:20 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 10 Jan 2006 18:35:20 +0100 (CET) Subject: [pypy-svn] r21896 - in pypy/extradoc/talk/chalmers: . image Message-ID: <20060110173520.3E94827B45@code1.codespeak.net> Author: arigo Date: Tue Jan 10 18:35:17 2006 New Revision: 21896 Added: pypy/extradoc/talk/chalmers/image/pipe-title.jpg (contents, props changed) pypy/extradoc/talk/chalmers/image/py-web1.png - copied unchanged from r21895, pypy/extradoc/talk/chalmers/py-web1.png Removed: pypy/extradoc/talk/chalmers/py-web1.png Modified: pypy/extradoc/talk/chalmers/talk-wm2006.txt Log: Added the conclusion. Minor tweaks. Added: pypy/extradoc/talk/chalmers/image/pipe-title.jpg ============================================================================== Binary file. No diff available. Modified: pypy/extradoc/talk/chalmers/talk-wm2006.txt ============================================================================== --- pypy/extradoc/talk/chalmers/talk-wm2006.txt (original) +++ pypy/extradoc/talk/chalmers/talk-wm2006.txt Tue Jan 10 18:35:17 2006 @@ -10,7 +10,7 @@ :Author: Armin Rigo :URL: http://codespeak.net/pypy -.. image:: py-web1.png +.. image:: image/py-web1.png :align: right @@ -53,7 +53,7 @@ Translation Overview ========================= -.. image:: translation-overview.png +.. image:: image/translation-overview.png :align: center Translation Aspects @@ -68,7 +68,7 @@ Translation Aspects (2) ========================= -- the execution stragegy can be tweaked at translation time too +- the execution strategy can be tweaked at translation time too - if the input RPython program is an Interpreter_... - ...the output low-level program can be a Compiler_ @@ -92,42 +92,52 @@ - translation aspect -:input: - PyPy Interpreter written in RPython +:input: PyPy Interpreter written in RPython +:normal output: an Interpreter -- like CPython +:new output: a Compiler -- like Psyco -:prev output: - an Interpreter -- like CPython -:new output: - a Compiler -- like Psyco - - -Recipe for a JIT +Recipe for a JIT (1) ========================== -- JIT part 1: high level, language dependent -- JIT part 2: platform-specific code generation and optimizations -- separation of concerns - -PyPy JIT part 1 -========================== +JIT Part 1: high level, language dependent - generated as translation aspect from PyPy Interpreter - stays in sync with the evolution of the language +- separation of concerns - same technique works for other languages -PyPy JIT part 2 -========================= +Recipe for a JIT (2) +========================== + +JIT Part 2: platform-specific code generation and optimizations - no cross-platform code generation toolkit - needs to be manually rewritten for each platform - orthogonal to the source language - can reuse existing code generation and optimization projects -Conclusion +Summary +========================= + +- don't port new languages to new platforms manually +- manually port general translation toolsuites +- start from high-level Language Definitions +- regenerate Interpreters automatically +- get Just-In-Time Compilers for free :-) + +For More Information ========================= -XXX +- http://codespeak.net/pypy + + .. image:: image/py-web1.png + :align: center + +- http://psyco.sf.net + + .. image:: image/pipe-title.jpg + :align: center .. _Psyco: http://psyco.sourceforge.net From arigo at codespeak.net Tue Jan 10 19:20:32 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 10 Jan 2006 19:20:32 +0100 (CET) Subject: [pypy-svn] r21899 - in pypy/extradoc/talk/chalmers: . image Message-ID: <20060110182032.05EB927B58@code1.codespeak.net> Author: arigo Date: Tue Jan 10 19:20:29 2006 New Revision: 21899 Added: pypy/extradoc/talk/chalmers/image/stackless_informal.dot - copied, changed from r21748, pypy/dist/pypy/doc/image/stackless_informal.dot Modified: pypy/extradoc/talk/chalmers/talk-wm2006.txt Log: Simplify stackless_informal.dot more Modified: pypy/extradoc/talk/chalmers/talk-wm2006.txt ============================================================================== --- pypy/extradoc/talk/chalmers/talk-wm2006.txt (original) +++ pypy/extradoc/talk/chalmers/talk-wm2006.txt Tue Jan 10 19:20:29 2006 @@ -126,7 +126,7 @@ - regenerate Interpreters automatically - get Just-In-Time Compilers for free :-) -For More Information +Links ========================= - http://codespeak.net/pypy From arigo at codespeak.net Tue Jan 10 19:20:52 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 10 Jan 2006 19:20:52 +0100 (CET) Subject: [pypy-svn] r21900 - pypy/extradoc/talk/chalmers/image Message-ID: <20060110182052.913CA27B58@code1.codespeak.net> Author: arigo Date: Tue Jan 10 19:20:51 2006 New Revision: 21900 Modified: pypy/extradoc/talk/chalmers/image/stackless_informal.png Log: Regenerated. Modified: pypy/extradoc/talk/chalmers/image/stackless_informal.png ============================================================================== Binary files. No diff available. From ac at codespeak.net Wed Jan 11 13:23:36 2006 From: ac at codespeak.net (ac at codespeak.net) Date: Wed, 11 Jan 2006 13:23:36 +0100 (CET) Subject: [pypy-svn] r21924 - pypy/branch/arre-experiments Message-ID: <20060111122336.5464C27B46@code1.codespeak.net> Author: ac Date: Wed Jan 11 13:23:36 2006 New Revision: 21924 Added: pypy/branch/arre-experiments/ - copied from r21923, pypy/dist/ Log: Create a branch to do experiments without disturing others. From ac at codespeak.net Wed Jan 11 13:35:07 2006 From: ac at codespeak.net (ac at codespeak.net) Date: Wed, 11 Jan 2006 13:35:07 +0100 (CET) Subject: [pypy-svn] r21927 - pypy/branch/arre-experiments/pypy/interpreter Message-ID: <20060111123507.24F4927B58@code1.codespeak.net> Author: ac Date: Wed Jan 11 13:35:06 2006 New Revision: 21927 Modified: pypy/branch/arre-experiments/pypy/interpreter/argument.py Log: Have Arguments.prepend() not copy data from self. Modified: pypy/branch/arre-experiments/pypy/interpreter/argument.py ============================================================================== --- pypy/branch/arre-experiments/pypy/interpreter/argument.py (original) +++ pypy/branch/arre-experiments/pypy/interpreter/argument.py Wed Jan 11 13:35:06 2006 @@ -4,8 +4,104 @@ from pypy.interpreter.error import OperationError +class AbstractArguments: + def frompacked(space, w_args=None, w_kwds=None): + """Convenience static method to build an Arguments + from a wrapped sequence and a wrapped dictionary.""" + return Arguments(space, [], w_stararg=w_args, w_starstararg=w_kwds) + frompacked = staticmethod(frompacked) + + def fromshape(space, (shape_cnt,shape_keys,shape_star,shape_stst), data_w): + args_w = data_w[:shape_cnt] + p = shape_cnt + kwds_w = {} + for i in range(len(shape_keys)): + kwds_w[shape_keys[i]] = data_w[p] + p += 1 + if shape_star: + w_star = data_w[p] + p += 1 + else: + w_star = None + if shape_stst: + w_starstar = data_w[p] + p += 1 + else: + w_starstar = None + return Arguments(space, args_w, kwds_w, w_star, w_starstar) + fromshape = staticmethod(fromshape) + + def prepend(self, w_firstarg): + "Return a new Arguments with a new argument inserted first." + return ArgumentsPrepended(self, w_firstarg) + + +class ArgumentsPrepended(AbstractArguments): + def __init__(self, args, w_firstarg): + self.args = args + self.w_firstarg = w_firstarg + + def firstarg(self): + "Return the first argument for inspection." + return self.w_firstarg + + def __repr__(self): + return 'ArgumentsPrepended(%r, %r)' % (self.args, self.w_firstarg) + + def has_keywords(self): + return self.args.has_keywords() + + def unpack(self): + arguments_w, kwds_w = self.args.unpack() + return ([self.w_firstarg] + arguments_w), kwds_w + + def fixedunpack(self, argcount): + if argcount <= 0: + raise ValueError, "too many arguments (%d expected)" % argcount # XXX: Incorrect + return [self.w_firstarg] + self.args.fixedunpack(argcount - 1) + + def _rawshape(self, nextra=0): + return self.args._rawshape(nextra + 1) + + def _unpack_stararg(self): + self.args._unpack_stararg() + + def parse(self, fnname, signature, defaults_w=[]): + """Parse args and kwargs to initialize a frame + according to the signature of code object. + """ + self._unpack_stararg() + try: + return self.match_signature(signature, defaults_w) + except ArgErr, e: + raise OperationError(self.args.space.w_TypeError, + self.args.space.wrap(e.getmsg(self, fnname))) -class Arguments: + def match_signature(self, signature, defaults_w=[]): + """Parse args and kwargs according to the signature of a code object, + or raise an ArgErr in case of failure. + """ + argnames, varargname, kwargname = signature + scope_w = self.args.match_signature((argnames[1:], varargname, kwargname), defaults_w) + if len(argnames) == 0: + if varargname is None: + raise ArgErrCount(signature, defaults_w, 0) + space = self.args.space + if kwargname is not None: + scope_w[-2] = space.newtuple([self.w_firstarg] + space.unpackiterable(scope_w[-2])) + else: + scope_w[-1] = space.newtuple([self.w_firstarg] + space.unpackiterable(scope_w[-1])) + else: + scope_w.insert(0, self.w_firstarg) + return scope_w + + def flatten(self): + (shape_cnt, shape_keys, shape_star, shape_stst), data_w = self.args.flatten() + data_w.insert(0, self.w_firstarg) + return (shape_cnt + 1, shape_keys, shape_star, shape_stst), data_w + + +class Arguments(AbstractArguments): """ Collects the arguments of a function call. @@ -24,12 +120,6 @@ self.w_stararg = w_stararg self.w_starstararg = w_starstararg - def frompacked(space, w_args=None, w_kwds=None): - """Convenience static method to build an Arguments - from a wrapped sequence and a wrapped dictionary.""" - return Arguments(space, [], w_stararg=w_args, w_starstararg=w_kwds) - frompacked = staticmethod(frompacked) - def __repr__(self): if self.w_starstararg is not None: return 'Arguments(%s, %s, %s, %s)' % (self.arguments_w, @@ -84,13 +174,6 @@ self.w_starstararg = None return self.arguments_w, self.kwds_w - def prepend(self, w_firstarg): - "Return a new Arguments with a new argument inserted first." - args = Arguments(self.space, [w_firstarg] + self.arguments_w, - self.kwds_w, self.w_stararg, self.w_starstararg) - args.blind_arguments = self.blind_arguments + 1 - return args - def has_keywords(self): return bool(self.kwds_w) or (self.w_starstararg is not None and self.space.is_true(self.w_starstararg)) @@ -126,10 +209,7 @@ ### Parsing for function calls ### - def parse(self, fnname, signature, defaults_w=[]): - """Parse args and kwargs to initialize a frame - according to the signature of code object. - """ + def _unpack_stararg(self): space = self.space # If w_stararg is not exactly a tuple, unpack it now: # self.match_signature() assumes that it can use it directly for @@ -137,11 +217,17 @@ if self.w_stararg is not None: if not space.is_w(space.type(self.w_stararg), space.w_tuple): self.unpack() + + def parse(self, fnname, signature, defaults_w=[]): + """Parse args and kwargs to initialize a frame + according to the signature of code object. + """ + self._unpack_stararg() try: return self.match_signature(signature, defaults_w) except ArgErr, e: - raise OperationError(space.w_TypeError, - space.wrap(e.getmsg(self, fnname))) + raise OperationError(self.space.w_TypeError, + self.space.wrap(e.getmsg(self, fnname))) def match_signature(self, signature, defaults_w=[]): """Parse args and kwargs according to the signature of a code object, @@ -253,26 +339,6 @@ data_w.append(self.w_starstararg) return (shape_cnt, shape_keys, shape_star, shape_stst), data_w - def fromshape(space, (shape_cnt,shape_keys,shape_star,shape_stst), data_w): - args_w = data_w[:shape_cnt] - p = shape_cnt - kwds_w = {} - for i in range(len(shape_keys)): - kwds_w[shape_keys[i]] = data_w[p] - p += 1 - if shape_star: - w_star = data_w[p] - p += 1 - else: - w_star = None - if shape_stst: - w_starstar = data_w[p] - p += 1 - else: - w_starstar = None - return Arguments(space, args_w, kwds_w, w_star, w_starstar) - fromshape = staticmethod(fromshape) - def rawshape(args, nextra=0): return args._rawshape(nextra) From bea at codespeak.net Wed Jan 11 14:47:24 2006 From: bea at codespeak.net (bea at codespeak.net) Date: Wed, 11 Jan 2006 14:47:24 +0100 (CET) Subject: [pypy-svn] r21934 - pypy/extradoc/sprintinfo/mallorca Message-ID: <20060111134724.89E7027B55@code1.codespeak.net> Author: bea Date: Wed Jan 11 14:47:23 2006 New Revision: 21934 Modified: pypy/extradoc/sprintinfo/mallorca/sprint-announcement.txt Log: updated announcement with description on how to get to Gines place. Also "deleted" Sten - it seems like he and Simone will not join us there.....;-( Modified: pypy/extradoc/sprintinfo/mallorca/sprint-announcement.txt ============================================================================== --- pypy/extradoc/sprintinfo/mallorca/sprint-announcement.txt (original) +++ pypy/extradoc/sprintinfo/mallorca/sprint-announcement.txt Wed Jan 11 14:47:23 2006 @@ -116,9 +116,15 @@ http://www.hermetex4.com/damnans/ +Here is a description on how to get to Gin?s studio: + +http://www.maporama.com/affiliates/popup/share/iti.asp?ItineraryID=F32E624F +-BFB1-4751-A5D3-5EA4DBDC9C9B&language=en&NbStep=0&SESSIONID={CD058830-6B8B-4 +142-887F-94C01253E33C} + For those interested in playing collectable card games, this will also be an opportunity to get aquainted with V:TES which will be demoed by -Gin?s and Beatrice and Sten D?ring. For more information on this +Gin?s and Beatrice D?ring. For more information on this cardgame - see: http://www.white-wolf.com/vtes/index.php. (The Mallorca sprint was organized through contacts within the V:TES community). From mwh at codespeak.net Wed Jan 11 14:54:22 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Wed, 11 Jan 2006 14:54:22 +0100 (CET) Subject: [pypy-svn] r21936 - pypy/extradoc/sprintinfo/mallorca Message-ID: <20060111135422.8F14227B58@code1.codespeak.net> Author: mwh Date: Wed Jan 11 14:54:20 2006 New Revision: 21936 Modified: pypy/extradoc/sprintinfo/mallorca/people.txt Log: my dates and armins and my details. Modified: pypy/extradoc/sprintinfo/mallorca/people.txt ============================================================================== --- pypy/extradoc/sprintinfo/mallorca/people.txt (original) +++ pypy/extradoc/sprintinfo/mallorca/people.txt Wed Jan 11 14:54:20 2006 @@ -10,10 +10,10 @@ Name Arrive/Depart Accomodation ==================== ============== ===================== Beatrice Duering 22/1 -29/1 flat? -Armin Rigo 21/1 - 30?/1 ? +Armin Rigo 21/1 - 30?/1 Hostal Terminus +Michael Hudson 22/1 - 30/1 Hostal Terminus Samuele Pedroni 22/1 - 30/1 Hotel Colon Eric van Riet Paap 22/1 - 28/1 casa M. -Michael Hudson ? ? Carl Friedrich Bolz ? ? Anders Chrigstroem 22/1 - 30/1 Hotel Colon Christian Tismer 22/1 - 30/1 ? From mwh at codespeak.net Wed Jan 11 15:37:21 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Wed, 11 Jan 2006 15:37:21 +0100 (CET) Subject: [pypy-svn] r21943 - pypy/dist/pypy/rpython/l3interp Message-ID: <20060111143721.E425927B52@code1.codespeak.net> Author: mwh Date: Wed Jan 11 15:37:20 2006 New Revision: 21943 Modified: pypy/dist/pypy/rpython/l3interp/l3interp.py Log: actually install missing handlers! Modified: pypy/dist/pypy/rpython/l3interp/l3interp.py ============================================================================== --- pypy/dist/pypy/rpython/l3interp/l3interp.py (original) +++ pypy/dist/pypy/rpython/l3interp/l3interp.py Wed Jan 11 15:37:20 2006 @@ -87,8 +87,8 @@ "NOT_RPYTHON" def make_missing_handler(opname): def missing_handler(self): - print 'XXX missing handler for operation', opname - raise NotImplementedError + raise NotImplementedError('XXX missing handler for %r'%opname) + return missing_handler cls.dispatch_table = [] for opname in model.very_low_level_ops: try: From ale at codespeak.net Wed Jan 11 17:04:34 2006 From: ale at codespeak.net (ale at codespeak.net) Date: Wed, 11 Jan 2006 17:04:34 +0100 (CET) Subject: [pypy-svn] r21958 - pypy/dist/pypy/lib/pyontology/test Message-ID: <20060111160434.2A9A627B46@code1.codespeak.net> Author: ale Date: Wed Jan 11 17:04:33 2006 New Revision: 21958 Modified: pypy/dist/pypy/lib/pyontology/test/test_ontology.py Log: refactored creation of rdf lists into a function -> nicer test Modified: pypy/dist/pypy/lib/pyontology/test/test_ontology.py ============================================================================== --- pypy/dist/pypy/lib/pyontology/test/test_ontology.py (original) +++ pypy/dist/pypy/lib/pyontology/test/test_ontology.py Wed Jan 11 17:04:33 2006 @@ -11,6 +11,21 @@ from pypy.lib.pyontology.pyontology import * # Ontology, ClassDomain, SubClassConstraint from rdflib import Graph, URIRef, BNode +def rdf_list(ont, name, data): + owllist = URIRef(name) + obj = URIRef(namespaces['rdf']+'#List') + ont.type(owllist, obj) + own =owllist + for i,dat in enumerate(data[:-1]): + next = URIRef( 'anon'+str(i)) + print next,i,dat,own + ont.first(own, URIRef(dat)) + ont.rest(own, next) + own = next + ont.first(own, URIRef(data[-1])) + ont.rest(own, URIRef(namespaces['rdf']+'#nil')) + return owllist + def test_makevar(): O = Ontology() var = URIRef(u'http://www.w3.org/2002/03owlt/unionOf/premises004#A-and-B') @@ -350,15 +365,7 @@ O.type(p, obj) p = URIRef('favlist') O.oneOf(restrict, p) - own = URIRef('favlist') - obj = URIRef(namespaces['rdf']+'#List') - O.type(own, obj) - O.first(own, 0) - O.rest(own, URIRef('1')) - O.first( URIRef('1'), 1) - O.rest( URIRef('1'), URIRef('2')) - O.first( URIRef('2'), 2) - O.rest( URIRef('2'), URIRef(namespaces['rdf']+'#nil')) + own = rdf_list(O, 'favlist', [0, 1, 2]) own = URIRef('class') obj = URIRef(namespaces['rdf']+'#Class') O.type(own, obj) @@ -368,49 +375,27 @@ def test_oneofclassenumeration(): O = Ontology() restrict = BNode('anon') - own = URIRef('favlist') - obj = URIRef(namespaces['rdf']+'#List') - O.type(own, obj) - O.first(own, URIRef('first')) - O.rest(own, URIRef('1')) - O.first( URIRef('1'), URIRef('second')) - O.rest( URIRef('1'), URIRef('2')) - O.first( URIRef('2'), URIRef('third')) - O.rest( URIRef('2'), URIRef(namespaces['rdf']+'#nil')) + own = rdf_list(O, 'favlist', ['first', 'second', 'third']) O.oneOf(restrict, own) O.type(restrict, namespaces['owl']+'#Class') O.consistency(4) + print O.rep._domains assert len(O.rep._domains[restrict].getValues()) == 3 def test_oneofdatarange(): O = Ontology() restrict = BNode('anon') - own = URIRef('favlist') - obj = URIRef(namespaces['rdf']+'#List') - O.type(own, obj) - O.first(own, URIRef('first')) - O.rest(own, URIRef('1')) - O.first( URIRef('1'), URIRef('second')) - O.rest( URIRef('1'), URIRef('2')) - O.first( URIRef('2'), URIRef('third')) - O.rest( URIRef('2'), URIRef(namespaces['rdf']+'#nil')) + own = rdf_list(O, 'favlist', ['1','2','3']) O.oneOf(restrict, own) O.type(restrict, namespaces['owl']+'#DataRange') O.consistency(4) + print O.rep._domains assert len(O.rep._domains[restrict].getValues()) == 3 def test_somevaluesfrom_datarange(): O = Ontology() datarange = BNode('anon') - own = URIRef('favlist') - obj = URIRef(namespaces['rdf']+'#List') - O.type(own, obj) - O.first(own, URIRef('first')) - O.rest(own, URIRef('1')) - O.first( URIRef('1'), URIRef('second')) - O.rest( URIRef('1'), URIRef('2')) - O.first( URIRef('2'), URIRef('third')) - O.rest( URIRef('2'), URIRef(namespaces['rdf']+'#nil')) + own = rdf_list(O, 'favlist', ['1','2','3']) O.oneOf(datarange, own) O.type(datarange, namespaces['owl']+'#DataRange') restrict = BNode('anon1') @@ -431,6 +416,27 @@ def test_allvaluesfrom_datarange(): O = Ontology() datarange = BNode('anon') + own = rdf_list(O, 'favlist', ['1','2','3']) + O.oneOf(datarange, own) + O.type(datarange, namespaces['owl']+'#DataRange') + restrict = BNode('anon1') + obj = URIRef(namespaces['owl']+'#Restriction') + O.type(restrict, obj) + p = URIRef('p') + O.onProperty(restrict,p) + obj = URIRef(namespaces['owl']+'#ObjectProperty') + O.type(p, obj) + O.allValuesFrom(restrict, datarange) + cls = URIRef('class') + obj = URIRef(namespaces['owl']+'#Class') + O.type(cls, obj) + O.subClassOf(cls,restrict) + O.variables['p_'].setValues([(cls,1)]) + py.test.raises(ConsistencyFailure, O.consistency) + +def no_test_unionof(): + O = Ontology() + datarange = BNode('anon') own = URIRef('favlist') obj = URIRef(namespaces['rdf']+'#List') O.type(own, obj) @@ -440,7 +446,7 @@ O.rest( URIRef('1'), URIRef('2')) O.first( URIRef('2'), URIRef('third')) O.rest( URIRef('2'), URIRef(namespaces['rdf']+'#nil')) - O.oneOf(datarange, own) + O.unionOf(datarange, own) O.type(datarange, namespaces['owl']+'#DataRange') restrict = BNode('anon1') obj = URIRef(namespaces['owl']+'#Restriction') From ale at codespeak.net Wed Jan 11 18:22:49 2006 From: ale at codespeak.net (ale at codespeak.net) Date: Wed, 11 Jan 2006 18:22:49 +0100 (CET) Subject: [pypy-svn] r21964 - in pypy/dist/pypy/lib/pyontology: . test Message-ID: <20060111172249.49E8527B3F@code1.codespeak.net> Author: ale Date: Wed Jan 11 18:22:48 2006 New Revision: 21964 Modified: pypy/dist/pypy/lib/pyontology/pyontology.py pypy/dist/pypy/lib/pyontology/test/test_ontology.py Log: Added tests for unionof/intersectionof. Refactored the subclassing of rdflib.Graph removed get_list Modified: pypy/dist/pypy/lib/pyontology/pyontology.py ============================================================================== --- pypy/dist/pypy/lib/pyontology/pyontology.py (original) +++ pypy/dist/pypy/lib/pyontology/pyontology.py Wed Jan 11 18:22:48 2006 @@ -189,15 +189,18 @@ getUriref('rdf', 'List') : List } -class Ontology(Graph): +class Ontology: def __init__(self): - Graph.__init__(self) + self.graph = Graph() self.variables = {} self.constraints = [] self.seen = {} self.var2ns ={} + def add(self, triples): + self.graph.add(triples) + def add_file(self, f): tmp = Graph() tmp.load(f) @@ -205,7 +208,7 @@ self.add(i) def attach_fd(self): - for (s, p, o) in (self.triples((None, None, None))): + for (s, p, o) in (self.graph.triples((None,)*3)): if p.find('#') != -1: ns, func = p.split('#') else: @@ -238,21 +241,6 @@ self.rep = Repository(self.variables.keys(), self.variables, self.constraints) self.rep.consistency(verbose) - def get_list(self, subject): - res = [] - first = list(self.objects(subject, rdf_first)) - assert len(first) == 1 - if type(first[0]) == URIRef: - var = self.make_var(ClassDomain, first[0]) - res += first - - rest = list(self.objects(subject, rdf_rest)) - if "#nil" in rest[0] : - return res - else: - res += self.get_list(rest[0]) - return res - def make_var(self, cls=fd, a=''): if type(a) == URIRef: if a.find('#') != -1: @@ -349,17 +337,16 @@ self.constraints.append(cons) def unionOf(self,s, var): - res = self.get_list(var) - return res #There might be doubles (but fd takes care of that) + avar = self.make_var(List, var) + svar = self.make_var(ClassDomain, s) + cons = UnionofConstraint(svar, avar) + self.constraints.append(cons) def intersectionOf(self, s, var): - res = self.get_list(var) - result = {}.fromkeys(res[0]) - for el in res: - for cls in result.keys(): - if cls not in el: - result.pop(cls) - return result.keys() + avar = self.make_var(List, var) + svar = self.make_var(ClassDomain, s) + cons = IntersectionofConstraint(svar, avar) + self.constraints.append(cons) #---Property Axioms---#000000#FFFFFF-------------------------------------------- @@ -424,10 +411,11 @@ constrain = BinaryExpression([s_var, var_var],"%s != %s" %(s_var, var_var)) self.constraints.append(constrain) - def distinctMembers(self, s, var): - res = self.get_list(var) - self.constraints.append(AllDistinct([self.make_var(ClassDomain, y) for y in res])) - return res +#XXX need to change this +## def distinctMembers(self, s, var): +## res = self.get_list(var) +## self.constraints.append(AllDistinct([self.make_var(ClassDomain, y) for y in res])) +## return res def sameAs(self, s, var): s_var = self.make_var(None, s) @@ -780,6 +768,49 @@ domains[self.variable].setValues(val) return 1 +class UnionofConstraint(AbstractConstraint): + + def __init__(self, variable, List): + AbstractConstraint.__init__(self, [variable, List]) + self.variable = variable + self.List = List + + def estimateCost(self, domains): + return 200 + + def narrow(self, domains): + val = domains[self.List].getValues() + union = [] + for v in val: + for u in domains[v].getValues(): + if not u in union: + union.append(u) + cls = domains[self.variable].setValues(union) + +class IntersectionofConstraint(AbstractConstraint): + + def __init__(self, variable, List): + AbstractConstraint.__init__(self, [variable, List]) + self.variable = variable + self.List = List + + def estimateCost(self, domains): + return 200 + + def narrow(self, domains): + val = domains[self.List].getValues() + intersection = domains[val[0]].getValues() + for v in val[1:]: + vals= domains[v].getValues() + remove = [] + for u in intersection: + if not u in vals: + remove.append(u) + for u in remove: + intersection.remove(u) + cls = domains[self.variable].setValues(intersection) + + class SomeValueConstraint(AbstractConstraint): def __init__(self, variable, List): Modified: pypy/dist/pypy/lib/pyontology/test/test_ontology.py ============================================================================== --- pypy/dist/pypy/lib/pyontology/test/test_ontology.py (original) +++ pypy/dist/pypy/lib/pyontology/test/test_ontology.py Wed Jan 11 18:22:48 2006 @@ -17,12 +17,12 @@ ont.type(owllist, obj) own =owllist for i,dat in enumerate(data[:-1]): - next = URIRef( 'anon'+str(i)) + next = URIRef( name + str(i)) print next,i,dat,own - ont.first(own, URIRef(dat)) + ont.first(own, dat) ont.rest(own, next) own = next - ont.first(own, URIRef(data[-1])) + ont.first(own, data[-1]) ont.rest(own, URIRef(namespaces['rdf']+'#nil')) return owllist @@ -434,31 +434,32 @@ O.variables['p_'].setValues([(cls,1)]) py.test.raises(ConsistencyFailure, O.consistency) -def no_test_unionof(): +def test_unionof(): O = Ontology() - datarange = BNode('anon') - own = URIRef('favlist') - obj = URIRef(namespaces['rdf']+'#List') - O.type(own, obj) - O.first(own, URIRef('first')) - O.rest(own, URIRef('1')) - O.first( URIRef('1'), URIRef('second')) - O.rest( URIRef('1'), URIRef('2')) - O.first( URIRef('2'), URIRef('third')) - O.rest( URIRef('2'), URIRef(namespaces['rdf']+'#nil')) - O.unionOf(datarange, own) - O.type(datarange, namespaces['owl']+'#DataRange') - restrict = BNode('anon1') - obj = URIRef(namespaces['owl']+'#Restriction') - O.type(restrict, obj) - p = URIRef('p') - O.onProperty(restrict,p) - obj = URIRef(namespaces['owl']+'#ObjectProperty') - O.type(p, obj) - O.allValuesFrom(restrict, datarange) - cls = URIRef('class') - obj = URIRef(namespaces['owl']+'#Class') - O.type(cls, obj) - O.subClassOf(cls,restrict) - O.variables['p_'].setValues([(cls,1)]) - py.test.raises(ConsistencyFailure, O.consistency) + cls = BNode('anon') + own1 = BNode('liist1') + own2 = BNode('liist2') + list1 = rdf_list(O, 'favlist1', ['1', '2', '3']) + list2 = rdf_list(O, 'favlist2', ['3', '4', '5']) + own = rdf_list(O, 'favlist', [own1, own2]) + O.oneOf( own1, list1) + O.oneOf( own2, list2) + O.unionOf(cls, own) + O.type(cls, namespaces['owl']+'#Class') + O.consistency(3) + assert O.rep._domains[cls].getValues() == ['1', '2', '3', '4', '5'] + +def test_intersectionof(): + O = Ontology() + cls = BNode('anon') + own1 = BNode('liist1') + own2 = BNode('liist2') + list1 = rdf_list(O, 'favlist1', ['1', '2', '3']) + list2 = rdf_list(O, 'favlist2', ['3', '4', '5']) + own = rdf_list(O, 'favlist', [own1, own2]) + O.oneOf( own1, list1) + O.oneOf( own2, list2) + O.intersectionOf(cls, own) + O.type(cls, namespaces['owl']+'#Class') + O.consistency(3) + assert O.rep._domains[cls].getValues() == ['3'] From mwh at codespeak.net Wed Jan 11 18:43:12 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Wed, 11 Jan 2006 18:43:12 +0100 (CET) Subject: [pypy-svn] r21966 - in pypy/dist/pypy/rpython/l3interp: . test Message-ID: <20060111174312.B945F27B3F@code1.codespeak.net> Author: mwh Date: Wed Jan 11 18:43:11 2006 New Revision: 21966 Modified: pypy/dist/pypy/rpython/l3interp/l3interp.py pypy/dist/pypy/rpython/l3interp/test/test_convert.py Log: some more convertgraph tests and one more operation. Modified: pypy/dist/pypy/rpython/l3interp/l3interp.py ============================================================================== --- pypy/dist/pypy/rpython/l3interp/l3interp.py (original) +++ pypy/dist/pypy/rpython/l3interp/l3interp.py Wed Jan 11 18:43:11 2006 @@ -157,6 +157,13 @@ y = self.getint() self.stack_int.append(x + y) + def op_int_is_true(self): + x = self.getint() + if x: + self.stack_int.append(1) + else: + self.stack_int.append(0) + def op_direct_call(self): block = self.block assert block.called_graphs is not None Modified: pypy/dist/pypy/rpython/l3interp/test/test_convert.py ============================================================================== --- pypy/dist/pypy/rpython/l3interp/test/test_convert.py (original) +++ pypy/dist/pypy/rpython/l3interp/test/test_convert.py Wed Jan 11 18:43:11 2006 @@ -2,14 +2,42 @@ from pypy.rpython.l3interp import convertgraph, l3interp from pypy.translator.translator import TranslationContext -def test_convert_add(): - def f(x): - return x + 4 +def l3ify(f, inputargs): t = TranslationContext() - t.buildannotator().build_types(f, [int]) + t.buildannotator().build_types(f, inputargs) t.buildrtyper().specialize() conv = convertgraph.LL2L3Converter() - l3graph = conv.convert_graph(t.graphs[0]) + return conv.convert_graph(t.graphs[0]) + + +def test_convert_add(): + def f(x): + return x + 4 + l3graph = l3ify(f, [int]) result = l3interp.l3interpret(l3graph, [42], [], []) assert isinstance(result, l3interp.L3Integer) assert result.intval == 46 + +def test_convert_simple(): + def f(): + return 3 + 4 + l3graph = l3ify(f, []) + result = l3interp.l3interpret(l3graph, [], [], []) + assert isinstance(result, l3interp.L3Integer) + assert result.intval == 7 + +def test_convert_branch(): + def f(x): + if x: + return x + return 1 + l3graph = l3ify(f, [int]) + result = l3interp.l3interpret(l3graph, [2], [], []) + assert isinstance(result, l3interp.L3Integer) + assert result.intval == 2 + + result = l3interp.l3interpret(l3graph, [0], [], []) + assert isinstance(result, l3interp.L3Integer) + assert result.intval == 1 + + From cfbolz at codespeak.net Thu Jan 12 00:04:03 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Thu, 12 Jan 2006 00:04:03 +0100 (CET) Subject: [pypy-svn] r22000 - in pypy/dist/pypy/translator/backendopt: . test Message-ID: <20060111230403.5B78E27B42@code1.codespeak.net> Author: cfbolz Date: Thu Jan 12 00:04:01 2006 New Revision: 22000 Modified: pypy/dist/pypy/translator/backendopt/escape.py pypy/dist/pypy/translator/backendopt/test/test_escape.py Log: some small fixes to the adi. is now able to treat the whole rpystone - not sure whether it produces useful info. Modified: pypy/dist/pypy/translator/backendopt/escape.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/escape.py (original) +++ pypy/dist/pypy/translator/backendopt/escape.py Thu Jan 12 00:04:01 2006 @@ -77,6 +77,7 @@ self.constant_cps = {} # const: creationpoint self.dependencies = {} # creationpoint: {block: graph containing it} self.functionargs = {} # graph: list of state of args + self.flown_blocks = {} # block: True def getstate(self, var_or_const): if not isonheap(var_or_const): @@ -129,6 +130,7 @@ def flow_block(self, block, graph): print "flowing in block %s of function %s" % (block, graph.name) + self.flown_blocks[block] = True if block is graph.returnblock: if isonheap(block.inputargs[0]): changed = self.getstate(block.inputargs[0]).setescapes() @@ -139,7 +141,7 @@ changed = self.getstate(block.inputargs[0]).setescapes() self.handle_changed(changed) if isonheap(block.inputargs[0]): - changed = self.gestate(block.inputargs[1]).setescapes() + changed = self.getstate(block.inputargs[1]).setescapes() self.handle_changed(changed) return self.curr_block = block @@ -154,7 +156,9 @@ targetargs = self.getstates(exit.target.inputargs) print " newargs", args print " targetargs", targetargs - if multicontains(targetargs, args): + # flow every block at least once: + if (multicontains(targetargs, args) and + exit.target in self.flown_blocks): print " not necessary" continue else: @@ -203,6 +207,7 @@ def register_state_dependency(self, state1, state2): "state1 depends on state2: if state2 does escape/change, so does state1" + # change state1 according to how state2 is now escapes = state2.does_escape() if escapes and not state1.does_escape(): changed = state1.setescapes() @@ -211,19 +216,26 @@ if changes and not state1: changed = state1.setchanges() self.handle_changed(changed) + # register a dependency of the current block on state2: + # that means that if state2 changes the current block will be reflown + # triggering this function again and thus updating state1 self.register_block_dependency(state2) # _____________________________________________________________________ # operation implementations - def malloc(self, op, type): + def malloc(self, op, typestate): return VarState(self.get_creationpoint(op.result, "malloc")) + def malloc_varsize(self, op, typestate, lengthstate): + return VarState(self.get_creationpoint(op.result, "malloc_varsize")) + def cast_pointer(self, op, state): return state def setfield(self, op, objstate, fieldname, valuestate): changed = objstate.setchanges() + self.handle_changed(changed) if valuestate is not None: # be pessimistic for now: # everything that gets stored into a structure escapes and changes @@ -233,15 +245,46 @@ changed = valuestate.setescapes() self.handle_changed(changed) return None + + def setarrayitem(self, op, objstate, indexstate, valuestate): + changed = objstate.setchanges() + self.handle_changed(changed) + if valuestate is not None: + # everything that gets stored into a structure escapes and changes + self.handle_changed(changed) + changed = valuestate.setchanges() + self.handle_changed(changed) + changed = valuestate.setescapes() + self.handle_changed(changed) + return None + + def getarrayitem(self, op, objstate, indexstate): + if isonheap(op.result): + return VarState(self.get_creationpoint(op.result, "getarrayitem")) def getfield(self, op, objstate, fieldname): if isonheap(op.result): # assume that getfield creates a new value return VarState(self.get_creationpoint(op.result, "getfield")) + def getsubstruct(self, op, objstate, fieldname): + # since this is really an embedded struct, it has the same + # state, the same creationpoints, etc. + return objstate + + def getarraysize(self, op, arraystate): + pass + def direct_call(self, op, function, *args): graph = get_graph(op.args[0], self.translation_context) - result, funcargs = self.schedule_function(graph) + if graph is None: + if len(filter(None, args)): + raise NotImplementedError, "can't handle call %s" % (op, ) + else: + result = None + funcargs = [None] * len(args) + else: + result, funcargs = self.schedule_function(graph) assert len(args) == len(funcargs) for localarg, funcarg in zip(args, funcargs): if localarg is None: @@ -267,8 +310,15 @@ # assume that a call creates a new value return VarState(self.get_creationpoint(op.result, "indirect_call")) - def ptr_iszero(self, op, ptr): + def ptr_iszero(self, op, ptrstate): return None + + cast_ptr_to_int = ptr_iszero + + def same_as(self, op, objstate): + return objstate + + def isonheap(var_or_const): return isinstance(var_or_const.concretetype, lltype.Ptr) Modified: pypy/dist/pypy/translator/backendopt/test/test_escape.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/test/test_escape.py (original) +++ pypy/dist/pypy/translator/backendopt/test/test_escape.py Thu Jan 12 00:04:01 2006 @@ -245,3 +245,55 @@ a2crep = a2state.creation_points.keys()[0] assert a1crep.changes and a2crep.changes assert not a1crep.escapes and a2crep.escapes + +def test_getarray(): + class A(object): + pass + def f(): + a = A() + l = [None] + l[0] = a + t, adi, graph = build_adi(f, []) + avar = graph.startblock.operations[0].result + state = adi.getstate(avar) + assert len(state.creation_points) == 1 + crep = state.creation_points.keys()[0] + assert crep.changes + assert crep.escapes + +def test_flow_blocksonce(): + class A(object): + pass + def f(): + a = 0 + for i in range(10): + a += i + b = A() + b.x = 1 + return b.x + 2 + t, adi, graph = build_adi(f, []) + avar = graph.startblock.exits[0].target.exits[1].target.operations[0].result + state = adi.getstate(avar) + assert not state.does_escape() + +def test_call_external(): + import time + def f(): + return time.time() + #does not crash + t, adi, graph = build_adi(f, []) + +def test_getsubstruct(): + def f(i): + s = "hello" + return s[i] + # does not crash + t, adi, graph = build_adi(f, [int]) + +def test_big(): + from pypy.translator.goal.targetrpystonex import make_target_definition + entrypoint, _, _ = make_target_definition(10) + # does not crash + t, adi, graph = build_adi(entrypoint, [int]) + + From adim at codespeak.net Thu Jan 12 09:50:36 2006 From: adim at codespeak.net (adim at codespeak.net) Date: Thu, 12 Jan 2006 09:50:36 +0100 (CET) Subject: [pypy-svn] r22003 - pypy/dist/pypy/module/recparser Message-ID: <20060112085036.DDDF427B60@code1.codespeak.net> Author: adim Date: Thu Jan 12 09:50:35 2006 New Revision: 22003 Modified: pypy/dist/pypy/module/recparser/__init__.py pypy/dist/pypy/module/recparser/pyparser.py Log: provide a small utility function in the parser module to generate the ast from source code Modified: pypy/dist/pypy/module/recparser/__init__.py ============================================================================== --- pypy/dist/pypy/module/recparser/__init__.py (original) +++ pypy/dist/pypy/module/recparser/__init__.py Thu Jan 12 09:50:35 2006 @@ -44,6 +44,7 @@ ## #'compilest' : 'pyparser.compilest', # PyPy extension + 'source2ast' : "pyparser.source2ast", 'decode_string_literal': 'pyparser.decode_string_literal', 'install_compiler_hook' : 'pypy.interpreter.pycompiler.install_compiler_hook', 'rules' : 'pypy.interpreter.pyparser.pythonparse.grammar_rules', Modified: pypy/dist/pypy/module/recparser/pyparser.py ============================================================================== --- pypy/dist/pypy/module/recparser/pyparser.py (original) +++ pypy/dist/pypy/module/recparser/pyparser.py Thu Jan 12 09:50:35 2006 @@ -200,6 +200,18 @@ return space.wrap( STType(space, syntaxtree) ) +def source2ast(space, source): + from pypy.interpreter.pyparser.pythonutil import AstBuilder, PYTHON_PARSER + builder = AstBuilder(space=space) + PYTHON_PARSER.parse_source(source, 'file_input', builder) + ast_tree = builder.rule_stack[-1] + return space.wrap(ast_tree) +source2ast.unwrap_spec = [ObjSpace, str] + + + + + def decode_string_literal(space, s, w_encoding=None): from pypy.interpreter.pyparser.parsestring import parsestr if space.is_true(w_encoding): From ericvrp at codespeak.net Thu Jan 12 10:31:21 2006 From: ericvrp at codespeak.net (ericvrp at codespeak.net) Date: Thu, 12 Jan 2006 10:31:21 +0100 (CET) Subject: [pypy-svn] r22006 - in pypy/dist/pypy/translator/c: . src test Message-ID: <20060112093121.8F72427B58@code1.codespeak.net> Author: ericvrp Date: Thu Jan 12 10:31:20 2006 New Revision: 22006 Modified: pypy/dist/pypy/translator/c/funcgen.py pypy/dist/pypy/translator/c/src/address.h pypy/dist/pypy/translator/c/test/test_lladdresses.py Log: Added genc support of stack flavored_malloc Modified: pypy/dist/pypy/translator/c/funcgen.py ============================================================================== --- pypy/dist/pypy/translator/c/funcgen.py (original) +++ pypy/dist/pypy/translator/c/funcgen.py Thu Jan 12 10:31:20 2006 @@ -568,6 +568,8 @@ flavor = op.args[0].value if flavor == "raw": return "OP_RAW_MALLOC(%s, %s, %s);" % (esize, eresult, err) + elif flavor == "stack": + return "OP_STACK_MALLOC(%s, %s, %s);" % (esize, eresult, err) else: raise NotImplementedError Modified: pypy/dist/pypy/translator/c/src/address.h ============================================================================== --- pypy/dist/pypy/translator/c/src/address.h (original) +++ pypy/dist/pypy/translator/c/src/address.h Thu Jan 12 10:31:20 2006 @@ -20,5 +20,9 @@ r = (void*) malloc(size); \ if (r == NULL) FAIL_EXCEPTION(err, PyExc_MemoryError, "out of memory");\ +#define OP_STACK_MALLOC(size,r,err) \ + r = (void*) alloca(size); \ + if (r == NULL) FAIL_EXCEPTION(err, PyExc_MemoryError, "out of memory");\ + #define OP_RAW_FREE(x,r,err) free(x); #define OP_RAW_MEMCOPY(x,y,size,r,err) memcpy(y,x,size); Modified: pypy/dist/pypy/translator/c/test/test_lladdresses.py ============================================================================== --- pypy/dist/pypy/translator/c/test/test_lladdresses.py (original) +++ pypy/dist/pypy/translator/c/test/test_lladdresses.py Thu Jan 12 10:31:20 2006 @@ -88,7 +88,7 @@ res = fc() assert res == int('011100' * 2, 2) -def test_flavored_malloc(): +def test_flavored_malloc_raw(): class A(object): _alloc_flavor_ = "raw" def __init__(self, val): @@ -100,3 +100,15 @@ return result fn = compile(f, [int]) assert fn(1) == 2 + +def test_flavored_malloc_stack(): + class A(object): + _alloc_flavor_ = "stack" + def __init__(self, val): + self.val = val + def f(x): + a = A(x + 1) + result = a.val + return result + fn = compile(f, [int]) + assert fn(1) == 2 From ericvrp at codespeak.net Thu Jan 12 10:50:10 2006 From: ericvrp at codespeak.net (ericvrp at codespeak.net) Date: Thu, 12 Jan 2006 10:50:10 +0100 (CET) Subject: [pypy-svn] r22007 - pypy/dist/pypy/translator/c/src Message-ID: <20060112095010.4452327B84@code1.codespeak.net> Author: ericvrp Date: Thu Jan 12 10:50:09 2006 New Revision: 22007 Modified: pypy/dist/pypy/translator/c/src/address.h Log: alloca is called _alloca on Windows (thanks Ben) Modified: pypy/dist/pypy/translator/c/src/address.h ============================================================================== --- pypy/dist/pypy/translator/c/src/address.h (original) +++ pypy/dist/pypy/translator/c/src/address.h Thu Jan 12 10:50:09 2006 @@ -19,7 +19,11 @@ #define OP_RAW_MALLOC(size,r,err) \ r = (void*) malloc(size); \ if (r == NULL) FAIL_EXCEPTION(err, PyExc_MemoryError, "out of memory");\ - + +#ifdef MS_WINDOWS +#define alloca _alloca +#endif + #define OP_STACK_MALLOC(size,r,err) \ r = (void*) alloca(size); \ if (r == NULL) FAIL_EXCEPTION(err, PyExc_MemoryError, "out of memory");\ From auc at codespeak.net Thu Jan 12 11:17:05 2006 From: auc at codespeak.net (auc at codespeak.net) Date: Thu, 12 Jan 2006 11:17:05 +0100 (CET) Subject: [pypy-svn] r22009 - pypy/dist/pypy/doc Message-ID: <20060112101705.9C54027B6C@code1.codespeak.net> Author: auc Date: Thu Jan 12 11:17:03 2006 New Revision: 22009 Added: pypy/dist/pypy/doc/plan-for-wp9-10.txt Log: This is a test for svn access. Added: pypy/dist/pypy/doc/plan-for-wp9-10.txt ============================================================================== From auc at codespeak.net Thu Jan 12 11:23:26 2006 From: auc at codespeak.net (auc at codespeak.net) Date: Thu, 12 Jan 2006 11:23:26 +0100 (CET) Subject: [pypy-svn] r22010 - pypy/dist/pypy/doc Message-ID: <20060112102326.793F227B64@code1.codespeak.net> Author: auc Date: Thu Jan 12 11:23:25 2006 New Revision: 22010 Removed: pypy/dist/pypy/doc/plan-for-wp9-10.txt Log: removal of the test stuff From arigo at codespeak.net Thu Jan 12 11:29:10 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Thu, 12 Jan 2006 11:29:10 +0100 (CET) Subject: [pypy-svn] r22011 - pypy/extradoc/talk/chalmers Message-ID: <20060112102910.6D7DA27B70@code1.codespeak.net> Author: arigo Date: Thu Jan 12 11:29:06 2006 New Revision: 22011 Modified: pypy/extradoc/talk/chalmers/talk-wm2006.txt Log: Tweaks. Modified: pypy/extradoc/talk/chalmers/talk-wm2006.txt ============================================================================== --- pypy/extradoc/talk/chalmers/talk-wm2006.txt (original) +++ pypy/extradoc/talk/chalmers/talk-wm2006.txt Thu Jan 12 11:29:06 2006 @@ -13,7 +13,6 @@ .. image:: image/py-web1.png :align: right - Python Implementation Facts =========================== @@ -26,15 +25,15 @@ Python Implementations =========================== -- CPython: main Python version -- Jython: compiles to Java Bytecode -- IronPython (MS): compiles to .NET's CLR -- PyPy: self-contained - self-translating - flexible +:CPython: main Python version +:Jython: compiles to Java Bytecode +:IronPython: compiles to .NET's CLR +:PyPy: self-contained - self-translating - flexible PyPy Project Facts ======================= -- started 2002 as a grass-root effort +- Open Source project, started 2002 - aims: flexibility, research, speed - test-driven development - received EU-funding from end 2004 on @@ -50,12 +49,35 @@ - but no low-level details - suitable for translation to many platforms +Goals +========================= + +Framework for language implementors: + +- Write interpreter once +- compile it to very different platforms +- low-level platform-specific details are modular +- experiment with language-level extensions: + concurrency models, memory models, security, distribution, persistence... + Translation Overview ========================= .. image:: image/translation-overview.png :align: center +References +==================== + +- http://codespeak.net/pypy + + .. image:: image/py-web1.png + :align: center + +==== + +- *Compiling dynamic language implementations*, to be submitted + Translation Aspects ==================== @@ -68,7 +90,7 @@ Translation Aspects (2) ========================= -- the execution strategy can be tweaked at translation time too +- even the execution strategy can be tweaked at translation time - if the input RPython program is an Interpreter_... - ...the output low-level program can be a Compiler_ @@ -87,6 +109,20 @@ - produces x86 machine code at run-time - a "generating extension": generates specialized versions of the CPython Interpreter +References +========================= + +- http://psyco.sf.net + + .. image:: image/pipe-title.jpg + :align: center + +==== + +- Representation-Based Just-In-Time Specialization and the + Psyco Prototype for Python, ACM SIGPLAN PEPM'04, August 24-26, 2004, + Verona, Italy. + PyPy JIT Overview ==================== @@ -96,7 +132,6 @@ :normal output: an Interpreter -- like CPython :new output: a Compiler -- like Psyco - Recipe for a JIT (1) ========================== @@ -125,20 +160,8 @@ - start from high-level Language Definitions - regenerate Interpreters automatically - get Just-In-Time Compilers for free :-) - -Links -========================= - - http://codespeak.net/pypy - .. image:: image/py-web1.png - :align: center - -- http://psyco.sf.net - - .. image:: image/pipe-title.jpg - :align: center - .. _Psyco: http://psyco.sourceforge.net .. _threads: image/stackless_informal.png From auc at codespeak.net Thu Jan 12 12:39:24 2006 From: auc at codespeak.net (auc at codespeak.net) Date: Thu, 12 Jan 2006 12:39:24 +0100 (CET) Subject: [pypy-svn] r22015 - pypy/dist/pypy/doc Message-ID: <20060112113924.DC35227B6C@code1.codespeak.net> Author: auc Date: Thu Jan 12 12:39:22 2006 New Revision: 22015 Added: pypy/dist/pypy/doc/constraints-and-logic.txt Log: Early reflexions on constraints and logic programming in Pypy. Added: pypy/dist/pypy/doc/constraints-and-logic.txt ============================================================================== --- (empty file) +++ pypy/dist/pypy/doc/constraints-and-logic.txt Thu Jan 12 12:39:22 2006 @@ -0,0 +1,322 @@ +====================================================================== + Ideas about syntactic and algorithmic aspects of Constraint and Logic + Programming in Python (CLPiP) +====================================================================== + +** WORK IN PROGRESS ** + +Introduction +============ + +Scope +----- + +This document tries to shed some light on integration of logic and +constraint programming into Python using the Pypy +framework. + +Pypy has been progressively equiped with a parser and compiler +flexible enough that it is hoped that developpers can leverage it to +extend the language *at runtime*. This is quite in the spirit of Lisp +macros, if not the exact manner. + +This takes place in Working Packages 09 and 10 of the EU Pypy funding +project. It is expected that an aspect oriented programming toolkit be +built using the compiler and parser infrastructure (WP10), and that the +logic and constraint programming features be added to Pypy (WP9). An +ontology library will be provided and will serve as our first use +case for logic programming. + +Clarifications +-------------- + +This work was described as integration of logic programming *and* +constraint programming into Pypy. Both are obviously related but we +don't know if they will share, in Pypy, the same syntax and the same +machinery. + +Also it has been said that CLP will be inspired from what is done in +the Oz programming language, which provides different programming +'paradigms' with a very well thought-out integration between said +paradigms. For instance, imperative features can be mixed, to some +extent, with logic programming (the later can be 'embedded' in the +former); or constraint programming goes with stateless concurrency, +another important feature of Oz. We still don't know to what extent +the Oz level of integration will be pursued in the context of Pypy, +esp. WP 9 and 10. + +Lastly, here we mainly discuss syntactical issues: those are probably +the least difficult aspects of getting CLP into python; getting an +efficient implementation of the canonical algorithms into Pypy will be +the bulk of the work. + +Constraint programming +====================== + +In constraint programming, a 'problem' is a set of variables, their +(finite discrete) domains, and the constraints that restrict their +possible values. When all these have been given to a constraint +solver, it is possible to find all possible solutions, that is the +sets of valuations that satisfies simultaneously all constraints. The +solver is solely responsible for finding solutions (or lack thereof). + +Overview in python +------------------ + +At the time being, there exists a *constraints* package made by +Logilab and written in pure python, which implements the solver found +in Mozart (the reference Oz implementation). We use it to illustrate +where we want to go, syntactically-wise. + +Let's start with a quite standard example (the problem being solved +here is fully described on +http://www.logilab.org/projects/constraint/documentation) : + + + # import Repository class and fd module, + from logilab.constraint import * + variables = ('c01','c02','c03','c04','c05','c06','c07','c08','c09','c10') + +Variables are represented as any string object. + + values = [(room,slot) + for room in ('room A','room B','room C') + for slot in ('day 1 AM','day 1 PM','day 2 AM','day 2 PM')] + +Values can be freely pre-computed using standard python constructs; +they can be any object; here, tuples of strings. + + domains = {} + for v in variables: + domains[v]=fd.FiniteDomain(values) + +The relationship between variables and their possible values is set in +a dictionnary whose keys are variable designators (strings). Values +are wrapped into FiniteDomain instances (FiniteDomain has set +behaviour, plus some implementation subtleties). + + groups = (('c01','c02','c03','c10'), + ('c02','c06','c08','c09'), + ('c03','c05','c06','c07'), + ('c01','c03','c07','c08')) + for g in groups: + for conf1 in g: + for conf2 in g: + if conf2 > conf1: + constraints.append(fd.make_expression((conf1,conf2), + '%s[1] != %s[1]'%\ + (conf1,conf2))) + +Constraints are built by make_expression which takes a tuple of one or +two variables and a string representing an unary or binary +relationship. The example complete with all constraints is provided at +the url mentioned supra. + +Then, when everything has been settled, comes the last step : + + r = Repository(variables,domains,constraints) + solutions = Solver().solve(r) + print solutions + +Remarks +------- + +Due to the compactness of Python syntax, this sample problem +specification remains quite small and readable. It is not obvious what +could be done to make it smaller and still readable. + +Variables are not first-class (but close ...) and have nothing to do +with Python standard variables. The good side of this is that we can't +misuse a CSP variable with an ordinary variable. + +Specifiying a constraint is clunky : variables and operator have to be +provided separately, and the operator has to be a string. This last +restriction because Python doesn't allow passing builtin (infix) +operators as functional parameters. + +Special operator for CLP variable bindings +------------------------------------------ + +First, promote variables from third-class to second-class +citizenry. Be able to write something like : + + domain = [(room,slot) + for room in ('room A','room B','room C') + for slot in ('day 1 AM','day 1 PM','day 2 AM','day 2 PM')] + c01 := domain + c02 := domain + +This introduces a special operator ':=' which binds a logical variable +to a domain. More generally : + + var := + +With respect to normal assignment, we can imagine the following : + + c01 = 'foo' # raises a NotAssignable or ReadOnly exception + bar = c01 # takes a reference to the current value of c01 into bar + # also, meaningless (so ... None) before the solver has run + +Problem ... we can't anymore do : + + for conf in ('c01','c05','c10'): ... + +It should be good to define a kind of first-class designator for these +kind of variables. A specially-crafted class representing variables +(in the manner of Lisp's symbols) would suffice : + + for conf in (c01, c05, c10): ... + +Is it worth the price ? Quite unsure. + +Domain-specific blocks +---------------------- + +An alternative which avoids the special operator and uses a keyword +instead could be : + + domain: + c01 = + c02 = + +It makes us reuse '=', with twisted (non-standard Python) semantics +but under a clear lexical umbrella (a 'domain:' block). + +It is possible to get further in this direction : + + problem toto: + D1 = + a,b,c in D1 + def constraint1(a,b,c): + a == b + + for sol in toto: + print sol + +There, we put a full constraints mini-language under a named 'problem' +block. The *problem* becomes a first class object (in the manner of +Python classes) and we can (lazily) extract solutions from it. + + +Stuffing constraints +-------------------- + +The ugly aspect of py-constraints is the definition of custom +unary/binary constraints through make_expression, as in : + + fd.make_expression ('var1', 'var2', "frob(var1,var2)") + +One solution might be to parse the string at runtime to recover the +variable names : + + fd.make_expression ('frob(var1,var2)') + +A simple hand-written parser could be sufficient for this. On the +other hand, the lexically bounded mini-language proposed above helps +solve this more uniformly. + +Logic Programming, Prolog-style +=============================== + +Good old logic programming has an already long syntactic tradition; we +should probably not try to evade it too much. Thus we propose again a +mini-language embedding horn-clauses and logic stuff into 'normal' +(unsuspecting) Python programs. + +Databases and Requests +---------------------- + +Databases contain facts, rules and requests. + + >>> database Numbers: # we def a logic db and enter prolog space + ... natural(z) + ... natural(s(X)) if natural(X) + ... r1: natural(s(s(s(z)))) # kind of named request + ... + >>> Numbers + + >>> Numbers = 42 + NotAssignable exception, etc. + + +Integration in Python +--------------------- + + # databases as first class objects + get_logic_databases() # -> [Numbers] + + # return values in Python space + d1 = get_logic_databases()[0] + d1.ask('r1') -> True + + # extending an existing database + database Numbers: # adds up to existing Numbers db (eyebrows raise) + pair(z) + pair(s(s(X))) if pair(X) + + # database acquisition (inheritance ?) + database Ratios(Numbers): + pass + + # adding rules and requests from Python space + d2 = get_logic_databases[1] # the Ratios db + d2.add_rule('ratio(X,Y) if Y != z, natural(X), natural(Y)') + + d2.add_request('r2', 'ratio(s(z),toto)') + d2.ask('r2') -> False + + +Thus, the logic and Python stuff are cleanly separated, and of course +we forbid lexical capture as in : + + z = 42 + database Numbers: + natural(z) + +A (maybe) nicer way to ask could be : + + database Family : + + father(jean, pierre) + father(paul, jean) + grandfather(X,Y) if father(X,Z) and father(Z,Y) + + Family.grandfather(jean, A) + + +Conclusion +---------- + +Syntax is not all-important but can touch on the nerves of some. It +sometimes involves relogious arguments. We hope to irritate none with +this proposal; we merely expect counter-proposals or more detailed +proposals or agreement or just indiference on this. + + +CLP and Oz +========== + +The Oz programming language is and will be a driving inspiration for +all deep integration work of CLP. Logilab constraints pure python +package already implements search according to Oz (without the +concurrency parts). + +What shall be done +------------------ + +For constraint programming: + +* Adapt the core algorithms from logilab.constraints to RPython +* Stuff it into Pypy +* Enhance (tm) it with concurrency aspects + +Logic programming: + +* Investigate how much of above can be reused +* Pick a strategy (reuse constraints or implement unification) +* Implement deep into Pypy + +For both: + +* Adapt the parser/lexer to the syntactic requirements + From cfbolz at codespeak.net Thu Jan 12 12:40:07 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Thu, 12 Jan 2006 12:40:07 +0100 (CET) Subject: [pypy-svn] r22016 - pypy/extradoc/talk/ui Message-ID: <20060112114007.8BBA727B6C@code1.codespeak.net> Author: cfbolz Date: Thu Jan 12 12:40:05 2006 New Revision: 22016 Added: pypy/extradoc/talk/ui/ - copied from r22006, pypy/eu-tracking/talk/review1/ui/ Log: add ui there too From auc at codespeak.net Thu Jan 12 13:12:07 2006 From: auc at codespeak.net (auc at codespeak.net) Date: Thu, 12 Jan 2006 13:12:07 +0100 (CET) Subject: [pypy-svn] r22018 - pypy/dist/pypy/doc Message-ID: <20060112121207.598A027B6C@code1.codespeak.net> Author: auc Date: Thu Jan 12 13:12:06 2006 New Revision: 22018 Modified: pypy/dist/pypy/doc/constraints-and-logic.txt Log: add url for Oz, mention tasklets Modified: pypy/dist/pypy/doc/constraints-and-logic.txt ============================================================================== --- pypy/dist/pypy/doc/constraints-and-logic.txt (original) +++ pypy/dist/pypy/doc/constraints-and-logic.txt Thu Jan 12 13:12:06 2006 @@ -36,14 +36,14 @@ machinery. Also it has been said that CLP will be inspired from what is done in -the Oz programming language, which provides different programming -'paradigms' with a very well thought-out integration between said -paradigms. For instance, imperative features can be mixed, to some -extent, with logic programming (the later can be 'embedded' in the -former); or constraint programming goes with stateless concurrency, -another important feature of Oz. We still don't know to what extent -the Oz level of integration will be pursued in the context of Pypy, -esp. WP 9 and 10. +the Oz programming language (http://www.mozart-oz.org/), which +provides different programming 'paradigms' with a very well +thought-out integration between said paradigms. For instance, +imperative features can be mixed, to some extent, with logic +programming (the later can be 'embedded' in the former); or constraint +programming goes with stateless concurrency, another important feature +of Oz. We still don't know to what extent the Oz level of integration +will be pursued in the context of Pypy, esp. WP 9 and 10. Lastly, here we mainly discuss syntactical issues: those are probably the least difficult aspects of getting CLP into python; getting an @@ -308,7 +308,7 @@ * Adapt the core algorithms from logilab.constraints to RPython * Stuff it into Pypy -* Enhance (tm) it with concurrency aspects +* Enhance (tm) it with concurrency aspects (using tis's tasklets) Logic programming: From auc at codespeak.net Thu Jan 12 13:55:33 2006 From: auc at codespeak.net (auc at codespeak.net) Date: Thu, 12 Jan 2006 13:55:33 +0100 (CET) Subject: [pypy-svn] r22019 - pypy/dist/pypy/doc Message-ID: <20060112125533.B67C627B6C@code1.codespeak.net> Author: auc Date: Thu Jan 12 13:55:32 2006 New Revision: 22019 Modified: pypy/dist/pypy/doc/constraints-and-logic.txt Log: add refs to prolog-in-python stuff Modified: pypy/dist/pypy/doc/constraints-and-logic.txt ============================================================================== --- pypy/dist/pypy/doc/constraints-and-logic.txt (original) +++ pypy/dist/pypy/doc/constraints-and-logic.txt Thu Jan 12 13:55:32 2006 @@ -223,6 +223,12 @@ mini-language embedding horn-clauses and logic stuff into 'normal' (unsuspecting) Python programs. +Existing python/prolog projects can be found at : +http://christophe.delord.free.fr/en/pylog/ and +http://aima.cs.berkeley.edu/python/logic.html. There might be more of +them. + + Databases and Requests ---------------------- From ale at codespeak.net Thu Jan 12 14:10:16 2006 From: ale at codespeak.net (ale at codespeak.net) Date: Thu, 12 Jan 2006 14:10:16 +0100 (CET) Subject: [pypy-svn] r22022 - in pypy/dist/pypy/lib/pyontology: . test Message-ID: <20060112131016.AC44827B6C@code1.codespeak.net> Author: ale Date: Thu Jan 12 14:10:15 2006 New Revision: 22022 Modified: pypy/dist/pypy/lib/pyontology/pyontology.py pypy/dist/pypy/lib/pyontology/test/test_ontology.py Log: Added a test for differentfrom. Light refactoring of constraint classes Modified: pypy/dist/pypy/lib/pyontology/pyontology.py ============================================================================== --- pypy/dist/pypy/lib/pyontology/pyontology.py (original) +++ pypy/dist/pypy/lib/pyontology/pyontology.py Thu Jan 12 14:10:15 2006 @@ -70,10 +70,11 @@ self.values = values class List(ClassDomain): + def __init__(self, name='', values=[], bases = []): ClassDomain.__init__(self, name, values, bases) self.constraint = ListConstraint(name) - + class Property(ClassDomain): # Property contains the relationship between a class instance and a value # - a pair. To accomodate global assertions like 'range' and 'domain' attributes @@ -450,24 +451,25 @@ class OwlConstraint(AbstractConstraint): + cost = 1 + def __init__(self, variable): AbstractConstraint.__init__(self, [variable]) self.variable = variable - self.__cost = 1 def __repr__(self): return '<%s %s>' % (self.__class__.__name__, str(self._variables[0])) def estimateCost(self, domains): - return self.__cost + return self.cost class MaxCardinality(AbstractConstraint): """Contraint: all values must be distinct""" - def __init__(self, variable, cls, cardinality): + def __init__(self, variable, cardinality): AbstractConstraint.__init__(self, [variable]) - self.__cost = 2 + self.cost = 80 self.variable = variable self.cardinality = cardinality #self.cls = cls @@ -476,7 +478,7 @@ return '<%s %s %i>' % (self.__class__.__name__, str(self._variables[0]), self.cardinality) def estimateCost(self, domains): - return 80 + return self.cost def narrow(self, domains): """narrowing algorithm for the constraint""" @@ -512,7 +514,6 @@ else: return 1 - def get_values(dom, domains, attr = 'getValues'): res = [] if type(dom) == Literal: @@ -527,11 +528,16 @@ class SubClassConstraint(AbstractConstraint): + cost=1 + def __init__(self, variable, cls_or_restriction): AbstractConstraint.__init__(self, [variable, cls_or_restriction]) self.object = cls_or_restriction self.variable = variable + def estimateCost(self, domains): + return self.cost + def __repr__(self): return '<%s %s %s>' % (self.__class__.__name__, str(self._variables[0]), self.object) @@ -566,9 +572,8 @@ class RangeConstraint(SubClassConstraint): - def estimateCost(self, domains): - return 200 - + cost = 200 + def narrow(self, domains): propdom = domains[self.variable] rangedom = domains[self.object] @@ -706,20 +711,17 @@ if not (val,cls) in obj_domain: raise ConsistencyFailure("Inverseof failed") -class ListConstraint(AbstractConstraint): +class ListConstraint(OwlConstraint): """Contraint: all values must be distinct""" - def __init__(self, variable): - AbstractConstraint.__init__(self, [variable]) - self.variable = variable - self.__cost = 10 - def estimateCost(self, domains): - return self.__cost + cost = 10 def narrow(self, domains): """narrowing algorithm for the constraint""" vals =[] vals += domains[self.variable].getValues() + if vals == []: + return 0 while True: if vals[-1] in domains.keys() and isinstance(domains[vals[-1]], List): vals = vals[:-1] + domains[vals[-1]].getValues() @@ -728,13 +730,9 @@ domains[self.variable].setValues(vals) return 1 -class RestrictionConstraint(AbstractConstraint): - def __init__(self, variable): - AbstractConstraint.__init__(self, [variable]) - self.variable = variable +class RestrictionConstraint(OwlConstraint): - def estimateCost(self, domains): - return 90 + cost = 90 def narrow(self, domains): prop = domains[self.variable].property @@ -750,9 +748,11 @@ self.variable = variable self.List = List + cost = 100 + def estimateCost(self, domains): - return 100 - + return self.cost + def narrow(self, domains): val = domains[self.List].getValues() if isinstance(domains[self.variable],Restriction): @@ -768,16 +768,10 @@ domains[self.variable].setValues(val) return 1 -class UnionofConstraint(AbstractConstraint): +class UnionofConstraint(OneofPropertyConstraint): - def __init__(self, variable, List): - AbstractConstraint.__init__(self, [variable, List]) - self.variable = variable - self.List = List + cost = 200 - def estimateCost(self, domains): - return 200 - def narrow(self, domains): val = domains[self.List].getValues() union = [] @@ -787,16 +781,10 @@ union.append(u) cls = domains[self.variable].setValues(union) -class IntersectionofConstraint(AbstractConstraint): +class IntersectionofConstraint(OneofPropertyConstraint): - def __init__(self, variable, List): - AbstractConstraint.__init__(self, [variable, List]) - self.variable = variable - self.List = List + cost = 200 - def estimateCost(self, domains): - return 200 - def narrow(self, domains): val = domains[self.List].getValues() intersection = domains[val[0]].getValues() @@ -811,15 +799,9 @@ cls = domains[self.variable].setValues(intersection) -class SomeValueConstraint(AbstractConstraint): +class SomeValueConstraint(OneofPropertyConstraint): - def __init__(self, variable, List): - AbstractConstraint.__init__(self, [variable, List]) - self.variable = variable - self.List = List - - def estimateCost(self, domains): - return 100 + cost = 100 def narrow(self, domains): val = domains[self.List].getValues() @@ -834,15 +816,9 @@ "The value of the property %s in the class %s has no values from %r" %(property, cls, val)) -class AllValueConstraint(AbstractConstraint): - - def __init__(self, variable, List): - AbstractConstraint.__init__(self, [variable, List]) - self.variable = variable - self.List = List +class AllValueConstraint(OneofPropertyConstraint): - def estimateCost(self, domains): - return 100 + cost = 100 def narrow(self, domains): val = domains[self.List].getValues() @@ -856,16 +832,19 @@ %(property, cls, val)) class HasvalueConstraint(AbstractConstraint): + def __init__(self, variable, List): AbstractConstraint.__init__(self, [variable]) self.variable = variable self.List = List + cost = 100 + def estimateCost(self, domains): - return 100 - + return self.cost + def narrow(self, domains): - val = self.List #domains[self.List].getValues() + val = self.List property = domains[self.variable].property cls = domains[self.variable].cls prop = Linkeddict(domains[property].getValues()) Modified: pypy/dist/pypy/lib/pyontology/test/test_ontology.py ============================================================================== --- pypy/dist/pypy/lib/pyontology/test/test_ontology.py (original) +++ pypy/dist/pypy/lib/pyontology/test/test_ontology.py Thu Jan 12 14:10:15 2006 @@ -463,3 +463,18 @@ O.type(cls, namespaces['owl']+'#Class') O.consistency(3) assert O.rep._domains[cls].getValues() == ['3'] + +def test_differentfrom(): + O = Ontology() + cls = BNode('anon') + own1 = BNode('liist1') + own2 = BNode('liist2') + list1 = rdf_list(O, 'favlist1', ['1', '2', '3']) + list2 = rdf_list(O, 'favlist2', ['3', '4', '5']) + own = rdf_list(O, 'favlist', [own1, own2]) + O.oneOf( own1, list1) + O.oneOf( own2, list2) + O.intersectionOf(cls, own) + O.type(cls, namespaces['owl']+'#Class') + O.consistency(3) + assert O.rep._domains[cls].getValues() == ['3'] From adim at codespeak.net Thu Jan 12 15:13:36 2006 From: adim at codespeak.net (adim at codespeak.net) Date: Thu, 12 Jan 2006 15:13:36 +0100 (CET) Subject: [pypy-svn] r22032 - pypy/dist/pypy/module/recparser Message-ID: <20060112141336.3504227B64@code1.codespeak.net> Author: adim Date: Thu Jan 12 15:13:34 2006 New Revision: 22032 Modified: pypy/dist/pypy/module/recparser/__init__.py pypy/dist/pypy/module/recparser/pyparser.py Log: tab /spaces mess Modified: pypy/dist/pypy/module/recparser/__init__.py ============================================================================== --- pypy/dist/pypy/module/recparser/__init__.py (original) +++ pypy/dist/pypy/module/recparser/__init__.py Thu Jan 12 15:13:34 2006 @@ -44,7 +44,7 @@ ## #'compilest' : 'pyparser.compilest', # PyPy extension - 'source2ast' : "pyparser.source2ast", + 'source2ast' : "pyparser.source2ast", 'decode_string_literal': 'pyparser.decode_string_literal', 'install_compiler_hook' : 'pypy.interpreter.pycompiler.install_compiler_hook', 'rules' : 'pypy.interpreter.pyparser.pythonparse.grammar_rules', Modified: pypy/dist/pypy/module/recparser/pyparser.py ============================================================================== --- pypy/dist/pypy/module/recparser/pyparser.py (original) +++ pypy/dist/pypy/module/recparser/pyparser.py Thu Jan 12 15:13:34 2006 @@ -209,9 +209,6 @@ source2ast.unwrap_spec = [ObjSpace, str] - - - def decode_string_literal(space, s, w_encoding=None): from pypy.interpreter.pyparser.parsestring import parsestr if space.is_true(w_encoding): From guido at codespeak.net Thu Jan 12 15:52:41 2006 From: guido at codespeak.net (guido at codespeak.net) Date: Thu, 12 Jan 2006 15:52:41 +0100 (CET) Subject: [pypy-svn] r22040 - pypy/extradoc/talk/ui Message-ID: <20060112145241.3970827B6C@code1.codespeak.net> Author: guido Date: Thu Jan 12 15:52:40 2006 New Revision: 22040 Added: pypy/extradoc/talk/ui/py-web.png (contents, props changed) pypy/extradoc/talk/ui/py.css Log: Added PyPy logo and PyPy stylesheet. Added: pypy/extradoc/talk/ui/py-web.png ============================================================================== Binary file. No diff available. Added: pypy/extradoc/talk/ui/py.css ============================================================================== --- (empty file) +++ pypy/extradoc/talk/ui/py.css Thu Jan 12 15:52:40 2006 @@ -0,0 +1,72 @@ +body, h1, h2, h3, h4, td, p, div { + font-family: sans-serif; + font-size: 0.9em; +} + +#slide0 h1.title { + text-align: center; + font-family: sans-serif; +} + +div#header, div#footer, div#controls { + background-color: white; +} + +div#header { + background-image: url("py-web.png"); + background-repeat: no-repeat; + margin: 3px; + height: 120px; + border-bottom: 1px solid black; +} + +.slide h1 { + background-color: white; + margin-left: 180px; + padding-left: 0px; + color: black; + font-size: 1.2em; +} + +div#footer { + padding: 3px; + height: 4em; + border-top: 1px solid black; +} + +div#footer h1, div#footer h2, div#footer h3 { + font-family: "Times New Roman"; + font-style: italic; + padding: 0px; +} + +div#footer h1 { + font-size: 2em; +} + +#controls { + border: 1px solid red; + background-color: red; + width: 100px; + visibility: hidden; +} + +#controls #controlForm { + visibility: visible; +} + +#navList, #navLinks { + background-color: transparent; +} + +#navLinks a#toggle, #navLinks a#prev, #navLinks a#next { + background-color: transparent; +} + +/* some hacks to fix whitespace between list items */ + +li, li p { + line-height: 1.2em; + font-size: 1em; + margin: 0px; +} From guido at codespeak.net Thu Jan 12 16:49:15 2006 From: guido at codespeak.net (guido at codespeak.net) Date: Thu, 12 Jan 2006 16:49:15 +0100 (CET) Subject: [pypy-svn] r22050 - pypy/extradoc/talk/ui Message-ID: <20060112154915.115B827B6C@code1.codespeak.net> Author: guido Date: Thu Jan 12 16:49:15 2006 New Revision: 22050 Modified: pypy/extradoc/talk/ui/py.css Log: Made global fonts a bit smaller. Modified: pypy/extradoc/talk/ui/py.css ============================================================================== --- pypy/extradoc/talk/ui/py.css (original) +++ pypy/extradoc/talk/ui/py.css Thu Jan 12 16:49:15 2006 @@ -1,6 +1,6 @@ body, h1, h2, h3, h4, td, p, div { font-family: sans-serif; - font-size: 0.9em; + font-size: 0.7em; } #slide0 h1.title { From guido at codespeak.net Thu Jan 12 16:49:42 2006 From: guido at codespeak.net (guido at codespeak.net) Date: Thu, 12 Jan 2006 16:49:42 +0100 (CET) Subject: [pypy-svn] r22051 - pypy/extradoc/talk/ui/default Message-ID: <20060112154942.097E127B70@code1.codespeak.net> Author: guido Date: Thu Jan 12 16:49:41 2006 New Revision: 22051 Modified: pypy/extradoc/talk/ui/default/slides.css Log: Made slides.css import py.css so no changes to the documents have to be made. Modified: pypy/extradoc/talk/ui/default/slides.css ============================================================================== --- pypy/extradoc/talk/ui/default/slides.css (original) +++ pypy/extradoc/talk/ui/default/slides.css Thu Jan 12 16:49:41 2006 @@ -8,3 +8,6 @@ /* styles that make the slides look good */ @import url(pretty.css); + +/* pypy override */ + at import url(../py.css); From ac at codespeak.net Thu Jan 12 19:54:25 2006 From: ac at codespeak.net (ac at codespeak.net) Date: Thu, 12 Jan 2006 19:54:25 +0100 (CET) Subject: [pypy-svn] r22069 - in pypy/branch/arre-experiments/pypy: annotation interpreter rpython translator/tool Message-ID: <20060112185425.A219F27B6C@code1.codespeak.net> Author: ac Date: Thu Jan 12 19:54:24 2006 New Revision: 22069 Modified: pypy/branch/arre-experiments/pypy/annotation/bookkeeper.py pypy/branch/arre-experiments/pypy/annotation/description.py pypy/branch/arre-experiments/pypy/interpreter/argument.py pypy/branch/arre-experiments/pypy/rpython/callparse.py pypy/branch/arre-experiments/pypy/translator/tool/cbuild.py Log: Slight refactoring. Modified: pypy/branch/arre-experiments/pypy/annotation/bookkeeper.py ============================================================================== --- pypy/branch/arre-experiments/pypy/annotation/bookkeeper.py (original) +++ pypy/branch/arre-experiments/pypy/annotation/bookkeeper.py Thu Jan 12 19:54:24 2006 @@ -632,6 +632,7 @@ For the Arguments class: if it really needs other operations, it means that the call pattern is too complex for R-Python. """ + w_tuple = SomeTuple def newtuple(self, items_s): return SomeTuple(items_s) @@ -646,6 +647,13 @@ return s_obj.items raise CallPatternTooComplex, "'*' argument must be SomeTuple" + def is_w(self, one, other): + return one is other + + def type(self, item): + return type(item) + + class CallPatternTooComplex(Exception): pass Modified: pypy/branch/arre-experiments/pypy/annotation/description.py ============================================================================== --- pypy/branch/arre-experiments/pypy/annotation/description.py (original) +++ pypy/branch/arre-experiments/pypy/annotation/description.py Thu Jan 12 19:54:24 2006 @@ -202,7 +202,7 @@ try: inputcells = args.match_signature(signature, defs_s) except ArgErr, e: - raise TypeError, "signature mismatch: %s" % e.getmsg(args, self.name) + raise TypeError, "signature mismatch: %s" % e.getmsg(self.name) return inputcells def specialize(self, inputcells): Modified: pypy/branch/arre-experiments/pypy/interpreter/argument.py ============================================================================== --- pypy/branch/arre-experiments/pypy/interpreter/argument.py (original) +++ pypy/branch/arre-experiments/pypy/interpreter/argument.py Thu Jan 12 19:54:24 2006 @@ -35,7 +35,6 @@ "Return a new Arguments with a new argument inserted first." return ArgumentsPrepended(self, w_firstarg) - class ArgumentsPrepended(AbstractArguments): def __init__(self, args, w_firstarg): self.args = args @@ -63,29 +62,30 @@ def _rawshape(self, nextra=0): return self.args._rawshape(nextra + 1) - def _unpack_stararg(self): - self.args._unpack_stararg() - def parse(self, fnname, signature, defaults_w=[]): """Parse args and kwargs to initialize a frame according to the signature of code object. """ - self._unpack_stararg() try: return self.match_signature(signature, defaults_w) except ArgErr, e: raise OperationError(self.args.space.w_TypeError, - self.args.space.wrap(e.getmsg(self, fnname))) + self.args.space.wrap(e.getmsg(fnname))) def match_signature(self, signature, defaults_w=[]): """Parse args and kwargs according to the signature of a code object, or raise an ArgErr in case of failure. """ argnames, varargname, kwargname = signature - scope_w = self.args.match_signature((argnames[1:], varargname, kwargname), defaults_w) + try: + scope_w = self.args.match_signature((argnames[1:], varargname, kwargname), defaults_w) + except ArgErrCount, e: + e.num_args += 1 # Add our first arg + raise + if len(argnames) == 0: if varargname is None: - raise ArgErrCount(signature, defaults_w, 0) + raise ArgErrCount(self, signature, defaults_w, 0) space = self.args.space if kwargname is not None: scope_w[-2] = space.newtuple([self.w_firstarg] + space.unpackiterable(scope_w[-2])) @@ -100,7 +100,12 @@ data_w.insert(0, self.w_firstarg) return (shape_cnt + 1, shape_keys, shape_star, shape_stst), data_w - + def num_args(self): + return self.args.num_args() + 1 + + def num_kwds(self): + return self.args.num_kwds() + class Arguments(AbstractArguments): """ Collects the arguments of a function call. @@ -120,6 +125,14 @@ self.w_stararg = w_stararg self.w_starstararg = w_starstararg + def num_args(self): + self._unpack() + return len(self.arguments_w) + + def num_kwds(self): + self._unpack() + return len(self.kwds_w) + def __repr__(self): if self.w_starstararg is not None: return 'Arguments(%s, %s, %s, %s)' % (self.arguments_w, @@ -140,6 +153,11 @@ def unpack(self): "Return a ([w1,w2...], {'kw':w3...}) pair." + self._unpack() + return self.arguments_w, self.kwds_w + + def _unpack(self): + "unpack the *arg and **kwd into w_arguments and kwds_w" # --- unpack the * argument now --- if self.w_stararg is not None: self.arguments_w += self.space.unpackiterable(self.w_stararg) @@ -172,7 +190,6 @@ d[key] = space.getitem(w_starstararg, w_key) self.kwds_w = d self.w_starstararg = None - return self.arguments_w, self.kwds_w def has_keywords(self): return bool(self.kwds_w) or (self.w_starstararg is not None and @@ -209,25 +226,16 @@ ### Parsing for function calls ### - def _unpack_stararg(self): - space = self.space - # If w_stararg is not exactly a tuple, unpack it now: - # self.match_signature() assumes that it can use it directly for - # a matching *arg in the callee's signature. - if self.w_stararg is not None: - if not space.is_w(space.type(self.w_stararg), space.w_tuple): - self.unpack() - def parse(self, fnname, signature, defaults_w=[]): """Parse args and kwargs to initialize a frame according to the signature of code object. """ - self._unpack_stararg() try: return self.match_signature(signature, defaults_w) except ArgErr, e: raise OperationError(self.space.w_TypeError, - self.space.wrap(e.getmsg(self, fnname))) + self.space.wrap(e.getmsg(fnname))) + def match_signature(self, signature, defaults_w=[]): """Parse args and kwargs according to the signature of a code object, @@ -244,13 +252,16 @@ if self.w_stararg is not None: # There is a case where we don't have to unpack() a w_stararg: # if it matches exactly a *arg in the signature. - if len(self.arguments_w) == co_argcount and varargname is not None: + if (len(self.arguments_w) == co_argcount and + varargname is not None and + self.space.is_w(self.space.type(self.w_stararg), + self.space.w_tuple)): pass else: - self.unpack() # sets self.w_stararg to None + self._unpack() # sets self.w_stararg to None # always unpack the ** arguments if self.w_starstararg is not None: - self.unpack() + self._unpack() args_w = self.arguments_w kwds_w = self.kwds_w @@ -301,7 +312,7 @@ else: # shortcut for the non-unpack() case above scope_w.append(self.w_stararg) elif len(args_w) > co_argcount: - raise ArgErrCount(signature, defaults_w, 0) + raise ArgErrCount(self, signature, defaults_w, 0) # collect extra keyword arguments into the **kwarg if kwargname is not None: @@ -314,7 +325,7 @@ raise ArgErrUnknownKwds(remainingkwds_w) if missing: - raise ArgErrCount(signature, defaults_w, missing) + raise ArgErrCount(self, signature, defaults_w, missing) return scope_w ### Argument <-> list of w_objects together with "shape" information @@ -350,29 +361,32 @@ class ArgErr(Exception): - def getmsg(self, args, fnname): + def getmsg(self, fnname): raise NotImplementedError class ArgErrCount(ArgErr): - def __init__(self, signature, defaults_w, missing_args): + def __init__(self, args, signature, defaults_w, missing_args): self.signature = signature self.defaults_w = defaults_w self.missing_args = missing_args - - def getmsg(self, args, fnname): + self.num_args = args.num_args() + self.num_kwds = args.num_kwds() + + def getmsg(self, fnname): + args = None argnames, varargname, kwargname = self.signature - args_w, kwds_w = args.unpack() - if kwargname is not None or (kwds_w and self.defaults_w): + #args_w, kwds_w = args.unpack() + if kwargname is not None or (self.num_kwds and self.defaults_w): msg2 = "non-keyword " if self.missing_args: required_args = len(argnames) - len(self.defaults_w) nargs = required_args - self.missing_args else: - nargs = len(args_w) + nargs = self.num_args else: msg2 = "" - nargs = len(args_w) + len(kwds_w) + nargs = self.num_args + self.num_kwds n = len(argnames) if n == 0: msg = "%s() takes no %sargument (%d given)" % ( @@ -388,7 +402,7 @@ else: msg1 = "at least" n -= defcount - if not kwds_w: # msg "f() takes at least X non-keyword args" + if not self.num_kwds: # msg "f() takes at least X non-keyword args" msg2 = "" # is confusing if no kwd arg actually provided if n == 1: plural = "" @@ -408,7 +422,7 @@ def __init__(self, argname): self.argname = argname - def getmsg(self, args, fnname): + def getmsg(self, fnname): msg = "%s() got multiple values for keyword argument '%s'" % ( fnname, self.argname) @@ -417,16 +431,18 @@ class ArgErrUnknownKwds(ArgErr): def __init__(self, kwds_w): - self.kwds_w = kwds_w + self.kwd_name = '' + self.num_kwds = len(kwds_w) + if self.num_kwds == 1: + self.kwd_name = kwds_w.keys()[0] - def getmsg(self, args, fnname): - kwds_w = self.kwds_w - if len(kwds_w) == 1: + def getmsg(self, fnname): + if self.num_kwds == 1: msg = "%s() got an unexpected keyword argument '%s'" % ( fnname, - kwds_w.keys()[0]) + self.kwd_name) else: msg = "%s() got %d unexpected keyword arguments" % ( fnname, - len(kwds_w)) + self.num_kwds) return msg Modified: pypy/branch/arre-experiments/pypy/rpython/callparse.py ============================================================================== --- pypy/branch/arre-experiments/pypy/rpython/callparse.py (original) +++ pypy/branch/arre-experiments/pypy/rpython/callparse.py Thu Jan 12 19:54:24 2006 @@ -8,28 +8,6 @@ pass -# for parsing call arguments -class RPythonCallsSpace: - """Pseudo Object Space providing almost no real operation. - For the Arguments class: if it really needs other operations, it means - that the call pattern is too complex for R-Python. - """ - def newtuple(self, items): - return NewTupleHolder(items) - - def newdict(self, stuff): - raise CallPatternTooComplex, "'**' argument" - - def unpackiterable(self, it, expected_length=None): - if it.is_tuple(): - items = it.items() - if (expected_length is not None and - expected_length != len(items)): - raise ValueError - return items - raise CallPatternTooComplex, "'*' argument must be a tuple" - - def getrinputs(rtyper, graph): """Return the list of reprs of the input arguments to the 'graph'.""" return [rtyper.bindingrepr(v) for v in graph.getargs()] @@ -69,7 +47,7 @@ try: holders = arguments.match_signature(signature, defs_h) except ArgErr, e: - raise TyperError, "signature mismatch: %s" % e.getmsg(arguments, graph.name) + raise TyperError, "signature mismatch: %s" % e.getmsg(graph.name) assert len(holders) == len(rinputs), "argument parsing mismatch" vlist = [] @@ -171,3 +149,32 @@ r_tup, v_tuple = self.holder.access(hop) v = r_tup.getitem(hop, v_tuple, index) return hop.llops.convertvar(v, r_tup.items_r[index], repr) + +# for parsing call arguments +class RPythonCallsSpace: + """Pseudo Object Space providing almost no real operation. + For the Arguments class: if it really needs other operations, it means + that the call pattern is too complex for R-Python. + """ + w_tuple = NewTupleHolder + def newtuple(self, items): + return NewTupleHolder(items) + + def newdict(self, stuff): + raise CallPatternTooComplex, "'**' argument" + + def unpackiterable(self, it, expected_length=None): + if it.is_tuple(): + items = it.items() + if (expected_length is not None and + expected_length != len(items)): + raise ValueError + return items + raise CallPatternTooComplex, "'*' argument must be a tuple" + + def is_w(self, one, other): + return one is other + + def type(self, item): + return type(item) + Modified: pypy/branch/arre-experiments/pypy/translator/tool/cbuild.py ============================================================================== --- pypy/branch/arre-experiments/pypy/translator/tool/cbuild.py (original) +++ pypy/branch/arre-experiments/pypy/translator/tool/cbuild.py Thu Jan 12 19:54:24 2006 @@ -271,14 +271,15 @@ self.libraries.append('m') if 'pthread' not in self.libraries: self.libraries.append('pthread') - self.compile_extra += ['-O2', '-pthread'] - self.link_extra += ['-pthread'] + self.compile_extra += ['-g', '-O2', '-pthread'] + self.link_extra += ['-g', '-pthread'] if sys.platform == 'darwin': if '/sw/include' not in self.include_dirs: self.include_dirs.append('/sw/include') if '/sw/lib' not in self.library_dirs: self.library_dirs.append('/sw/lib') - self.compile_extra += ['-O2'] + self.compile_extra += ['-g', '-O2'] + self.link_extra += ['-g'] if outputfilename is None: self.outputfilename = py.path.local(cfilenames[0]).new(ext=ext) From cfbolz at codespeak.net Thu Jan 12 21:20:52 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Thu, 12 Jan 2006 21:20:52 +0100 (CET) Subject: [pypy-svn] r22075 - in pypy/dist/pypy/rpython: . test Message-ID: <20060112202052.DCAFB27B6C@code1.codespeak.net> Author: cfbolz Date: Thu Jan 12 21:20:51 2006 New Revision: 22075 Modified: pypy/dist/pypy/rpython/llinterp.py pypy/dist/pypy/rpython/test/test_llinterp.py Log: make alloca (flavored_malloc(stack, ...)) work in llinterp. also invalidify the object when leaving the stack as a safety check. Modified: pypy/dist/pypy/rpython/llinterp.py ============================================================================== --- pypy/dist/pypy/rpython/llinterp.py (original) +++ pypy/dist/pypy/rpython/llinterp.py Thu Jan 12 21:20:51 2006 @@ -3,6 +3,7 @@ from pypy.rpython.lltypesystem import lltype from pypy.rpython.memory import lladdress from pypy.rpython.ootypesystem import ootype +from pypy.rpython.objectmodel import FREED_OBJECT import sys import math @@ -100,6 +101,7 @@ self.f_back = f_back self.curr_block = None self.curr_operation_index = 0 + self.alloca_objects = [] # _______________________________________________________ # variable setters/getters helpers @@ -156,6 +158,9 @@ nextblock, args = self.eval_block(nextblock) if nextblock is None: self.llinterpreter.active_frame = self.f_back + for obj in self.alloca_objects: + #XXX slighly unclean + obj._setobj(None) return args def eval_block(self, block): @@ -353,6 +358,13 @@ def op_flavored_malloc(self, flavor, obj): assert isinstance(flavor, str) + if flavor == "stack": + if isinstance(obj, self.llt.Struct) and obj._arrayfld is None: + result = self.llt.malloc(obj) + self.alloca_objects.append(result) + return result + else: + raise ValueError("cannot allocate variable-sized things on the stack") return self.llt.malloc(obj, flavor=flavor) def op_flavored_free(self, flavor, obj): Modified: pypy/dist/pypy/rpython/test/test_llinterp.py ============================================================================== --- pypy/dist/pypy/rpython/test/test_llinterp.py (original) +++ pypy/dist/pypy/rpython/test/test_llinterp.py Thu Jan 12 21:20:51 2006 @@ -1,8 +1,9 @@ import py -from pypy.rpython.lltypesystem.lltype import typeOf, pyobjectptr, Ptr, PyObject +from pypy.rpython.lltypesystem.lltype import typeOf, pyobjectptr, Ptr, PyObject, Void from pypy.rpython.rtyper import RPythonTyper from pypy.rpython.llinterp import LLInterpreter, LLException,log +from pypy.rpython.rmodel import inputconst from pypy.translator.translator import TranslationContext from pypy.rpython.rlist import * from pypy.rpython.rint import signed_repr @@ -384,6 +385,40 @@ for j in [0, 1]: result = interpret(getids, [i, j]) assert result + +def test_stack_malloc(): + class A(object): + pass + def f(): + a = A() + a.i = 1 + return a.i + interp, graph = get_interpreter(f, []) + graph.startblock.operations[0].opname = "flavored_malloc" + graph.startblock.operations[0].args.insert(0, inputconst(Void, "stack")) + result = interp.eval_graph(graph, []) + assert result == 1 + +def test_invalid_stack_access(): + class A(object): + pass + globala = A() + globala.next = None + globala.i = 1 + def g(a): + globala.next = a + def f(): + a = A() + a.i = 2 + g(a) + def h(): + f() + return globala.next.i + interp, graph = get_interpreter(h, []) + fgraph = graph.startblock.operations[0].args[0].value._obj.graph + fgraph.startblock.operations[0].opname = "flavored_malloc" + fgraph.startblock.operations[0].args.insert(0, inputconst(Void, "stack")) + py.test.raises(AttributeError, "interp.eval_graph(graph, [])") #__________________________________________________________________ # example functions for testing the LLInterpreter _snap = globals().copy() From ericvrp at codespeak.net Thu Jan 12 21:31:36 2006 From: ericvrp at codespeak.net (ericvrp at codespeak.net) Date: Thu, 12 Jan 2006 21:31:36 +0100 (CET) Subject: [pypy-svn] r22078 - in pypy/dist/pypy/translator/llvm: . module Message-ID: <20060112203136.A4B4927B70@code1.codespeak.net> Author: ericvrp Date: Thu Jan 12 21:31:29 2006 New Revision: 22078 Modified: pypy/dist/pypy/translator/llvm/exception.py pypy/dist/pypy/translator/llvm/funcnode.py pypy/dist/pypy/translator/llvm/gc.py pypy/dist/pypy/translator/llvm/module/genexterns.c pypy/dist/pypy/translator/llvm/module/support.py Log: WIP fixes and todo markers for making genllvm work on ia64 Modified: pypy/dist/pypy/translator/llvm/exception.py ============================================================================== --- pypy/dist/pypy/translator/llvm/exception.py (original) +++ pypy/dist/pypy/translator/llvm/exception.py Thu Jan 12 21:31:29 2006 @@ -168,7 +168,11 @@ codewriter.unwind() def llc_options(self): - return '-enable-correct-eh-support' + if sys.platform == 'linux2' and sys.maxint == 2**64-1: + s = ' -enable-ia64-dag-isel' + else: + s = '' + return '-enable-correct-eh-support' + s class ExplicitExceptionPolicy(ExceptionPolicy): """ uses issubclass() and last_exception tests after each call """ @@ -347,5 +351,9 @@ codewriter.ret(returntype, self._noresult2(returntype)) def llc_options(self): - return '' + if sys.platform == 'linux2' and sys.maxint == 2**64-1: + s = '-enable-ia64-dag-isel' + else: + s = '' + return s Modified: pypy/dist/pypy/translator/llvm/funcnode.py ============================================================================== --- pypy/dist/pypy/translator/llvm/funcnode.py (original) +++ pypy/dist/pypy/translator/llvm/funcnode.py Thu Jan 12 21:31:29 2006 @@ -72,7 +72,9 @@ def post_setup_transform(self): self.db.exceptionpolicy.transform(self.db.translator, self.graph) - remove_exception_mallocs(self.db.translator, self.graph, self.ref) + import sys + if sys.maxint == 2**31-1: #XXX not yet 64bit compatible + remove_exception_mallocs(self.db.translator, self.graph, self.ref) remove_double_links(self.db.translator, self.graph) def writedecl(self, codewriter): Modified: pypy/dist/pypy/translator/llvm/gc.py ============================================================================== --- pypy/dist/pypy/translator/llvm/gc.py (original) +++ pypy/dist/pypy/translator/llvm/gc.py Thu Jan 12 21:31:29 2006 @@ -130,7 +130,7 @@ if is_atomic: t += ''' - call ccc void %%llvm.memset(sbyte* %%malloc_Ptr%(cnt)s, ubyte 0, uint %%malloc_SizeU%(cnt)s, uint 0) + call ccc void %%llvm.memset(sbyte* %%malloc_Ptr%(cnt)s, ubyte 0, %(uword)s %%malloc_SizeU%(cnt)s, uint 0) ''' % locals() codewriter.write_lines(t) Modified: pypy/dist/pypy/translator/llvm/module/genexterns.c ============================================================================== --- pypy/dist/pypy/translator/llvm/module/genexterns.c (original) +++ pypy/dist/pypy/translator/llvm/module/genexterns.c Thu Jan 12 21:31:29 2006 @@ -1,7 +1,7 @@ // we hand craft these in module/support.ll char *RPyString_AsString(RPyString*); -int RPyString_Size(RPyString*); +int RPyString_Size(RPyString*); //XXX int->long RPyString *RPyString_FromString(char *); int RPyExceptionOccurred(void); char* LLVM_RPython_StartupCode(void); Modified: pypy/dist/pypy/translator/llvm/module/support.py ============================================================================== --- pypy/dist/pypy/translator/llvm/module/support.py (original) +++ pypy/dist/pypy/translator/llvm/module/support.py Thu Jan 12 21:31:29 2006 @@ -17,7 +17,7 @@ internal fastcc int %RPyString_Size(%RPyString* %structstring) { %sizeptr = getelementptr %RPyString* %structstring, int 0, uint 1, uint 0 - %size = load int* %sizeptr + %size = load int* %sizeptr ;XXX int ->long ret int %size } From ericvrp at codespeak.net Thu Jan 12 21:36:49 2006 From: ericvrp at codespeak.net (ericvrp at codespeak.net) Date: Thu, 12 Jan 2006 21:36:49 +0100 (CET) Subject: [pypy-svn] r22079 - pypy/dist/pypy/translator/llvm Message-ID: <20060112203649.B3DB727B6C@code1.codespeak.net> Author: ericvrp Date: Thu Jan 12 21:36:46 2006 New Revision: 22079 Modified: pypy/dist/pypy/translator/llvm/exception.py Log: oops Modified: pypy/dist/pypy/translator/llvm/exception.py ============================================================================== --- pypy/dist/pypy/translator/llvm/exception.py (original) +++ pypy/dist/pypy/translator/llvm/exception.py Thu Jan 12 21:36:46 2006 @@ -168,7 +168,7 @@ codewriter.unwind() def llc_options(self): - if sys.platform == 'linux2' and sys.maxint == 2**64-1: + if sys.platform == 'linux2' and sys.maxint == 2**63-1: s = ' -enable-ia64-dag-isel' else: s = '' @@ -351,7 +351,7 @@ codewriter.ret(returntype, self._noresult2(returntype)) def llc_options(self): - if sys.platform == 'linux2' and sys.maxint == 2**64-1: + if sys.platform == 'linux2' and sys.maxint == 2**63-1: s = '-enable-ia64-dag-isel' else: s = '' From cfbolz at codespeak.net Thu Jan 12 22:06:43 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Thu, 12 Jan 2006 22:06:43 +0100 (CET) Subject: [pypy-svn] r22082 - in pypy/dist/pypy/translator/backendopt: . test Message-ID: <20060112210643.93B0527B68@code1.codespeak.net> Author: cfbolz Date: Thu Jan 12 22:06:42 2006 New Revision: 22082 Modified: pypy/dist/pypy/translator/backendopt/escape.py pypy/dist/pypy/translator/backendopt/test/test_escape.py Log: actually implement a transformation that removes puts structures on the stack - not safe yet. Modified: pypy/dist/pypy/translator/backendopt/escape.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/escape.py (original) +++ pypy/dist/pypy/translator/backendopt/escape.py Thu Jan 12 22:06:42 2006 @@ -2,6 +2,7 @@ from pypy.objspace.flow.model import Variable, Constant from pypy.rpython.lltypesystem import lltype from pypy.translator.simplify import get_graph +from pypy.rpython.rmodel import inputconst class CreationPoint(object): def __init__(self, creation_method="?"): @@ -147,6 +148,7 @@ self.curr_block = block self.curr_graph = graph print "inputargs", self.getstates(block.inputargs) + for op in block.operations: self.flow_operation(op) print "checking exits..." @@ -313,13 +315,14 @@ def ptr_iszero(self, op, ptrstate): return None - cast_ptr_to_int = ptr_iszero + cast_ptr_to_int = keepalive = ptr_nonzero = ptr_iszero + + def ptr_eq(self, op, ptr1state, ptr2state): + return None def same_as(self, op, objstate): return objstate - - def isonheap(var_or_const): return isinstance(var_or_const.concretetype, lltype.Ptr) @@ -331,3 +334,20 @@ elif not a.contains(b): return False return True + +def malloc_to_stack(t): + aib = AbstractDataFlowInterpreter(t) + for graph in t.graphs: + for block in graph.iterblocks(): + for op in block.operations: + if op.opname == 'malloc': + if graph.startblock not in aib.flown_blocks: + aib.schedule_function(graph) + aib.complete() + varstate = aib.getstate(op.result) + assert len(varstate.creation_points) == 1 + crep = varstate.creation_points.keys()[0] + if not crep.escapes: + print "moving object from heap to stack %s in %s" % (op, graph.name) + op.opname = 'flavored_malloc' + op.args.insert(0, inputconst(lltype.Void, 'stack')) Modified: pypy/dist/pypy/translator/backendopt/test/test_escape.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/test/test_escape.py (original) +++ pypy/dist/pypy/translator/backendopt/test/test_escape.py Thu Jan 12 22:06:42 2006 @@ -1,5 +1,6 @@ from pypy.translator.translator import TranslationContext, graphof -from pypy.translator.backendopt.escape import AbstractDataFlowInterpreter +from pypy.translator.backendopt.escape import AbstractDataFlowInterpreter, malloc_to_stack +from pypy.rpython.llinterp import LLInterpreter def build_adi(function, types): t = TranslationContext() @@ -11,6 +12,23 @@ adi.complete() return t, adi, graph +def check_malloc_removal(function, types, args, expected_result, must_remove=True): + t = TranslationContext() + t.buildannotator().build_types(function, types) + t.buildrtyper().specialize() + interp = LLInterpreter(t.rtyper) + graph = graphof(t, function) + res = interp.eval_graph(graph, args) + assert res == expected_result + malloc_to_stack(t) + if must_remove: + for block in graph.iterblocks(): + for op in block.operations: + assert op.opname != "malloc" + res = interp.eval_graph(graph, args) + assert res == expected_result + return t + def test_simple(): class A(object): pass @@ -64,7 +82,6 @@ assert crep.changes assert not crep.escapes avarinloop = graph.startblock.exits[0].target.inputargs[1] - #t.view() state1 = adi.getstate(avarinloop) assert crep in state1.creation_points assert len(state1.creation_points) == 2 @@ -126,7 +143,6 @@ def test_call(): class A(object): pass - globala = A() def g(b): return b.i + 2 def f(): @@ -290,10 +306,60 @@ # does not crash t, adi, graph = build_adi(f, [int]) +def test_raise_escapes(): + def f(): + a = ValueError() + raise a + t, adi, graph = build_adi(f, []) + avar = graph.startblock.operations[0].result + state = adi.getstate(avar) + assert state.does_escape() + assert state.does_change() + + def test_big(): from pypy.translator.goal.targetrpystonex import make_target_definition entrypoint, _, _ = make_target_definition(10) # does not crash t, adi, graph = build_adi(entrypoint, [int]) - + +#__________________________________________________________ +# malloc removal tests + +def test_remove_simple(): + class A(object): + pass + def f(): + a = A() + a.x = 1 + return a.x + check_malloc_removal(f, [], [], 1) + +def test_remove_aliasing(): + class A: + pass + def fn6(n): + a1 = A() + a1.x = 5 + a2 = A() + a2.x = 6 + if n > 0: + a = a1 + else: + a = a2 + a.x = 12 + return a1.x + t = check_malloc_removal(fn6, [int], [2], 12) + +def test_remove_call(): + class A(object): + pass + def g(b): + return b.i + 2 + def f(): + a = A() + a.i = 2 + return g(a) + t = check_malloc_removal(f, [], [], 4) + From arigo at codespeak.net Fri Jan 13 10:13:27 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Fri, 13 Jan 2006 10:13:27 +0100 (CET) Subject: [pypy-svn] r22096 - pypy/dist/pypy/rpython Message-ID: <20060113091327.7BC3C27B55@code1.codespeak.net> Author: arigo Date: Fri Jan 13 10:13:25 2006 New Revision: 22096 Modified: pypy/dist/pypy/rpython/rlist.py Log: Two lines no longer used. Modified: pypy/dist/pypy/rpython/rlist.py ============================================================================== --- pypy/dist/pypy/rpython/rlist.py (original) +++ pypy/dist/pypy/rpython/rlist.py Fri Jan 13 10:13:25 2006 @@ -518,8 +518,6 @@ def dum_checkidx(): pass def dum_nocheck(): pass -def dum_inplace():pass -def dum_newlist():pass def ll_pop_nonneg(func, l, index): if func is dum_checkidx and (index >= l.length): From ale at codespeak.net Fri Jan 13 12:47:00 2006 From: ale at codespeak.net (ale at codespeak.net) Date: Fri, 13 Jan 2006 12:47:00 +0100 (CET) Subject: [pypy-svn] r22103 - in pypy/dist/pypy/lib/pyontology: . test Message-ID: <20060113114700.3A2D727B55@code1.codespeak.net> Author: ale Date: Fri Jan 13 12:46:59 2006 New Revision: 22103 Modified: pypy/dist/pypy/lib/pyontology/pyontology.py pypy/dist/pypy/lib/pyontology/test/test_ontology.py Log: Refactor the way "type" works. The goal is to get rid of the make_var function/cludge. To be able to do this I need to get rid of BinaryExpressions. Changed the tests accordingly Modified: pypy/dist/pypy/lib/pyontology/pyontology.py ============================================================================== --- pypy/dist/pypy/lib/pyontology/pyontology.py (original) +++ pypy/dist/pypy/lib/pyontology/pyontology.py Fri Jan 13 12:46:59 2006 @@ -160,14 +160,13 @@ the property is saved in the Restriction class' property attribute 2. The restriction itself. This comes from one of the property restrictions triples (oneOf, maxCardinality ....). It adds a constraint class - 3. The class in which context the restriction should be applied. This comes from subClassOf, - The class is saved in the restrictions clsattribute + 3. The class in which context the restriction should be applied. This comes from subClassOf, type... + The class is saved in the restrictions cls attribute """ def __init__(self, name='', values=[], bases = []): ClassDomain.__init__(self, name, values, bases) self.constraint = RestrictionConstraint(name) self.property = None - self.cls = None builtin_voc = { getUriref('owl', 'Thing') : Thing, @@ -267,7 +266,7 @@ # var is not one of the builtin classes avar = self.make_var(ClassDomain, var) svar = self.make_var(self.variables[avar].__class__, s) - constrain = BinaryExpression([svar, avar],"%s in %s" %(svar, avar)) + constrain = TypeConstraint(svar, avar) self.constraints.append(constrain) else: # var is a builtin class @@ -388,22 +387,20 @@ def maxCardinality(self, s, var): """ Len of finite domain of the property shall be less than or equal to var""" svar =self.make_var(Restriction, s) - constrain = MaxCardinality(svar, None, int(var)) + constrain = MaxCardinality(svar, int(var)) self.constraints.append(constrain) def minCardinality(self, s, var): """ Len of finite domain of the property shall be greater than or equal to var""" - avar = self.find_property(s) - cls =self.make_var(None, self.find_cls(s)) - constrain = MinCardinality(avar, None, int(var)) + svar =self.make_var(Restriction, s) + constrain = MinCardinality(svar, int(var)) self.constraints.append(constrain) def cardinality(self, s, var): """ Len of finite domain of the property shall be equal to var""" - avar = self.find_property(s) - cls =self.make_var(None, self.find_cls(s)) + svar =self.make_var(Restriction, s) # Check if var is an int, else find the int buried in the structure - constrain = Cardinality(avar, None, int(var)) + constrain = Cardinality(svar, int(var)) self.constraints.append(constrain) def differentFrom(self, s, var): @@ -472,7 +469,6 @@ self.cost = 80 self.variable = variable self.cardinality = cardinality - #self.cls = cls def __repr__(self): return '<%s %s %i>' % (self.__class__.__name__, str(self._variables[0]), self.cardinality) @@ -484,7 +480,7 @@ """narrowing algorithm for the constraint""" prop = domains[self.variable].property props = Linkeddict(domains[prop].getValues()) - cls = domains[self.variable].cls + cls = domains[self.variable].getValues()[0] if len(props[cls]) > self.cardinality: raise ConsistencyFailure("Maxcardinality of %i exceeded by the value %i" %(self.cardinality,len(props[cls]))) else: @@ -496,7 +492,7 @@ """narrowing algorithm for the constraint""" prop = domains[self.variable].property props = Linkeddict(domains[prop].getValues()) - cls = domains[self.variable].cls + cls = domains[self.variable].getValues()[0] if len(props[cls]) < self.cardinality: raise ConsistencyFailure("MinCardinality of %i not achieved by the value %i" %(self.cardinality,len(props[cls]))) else: @@ -508,7 +504,7 @@ """narrowing algorithm for the constraint""" prop = domains[self.variable].property props = Linkeddict(domains[prop].getValues()) - cls = domains[self.variable].cls + cls = domains[self.variable].getValues()[0] if len(props[cls]) != self.cardinality: raise ConsistencyFailure("Cardinality of %i exceeded by the value %i" %(self.cardinality,len(props[cls]))) else: @@ -544,12 +540,11 @@ def narrow(self, domains): subdom = domains[self.variable] superdom = domains[self.object] - if isinstance(superdom, Restriction): - superdom.cls = self.variable - bases = get_values(superdom, domains, 'getBases') - subdom.bases += [bas for bas in bases if bas not in subdom.bases] - vals = get_values(subdom, domains, 'getValues') - superdom.values += [val for val in vals if val not in superdom.values] + vals = [] + vals += superdom.getValues() + vals += subdom.getValues() +[self.variable] + superdom.setValues(vals) + return 0 class DisjointClassConstraint(SubClassConstraint): @@ -633,6 +628,17 @@ if not val in vals: raise ConsistencyFailure("Value not in prescribed range") +class TypeConstraint(SubClassConstraint): + cost = 1 + def narrow(self, domains): + subdom = domains[self.variable] + superdom = domains[self.object] + vals = [] + vals += superdom.getValues() + vals.append(self.variable) + superdom.setValues(vals) + return 1 + class FunctionalCardinality(OwlConstraint): """Contraint: all values must be distinct""" @@ -736,10 +742,15 @@ def narrow(self, domains): prop = domains[self.variable].property - cls = domains[self.variable].cls - props = domains[prop].getValues() - props.append((cls, None)) - domains[prop].setValues(props) + vals = domains[self.variable].getValues() + if vals: + cls = vals[0] + props = domains[prop].getValues() + props.append((cls, None)) + domains[prop].setValues(props) + return 1 + else: + return 0 class OneofPropertyConstraint(AbstractConstraint): @@ -757,7 +768,7 @@ val = domains[self.List].getValues() if isinstance(domains[self.variable],Restriction): property = domains[self.variable].property - cls = domains[self.variable].cls + cls = domains[self.variable].getValues()[0] prop = Linkeddict(domains[property].getValues()) for v in prop[cls]: if not v in val: @@ -806,7 +817,7 @@ def narrow(self, domains): val = domains[self.List].getValues() property = domains[self.variable].property - cls = domains[self.variable].cls + cls = domains[self.variable].getValues()[0] prop = Linkeddict(domains[property].getValues()) for v in prop[cls]: if v in val: @@ -823,7 +834,7 @@ def narrow(self, domains): val = domains[self.List].getValues() property = domains[self.variable].property - cls = domains[self.variable].cls + cls = domains[self.variable].getValues()[0] prop = Linkeddict(domains[property].getValues()) for v in prop[cls]: if not v in val: @@ -846,7 +857,7 @@ def narrow(self, domains): val = self.List property = domains[self.variable].property - cls = domains[self.variable].cls + cls = domains[self.variable].getValues()[0] prop = Linkeddict(domains[property].getValues()) for v in prop[cls]: if v == val: Modified: pypy/dist/pypy/lib/pyontology/test/test_ontology.py ============================================================================== --- pypy/dist/pypy/lib/pyontology/test/test_ontology.py (original) +++ pypy/dist/pypy/lib/pyontology/test/test_ontology.py Fri Jan 13 12:46:59 2006 @@ -37,27 +37,24 @@ def test_subClassof(): O = Ontology() - a = URIRef(u'http://www.w3.org/2002/03owlt/unionOf/premises004#A') - b = URIRef(u'http://www.w3.org/2002/03owlt/unionOf/premises004#B') - c = URIRef(u'http://www.w3.org/2002/03owlt/unionOf/premises004#C') - A = O.make_var(ClassDomain, a) - C = O.make_var(ClassDomain, c) - C = O.make_var(ClassDomain, b) + a = URIRef(u'A') + b = URIRef(u'B') + c = URIRef(u'C') O.subClassOf(b, a) O.subClassOf(c, b) - for con in O.constraints: - con.narrow(O.variables) + obj = URIRef(namespaces['owl']+'#Class') + O.type(a,obj) + O.consistency() + O.consistency() assert len(O.variables) == 3 - assert O.variables[A] in O.variables[C].bases + assert 'C_' in O.variables['A_'].getValues() -def test_ClassDomain(): +def no_test_ClassDomain(): a = ClassDomain() - assert a.bases == [a] cls = 1 b = ClassDomain('B',[],[a]) - assert b in b.bases - assert a in b.bases - assert len(b.bases) == 2 + assert b in b.getValues() + assert a in b.getValues() def test_subClassconstraint(): a = ClassDomain('A') @@ -67,9 +64,9 @@ con2 = SubClassConstraint('c','b') con.narrow({'a': a, 'b': b, 'c': c}) con2.narrow({'a': a, 'b': b, 'c': c}) - assert a in c.bases - assert b in c.bases - assert c in c.bases + con.narrow({'a': a, 'b': b, 'c': c}) + assert 'b' in a.getValues() + assert 'c' in a.getValues() def test_subClassconstraintMulti(): a = ClassDomain('A') @@ -79,9 +76,8 @@ con2 = SubClassConstraint('c','b') con.narrow({'a': a, 'b': b, 'c': c}) con2.narrow({'a': a, 'b': b, 'c': c}) - assert a in c.bases - assert b in c.bases - assert c in c.bases + assert 'c' in a.getValues() + assert 'c' in b.getValues() def test_subClassconstraintMulti2(): a = ClassDomain('A') @@ -93,12 +89,9 @@ con.narrow({'a': a, 'b': b, 'c': c}) con2.narrow({'a': a, 'b': b, 'c': c}) con3.narrow({'a': a, 'b': b, 'c': c}) - assert a in c.bases - assert b in c.bases - assert c in c.bases - assert c in a.bases - assert len(c.bases) == len(a.bases) - assert [bas in a.bases for bas in c.bases] == [True]*len(a.bases) + assert 'c' in a.getValues() + assert 'c' in b.getValues() + assert 'a' in c.getValues() def test_equivalentClass(): a = URIRef('A') From ale at codespeak.net Fri Jan 13 13:54:37 2006 From: ale at codespeak.net (ale at codespeak.net) Date: Fri, 13 Jan 2006 13:54:37 +0100 (CET) Subject: [pypy-svn] r22108 - in pypy/dist/pypy/lib/pyontology: . test Message-ID: <20060113125437.7E57D27B5A@code1.codespeak.net> Author: ale Date: Fri Jan 13 13:54:36 2006 New Revision: 22108 Modified: pypy/dist/pypy/lib/pyontology/pyontology.py pypy/dist/pypy/lib/pyontology/test/test_ontology.py Log: Changed differentFrom and added extra test. Modified: pypy/dist/pypy/lib/pyontology/pyontology.py ============================================================================== --- pypy/dist/pypy/lib/pyontology/pyontology.py (original) +++ pypy/dist/pypy/lib/pyontology/pyontology.py Fri Jan 13 13:54:36 2006 @@ -1,6 +1,6 @@ from rdflib import Graph, URIRef, BNode, Literal from logilab.constraint import Repository, Solver -from logilab.constraint.fd import Equals, AllDistinct, BinaryExpression, Expression +from logilab.constraint.fd import BinaryExpression from logilab.constraint.fd import FiniteDomain as fd from logilab.constraint.propagation import AbstractDomain, AbstractConstraint, ConsistencyFailure import sys @@ -404,9 +404,9 @@ self.constraints.append(constrain) def differentFrom(self, s, var): - s_var = self.make_var(ClassDomain, s) + s_var = self.make_var(Thing, s) var_var = self.make_var(Thing, var) - constrain = BinaryExpression([s_var, var_var],"%s != %s" %(s_var, var_var)) + constrain = DifferentfromConstraint(s_var, var_var) self.constraints.append(constrain) #XXX need to change this @@ -717,6 +717,15 @@ if not (val,cls) in obj_domain: raise ConsistencyFailure("Inverseof failed") +class DifferentfromConstraint(SubClassConstraint): + + def narrow(self, domains): + if self.variable == self.object: + raise ConsistencyFailure("%s can't be differentFrom itself" % self.variable) + else: + return 0 + + class ListConstraint(OwlConstraint): """Contraint: all values must be distinct""" Modified: pypy/dist/pypy/lib/pyontology/test/test_ontology.py ============================================================================== --- pypy/dist/pypy/lib/pyontology/test/test_ontology.py (original) +++ pypy/dist/pypy/lib/pyontology/test/test_ontology.py Fri Jan 13 13:54:36 2006 @@ -462,12 +462,19 @@ cls = BNode('anon') own1 = BNode('liist1') own2 = BNode('liist2') - list1 = rdf_list(O, 'favlist1', ['1', '2', '3']) - list2 = rdf_list(O, 'favlist2', ['3', '4', '5']) - own = rdf_list(O, 'favlist', [own1, own2]) - O.oneOf( own1, list1) - O.oneOf( own2, list2) - O.intersectionOf(cls, own) - O.type(cls, namespaces['owl']+'#Class') + O.differentFrom(cls, own1) + O.differentFrom(own1, own2) + O.differentFrom(cls, own2) + O.differentFrom(own2,cls) + O.type(cls, namespaces['owl']+'#Thing') + O.type(own1, namespaces['owl']+'#Thing') + O.type(own2, namespaces['owl']+'#Thing') O.consistency(3) - assert O.rep._domains[cls].getValues() == ['3'] + assert len(O.rep._constraints) == 4 + +def test_differentfromconsistency(): + O = Ontology() + cls = BNode('anon') + O.differentFrom(cls, cls) + O.type(cls, namespaces['owl']+'#Thing') + py.test.raises(ConsistencyFailure, O.consistency, 3) From ac at codespeak.net Fri Jan 13 14:39:32 2006 From: ac at codespeak.net (ac at codespeak.net) Date: Fri, 13 Jan 2006 14:39:32 +0100 (CET) Subject: [pypy-svn] r22111 - pypy/branch/arre-experiments Message-ID: <20060113133932.AB6FB27B6C@code1.codespeak.net> Author: ac Date: Fri Jan 13 14:39:32 2006 New Revision: 22111 Modified: pypy/branch/arre-experiments/ (props changed) Log: Initialized merge tracking via "svnmerge" with revisions "1-21923" from svn+ssh://codespeak.net/svn/pypy/dist From ericvrp at codespeak.net Fri Jan 13 14:45:38 2006 From: ericvrp at codespeak.net (ericvrp at codespeak.net) Date: Fri, 13 Jan 2006 14:45:38 +0100 (CET) Subject: [pypy-svn] r22112 - in pypy/dist/pypy/translator/llvm: . module test Message-ID: <20060113134538.7CC0D27B68@code1.codespeak.net> Author: ericvrp Date: Fri Jan 13 14:45:28 2006 New Revision: 22112 Modified: pypy/dist/pypy/translator/llvm/build_llvm_module.py pypy/dist/pypy/translator/llvm/exception.py pypy/dist/pypy/translator/llvm/gc.py pypy/dist/pypy/translator/llvm/genllvm.py pypy/dist/pypy/translator/llvm/module/boehm.h pypy/dist/pypy/translator/llvm/module/excsupport.py pypy/dist/pypy/translator/llvm/module/genexterns.c pypy/dist/pypy/translator/llvm/module/support.py pypy/dist/pypy/translator/llvm/test/test_genllvm1.py Log: Fixes to be pass some tests on ia64. Modified: pypy/dist/pypy/translator/llvm/build_llvm_module.py ============================================================================== --- pypy/dist/pypy/translator/llvm/build_llvm_module.py (original) +++ pypy/dist/pypy/translator/llvm/build_llvm_module.py Fri Jan 13 14:45:28 2006 @@ -80,7 +80,7 @@ #ball = str(dirpath.join('%s_all.bc' % b)) #cmds.append("opt %s %s -f -o %s.bc" % (OPTIMIZATION_SWITCHES, ball, b)) - use_gcc = sys.platform == 'linux2' + use_gcc = sys.platform == 'linux2' and sys.maxint == 2**31-1 profile = False cleanup = False Modified: pypy/dist/pypy/translator/llvm/exception.py ============================================================================== --- pypy/dist/pypy/translator/llvm/exception.py (original) +++ pypy/dist/pypy/translator/llvm/exception.py Fri Jan 13 14:45:28 2006 @@ -168,6 +168,7 @@ codewriter.unwind() def llc_options(self): + import sys if sys.platform == 'linux2' and sys.maxint == 2**63-1: s = ' -enable-ia64-dag-isel' else: @@ -351,6 +352,7 @@ codewriter.ret(returntype, self._noresult2(returntype)) def llc_options(self): + import sys if sys.platform == 'linux2' and sys.maxint == 2**63-1: s = '-enable-ia64-dag-isel' else: Modified: pypy/dist/pypy/translator/llvm/gc.py ============================================================================== --- pypy/dist/pypy/translator/llvm/gc.py (original) +++ pypy/dist/pypy/translator/llvm/gc.py Fri Jan 13 14:45:28 2006 @@ -128,9 +128,9 @@ %(targetvar)s = cast sbyte* %%malloc_Ptr%(cnt)s to %(type_)s ''' % locals() - if is_atomic: + if is_atomic: #Boehm aligns on 8 byte boundaries t += ''' - call ccc void %%llvm.memset(sbyte* %%malloc_Ptr%(cnt)s, ubyte 0, %(uword)s %%malloc_SizeU%(cnt)s, uint 0) + call ccc void %%llvm.memset(sbyte* %%malloc_Ptr%(cnt)s, ubyte 0, %(uword)s %%malloc_SizeU%(cnt)s, %(uword)s 8) ''' % locals() codewriter.write_lines(t) Modified: pypy/dist/pypy/translator/llvm/genllvm.py ============================================================================== --- pypy/dist/pypy/translator/llvm/genllvm.py (original) +++ pypy/dist/pypy/translator/llvm/genllvm.py Fri Jan 13 14:45:28 2006 @@ -99,7 +99,12 @@ codewriter, self.filename = self.create_codewriter() self._checkpoint('open file and create codewriter') return codewriter - + + def _set_wordsize(self, s): + s = s.replace('UWORD', self.db.get_machine_uword()) + s = s.replace( 'WORD', self.db.get_machine_word()) + return s + def write_headers(self, codewriter): # write external function headers codewriter.header_comment('External Function Headers') @@ -130,7 +135,7 @@ codewriter.header_comment("Function Prototypes") # write external protos - codewriter.write_lines(extdeclarations) + codewriter.write_lines(self._set_wordsize(extdeclarations)) # write node protos for typ_decl in self.db.getnodes(): @@ -144,7 +149,7 @@ # write external function implementations codewriter.header_comment('External Function Implementation') codewriter.write_lines(self.llexterns_functions) - codewriter.write_lines(extfunctions) + codewriter.write_lines(self._set_wordsize(extfunctions)) self.write_extern_impls(codewriter) self.write_setup_impl(codewriter) Modified: pypy/dist/pypy/translator/llvm/module/boehm.h ============================================================================== --- pypy/dist/pypy/translator/llvm/module/boehm.h (original) +++ pypy/dist/pypy/translator/llvm/module/boehm.h Fri Jan 13 14:45:28 2006 @@ -15,11 +15,11 @@ #define USING_BOEHM_GC -char *pypy_malloc(unsigned int size) { +char *pypy_malloc(unsigned long size) { return GC_MALLOC(size); } -char *pypy_malloc_atomic(unsigned int size) { +char *pypy_malloc_atomic(unsigned long size) { return GC_MALLOC_ATOMIC(size); } Modified: pypy/dist/pypy/translator/llvm/module/excsupport.py ============================================================================== --- pypy/dist/pypy/translator/llvm/module/excsupport.py (original) +++ pypy/dist/pypy/translator/llvm/module/excsupport.py Fri Jan 13 14:45:28 2006 @@ -29,6 +29,10 @@ } ''' % (RINGBUFFER_ENTRY_MAXSIZE, RINGBUFFER_OVERSIZE, RINGBUFFER_SIZE - 1) +import sys +if sys.maxint != 2**31-1: #XXX need to move the ringbuffer code to another level anyway + ringbuffer_decl = ringbuffer_code = '' + invokeunwind_code = ''' ccc %(returntype)s%%__entrypoint__%(entrypointname)s { %%result = invoke %(cconv)s %(returntype)s%%%(entrypointname)s to label %%no_exception except label %%exception Modified: pypy/dist/pypy/translator/llvm/module/genexterns.c ============================================================================== --- pypy/dist/pypy/translator/llvm/module/genexterns.c (original) +++ pypy/dist/pypy/translator/llvm/module/genexterns.c Fri Jan 13 14:45:28 2006 @@ -1,7 +1,7 @@ // we hand craft these in module/support.ll char *RPyString_AsString(RPyString*); -int RPyString_Size(RPyString*); //XXX int->long +long RPyString_Size(RPyString*); RPyString *RPyString_FromString(char *); int RPyExceptionOccurred(void); char* LLVM_RPython_StartupCode(void); @@ -13,7 +13,7 @@ RPyMODF_RESULT *ll_modf_result(double, double); RPySTAT_RESULT *ll_stat_result(int, int, int, int, int, int, int, int, int, int); void RPYTHON_RAISE_OSERROR(int error); -RPyListOfString *_RPyListOfString_New(int); +RPyListOfString *_RPyListOfString_New(long); void _RPyListOfString_SetItem(RPyListOfString *, int, RPyString *); // overflows/zeros/values raising operations @@ -45,7 +45,7 @@ } // raw malloc code -char *raw_malloc(int size) { +char *raw_malloc(long size) { return malloc(size); } @@ -53,7 +53,7 @@ free(ptr); } -void raw_memcopy(char *ptr1, char *ptr2, int size) { +void raw_memcopy(char *ptr1, char *ptr2, long size) { memcpy((void *) ptr2, (void *) ptr1, size); } Modified: pypy/dist/pypy/translator/llvm/module/support.py ============================================================================== --- pypy/dist/pypy/translator/llvm/module/support.py (original) +++ pypy/dist/pypy/translator/llvm/module/support.py Fri Jan 13 14:45:28 2006 @@ -4,8 +4,8 @@ %last_exception_value = internal global %RPYTHON_EXCEPTION* null declare ccc uint %strlen(sbyte*) -declare ccc void %llvm.memset(sbyte*, ubyte, uint, uint) -declare ccc void %llvm.memcpy(sbyte*, sbyte*, uint, uint) +declare ccc void %llvm.memset(sbyte*, ubyte, UWORD, UWORD) +declare ccc void %llvm.memcpy(sbyte*, sbyte*, UWORD, UWORD) """ extfunctions = """ @@ -15,10 +15,10 @@ ret sbyte* %source1 } -internal fastcc int %RPyString_Size(%RPyString* %structstring) { +internal fastcc WORD %RPyString_Size(%RPyString* %structstring) { %sizeptr = getelementptr %RPyString* %structstring, int 0, uint 1, uint 0 - %size = load int* %sizeptr ;XXX int ->long - ret int %size + %size = load WORD* %sizeptr + ret WORD %size } internal fastcc int %RPyExceptionOccurred() { @@ -30,12 +30,13 @@ internal fastcc %RPyString* %RPyString_FromString(sbyte* %s) { %lenu = call ccc uint %strlen(sbyte* %s) - %len = cast uint %lenu to int - %rpy = call fastcc %RPyString* %pypy_RPyString_New__Signed(int %len) + %lenuword = cast uint %lenu to UWORD + %lenword = cast uint %lenu to WORD + %rpy = call fastcc %RPyString* %pypy_RPyString_New__Signed(WORD %lenword) %rpystrptr = getelementptr %RPyString* %rpy, int 0, uint 1, uint 1 %rpystr = cast [0 x sbyte]* %rpystrptr to sbyte* - call ccc void %llvm.memcpy(sbyte* %rpystr, sbyte* %s, uint %lenu, uint 0) + call ccc void %llvm.memcpy(sbyte* %rpystr, sbyte* %s, UWORD %lenuword, UWORD 0) ret %RPyString* %rpy } Modified: pypy/dist/pypy/translator/llvm/test/test_genllvm1.py ============================================================================== --- pypy/dist/pypy/translator/llvm/test/test_genllvm1.py (original) +++ pypy/dist/pypy/translator/llvm/test/test_genllvm1.py Fri Jan 13 14:45:28 2006 @@ -54,13 +54,17 @@ assert shr(42,2) == llvmsnippet.shiftright(42,2) class TestFloat(object): + def _almost_equal(self, a, b): + _epsilon = 0.0000001 + return a >= b - _epsilon and a <= b + _epsilon + def test_float_f1(self): f = compile_function(llvmsnippet.float_f1, [float]) - assert f(1.0) == 2.2 + assert self._almost_equal(f(1.0), 2.2) def test_float_int_bool(self): f = compile_function(llvmsnippet.float_int_bool, [float]) - assert f(3.0) == 9.0 + assert self._almost_equal(f(3.0), 9.0) class TestString(object): From rxe at codespeak.net Sat Jan 14 13:34:59 2006 From: rxe at codespeak.net (rxe at codespeak.net) Date: Sat, 14 Jan 2006 13:34:59 +0100 (CET) Subject: [pypy-svn] r22134 - pypy/extradoc/sprintinfo/mallorca Message-ID: <20060114123459.053E427B44@code1.codespeak.net> Author: rxe Date: Sat Jan 14 13:34:58 2006 New Revision: 22134 Modified: pypy/extradoc/sprintinfo/mallorca/people.txt Log: My details. Modified: pypy/extradoc/sprintinfo/mallorca/people.txt ============================================================================== --- pypy/extradoc/sprintinfo/mallorca/people.txt (original) +++ pypy/extradoc/sprintinfo/mallorca/people.txt Sat Jan 14 13:34:58 2006 @@ -22,6 +22,7 @@ Jacob Hallen ? ? Anders Lehmann 22/1 -29/1 Hotel Almudainal Holger Krekel ? ? +Richard Emslie 22/1 - 29/1 catalonia hotel ==================== ============== ===================== People on the following list were present at previous sprints: @@ -35,7 +36,6 @@ Lene Wagner ? ? Anders Lehmann ? ? Niklaus Haldimann ? ? -Richard Emslie ? ? Johan Hahn ? ? Amaury Forgeot d'Arc ? ? Valentino Volonghi ? ? From arigo at codespeak.net Sat Jan 14 13:37:14 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sat, 14 Jan 2006 13:37:14 +0100 (CET) Subject: [pypy-svn] r22135 - in pypy/dist/pypy: jit jit/test rpython Message-ID: <20060114123714.4B81127B44@code1.codespeak.net> Author: arigo Date: Sat Jan 14 13:37:09 2006 New Revision: 22135 Added: pypy/dist/pypy/jit/test/test_vlist.py (contents, props changed) pypy/dist/pypy/jit/vlist.py (contents, props changed) Modified: pypy/dist/pypy/jit/llabstractinterp.py pypy/dist/pypy/jit/llcontainer.py pypy/dist/pypy/jit/llvalue.py pypy/dist/pypy/jit/test/test_llabstractinterp.py pypy/dist/pypy/rpython/rlist.py Log: Starting LLAbstractInterp support for RPython list objects, virtualized directly instead of as structs and arrays with multiple low-level representations (e.g. in the presence of over-allocation). Experimental and incomplete! Modified: pypy/dist/pypy/jit/llabstractinterp.py ============================================================================== --- pypy/dist/pypy/jit/llabstractinterp.py (original) +++ pypy/dist/pypy/jit/llabstractinterp.py Sat Jan 14 13:37:09 2006 @@ -5,8 +5,10 @@ from pypy.rpython.lltypesystem import lltype from pypy.translator.simplify import eliminate_empty_blocks, join_blocks from pypy.jit.llvalue import LLAbstractValue, const, newvar, dupvar +from pypy.jit.llvalue import ll_no_return_value from pypy.jit.llvalue import FlattenMemo, MatchMemo, FreezeMemo, UnfreezeMemo from pypy.jit.llcontainer import LLAbstractContainer, virtualcontainervalue +from pypy.jit.llcontainer import hasllcontent # ____________________________________________________________ @@ -138,16 +140,16 @@ # ____________________________________________________________ -ll_no_return_value = LLAbstractValue(const(None, lltype.Void)) - class Policy(object): def __init__(self, inlining=False, const_propagate=False, - concrete_propagate=True, concrete_args=True): + concrete_propagate=True, concrete_args=True, + oopspec=False): self.inlining = inlining self.const_propagate = const_propagate self.concrete_propagate = concrete_propagate self.concrete_args = concrete_args + self.oopspec = oopspec # hint-driven policy best_policy = Policy(inlining=True, const_propagate=True, concrete_args=False) @@ -689,6 +691,13 @@ fnobj = v_func.value._obj if not hasattr(fnobj, 'graph'): return None + + if self.interp.policy.oopspec and hasattr(fnobj._callable, 'oopspec'): + a_result = self.handle_highlevel_operation(op, fnobj._callable, + *args_a) + if a_result is not None: + return a_result + if getattr(fnobj._callable, 'suggested_primitive', False): return None @@ -745,7 +754,7 @@ return a_real_result def op_getfield(self, op, a_ptr, a_attrname): - if a_ptr.content is not None: + if hasllcontent(a_ptr): c_attrname = a_attrname.maybe_get_constant() assert c_attrname is not None return a_ptr.content.getfield(c_attrname.value) @@ -756,7 +765,7 @@ return self.residualize(op, [a_ptr, a_attrname], constant_op) def op_getsubstruct(self, op, a_ptr, a_attrname): - if a_ptr.content is not None: + if hasllcontent(a_ptr): c_attrname = a_attrname.maybe_get_constant() assert c_attrname is not None # the difference with op_getfield() is only that the following @@ -765,12 +774,12 @@ return self.residualize(op, [a_ptr, a_attrname], getattr) def op_getarraysize(self, op, a_ptr): - if a_ptr.content is not None: + if hasllcontent(a_ptr): return LLAbstractValue(const(a_ptr.content.length)) return self.residualize(op, [a_ptr], len) def op_getarrayitem(self, op, a_ptr, a_index): - if a_ptr.content is not None: + if hasllcontent(a_ptr): c_index = a_index.maybe_get_constant() if c_index is not None: return a_ptr.content.getfield(c_index.value) @@ -794,7 +803,7 @@ return self.residualize(op, [a_T, a_size]) def op_setfield(self, op, a_ptr, a_attrname, a_value): - if a_ptr.content is not None: + if hasllcontent(a_ptr): c_attrname = a_attrname.maybe_get_constant() assert c_attrname is not None a_ptr.content.setfield(c_attrname.value, a_value) @@ -802,7 +811,7 @@ return self.residualize(op, [a_ptr, a_attrname, a_value]) def op_setarrayitem(self, op, a_ptr, a_index, a_value): - if a_ptr.content is not None: + if hasllcontent(a_ptr): c_index = a_index.maybe_get_constant() if c_index is not None: a_ptr.content.setfield(c_index.value, a_value) @@ -810,7 +819,7 @@ return self.residualize(op, [a_ptr, a_index, a_value]) def op_cast_pointer(self, op, a_ptr): - if a_ptr.content is not None: + if hasllcontent(a_ptr): down_or_up = lltype.castable(op.result.concretetype, a_ptr.content.getconcretetype()) # the following works because if a structure is virtual, then @@ -838,6 +847,50 @@ self.residual_operations.append(op) return ll_no_return_value + # High-level operation dispatcher + def handle_highlevel_operation(self, op, ll_func, *args_a): + # parse the oopspec and fill in the arguments + operation_name, args = ll_func.oopspec.split('(', 1) + assert args.endswith(')') + args = args[:-1] + ',' # trailing comma to force tuple syntax + argnames = ll_func.func_code.co_varnames[:len(args_a)] + d = dict(zip(argnames, args_a)) + argtuple = eval(args, d) + args_a = [] + for a in argtuple: + if not isinstance(a, LLAbstractValue): + a = LLAbstractValue(const(a)) + args_a.append(a) + # end of rather XXX'edly hackish parsing + + if operation_name == 'newlist': + from pypy.jit.vlist import oop_newlist + handler = oop_newlist + else: + # dispatch on the 'self' argument if it is virtual + a_self = args_a[0] + args_a = args_a[1:] + if not isinstance(a_self, LLAbstractContainer): + return None + type_name, operation_name = operation_name.split('.') + if a_self.type_name != type_name: + return None + try: + handler = getattr(a_self, 'oop_' + operation_name) + except AttributeError: + print 'MISSING HANDLER: oop_%s' % (operation_name,) + return None + try: + a_result = handler(op, *args_a) + except NotImplementedError: + return None + if a_result is None: + a_result = ll_no_return_value + assert op.result.concretetype == a_result.getconcretetype(), ( + "type mismatch: %s\nreturned %s\nexpected %s" % ( + handler, a_result.getconcretetype(), op.result.concretetype)) + return a_result + class InsertNextLink(Exception): def __init__(self, link): Modified: pypy/dist/pypy/jit/llcontainer.py ============================================================================== --- pypy/dist/pypy/jit/llcontainer.py (original) +++ pypy/dist/pypy/jit/llcontainer.py Sat Jan 14 13:37:09 2006 @@ -230,3 +230,6 @@ # primitive initialized to zero a = LLAbstractValue(const(T._defl())) return a + +def hasllcontent(a_ptr): + return isinstance(a_ptr.content, LLVirtualContainer) Modified: pypy/dist/pypy/jit/llvalue.py ============================================================================== --- pypy/dist/pypy/jit/llvalue.py (original) +++ pypy/dist/pypy/jit/llvalue.py Sat Jan 14 13:37:09 2006 @@ -24,7 +24,7 @@ def __repr__(self): if self.runtimevar is None: if self.content is None: - return '' + return '' else: return '' % (self.content,) else: @@ -47,7 +47,7 @@ # don't use 'memo' here: for now, shared LLAbstractValues # need to become distinct LLFrozenRuntimeValues. return LLFrozenRuntimeValue(self) - else: + elif self.content is not None: # virtual container: preserve sharing if self in memo.seen: return memo.seen[self] # already seen @@ -56,15 +56,21 @@ memo.seen[self] = result result.fz_content = self.content.freeze(memo) return result + else: + return frozen_dummy_value # dummy def getconcretetype(self): if self.runtimevar is not None: return self.runtimevar.concretetype - else: + elif self.content is not None: return lltype.Ptr(self.content.T) + else: + raise ValueError("ll_dummy_value.getconcretetype()") def forcevarorconst(self, builder): if self.runtimevar is None: + if self.content is None: + raise ValueError("ll_dummy_value.forcevarorconst()") self.runtimevar = self.content.build_runtime_container(builder) self.content = None return self.runtimevar @@ -83,10 +89,12 @@ if not self.concrete: # skip concrete values, they don't need # to be present in the residual graph at all memo.result.append(self) - else: + elif self.content is not None: if self not in memo.seen: memo.seen[self] = True self.content.flatten(memo) + else: + pass # dummy # ____________________________________________________________ @@ -99,6 +107,18 @@ # relevant in the finished block. +class LLFrozenDummyValue(LLFrozenValue): + + def flatten(self, memo): + pass + + def unfreeze(self, memo): + return ll_dummy_value + + def match(self, a_value, memo): + return True # a dummy matches anything + + class LLFrozenConcreteValue(LLFrozenValue): def __init__(self, a_source): @@ -236,3 +256,7 @@ v1 = Variable(v) v1.concretetype = v.concretetype return v1 + +ll_no_return_value = LLAbstractValue(const(None, lltype.Void)) +ll_dummy_value = LLAbstractValue() +frozen_dummy_value = LLFrozenDummyValue() Modified: pypy/dist/pypy/jit/test/test_llabstractinterp.py ============================================================================== --- pypy/dist/pypy/jit/test/test_llabstractinterp.py (original) +++ pypy/dist/pypy/jit/test/test_llabstractinterp.py Sat Jan 14 13:37:09 2006 @@ -54,13 +54,16 @@ result1 = llinterp.eval_graph(graph1, argvalues) result2 = llinterp.eval_graph(graph2, argvalues2) assert result1 == result2 - # return a summary of the instructions left in graph2 + return graph2, summary(interp) + +def summary(interp): + # return a summary of the instructions left in all the residual graphs insns = {} for copygraph in interp.itercopygraphs(): for block in copygraph.iterblocks(): for op in block.operations: insns[op.opname] = insns.get(op.opname, 0) + 1 - return graph2, insns + return insns P_INLINE = Policy(inlining=True) P_CONST_INLINE = Policy(inlining=True, const_propagate=True) Added: pypy/dist/pypy/jit/test/test_vlist.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/jit/test/test_vlist.py Sat Jan 14 13:37:09 2006 @@ -0,0 +1,45 @@ +import py +from pypy.translator.translator import TranslationContext +from pypy.jit.llabstractinterp import LLAbstractInterp, Policy +from pypy.jit.test.test_llabstractinterp import summary +from pypy.rpython.llinterp import LLInterpreter +from pypy.rpython.rstr import string_repr +from pypy.rpython.objectmodel import hint + +policy = Policy(inlining=True, const_propagate=True, concrete_args=False, + oopspec=True) + +def run(fn, argvalues): + t = TranslationContext() + t.buildannotator().build_types(fn, [type(x) for x in argvalues]) + rtyper = t.buildrtyper() + rtyper.specialize() + graph1 = t.graphs[0] + + interp = LLAbstractInterp(policy) + hints = {} + llvalues = [] + for i, value in enumerate(argvalues): + if isinstance(value, str): + value = string_repr.convert_const(value) + llvalues.append(value) + hints[i] = value + graph2 = interp.eval(graph1, hints) + #graph2.show() + + llinterp = LLInterpreter(rtyper) + result1 = llinterp.eval_graph(graph1, llvalues) + result2 = llinterp.eval_graph(graph2, []) + + assert result1 == result2 + + return graph2, summary(interp) + + +def test_newlist(): + py.test.skip("in-progress") + def fn(n): + lst = [5] * n + return len(lst) + graph2, insns = run(fn, [12]) + assert insns == {} Added: pypy/dist/pypy/jit/vlist.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/jit/vlist.py Sat Jan 14 13:37:09 2006 @@ -0,0 +1,83 @@ +from pypy.rpython.lltypesystem import lltype +from pypy.jit.llvalue import LLAbstractValue, newvar, const, ll_dummy_value +from pypy.jit.llcontainer import LLAbstractContainer + + +class LLVirtualList(LLAbstractContainer): + type_name = 'list' + + def __init__(self, T, items_a=None): + self.T = T + if items_a is None: + self.items_a = [] + else: + self.items_a = items_a + + def flatten(self, memo): + assert self not in memo.seen; memo.seen[self] = True # debugging only + for a_value in self.items_a: + a_value.flatten(memo) + + def match(self, live, memo): + if self.__class__ is not live.__class__: + return False + if len(self.items_a) != len(live.items_a): + return False + for a1, a2 in zip(self.items_a, live.items_a): + if not a1.match(a2, memo): + return False + else: + return True + + def freeze(self, memo): + items_a = [a.freeze(memo) for a in self.items_a] + return LLVirtualList(self.T, items_a) + + def unfreeze(self, memo): + items_a = [a.unfreeze(memo) for a in self.items_a] + return LLVirtualList(self.T, items_a) + + def build_runtime_container(self, builder): + cno = const(len(self.items_a)) + v_result = builder.gendirectcall(self.T.ll_newlist, cno) + cdum = const(rlist.dum_nocheck, lltype.Void) + for i, a in enumerate(self.items_a): + ci = const(i) + v_item = a.forcevarorconst(builder) + builder.gendirectcall(rlist.ll_setitem_nonneg, + cdum, v_result, ci, v_item) + return v_result + + # ____________________________________________________________ + # High-level operations + + def oop_getitem(self, op, a_index): + c_index = a_index.maybe_get_constant() + if c_index is None: + raise NotImplementedError + return self.items_a[c_index.value] + + def oop_setitem(self, op, a_index, a_newobj): + c_index = a_index.maybe_get_constant() + if c_index is None: + raise NotImplementedError + self.items_a[c_index.value] = a_newobj + + def oop_append(self, op, a_newobj): + self.items_a.append(a_newobj) + + def oop_pop(self, op, a_index): + c_index = a_index.maybe_get_constant() + if c_index is None: + raise NotImplementedError + return self.items_a.pop(c_index.value) + + +def oop_newlist(op, a_numitems): + c_numitems = a_numitems.maybe_get_constant() + if c_numitems is None: + raise NotImplementedError + LIST = op.result.concretetype.TO + items_a = [ll_dummy_value] * c_numitems.value + virtuallist = LLVirtualList(LIST, items_a) + return LLAbstractValue(content=virtuallist) Modified: pypy/dist/pypy/rpython/rlist.py ============================================================================== --- pypy/dist/pypy/rpython/rlist.py (original) +++ pypy/dist/pypy/rpython/rlist.py Sat Jan 14 13:37:09 2006 @@ -558,6 +558,7 @@ items[newlength] = nullptr(ITEM.TO) _ll_list_resize_le(l, newlength) return res +ll_pop_zero.oopspec = 'list.pop(l, 0)' def ll_pop(func, l, index): length = l.length @@ -861,6 +862,7 @@ l.items = malloc(LIST.items.TO, length) return l ll_newlist = typeMethod(ll_newlist) +ll_newlist.oopspec = 'newlist(length)' def ll_length(l): return l.length @@ -874,6 +876,7 @@ l = malloc(LIST, length) return l ll_fixed_newlist = typeMethod(ll_fixed_newlist) +ll_fixed_newlist.oopspec = 'newlist(length)' def ll_fixed_length(l): return len(l) From pedronis at codespeak.net Sat Jan 14 14:41:43 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Sat, 14 Jan 2006 14:41:43 +0100 (CET) Subject: [pypy-svn] r22142 - in pypy/dist/pypy: annotation rpython rpython/memory rpython/memory/test Message-ID: <20060114134143.86DD827B46@code1.codespeak.net> Author: pedronis Date: Sat Jan 14 14:41:41 2006 New Revision: 22142 Modified: pypy/dist/pypy/annotation/bookkeeper.py pypy/dist/pypy/annotation/builtin.py pypy/dist/pypy/rpython/memory/lladdress.py pypy/dist/pypy/rpython/memory/test/test_address.py pypy/dist/pypy/rpython/objectmodel.py Log: expererimenting with adding offset/sizeof/itemoffsetof to the lltype model. Annotation for offsetof and offsetof constants. Modified: pypy/dist/pypy/annotation/bookkeeper.py ============================================================================== --- pypy/dist/pypy/annotation/bookkeeper.py (original) +++ pypy/dist/pypy/annotation/bookkeeper.py Sat Jan 14 14:41:41 2006 @@ -18,7 +18,7 @@ from pypy.annotation import description from pypy.interpreter.argument import Arguments, ArgErr from pypy.rpython.rarithmetic import r_uint, r_ulonglong, r_longlong -from pypy.rpython.objectmodel import r_dict +from pypy.rpython.objectmodel import r_dict, Symbolic from pypy.tool.algo.unionfind import UnionFind from pypy.rpython.lltypesystem import lltype from pypy.rpython.ootypesystem import ootype @@ -302,6 +302,8 @@ if x is sys: # special case constant sys to someobject return SomeObject() tp = type(x) + if issubclass(tp, Symbolic): # symbolic constants support + return x.annotation() if tp is bool: result = SomeBool() elif tp is int: Modified: pypy/dist/pypy/annotation/builtin.py ============================================================================== --- pypy/dist/pypy/annotation/builtin.py (original) +++ pypy/dist/pypy/annotation/builtin.py Sat Jan 14 14:41:41 2006 @@ -502,6 +502,15 @@ BUILTIN_ANALYZERS[lladdress.raw_memcopy] = raw_memcopy #_________________________________ +# offsetof/sizeof + +def offsetof(TYPE, fldname): + return SomeInteger() + +BUILTIN_ANALYZERS[lladdress.offsetof] = offsetof + + +#_________________________________ # external functions Modified: pypy/dist/pypy/rpython/memory/lladdress.py ============================================================================== --- pypy/dist/pypy/rpython/memory/lladdress.py (original) +++ pypy/dist/pypy/rpython/memory/lladdress.py Sat Jan 14 14:41:41 2006 @@ -119,3 +119,29 @@ "char": lltype.Char, "address": Address, } + +# sizeof, offsetof + +from pypy.rpython.objectmodel import Symbolic + +class OffsetOf(Symbolic): + + def __init__(self, TYPE, fldname): + self.TYPE = TYPE + self.fldname = fldname + + def annotation(self): + from pypy.annotation import model + return model.SomeInteger() + + def __repr__(self): + return "" % (self.TYPE, self.fldname) + +def sizeof(TYPE, n=None): + pass + +def offsetof(TYPE, fldname): + return OffsetOf(TYPE, fldname) + +def itemoffsetof(TYPE, n=None): + pass Modified: pypy/dist/pypy/rpython/memory/test/test_address.py ============================================================================== --- pypy/dist/pypy/rpython/memory/test/test_address.py (original) +++ pypy/dist/pypy/rpython/memory/test/test_address.py Sat Jan 14 14:41:41 2006 @@ -112,6 +112,21 @@ assert not f(0) assert not f(-1) + def test_simple_offsetof(self): + from pypy.rpython.lltypesystem import lltype + from pypy.rpython.memory.lladdress import offsetof + S = lltype.GcStruct(('x', lltype.Bool), ('y', lltype.Signed)) + def f(): + return offsetof(S, 'x') + offsetof(S, 'y') + a = RPythonAnnotator() + s = a.build_types(f, []) + assert s.knowntype == int + coff = offsetof(S, 'y') + def f(): + return coff + a = RPythonAnnotator() + s = a.build_types(f, []) + assert s.knowntype == int class TestAddressRTyping(object): def test_null(self): Modified: pypy/dist/pypy/rpython/objectmodel.py ============================================================================== --- pypy/dist/pypy/rpython/objectmodel.py (original) +++ pypy/dist/pypy/rpython/objectmodel.py Sat Jan 14 14:41:41 2006 @@ -3,6 +3,11 @@ RPython-compliant way. """ +class Symbolic(object): + + def annotation(self): + return None + import new import weakref From hpk at codespeak.net Sat Jan 14 15:59:31 2006 From: hpk at codespeak.net (hpk at codespeak.net) Date: Sat, 14 Jan 2006 15:59:31 +0100 (CET) Subject: [pypy-svn] r22147 - pypy/extradoc/talk/22c3 Message-ID: <20060114145931.AFCB327B42@code1.codespeak.net> Author: hpk Date: Sat Jan 14 15:59:27 2006 New Revision: 22147 Added: pypy/extradoc/talk/22c3/saarbruecken_consortium.jpg (contents, props changed) Modified: pypy/extradoc/talk/22c3/hpk-tech.txt pypy/extradoc/talk/22c3/slides-agility.txt Log: added missing picture and final slides Modified: pypy/extradoc/talk/22c3/hpk-tech.txt ============================================================================== --- pypy/extradoc/talk/22c3/hpk-tech.txt (original) +++ pypy/extradoc/talk/22c3/hpk-tech.txt Sat Jan 14 15:59:27 2006 @@ -35,7 +35,7 @@ PyPy project facts ======================= -- started 2002 as a grass-root effort +- started 2003 as a grass-root effort - aims: flexibility, research, speed - test-driven development - received EU-funding from end 2004 on Added: pypy/extradoc/talk/22c3/saarbruecken_consortium.jpg ============================================================================== Binary file. No diff available. Modified: pypy/extradoc/talk/22c3/slides-agility.txt ============================================================================== --- pypy/extradoc/talk/22c3/slides-agility.txt (original) +++ pypy/extradoc/talk/22c3/slides-agility.txt Sat Jan 14 15:59:27 2006 @@ -2,7 +2,7 @@ Open Source, EU Funding and Agile Methods ============================================================ -:Authors: Bea During (changemaker), Holger Krekel (merlinux) +:Authors: Bea During (Change Maker), Holger Krekel (merlinux) :Date: 29th December 2005 How it got started @@ -23,6 +23,7 @@ - welcomed by python community - based on zen of python / python culture - focus on correctness of concepts, then speed +- evolutionary step by step development Lines of Code and tests ============================================================ @@ -38,6 +39,10 @@ - strong focus on glue & integration esp. with C/other languages - few flaming fights inside / much collaboration - has many windows and unix hackers +- CPython: main Python version (BDFL'ed by Guido) +- Jython: top scripting langauge for Java +- IronPython (MS): compiles to .NET's CLR +- PyPy: self-contained - self-translating - flexible Evolving agility ================================================== @@ -46,6 +51,8 @@ - several projects regularly "sprint" and work together distributedly - community conference EuroPython in 2002 (now yearly) - many test tools and methods available +- next EuroPython 2006: June at CERN (Geneva) +- ... with dedicated agile development track PyPy test-driven development ================================================== @@ -328,7 +335,7 @@ - improve interactions with community & contribution - taking care about post-EU development (2007++) - visiting Mallorca, Texas, Tokyo, Ireland, ... -- commercial opportunities ... +- commercial opportunities ... hiring opportunities ... - *Questions?* (talk to us ...) http://codespeak.net/pypy and http://pypy.org From arigo at codespeak.net Sat Jan 14 22:56:38 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sat, 14 Jan 2006 22:56:38 +0100 (CET) Subject: [pypy-svn] r22181 - pypy/dist/pypy/doc/discussion Message-ID: <20060114215638.DD9E827B44@code1.codespeak.net> Author: arigo Date: Sat Jan 14 22:56:37 2006 New Revision: 22181 Modified: pypy/dist/pypy/doc/discussion/draft-jit-ideas.txt Log: Check-in a paragraph that is meant to be followed by more. Modified: pypy/dist/pypy/doc/discussion/draft-jit-ideas.txt ============================================================================== --- pypy/dist/pypy/doc/discussion/draft-jit-ideas.txt (original) +++ pypy/dist/pypy/doc/discussion/draft-jit-ideas.txt Sat Jan 14 22:56:37 2006 @@ -83,4 +83,14 @@ Machine code backends and register allocation ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -xxx \ No newline at end of file +xxx + +Alternative: RPython interface to code generation +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +As a first step, to allow experimentation with code generation without +all the difficulties of the L3 graph model and L3 interpreter, we could +provide a set of primitive RPython functions that enable simple code +generation. + +TBD From arigo at codespeak.net Sat Jan 14 23:00:35 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sat, 14 Jan 2006 23:00:35 +0100 (CET) Subject: [pypy-svn] r22182 - in pypy/dist/pypy/rpython: . test Message-ID: <20060114220035.6C9BD27B46@code1.codespeak.net> Author: arigo Date: Sat Jan 14 23:00:32 2006 New Revision: 22182 Modified: pypy/dist/pypy/rpython/rlist.py pypy/dist/pypy/rpython/rtyper.py pypy/dist/pypy/rpython/test/test_llinterp.py pypy/dist/pypy/rpython/test/test_rlist.py Log: The RTyper side of the JIT problem of building real lists from virtual lists at any time. Now each LIST GcStruct has a list_builder() adt-method that produces lloperations that create a corresponding list. The ListBuilder class exists because of some order-of-initialization fun. Modified: pypy/dist/pypy/rpython/rlist.py ============================================================================== --- pypy/dist/pypy/rpython/rlist.py (original) +++ pypy/dist/pypy/rpython/rlist.py Sat Jan 14 23:00:32 2006 @@ -68,8 +68,12 @@ self.external_item_repr, self.item_repr = externalvsinternal(rtyper, item_repr) self.listitem = listitem self.list_cache = {} + self.list_builder = ListBuilder() # setup() needs to be called to finish this initialization - + + def _setup_repr_final(self): + self.list_builder.setup(self) + def recast(self, llops, v): return llops.convertvar(v, self.item_repr, self.external_item_repr) @@ -146,6 +150,46 @@ temp), rstr.list_str_close_bracket)) +class ListBuilder(object): + """Interface to allow lazy list building by the JIT.""" + # This should not keep a reference to the RTyper, even indirectly via + # the list_repr. + + def setup(self, list_repr): + # Precompute the c_newitem and c_setitem_nonneg function pointers, + # needed below. + if list_repr.rtyper is None: + return # only for test_rlist, which doesn't need this anyway + LIST = list_repr.LIST + LISTPTR = list_repr.lowleveltype + ITEM = list_repr.item_repr.lowleveltype + self.LIST = LIST + self.LISTPTR = LISTPTR + + argtypes = [Signed] + fnptr = list_repr.rtyper.annotate_helper_fn(LIST.ll_newlist, argtypes) + self.c_newlist = inputconst(typeOf(fnptr), fnptr) + + bk = list_repr.rtyper.annotator.bookkeeper + argtypes = [bk.immutablevalue(dum_nocheck), LISTPTR, Signed, ITEM] + fnptr = list_repr.rtyper.annotate_helper_fn(ll_setitem_nonneg, argtypes) + self.c_setitem_nonneg = inputconst(typeOf(fnptr), fnptr) + + def build(self, llop, items_v): + """Make the operations that would build a list containing the + provided items.""" + # This is called by the JIT with llop.rtyper == None, so + # llop.gendirectcall() doesn't work. Instead, we need to call + # directly the c_newitem and c_setitem_nonneg precomputed above. + c_list = inputconst(Void, self.LIST) + c_len = inputconst(Signed, len(items_v)) + v_result = llop.genop('direct_call', [self.c_newlist, c_list, c_len], + resulttype = self.LISTPTR) + for i, v in enumerate(items_v): + llop.genop('direct_call', [self.c_setitem_nonneg, v_result, + inputconst(Signed, i), v]) + return v_result + class ListRepr(BaseListRepr): def _setup_repr(self): @@ -161,6 +205,7 @@ "ll_newlist": ll_newlist, "ll_length": ll_length, "ll_items": ll_items, + "list_builder": self.list_builder.build, }) ) @@ -233,6 +278,7 @@ "ll_newlist": ll_fixed_newlist, "ll_length": ll_fixed_length, "ll_items": ll_fixed_items, + "list_builder": self.list_builder.build, }) self.LIST.become(ITEMARRAY) Modified: pypy/dist/pypy/rpython/rtyper.py ============================================================================== --- pypy/dist/pypy/rpython/rtyper.py (original) +++ pypy/dist/pypy/rpython/rtyper.py Sat Jan 14 23:00:32 2006 @@ -538,15 +538,31 @@ return self.type_system.getcallable(graph, getconcretetype) - def annotate_helper(self, ll_function, arglltypes): - """Annotate the given low-level helper function - and return it as a function pointer object. + def annotate_helper(self, ll_function, argtypes): + """Annotate the given low-level helper function and return its graph """ - args_s = [annmodel.lltype_to_annotation(T) for T in arglltypes] + args_s = [] + for s in argtypes: + # assume 's' is a low-level type, unless it is already an annotation + if not isinstance(s, annmodel.SomeObject): + s = annmodel.lltype_to_annotation(s) + args_s.append(s) + # hack for bound methods + if hasattr(ll_function, 'im_func'): + bk = self.annotator.bookkeeper + args_s.insert(0, bk.immutablevalue(ll_function.im_self)) + ll_function = ll_function.im_func helper_graph = annotate_lowlevel_helper(self.annotator, ll_function, args_s) return helper_graph + def annotate_helper_fn(self, ll_function, argtypes): + """Annotate the given low-level helper function + and return it as a function pointer + """ + graph = self.annotate_helper(ll_function, argtypes) + return self.getcallable(graph) + def attachRuntimeTypeInfoFunc(self, GCSTRUCT, func, ARG_GCSTRUCT=None, destrptr=None): self.call_all_setups() # compute ForwardReferences now Modified: pypy/dist/pypy/rpython/test/test_llinterp.py ============================================================================== --- pypy/dist/pypy/rpython/test/test_llinterp.py (original) +++ pypy/dist/pypy/rpython/test/test_llinterp.py Sat Jan 14 23:00:32 2006 @@ -1,7 +1,6 @@ import py from pypy.rpython.lltypesystem.lltype import typeOf, pyobjectptr, Ptr, PyObject, Void -from pypy.rpython.rtyper import RPythonTyper from pypy.rpython.llinterp import LLInterpreter, LLException,log from pypy.rpython.rmodel import inputconst from pypy.translator.translator import TranslationContext Modified: pypy/dist/pypy/rpython/test/test_rlist.py ============================================================================== --- pypy/dist/pypy/rpython/test/test_rlist.py (original) +++ pypy/dist/pypy/rpython/test/test_rlist.py Sat Jan 14 23:00:32 2006 @@ -1,10 +1,12 @@ import sys from pypy.rpython.lltypesystem.lltype import * -from pypy.rpython.rtyper import RPythonTyper +from pypy.rpython.rtyper import LowLevelOpList from pypy.rpython.rlist import * from pypy.rpython.rslice import ll_newslice from pypy.rpython.rint import signed_repr from pypy.rpython.test.test_llinterp import interpret, interpret_raises +from pypy.translator.translator import TranslationContext +from pypy.objspace.flow.model import Constant, Variable # undo the specialization parameter for n1 in 'get set del'.split(): @@ -891,3 +893,37 @@ res = interpret(fn, [2]) assert res == 0 interpret_raises(MemoryError, fn, [sys.maxint]) + +def test_list_builder(): + def fixed_size_case(): + return [42] + def variable_size_case(): + lst = [] + lst.append(42) + return lst + + for fn in [fixed_size_case, variable_size_case]: + t = TranslationContext() + t.buildannotator().build_types(fn, []) + t.buildrtyper().specialize() + LIST = t.graphs[0].getreturnvar().concretetype.TO + llop = LowLevelOpList(None) + v0 = Constant(42) + v0.concretetype = Signed + v1 = Variable() + v1.concretetype = Signed + vr = LIST.list_builder(llop, [v0, v1]) + assert len(llop) == 3 + assert llop[0].opname == 'direct_call' + assert llop[0].args[1].concretetype == Void + assert llop[0].args[1].value == LIST + assert llop[0].args[2].concretetype == Signed + assert llop[0].args[2].value == 2 + assert llop[0].result is vr + for op, i, vi in [(llop[1], 0, v0), (llop[2], 1, v1)]: + assert op.opname == 'direct_call' + assert op.args[-3] is vr + assert op.args[-2].concretetype == Signed + assert op.args[-2].value == i + assert op.args[-1] is vi + assert op.result.concretetype is Void From arigo at codespeak.net Sat Jan 14 23:29:13 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sat, 14 Jan 2006 23:29:13 +0100 (CET) Subject: [pypy-svn] r22183 - in pypy/dist/pypy/rpython: . test Message-ID: <20060114222913.F087A27B46@code1.codespeak.net> Author: arigo Date: Sat Jan 14 23:29:11 2006 New Revision: 22183 Modified: pypy/dist/pypy/rpython/rlist.py pypy/dist/pypy/rpython/test/test_rlist.py Log: Bug fix, more precise test. A few more oopspec tags. Modified: pypy/dist/pypy/rpython/rlist.py ============================================================================== --- pypy/dist/pypy/rpython/rlist.py (original) +++ pypy/dist/pypy/rpython/rlist.py Sat Jan 14 23:29:11 2006 @@ -174,6 +174,7 @@ argtypes = [bk.immutablevalue(dum_nocheck), LISTPTR, Signed, ITEM] fnptr = list_repr.rtyper.annotate_helper_fn(ll_setitem_nonneg, argtypes) self.c_setitem_nonneg = inputconst(typeOf(fnptr), fnptr) + self.c_dum_nocheck = inputconst(Void, dum_nocheck) def build(self, llop, items_v): """Make the operations that would build a list containing the @@ -186,8 +187,9 @@ v_result = llop.genop('direct_call', [self.c_newlist, c_list, c_len], resulttype = self.LISTPTR) for i, v in enumerate(items_v): - llop.genop('direct_call', [self.c_setitem_nonneg, v_result, - inputconst(Signed, i), v]) + llop.genop('direct_call', [self.c_setitem_nonneg, + self.c_dum_nocheck, + v_result, inputconst(Signed, i), v]) return v_result class ListRepr(BaseListRepr): @@ -645,6 +647,7 @@ if func is dum_checkidx and (index >= l.ll_length()): raise IndexError l.ll_items()[index] = newitem +ll_setitem_nonneg.oopspec = 'list.setitem(l, index, newitem)' def ll_setitem(func, l, index, newitem): length = l.ll_length() @@ -653,6 +656,7 @@ if func is dum_checkidx and (index < 0 or index >= length): raise IndexError l.ll_items()[index] = newitem +ll_setitem.oopspec = 'list.setitem(l, index, newitem)' def ll_delitem_nonneg(func, l, index): length = l.length @@ -964,6 +968,7 @@ items[i] = item i += 1 return l +ll_alloc_and_set.oopspec = 'newlist(count, item)' def rtype_alloc_and_set(hop): r_list = hop.r_result Modified: pypy/dist/pypy/rpython/test/test_rlist.py ============================================================================== --- pypy/dist/pypy/rpython/test/test_rlist.py (original) +++ pypy/dist/pypy/rpython/test/test_rlist.py Sat Jan 14 23:29:11 2006 @@ -915,6 +915,7 @@ vr = LIST.list_builder(llop, [v0, v1]) assert len(llop) == 3 assert llop[0].opname == 'direct_call' + assert len(llop[0].args) == 3 assert llop[0].args[1].concretetype == Void assert llop[0].args[1].value == LIST assert llop[0].args[2].concretetype == Signed @@ -922,8 +923,10 @@ assert llop[0].result is vr for op, i, vi in [(llop[1], 0, v0), (llop[2], 1, v1)]: assert op.opname == 'direct_call' - assert op.args[-3] is vr - assert op.args[-2].concretetype == Signed - assert op.args[-2].value == i - assert op.args[-1] is vi + assert len(op.args) == 5 + assert op.args[1].value is dum_nocheck + assert op.args[2] is vr + assert op.args[3].concretetype == Signed + assert op.args[3].value == i + assert op.args[4] is vi assert op.result.concretetype is Void From arigo at codespeak.net Sat Jan 14 23:38:21 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sat, 14 Jan 2006 23:38:21 +0100 (CET) Subject: [pypy-svn] r22185 - in pypy/dist/pypy/jit: . test Message-ID: <20060114223821.165FA27B44@code1.codespeak.net> Author: arigo Date: Sat Jan 14 23:38:19 2006 New Revision: 22185 Modified: pypy/dist/pypy/jit/llabstractinterp.py pypy/dist/pypy/jit/test/test_vlist.py pypy/dist/pypy/jit/vlist.py Log: Virtual lists start to work. Modified: pypy/dist/pypy/jit/llabstractinterp.py ============================================================================== --- pypy/dist/pypy/jit/llabstractinterp.py (original) +++ pypy/dist/pypy/jit/llabstractinterp.py Sat Jan 14 23:38:19 2006 @@ -667,6 +667,8 @@ raise RestartCompleting a = LLAbstractValue(c) a.concrete = True + if hints.get('nonvirtual'): + a.forcevarorconst(self) # for testing only return a def op_direct_call(self, op, *args_a): Modified: pypy/dist/pypy/jit/test/test_vlist.py ============================================================================== --- pypy/dist/pypy/jit/test/test_vlist.py (original) +++ pypy/dist/pypy/jit/test/test_vlist.py Sat Jan 14 23:38:19 2006 @@ -36,10 +36,30 @@ return graph2, summary(interp) -def test_newlist(): - py.test.skip("in-progress") +def test_fixed_newlistfill_force(): + def fn(n): + lst = [5] * n + hint(lst, nonvirtual=True) + graph2, insns = run(fn, [12]) + assert insns == {'direct_call': 13} + +def test_newlistfill_force(): def fn(n): lst = [5] * n - return len(lst) + if n < 0: + lst.append(6) + hint(lst, nonvirtual=True) + graph2, insns = run(fn, [12]) + assert insns == {'direct_call': 13} + +def test_newlist_force(): + py.test.skip("in-progress") + def fn(n): + lst = [] + lst.append(n) + lst.append(5) + lst.append(12) + lst.pop() + hint(lst, nonvirtual=True) graph2, insns = run(fn, [12]) - assert insns == {} + assert insns == {'direct_call': 3} Modified: pypy/dist/pypy/jit/vlist.py ============================================================================== --- pypy/dist/pypy/jit/vlist.py (original) +++ pypy/dist/pypy/jit/vlist.py Sat Jan 14 23:38:19 2006 @@ -1,4 +1,5 @@ from pypy.rpython.lltypesystem import lltype +from pypy.rpython.rtyper import LowLevelOpList from pypy.jit.llvalue import LLAbstractValue, newvar, const, ll_dummy_value from pypy.jit.llcontainer import LLAbstractContainer @@ -38,14 +39,13 @@ return LLVirtualList(self.T, items_a) def build_runtime_container(self, builder): - cno = const(len(self.items_a)) - v_result = builder.gendirectcall(self.T.ll_newlist, cno) - cdum = const(rlist.dum_nocheck, lltype.Void) - for i, a in enumerate(self.items_a): - ci = const(i) - v_item = a.forcevarorconst(builder) - builder.gendirectcall(rlist.ll_setitem_nonneg, - cdum, v_result, ci, v_item) + items_v = [a.forcevarorconst(builder) for a in self.items_a] + llop = LowLevelOpList(None) + v_result = self.T.list_builder(llop, items_v) + print 'list_builder:' + for op in llop: + print '\t', op + builder.residual_operations.append(op) return v_result # ____________________________________________________________ @@ -73,11 +73,11 @@ return self.items_a.pop(c_index.value) -def oop_newlist(op, a_numitems): +def oop_newlist(op, a_numitems, a_item=ll_dummy_value): c_numitems = a_numitems.maybe_get_constant() if c_numitems is None: raise NotImplementedError LIST = op.result.concretetype.TO - items_a = [ll_dummy_value] * c_numitems.value + items_a = [a_item] * c_numitems.value virtuallist = LLVirtualList(LIST, items_a) return LLAbstractValue(content=virtuallist) From arigo at codespeak.net Sat Jan 14 23:48:51 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sat, 14 Jan 2006 23:48:51 +0100 (CET) Subject: [pypy-svn] r22187 - in pypy/dist/pypy: jit jit/test rpython Message-ID: <20060114224851.8D4B627B44@code1.codespeak.net> Author: arigo Date: Sat Jan 14 23:48:48 2006 New Revision: 22187 Modified: pypy/dist/pypy/jit/llabstractinterp.py pypy/dist/pypy/jit/test/test_vlist.py pypy/dist/pypy/jit/vlist.py pypy/dist/pypy/rpython/rlist.py Log: Some more oopspecs, and some virtual methods working. Nothing systematic yet. Modified: pypy/dist/pypy/jit/llabstractinterp.py ============================================================================== --- pypy/dist/pypy/jit/llabstractinterp.py (original) +++ pypy/dist/pypy/jit/llabstractinterp.py Sat Jan 14 23:48:48 2006 @@ -872,13 +872,13 @@ # dispatch on the 'self' argument if it is virtual a_self = args_a[0] args_a = args_a[1:] - if not isinstance(a_self, LLAbstractContainer): + if not isinstance(a_self.content, LLAbstractContainer): return None type_name, operation_name = operation_name.split('.') - if a_self.type_name != type_name: + if a_self.content.type_name != type_name: return None try: - handler = getattr(a_self, 'oop_' + operation_name) + handler = getattr(a_self.content, 'oop_' + operation_name) except AttributeError: print 'MISSING HANDLER: oop_%s' % (operation_name,) return None Modified: pypy/dist/pypy/jit/test/test_vlist.py ============================================================================== --- pypy/dist/pypy/jit/test/test_vlist.py (original) +++ pypy/dist/pypy/jit/test/test_vlist.py Sat Jan 14 23:48:48 2006 @@ -53,7 +53,6 @@ assert insns == {'direct_call': 13} def test_newlist_force(): - py.test.skip("in-progress") def fn(n): lst = [] lst.append(n) Modified: pypy/dist/pypy/jit/vlist.py ============================================================================== --- pypy/dist/pypy/jit/vlist.py (original) +++ pypy/dist/pypy/jit/vlist.py Sat Jan 14 23:48:48 2006 @@ -66,11 +66,14 @@ def oop_append(self, op, a_newobj): self.items_a.append(a_newobj) - def oop_pop(self, op, a_index): - c_index = a_index.maybe_get_constant() - if c_index is None: - raise NotImplementedError - return self.items_a.pop(c_index.value) + def oop_pop(self, op, a_index=None): + if a_index is None: + return self.items_a.pop() + else: + c_index = a_index.maybe_get_constant() + if c_index is None: + raise NotImplementedError + return self.items_a.pop(c_index.value) def oop_newlist(op, a_numitems, a_item=ll_dummy_value): Modified: pypy/dist/pypy/rpython/rlist.py ============================================================================== --- pypy/dist/pypy/rpython/rlist.py (original) +++ pypy/dist/pypy/rpython/rlist.py Sat Jan 14 23:48:48 2006 @@ -540,6 +540,7 @@ length = l.length _ll_list_resize_ge(l, length+1) l.items[length] = newitem +ll_append.oopspec = 'list.append(l, newitem)' # this one is for the special case of insert(0, x) def ll_prepend(l, newitem): @@ -552,6 +553,7 @@ items[dst] = items[src] dst = src items[0] = newitem +ll_prepend.oopspec = 'list.insert(l, 0, newitem)' def ll_insert_nonneg(l, index, newitem): length = l.length @@ -563,6 +565,7 @@ items[dst] = items[src] dst = src items[index] = newitem +ll_insert_nonneg.oopspec = 'list.insert(l, index, newitem)' def dum_checkidx(): pass def dum_nocheck(): pass @@ -573,6 +576,7 @@ res = l.items[index] ll_delitem_nonneg(dum_nocheck, l, index) return res +ll_pop_nonneg.oopspec = 'list.pop(l, index)' def ll_pop_default(func, l): length = l.length @@ -587,6 +591,7 @@ items[index] = nullptr(ITEM.TO) _ll_list_resize_le(l, newlength) return res +ll_pop_default.oopspec = 'list.pop(l)' def ll_pop_zero(func, l): length = l.length @@ -617,6 +622,7 @@ res = l.items[index] ll_delitem_nonneg(dum_nocheck, l, index) return res +ll_pop.oopspec = 'list.pop(l, index)' def ll_reverse(l): length = l.ll_length() @@ -629,11 +635,13 @@ items[length_1_i] = tmp i += 1 length_1_i -= 1 +ll_reverse.oopspec = 'list.reverse(l)' def ll_getitem_nonneg(func, l, index): if func is dum_checkidx and (index >= l.ll_length()): raise IndexError return l.ll_items()[index] +ll_getitem_nonneg.oopspec = 'list.getitem(l, index)' def ll_getitem(func, l, index): length = l.ll_length() @@ -642,6 +650,7 @@ if func is dum_checkidx and (index < 0 or index >= length): raise IndexError return l.ll_items()[index] +ll_getitem.oopspec = 'list.getitem(l, index)' def ll_setitem_nonneg(func, l, index, newitem): if func is dum_checkidx and (index >= l.ll_length()): @@ -674,6 +683,7 @@ if isinstance(ITEM, Ptr): items[newlength] = nullptr(ITEM.TO) _ll_list_resize_le(l, newlength) +ll_delitem_nonneg.oopspec = 'list.delitem(l, index)' def ll_delitem(func, l, i): length = l.length @@ -682,6 +692,7 @@ if func is dum_checkidx and (i < 0 or i >= length): raise IndexError ll_delitem_nonneg(dum_nocheck, l, i) +ll_delitem.oopspec = 'list.delitem(l, i)' def ll_concat(RESLIST, l1, l2): len1 = l1.ll_length() From arigo at codespeak.net Sun Jan 15 00:03:05 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sun, 15 Jan 2006 00:03:05 +0100 (CET) Subject: [pypy-svn] r22188 - in pypy/dist/pypy: jit jit/test rpython Message-ID: <20060114230305.BE80327B44@code1.codespeak.net> Author: arigo Date: Sun Jan 15 00:03:03 2006 New Revision: 22188 Modified: pypy/dist/pypy/jit/test/test_vlist.py pypy/dist/pypy/jit/vlist.py pypy/dist/pypy/rpython/rlist.py Log: A bunch of new list methods. Modified: pypy/dist/pypy/jit/test/test_vlist.py ============================================================================== --- pypy/dist/pypy/jit/test/test_vlist.py (original) +++ pypy/dist/pypy/jit/test/test_vlist.py Sun Jan 15 00:03:03 2006 @@ -62,3 +62,36 @@ hint(lst, nonvirtual=True) graph2, insns = run(fn, [12]) assert insns == {'direct_call': 3} + +def test_simple_purely_virtual(): + def fn(n): + return len([5]*n) + graph2, insns = run(fn, [12]) + assert insns == {} + +def test_copy(): + def fn(n): + lst = [] + lst.append(n) + lst.append(n) + return len(list(lst)) + graph2, insns = run(fn, [12]) + assert insns == {} + +def test_is_true(): + def fn(n): + lst = [5] * n + if lst: + return 654 + else: + return 321 + graph2, insns = run(fn, [12]) + assert insns == {} + +def test_concat(): + def fn(n): + lst1 = [2, 3] + lst2 = [4] * n + return len(lst1 + lst2) + graph2, insns = run(fn, [12]) + assert insns == {} Modified: pypy/dist/pypy/jit/vlist.py ============================================================================== --- pypy/dist/pypy/jit/vlist.py (original) +++ pypy/dist/pypy/jit/vlist.py Sun Jan 15 00:03:03 2006 @@ -51,6 +51,12 @@ # ____________________________________________________________ # High-level operations + def oop_len(self, op): + return LLAbstractValue(const(len(self.items_a))) + + def oop_nonzero(self, op): + return LLAbstractValue(const(bool(self.items_a))) + def oop_getitem(self, op, a_index): c_index = a_index.maybe_get_constant() if c_index is None: @@ -63,9 +69,21 @@ raise NotImplementedError self.items_a[c_index.value] = a_newobj + def oop_delitem(self, op, a_index): + c_index = a_index.maybe_get_constant() + if c_index is None: + raise NotImplementedError + del self.items_a[c_index.value] + def oop_append(self, op, a_newobj): self.items_a.append(a_newobj) + def oop_insert(self, op, a_index, a_newobj): + c_index = a_index.maybe_get_constant() + if c_index is None: + raise NotImplementedError + self.items_a.insert(c_index.value, a_newobj) + def oop_pop(self, op, a_index=None): if a_index is None: return self.items_a.pop() @@ -75,6 +93,23 @@ raise NotImplementedError return self.items_a.pop(c_index.value) + def oop_reverse(self, op): + self.items_a.reverse() + + def oop_copy(self, op): + items_a = list(self.items_a) + LIST = op.result.concretetype.TO + virtuallist = LLVirtualList(LIST, items_a) + return LLAbstractValue(content=virtuallist) + + def oop_concat(self, op, a_other): + if not isinstance(a_other.content, LLVirtualList): + raise NotImplementedError + items_a = self.items_a + a_other.content.items_a + LIST = op.result.concretetype.TO + virtuallist = LLVirtualList(LIST, items_a) + return LLAbstractValue(content=virtuallist) + def oop_newlist(op, a_numitems, a_item=ll_dummy_value): c_numitems = a_numitems.maybe_get_constant() Modified: pypy/dist/pypy/rpython/rlist.py ============================================================================== --- pypy/dist/pypy/rpython/rlist.py (original) +++ pypy/dist/pypy/rpython/rlist.py Sun Jan 15 00:03:03 2006 @@ -528,13 +528,16 @@ new_items[i] = items[i] i += 1 return new_lst +ll_copy.oopspec = 'list.copy(l)' def ll_len(l): return l.ll_length() +ll_len.oopspec = 'list.len(l)' def ll_list_is_true(l): # check if a list is True, allowing for None return bool(l) and l.ll_length() != 0 +ll_list_is_true.oopspec = 'list.nonzero(l)' def ll_append(l, newitem): length = l.length @@ -712,6 +715,7 @@ i += 1 j += 1 return l +ll_concat.oopspec = 'list.concat(l1, l2)' def ll_extend(l1, l2): len1 = l1.length From pedronis at codespeak.net Sun Jan 15 00:08:59 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Sun, 15 Jan 2006 00:08:59 +0100 (CET) Subject: [pypy-svn] r22189 - pypy/dist/pypy/jit Message-ID: <20060114230859.D88E027B44@code1.codespeak.net> Author: pedronis Date: Sun Jan 15 00:08:58 2006 New Revision: 22189 Modified: pypy/dist/pypy/jit/llcontainer.py Log: small fix Modified: pypy/dist/pypy/jit/llcontainer.py ============================================================================== --- pypy/dist/pypy/jit/llcontainer.py (original) +++ pypy/dist/pypy/jit/llcontainer.py Sun Jan 15 00:08:58 2006 @@ -52,7 +52,7 @@ def build_runtime_container(self, builder): v_result = newvar(lltype.Ptr(self.T)) if self.a_parent is not None: - v_parent = self.a_parent.build_runtime_container(builder) + v_parent = self.a_parent.content.build_runtime_container(builder) op = SpaceOperation('getsubstruct', [v_parent, const(self.parentindex, lltype.Void)], From pedronis at codespeak.net Sun Jan 15 00:51:11 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Sun, 15 Jan 2006 00:51:11 +0100 (CET) Subject: [pypy-svn] r22190 - in pypy/dist/pypy/jit: . test Message-ID: <20060114235111.9839127B44@code1.codespeak.net> Author: pedronis Date: Sun Jan 15 00:51:10 2006 New Revision: 22190 Modified: pypy/dist/pypy/jit/llabstractinterp.py pypy/dist/pypy/jit/test/test_jit_tl.py Log: test_jit_tl passes with oopspecs Modified: pypy/dist/pypy/jit/llabstractinterp.py ============================================================================== --- pypy/dist/pypy/jit/llabstractinterp.py (original) +++ pypy/dist/pypy/jit/llabstractinterp.py Sun Jan 15 00:51:10 2006 @@ -153,6 +153,8 @@ # hint-driven policy best_policy = Policy(inlining=True, const_propagate=True, concrete_args=False) +best_policy_oopspec = Policy(inlining=True, const_propagate=True, concrete_args=False, + oopspec=True) class LLAbstractInterp(object): Modified: pypy/dist/pypy/jit/test/test_jit_tl.py ============================================================================== --- pypy/dist/pypy/jit/test/test_jit_tl.py (original) +++ pypy/dist/pypy/jit/test/test_jit_tl.py Sun Jan 15 00:51:10 2006 @@ -3,7 +3,7 @@ import py from pypy.translator.translator import TranslationContext from pypy.jit import tl -from pypy.jit.llabstractinterp import LLAbstractInterp +from pypy.jit.llabstractinterp import LLAbstractInterp, best_policy_oopspec from pypy.rpython.rstr import string_repr from pypy.rpython.llinterp import LLInterpreter #from pypy.translator.backendopt import inline @@ -22,7 +22,7 @@ def jit_tl(code): - interp = LLAbstractInterp() + interp = LLAbstractInterp(policy=best_policy_oopspec) hints = {0: string_repr.convert_const(code), 1: 0} graph2 = interp.eval(graph1, hints) From rxe at codespeak.net Sun Jan 15 00:56:47 2006 From: rxe at codespeak.net (rxe at codespeak.net) Date: Sun, 15 Jan 2006 00:56:47 +0100 (CET) Subject: [pypy-svn] r22191 - pypy/dist/pypy/translator/llvm Message-ID: <20060114235647.4CF4127B44@code1.codespeak.net> Author: rxe Date: Sun Jan 15 00:56:46 2006 New Revision: 22191 Modified: pypy/dist/pypy/translator/llvm/exception.py Log: cosmetic Modified: pypy/dist/pypy/translator/llvm/exception.py ============================================================================== --- pypy/dist/pypy/translator/llvm/exception.py (original) +++ pypy/dist/pypy/translator/llvm/exception.py Sun Jan 15 00:56:46 2006 @@ -170,9 +170,9 @@ def llc_options(self): import sys if sys.platform == 'linux2' and sys.maxint == 2**63-1: - s = ' -enable-ia64-dag-isel' + s = ' -enable-ia64-dag-isel' else: - s = '' + s = '' return '-enable-correct-eh-support' + s class ExplicitExceptionPolicy(ExceptionPolicy): @@ -354,8 +354,8 @@ def llc_options(self): import sys if sys.platform == 'linux2' and sys.maxint == 2**63-1: - s = '-enable-ia64-dag-isel' + s = '-enable-ia64-dag-isel' else: - s = '' + s = '' return s From rxe at codespeak.net Sun Jan 15 00:59:34 2006 From: rxe at codespeak.net (rxe at codespeak.net) Date: Sun, 15 Jan 2006 00:59:34 +0100 (CET) Subject: [pypy-svn] r22192 - pypy/dist/pypy/translator/llvm Message-ID: <20060114235934.2A3BC27B44@code1.codespeak.net> Author: rxe Date: Sun Jan 15 00:59:33 2006 New Revision: 22192 Modified: pypy/dist/pypy/translator/llvm/gc.py Log: Some minor tweaks (and possible temporary ones until we understand what is going on) to make our nightly build tests work again. Modified: pypy/dist/pypy/translator/llvm/gc.py ============================================================================== --- pypy/dist/pypy/translator/llvm/gc.py (original) +++ pypy/dist/pypy/translator/llvm/gc.py Sun Jan 15 00:59:33 2006 @@ -113,8 +113,13 @@ def gc_libraries(self): return ['gc', 'pthread'] # XXX on windows? - def malloc(self, codewriter, targetvar, type_, size=1, atomic=False): + import sys + if sys.platform == 'linux2' and sys.maxint == 2**63-1: + boundary_size = 8 + else: + boundary_size = 0 + is_atomic = atomic uword = self.db.get_machine_uword() s = str(size) @@ -130,10 +135,11 @@ if is_atomic: #Boehm aligns on 8 byte boundaries t += ''' - call ccc void %%llvm.memset(sbyte* %%malloc_Ptr%(cnt)s, ubyte 0, %(uword)s %%malloc_SizeU%(cnt)s, %(uword)s 8) + call ccc void %%llvm.memset(sbyte* %%malloc_Ptr%(cnt)s, ubyte 0, %(uword)s %%malloc_SizeU%(cnt)s, %(uword)s %(boundary_size)s) ''' % locals() codewriter.write_lines(t) + def pyrex_code(self): return ''' cdef extern int GC_get_heap_size() From arigo at codespeak.net Sun Jan 15 13:16:16 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sun, 15 Jan 2006 13:16:16 +0100 (CET) Subject: [pypy-svn] r22194 - pypy/dist/pypy/jit Message-ID: <20060115121616.63A3827B52@code1.codespeak.net> Author: arigo Date: Sun Jan 15 13:16:14 2006 New Revision: 22194 Modified: pypy/dist/pypy/jit/llabstractinterp.py Log: Some bug fixes. Modified: pypy/dist/pypy/jit/llabstractinterp.py ============================================================================== --- pypy/dist/pypy/jit/llabstractinterp.py (original) +++ pypy/dist/pypy/jit/llabstractinterp.py Sun Jan 15 13:16:14 2006 @@ -152,7 +152,8 @@ self.oopspec = oopspec # hint-driven policy -best_policy = Policy(inlining=True, const_propagate=True, concrete_args=False) +best_policy = Policy(inlining=True, const_propagate=True, concrete_args=False, + oopspec=True) best_policy_oopspec = Policy(inlining=True, const_propagate=True, concrete_args=False, oopspec=True) @@ -874,10 +875,8 @@ # dispatch on the 'self' argument if it is virtual a_self = args_a[0] args_a = args_a[1:] - if not isinstance(a_self.content, LLAbstractContainer): - return None type_name, operation_name = operation_name.split('.') - if a_self.content.type_name != type_name: + if getattr(a_self.content, 'type_name', None) != type_name: return None try: handler = getattr(a_self.content, 'oop_' + operation_name) From arigo at codespeak.net Sun Jan 15 13:24:25 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sun, 15 Jan 2006 13:24:25 +0100 (CET) Subject: [pypy-svn] r22195 - in pypy/dist/pypy/jit: . test Message-ID: <20060115122425.628A127B54@code1.codespeak.net> Author: arigo Date: Sun Jan 15 13:24:23 2006 New Revision: 22195 Modified: pypy/dist/pypy/jit/llabstractinterp.py pypy/dist/pypy/jit/llcontainer.py pypy/dist/pypy/jit/test/test_jit_tl.py Log: Synchronized small fixes with Samuele's, some of which accidentally checked in twice in two different ways -- I got out of sync during codespeak downtime. Modified: pypy/dist/pypy/jit/llabstractinterp.py ============================================================================== --- pypy/dist/pypy/jit/llabstractinterp.py (original) +++ pypy/dist/pypy/jit/llabstractinterp.py Sun Jan 15 13:24:23 2006 @@ -154,8 +154,6 @@ # hint-driven policy best_policy = Policy(inlining=True, const_propagate=True, concrete_args=False, oopspec=True) -best_policy_oopspec = Policy(inlining=True, const_propagate=True, concrete_args=False, - oopspec=True) class LLAbstractInterp(object): Modified: pypy/dist/pypy/jit/llcontainer.py ============================================================================== --- pypy/dist/pypy/jit/llcontainer.py (original) +++ pypy/dist/pypy/jit/llcontainer.py Sun Jan 15 13:24:23 2006 @@ -52,7 +52,7 @@ def build_runtime_container(self, builder): v_result = newvar(lltype.Ptr(self.T)) if self.a_parent is not None: - v_parent = self.a_parent.content.build_runtime_container(builder) + v_parent = self.a_parent.forcevarorconst(builder) op = SpaceOperation('getsubstruct', [v_parent, const(self.parentindex, lltype.Void)], Modified: pypy/dist/pypy/jit/test/test_jit_tl.py ============================================================================== --- pypy/dist/pypy/jit/test/test_jit_tl.py (original) +++ pypy/dist/pypy/jit/test/test_jit_tl.py Sun Jan 15 13:24:23 2006 @@ -3,7 +3,7 @@ import py from pypy.translator.translator import TranslationContext from pypy.jit import tl -from pypy.jit.llabstractinterp import LLAbstractInterp, best_policy_oopspec +from pypy.jit.llabstractinterp import LLAbstractInterp from pypy.rpython.rstr import string_repr from pypy.rpython.llinterp import LLInterpreter #from pypy.translator.backendopt import inline @@ -22,7 +22,7 @@ def jit_tl(code): - interp = LLAbstractInterp(policy=best_policy_oopspec) + interp = LLAbstractInterp() hints = {0: string_repr.convert_const(code), 1: 0} graph2 = interp.eval(graph1, hints) From arigo at codespeak.net Sun Jan 15 17:06:51 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sun, 15 Jan 2006 17:06:51 +0100 (CET) Subject: [pypy-svn] r22197 - pypy/dist/pypy/doc/discussion Message-ID: <20060115160651.7A4F527B60@code1.codespeak.net> Author: arigo Date: Sun Jan 15 17:06:49 2006 New Revision: 22197 Modified: pypy/dist/pypy/doc/discussion/draft-jit-ideas.txt Log: (pedronis, arigo) Refactored and updated the JIT ideas and status. Don't miss the new paragraph 'Functions/operations to generate code', which is sort of the next thing to do. Modified: pypy/dist/pypy/doc/discussion/draft-jit-ideas.txt ============================================================================== --- pypy/dist/pypy/doc/discussion/draft-jit-ideas.txt (original) +++ pypy/dist/pypy/doc/discussion/draft-jit-ideas.txt Sun Jan 15 17:06:49 2006 @@ -6,14 +6,19 @@ Short-term plans: -1. Write a small interpreter in RPython for whatever bytecode language, - as an example and for testing. The goal is to turn that interpreter - into a JIT. +1. DONE (jit/tl.py): Write a small interpreter in RPython for whatever + bytecode language, as an example and for testing. The goal is to turn + that interpreter into a JIT. -2. Write code that takes LL graphs and "specializes" them, by making a - variable constant and propagating it. +2. MOSTLY DONE (jit/llabstractinterp.py): Write code that takes LL + graphs and "specializes" them, by making a variable constant and + propagating it. -3. Think more about how to plug 1 into 2 :-) +3. DONE (jit/test/test_jit_tl.py): Think more about how to plug 1 into 2 :-) + +4. Refactor 2 to use `Functions/operations to generate code`_ + +5. Think about `how to do at run-time what llabstractinterp does statically`_ Discussion and details @@ -22,45 +27,82 @@ Low-level graphs abstract interpreter ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -In the context of PyPy architecture a JIT can be envisioned as a -run-time specialiser (doing constant folding, partial evaluation and -keeping allocation virtual as much as possible). The specialiser would -be able to produce for some runtime execution state of a function new -low-level graphs from a predetermined relevant subset of the forest of -low-level graphs making up PyPy and given entry points and parameter -variables whose run-time constantness can be exploited (think the eval -loop for an entry-point and the bytecode of the function for such a -variable and the graphs for the directly involved PyPy functions as the -subset). This new low-level graphs could then be turned into machine -code by a run-time machine code backend, mapping progressively -the function bytecode into machine code. +Now done in pypy/jit/llabstractinterp.py. XXX To be documented. -Ideally PyPy translation should generate code from this determined -subset, list of entry-points and variables that implements run-time -specialisation for it, plus management/bookkeeping and instrumentation -code. - -To explore and understand this problem space, we should probably start -by writing a pure Python abstract interpreter doing constant-folding -and partial evaluation of low-level graphs. Increasing abstraction -this could maybe evolve in the code for generating the specialiser or -at least be used to analyse which subset of the graphs is relevant. +The LLAbstractInterp is a kind of generalized constant propagation, +malloc removal, and inlining tool. It takes L2 graphs, uses hints about +the value of some arguments, and produce a new L2 graph. It is tuned to +the following goal: given the L2 graph of an interpreter +'interp(bytecode)', and a known 'bytecode' to interpret, it produces a +residual L2 graph in which the interpretation overhead was removed. In +other words the combination of LLAbstractInterp and the L2 graph of +'interp()' works like a compiler for the language accepted by interp(). -issue: too fine granularity of low-level implementations of rpython dicts Simple target interpreter for experimentation and testing ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Given that PyPy eval loop is quite a large chunk to swallow, ahem, -analyse, it would be nice to have some kind of minimal bytecode eval -loop for some very simple bytecode written in RPython to use for -testing/experimenting and as first target. Ideally the interpreter -state for this should not be much more than an instruction counter and -a value stack. +analyse, it is nice to have some kind of minimal bytecode eval loop for +some very simple bytecode written in RPython to use for +testing/experimenting and as first target. This is the Toy Language, +for which we have an interpreter in pypy/jit/tl.py. The state for this +interpreter is not much more than an instruction counter and a value +stack (easy for LLAbstractInterp). We should try to add more features +incrementally when we extend LLAbstractInterp. + + +How to do at run-time what llabstractinterp does statically +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Remember from above that LLAbstractInterp together with the L2 graph of +'interp()' works like a compiler for the language accepted by interp(). +The next goal is to really produce a stand-alone compiler, +i.e. something that does the same job but more efficiently (and that can +be translated and executed at run-time, too). So we need to divide the +job of LLAbstractInterp into the bits that analyse the L2 graphs of +interp() and the bits that actually flow real constants, like the value +of 'bytecode', in the L2 graphs. The final compiler should only do an +efficient flowing of the 'bytecode' and the production of new L2 graphs, +but not the analysis of the L2 graphs of interp(), done in advance. + + +Functions/operations to generate code +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +As a first step, to allow experimentation with code generation without +all the difficulties of the L3 graph model and L3 interpreter, we could +provide a set of primitive functions that enable simple code generation. + +These functions define a reusable interface: when they are run normally +on top of CPython, they produce -- say -- L2 graphs that can be executed +normally with the LLInterpreter (or compiled by GenC, etc). When the +functions themselves appear in an RPython function, they are turned into +new, special low-level operations (like lltype.malloc() becomes a +'malloc' low-level operation). What these special operations do depends +on the back-end. On top of LLInterpreter, these operations would again +produce L2 graphs, runnable by the same LLInterpreter. The other cases +need to be thought out later -- for example, some extended GenC could +turn them into C code that generates assembler at run-time. + +To define these operations, we plan to go over jit/llabstractinterp.py +and invent the operations while refactoring it. The goal would be to +replace all the explicit building of Graph, Block, Link, Operation and +Variable with calls to such operations. + L3 interpreter ~~~~~~~~~~~~~~~~~~~ +L3 is meant to be an RPython-friendly format for graphs, with an +L3Interpreter written in RPython. It would be used as a possible target +for the above code-generating operations: they would produce L3 graphs, +and when these graphs must be executed, we would use the L3Interpreter. +This gives a completely RPython implementation of the code-generating +operations. + +About L3Interpreter: + * in RPython * the code should try to be straightforward (also for efficiency) @@ -70,7 +112,8 @@ * one major issue to keep in mind is where the information about offsets into low-level data types comes from/is computed - (C compiler offset values, vs. made-up values) + (C compiler offset values, vs. made-up values). This could be + factored into a generic 'heap' interface. * translatable together with an RPython program, and capable of accepting (constant) data and functions from it @@ -83,14 +126,38 @@ Machine code backends and register allocation ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -xxx +Open. This is about what to do in GenC with the operations that +generate code. We could have code that takes the L3 graph data +structures and turn them into machine code. Or we could directly take +the code-generation operations and turn them into machine code. Or some +combination, e.g. with yet other intermediate compilable formats. -Alternative: RPython interface to code generation -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -As a first step, to allow experimentation with code generation without -all the difficulties of the L3 graph model and L3 interpreter, we could -provide a set of primitive RPython functions that enable simple code -generation. +This should be rewritten into a real nice intro +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +XXX kill this stuff and write a real nice intro + +In the context of PyPy architecture a JIT can be envisioned as a +run-time specialiser (doing constant folding, partial evaluation and +keeping allocation virtual as much as possible). The specialiser would +be able to produce for some runtime execution state of a function new +low-level graphs from a predetermined relevant subset of the forest of +low-level graphs making up PyPy and given entry points and parameter +variables whose run-time constantness can be exploited (think the eval +loop for an entry-point and the bytecode of the function for such a +variable and the graphs for the directly involved PyPy functions as the +subset). This new low-level graphs could then be turned into machine +code by a run-time machine code backend, mapping progressively +the function bytecode into machine code. -TBD +Ideally PyPy translation should generate code from this determined +subset, list of entry-points and variables that implements run-time +specialisation for it, plus management/bookkeeping and instrumentation +code. + +To explore and understand this problem space, we should probably start +by writing a pure Python abstract interpreter doing constant-folding +and partial evaluation of low-level graphs. Increasing abstraction +this could maybe evolve in the code for generating the specialiser or +at least be used to analyse which subset of the graphs is relevant. From arigo at codespeak.net Sun Jan 15 17:19:42 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sun, 15 Jan 2006 17:19:42 +0100 (CET) Subject: [pypy-svn] r22199 - pypy/dist/pypy/jit/test Message-ID: <20060115161942.10EB327B66@code1.codespeak.net> Author: arigo Date: Sun Jan 15 17:19:38 2006 New Revision: 22199 Added: pypy/dist/pypy/jit/test/test_jit_tlr.py (contents, props changed) Modified: pypy/dist/pypy/jit/test/test_jit_tl.py Log: Checking in another example toy language test that I wrote some time ago for demonstration purposes. Modified: pypy/dist/pypy/jit/test/test_jit_tl.py ============================================================================== --- pypy/dist/pypy/jit/test/test_jit_tl.py (original) +++ pypy/dist/pypy/jit/test/test_jit_tl.py Sun Jan 15 17:19:38 2006 @@ -6,6 +6,7 @@ from pypy.jit.llabstractinterp import LLAbstractInterp from pypy.rpython.rstr import string_repr from pypy.rpython.llinterp import LLInterpreter +from pypy.jit.test.test_llabstractinterp import summary #from pypy.translator.backendopt import inline #py.test.skip("in-progress") @@ -35,12 +36,7 @@ #interp.graphs[0].show() # return a summary of the instructions left in graph2 - insns = {} - for copygraph in interp.itercopygraphs(): - for block in copygraph.iterblocks(): - for op in block.operations: - insns[op.opname] = insns.get(op.opname, 0) + 1 - return insns + return summary(interp) def run_jit(code): Added: pypy/dist/pypy/jit/test/test_jit_tlr.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/jit/test/test_jit_tlr.py Sun Jan 15 17:19:38 2006 @@ -0,0 +1,102 @@ +from pypy.annotation import model as annmodel +from pypy.annotation.listdef import ListDef +from pypy.translator.translator import TranslationContext +from pypy.jit.llabstractinterp import LLAbstractInterp +from pypy.jit.test.test_llabstractinterp import summary +from pypy.rpython.llinterp import LLInterpreter +from pypy.rpython.objectmodel import hint +from pypy.rpython.rstr import string_repr + +MOV_A_R = 1 +MOV_R_A = 2 +JUMP_IF_A = 3 +SET_A = 4 +ADD_R_TO_A = 5 +RETURN_A = 6 +ALLOCATE = 7 +NEG_A = 8 + + +def interpret(bytecode, a): + """Another Toy Language interpreter, this one register-based.""" + regs = [] + pc = 0 + while True: + opcode = hint(ord(bytecode[pc]), concrete=True) + pc += 1 + if opcode == MOV_A_R: + n = ord(bytecode[pc]) + pc += 1 + regs[n] = a + elif opcode == MOV_R_A: + n = ord(bytecode[pc]) + pc += 1 + a = regs[n] + elif opcode == JUMP_IF_A: + target = ord(bytecode[pc]) + pc += 1 + if a: + pc = target + elif opcode == SET_A: + a = ord(bytecode[pc]) + pc += 1 + elif opcode == ADD_R_TO_A: + n = ord(bytecode[pc]) + pc += 1 + a += regs[n] + elif opcode == RETURN_A: + return a + elif opcode == ALLOCATE: + n = ord(bytecode[pc]) + pc += 1 + regs = [0] * n + elif opcode == NEG_A: + a = -a + +SQUARE_LIST = [ + # compute the square of 'a' >= 1 + ALLOCATE, 3, + MOV_A_R, 0, # counter + MOV_A_R, 1, # copy of 'a' + SET_A, 0, + MOV_A_R, 2, # accumulator for the result + # 10: + SET_A, 1, + NEG_A, + ADD_R_TO_A, 0, + MOV_A_R, 0, + MOV_R_A, 2, + ADD_R_TO_A, 1, + MOV_A_R, 2, + MOV_R_A, 0, + JUMP_IF_A, 10, + + MOV_R_A, 2, + RETURN_A ] + +SQUARE = ''.join([chr(n) for n in SQUARE_LIST]) + + +def test_multiply(): + assert interpret(SQUARE, 1) == 1 + assert interpret(SQUARE, 7) == 49 + assert interpret(SQUARE, 9) == 81 + +def test_compile(): + t = TranslationContext() + t.buildannotator().build_types(interpret, [str, int]) + rtyper = t.buildrtyper() + rtyper.specialize() + + interp = LLAbstractInterp() + hints = {0: string_repr.convert_const(SQUARE)} + graph2 = interp.eval(t.graphs[0], hints) + #graph2.show() + + llinterp = LLInterpreter(rtyper) + res = llinterp.eval_graph(graph2, [17]) + assert res == 289 + + insns = summary(interp) + assert insns == {'int_add': 2, + 'int_is_true': 1} From arigo at codespeak.net Sun Jan 15 17:35:03 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sun, 15 Jan 2006 17:35:03 +0100 (CET) Subject: [pypy-svn] r22200 - pypy/dist/pypy/translator Message-ID: <20060115163503.970CF27B68@code1.codespeak.net> Author: arigo Date: Sun Jan 15 17:35:00 2006 New Revision: 22200 Modified: pypy/dist/pypy/translator/interactive.py Log: (pedronis, arigo) For -i usage. Modified: pypy/dist/pypy/translator/interactive.py ============================================================================== --- pypy/dist/pypy/translator/interactive.py (original) +++ pypy/dist/pypy/translator/interactive.py Sun Jan 15 17:35:00 2006 @@ -1,5 +1,6 @@ import optparse +import autopath from pypy.translator.translator import TranslationContext from pypy.translator import driver From arigo at codespeak.net Sun Jan 15 17:35:31 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sun, 15 Jan 2006 17:35:31 +0100 (CET) Subject: [pypy-svn] r22201 - pypy/dist/pypy/rpython Message-ID: <20060115163531.572B227B82@code1.codespeak.net> Author: arigo Date: Sun Jan 15 17:35:29 2006 New Revision: 22201 Modified: pypy/dist/pypy/rpython/rint.py Log: (pedronis, arigo) Probable typo. Modified: pypy/dist/pypy/rpython/rint.py ============================================================================== --- pypy/dist/pypy/rpython/rint.py (original) +++ pypy/dist/pypy/rpython/rint.py Sun Jan 15 17:35:29 2006 @@ -443,7 +443,7 @@ UnsignedLongLong: ('PyLong_FromUnsignedLongLong', lambda i: pyobjectptr(i)), SignedLongLong: ('PyLong_FromLongLong', lambda i: pyobjectptr(i)), Unsigned: ('PyLong_FromUnsignedLong', lambda i: pyobjectptr(i)), - Signed: ('PyLong_FromLong', lambda i: pyobjectptr(i)), + Signed: ('PyInt_FromLong', lambda i: pyobjectptr(i)), } From arigo at codespeak.net Sun Jan 15 18:35:55 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sun, 15 Jan 2006 18:35:55 +0100 (CET) Subject: [pypy-svn] r22202 - pypy/dist/pypy/doc Message-ID: <20060115173555.0971927DB5@code1.codespeak.net> Author: arigo Date: Sun Jan 15 18:35:52 2006 New Revision: 22202 Modified: pypy/dist/pypy/doc/coding-guide.txt Log: Clarified a few paragraphs of the "RPython specs". Modified: pypy/dist/pypy/doc/coding-guide.txt ============================================================================== --- pypy/dist/pypy/doc/coding-guide.txt (original) +++ pypy/dist/pypy/doc/coding-guide.txt Sun Jan 15 18:35:52 2006 @@ -190,13 +190,19 @@ variables should contain values of at most one type as described in `Object restrictions`_ at each control flow point, that means for example that joining control paths using the same variable to - contain both a float and a int should be avoided. Mixing None - (basically with the role of a null pointer) and `wrapped objects` - and class instances is allowed. + contain both a string and a int must be avoided. It is allowed to + mix None (basically with the role of a null pointer) with many other + types: `wrapped objects`, class instances, lists, dicts, strings, etc. + but *not* with int and floats. **constants** - all module globals are considered constants. + all module globals are considered constants. Their binding must not + be changed at run-time. Moreover, global (i.e. prebuilt) lists and + dictionaries are supposed to be immutable: modifying e.g. a global + list will give inconsistent results. However, global instances don't + have this restriction, so if you need mutable global state, store it + in the attributes of some prebuilt singleton instance. **control structures** @@ -280,26 +286,24 @@ arguments (which may be passed as a list instead of a tuple, so write code that does not depend on it being a tuple). -+ dynamic dispatch enforces use of very simple signatures, equal for all - functions to be called in that context. At the moment, this occurs in the - opcode dispatch, only. ++ dynamic dispatch enforces the use of signatures that are equal for all + possible called function, or at least "compatible enough". This + concerns mainly method calls, when the method is overridden or in any + way given different definitions in different classes. It also concerns + the less common case of explicitly manipulated function objects. + Describing the exact compability rules is rather involved (but if you + break them, you should get explicit errors from the rtyper and not + obscure crashes.) **builtin functions** - A few builtin functions will be used, while this set is not defined - completely, yet. Some builtin functions are special forms: - -**len** - -+ may be used with basic types that have a length. But len is a special form - that is recognized by the compiler. - -+ If a certain structure is never touched by len, the compiler might save the - length field from the underlying structure. - -``int, float, ord, chr``... are available as simple conversion functions. - -``int, float, str``... have a special meaning as a type inside of isinstance only. + A number of builtin functions can be used. The precise set can be + found in `pypy/annotation/builtin.py`_ (see ``def builtin_xxx()``). + Some builtin functions may be limited in what they support, though. + + ``int, float, str, ord, chr``... are available as simple conversion + functions. Note that ``int, float, str``... have a special meaning as + a type inside of isinstance only. **classes** @@ -309,9 +313,9 @@ **objects** - wrapped objects are borrowed from the object space. Just like in CPython, - code that needs e.g. a dictionary can use a wrapped dict and the object space - operations on it. + in PyPy, wrapped objects are borrowed from the object space. Just like + in CPython, code that needs e.g. a dictionary can use a wrapped dict + and the object space operations on it. This layout makes the number of types to take care about quite limited. From arigo at codespeak.net Sun Jan 15 18:42:01 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sun, 15 Jan 2006 18:42:01 +0100 (CET) Subject: [pypy-svn] r22203 - pypy/dist/pypy/doc Message-ID: <20060115174201.6013C27DB8@code1.codespeak.net> Author: arigo Date: Sun Jan 15 18:41:58 2006 New Revision: 22203 Modified: pypy/dist/pypy/doc/constraints-and-logic.txt Log: ReSTified. Modified: pypy/dist/pypy/doc/constraints-and-logic.txt ============================================================================== --- pypy/dist/pypy/doc/constraints-and-logic.txt (original) +++ pypy/dist/pypy/doc/constraints-and-logic.txt Sun Jan 15 18:41:58 2006 @@ -1,7 +1,6 @@ -====================================================================== - Ideas about syntactic and algorithmic aspects of Constraint and Logic - Programming in Python (CLPiP) -====================================================================== +=================================================================================================== +Ideas about syntactic and algorithmic aspects of Constraint and Logic Programming in Python (CLPiP) +=================================================================================================== ** WORK IN PROGRESS ** @@ -70,21 +69,21 @@ Let's start with a quite standard example (the problem being solved here is fully described on -http://www.logilab.org/projects/constraint/documentation) : +http://www.logilab.org/projects/constraint/documentation):: # import Repository class and fd module, from logilab.constraint import * variables = ('c01','c02','c03','c04','c05','c06','c07','c08','c09','c10') -Variables are represented as any string object. +Variables are represented as any string object:: values = [(room,slot) for room in ('room A','room B','room C') for slot in ('day 1 AM','day 1 PM','day 2 AM','day 2 PM')] Values can be freely pre-computed using standard python constructs; -they can be any object; here, tuples of strings. +they can be any object; here, tuples of strings:: domains = {} for v in variables: @@ -93,7 +92,7 @@ The relationship between variables and their possible values is set in a dictionnary whose keys are variable designators (strings). Values are wrapped into FiniteDomain instances (FiniteDomain has set -behaviour, plus some implementation subtleties). +behaviour, plus some implementation subtleties):: groups = (('c01','c02','c03','c10'), ('c02','c06','c08','c09'), @@ -112,7 +111,7 @@ relationship. The example complete with all constraints is provided at the url mentioned supra. -Then, when everything has been settled, comes the last step : +Then, when everything has been settled, comes the last step:: r = Repository(variables,domains,constraints) solutions = Solver().solve(r) @@ -138,7 +137,7 @@ ------------------------------------------ First, promote variables from third-class to second-class -citizenry. Be able to write something like : +citizenry. Be able to write something like:: domain = [(room,slot) for room in ('room A','room B','room C') @@ -146,24 +145,24 @@ c01 := domain c02 := domain -This introduces a special operator ':=' which binds a logical variable -to a domain. More generally : +This introduces a special operator `:=` which binds a logical variable +to a domain. More generally:: var := -With respect to normal assignment, we can imagine the following : +With respect to normal assignment, we can imagine the following:: c01 = 'foo' # raises a NotAssignable or ReadOnly exception bar = c01 # takes a reference to the current value of c01 into bar # also, meaningless (so ... None) before the solver has run -Problem ... we can't anymore do : +Problem ... we can't anymore do:: for conf in ('c01','c05','c10'): ... It should be good to define a kind of first-class designator for these kind of variables. A specially-crafted class representing variables -(in the manner of Lisp's symbols) would suffice : +(in the manner of Lisp's symbols) would suffice:: for conf in (c01, c05, c10): ... @@ -173,7 +172,7 @@ ---------------------- An alternative which avoids the special operator and uses a keyword -instead could be : +instead could be:: domain: c01 = @@ -182,7 +181,7 @@ It makes us reuse '=', with twisted (non-standard Python) semantics but under a clear lexical umbrella (a 'domain:' block). -It is possible to get further in this direction : +It is possible to get further in this direction:: problem toto: D1 = @@ -202,12 +201,12 @@ -------------------- The ugly aspect of py-constraints is the definition of custom -unary/binary constraints through make_expression, as in : +unary/binary constraints through make_expression, as in:: fd.make_expression ('var1', 'var2', "frob(var1,var2)") One solution might be to parse the string at runtime to recover the -variable names : +variable names:: fd.make_expression ('frob(var1,var2)') @@ -232,7 +231,7 @@ Databases and Requests ---------------------- -Databases contain facts, rules and requests. +Databases contain facts, rules and requests:: >>> database Numbers: # we def a logic db and enter prolog space ... natural(z) @@ -248,6 +247,8 @@ Integration in Python --------------------- +:: + # databases as first class objects get_logic_databases() # -> [Numbers] @@ -273,13 +274,13 @@ Thus, the logic and Python stuff are cleanly separated, and of course -we forbid lexical capture as in : +we forbid lexical capture as in:: z = 42 database Numbers: natural(z) -A (maybe) nicer way to ask could be : +A (maybe) nicer way to ask could be:: database Family : From arigo at codespeak.net Sun Jan 15 18:45:58 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sun, 15 Jan 2006 18:45:58 +0100 (CET) Subject: [pypy-svn] r22204 - pypy/dist/pypy/doc Message-ID: <20060115174558.B572427DBB@code1.codespeak.net> Author: arigo Date: Sun Jan 15 18:45:56 2006 New Revision: 22204 Modified: pypy/dist/pypy/doc/constraints-and-logic.txt Log: Pypy -> PyPy Modified: pypy/dist/pypy/doc/constraints-and-logic.txt ============================================================================== --- pypy/dist/pypy/doc/constraints-and-logic.txt (original) +++ pypy/dist/pypy/doc/constraints-and-logic.txt Sun Jan 15 18:45:56 2006 @@ -11,18 +11,18 @@ ----- This document tries to shed some light on integration of logic and -constraint programming into Python using the Pypy +constraint programming into Python using the PyPy framework. -Pypy has been progressively equiped with a parser and compiler +PyPy has been progressively equiped with a parser and compiler flexible enough that it is hoped that developpers can leverage it to extend the language *at runtime*. This is quite in the spirit of Lisp macros, if not the exact manner. -This takes place in Working Packages 09 and 10 of the EU Pypy funding +This takes place in Working Packages 09 and 10 of the EU PyPy funding project. It is expected that an aspect oriented programming toolkit be built using the compiler and parser infrastructure (WP10), and that the -logic and constraint programming features be added to Pypy (WP9). An +logic and constraint programming features be added to PyPy (WP9). An ontology library will be provided and will serve as our first use case for logic programming. @@ -30,8 +30,8 @@ -------------- This work was described as integration of logic programming *and* -constraint programming into Pypy. Both are obviously related but we -don't know if they will share, in Pypy, the same syntax and the same +constraint programming into PyPy. Both are obviously related but we +don't know if they will share, in PyPy, the same syntax and the same machinery. Also it has been said that CLP will be inspired from what is done in @@ -42,11 +42,11 @@ programming (the later can be 'embedded' in the former); or constraint programming goes with stateless concurrency, another important feature of Oz. We still don't know to what extent the Oz level of integration -will be pursued in the context of Pypy, esp. WP 9 and 10. +will be pursued in the context of PyPy, esp. WP 9 and 10. Lastly, here we mainly discuss syntactical issues: those are probably the least difficult aspects of getting CLP into python; getting an -efficient implementation of the canonical algorithms into Pypy will be +efficient implementation of the canonical algorithms into PyPy will be the bulk of the work. Constraint programming @@ -145,7 +145,7 @@ c01 := domain c02 := domain -This introduces a special operator `:=` which binds a logical variable +This introduces a special operator ``:=`` which binds a logical variable to a domain. More generally:: var := @@ -178,8 +178,8 @@ c01 = c02 = -It makes us reuse '=', with twisted (non-standard Python) semantics -but under a clear lexical umbrella (a 'domain:' block). +It makes us reuse ``=``, with twisted (non-standard Python) semantics +but under a clear lexical umbrella (a ``domain:`` block). It is possible to get further in this direction:: @@ -314,14 +314,14 @@ For constraint programming: * Adapt the core algorithms from logilab.constraints to RPython -* Stuff it into Pypy +* Stuff it into PyPy * Enhance (tm) it with concurrency aspects (using tis's tasklets) Logic programming: * Investigate how much of above can be reused * Pick a strategy (reuse constraints or implement unification) -* Implement deep into Pypy +* Implement deep into PyPy For both: From cfbolz at codespeak.net Mon Jan 16 10:34:51 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Mon, 16 Jan 2006 10:34:51 +0100 (CET) Subject: [pypy-svn] r22206 - in pypy/dist/pypy/translator/backendopt: . test Message-ID: <20060116093451.2717627B3D@code1.codespeak.net> Author: cfbolz Date: Mon Jan 16 10:34:49 2006 New Revision: 22206 Modified: pypy/dist/pypy/translator/backendopt/escape.py pypy/dist/pypy/translator/backendopt/test/test_escape.py Log: don't just give up on external functions that take arguments that are on the heap Modified: pypy/dist/pypy/translator/backendopt/escape.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/escape.py (original) +++ pypy/dist/pypy/translator/backendopt/escape.py Mon Jan 16 10:34:49 2006 @@ -280,11 +280,16 @@ def direct_call(self, op, function, *args): graph = get_graph(op.args[0], self.translation_context) if graph is None: - if len(filter(None, args)): + if isonheap(op.result): raise NotImplementedError, "can't handle call %s" % (op, ) - else: - result = None - funcargs = [None] * len(args) + for arg in args: + if arg is None: + continue + # an external function can change every parameter: + changed = arg.setchanges() + self.handle_changed(changed) + result = None + funcargs = [None] * len(args) else: result, funcargs = self.schedule_function(graph) assert len(args) == len(funcargs) @@ -292,7 +297,8 @@ if localarg is None: assert funcarg is None continue - self.register_state_dependency(localarg, funcarg) + if funcarg is not None: + self.register_state_dependency(localarg, funcarg) if isonheap(op.result): # assume that a call creates a new value return VarState(self.get_creationpoint(op.result, "direct_call")) Modified: pypy/dist/pypy/translator/backendopt/test/test_escape.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/test/test_escape.py (original) +++ pypy/dist/pypy/translator/backendopt/test/test_escape.py Mon Jan 16 10:34:49 2006 @@ -323,6 +323,17 @@ # does not crash t, adi, graph = build_adi(entrypoint, [int]) +def test_extfunc_onheaparg(): + import os + def f(i): + s = str(i) + os.write(2, s) + return len(s) + t, adi, graph = build_adi(f, [int]) + svar = graph.startblock.operations[0].result + state = adi.getstate(svar) + assert not state.does_escape() + assert state.does_change() #__________________________________________________________ # malloc removal tests From bea at codespeak.net Mon Jan 16 10:52:24 2006 From: bea at codespeak.net (bea at codespeak.net) Date: Mon, 16 Jan 2006 10:52:24 +0100 (CET) Subject: [pypy-svn] r22208 - pypy/extradoc/talk Message-ID: <20060116095224.BD21927B4F@code1.codespeak.net> Author: bea Date: Mon Jan 16 10:52:23 2006 New Revision: 22208 Added: pypy/extradoc/talk/ireland_planning_2006.txt Log: text file for planning of the ireland trip for CM and merlinux -talks at UCD and workshop with IONA 6-7th Febr Added: pypy/extradoc/talk/ireland_planning_2006.txt ============================================================================== --- (empty file) +++ pypy/extradoc/talk/ireland_planning_2006.txt Mon Jan 16 10:52:23 2006 @@ -0,0 +1,68 @@ +Planning Ireland visit (IONA/UCD) +===================== + +Time: 2006-02-06/2006-02-07 + +Participants PyPy: +- Beatrice D?ring/Change Maker +- Holger Krekel/merlinux + + +IONA workshop 2006-02-06 +================= +- contact: Niall Donnelly, manager of engineering operations (Sean Baker) +- agenda 17th of January, 13:00-17:00 +- participants IONA: good cross of management, senior engineers and + product management people, so maybe a group of 6-8 people. +-idea: + a round table meeting at IONA between IONA/PyPy and + in which we explore our different perspectives + in sprinting and agile/distributed work + + +Information about IONA (agile, distributed, OSS): +------------------------------------------------ +- we are a combination of proprietary and open source projects development. +- have 4 development sites - a large number for a company of relatively small size. +- Certainly we are very much interested in agile distributed process. +- OSS too, because we are new-ish to open source development, open source community processes + would be of big interest to us as well. + +- Some of the more formal processes like sprinting (we do 2-week + iterations, and some pairing, but not in a formal manner and my guess is + we are not likely to change this too much either for now), test-driven + (this we do already), and open communnication (do this also), no need to + go into these in much detail, we are fairly familiar with all this. + +- It would be worth a small session on EU funding and requirements (saw + this on a document you co-wrote from the Web). It is not relevant to us + now, but that may only be because we know nothing about it, I think an + introduction to this would be worthwhile, we will certainly learn + something new. + +- work within Eclipse, currently interally but also soon in a OOS environment. +- Internally, IONA is a C++ and JAVA development house, about 50/50 for proprietary products. +- OSS, we are so far exclusively Java development (but may in the near future also look at C++ OSS stuff). +- An example of an OSS project can be found with Celtix,go to http://forge.objectweb.org/projects/celtix/ + +Methodology and agenda IONA workshop: +---------------------------------------- + +UCD talk 2006-02-07 +============== +- contact: Joe Kiniry, School of Computer Science and Informatics +- agenda: send out 17th of January +- participants: students +- idea: 2 talks at UCD, one technical and one methodology talk (ca 3hrs in total) + +Information about UCD: +----------------------- +I teach three courses concurrently in the second term, all of which focus on software engineering with team-based project development. A lecture +about the PyPy methodology (lightweight, sprint-based development, how you identify and track bugs, assign tasks, communicate, etc.) would be of +interest to many of my students. We could either organise your giving one talk to many of them at the same time (big event) or a few shorter/smaller +talks during normal lecture times (these are < 1hr). + +(Do we fit in to your curriculum?) + +In general, yes, as I lecture on the above topics and emphasise that different teams have different practices, having an outside person confirm this +for the students and give a new point of view would be a Good Thing. From cfbolz at codespeak.net Mon Jan 16 11:53:09 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Mon, 16 Jan 2006 11:53:09 +0100 (CET) Subject: [pypy-svn] r22209 - in pypy/dist/pypy/translator/backendopt: . test Message-ID: <20060116105309.4F2FE27B4C@code1.codespeak.net> Author: cfbolz Date: Mon Jan 16 11:53:08 2006 New Revision: 22209 Modified: pypy/dist/pypy/translator/backendopt/escape.py pypy/dist/pypy/translator/backendopt/test/test_escape.py Log: make external functions work that return something on the heap Modified: pypy/dist/pypy/translator/backendopt/escape.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/escape.py (original) +++ pypy/dist/pypy/translator/backendopt/escape.py Mon Jan 16 11:53:08 2006 @@ -280,15 +280,12 @@ def direct_call(self, op, function, *args): graph = get_graph(op.args[0], self.translation_context) if graph is None: - if isonheap(op.result): - raise NotImplementedError, "can't handle call %s" % (op, ) for arg in args: if arg is None: continue # an external function can change every parameter: changed = arg.setchanges() self.handle_changed(changed) - result = None funcargs = [None] * len(args) else: result, funcargs = self.schedule_function(graph) Modified: pypy/dist/pypy/translator/backendopt/test/test_escape.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/test/test_escape.py (original) +++ pypy/dist/pypy/translator/backendopt/test/test_escape.py Mon Jan 16 11:53:08 2006 @@ -334,7 +334,17 @@ state = adi.getstate(svar) assert not state.does_escape() assert state.does_change() - + +def test_extfunc_resultonheap(): + import os + def f(i): + s = str(i) + return len(s) + t, adi, graph = build_adi(f, [float]) + svar = graph.startblock.operations[0].result + state = adi.getstate(svar) + assert not state.does_escape() + #__________________________________________________________ # malloc removal tests From rxe at codespeak.net Mon Jan 16 12:08:02 2006 From: rxe at codespeak.net (rxe at codespeak.net) Date: Mon, 16 Jan 2006 12:08:02 +0100 (CET) Subject: [pypy-svn] r22210 - pypy/dist/pypy/translator/llvm Message-ID: <20060116110802.767B727B4F@code1.codespeak.net> Author: rxe Date: Mon Jan 16 12:08:01 2006 New Revision: 22210 Modified: pypy/dist/pypy/translator/llvm/codewriter.py pypy/dist/pypy/translator/llvm/genllvm.py Log: Add an explicit close to codewriter. Modified: pypy/dist/pypy/translator/llvm/codewriter.py ============================================================================== --- pypy/dist/pypy/translator/llvm/codewriter.py (original) +++ pypy/dist/pypy/translator/llvm/codewriter.py Mon Jan 16 12:08:01 2006 @@ -10,6 +10,9 @@ self.file = file self.word_repr = db.get_machine_word() + def close(self): + self.file.close() + def _resolvetail(self, tail, cconv): # from: http://llvm.cs.uiuc.edu/docs/LangRef.html # The optional "tail" marker indicates whether the callee function Modified: pypy/dist/pypy/translator/llvm/genllvm.py ============================================================================== --- pypy/dist/pypy/translator/llvm/genllvm.py (original) +++ pypy/dist/pypy/translator/llvm/genllvm.py Mon Jan 16 12:08:01 2006 @@ -63,6 +63,7 @@ self.source_generated = True self._checkpoint('done') + codewriter.close() return self.filename def setup(self, func): From mwh at codespeak.net Mon Jan 16 12:41:59 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Mon, 16 Jan 2006 12:41:59 +0100 (CET) Subject: [pypy-svn] r22213 - in pypy/extradoc/talk/pycon2006: . pypy-selfcontained.key pypy-selfcontained.key/Contents pypy-selfcontained.key/thumbs Message-ID: <20060116114159.754E527B52@code1.codespeak.net> Author: mwh Date: Mon Jan 16 12:41:52 2006 New Revision: 22213 Added: pypy/extradoc/talk/pycon2006/pypy-selfcontained.key/ pypy/extradoc/talk/pycon2006/pypy-selfcontained.key/.typeAttributes.dict pypy/extradoc/talk/pycon2006/pypy-selfcontained.key/Contents/ pypy/extradoc/talk/pycon2006/pypy-selfcontained.key/Contents/PkgInfo pypy/extradoc/talk/pycon2006/pypy-selfcontained.key/index.apxl.gz (contents, props changed) pypy/extradoc/talk/pycon2006/pypy-selfcontained.key/py-web1.tiff (contents, props changed) pypy/extradoc/talk/pycon2006/pypy-selfcontained.key/thumbs/ pypy/extradoc/talk/pycon2006/pypy-selfcontained.key/thumbs/mt0-0.tiff (contents, props changed) pypy/extradoc/talk/pycon2006/pypy-selfcontained.key/thumbs/mt0-10.tiff (contents, props changed) pypy/extradoc/talk/pycon2006/pypy-selfcontained.key/thumbs/mt0-4.tiff (contents, props changed) pypy/extradoc/talk/pycon2006/pypy-selfcontained.key/thumbs/st0.tiff (contents, props changed) pypy/extradoc/talk/pycon2006/pypy-selfcontained.key/thumbs/st1.tiff (contents, props changed) pypy/extradoc/talk/pycon2006/pypy-selfcontained.key/thumbs/st10.tiff (contents, props changed) pypy/extradoc/talk/pycon2006/pypy-selfcontained.key/thumbs/st2.tiff (contents, props changed) pypy/extradoc/talk/pycon2006/pypy-selfcontained.key/thumbs/st3.tiff (contents, props changed) pypy/extradoc/talk/pycon2006/pypy-selfcontained.key/thumbs/st4.tiff (contents, props changed) pypy/extradoc/talk/pycon2006/pypy-selfcontained.key/thumbs/st5.tiff (contents, props changed) pypy/extradoc/talk/pycon2006/pypy-selfcontained.key/thumbs/st6.tiff (contents, props changed) pypy/extradoc/talk/pycon2006/pypy-selfcontained.key/thumbs/st7-1.tiff (contents, props changed) pypy/extradoc/talk/pycon2006/pypy-selfcontained.key/thumbs/st7.tiff (contents, props changed) pypy/extradoc/talk/pycon2006/pypy-selfcontained.key/thumbs/st8.tiff (contents, props changed) pypy/extradoc/talk/pycon2006/pypy-selfcontained.key/thumbs/st9.tiff (contents, props changed) pypy/extradoc/talk/pycon2006/pypy-selfcontained.pdf (contents, props changed) Log: a first cut at a talk for pycon. very high level, not much meat -- but how much meat can you cram in to 30 minutes? look at the pdf if you don't have keynote (most of you, presumably) Added: pypy/extradoc/talk/pycon2006/pypy-selfcontained.key/.typeAttributes.dict ============================================================================== Added: pypy/extradoc/talk/pycon2006/pypy-selfcontained.key/Contents/PkgInfo ============================================================================== --- (empty file) +++ pypy/extradoc/talk/pycon2006/pypy-selfcontained.key/Contents/PkgInfo Mon Jan 16 12:41:52 2006 @@ -0,0 +1 @@ +???????? \ No newline at end of file Added: pypy/extradoc/talk/pycon2006/pypy-selfcontained.key/index.apxl.gz ============================================================================== Binary file. No diff available. Added: pypy/extradoc/talk/pycon2006/pypy-selfcontained.key/py-web1.tiff ============================================================================== Binary file. No diff available. Added: pypy/extradoc/talk/pycon2006/pypy-selfcontained.key/thumbs/mt0-0.tiff ============================================================================== Binary file. No diff available. Added: pypy/extradoc/talk/pycon2006/pypy-selfcontained.key/thumbs/mt0-10.tiff ============================================================================== Binary file. No diff available. Added: pypy/extradoc/talk/pycon2006/pypy-selfcontained.key/thumbs/mt0-4.tiff ============================================================================== Binary file. No diff available. Added: pypy/extradoc/talk/pycon2006/pypy-selfcontained.key/thumbs/st0.tiff ============================================================================== Binary file. No diff available. Added: pypy/extradoc/talk/pycon2006/pypy-selfcontained.key/thumbs/st1.tiff ============================================================================== Binary file. No diff available. Added: pypy/extradoc/talk/pycon2006/pypy-selfcontained.key/thumbs/st10.tiff ============================================================================== Binary file. No diff available. Added: pypy/extradoc/talk/pycon2006/pypy-selfcontained.key/thumbs/st2.tiff ============================================================================== Binary file. No diff available. Added: pypy/extradoc/talk/pycon2006/pypy-selfcontained.key/thumbs/st3.tiff ============================================================================== Binary file. No diff available. Added: pypy/extradoc/talk/pycon2006/pypy-selfcontained.key/thumbs/st4.tiff ============================================================================== Binary file. No diff available. Added: pypy/extradoc/talk/pycon2006/pypy-selfcontained.key/thumbs/st5.tiff ============================================================================== Binary file. No diff available. Added: pypy/extradoc/talk/pycon2006/pypy-selfcontained.key/thumbs/st6.tiff ============================================================================== Binary file. No diff available. Added: pypy/extradoc/talk/pycon2006/pypy-selfcontained.key/thumbs/st7-1.tiff ============================================================================== Binary file. No diff available. Added: pypy/extradoc/talk/pycon2006/pypy-selfcontained.key/thumbs/st7.tiff ============================================================================== Binary file. No diff available. Added: pypy/extradoc/talk/pycon2006/pypy-selfcontained.key/thumbs/st8.tiff ============================================================================== Binary file. No diff available. Added: pypy/extradoc/talk/pycon2006/pypy-selfcontained.key/thumbs/st9.tiff ============================================================================== Binary file. No diff available. Added: pypy/extradoc/talk/pycon2006/pypy-selfcontained.pdf ============================================================================== Binary file. No diff available. From bea at codespeak.net Mon Jan 16 12:48:08 2006 From: bea at codespeak.net (bea at codespeak.net) Date: Mon, 16 Jan 2006 12:48:08 +0100 (CET) Subject: [pypy-svn] r22214 - pypy/extradoc/planning Message-ID: <20060116114808.A68E827B5A@code1.codespeak.net> Author: bea Date: Mon Jan 16 12:48:07 2006 New Revision: 22214 Modified: pypy/extradoc/planning/conference-planning.txt Log: XP Finland conference had extended their deadline for submissions - to the 30th of Jan Modified: pypy/extradoc/planning/conference-planning.txt ============================================================================== --- pypy/extradoc/planning/conference-planning.txt (original) +++ pypy/extradoc/planning/conference-planning.txt Mon Jan 16 12:48:07 2006 @@ -108,7 +108,7 @@ Time & location: 17-22th June 2006 Oulu, Finland Call for papers: -Submission date: 13th of January 2006 +Submission date: 30th of January 2006 (Bea plan to submit an experience report) http://virtual.vtt.fi/virtual/xp2006/sivu1.html From hpk at codespeak.net Mon Jan 16 13:02:04 2006 From: hpk at codespeak.net (hpk at codespeak.net) Date: Mon, 16 Jan 2006 13:02:04 +0100 (CET) Subject: [pypy-svn] r22215 - in pypy: eu-tracking/talk extradoc/talk Message-ID: <20060116120204.8C78027B5A@code1.codespeak.net> Author: hpk Date: Mon Jan 16 13:02:02 2006 New Revision: 22215 Added: pypy/eu-tracking/talk/ireland_planning_2006.txt - copied unchanged from r22214, pypy/extradoc/talk/ireland_planning_2006.txt Removed: pypy/extradoc/talk/ireland_planning_2006.txt Log: move to different location From cfbolz at codespeak.net Mon Jan 16 15:21:22 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Mon, 16 Jan 2006 15:21:22 +0100 (CET) Subject: [pypy-svn] r22221 - in pypy/dist/pypy/translator/backendopt: . test Message-ID: <20060116142122.D373D27B54@code1.codespeak.net> Author: cfbolz Date: Mon Jan 16 15:21:21 2006 New Revision: 22221 Modified: pypy/dist/pypy/translator/backendopt/escape.py pypy/dist/pypy/translator/backendopt/test/test_escape.py Log: detect loops Modified: pypy/dist/pypy/translator/backendopt/escape.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/escape.py (original) +++ pypy/dist/pypy/translator/backendopt/escape.py Mon Jan 16 15:21:21 2006 @@ -338,6 +338,59 @@ return False return True +def find_backedges(graph): + """finds the backedges in the flow graph""" + scheduled = [graph.startblock] + seen = {} + backedges = [] + while scheduled: + current = scheduled.pop() + seen[current] = True + for link in current.exits: + if link.target in seen: + backedges.append(link) + else: + scheduled.append(link.target) + return backedges + +def compute_reachability(graph): + reachable = {} + for block in graph.iterblocks(): + reach = {} + scheduled = [block] + while scheduled: + current = scheduled.pop() + for link in current.exits: + if link.target in reachable: + reach = setunion(reach, reachable[link.target]) + continue + if link.target not in reach: + reach[link.target] = True + reachable[block] = reach + return reachable + +def find_loop_blocks(graph): + """find the blocks in a graph that are part of a loop""" + loop = {} + reachable = compute_reachability(graph) + for backedge in find_backedges(graph): + start = backedge.target + end = backedge.prevblock + loop[start] = start + loop[end] = start + scheduled = [start] + seen = {} + while scheduled: + current = scheduled.pop() + connects = end in reachable[current] + seen[current] = True + if connects: + loop[current] = start + for link in current.exits: + if link.target not in seen: + scheduled.append(link.target) + return loop + def malloc_to_stack(t): aib = AbstractDataFlowInterpreter(t) for graph in t.graphs: Modified: pypy/dist/pypy/translator/backendopt/test/test_escape.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/test/test_escape.py (original) +++ pypy/dist/pypy/translator/backendopt/test/test_escape.py Mon Jan 16 15:21:21 2006 @@ -1,5 +1,6 @@ from pypy.translator.translator import TranslationContext, graphof from pypy.translator.backendopt.escape import AbstractDataFlowInterpreter, malloc_to_stack +from pypy.translator.backendopt.escape import find_backedges, find_loop_blocks from pypy.rpython.llinterp import LLInterpreter def build_adi(function, types): @@ -346,6 +347,33 @@ assert not state.does_escape() #__________________________________________________________ +# test loop detection + +def test_find_backedges(): + def f(k): + result = 0 + for i in range(k): + result += 1 + for j in range(k): + result += 1 + return result + t, adi, graph = build_adi(f, [int]) + backedges = find_backedges(graph) + assert len(backedges) == 2 + +def test_find_loop_blocks(): + def f(k): + result = 0 + for i in range(k): + result += 1 + for j in range(k): + result += 1 + return result + t, adi, graph = build_adi(f, [int]) + loop_blocks = find_loop_blocks(graph) + assert len(loop_blocks) == 4 + +#__________________________________________________________ # malloc removal tests def test_remove_simple(): From cfbolz at codespeak.net Mon Jan 16 15:40:05 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Mon, 16 Jan 2006 15:40:05 +0100 (CET) Subject: [pypy-svn] r22223 - in pypy/dist/pypy/translator/backendopt: . test Message-ID: <20060116144005.E886027B5A@code1.codespeak.net> Author: cfbolz Date: Mon Jan 16 15:40:04 2006 New Revision: 22223 Modified: pypy/dist/pypy/translator/backendopt/escape.py pypy/dist/pypy/translator/backendopt/test/test_escape.py Log: don't convert mallocs that are in loops to allocas Modified: pypy/dist/pypy/translator/backendopt/escape.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/escape.py (original) +++ pypy/dist/pypy/translator/backendopt/escape.py Mon Jan 16 15:40:04 2006 @@ -394,6 +394,7 @@ def malloc_to_stack(t): aib = AbstractDataFlowInterpreter(t) for graph in t.graphs: + loop_blocks = find_loop_blocks(graph) for block in graph.iterblocks(): for op in block.operations: if op.opname == 'malloc': @@ -403,7 +404,7 @@ varstate = aib.getstate(op.result) assert len(varstate.creation_points) == 1 crep = varstate.creation_points.keys()[0] - if not crep.escapes: + if not crep.escapes and block not in loop_blocks: print "moving object from heap to stack %s in %s" % (op, graph.name) op.opname = 'flavored_malloc' op.args.insert(0, inputconst(lltype.Void, 'stack')) Modified: pypy/dist/pypy/translator/backendopt/test/test_escape.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/test/test_escape.py (original) +++ pypy/dist/pypy/translator/backendopt/test/test_escape.py Mon Jan 16 15:40:04 2006 @@ -412,3 +412,16 @@ return g(a) t = check_malloc_removal(f, [], [], 4) +def test_dont_alloca_in_loops(): + class A(object): + pass + def f(x): + result = 0 + for i in range(x): + a = A() + a.i = i + result += a.i + return result + t = check_malloc_removal(f, [int], [3], 3,must_remove=False) + graph = graphof(t, f) + assert graph.startblock.exits[0].target.exits[0].target.operations[0].opname == "malloc" From ac at codespeak.net Mon Jan 16 17:29:58 2006 From: ac at codespeak.net (ac at codespeak.net) Date: Mon, 16 Jan 2006 17:29:58 +0100 (CET) Subject: [pypy-svn] r22227 - pypy/branch/arre-experiments/pypy/interpreter Message-ID: <20060116162958.B194127B55@code1.codespeak.net> Author: ac Date: Mon Jan 16 17:29:58 2006 New Revision: 22227 Modified: pypy/branch/arre-experiments/pypy/interpreter/argument.py pypy/branch/arre-experiments/pypy/interpreter/miscutils.py pypy/branch/arre-experiments/pypy/interpreter/pyopcode.py Log: Specialcase positional arguments on valuestack. Modified: pypy/branch/arre-experiments/pypy/interpreter/argument.py ============================================================================== --- pypy/branch/arre-experiments/pypy/interpreter/argument.py (original) +++ pypy/branch/arre-experiments/pypy/interpreter/argument.py Mon Jan 16 17:29:58 2006 @@ -5,6 +5,17 @@ from pypy.interpreter.error import OperationError class AbstractArguments: + + def parse(self, fnname, signature, defaults_w=[]): + """Parse args and kwargs to initialize a frame + according to the signature of code object. + """ + try: + return self.match_signature(signature, defaults_w) + except ArgErr, e: + raise OperationError(self.space.w_TypeError, + self.space.wrap(e.getmsg(fnname))) + def frompacked(space, w_args=None, w_kwds=None): """Convenience static method to build an Arguments from a wrapped sequence and a wrapped dictionary.""" @@ -37,6 +48,7 @@ class ArgumentsPrepended(AbstractArguments): def __init__(self, args, w_firstarg): + self.space = args.space self.args = args self.w_firstarg = w_firstarg @@ -62,16 +74,6 @@ def _rawshape(self, nextra=0): return self.args._rawshape(nextra + 1) - def parse(self, fnname, signature, defaults_w=[]): - """Parse args and kwargs to initialize a frame - according to the signature of code object. - """ - try: - return self.match_signature(signature, defaults_w) - except ArgErr, e: - raise OperationError(self.args.space.w_TypeError, - self.args.space.wrap(e.getmsg(fnname))) - def match_signature(self, signature, defaults_w=[]): """Parse args and kwargs according to the signature of a code object, or raise an ArgErr in case of failure. @@ -106,6 +108,100 @@ def num_kwds(self): return self.args.num_kwds() +class ArgumentsFromValuestack(AbstractArguments): + """ + Collects the arguments of a fuction call as stored on a PyFrame valuestack. + + Only for the case of purely positional arguments, for now. + """ + + def __init__(self, space, valuestack, nargs=0): + self.space = space + self.valuestack = valuestack + self.nargs = nargs + + def firstarg(self): + if self.nargs <= 0: + return None + return valuestack.top(nargs - 1) + + def __repr__(self): + return 'ArgumentsFromValuestack(%r, %r)' % (self.valuestack, self.nargs) + + def has_keywords(self): + return False + + def unpack(self): + args_w = [None] * self.nargs + for i in range(self.nargs): + args_w[i] = self.valuestack.top(self.nargs - 1 - i) + return args_w, {} + + def fixedunpack(self, argcount): + if self.nargs > argcount: + raise ValueError, "too many arguments (%d expected)" % argcount + elif self.nargs < argcount: + raise ValueError, "not enough arguments (%d expected)" % argcount + data_w = [None] * self.nargs + for i in range(self.nargs): + data_w[i] = self.valuestack.top(nargs - 1 - i) + return data_w + + def _rawshape(self, nextra=0): + return nextra + self.nargs, (), False, False + + def match_signature(self, signature, defaults_w=[]): + argnames, varargname, kwargname = signature + co_argcount = len(argnames) + if self.nargs + len(defaults_w) < co_argcount: + raise ArgErrCount(self, signature, defaults_w, + co_argcount - self.nargs - len(defaults_w)) + if self.nargs > co_argcount and varargname is None: + raise ArgErrCount(self, signature, defaults_w, 0) + + scopesize = co_argcount + if varargname: + scopesize += 1 + if kwargname: + scopesize += 1 + scope_w = [None] * scopesize + if self.nargs >= co_argcount: + for i in range(co_argcount): + scope_w[i] = self.valuestack.top(self.nargs - 1 - i) + if varargname is not None: + stararg_w = [None] * (self.nargs - co_argcount) + for i in range(co_argcount, self.nargs): + stararg_w[i - co_argcount] = self.valuestack.top(self.nargs - 1 - i) + scope_w[co_argcount] = self.space.newtuple(stararg_w) + co_argcount += 1 + else: + for i in range(self.nargs): + scope_w[i] = self.valuestack.top(self.nargs - 1 - i) + ndefaults = len(defaults_w) + missing = co_argcount - self.nargs + first_default = ndefaults - missing + for i in range(missing): + scope_w[self.nargs + i] = defaults_w[first_default + i] + if varargname is not None: + scope_w[co_argcount] = self.space.newtuple([]) + co_argcount += 1 + + if kwargname is not None: + scope_w[co_argcount] = self.space.newdict([]) + return scope_w + + def flatten(self): + data_w = [None] * self.nargs + for i in range(self.nargs): + data_w[i] = self.valuestack.top(nargs - 1 - i) + return nextra + self.nargs, (), False, False, data_w + + def num_args(self): + return self.nargs + + def num_kwds(self): + return 0 + class Arguments(AbstractArguments): """ Collects the arguments of a function call. @@ -226,17 +322,6 @@ ### Parsing for function calls ### - def parse(self, fnname, signature, defaults_w=[]): - """Parse args and kwargs to initialize a frame - according to the signature of code object. - """ - try: - return self.match_signature(signature, defaults_w) - except ArgErr, e: - raise OperationError(self.space.w_TypeError, - self.space.wrap(e.getmsg(fnname))) - - def match_signature(self, signature, defaults_w=[]): """Parse args and kwargs according to the signature of a code object, or raise an ArgErr in case of failure. Modified: pypy/branch/arre-experiments/pypy/interpreter/miscutils.py ============================================================================== --- pypy/branch/arre-experiments/pypy/interpreter/miscutils.py (original) +++ pypy/branch/arre-experiments/pypy/interpreter/miscutils.py Mon Jan 16 17:29:58 2006 @@ -33,6 +33,10 @@ def pop(self): return self.items.pop() + def drop(self, n): + if n > 0: + del self.items[-n:] + def top(self, position=0): """'position' is 0 for the top of the stack, 1 for the item below, and so on. It must not be negative.""" @@ -84,6 +88,12 @@ self.ptr = ptr return ret + def drop(self, n): + while n > 0: + n -= 1 + self.ptr -= 1 + self.items[self.ptr] = None + def top(self, position=0): # for a fixed stack, we assume correct indices return self.items[self.ptr + ~position] Modified: pypy/branch/arre-experiments/pypy/interpreter/pyopcode.py ============================================================================== --- pypy/branch/arre-experiments/pypy/interpreter/pyopcode.py (original) +++ pypy/branch/arre-experiments/pypy/interpreter/pyopcode.py Mon Jan 16 17:29:58 2006 @@ -9,7 +9,7 @@ from pypy.interpreter import gateway, function, eval from pypy.interpreter import pyframe, pytraceback from pypy.interpreter.miscutils import InitializedClass -from pypy.interpreter.argument import Arguments +from pypy.interpreter.argument import Arguments, ArgumentsFromValuestack from pypy.interpreter.pycode import PyCode from pypy.tool.sourcetools import func_with_new_name from pypy.rpython.objectmodel import we_are_translated @@ -678,6 +678,14 @@ w_function = f.valuestack.pop() w_result = f.space.call_function(w_function, w_arg1, w_arg2, w_arg3) f.valuestack.push(w_result) + elif (oparg >> 8) & 0xff == 0: + # Only positional arguments + nargs = oparg & 0xff + args = ArgumentsFromValuestack(f.space, f.valuestack, nargs) + w_function = f.valuestack.top(nargs) + w_result = f.space.call_args(w_function, args) + f.valuestack.drop(nargs + 1) + f.valuestack.push(w_result) # XXX end of hack for performance else: # general case From ac at codespeak.net Mon Jan 16 18:13:37 2006 From: ac at codespeak.net (ac at codespeak.net) Date: Mon, 16 Jan 2006 18:13:37 +0100 (CET) Subject: [pypy-svn] r22230 - pypy/branch/arre-experiments/pypy/interpreter Message-ID: <20060116171337.B33E527B60@code1.codespeak.net> Author: ac Date: Mon Jan 16 18:13:37 2006 New Revision: 22230 Modified: pypy/branch/arre-experiments/pypy/interpreter/argument.py pypy/branch/arre-experiments/pypy/interpreter/pyopcode.py Log: Fix some typos. Modified: pypy/branch/arre-experiments/pypy/interpreter/argument.py ============================================================================== --- pypy/branch/arre-experiments/pypy/interpreter/argument.py (original) +++ pypy/branch/arre-experiments/pypy/interpreter/argument.py Mon Jan 16 18:13:37 2006 @@ -123,7 +123,7 @@ def firstarg(self): if self.nargs <= 0: return None - return valuestack.top(nargs - 1) + return self.valuestack.top(self.nargs - 1) def __repr__(self): return 'ArgumentsFromValuestack(%r, %r)' % (self.valuestack, self.nargs) Modified: pypy/branch/arre-experiments/pypy/interpreter/pyopcode.py ============================================================================== --- pypy/branch/arre-experiments/pypy/interpreter/pyopcode.py (original) +++ pypy/branch/arre-experiments/pypy/interpreter/pyopcode.py Mon Jan 16 18:13:37 2006 @@ -683,8 +683,10 @@ nargs = oparg & 0xff args = ArgumentsFromValuestack(f.space, f.valuestack, nargs) w_function = f.valuestack.top(nargs) - w_result = f.space.call_args(w_function, args) - f.valuestack.drop(nargs + 1) + try: + w_result = f.space.call_args(w_function, args) + finally: + f.valuestack.drop(nargs + 1) f.valuestack.push(w_result) # XXX end of hack for performance else: From cfbolz at codespeak.net Mon Jan 16 18:18:30 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Mon, 16 Jan 2006 18:18:30 +0100 (CET) Subject: [pypy-svn] r22231 - in pypy/dist/pypy/translator/backendopt: . test Message-ID: <20060116171830.D518527B64@code1.codespeak.net> Author: cfbolz Date: Mon Jan 16 18:18:29 2006 New Revision: 22231 Modified: pypy/dist/pypy/translator/backendopt/escape.py pypy/dist/pypy/translator/backendopt/test/test_escape.py Log: implemet getarraysubstruct + test Modified: pypy/dist/pypy/translator/backendopt/escape.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/escape.py (original) +++ pypy/dist/pypy/translator/backendopt/escape.py Mon Jan 16 18:18:29 2006 @@ -274,6 +274,12 @@ # state, the same creationpoints, etc. return objstate + def getarraysubstruct(self, op, arraystate, indexstate): + # since this is really a struct embedded somewhere in the array it has + # the same state, creationpoints, etc. in most cases the resulting + # pointer should not be used much anyway + return arraystate + def getarraysize(self, op, arraystate): pass @@ -323,6 +329,8 @@ def ptr_eq(self, op, ptr1state, ptr2state): return None + ptr_neq = ptr_eq + def same_as(self, op, objstate): return objstate Modified: pypy/dist/pypy/translator/backendopt/test/test_escape.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/test/test_escape.py (original) +++ pypy/dist/pypy/translator/backendopt/test/test_escape.py Mon Jan 16 18:18:29 2006 @@ -165,6 +165,29 @@ assert acrep.changes assert not acrep.escapes +def test_dependencies(): + class A(object): + pass + globala = A() + globala.l = [1] + def g(a): + a.append(1) + globala.l = a + def f(): + a = [0] + a.append(1) + # g(a) + return globala.l[0] + t, adi, graph = build_adi(f, []) + t.view() + avar = graph.startblock.operations[0].result + astate = adi.getstate(avar) + appendgraph = graph.startblock.operations[3].args[0].value._obj.graph + resizegraph = appendgraph.startblock.operations[2].args[0].value._obj.graph + reallygraph = resizegraph.startblock.exits[0].target.operations[0].args[0].value._obj.graph +# assert astate.does_escape() + assert astate.does_change() + def test_substruct(): class A(object): pass @@ -307,6 +330,22 @@ # does not crash t, adi, graph = build_adi(f, [int]) +def test_getarraysubstruct(): + def createdict(i, j): + d = {'hello' : 23, + 'world' : 21} + l = ["hello", "world"] + return d[l[i]] + d[l[j]] + # does not crash, for now + t, adi, graph = build_adi(createdict, [int, int]) + t.view() + dvar = graph.startblock.operations[0].result + lvar = graph.startblock.operations[3].result + dstate = adi.getstate(dvar) + lstate = adi.getstate(lvar) + assert dstate.does_change() + assert lstate.does_change() + def test_raise_escapes(): def f(): a = ValueError() @@ -346,6 +385,7 @@ state = adi.getstate(svar) assert not state.does_escape() + #__________________________________________________________ # test loop detection @@ -422,6 +462,6 @@ a.i = i result += a.i return result - t = check_malloc_removal(f, [int], [3], 3,must_remove=False) + t = check_malloc_removal(f, [int], [3], 3, must_remove=False) graph = graphof(t, f) assert graph.startblock.exits[0].target.exits[0].target.operations[0].opname == "malloc" From cfbolz at codespeak.net Mon Jan 16 18:19:40 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Mon, 16 Jan 2006 18:19:40 +0100 (CET) Subject: [pypy-svn] r22232 - pypy/dist/pypy/translator/backendopt/test Message-ID: <20060116171940.36CDF27B60@code1.codespeak.net> Author: cfbolz Date: Mon Jan 16 18:19:39 2006 New Revision: 22232 Modified: pypy/dist/pypy/translator/backendopt/test/test_escape.py Log: oops: this test is work in progress Modified: pypy/dist/pypy/translator/backendopt/test/test_escape.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/test/test_escape.py (original) +++ pypy/dist/pypy/translator/backendopt/test/test_escape.py Mon Jan 16 18:19:39 2006 @@ -165,7 +165,7 @@ assert acrep.changes assert not acrep.escapes -def test_dependencies(): +def DONOTtest_dependencies(): class A(object): pass globala = A() From ericvrp at codespeak.net Mon Jan 16 18:29:34 2006 From: ericvrp at codespeak.net (ericvrp at codespeak.net) Date: Mon, 16 Jan 2006 18:29:34 +0100 (CET) Subject: [pypy-svn] r22233 - pypy/dist/pypy/translator/backendopt Message-ID: <20060116172934.60B6727B60@code1.codespeak.net> Author: ericvrp Date: Mon Jan 16 18:29:33 2006 New Revision: 22233 Modified: pypy/dist/pypy/translator/backendopt/escape.py Log: fixed typo Modified: pypy/dist/pypy/translator/backendopt/escape.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/escape.py (original) +++ pypy/dist/pypy/translator/backendopt/escape.py Mon Jan 16 18:29:33 2006 @@ -329,7 +329,7 @@ def ptr_eq(self, op, ptr1state, ptr2state): return None - ptr_neq = ptr_eq + ptr_ne = ptr_eq def same_as(self, op, objstate): return objstate From cfbolz at codespeak.net Mon Jan 16 18:45:06 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Mon, 16 Jan 2006 18:45:06 +0100 (CET) Subject: [pypy-svn] r22234 - in pypy/dist/pypy/translator/backendopt: . test Message-ID: <20060116174506.1141027B60@code1.codespeak.net> Author: cfbolz Date: Mon Jan 16 18:45:04 2006 New Revision: 22234 Modified: pypy/dist/pypy/translator/backendopt/escape.py pypy/dist/pypy/translator/backendopt/test/test_escape.py Log: fixing bug in change register_state_dependency simplifying getarraysubstruct test Modified: pypy/dist/pypy/translator/backendopt/escape.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/escape.py (original) +++ pypy/dist/pypy/translator/backendopt/escape.py Mon Jan 16 18:45:04 2006 @@ -210,12 +210,13 @@ def register_state_dependency(self, state1, state2): "state1 depends on state2: if state2 does escape/change, so does state1" # change state1 according to how state2 is now + print "registering dependency of %s on %s" % (state1, state2) escapes = state2.does_escape() if escapes and not state1.does_escape(): changed = state1.setescapes() self.handle_changed(changed) changes = state2.does_change() - if changes and not state1: + if changes and not state1.does_change(): changed = state1.setchanges() self.handle_changed(changed) # register a dependency of the current block on state2: Modified: pypy/dist/pypy/translator/backendopt/test/test_escape.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/test/test_escape.py (original) +++ pypy/dist/pypy/translator/backendopt/test/test_escape.py Mon Jan 16 18:45:04 2006 @@ -165,7 +165,7 @@ assert acrep.changes assert not acrep.escapes -def DONOTtest_dependencies(): +def test_dependencies(): class A(object): pass globala = A() @@ -179,13 +179,20 @@ # g(a) return globala.l[0] t, adi, graph = build_adi(f, []) - t.view() avar = graph.startblock.operations[0].result astate = adi.getstate(avar) appendgraph = graph.startblock.operations[3].args[0].value._obj.graph + appendarg0 = appendgraph.startblock.inputargs[0] + appendstate = adi.getstate(appendarg0) resizegraph = appendgraph.startblock.operations[2].args[0].value._obj.graph + resizearg0 = resizegraph.startblock.inputargs[0] + resizestate = adi.getstate(resizearg0) reallygraph = resizegraph.startblock.exits[0].target.operations[0].args[0].value._obj.graph -# assert astate.does_escape() + reallyarg0 = reallygraph.startblock.inputargs[0] + reallystate = adi.getstate(reallyarg0) + assert reallystate.does_change() + assert resizestate.does_change() + assert appendstate.does_change() assert astate.does_change() def test_substruct(): @@ -332,19 +339,15 @@ def test_getarraysubstruct(): def createdict(i, j): - d = {'hello' : 23, - 'world' : 21} - l = ["hello", "world"] - return d[l[i]] + d[l[j]] + d = {2 : 23, + 3 : 21} + return d[i] + d[j] # does not crash, for now t, adi, graph = build_adi(createdict, [int, int]) - t.view() dvar = graph.startblock.operations[0].result - lvar = graph.startblock.operations[3].result dstate = adi.getstate(dvar) - lstate = adi.getstate(lvar) assert dstate.does_change() - assert lstate.does_change() + assert not dstate.does_escape() def test_raise_escapes(): def f(): From cfbolz at codespeak.net Mon Jan 16 19:39:26 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Mon, 16 Jan 2006 19:39:26 +0100 (CET) Subject: [pypy-svn] r22235 - in pypy/dist/pypy/translator/backendopt: . test Message-ID: <20060116183926.8112527B60@code1.codespeak.net> Author: cfbolz Date: Mon Jan 16 19:39:25 2006 New Revision: 22235 Modified: pypy/dist/pypy/translator/backendopt/escape.py pypy/dist/pypy/translator/backendopt/test/test_escape.py Log: fix indirect_call when the set of graphs that can be called is unknown Modified: pypy/dist/pypy/translator/backendopt/escape.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/escape.py (original) +++ pypy/dist/pypy/translator/backendopt/escape.py Mon Jan 16 19:39:25 2006 @@ -310,14 +310,23 @@ def indirect_call(self, op, function, *args): graphs = op.args[-1].value args = args[:-1] - for graph in graphs: - result, funcargs = self.schedule_function(graph) - assert len(args) == len(funcargs) - for localarg, funcarg in zip(args, funcargs): + if graphs is None: + for localarg in args: if localarg is None: - assert funcarg is None continue - self.register_state_dependency(localarg, funcarg) + changed = localarg.setescapes() + self.handle_changed(changed) + changed = localarg.setchanges() + self.handle_changed(changed) + else: + for graph in graphs: + result, funcargs = self.schedule_function(graph) + assert len(args) == len(funcargs) + for localarg, funcarg in zip(args, funcargs): + if localarg is None: + assert funcarg is None + continue + self.register_state_dependency(localarg, funcarg) if isonheap(op.result): # assume that a call creates a new value return VarState(self.get_creationpoint(op.result, "indirect_call")) Modified: pypy/dist/pypy/translator/backendopt/test/test_escape.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/test/test_escape.py (original) +++ pypy/dist/pypy/translator/backendopt/test/test_escape.py Mon Jan 16 19:39:25 2006 @@ -2,6 +2,7 @@ from pypy.translator.backendopt.escape import AbstractDataFlowInterpreter, malloc_to_stack from pypy.translator.backendopt.escape import find_backedges, find_loop_blocks from pypy.rpython.llinterp import LLInterpreter +from pypy.rpython.objectmodel import instantiate def build_adi(function, types): t = TranslationContext() @@ -293,6 +294,20 @@ assert a1crep.changes and a2crep.changes assert not a1crep.escapes and a2crep.escapes +def test_indirect_call_unknown_graphs(): + class A: + pass + class B: + pass + def f(i): + if i: + klass = A + else: + klass = B + a = instantiate(klass) + # does not crash + t, adi, graph = build_adi(f, [int]) + def test_getarray(): class A(object): pass From arigo at codespeak.net Mon Jan 16 21:31:57 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Mon, 16 Jan 2006 21:31:57 +0100 (CET) Subject: [pypy-svn] r22237 - pypy/dist/pypy/translator/backendopt Message-ID: <20060116203157.1059727B60@code1.codespeak.net> Author: arigo Date: Mon Jan 16 21:31:55 2006 New Revision: 22237 Modified: pypy/dist/pypy/translator/backendopt/escape.py Log: Typo Modified: pypy/dist/pypy/translator/backendopt/escape.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/escape.py (original) +++ pypy/dist/pypy/translator/backendopt/escape.py Mon Jan 16 21:31:55 2006 @@ -31,7 +31,6 @@ return True def merge(self, other): - newstate = VarState() creation_points = setunion(self.creation_points, other.creation_points) newstate = VarState() newstate.creation_points = creation_points From arigo at codespeak.net Mon Jan 16 21:46:40 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Mon, 16 Jan 2006 21:46:40 +0100 (CET) Subject: [pypy-svn] r22238 - pypy/dist/pypy/translator/backendopt Message-ID: <20060116204640.5244D27B60@code1.codespeak.net> Author: arigo Date: Mon Jan 16 21:46:38 2006 New Revision: 22238 Modified: pypy/dist/pypy/translator/backendopt/escape.py Log: - typoish bug fix - direct uses of popitem() and update() -- in particular, iterkeys().next() followed by pop() is extremely inefficient (wrong complexity!) Modified: pypy/dist/pypy/translator/backendopt/escape.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/escape.py (original) +++ pypy/dist/pypy/translator/backendopt/escape.py Mon Jan 16 21:46:38 2006 @@ -140,7 +140,7 @@ if isonheap(block.inputargs[0]): changed = self.getstate(block.inputargs[0]).setescapes() self.handle_changed(changed) - if isonheap(block.inputargs[0]): + if isonheap(block.inputargs[1]): changed = self.getstate(block.inputargs[1]).setescapes() self.handle_changed(changed) return @@ -188,16 +188,14 @@ def complete(self): while self.scheduled: - block = self.scheduled.iterkeys().next() - graph = self.scheduled.pop(block) + block, graph = self.scheduled.popitem() self.flow_block(block, graph) def handle_changed(self, changed): for crep in changed: if crep not in self.dependencies: continue - for block, graph in self.dependencies[crep].iteritems(): - self.scheduled[block] = graph + self.scheduled.update(self.dependencies[crep]) def register_block_dependency(self, state, block=None, graph=None): if block is None: From adim at codespeak.net Mon Jan 16 22:08:45 2006 From: adim at codespeak.net (adim at codespeak.net) Date: Mon, 16 Jan 2006 22:08:45 +0100 (CET) Subject: [pypy-svn] r22239 - pypy/dist/pypy/interpreter Message-ID: <20060116210845.DF2C427B60@code1.codespeak.net> Author: adim Date: Mon Jan 16 22:08:45 2006 New Revision: 22239 Modified: pypy/dist/pypy/interpreter/pycompiler.py Log: - use the AST returned by the compiler hook - renamed compile_hook into w_compile_hook (it really is a wrapped function) Modified: pypy/dist/pypy/interpreter/pycompiler.py ============================================================================== --- pypy/dist/pypy/interpreter/pycompiler.py (original) +++ pypy/dist/pypy/interpreter/pycompiler.py Mon Jan 16 22:08:45 2006 @@ -13,7 +13,7 @@ def __init__(self, space): self.space = space - self.compile_hook = None + self.w_compile_hook = space.w_None def compile(self, source, filename, mode, flags): """Compile and return an pypy.interpreter.eval.Code instance.""" @@ -204,7 +204,8 @@ from pypy.interpreter.astcompiler.pycodegen import ModuleCodeGenerator from pypy.interpreter.astcompiler.pycodegen import InteractiveCodeGenerator from pypy.interpreter.astcompiler.pycodegen import ExpressionCodeGenerator - from pyparser.pythonutil import AstBuilder, PYTHON_PARSER, TARGET_DICT + from pypy.interpreter.astcompiler.ast import Node + from pyparser.pythonutil import AstBuilder, PYTHON_PARSER, TARGET_DICT from pypy.interpreter.pycode import PyCode flags |= __future__.generators.compiler_flag # always on (2.2 compat) @@ -219,10 +220,11 @@ raise OperationError(space.w_SyntaxError, e.wrap_info(space, filename)) - if self.compile_hook is not None: - new_tree = space.call_function(self.compile_hook, - space.wrap(ast_tree), - space.wrap(encoding)) + if not space.is_w(self.w_compile_hook, space.w_None): + w_ast_tree = space.call_function(self.w_compile_hook, + space.wrap(ast_tree), + space.wrap(encoding)) + ast_tree = space.interp_w(Node, w_ast_tree) try: astcompiler.misc.set_filename(filename, ast_tree) flag_names = get_flag_names(space, flags) @@ -246,9 +248,6 @@ def install_compiler_hook(space, w_callable): - if space.is_w(w_callable, space.w_None): - space.default_compiler.compile_hook = None - else: # if not space.get( w_callable ): # raise OperationError( space.w_TypeError( space.wrap( "must have a callable" ) ) - space.default_compiler.compile_hook = w_callable + space.default_compiler.w_compile_hook = w_callable From aft at codespeak.net Mon Jan 16 22:08:57 2006 From: aft at codespeak.net (aft at codespeak.net) Date: Mon, 16 Jan 2006 22:08:57 +0100 (CET) Subject: [pypy-svn] r22240 - pypy/extradoc/sprintinfo/mallorca Message-ID: <20060116210857.41E5B27B6C@code1.codespeak.net> Author: aft Date: Mon Jan 16 22:08:55 2006 New Revision: 22240 Modified: pypy/extradoc/sprintinfo/mallorca/people.txt Log: Added A.Thompson (AFT) to Mallorca Attendance Modified: pypy/extradoc/sprintinfo/mallorca/people.txt ============================================================================== --- pypy/extradoc/sprintinfo/mallorca/people.txt (original) +++ pypy/extradoc/sprintinfo/mallorca/people.txt Mon Jan 16 22:08:55 2006 @@ -23,6 +23,7 @@ Anders Lehmann 22/1 -29/1 Hotel Almudainal Holger Krekel ? ? Richard Emslie 22/1 - 29/1 catalonia hotel +Andrew Thompson 22/1 - 29/1 Hotel Terminus ==================== ============== ===================== People on the following list were present at previous sprints: @@ -40,6 +41,5 @@ Amaury Forgeot d'Arc ? ? Valentino Volonghi ? ? Boris Feigin ? ? -Andrew Thompson ? ? Bert Freudenberg ? ? ==================== ============== ===================== From arigo at codespeak.net Mon Jan 16 22:09:36 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Mon, 16 Jan 2006 22:09:36 +0100 (CET) Subject: [pypy-svn] r22241 - pypy/dist/pypy/translator/backendopt Message-ID: <20060116210936.7B6E927B60@code1.codespeak.net> Author: arigo Date: Mon Jan 16 22:09:35 2006 New Revision: 22241 Modified: pypy/dist/pypy/translator/backendopt/escape.py Log: 'op_' prefix to avoid name clashes. Yup, I know it's easy to fix someone else's code and pretend I don't know that the annotator itself has the same problem :-/ Modified: pypy/dist/pypy/translator/backendopt/escape.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/escape.py (original) +++ pypy/dist/pypy/translator/backendopt/escape.py Mon Jan 16 22:09:35 2006 @@ -177,7 +177,7 @@ print "handling", op args = self.getstates(op.args) print "args:", args - opimpl = getattr(self, op.opname, None) + opimpl = getattr(self, 'op_'+op.opname, None) if opimpl is None: if isonheap(op.result) or filter(None, args): raise NotImplementedError("can't handle %s" % (op.opname, )) @@ -224,16 +224,16 @@ # _____________________________________________________________________ # operation implementations - def malloc(self, op, typestate): + def op_malloc(self, op, typestate): return VarState(self.get_creationpoint(op.result, "malloc")) - def malloc_varsize(self, op, typestate, lengthstate): + def op_malloc_varsize(self, op, typestate, lengthstate): return VarState(self.get_creationpoint(op.result, "malloc_varsize")) - def cast_pointer(self, op, state): + def op_cast_pointer(self, op, state): return state - def setfield(self, op, objstate, fieldname, valuestate): + def op_setfield(self, op, objstate, fieldname, valuestate): changed = objstate.setchanges() self.handle_changed(changed) if valuestate is not None: @@ -246,7 +246,7 @@ self.handle_changed(changed) return None - def setarrayitem(self, op, objstate, indexstate, valuestate): + def op_setarrayitem(self, op, objstate, indexstate, valuestate): changed = objstate.setchanges() self.handle_changed(changed) if valuestate is not None: @@ -258,30 +258,30 @@ self.handle_changed(changed) return None - def getarrayitem(self, op, objstate, indexstate): + def op_getarrayitem(self, op, objstate, indexstate): if isonheap(op.result): return VarState(self.get_creationpoint(op.result, "getarrayitem")) - def getfield(self, op, objstate, fieldname): + def op_getfield(self, op, objstate, fieldname): if isonheap(op.result): # assume that getfield creates a new value return VarState(self.get_creationpoint(op.result, "getfield")) - def getsubstruct(self, op, objstate, fieldname): + def op_getsubstruct(self, op, objstate, fieldname): # since this is really an embedded struct, it has the same # state, the same creationpoints, etc. return objstate - def getarraysubstruct(self, op, arraystate, indexstate): + def op_getarraysubstruct(self, op, arraystate, indexstate): # since this is really a struct embedded somewhere in the array it has # the same state, creationpoints, etc. in most cases the resulting # pointer should not be used much anyway return arraystate - def getarraysize(self, op, arraystate): + def op_getarraysize(self, op, arraystate): pass - def direct_call(self, op, function, *args): + def op_direct_call(self, op, function, *args): graph = get_graph(op.args[0], self.translation_context) if graph is None: for arg in args: @@ -304,7 +304,7 @@ # assume that a call creates a new value return VarState(self.get_creationpoint(op.result, "direct_call")) - def indirect_call(self, op, function, *args): + def op_indirect_call(self, op, function, *args): graphs = op.args[-1].value args = args[:-1] if graphs is None: @@ -328,17 +328,17 @@ # assume that a call creates a new value return VarState(self.get_creationpoint(op.result, "indirect_call")) - def ptr_iszero(self, op, ptrstate): + def op_ptr_iszero(self, op, ptrstate): return None - cast_ptr_to_int = keepalive = ptr_nonzero = ptr_iszero + op_cast_ptr_to_int = op_keepalive = op_ptr_nonzero = op_ptr_iszero - def ptr_eq(self, op, ptr1state, ptr2state): + def op_ptr_eq(self, op, ptr1state, ptr2state): return None - ptr_ne = ptr_eq + op_ptr_ne = op_ptr_eq - def same_as(self, op, objstate): + def op_same_as(self, op, objstate): return objstate def isonheap(var_or_const): From cfbolz at codespeak.net Mon Jan 16 23:31:42 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Mon, 16 Jan 2006 23:31:42 +0100 (CET) Subject: [pypy-svn] r22242 - pypy/dist/pypy/translator/backendopt Message-ID: <20060116223142.6879627B64@code1.codespeak.net> Author: cfbolz Date: Mon Jan 16 23:31:41 2006 New Revision: 22242 Modified: pypy/dist/pypy/translator/backendopt/escape.py Log: * comment out most debug prints as they take a lot of time * actually use the functionargs cache (thanks samuele) * finish the analysis _before_ changing something -- duh Modified: pypy/dist/pypy/translator/backendopt/escape.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/escape.py (original) +++ pypy/dist/pypy/translator/backendopt/escape.py Mon Jan 16 23:31:41 2006 @@ -110,7 +110,7 @@ return crep def schedule_function(self, graph): - print "scheduling function:", graph.name + #print "scheduling function:", graph.name startblock = graph.startblock if graph in self.functionargs: args = self.functionargs[graph] @@ -125,6 +125,7 @@ self.setstate(var, varstate) args.append(varstate) self.scheduled[startblock] = graph + self.functionargs[graph] = args resultstate = self.getstate(graph.returnblock.inputargs[0]) return resultstate, args @@ -146,42 +147,42 @@ return self.curr_block = block self.curr_graph = graph - print "inputargs", self.getstates(block.inputargs) + #print "inputargs", self.getstates(block.inputargs) for op in block.operations: self.flow_operation(op) - print "checking exits..." + #print "checking exits..." for exit in block.exits: - print "exit", exit + #print "exit", exit args = self.getstates(exit.args) targetargs = self.getstates(exit.target.inputargs) - print " newargs", args - print " targetargs", targetargs + #print " newargs", args + #print " targetargs", targetargs # flow every block at least once: if (multicontains(targetargs, args) and exit.target in self.flown_blocks): - print " not necessary" + #print " not necessary" continue - else: - print " scheduling for flowin" + #else: + #print " scheduling for flowin" for prevstate, origstate, var in zip(args, targetargs, exit.target.inputargs): if not isonheap(var): continue newstate = prevstate.merge(origstate) self.setstate(var, newstate) - print " args", self.getstates(exit.target.inputargs) + #print " args", self.getstates(exit.target.inputargs) self.scheduled[exit.target] = graph def flow_operation(self, op): - print "handling", op + #print "handling", op args = self.getstates(op.args) - print "args:", args + #print "args:", args opimpl = getattr(self, 'op_'+op.opname, None) if opimpl is None: if isonheap(op.result) or filter(None, args): raise NotImplementedError("can't handle %s" % (op.opname, )) - print "assuming that '%s' is irrelevant" % op + #print "assuming that '%s' is irrelevant" % op return res = opimpl(op, *args) self.setstate(op.result, res) @@ -207,7 +208,7 @@ def register_state_dependency(self, state1, state2): "state1 depends on state2: if state2 does escape/change, so does state1" # change state1 according to how state2 is now - print "registering dependency of %s on %s" % (state1, state2) + #print "registering dependency of %s on %s" % (state1, state2) escapes = state2.does_escape() if escapes and not state1.does_escape(): changed = state1.setescapes() @@ -409,17 +410,21 @@ def malloc_to_stack(t): aib = AbstractDataFlowInterpreter(t) for graph in t.graphs: + if graph.startblock not in aib.flown_blocks: + aib.schedule_function(graph) + aib.complete() + for graph in t.graphs: loop_blocks = find_loop_blocks(graph) for block in graph.iterblocks(): for op in block.operations: if op.opname == 'malloc': - if graph.startblock not in aib.flown_blocks: - aib.schedule_function(graph) - aib.complete() varstate = aib.getstate(op.result) assert len(varstate.creation_points) == 1 crep = varstate.creation_points.keys()[0] - if not crep.escapes and block not in loop_blocks: - print "moving object from heap to stack %s in %s" % (op, graph.name) - op.opname = 'flavored_malloc' - op.args.insert(0, inputconst(lltype.Void, 'stack')) + if not crep.escapes: + if block not in loop_blocks: + print "moving object from heap to stack %s in %s" % (op, graph.name) + op.opname = 'flavored_malloc' + op.args.insert(0, inputconst(lltype.Void, 'stack')) + else: + print "%s in %s is a non-escaping malloc in a loop" % (op, graph.name) From cfbolz at codespeak.net Tue Jan 17 02:25:31 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Tue, 17 Jan 2006 02:25:31 +0100 (CET) Subject: [pypy-svn] r22243 - pypy/dist/pypy/translator/backendopt Message-ID: <20060117012531.BB26E27B5A@code1.codespeak.net> Author: cfbolz Date: Tue Jan 17 02:25:30 2006 New Revision: 22243 Modified: pypy/dist/pypy/translator/backendopt/escape.py Log: just assume the worst about operations that are unknown Modified: pypy/dist/pypy/translator/backendopt/escape.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/escape.py (original) +++ pypy/dist/pypy/translator/backendopt/escape.py Tue Jan 17 02:25:30 2006 @@ -181,7 +181,13 @@ opimpl = getattr(self, 'op_'+op.opname, None) if opimpl is None: if isonheap(op.result) or filter(None, args): - raise NotImplementedError("can't handle %s" % (op.opname, )) + for arg in args: + if arg is not None: + changed = arg.setchanges() + self.handle_changed(changed) + changed = arg.setescapes() + self.handle_changed(changed) + #raise NotImplementedError("can't handle %s" % (op.opname, )) #print "assuming that '%s' is irrelevant" % op return res = opimpl(op, *args) From mwh at codespeak.net Tue Jan 17 12:42:10 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Tue, 17 Jan 2006 12:42:10 +0100 (CET) Subject: [pypy-svn] r22247 - pypy/dist/pypy/rpython/memory/test Message-ID: <20060117114210.CCC3827B43@code1.codespeak.net> Author: mwh Date: Tue Jan 17 12:42:09 2006 New Revision: 22247 Modified: pypy/dist/pypy/rpython/memory/test/test_address.py Log: don't give the tested struct a name of "('x', Bool)" :) Modified: pypy/dist/pypy/rpython/memory/test/test_address.py ============================================================================== --- pypy/dist/pypy/rpython/memory/test/test_address.py (original) +++ pypy/dist/pypy/rpython/memory/test/test_address.py Tue Jan 17 12:42:09 2006 @@ -115,7 +115,7 @@ def test_simple_offsetof(self): from pypy.rpython.lltypesystem import lltype from pypy.rpython.memory.lladdress import offsetof - S = lltype.GcStruct(('x', lltype.Bool), ('y', lltype.Signed)) + S = lltype.GcStruct('S', ('x', lltype.Bool), ('y', lltype.Signed)) def f(): return offsetof(S, 'x') + offsetof(S, 'y') a = RPythonAnnotator() From mwh at codespeak.net Tue Jan 17 12:50:19 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Tue, 17 Jan 2006 12:50:19 +0100 (CET) Subject: [pypy-svn] r22248 - pypy/dist/pypy/rpython/memory Message-ID: <20060117115019.DD8D727B43@code1.codespeak.net> Author: mwh Date: Tue Jan 17 12:50:19 2006 New Revision: 22248 Modified: pypy/dist/pypy/rpython/memory/lladdress.py Log: an assert that would have caught the above mentioned slip, and various others i've been making in the last half hour. Modified: pypy/dist/pypy/rpython/memory/lladdress.py ============================================================================== --- pypy/dist/pypy/rpython/memory/lladdress.py (original) +++ pypy/dist/pypy/rpython/memory/lladdress.py Tue Jan 17 12:50:19 2006 @@ -141,6 +141,7 @@ pass def offsetof(TYPE, fldname): + assert fldname in TYPE._flds return OffsetOf(TYPE, fldname) def itemoffsetof(TYPE, n=None): From ericvrp at codespeak.net Tue Jan 17 13:16:25 2006 From: ericvrp at codespeak.net (ericvrp at codespeak.net) Date: Tue, 17 Jan 2006 13:16:25 +0100 (CET) Subject: [pypy-svn] r22249 - in pypy/dist/pypy: annotation rpython/lltypesystem Message-ID: <20060117121625.5E22427B44@code1.codespeak.net> Author: ericvrp Date: Tue Jan 17 13:16:20 2006 New Revision: 22249 Modified: pypy/dist/pypy/annotation/model.py pypy/dist/pypy/rpython/lltypesystem/lltype.py Log: Some small fixes to not use longlong on non 32-bit systems. (could someone please review this?) Modified: pypy/dist/pypy/annotation/model.py ============================================================================== --- pypy/dist/pypy/annotation/model.py (original) +++ pypy/dist/pypy/annotation/model.py Tue Jan 17 13:16:20 2006 @@ -34,6 +34,7 @@ from pypy.tool.tls import tlsobject from pypy.rpython.rarithmetic import r_uint, r_longlong, r_ulonglong import inspect +from sys import maxint DEBUG = True # set to False to disable recording of debugging information @@ -168,6 +169,8 @@ def __init__(self, nonneg=False, unsigned=False, size=1): self.nonneg = unsigned or nonneg self.unsigned = unsigned # pypy.rpython.rarithmetic.r_uint + if maxint != 2**31-1: + size = 1 #XXX don't support longlong on 64 bits systems self.size = size if self.unsigned: if self.size == 2: Modified: pypy/dist/pypy/rpython/lltypesystem/lltype.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/lltype.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/lltype.py Tue Jan 17 13:16:20 2006 @@ -4,6 +4,7 @@ from pypy.tool.uid import Hashable from pypy.tool.tls import tlsobject from types import NoneType +from sys import maxint log = py.log.Producer('lltype') @@ -428,9 +429,13 @@ Signed = Primitive("Signed", 0) -SignedLongLong = Primitive("SignedLongLong", r_longlong(0)) Unsigned = Primitive("Unsigned", r_uint(0)) -UnsignedLongLong = Primitive("UnsignedLongLong", r_ulonglong(0)) +if maxint == 2**31-1: + SignedLongLong = Primitive("SignedLongLong", r_longlong(0)) + UnsignedLongLong = Primitive("UnsignedLongLong", r_ulonglong(0)) +else: + SignedLongLong = Signed + UnsignedLongLong = Unsigned Float = Primitive("Float", 0.0) Char = Primitive("Char", '\x00') Bool = Primitive("Bool", False) From mwh at codespeak.net Tue Jan 17 13:28:54 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Tue, 17 Jan 2006 13:28:54 +0100 (CET) Subject: [pypy-svn] r22250 - pypy/dist/pypy/annotation Message-ID: <20060117122854.9E5EE27B44@code1.codespeak.net> Author: mwh Date: Tue Jan 17 13:28:53 2006 New Revision: 22250 Modified: pypy/dist/pypy/annotation/bookkeeper.py Log: check for r_int-ness before r_longlong-ness -- makes a difference when they are in fact the same. Modified: pypy/dist/pypy/annotation/bookkeeper.py ============================================================================== --- pypy/dist/pypy/annotation/bookkeeper.py (original) +++ pypy/dist/pypy/annotation/bookkeeper.py Tue Jan 17 13:28:53 2006 @@ -17,7 +17,7 @@ from pypy.annotation.dictdef import DictDef, MOST_GENERAL_DICTDEF from pypy.annotation import description from pypy.interpreter.argument import Arguments, ArgErr -from pypy.rpython.rarithmetic import r_uint, r_ulonglong, r_longlong +from pypy.rpython.rarithmetic import r_int, r_uint, r_ulonglong, r_longlong from pypy.rpython.objectmodel import r_dict, Symbolic from pypy.tool.algo.unionfind import UnionFind from pypy.rpython.lltypesystem import lltype @@ -306,7 +306,7 @@ return x.annotation() if tp is bool: result = SomeBool() - elif tp is int: + elif tp is int or tp is r_int: result = SomeInteger(nonneg = x>=0) elif tp is r_uint: result = SomeInteger(nonneg = True, unsigned = True) @@ -476,7 +476,7 @@ assert isinstance(t, (type, types.ClassType)) if t is bool: return SomeBool() - elif t is int: + elif t is int or t is r_int: return SomeInteger() elif t is r_uint: return SomeInteger(nonneg = True, unsigned = True) From mwh at codespeak.net Tue Jan 17 14:50:32 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Tue, 17 Jan 2006 14:50:32 +0100 (CET) Subject: [pypy-svn] r22251 - pypy/dist/pypy/rpython/memory Message-ID: <20060117135032.C49A727B45@code1.codespeak.net> Author: mwh Date: Tue Jan 17 14:50:31 2006 New Revision: 22251 Modified: pypy/dist/pypy/rpython/memory/gc.py Log: fix misspelt method name. Modified: pypy/dist/pypy/rpython/memory/gc.py ============================================================================== --- pypy/dist/pypy/rpython/memory/gc.py (original) +++ pypy/dist/pypy/rpython/memory/gc.py Tue Jan 17 14:50:31 2006 @@ -263,7 +263,7 @@ if not self.fromspace <= obj < self.fromspace + self.space_size: return self.copy_non_managed_obj(obj) ## print "copying regularly", obj, - if self.is_forwared(obj): + if self.is_forwarded(obj): ## print "already copied to", self.get_forwarding_address(obj) return self.get_forwarding_address(obj) else: @@ -310,7 +310,7 @@ j += 1 i += 1 - def is_forwared(self, obj): + def is_forwarded(self, obj): return (obj - self.size_gc_header()).signed[1] < 0 def get_forwarding_address(self, obj): From cfbolz at codespeak.net Tue Jan 17 15:28:14 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Tue, 17 Jan 2006 15:28:14 +0100 (CET) Subject: [pypy-svn] r22253 - in pypy/dist/pypy/translator/backendopt: . test Message-ID: <20060117142814.0621327B52@code1.codespeak.net> Author: cfbolz Date: Tue Jan 17 15:28:12 2006 New Revision: 22253 Modified: pypy/dist/pypy/translator/backendopt/escape.py pypy/dist/pypy/translator/backendopt/test/test_escape.py Log: as long as gc interaction is unclear at best, don't put allocations of objects to the stack that have a deallocator. Modified: pypy/dist/pypy/translator/backendopt/escape.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/escape.py (original) +++ pypy/dist/pypy/translator/backendopt/escape.py Tue Jan 17 15:28:12 2006 @@ -424,6 +424,14 @@ for block in graph.iterblocks(): for op in block.operations: if op.opname == 'malloc': + STRUCT = op.args[0].value + # must not remove mallocs of structures that have a RTTI with a destructor + try: + destr_ptr = lltype.getRuntimeTypeInfo(STRUCT)._obj.destructor_funcptr + if destr_ptr: + continue + except (ValueError, AttributeError), e: + pass varstate = aib.getstate(op.result) assert len(varstate.creation_points) == 1 crep = varstate.creation_points.keys()[0] Modified: pypy/dist/pypy/translator/backendopt/test/test_escape.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/test/test_escape.py (original) +++ pypy/dist/pypy/translator/backendopt/test/test_escape.py Tue Jan 17 15:28:12 2006 @@ -483,3 +483,17 @@ t = check_malloc_removal(f, [int], [3], 3, must_remove=False) graph = graphof(t, f) assert graph.startblock.exits[0].target.exits[0].target.operations[0].opname == "malloc" + +def test_dont_remove_del_objects(): + class A(object): + def __del__(self): + pass + def f(): + a = A() + a.i = 1 + return a.i + t = check_malloc_removal(f, [], [], 1, must_remove=False) + graph = graphof(t, f) + assert graph.startblock.operations[0].opname == "malloc" + + From adim at codespeak.net Tue Jan 17 16:39:34 2006 From: adim at codespeak.net (adim at codespeak.net) Date: Tue, 17 Jan 2006 16:39:34 +0100 (CET) Subject: [pypy-svn] r22254 - pypy/dist/pypy/interpreter/astcompiler Message-ID: <20060117153934.56A3227B45@code1.codespeak.net> Author: adim Date: Tue Jan 17 16:39:30 2006 New Revision: 22254 Modified: pypy/dist/pypy/interpreter/astcompiler/ast.py pypy/dist/pypy/interpreter/astcompiler/ast.txt pypy/dist/pypy/interpreter/astcompiler/astgen.py Log: lots of ws / tabs fixes Modified: pypy/dist/pypy/interpreter/astcompiler/ast.py ============================================================================== --- pypy/dist/pypy/interpreter/astcompiler/ast.py (original) +++ pypy/dist/pypy/interpreter/astcompiler/ast.py Tue Jan 17 16:39:30 2006 @@ -55,10 +55,10 @@ return res def __repr__(self): - return "Node()" + return "Node()" def descr_repr( self, space ): - return space.wrap( self.__repr__() ) + return space.wrap( self.__repr__() ) def descr_getChildNodes( self, space ): lst = self.getChildNodes() @@ -76,12 +76,12 @@ Node.typedef = TypeDef('ASTNode', __new__ = interp2app(descr_Node_new, unwrap_spec=[ObjSpace, W_Root, int]), - #__repr__ = interp2app(descr_node_repr, unwrap_spec=['self', ObjSpace] ), - getChildNodes = interp2app(Node.descr_getChildNodes, unwrap_spec=[ 'self', ObjSpace ] ), - accept = interp2app(descr_node_accept, unwrap_spec=[ ObjSpace, W_Root, W_Root ] ), + #__repr__ = interp2app(descr_node_repr, unwrap_spec=['self', ObjSpace] ), + getChildNodes = interp2app(Node.descr_getChildNodes, unwrap_spec=[ 'self', ObjSpace ] ), + accept = interp2app(descr_node_accept, unwrap_spec=[ ObjSpace, W_Root, W_Root ] ), lineno = interp_attrproperty('lineno', cls=Node), filename = interp_attrproperty('filename', cls=Node), - ) + ) class EmptyNode(Node): @@ -1212,7 +1212,7 @@ w_opname = space.getitem(w_tuple, space.wrap(0)) w_node = space.getitem(w_tuple, space.wrap(1)) opname = space.str_w(w_opname) - node = space.interp_w(Node, w_node) + node = space.interp_w(Node, w_node) ops.append((opname, node)) self.ops = ops self.lineno = lineno @@ -3869,9 +3869,9 @@ w_expr1 = space.getitem( w_tuple, space.wrap(0) ) w_expr2 = space.getitem( w_tuple, space.wrap(1) ) w_body = space.getitem( w_tuple, space.wrap(2) ) - expr1 = space.interp_w(Node, w_expr1, can_be_None=True) - expr2 = space.interp_w(Node, w_expr2, can_be_None=True) - body = space.interp_w(Node, w_body, can_be_None=False) + expr1 = space.interp_w(Node, w_expr1, can_be_None=True) + expr2 = space.interp_w(Node, w_expr2, can_be_None=True) + body = space.interp_w(Node, w_body, can_be_None=False) handlers.append((expr1, expr2, body)) self.handlers = handlers self.else_ = space.interp_w(Node, w_else_, can_be_None=True) Modified: pypy/dist/pypy/interpreter/astcompiler/ast.txt ============================================================================== --- pypy/dist/pypy/interpreter/astcompiler/ast.txt (original) +++ pypy/dist/pypy/interpreter/astcompiler/ast.txt Tue Jan 17 16:39:30 2006 @@ -290,7 +290,7 @@ w_opname = space.getitem(w_tuple, space.wrap(0)) w_node = space.getitem(w_tuple, space.wrap(1)) opname = space.str_w(w_opname) - node = space.interp_w(Node, w_node) + node = space.interp_w(Node, w_node) ops.append((opname, node)) self.ops = ops self.lineno = lineno @@ -333,9 +333,9 @@ w_expr1 = space.getitem( w_tuple, space.wrap(0) ) w_expr2 = space.getitem( w_tuple, space.wrap(1) ) w_body = space.getitem( w_tuple, space.wrap(2) ) - expr1 = space.interp_w(Node, w_expr1, can_be_None=True) - expr2 = space.interp_w(Node, w_expr2, can_be_None=True) - body = space.interp_w(Node, w_body, can_be_None=False) + expr1 = space.interp_w(Node, w_expr1, can_be_None=True) + expr2 = space.interp_w(Node, w_expr2, can_be_None=True) + body = space.interp_w(Node, w_body, can_be_None=False) handlers.append((expr1, expr2, body)) self.handlers = handlers self.else_ = space.interp_w(Node, w_else_, can_be_None=True) Modified: pypy/dist/pypy/interpreter/astcompiler/astgen.py ============================================================================== --- pypy/dist/pypy/interpreter/astcompiler/astgen.py (original) +++ pypy/dist/pypy/interpreter/astcompiler/astgen.py Tue Jan 17 16:39:30 2006 @@ -57,7 +57,7 @@ self.parent = Node_NodeInfo def get_argnames(self): - args = self.args + args = self.args return [strip_default(arg.strip()) for arg in args.split(',') if arg] @@ -295,56 +295,56 @@ def _gen_fget_func(self, buf, attr, prop ): - # FGET - print >> buf, " def fget_%s( space, self):" % attr - if prop[attr]==P_WRAPPED: - print >> buf, " return self.%s" % attr - elif prop[attr] in (P_INT,P_STR, P_NODE): - print >> buf, " return space.wrap(self.%s)" % attr - elif prop[attr] in (P_INT_LIST, P_STR_LIST, P_NESTED ): - print >> buf, " return space.newlist( [space.wrap(itm) for itm in self.%s] )" % attr - elif prop[attr]==P_NONE: - print >> buf, " if self.%s is None:" % attr - print >> buf, " return space.w_None" - print >> buf, " else:" - print >> buf, " return space.wrap(self.%s)" % attr - else: - assert False, "Unkown node type" + # FGET + print >> buf, " def fget_%s( space, self):" % attr + if prop[attr]==P_WRAPPED: + print >> buf, " return self.%s" % attr + elif prop[attr] in (P_INT,P_STR, P_NODE): + print >> buf, " return space.wrap(self.%s)" % attr + elif prop[attr] in (P_INT_LIST, P_STR_LIST, P_NESTED ): + print >> buf, " return space.newlist( [space.wrap(itm) for itm in self.%s] )" % attr + elif prop[attr]==P_NONE: + print >> buf, " if self.%s is None:" % attr + print >> buf, " return space.w_None" + print >> buf, " else:" + print >> buf, " return space.wrap(self.%s)" % attr + else: + assert False, "Unkown node type" def _gen_fset_func(self, buf, attr, prop ): - # FSET - print >> buf, " def fset_%s( space, self, w_arg):" % attr - if prop[attr] == P_WRAPPED: - print >> buf, " self.%s = w_arg" % attr - elif prop[attr] == P_INT: - print >> buf, " self.%s = space.int_w(w_arg)" % attr - elif prop[attr] == P_STR: - print >> buf, " self.%s = space.str_w(w_arg)" % attr - elif prop[attr] == P_INT_LIST: - print >> buf, " del self.%s[:]" % attr - print >> buf, " for itm in space.unpackiterable(w_arg):" - print >> buf, " self.%s.append( space.int_w(itm) )" % attr - elif prop[attr] == P_STR_LIST: - print >> buf, " del self.%s[:]" % attr - print >> buf, " for itm in space.unpackiterable(w_arg):" - print >> buf, " self.%s.append( space.str_w(itm) )" % attr - elif prop[attr] == P_NESTED: - print >> buf, " del self.%s[:]" % attr - print >> buf, " for w_itm in space.unpackiterable(w_arg):" - print >> buf, " self.%s.append( space.interp_w(Node, w_itm))" % attr - elif prop[attr] == P_NONE: + # FSET + print >> buf, " def fset_%s( space, self, w_arg):" % attr + if prop[attr] == P_WRAPPED: + print >> buf, " self.%s = w_arg" % attr + elif prop[attr] == P_INT: + print >> buf, " self.%s = space.int_w(w_arg)" % attr + elif prop[attr] == P_STR: + print >> buf, " self.%s = space.str_w(w_arg)" % attr + elif prop[attr] == P_INT_LIST: + print >> buf, " del self.%s[:]" % attr + print >> buf, " for itm in space.unpackiterable(w_arg):" + print >> buf, " self.%s.append( space.int_w(itm) )" % attr + elif prop[attr] == P_STR_LIST: + print >> buf, " del self.%s[:]" % attr + print >> buf, " for itm in space.unpackiterable(w_arg):" + print >> buf, " self.%s.append( space.str_w(itm) )" % attr + elif prop[attr] == P_NESTED: + print >> buf, " del self.%s[:]" % attr + print >> buf, " for w_itm in space.unpackiterable(w_arg):" + print >> buf, " self.%s.append( space.interp_w(Node, w_itm))" % attr + elif prop[attr] == P_NONE: print >> buf, " self.%s = space.interp_w(Node, w_arg, can_be_None=True)" % attr - else: # P_NODE + else: # P_NODE print >> buf, " self.%s = space.interp_w(Node, w_arg, can_be_None=False)" % attr def _gen_attrs(self, buf): - prop = self.argprops - for attr in self.argnames: - if "fget_%s" % attr not in self.additional_methods: - self._gen_fget_func( buf, attr, prop ) + prop = self.argprops + for attr in self.argnames: + if "fget_%s" % attr not in self.additional_methods: + self._gen_fget_func( buf, attr, prop ) - if "fset_%s" % attr not in self.additional_methods: - self._gen_fset_func( buf, attr, prop ) + if "fset_%s" % attr not in self.additional_methods: + self._gen_fset_func( buf, attr, prop ) def _gen_typedef(self, buf): initargs = [strip_default(arg.strip()) @@ -353,7 +353,7 @@ new_unwrap_spec = ['ObjSpace', 'W_Root'] + ['W_Root'] * len(initargs) + ['int'] else: new_unwrap_spec = ['ObjSpace', 'W_Root', 'int'] - parent_type = "%s.typedef" % self.parent.name + parent_type = "%s.typedef" % self.parent.name print >> buf, "def descr_%s_accept( space, w_self, w_visitor):" %self.name print >> buf, " w_callable = space.getattr(w_visitor, space.wrap('visit%s'))" % self.name print >> buf, " args = Arguments(space, [ w_self ])" @@ -362,38 +362,38 @@ print >> buf, "%s.typedef = TypeDef('%s', %s, " % (self.name, self.name, parent_type) print >> buf, " __new__ = interp2app(descr_%s_new, unwrap_spec=[%s])," % (self.name, ', '.join(new_unwrap_spec)) print >> buf, " accept=interp2app(descr_%s_accept, unwrap_spec=[ObjSpace, W_Root, W_Root] )," % self.name - for attr in self.argnames: - print >> buf, " %s=GetSetProperty(%s.fget_%s, %s.fset_%s )," % (attr,self.name,attr,self.name,attr) - print >> buf, " )" + for attr in self.argnames: + print >> buf, " %s=GetSetProperty(%s.fget_%s, %s.fset_%s )," % (attr,self.name,attr,self.name,attr) + print >> buf, " )" def _gen_additional_methods(self, buf): for key, value in self.additional_methods.iteritems(): - print >> buf, ''.join(value) + print >> buf, ''.join(value) def gen_base_visit(self, buf): print >> buf, " def visit%s(self, node):" % self.name print >> buf, " return self.default( node )" def gen_print_visit(self, buf): - # This is a print visitor for application level tests + # This is a print visitor for application level tests print >> buf, " def visit%s(self, node):" % self.name - print >> buf, " print '%s('," % self.name - for attr in self.argnames: - if self.argprops[attr] == P_NODE: - print >> buf, " node.%s.accept(self)" % attr - print >> buf, " print ','," - if self.argprops[attr] == P_NONE: - print >> buf, " if node.%s: node.%s.accept(self)" % (attr,attr) - print >> buf, " print ','," - elif self.argprops[attr] == P_NESTED: - print >> buf, " for nd in node.%s:" % attr - print >> buf, " nd.accept(self)" - print >> buf, " print ','," - else: - print >> buf, " print node.%s,','," % attr - print >> buf, " print ')'," - + print >> buf, " print '%s('," % self.name + for attr in self.argnames: + if self.argprops[attr] == P_NODE: + print >> buf, " node.%s.accept(self)" % attr + print >> buf, " print ','," + if self.argprops[attr] == P_NONE: + print >> buf, " if node.%s: node.%s.accept(self)" % (attr,attr) + print >> buf, " print ','," + elif self.argprops[attr] == P_NESTED: + print >> buf, " for nd in node.%s:" % attr + print >> buf, " nd.accept(self)" + print >> buf, " print ','," + else: + print >> buf, " print node.%s,','," % attr + print >> buf, " print ')'," + Node_NodeInfo = NodeInfo("Node","") @@ -408,77 +408,77 @@ kind = None fiter = fileinput.input(file) for line in fiter: - if line.startswith("== OVERRIDES =="): - break + if line.startswith("== OVERRIDES =="): + break comment = line.strip().startswith('#') - if comment: - continue - # a normal entry - try: - name, args = line.split(':') - except ValueError: - continue - if "(" in name: - name, parent = name.split("(") - parent = parent[:-1] - else: - parent = None - classes[name] = NodeInfo(name, args, parent) + if comment: + continue + # a normal entry + try: + name, args = line.split(':') + except ValueError: + continue + if "(" in name: + name, parent = name.split("(") + parent = parent[:-1] + else: + parent = None + classes[name] = NodeInfo(name, args, parent) for line in fiter: mo = None - mo = rx_init.match(line) - if mo: - kind = 'init' - # some extra code for a Node's __init__ method - name = mo.group(1) - cur = classes[name] - continue - - mo = rx_flatten_nodes.match(line) - if mo: - kind = 'flatten_nodes' - # special case for getChildNodes flattening - name = mo.group(1) - attr = mo.group(2) - cur = classes[name] - _cur_ = attr - cur.flatten_nodes[attr] = [] - flatten_expect_comment = True - continue - - mo = rx_additional_methods.match(line) - if mo: - kind = 'additional_method' - name = mo.group(1) - methname = mo.group(2) - params = mo.group(3) - cur = classes[name] - _cur_ = methname - cur.additional_methods[_cur_] = [' def %s(%s):\n' % (methname, params)] - continue - - mo = rx_descr_news_methods.match(line) - if mo: - kind = 'applevel_new' - name = mo.group(1) - cur = classes[name] - cur.applevel_new = [mo.group(0) + '\n'] - continue - - if kind == 'init': - # some code for the __init__ method - cur.init.append(line) - elif kind == 'flatten_nodes': - if flatten_expect_comment: - assert line.strip().startswith("#") - flatten_expect_comment=False - cur.flatten_nodes[_cur_].append(line) - elif kind == 'additional_method': - cur.additional_methods[_cur_].append(' '*4 + line) - elif kind == 'applevel_new': - cur.applevel_new.append(line) + mo = rx_init.match(line) + if mo: + kind = 'init' + # some extra code for a Node's __init__ method + name = mo.group(1) + cur = classes[name] + continue + + mo = rx_flatten_nodes.match(line) + if mo: + kind = 'flatten_nodes' + # special case for getChildNodes flattening + name = mo.group(1) + attr = mo.group(2) + cur = classes[name] + _cur_ = attr + cur.flatten_nodes[attr] = [] + flatten_expect_comment = True + continue + + mo = rx_additional_methods.match(line) + if mo: + kind = 'additional_method' + name = mo.group(1) + methname = mo.group(2) + params = mo.group(3) + cur = classes[name] + _cur_ = methname + cur.additional_methods[_cur_] = [' def %s(%s):\n' % (methname, params)] + continue + + mo = rx_descr_news_methods.match(line) + if mo: + kind = 'applevel_new' + name = mo.group(1) + cur = classes[name] + cur.applevel_new = [mo.group(0) + '\n'] + continue + + if kind == 'init': + # some code for the __init__ method + cur.init.append(line) + elif kind == 'flatten_nodes': + if flatten_expect_comment: + assert line.strip().startswith("#") + flatten_expect_comment=False + cur.flatten_nodes[_cur_].append(line) + elif kind == 'additional_method': + cur.additional_methods[_cur_].append(' '*4 + line) + elif kind == 'applevel_new': + cur.applevel_new.append(line) for node in classes.values(): node.setup_parent(classes) @@ -597,10 +597,10 @@ return res def __repr__(self): - return "Node()" + return "Node()" def descr_repr( self, space ): - return space.wrap( self.__repr__() ) + return space.wrap( self.__repr__() ) def descr_getChildNodes( self, space ): lst = self.getChildNodes() @@ -618,12 +618,12 @@ Node.typedef = TypeDef('ASTNode', __new__ = interp2app(descr_Node_new, unwrap_spec=[ObjSpace, W_Root, int]), - #__repr__ = interp2app(descr_node_repr, unwrap_spec=['self', ObjSpace] ), - getChildNodes = interp2app(Node.descr_getChildNodes, unwrap_spec=[ 'self', ObjSpace ] ), - accept = interp2app(descr_node_accept, unwrap_spec=[ ObjSpace, W_Root, W_Root ] ), + #__repr__ = interp2app(descr_node_repr, unwrap_spec=['self', ObjSpace] ), + getChildNodes = interp2app(Node.descr_getChildNodes, unwrap_spec=[ 'self', ObjSpace ] ), + accept = interp2app(descr_node_accept, unwrap_spec=[ ObjSpace, W_Root, W_Root ] ), lineno = interp_attrproperty('lineno', cls=Node), filename = interp_attrproperty('filename', cls=Node), - ) + ) class EmptyNode(Node): From ac at codespeak.net Wed Jan 18 00:54:40 2006 From: ac at codespeak.net (ac at codespeak.net) Date: Wed, 18 Jan 2006 00:54:40 +0100 (CET) Subject: [pypy-svn] r22265 - pypy/branch/arre-experiments/pypy/interpreter Message-ID: <20060117235440.8BAB527B45@code1.codespeak.net> Author: ac Date: Wed Jan 18 00:54:39 2006 New Revision: 22265 Modified: pypy/branch/arre-experiments/pypy/interpreter/argument.py pypy/branch/arre-experiments/pypy/interpreter/function.py pypy/branch/arre-experiments/pypy/interpreter/gateway.py pypy/branch/arre-experiments/pypy/interpreter/pyopcode.py Log: Some nice progress on the Arguments handling. the microbench for object creation is now down to a factor of ~20 Modified: pypy/branch/arre-experiments/pypy/interpreter/argument.py ============================================================================== --- pypy/branch/arre-experiments/pypy/interpreter/argument.py (original) +++ pypy/branch/arre-experiments/pypy/interpreter/argument.py Wed Jan 18 00:54:39 2006 @@ -3,6 +3,7 @@ """ from pypy.interpreter.error import OperationError +import os class AbstractArguments: @@ -16,6 +17,22 @@ raise OperationError(self.space.w_TypeError, self.space.wrap(e.getmsg(fnname))) + def parse_into_scope(self, scope_w, fnname, signature, defaults_w=[]): + """Parse args and kwargs to initialize a frame + according to the signature of code object. + Store the argumentvalues into scope_w. + scope_w must be big enough for signature. + """ + argnames, varargname, kwargname = signature + has_vararg = varargname is not None + has_kwarg = kwargname is not None + try: + self._match_signature(scope_w, argnames, has_vararg, has_kwarg, + defaults_w, 0, []) + except ArgErr, e: + raise OperationError(self.space.w_TypeError, + self.space.wrap(e.getmsg(fnname))) + def frompacked(space, w_args=None, w_kwds=None): """Convenience static method to build an Arguments from a wrapped sequence and a wrapped dictionary.""" @@ -45,6 +62,22 @@ def prepend(self, w_firstarg): "Return a new Arguments with a new argument inserted first." return ArgumentsPrepended(self, w_firstarg) + + def match_signature(self, signature, defaults_w): + """Parse args and kwargs according to the signature of a code object, + or raise an ArgErr in case of failure. + """ + argnames, varargname, kwargname = signature + scopelen = len(argnames) + has_vararg = varargname is not None + has_kwarg = kwargname is not None + if has_vararg: + scopelen += 1 + if has_kwarg: + scopelen += 1 + scope_w = [None] * scopelen + self._match_signature(scope_w, argnames, has_vararg, has_kwarg, defaults_w, 0, []) + return scope_w class ArgumentsPrepended(AbstractArguments): def __init__(self, args, w_firstarg): @@ -74,28 +107,13 @@ def _rawshape(self, nextra=0): return self.args._rawshape(nextra + 1) - def match_signature(self, signature, defaults_w=[]): - """Parse args and kwargs according to the signature of a code object, - or raise an ArgErr in case of failure. - """ - argnames, varargname, kwargname = signature - try: - scope_w = self.args.match_signature((argnames[1:], varargname, kwargname), defaults_w) - except ArgErrCount, e: - e.num_args += 1 # Add our first arg - raise - - if len(argnames) == 0: - if varargname is None: - raise ArgErrCount(self, signature, defaults_w, 0) - space = self.args.space - if kwargname is not None: - scope_w[-2] = space.newtuple([self.w_firstarg] + space.unpackiterable(scope_w[-2])) - else: - scope_w[-1] = space.newtuple([self.w_firstarg] + space.unpackiterable(scope_w[-1])) + def _match_signature(self, scope_w, argnames, has_vararg=False, has_kwarg=False, defaults_w=[], blindargs=0, extravarargs=[]): + if blindargs < len(argnames): + scope_w[blindargs] = self.w_firstarg else: - scope_w.insert(0, self.w_firstarg) - return scope_w + extravarargs.append(self.w_firstarg) + self.args._match_signature(scope_w, argnames, has_vararg, has_kwarg, + defaults_w, blindargs + 1, extravarargs) def flatten(self): (shape_cnt, shape_keys, shape_star, shape_stst), data_w = self.args.flatten() @@ -150,45 +168,46 @@ def _rawshape(self, nextra=0): return nextra + self.nargs, (), False, False - def match_signature(self, signature, defaults_w=[]): - argnames, varargname, kwargname = signature + def _match_signature(self, scope_w, argnames, has_vararg=False, has_kwarg=False, defaults_w=[], blindargs=0, extravarargs=[]): co_argcount = len(argnames) - if self.nargs + len(defaults_w) < co_argcount: - raise ArgErrCount(self, signature, defaults_w, - co_argcount - self.nargs - len(defaults_w)) - if self.nargs > co_argcount and varargname is None: - raise ArgErrCount(self, signature, defaults_w, 0) - - scopesize = co_argcount - if varargname: - scopesize += 1 - if kwargname: - scopesize += 1 - scope_w = [None] * scopesize - if self.nargs >= co_argcount: - for i in range(co_argcount): - scope_w[i] = self.valuestack.top(self.nargs - 1 - i) - if varargname is not None: - stararg_w = [None] * (self.nargs - co_argcount) - for i in range(co_argcount, self.nargs): - stararg_w[i - co_argcount] = self.valuestack.top(self.nargs - 1 - i) + if blindargs + self.nargs + len(defaults_w) < co_argcount: + raise ArgErrCount(blindargs + self.nargs , 0, + (co_argcount, has_vararg, has_kwarg), + defaults_w, co_argcount - blindargs - + self.nargs - len(defaults_w)) + if self.nargs > co_argcount and has_vararg: + raise ArgErrCount(blindargs + self.nargs, 0, + (co_argcount, has_vararg, has_kwarg), + defaults_w, 0) + + if blindargs + self.nargs >= co_argcount: + for i in range(co_argcount - blindargs): + scope_w[i + blindargs] = self.valuestack.top(self.nargs - 1 - i) + if has_vararg: + if blindargs > co_argcount: + startarg_w = extravararg + for i in range(self.nargs): + startarg_w.append(self.valuestack.top(self.nargs - 1 - i)) + else: + stararg_w = [None] * (self.nargs + blindargs - co_argcount) + for i in range(co_argcount - blindargs, self.nargs): + stararg_w[i - co_argcount + blindargs] = self.valuestack.top(self.nargs - 1 - i) scope_w[co_argcount] = self.space.newtuple(stararg_w) co_argcount += 1 else: for i in range(self.nargs): - scope_w[i] = self.valuestack.top(self.nargs - 1 - i) + scope_w[i + blindargs] = self.valuestack.top(self.nargs - 1 - i) ndefaults = len(defaults_w) - missing = co_argcount - self.nargs + missing = co_argcount - self.nargs - blindargs first_default = ndefaults - missing for i in range(missing): - scope_w[self.nargs + i] = defaults_w[first_default + i] - if varargname is not None: + scope_w[self.nargs + blindargs + i] = defaults_w[first_default + i] + if has_vararg: scope_w[co_argcount] = self.space.newtuple([]) co_argcount += 1 - if kwargname is not None: + if has_kwarg: scope_w[co_argcount] = self.space.newdict([]) - return scope_w def flatten(self): data_w = [None] * self.nargs @@ -322,11 +341,13 @@ ### Parsing for function calls ### - def match_signature(self, signature, defaults_w=[]): + def _match_signature(self, scope_w, argnames, has_vararg=False, + has_kwarg=False, defaults_w=[], blindargs=0, + extravarargs=[]): """Parse args and kwargs according to the signature of a code object, or raise an ArgErr in case of failure. """ - argnames, varargname, kwargname = signature + # # args_w = list of the normal actual parameters, wrapped # kwds_w = real dictionary {'keyword': wrapped parameter} @@ -337,8 +358,8 @@ if self.w_stararg is not None: # There is a case where we don't have to unpack() a w_stararg: # if it matches exactly a *arg in the signature. - if (len(self.arguments_w) == co_argcount and - varargname is not None and + if (len(self.arguments_w) + blindargs == co_argcount and + has_vararg and self.space.is_w(self.space.type(self.w_stararg), self.space.w_tuple)): pass @@ -350,10 +371,22 @@ args_w = self.arguments_w kwds_w = self.kwds_w - + num_kwds = 0 + if kwds_w is not None: + num_kwds = len(kwds_w) + # put as many positional input arguments into place as available - scope_w = args_w[:co_argcount] - input_argcount = len(scope_w) + if blindargs >= co_argcount: + input_argcount = co_argcount + elif len(args_w) + blindargs > co_argcount: + for i in range(co_argcount - blindargs): + scope_w[i + blindargs] = args_w[i] + input_argcount = co_argcount + next_arg = co_argcount - blindargs + else: + for i in range(len(args_w)): + scope_w[i + blindargs] = args_w[i] + input_argcount = len(args_w) + blindargs # check that no keyword argument conflicts with these # note that for this purpose we ignore the first blind_arguments, @@ -376,42 +409,52 @@ for i in range(input_argcount, co_argcount): name = argnames[i] if name in remainingkwds_w: - scope_w.append(remainingkwds_w[name]) + scope_w[i] = remainingkwds_w[name] del remainingkwds_w[name] elif i >= def_first: - scope_w.append(defaults_w[i-def_first]) + scope_w[i] = defaults_w[i-def_first] else: # error: not enough arguments. Don't signal it immediately # because it might be related to a problem with */** or # keyword arguments, which will be checked for below. missing += 1 + # collect extra positional arguments into the *vararg - if varargname is not None: + if has_vararg: if self.w_stararg is None: # common case - if len(args_w) > co_argcount: # check required by rpython - starargs_w = args_w[co_argcount:] + args_left = co_argcount - blindargs + if args_left < 0: # check required by rpython + starargs_w = extravarargs + if len(args_w): + starargs_w.extend(args_w) + elif len(args_w) > args_left: + starargs_w = args_w[args_left:] else: starargs_w = [] - scope_w.append(self.space.newtuple(starargs_w)) + scope_w[co_argcount] = self.space.newtuple(starargs_w) else: # shortcut for the non-unpack() case above - scope_w.append(self.w_stararg) - elif len(args_w) > co_argcount: - raise ArgErrCount(self, signature, defaults_w, 0) + scope_w[co_argcount] = self.w_stararg + elif len(args_w) + blindargs > co_argcount: + raise ArgErrCount(len(args_w) + blindargs, num_kwds, + (co_argcount, has_vararg, has_kwarg), + defaults_w, 0) # collect extra keyword arguments into the **kwarg - if kwargname is not None: + if has_kwarg: w_kwds = self.space.newdict([]) if remainingkwds_w: for key, w_value in remainingkwds_w.items(): self.space.setitem(w_kwds, self.space.wrap(key), w_value) - scope_w.append(w_kwds) + scope_w[co_argcount + has_vararg] = w_kwds elif remainingkwds_w: raise ArgErrUnknownKwds(remainingkwds_w) if missing: - raise ArgErrCount(self, signature, defaults_w, missing) - return scope_w + raise ArgErrCount(len(args_w) + blindargs, num_kwds, + (co_argcount, has_vararg, has_kwarg), + defaults_w, missing) + ### Argument <-> list of w_objects together with "shape" information @@ -451,36 +494,36 @@ class ArgErrCount(ArgErr): - def __init__(self, args, signature, defaults_w, missing_args): + def __init__(self, nargs, nkwds, signature, defaults_w, missing_args): self.signature = signature - self.defaults_w = defaults_w + self.num_defaults = len(defaults_w) self.missing_args = missing_args - self.num_args = args.num_args() - self.num_kwds = args.num_kwds() + self.num_args = nargs + self.num_kwds = nkwds def getmsg(self, fnname): args = None - argnames, varargname, kwargname = self.signature + num_args, has_vararg, has_kwarg = self.signature #args_w, kwds_w = args.unpack() - if kwargname is not None or (self.num_kwds and self.defaults_w): + if has_kwarg or (self.num_kwds and self.num_defaults): msg2 = "non-keyword " if self.missing_args: - required_args = len(argnames) - len(self.defaults_w) + required_args = num_args - self.num_defaults nargs = required_args - self.missing_args else: nargs = self.num_args else: msg2 = "" nargs = self.num_args + self.num_kwds - n = len(argnames) + n = num_args if n == 0: msg = "%s() takes no %sargument (%d given)" % ( fnname, msg2, nargs) else: - defcount = len(self.defaults_w) - if defcount == 0 and varargname is None: + defcount = self.num_defaults + if defcount == 0 and has_vararg: msg1 = "exactly" elif not self.missing_args: msg1 = "at most" Modified: pypy/branch/arre-experiments/pypy/interpreter/function.py ============================================================================== --- pypy/branch/arre-experiments/pypy/interpreter/function.py (original) +++ pypy/branch/arre-experiments/pypy/interpreter/function.py Wed Jan 18 00:54:39 2006 @@ -9,6 +9,7 @@ from pypy.interpreter.error import OperationError from pypy.interpreter.baseobjspace import Wrappable from pypy.interpreter.eval import Code +from pypy.interpreter.pyframe import PyFrame class Function(Wrappable): """A function is a code object captured with some environment: @@ -34,7 +35,7 @@ def call_args(self, args): scope_w = args.parse(self.name, self.code.signature(), self.defs_w) frame = self.code.create_frame(self.space, self.w_func_globals, - self.closure) + self.closure) frame.setfastscope(scope_w) return frame.run() Modified: pypy/branch/arre-experiments/pypy/interpreter/gateway.py ============================================================================== --- pypy/branch/arre-experiments/pypy/interpreter/gateway.py (original) +++ pypy/branch/arre-experiments/pypy/interpreter/gateway.py Wed Jan 18 00:54:39 2006 @@ -312,9 +312,14 @@ del d['setfastscope_UWS_%s' % label] self.miniglobals['OperationError'] = OperationError + self.miniglobals['os'] = os source = """if 1: def _run_UWS_%s(self): - return self.behavior(%s) + try: + return self.behavior(%s) + except MemoryError: + os.write(2, 'Fail in _run() of ' + self.b_name + '\\n') + raise \n""" % (label, ','.join(self.run_args)) exec compile2(source) in self.miniglobals, d d['_run'] = d['_run_UWS_%s' % label] @@ -327,6 +332,7 @@ def create(self, space, code, w_globals): newframe = frame_cls(space, code, w_globals) newframe.behavior = self.behavior + newframe.b_name = self.b_name return newframe MyBuiltinFrameFactory.__name__ = 'BuiltinFrameFactory_UwS_%s' % label @@ -339,6 +345,7 @@ factory = frame_uw_factory_cls() factory.behavior = func + factory.b_name = func.__name__ return factory Modified: pypy/branch/arre-experiments/pypy/interpreter/pyopcode.py ============================================================================== --- pypy/branch/arre-experiments/pypy/interpreter/pyopcode.py (original) +++ pypy/branch/arre-experiments/pypy/interpreter/pyopcode.py Wed Jan 18 00:54:39 2006 @@ -678,7 +678,7 @@ w_function = f.valuestack.pop() w_result = f.space.call_function(w_function, w_arg1, w_arg2, w_arg3) f.valuestack.push(w_result) - elif (oparg >> 8) & 0xff == 0: + elif False and (oparg >> 8) & 0xff == 0: # Only positional arguments nargs = oparg & 0xff args = ArgumentsFromValuestack(f.space, f.valuestack, nargs) @@ -686,6 +686,7 @@ try: w_result = f.space.call_args(w_function, args) finally: + args.valuestack = None f.valuestack.drop(nargs + 1) f.valuestack.push(w_result) # XXX end of hack for performance From ericvrp at codespeak.net Wed Jan 18 11:19:14 2006 From: ericvrp at codespeak.net (ericvrp at codespeak.net) Date: Wed, 18 Jan 2006 11:19:14 +0100 (CET) Subject: [pypy-svn] r22270 - pypy/dist/pypy/translator/llvm/test Message-ID: <20060118101914.C3A4A27B54@code1.codespeak.net> Author: ericvrp Date: Wed Jan 18 11:19:10 2006 New Revision: 22270 Modified: pypy/dist/pypy/translator/llvm/test/test_exc_operation.py pypy/dist/pypy/translator/llvm/test/test_extfunc.py Log: Disable some tests on 64 bit machines that are work in progress. Part of the externalfunctions tests actualy start to work. Modified: pypy/dist/pypy/translator/llvm/test/test_exc_operation.py ============================================================================== --- pypy/dist/pypy/translator/llvm/test/test_exc_operation.py (original) +++ pypy/dist/pypy/translator/llvm/test/test_exc_operation.py Wed Jan 18 11:19:10 2006 @@ -4,6 +4,9 @@ from pypy.rpython.rarithmetic import r_uint, ovfcheck, ovfcheck_lshift from pypy.translator.test import snippet +if sys.maxint != 2**31-1: + py.test.skip("WIP") + def test_zerodiv_int(): def zerodiv_int(n): try: Modified: pypy/dist/pypy/translator/llvm/test/test_extfunc.py ============================================================================== --- pypy/dist/pypy/translator/llvm/test/test_extfunc.py (original) +++ pypy/dist/pypy/translator/llvm/test/test_extfunc.py Wed Jan 18 11:19:10 2006 @@ -9,6 +9,9 @@ from pypy.rpython.rarithmetic import r_uint from pypy.rpython import ros +if sys.maxint != 2**31-1: + py.test.skip("WIP") + def test_external_function_ll_os_dup(): def fn(): return os.dup(0) From ericvrp at codespeak.net Wed Jan 18 11:19:43 2006 From: ericvrp at codespeak.net (ericvrp at codespeak.net) Date: Wed, 18 Jan 2006 11:19:43 +0100 (CET) Subject: [pypy-svn] r22271 - in pypy/dist/pypy/translator/llvm: . module Message-ID: <20060118101943.48D7027B5B@code1.codespeak.net> Author: ericvrp Date: Wed Jan 18 11:19:39 2006 New Revision: 22271 Modified: pypy/dist/pypy/translator/llvm/extfuncnode.py pypy/dist/pypy/translator/llvm/module/support.py pypy/dist/pypy/translator/llvm/opwriter.py Log: Fixes for 64 bit machines Modified: pypy/dist/pypy/translator/llvm/extfuncnode.py ============================================================================== --- pypy/dist/pypy/translator/llvm/extfuncnode.py (original) +++ pypy/dist/pypy/translator/llvm/extfuncnode.py Wed Jan 18 11:19:39 2006 @@ -5,9 +5,9 @@ log = log.extfuncnode class ExtFuncSig(object): - def __init__(self, retval, args): + def __init__(self, rettype, args): + self.rettype = rettype self.args = args - self.rettype = retval # signature of external functions differ from C's implementation ext_func_sigs = { @@ -15,8 +15,23 @@ "%LL_stack_too_big" : ExtFuncSig("int", None), "%LL_os_lseek" : ExtFuncSig("int", None), "%LL_thread_acquirelock" : ExtFuncSig("int", [None, "int"]), - "%LL_thread_start" : ExtFuncSig(None, ["sbyte*", "sbyte*"])} - + "%LL_thread_start" : ExtFuncSig(None, ["sbyte*", "sbyte*"]), + } + +#on 64 bit systems we need a wrapper for everything the takes or returns an int +from sys import maxint +if maxint != 2**31-1: + extra_ext_func_sigs = { + "%LL_os_dup" : ExtFuncSig("int", ["int"]), + "%LL_os_open" : ExtFuncSig("int", [None, "int", "int"]), + "%LL_os_write" : ExtFuncSig(None, ["int", None]), + "%LL_read_into" : ExtFuncSig(None, ["int", None]), + "%LL_os_close" : ExtFuncSig(None, ["int"]), + "%pypy_ll_stat_result__Signed_Signed_Signed_Signed_Signed_Signed_Signed_Signed_Signed_Signed" : ExtFuncSig(None, ["int"]*10) + } + ext_func_sigs.update(extra_ext_func_sigs) + + class ExternalFuncNode(ConstantLLVMNode): def __init__(self, db, value): @@ -28,7 +43,11 @@ mapped_name = EXTERNALS[value._callable] self.ref = self.make_ref("%", mapped_name) self.wrapper = ext_func_sigs.get(self.ref, None) - + if self.wrapper is None and maxint != 2**31-1: + #XXX if returntype or param in ('int', 'uint'): + # add wrapper to return from long->int + pass + def __str__(self): return "" % self.ref Modified: pypy/dist/pypy/translator/llvm/module/support.py ============================================================================== --- pypy/dist/pypy/translator/llvm/module/support.py (original) +++ pypy/dist/pypy/translator/llvm/module/support.py Wed Jan 18 11:19:39 2006 @@ -79,6 +79,17 @@ """ +from sys import maxint +if maxint != 2**31-1: + extfunctions = """ +internal fastcc void %pypy_ll_raise_OSError__Signed(int %errno_0) { + %tmp = cast int %errno_0 to long + call void %pypy_ll_raise_OSError__Signed(long %tmp) + ret void +} + +""" + def write_raise_exc(c_name, exc_repr, codewriter): l = """ Modified: pypy/dist/pypy/translator/llvm/opwriter.py ============================================================================== --- pypy/dist/pypy/translator/llvm/opwriter.py (original) +++ pypy/dist/pypy/translator/llvm/opwriter.py Wed Jan 18 11:19:39 2006 @@ -81,6 +81,8 @@ def __init__(self, db, codewriter): self.db = db self.codewriter = codewriter + self.word = db.get_machine_word() + self.uword = db.get_machine_uword() def _tmp(self, count=1): if count == 1: @@ -89,6 +91,8 @@ return [self.db.repr_tmpvar() for ii in range(count)] def write_operation(self, op): + #log(op) + if op.opname in ("direct_call", 'indirect_call'): opr = OpReprCall(op, self.db) else: @@ -373,20 +377,17 @@ opr.argrefs[0], [("uint", 0)]) self.codewriter.load(opr.retref, opr.rettype, tmpvar) - - # XXX use machine word size - def adr_delta(self, opr): addr1, addr2 = self._tmp(2) - self.codewriter.cast(addr1, opr.argtypes[0], opr.argrefs[0], "int") - self.codewriter.cast(addr2, opr.argtypes[1], opr.argrefs[1], "int") + self.codewriter.cast(addr1, opr.argtypes[0], opr.argrefs[0], self.word) + self.codewriter.cast(addr2, opr.argtypes[1], opr.argrefs[1], self.word) self.codewriter.binaryop("sub", opr.retref, opr.rettype, addr1, addr2) def _op_adr_generic(self, opr, llvm_op): addr, res = self._tmp(2) - self.codewriter.cast(addr, opr.argtypes[0], opr.argrefs[0], "int") - self.codewriter.binaryop(llvm_op, res, "int", addr, opr.argrefs[1]) - self.codewriter.cast(opr.retref, "int", res, opr.rettype) + self.codewriter.cast(addr, opr.argtypes[0], opr.argrefs[0], self.word) + self.codewriter.binaryop(llvm_op, res, self.word, addr, opr.argrefs[1]) + self.codewriter.cast(opr.retref, self.word, res, opr.rettype) def adr_add(self, opr): self._op_adr_generic(opr, "add") @@ -396,10 +397,10 @@ def _op_adr_cmp(self, opr, llvm_op): addr1, addr2 = self._tmp(2) - self.codewriter.cast(addr1, opr.argtypes[0], opr.argrefs[0], "int") - self.codewriter.cast(addr2, opr.argtypes[1], opr.argrefs[1], "int") + self.codewriter.cast(addr1, opr.argtypes[0], opr.argrefs[0], self.word) + self.codewriter.cast(addr2, opr.argtypes[1], opr.argrefs[1], self.word) assert opr.rettype == "bool" - self.codewriter.binaryop(llvm_op, opr.retref, "int", addr1, addr2) + self.codewriter.binaryop(llvm_op, opr.retref, self.word, addr1, addr2) def adr_eq(self, opr): self._op_adr_cmp(opr, "seteq") From adim at codespeak.net Wed Jan 18 16:36:28 2006 From: adim at codespeak.net (adim at codespeak.net) Date: Wed, 18 Jan 2006 16:36:28 +0100 (CET) Subject: [pypy-svn] r22393 - pypy/branch/ast-experiments Message-ID: <20060118153628.E60E227B60@code1.codespeak.net> Author: adim Date: Wed Jan 18 16:36:14 2006 New Revision: 22393 Added: pypy/branch/ast-experiments/ - copied from r22392, pypy/dist/ Log: create a branch to explore ast manipulation without having to make sure PyPy compiles before each checkin From adim at codespeak.net Wed Jan 18 17:06:44 2006 From: adim at codespeak.net (adim at codespeak.net) Date: Wed, 18 Jan 2006 17:06:44 +0100 (CET) Subject: [pypy-svn] r22395 - pypy/dist/pypy/interpreter Message-ID: <20060118160644.DD86C27B70@code1.codespeak.net> Author: adim Date: Wed Jan 18 17:06:34 2006 New Revision: 22395 Modified: pypy/dist/pypy/interpreter/pycompiler.py Log: another space/tabs problem (hopefully the last onesvn diff) Modified: pypy/dist/pypy/interpreter/pycompiler.py ============================================================================== --- pypy/dist/pypy/interpreter/pycompiler.py (original) +++ pypy/dist/pypy/interpreter/pycompiler.py Wed Jan 18 17:06:34 2006 @@ -13,7 +13,7 @@ def __init__(self, space): self.space = space - self.w_compile_hook = space.w_None + self.w_compile_hook = space.w_None def compile(self, source, filename, mode, flags): """Compile and return an pypy.interpreter.eval.Code instance.""" @@ -220,8 +220,8 @@ raise OperationError(space.w_SyntaxError, e.wrap_info(space, filename)) - if not space.is_w(self.w_compile_hook, space.w_None): - w_ast_tree = space.call_function(self.w_compile_hook, + if not space.is_w(self.w_compile_hook, space.w_None): + w_ast_tree = space.call_function(self.w_compile_hook, space.wrap(ast_tree), space.wrap(encoding)) ast_tree = space.interp_w(Node, w_ast_tree) From ac at codespeak.net Wed Jan 18 17:24:29 2006 From: ac at codespeak.net (ac at codespeak.net) Date: Wed, 18 Jan 2006 17:24:29 +0100 (CET) Subject: [pypy-svn] r22398 - pypy/branch/arre-experiments/pypy/interpreter Message-ID: <20060118162429.E248927B82@code1.codespeak.net> Author: ac Date: Wed Jan 18 17:24:29 2006 New Revision: 22398 Modified: pypy/branch/arre-experiments/pypy/interpreter/argument.py pypy/branch/arre-experiments/pypy/interpreter/function.py pypy/branch/arre-experiments/pypy/interpreter/nestedscope.py pypy/branch/arre-experiments/pypy/interpreter/pyframe.py pypy/branch/arre-experiments/pypy/interpreter/pyopcode.py Log: Some more tweeking. Possibly a regression performance-wise. Modified: pypy/branch/arre-experiments/pypy/interpreter/argument.py ============================================================================== --- pypy/branch/arre-experiments/pypy/interpreter/argument.py (original) +++ pypy/branch/arre-experiments/pypy/interpreter/argument.py Wed Jan 18 17:24:29 2006 @@ -27,8 +27,8 @@ has_vararg = varargname is not None has_kwarg = kwargname is not None try: - self._match_signature(scope_w, argnames, has_vararg, has_kwarg, - defaults_w, 0, []) + return self._match_signature(scope_w, argnames, has_vararg, + has_kwarg, defaults_w, 0, None) except ArgErr, e: raise OperationError(self.space.w_TypeError, self.space.wrap(e.getmsg(fnname))) @@ -76,7 +76,7 @@ if has_kwarg: scopelen += 1 scope_w = [None] * scopelen - self._match_signature(scope_w, argnames, has_vararg, has_kwarg, defaults_w, 0, []) + self._match_signature(scope_w, argnames, has_vararg, has_kwarg, defaults_w, 0, None) return scope_w class ArgumentsPrepended(AbstractArguments): @@ -107,13 +107,21 @@ def _rawshape(self, nextra=0): return self.args._rawshape(nextra + 1) - def _match_signature(self, scope_w, argnames, has_vararg=False, has_kwarg=False, defaults_w=[], blindargs=0, extravarargs=[]): + def _match_signature(self, scope_w, argnames, has_vararg=False, has_kwarg=False, defaults_w=[], blindargs=0, extravarargs=None): + """Parse args and kwargs according to the signature of a code object, + or raise an ArgErr in case of failure. + Return the number of arguments filled in. + """ if blindargs < len(argnames): scope_w[blindargs] = self.w_firstarg else: - extravarargs.append(self.w_firstarg) - self.args._match_signature(scope_w, argnames, has_vararg, has_kwarg, - defaults_w, blindargs + 1, extravarargs) + if extravarargs is None: + extravarargs = [ self.w_firstarg ] + else: + extravarargs.append(self.w_firstarg) + return self.args._match_signature(scope_w, argnames, has_vararg, + has_kwarg, defaults_w, + blindargs + 1, extravarargs) def flatten(self): (shape_cnt, shape_keys, shape_star, shape_stst), data_w = self.args.flatten() @@ -168,7 +176,11 @@ def _rawshape(self, nextra=0): return nextra + self.nargs, (), False, False - def _match_signature(self, scope_w, argnames, has_vararg=False, has_kwarg=False, defaults_w=[], blindargs=0, extravarargs=[]): + def _match_signature(self, scope_w, argnames, has_vararg=False, has_kwarg=False, defaults_w=[], blindargs=0, extravarargs=None): + """Parse args and kwargs according to the signature of a code object, + or raise an ArgErr in case of failure. + Return the number of arguments filled in. + """ co_argcount = len(argnames) if blindargs + self.nargs + len(defaults_w) < co_argcount: raise ArgErrCount(blindargs + self.nargs , 0, @@ -185,7 +197,7 @@ scope_w[i + blindargs] = self.valuestack.top(self.nargs - 1 - i) if has_vararg: if blindargs > co_argcount: - startarg_w = extravararg + startarg_w = extravarargs for i in range(self.nargs): startarg_w.append(self.valuestack.top(self.nargs - 1 - i)) else: @@ -208,7 +220,9 @@ if has_kwarg: scope_w[co_argcount] = self.space.newdict([]) - + co_argcount += 1 + return co_argcount + def flatten(self): data_w = [None] * self.nargs for i in range(self.nargs): @@ -343,11 +357,11 @@ def _match_signature(self, scope_w, argnames, has_vararg=False, has_kwarg=False, defaults_w=[], blindargs=0, - extravarargs=[]): + extravarargs=None): """Parse args and kwargs according to the signature of a code object, or raise an ArgErr in case of failure. + Return the number of arguments filled in. """ - # # args_w = list of the normal actual parameters, wrapped # kwds_w = real dictionary {'keyword': wrapped parameter} @@ -455,7 +469,8 @@ (co_argcount, has_vararg, has_kwarg), defaults_w, missing) - + return co_argcount + has_vararg + has_kwarg + ### Argument <-> list of w_objects together with "shape" information def _rawshape(self, nextra=0): Modified: pypy/branch/arre-experiments/pypy/interpreter/function.py ============================================================================== --- pypy/branch/arre-experiments/pypy/interpreter/function.py (original) +++ pypy/branch/arre-experiments/pypy/interpreter/function.py Wed Jan 18 17:24:29 2006 @@ -33,10 +33,18 @@ return "" % self.name def call_args(self, args): - scope_w = args.parse(self.name, self.code.signature(), self.defs_w) frame = self.code.create_frame(self.space, self.w_func_globals, self.closure) - frame.setfastscope(scope_w) + sig = self.code.signature() + if (isinstance(frame, PyFrame) and + frame.setfastscope is PyFrame.setfastscope): + # XXX: Performance hack! + args_matched = args.parse_into_scope(frame.fastlocals_w, self.name, + sig, self.defs_w) + frame.init_cells(args_matched) + else: + scope_w = args.parse(self.name, sig, self.defs_w) + frame.setfastscope(scope_w) return frame.run() def getdict(self): Modified: pypy/branch/arre-experiments/pypy/interpreter/nestedscope.py ============================================================================== --- pypy/branch/arre-experiments/pypy/interpreter/nestedscope.py (original) +++ pypy/branch/arre-experiments/pypy/interpreter/nestedscope.py Wed Jan 18 17:24:29 2006 @@ -99,8 +99,7 @@ else: cell.set(w_value) - def setfastscope(self, scope_w): - PyInterpFrame.setfastscope(self, scope_w) + def init_cells(self, num_vars): if self.pycode.co_cellvars: # the first few cell vars could shadow already-set arguments, # in the same order as they appear in co_varnames @@ -109,10 +108,10 @@ cellvars = code.co_cellvars next = 0 nextname = cellvars[0] - for i in range(len(scope_w)): + for i in range(num_vars): if argvars[i] == nextname: # argument i has the same name as the next cell var - w_value = scope_w[i] + w_value = self.fastlocals_w[i] self.cells[next] = Cell(w_value) next += 1 try: Modified: pypy/branch/arre-experiments/pypy/interpreter/pyframe.py ============================================================================== --- pypy/branch/arre-experiments/pypy/interpreter/pyframe.py (original) +++ pypy/branch/arre-experiments/pypy/interpreter/pyframe.py Wed Jan 18 17:24:29 2006 @@ -80,7 +80,13 @@ if len(scope_w) > len(self.fastlocals_w): raise ValueError, "new fastscope is longer than the allocated area" self.fastlocals_w[:len(scope_w)] = scope_w - + self.init_cells(len(scope_w)) + + def init_cells(self, numvars): + """Initialize cellvars from self.fastlocals_w + This is overridden in PyNestedScopeFrame""" + pass + def getclosure(self): return None Modified: pypy/branch/arre-experiments/pypy/interpreter/pyopcode.py ============================================================================== --- pypy/branch/arre-experiments/pypy/interpreter/pyopcode.py (original) +++ pypy/branch/arre-experiments/pypy/interpreter/pyopcode.py Wed Jan 18 17:24:29 2006 @@ -647,13 +647,14 @@ w_key = f.valuestack.pop() key = f.space.str_w(w_key) keywords[key] = w_value - arguments = [f.valuestack.pop() for i in range(n_arguments)] - arguments.reverse() + arguments = [None] * n_arguments + for i in range(n_arguments - 1, -1, -1): + arguments[i] = f.valuestack.pop() args = Arguments(f.space, arguments, keywords, w_star, w_starstar) w_function = f.valuestack.pop() w_result = f.space.call_args(w_function, args) f.valuestack.push(w_result) - + def CALL_FUNCTION(f, oparg): # XXX start of hack for performance if oparg == 0: # 0 arg, 0 keyword arg From adim at codespeak.net Wed Jan 18 17:27:39 2006 From: adim at codespeak.net (adim at codespeak.net) Date: Wed, 18 Jan 2006 17:27:39 +0100 (CET) Subject: [pypy-svn] r22399 - in pypy/branch/ast-experiments/pypy: interpreter interpreter/astcompiler module/recparser/hooksamples Message-ID: <20060118162739.3764E27B64@code1.codespeak.net> Author: adim Date: Wed Jan 18 17:27:36 2006 New Revision: 22399 Added: pypy/branch/ast-experiments/pypy/module/recparser/hooksamples/ pypy/branch/ast-experiments/pypy/module/recparser/hooksamples/constchanger.py pypy/branch/ast-experiments/pypy/module/recparser/hooksamples/tracer.py Modified: pypy/branch/ast-experiments/pypy/interpreter/astcompiler/ast.py pypy/branch/ast-experiments/pypy/interpreter/astcompiler/astgen.py pypy/branch/ast-experiments/pypy/interpreter/pycompiler.py Log: - added a parent attribute to AST nodes - added a few hook examples (they will probably be used as a basis for improvements) Modified: pypy/branch/ast-experiments/pypy/interpreter/astcompiler/ast.py ============================================================================== --- pypy/branch/ast-experiments/pypy/interpreter/astcompiler/ast.py (original) +++ pypy/branch/ast-experiments/pypy/interpreter/astcompiler/ast.py Wed Jan 18 17:27:36 2006 @@ -31,6 +31,7 @@ def __init__(self, lineno = -1): self.lineno = lineno self.filename = "" + self.parent = None #self.scope = None def getChildren(self): @@ -60,6 +61,12 @@ def descr_repr( self, space ): return space.wrap( self.__repr__() ) + def fget_parent(space, self): + return space.wrap(self.parent) + + def fset_parent(space, self, w_parent): + self.parent = space.interp_w(Node, w_parent, can_be_None=False) + def descr_getChildNodes( self, space ): lst = self.getChildNodes() return space.newlist( [ space.wrap( it ) for it in lst ] ) @@ -81,6 +88,7 @@ accept = interp2app(descr_node_accept, unwrap_spec=[ ObjSpace, W_Root, W_Root ] ), lineno = interp_attrproperty('lineno', cls=Node), filename = interp_attrproperty('filename', cls=Node), + parent=GetSetProperty(Node.fget_parent, Node.fset_parent), ) Modified: pypy/branch/ast-experiments/pypy/interpreter/astcompiler/astgen.py ============================================================================== --- pypy/branch/ast-experiments/pypy/interpreter/astcompiler/astgen.py (original) +++ pypy/branch/ast-experiments/pypy/interpreter/astcompiler/astgen.py Wed Jan 18 17:27:36 2006 @@ -573,6 +573,7 @@ def __init__(self, lineno = -1): self.lineno = lineno self.filename = "" + self.parent = None #self.scope = None def getChildren(self): @@ -602,6 +603,12 @@ def descr_repr( self, space ): return space.wrap( self.__repr__() ) + def fget_parent(space, self): + return space.wrap(self.parent) + + def fset_parent(space, self, w_parent): + self.parent = space.interp_w(Node, w_parent, can_be_None=False) + def descr_getChildNodes( self, space ): lst = self.getChildNodes() return space.newlist( [ space.wrap( it ) for it in lst ] ) @@ -623,6 +630,7 @@ accept = interp2app(descr_node_accept, unwrap_spec=[ ObjSpace, W_Root, W_Root ] ), lineno = interp_attrproperty('lineno', cls=Node), filename = interp_attrproperty('filename', cls=Node), + parent=GetSetProperty(Node.fget_parent, Node.fset_parent), ) Modified: pypy/branch/ast-experiments/pypy/interpreter/pycompiler.py ============================================================================== --- pypy/branch/ast-experiments/pypy/interpreter/pycompiler.py (original) +++ pypy/branch/ast-experiments/pypy/interpreter/pycompiler.py Wed Jan 18 17:27:36 2006 @@ -13,7 +13,7 @@ def __init__(self, space): self.space = space - self.w_compile_hook = space.w_None + self.w_compile_hook = space.w_None def compile(self, source, filename, mode, flags): """Compile and return an pypy.interpreter.eval.Code instance.""" @@ -220,8 +220,8 @@ raise OperationError(space.w_SyntaxError, e.wrap_info(space, filename)) - if not space.is_w(self.w_compile_hook, space.w_None): - w_ast_tree = space.call_function(self.w_compile_hook, + if not space.is_w(self.w_compile_hook, space.w_None): + w_ast_tree = space.call_function(self.w_compile_hook, space.wrap(ast_tree), space.wrap(encoding)) ast_tree = space.interp_w(Node, w_ast_tree) Added: pypy/branch/ast-experiments/pypy/module/recparser/hooksamples/constchanger.py ============================================================================== --- (empty file) +++ pypy/branch/ast-experiments/pypy/module/recparser/hooksamples/constchanger.py Wed Jan 18 17:27:36 2006 @@ -0,0 +1,21 @@ +class ChangeConstVisitor: + def visitConst(self, node): + if node.value == 3: + node.value = 2 + + def defaultvisit(self, node): + for child in node.getChildNodes(): + child.accept(self) + + def __getattr__(self, attrname): + if attrname.startswith('visit'): + return self.defaultvisit + raise AttributeError(attrname) + +def threebecomestwo(ast, enc): + ast.accept(ChangeConstVisitor()) + return ast + +# install the hook +import parser +parser.install_compiler_hook(threebecomestwo) Added: pypy/branch/ast-experiments/pypy/module/recparser/hooksamples/tracer.py ============================================================================== --- (empty file) +++ pypy/branch/ast-experiments/pypy/module/recparser/hooksamples/tracer.py Wed Jan 18 17:27:36 2006 @@ -0,0 +1,42 @@ +"""this one logs simple assignments and somewhat clearly shows +that we need a nice API to define "joinpoints". Maybe a SAX-like +(i.e. event-based) API ? + +XXX: crashes on everything else than simple assignment (AssAttr, etc.) +""" + +from parser import ASTPrintnl, ASTConst, ASTName +from parser import install_compiler_hook + +class Tracer: + def visitModule(self, module): + module.node = module.node.accept(self) + return module + + def default(self, node): + for child in node.getChildNodes(): + # let's cheat a bit + child.parent = node + child.accept(self) + return node + + def visitAssign(self, assign): + stmt = assign.parent + varname = assign.nodes[0].name + lognode = ASTPrintnl([ASTConst('%s <--' % varname), ASTName(varname)], None) + index = stmt.nodes.index(assign) + newstmts = stmt.nodes + newstmts.insert(index + 1, lognode) + stmt.nodes = newstmts + return assign + + def __getattr__(self, attrname): + if attrname.startswith('visit'): + return self.default + raise AttributeError('No such attribute: %s' % attrname) + + +def _trace(ast, enc): + return ast.accept(Tracer()) + +install_compiler_hook(_trace) From rxe at codespeak.net Wed Jan 18 22:20:35 2006 From: rxe at codespeak.net (rxe at codespeak.net) Date: Wed, 18 Jan 2006 22:20:35 +0100 (CET) Subject: [pypy-svn] r22400 - pypy/dist/pypy/translator/llvm Message-ID: <20060118212035.4D20A27DB8@code1.codespeak.net> Author: rxe Date: Wed Jan 18 22:20:33 2006 New Revision: 22400 Modified: pypy/dist/pypy/translator/llvm/arraynode.py pypy/dist/pypy/translator/llvm/gc.py pypy/dist/pypy/translator/llvm/opwriter.py pypy/dist/pypy/translator/llvm/structnode.py Log: ooops forgot to complete refactoring the codewriter interface in gc.py - done now. Added an experimental variable size malloc to gc policy - heading towards using a policy object for flavoured mallocs (dont even know if this is a good idea, or even an idea at all) Might actually be (very) marginally faster now. Modified: pypy/dist/pypy/translator/llvm/arraynode.py ============================================================================== --- pypy/dist/pypy/translator/llvm/arraynode.py (original) +++ pypy/dist/pypy/translator/llvm/arraynode.py Wed Jan 18 22:20:33 2006 @@ -47,15 +47,22 @@ self.db.repr_type(self.arraytype)) def writedecl(self, codewriter): - # declaration for constructor + return #XXX tmp? + codewriter.declare(self.constructor_decl) def writeimpl(self, codewriter): + return #XXX tmp? + + current, indices_to_array = self.var_malloc_info() log.writeimpl(self.ref) gp = self.db.gcpolicy gp.write_constructor(codewriter, self.ref, self.constructor_decl, - self.array, atomic=self.array._is_atomic()) + current, atomic=self.array._is_atomic()) + def var_malloc_info(self): + return self.array, () + class VoidArrayTypeNode(LLVMNode): __slots__ = "db array ref".split() Modified: pypy/dist/pypy/translator/llvm/gc.py ============================================================================== --- pypy/dist/pypy/translator/llvm/gc.py (original) +++ pypy/dist/pypy/translator/llvm/gc.py Wed Jan 18 22:20:33 2006 @@ -1,3 +1,4 @@ +import sys from pypy.rpython.rstr import STR from pypy.translator.llvm.log import log @@ -6,149 +7,184 @@ class GcPolicy: def __init__(self, db): raise Exception, 'GcPolicy should not be used directly' - + def genextern_code(self): return "" def gc_libraries(self): return [] - - # malloc is not an codewriter specific thing - def malloc(self, codewriter, targetvar, type_, size=1, atomic=False): - # XXX _indent & test - codewriter._indent('%(targetvar)s = malloc %(type_)s, uint %(s)s' % locals()) - def write_constructor(self, codewriter, ref, constructor_decl, ARRAY, - indices_to_array=(), atomic=False, is_str=False): - - - #varsized arrays and structs look like this: - #Array: {int length , elemtype*} - #Struct: {...., Array} - - # the following indices access the last element in the array - elemtype = self.db.repr_type(ARRAY.OF) - word = lentype = self.db.get_machine_word() - uword = self.db.get_machine_uword() - - codewriter.openfunc(constructor_decl) - - # Need room for NUL terminator - if ARRAY is STR.chars: - codewriter.binaryop("add", "%actuallen", lentype, "%len", 1) - else: - codewriter.cast("%actuallen", lentype, "%len", lentype) - - elemindices = list(indices_to_array) - elemindices += [("uint", 1), (lentype, "%actuallen")] - codewriter.getelementptr("%size", ref + "*", "null", elemindices) - codewriter.cast("%usize", elemtype + "*", "%size", uword) - self.malloc(codewriter, "%ptr", "sbyte*", "%usize", atomic=atomic) - codewriter.cast("%result", "sbyte*", "%ptr", ref + "*") - - indices_to_arraylength = tuple(indices_to_array) + (("uint", 0),) - - # the following accesses the length field of the array - codewriter.getelementptr("%arraylength", ref + "*", - "%result", - indices_to_arraylength) - codewriter.store(lentype, "%len", "%arraylength") - - #if is_str: - # indices_to_hash = (("uint", 0),) - # codewriter.getelementptr("%ptrhash", ref + "*", - # "%result", - # indices_to_hash) - # codewriter.store("int", "0", "%ptrhash") - - - #if ARRAY is STR.chars: - # codewriter.getelementptr("%ptrendofchar", ref + "*", - # "%result", - # elemindices) - # codewriter.store(elemtype, "0", "%ptrendofchar") - - codewriter.ret(ref + "*", "%result") - codewriter.closefunc() - def pyrex_code(self): return '' + + #XXX tmp? +## def write_constructor(self, codewriter, ref, constructor_decl, ARRAY, +## indices_to_array=(), atomic=False): + +## # the following indices access the last element in the array +## elemtype = self.db.repr_type(ARRAY.OF) +## word = lentype = self.db.get_machine_word() +## uword = self.db.get_machine_uword() + +## codewriter.openfunc(constructor_decl) + +## # Need room for NUL terminator +## if ARRAY is STR.chars: +## codewriter.binaryop("add", "%actuallen", lentype, "%len", 1) +## else: +## codewriter.cast("%actuallen", lentype, "%len", lentype) + +## elemindices = list(indices_to_array) +## elemindices += [("uint", 1), (lentype, "%actuallen")] +## codewriter.getelementptr("%size", ref + "*", "null", elemindices) +## codewriter.cast("%usize", elemtype + "*", "%size", uword) + +## self._malloc(codewriter, "%ptr", "%usize", atomic=atomic) +## codewriter.cast("%result", "sbyte*", "%ptr", ref + "*") + +## indices_to_arraylength = tuple(indices_to_array) + (("uint", 0),) + +## # the following accesses the length field of the array +## codewriter.getelementptr("%arraylength", ref + "*", +## "%result", +## indices_to_arraylength) +## codewriter.store(lentype, "%len", "%arraylength") + +## codewriter.ret(ref + "*", "%result") +## codewriter.closefunc() - def new(db, gcpolicy=None): #factory - gcpolicy = gcpolicy or 'boehm' + def malloc(self, codewriter, targetvar, type_, size=1, atomic=False): + raise NotImplementedError, 'GcPolicy should not be used directly' + def new(db, gcpolicy=None): + """ factory """ + gcpolicy = gcpolicy or 'boehm' + + # XXX would be nice to localise this sort of thing? import distutils.sysconfig from os.path import exists libdir = distutils.sysconfig.EXEC_PREFIX + "/lib" boehm_on_path = exists(libdir + '/libgc.so') or exists(libdir + '/libgc.a') if gcpolicy == 'boehm' and not boehm_on_path: log.gc.WARNING('warning: Boehm GC libary not found in /usr/lib, falling back on no gc') - gcpolicy = 'none' + gcpolicy = 'raw' if gcpolicy == 'boehm': gcpolicy = BoehmGcPolicy(db) elif gcpolicy == 'ref': gcpolicy = RefcountingGcPolicy(db) - elif gcpolicy == 'none': - gcpolicy = NoneGcPolicy(db) + elif gcpolicy == 'raw': + gcpolicy = RawGcPolicy(db) else: raise Exception, 'unknown gcpolicy: ' + str(gcpolicy) return gcpolicy new = staticmethod(new) - -class NoneGcPolicy(GcPolicy): +class RawGcPolicy(GcPolicy): def __init__(self, db): self.db = db - + def malloc(self, codewriter, targetvar, type_, size=1, atomic=False): + codewriter.malloc(targetvar, type_, size) + #XXX memset + class BoehmGcPolicy(GcPolicy): def __init__(self, db): self.db = db self.n_malloced = 0 - + def genextern_code(self): return '#include "boehm.h"' - + def gc_libraries(self): return ['gc', 'pthread'] # XXX on windows? - def malloc(self, codewriter, targetvar, type_, size=1, atomic=False): - import sys + def pyrex_code(self): + return ''' +cdef extern int GC_get_heap_size() + +def GC_get_heap_size_wrapper(): + return GC_get_heap_size() +''' + + def _malloc(self, codewriter, targetvar, size=1, atomic=False): + """ assumes malloc of word size """ + # XXX Boehm aligns on 8 byte boundaries if sys.platform == 'linux2' and sys.maxint == 2**63-1: boundary_size = 8 else: - boundary_size = 0 + boundary_size = 0 - is_atomic = atomic uword = self.db.get_machine_uword() - s = str(size) + fnname = '%pypy_malloc' + (atomic and '_atomic' or '') + codewriter.call(targetvar, 'sbyte*', fnname, [uword], [size]) + + if atomic: + codewriter.call(None, 'void', '%llvm.memset', + ['sbyte*', 'ubyte', uword, uword], + [targetvar, 0, size, boundary_size], + cconv='ccc') + + def malloc(self, codewriter, targetvar, type_, size=1, atomic=False): + uword = self.db.get_machine_uword() self.n_malloced += 1 - cnt = '.%d' % self.n_malloced - atomic = is_atomic and '_atomic' or '' - t = ''' -%%malloc_Size%(cnt)s = getelementptr %(type_)s null, %(uword)s %(s)s -%%malloc_SizeU%(cnt)s = cast %(type_)s %%malloc_Size%(cnt)s to %(uword)s -%%malloc_Ptr%(cnt)s = call fastcc sbyte* %%pypy_malloc%(atomic)s(%(uword)s %%malloc_SizeU%(cnt)s) -%(targetvar)s = cast sbyte* %%malloc_Ptr%(cnt)s to %(type_)s -''' % locals() - - if is_atomic: #Boehm aligns on 8 byte boundaries - t += ''' - call ccc void %%llvm.memset(sbyte* %%malloc_Ptr%(cnt)s, ubyte 0, %(uword)s %%malloc_SizeU%(cnt)s, %(uword)s %(boundary_size)s) - ''' % locals() - codewriter.write_lines(t) + cnt = '_%d' % self.n_malloced + malloc_ptr = '%malloc_ptr' + cnt + malloc_size = '%malloc_size' + cnt + malloc_sizeu = '%malloc_sizeu' + cnt + + codewriter.getelementptr(malloc_size, type_, 'null', + [(uword, size)], getptr=False) + codewriter.cast(malloc_sizeu, type_, malloc_size, uword) + self._malloc(codewriter, malloc_ptr, malloc_sizeu, atomic) + codewriter.cast(targetvar, 'sbyte*', malloc_ptr, type_) + def var_malloc(self, codewriter, targetvar, + type_, node, len, atomic=False): - def pyrex_code(self): - return ''' -cdef extern int GC_get_heap_size() + word = lentype = self.db.get_machine_word() + uword = self.db.get_machine_uword() + self.n_malloced += 1 + cnt = '_%d' % self.n_malloced + malloc_ptr = '%malloc_ptr' + cnt + malloc_size = '%malloc_size' + cnt + malloc_sizeu = '%malloc_sizeu' + cnt + actuallen = '%actuallen' + cnt + arraylength = '%arraylength' + cnt + + ARRAY, indices_to_array = node.var_malloc_info() + + #varsized arrays and structs look like this: + #Array: {int length , elemtype*} + #Struct: {...., Array} + + # the following indices access the last element in the array + elemtype = self.db.repr_type(ARRAY.OF) + word = lentype = self.db.get_machine_word() + uword = self.db.get_machine_uword() + + # need room for NUL terminator + if ARRAY is STR.chars: + codewriter.binaryop('add', actuallen, lentype, len, 1) + else: + codewriter.cast(actuallen, lentype, len, lentype) + + elemindices = list(indices_to_array) + elemindices += [('uint', 1), (lentype, actuallen)] + codewriter.getelementptr(malloc_size, type_, 'null', elemindices) + codewriter.cast(malloc_sizeu, elemtype + '*', malloc_size, uword) + + self._malloc(codewriter, malloc_ptr, malloc_sizeu, atomic=atomic) -def GC_get_heap_size_wrapper(): - return GC_get_heap_size() -''' + indices_to_arraylength = tuple(indices_to_array) + (('uint', 0),) + + codewriter.cast(targetvar, 'sbyte*', malloc_ptr, type_) + # the following accesses the length field of the array + codewriter.getelementptr(arraylength, type_, + targetvar, indices_to_arraylength) + codewriter.store(lentype, len, arraylength) class RefcountingGcPolicy(GcPolicy): - def __init__(self): + def __init__(self, db): raise NotImplementedError, 'RefcountingGcPolicy' Modified: pypy/dist/pypy/translator/llvm/opwriter.py ============================================================================== --- pypy/dist/pypy/translator/llvm/opwriter.py (original) +++ pypy/dist/pypy/translator/llvm/opwriter.py Wed Jan 18 22:20:33 2006 @@ -269,7 +269,7 @@ def malloc_varsize(self, opr): - # XXX transformation + # XXX transformation perhaps? arg_type = opr.op.args[0].value if isinstance(arg_type, lltype.Array) and arg_type.OF is lltype.Void: # This is a backend decision to NOT represent a void array with @@ -277,9 +277,14 @@ self.malloc(opr) return - self.codewriter.call(opr.retref, opr.rettype, - self.db.repr_constructor(arg_type), - opr.argtypes[1:], opr.argrefs[1:]) + node = self.db.obj2node[arg_type] + self.db.gcpolicy.var_malloc(self.codewriter, opr.retref, opr.rettype, + node, opr.argrefs[1], + atomic=arg_type._is_atomic()) + #XXX tmp? + #self.codewriter.call(opr.retref, opr.rettype, + # self.db.repr_constructor(arg_type), + # opr.argtypes[1:], opr.argrefs[1:]) def flavored_malloc(self, opr): flavor = opr.op.args[0].value Modified: pypy/dist/pypy/translator/llvm/structnode.py ============================================================================== --- pypy/dist/pypy/translator/llvm/structnode.py (original) +++ pypy/dist/pypy/translator/llvm/structnode.py Wed Jan 18 22:20:33 2006 @@ -55,29 +55,37 @@ def writedecl(self, codewriter): # declaration for constructor + return #XXX + codewriter.declare(self.constructor_decl) def writeimpl(self, codewriter): + return #XXX + log.writeimpl(self.ref) + current, indices_to_array = self.var_malloc_info() + gp = self.db.gcpolicy + gp.write_constructor(codewriter, + self.ref, + self.constructor_decl, + current, + indices_to_array, + self.struct._is_atomic()) + def var_malloc_info(self): # build up a list of indices to get to the last # var-sized struct (or rather the according array) indices_to_array = [] current = self.struct while isinstance(current, lltype.Struct): last_pos = len(current._names_without_voids()) - 1 - indices_to_array.append(("uint", last_pos)) #struct requires uint consts + # struct requires uint consts + indices_to_array.append(("uint", last_pos)) name = current._names_without_voids()[-1] current = current._flds[name] assert isinstance(current, lltype.Array) - gp = self.db.gcpolicy - gp.write_constructor(codewriter, - self.ref, - self.constructor_decl, - current, - indices_to_array, - self.struct._is_atomic(), - is_str=self.struct._name == "rpy_string") + + return current, indices_to_array class StructNode(ConstantLLVMNode): """ A struct constant. Can simply contain From ac at codespeak.net Wed Jan 18 23:02:59 2006 From: ac at codespeak.net (ac at codespeak.net) Date: Wed, 18 Jan 2006 23:02:59 +0100 (CET) Subject: [pypy-svn] r22401 - pypy/branch/arre-experiments/pypy/interpreter Message-ID: <20060118220259.915F927B70@code1.codespeak.net> Author: ac Date: Wed Jan 18 23:02:58 2006 New Revision: 22401 Modified: pypy/branch/arre-experiments/pypy/interpreter/argument.py pypy/branch/arre-experiments/pypy/interpreter/function.py pypy/branch/arre-experiments/pypy/interpreter/pyopcode.py Log: Bugfixes. Modified: pypy/branch/arre-experiments/pypy/interpreter/argument.py ============================================================================== --- pypy/branch/arre-experiments/pypy/interpreter/argument.py (original) +++ pypy/branch/arre-experiments/pypy/interpreter/argument.py Wed Jan 18 23:02:58 2006 @@ -187,7 +187,7 @@ (co_argcount, has_vararg, has_kwarg), defaults_w, co_argcount - blindargs - self.nargs - len(defaults_w)) - if self.nargs > co_argcount and has_vararg: + if blindargs + self.nargs > co_argcount and not has_vararg: raise ArgErrCount(blindargs + self.nargs, 0, (co_argcount, has_vararg, has_kwarg), defaults_w, 0) Modified: pypy/branch/arre-experiments/pypy/interpreter/function.py ============================================================================== --- pypy/branch/arre-experiments/pypy/interpreter/function.py (original) +++ pypy/branch/arre-experiments/pypy/interpreter/function.py Wed Jan 18 23:02:58 2006 @@ -36,12 +36,13 @@ frame = self.code.create_frame(self.space, self.w_func_globals, self.closure) sig = self.code.signature() + # XXX start of hack for performance if (isinstance(frame, PyFrame) and frame.setfastscope is PyFrame.setfastscope): - # XXX: Performance hack! args_matched = args.parse_into_scope(frame.fastlocals_w, self.name, sig, self.defs_w) frame.init_cells(args_matched) + # XXX end of hack for performance else: scope_w = args.parse(self.name, sig, self.defs_w) frame.setfastscope(scope_w) Modified: pypy/branch/arre-experiments/pypy/interpreter/pyopcode.py ============================================================================== --- pypy/branch/arre-experiments/pypy/interpreter/pyopcode.py (original) +++ pypy/branch/arre-experiments/pypy/interpreter/pyopcode.py Wed Jan 18 23:02:58 2006 @@ -679,7 +679,7 @@ w_function = f.valuestack.pop() w_result = f.space.call_function(w_function, w_arg1, w_arg2, w_arg3) f.valuestack.push(w_result) - elif False and (oparg >> 8) & 0xff == 0: + elif (oparg >> 8) & 0xff == 0: # Only positional arguments nargs = oparg & 0xff args = ArgumentsFromValuestack(f.space, f.valuestack, nargs) From pedronis at codespeak.net Thu Jan 19 11:10:21 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Thu, 19 Jan 2006 11:10:21 +0100 (CET) Subject: [pypy-svn] r22417 - in pypy/dist/pypy/translator: . test Message-ID: <20060119101021.9A01B27DC2@code1.codespeak.net> Author: pedronis Date: Thu Jan 19 11:10:16 2006 New Revision: 22417 Modified: pypy/dist/pypy/translator/interactive.py pypy/dist/pypy/translator/test/test_interactive.py Log: Translation and driver must share translator context Modified: pypy/dist/pypy/translator/interactive.py ============================================================================== --- pypy/dist/pypy/translator/interactive.py (original) +++ pypy/dist/pypy/translator/interactive.py Thu Jan 19 11:10:16 2006 @@ -74,7 +74,7 @@ if not self.driver_setup: if argtypes is None: argtypes = [] - self.driver.setup(self.entry_point, argtypes, policy) + self.driver.setup(self.entry_point, argtypes, policy, empty_translator=self.context) self.ann_argtypes = argtypes self.ann_policy = policy self.driver_setup = True Modified: pypy/dist/pypy/translator/test/test_interactive.py ============================================================================== --- pypy/dist/pypy/translator/test/test_interactive.py (original) +++ pypy/dist/pypy/translator/test/test_interactive.py Thu Jan 19 11:10:16 2006 @@ -7,6 +7,8 @@ return x+y t = Translation(f, [int, int]) + assert t.context is t.driver.translator + s = t.annotate([int, int]) assert s.knowntype == int From ericvrp at codespeak.net Thu Jan 19 11:26:01 2006 From: ericvrp at codespeak.net (ericvrp at codespeak.net) Date: Thu, 19 Jan 2006 11:26:01 +0100 (CET) Subject: [pypy-svn] r22418 - in pypy/dist/pypy/translator/llvm: . module test Message-ID: <20060119102601.76A6227DC2@code1.codespeak.net> Author: ericvrp Date: Thu Jan 19 11:25:53 2006 New Revision: 22418 Modified: pypy/dist/pypy/translator/llvm/extfuncnode.py pypy/dist/pypy/translator/llvm/module/support.py pypy/dist/pypy/translator/llvm/test/test_extfunc.py Log: Create wrapper functions (on 64 bit architectures) where a int is expected but a long gives. The remaining problem for external functions has to do with data structures used by the ext.funcs. Modified: pypy/dist/pypy/translator/llvm/extfuncnode.py ============================================================================== --- pypy/dist/pypy/translator/llvm/extfuncnode.py (original) +++ pypy/dist/pypy/translator/llvm/extfuncnode.py Thu Jan 19 11:25:53 2006 @@ -18,20 +18,6 @@ "%LL_thread_start" : ExtFuncSig(None, ["sbyte*", "sbyte*"]), } -#on 64 bit systems we need a wrapper for everything the takes or returns an int -from sys import maxint -if maxint != 2**31-1: - extra_ext_func_sigs = { - "%LL_os_dup" : ExtFuncSig("int", ["int"]), - "%LL_os_open" : ExtFuncSig("int", [None, "int", "int"]), - "%LL_os_write" : ExtFuncSig(None, ["int", None]), - "%LL_read_into" : ExtFuncSig(None, ["int", None]), - "%LL_os_close" : ExtFuncSig(None, ["int"]), - "%pypy_ll_stat_result__Signed_Signed_Signed_Signed_Signed_Signed_Signed_Signed_Signed_Signed" : ExtFuncSig(None, ["int"]*10) - } - ext_func_sigs.update(extra_ext_func_sigs) - - class ExternalFuncNode(ConstantLLVMNode): def __init__(self, db, value): @@ -42,15 +28,45 @@ mapped_name = EXTERNALS[value._callable] self.ref = self.make_ref("%", mapped_name) - self.wrapper = ext_func_sigs.get(self.ref, None) - if self.wrapper is None and maxint != 2**31-1: - #XXX if returntype or param in ('int', 'uint'): - # add wrapper to return from long->int - pass + + def setup(self): + self.db.prepare_type(self.value._TYPE.RESULT) + self.db.prepare_type_multi(self.value._TYPE._trueargs()) def __str__(self): return "" % self.ref + def _get_wrapper(self): + wrapper = ext_func_sigs.get(self.ref, None) + + from sys import maxint + if wrapper is None and maxint != 2**31-1: + #log("ref=%s" % self.ref) + rettype, args = self.getdecl_parts() + conversions = False + if rettype == "long": + rettype = "int" + conversions = True + elif rettype == "ulong": + rettype = "uint" + conversions = True + else: + rettype = None + for i, a in enumerate(args): + if a == "long": + args[i] = "int" + conversions = True + elif a == "ulong": + args[i] = "uint" + conversions = True + else: + args[i] = None + if conversions: + wrapper = ExtFuncSig(rettype, args) + #log(" rettype=%s" % str(rettype)) + #log(" args =%s" % str(args)) + return wrapper + def getdecl_parts(self): T = self.value._TYPE rettype = self.db.repr_type(T.RESULT) @@ -65,8 +81,8 @@ codewriter.declare(self.getdecl()) def writeimpl(self, codewriter): - - if self.wrapper is None: + wrapper = self._get_wrapper() + if wrapper is None: return rettype, argtypes = self.getdecl_parts() @@ -85,10 +101,10 @@ expected_argtypes = [] # find out what the args/types should be - if self.wrapper.args is not None: - assert len(self.wrapper.args) == len(argtypes) + if wrapper.args is not None: + assert len(wrapper.args) == len(argtypes) - for expected_typ, typ, ref in zip(self.wrapper.args, + for expected_typ, typ, ref in zip(wrapper.args, argtypes, argrefs): if expected_typ is not None: @@ -108,17 +124,17 @@ expected_argtypes = argtypes # find out what the return type should be - expected_rettype = self.wrapper.rettype or rettype + expected_rettype = wrapper.rettype or rettype # call codewriter.call(returnval, expected_rettype, self.ref, expected_argtypes, expected_argrefs) - if self.wrapper.rettype: + if wrapper.rettype: # cast to desired return type tmpval = returnval returnval = self.db.repr_tmpvar() - codewriter.cast(returnval, self.wrapper.rettype, + codewriter.cast(returnval, wrapper.rettype, tmpval, rettype) codewriter.ret(rettype, returnval) Modified: pypy/dist/pypy/translator/llvm/module/support.py ============================================================================== --- pypy/dist/pypy/translator/llvm/module/support.py (original) +++ pypy/dist/pypy/translator/llvm/module/support.py Thu Jan 19 11:25:53 2006 @@ -81,7 +81,7 @@ from sys import maxint if maxint != 2**31-1: - extfunctions = """ + extfunctions += """ internal fastcc void %pypy_ll_raise_OSError__Signed(int %errno_0) { %tmp = cast int %errno_0 to long call void %pypy_ll_raise_OSError__Signed(long %tmp) Modified: pypy/dist/pypy/translator/llvm/test/test_extfunc.py ============================================================================== --- pypy/dist/pypy/translator/llvm/test/test_extfunc.py (original) +++ pypy/dist/pypy/translator/llvm/test/test_extfunc.py Thu Jan 19 11:25:53 2006 @@ -9,9 +9,6 @@ from pypy.rpython.rarithmetic import r_uint from pypy.rpython import ros -if sys.maxint != 2**31-1: - py.test.skip("WIP") - def test_external_function_ll_os_dup(): def fn(): return os.dup(0) @@ -60,6 +57,8 @@ assert os.path.exists(path) def test_os_file_ops_open_write_close(): + if sys.maxint != 2**31-1: + py.test.skip("WIP on 64 bit architectures") def openwriteclose(): fd = os.open(path, os.O_CREAT|os.O_RDWR, 0777) byteswritten = os.write(fd, path) @@ -74,6 +73,8 @@ assert open(path).read() == path def test_os_file_ops_open_write_read_close(): + if sys.maxint != 2**31-1: + py.test.skip("WIP on 64 bit architectures") def openwriteclose_openreadclose(): fd = os.open(path, os.O_CREAT|os.O_RDWR, 0777) byteswritten = os.write(fd, path+path+path) @@ -97,6 +98,8 @@ # following from translator/c/test/test_extfunc.py Revision: 15320 (jul 29th 2005) def test_os_stat(): + if sys.maxint != 2**31-1: + py.test.skip("WIP on 64 bit architectures") filename = str(py.magic.autopath()) def call_stat0(): st = os.stat(filename) @@ -116,6 +119,8 @@ assert f2() == st[2] def test_os_fstat(): + if sys.maxint != 2**31-1: + py.test.skip("WIP on 64 bit architectures") filename = str(py.magic.autopath()) def call_fstat0(): fd = os.open(filename, os.O_RDONLY, 0777) @@ -148,6 +153,8 @@ assert f1() def test_math_frexp(): + if sys.maxint != 2**31-1: + py.test.skip("WIP on 64 bit architectures") from math import frexp def fn(x): res = frexp(x) @@ -186,6 +193,8 @@ yield math_function_test, funcname def test_os_path_exists(): + if sys.maxint != 2**31-1: + py.test.skip("WIP on 64 bit architectures") tmpfile = str(udir.join('test_os_path_exists.TMP')) def fn(): return os.path.exists(tmpfile) @@ -196,6 +205,8 @@ assert f() == False def test_dynamic_string_null_termination(): + if sys.maxint != 2**31-1: + py.test.skip("WIP on 64 bit architectures") # forces malloc / versus pbc for NUL testing of C string tmpfile = str(udir.join('test_os_path_exists.TMP')) def fn(l): @@ -208,6 +219,8 @@ assert f(lfile-2) == False def test_os_path_isdir(): + if sys.maxint != 2**31-1: + py.test.skip("WIP on 64 bit architectures") directory = "./." def fn(): return os.path.isdir(directory) @@ -228,6 +241,8 @@ assert f(2) == os.isatty(2) def test_rarith_parts_to_float(): + if sys.maxint != 2**31-1: + py.test.skip("WIP on 64 bit architectures") from pypy.rpython.rarithmetic import parts_to_float parts = [ ["" ,"1","" ,""], @@ -246,6 +261,8 @@ assert f(i) == v def test_rarith_formatd(): + if sys.maxint != 2**31-1: + py.test.skip("WIP on 64 bit architectures") from pypy.rpython.rarithmetic import formatd as_float = [ 0.0 , 1.5 , 2.0 ] as_string = ["0.00", "1.50", "2.00"] @@ -421,6 +438,8 @@ assert res == 42 def test_start_new_thread(): + if sys.maxint != 2**31-1: + py.test.skip("WIP on 64 bit architectures") import thread import pypy.module.thread.rpython.exttable # for declare()/declaretype() class Arg: From nico at codespeak.net Thu Jan 19 11:30:42 2006 From: nico at codespeak.net (nico at codespeak.net) Date: Thu, 19 Jan 2006 11:30:42 +0100 (CET) Subject: [pypy-svn] r22419 - pypy/dist/pypy/lib/logic Message-ID: <20060119103042.04D0227DC3@code1.codespeak.net> Author: nico Date: Thu Jan 19 11:30:40 2006 New Revision: 22419 Added: pypy/dist/pypy/lib/logic/ pypy/dist/pypy/lib/logic/test_unification.py pypy/dist/pypy/lib/logic/unification.py Log: preliminary work on implementing a oz-like variable store Added: pypy/dist/pypy/lib/logic/test_unification.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/lib/logic/test_unification.py Thu Jan 19 11:30:40 2006 @@ -0,0 +1,85 @@ +import unification as u +from py.test import raises, skip + +class TestUnification: + + def setup_method(self, meth): + u._store = u.Store() + + def test_bind_var_var(self): + x = u.var('x') + y = u.var('y') + z = u.var('z') + u.bind(x, z) + assert u._store.unbound == set([u.EqSet([y]), + u.EqSet([x,z])]) + assert u.bound() == [] + + def test_bind_var_val(self): + y = u.var('y') + u.bind(y, 42) + assert u.unbound() == [] + assert u._store.bound == {y:42} + x = u.var('x') + z = u.var('z') + u.bind(x, z) + u.bind(z, 3.14) + assert u.unbound() == [] + assert u._store.bound == {x:3.14, y:42, z:3.14} + + def test_already_in_store(self): + x = u.var('x') + raises(u.AlreadyInStore, u.var, 'x') + + def test_already_bound(self): + x = u.var('x') + u.bind(x, 42) + raises(u.AlreadyBound, u.bind, x, 42) + + def test_unify_lists_success(self): + x,y,z,w = (u.var('x'), u.var('y'), + u.var('z'), u.var('w')) + u.bind(x, [42, z]) + u.bind(y, [w, 44]) + u.unify(x, y) + assert u._store.bound == {y: [w, 44], x: [42, z], + z: 44, w: 42} + + def test_unify_dicts_success(self): + x,y,z,w = (u.var('x'), u.var('y'), + u.var('z'), u.var('w')) + u.bind(x, {1:42, 2:z}) + u.bind(y, {1:w, 2:44}) + u.unify(x, y) + assert u._store.bound == {y: {1:w, 2:44}, + x: {1:42, 2:z}, + z: 44, w: 42} + + def test_unify_failure(self): + x,y,z = u.var('x'), u.var('y'), u.var('z') + u.bind(x, [42, z]) + u.bind(y, [z, 44]) + raises(u.UnificationFailure, u.unify, x, y) + + def test_unify_failure2(self): + x,y,z,w = (u.var('x'), u.var('y'), + u.var('z'), u.var('w')) + u.bind(x, [42, z]) + u.bind(y, [w, 44]) + u.bind(z, w) + raises(u.UnificationFailure, u.unify, x, y) + + def test_unify_circular(self): + x, y, z, w, a, b = (u.var('x'), u.var('y'), + u.var('z'), u.var('w'), + u.var('a'), u.var('b')) + u.bind(x, [y]) + u.bind(y, [x]) + raises(u.Circularity, u.unify, x, y) + u.bind(z, [1, w]) + u.bind(w, [z, 2]) + raises(u.UnificationFailure, u.unify, z, w) + u.bind(a, {1:42, 2:b}) + u.bind(b, {1:a, 2:42}) + raises(u.UnificationFailure, u.unify, a, b) + Added: pypy/dist/pypy/lib/logic/unification.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/lib/logic/unification.py Thu Jan 19 11:30:40 2006 @@ -0,0 +1,285 @@ +# Oz unification in Python 2.4 +# within a single assignment store +# crude and buggy ... + +#TODO : +# * ensure that the store is intact after a failure +# (maybe with some anti-bind ops) +# * provide a way to copy the store to a fresh one +# (clone operator) +# After reading more of the "book", I see some more ops +# are needed for the store to be part of a computation +# space ... + +#----------- Variables ---------------------------------- +class EqSet(frozenset): + """An equivalence set for variables""" + pass + +class Var(object): + + def __init__(self, name, store): + self.name = name + self.store = store + store.add_unbound(self) + + def __str__(self): + if self in self.store.bound.keys(): + return "%s = %s" % (self.name, + self.store.bound[self]) + return "%s" % self.name + + def __repr__(self): + return self.__str__() + + def __eq__(self, thing): + return type(thing) == Var and self.name == thing.name + + def __hash__(self): + return self.name.__hash__() + +def var(name): + return Var(name, _store) + +#----------- Store Exceptions ---------------------------- +class VariableException(Exception): + def __init__(self, name): + self.name = name + +class UnboundVariable(VariableException): + def __str__(self): + return "%s has no value yet" % self.name + +class AlreadyBound(VariableException): + def __str__(self): + return "%s is already bound" % self.name + +class AlreadyInStore(VariableException): + def __str__(self): + return "%s already in store" % self.name + +class UnificationFailure(Exception): + def __init__(self, var1, var2): + self.var1, self.var2 = (var1, var2) + def __str__(self): + return "%s %s can't be unified" % (self.var1, + self.var2) + +class Circularity(UnificationFailure): + def __str__(self): + return "cycle : % %s not unifiable" % (self.var1, + self.var2) + +#----------- Variable Exceptions------------------------- +class NotAVariable(VariableException): + def __str__(self): + return "%s is not a variable" % self.name + +#----------- Store ------------------------------------ +class Store(object): + """The Store consists of a set of k variables + x1,...,xk that are partitioned as follows: + * set of unbound variables that are equal + (also called equivalence sets of variables). + The variables in each set are equal to each + other but not to any other variables. + * variables bound to a number, record or procedure + (also called determined variables).""" + + def __init__(self): + # set of all known vars + # var->equivalence set mapping for unbound vars + # set of equisets (clusters of unbound variables) + # var->objects bindings + self.vars = set() + self.equisets = {} + self.unbound = set() + self.bound = {} + # memoizer for unify (avoids infinite loops when + # one wants to unify vars with cycles) + self.unify_memo = set() + + def add_unbound(self, var): + # register globally + if var in self.vars: + raise AlreadyInStore(var.name) + print "adding %s to the store" % var + self.vars.add(var) + # put into new singleton equiv. set + eqset = EqSet([var]) + self.equisets[var] = eqset + self.unbound.add(eqset) + + #-- BIND ------------------------------------------- + + def bind(self, var, val): + """1. (unbound)Variable/(unbound)Variable or + 2. (unbound)Variable/(bound)Variable or + 3. (unbound)Variable/Value binding + """ + assert(isinstance(var, Var) and (var in self.vars)) + if var == val: + return + if self._both_are_vars(var, val): + if self._both_are_bound(var, val): + raise AlreadyBound(var.name) + if self.bound.has_key(var): # 2. + self.bind(val, var) + elif self.bound.has_key(val): # 2. + self._bind(self.equisets[var], + self.bound[val]) + else: # 1. + self._merge(self.equisets[var], + self.equisets[val]) + else: # 3. + if self.bound.has_key(var): + raise AlreadyBound(var.name) + self._bind(self.equisets[var], val) + + def _both_are_vars(self, v1, v2): + try: + return v1 in self.vars and v2 in self.vars + except: + return False + + def _both_are_bound(self, v1, v2): + return self.bound.has_key(v1) and \ + self.bound.has_key(v2) + + def _bind(self, eqs, val): + print "variable - value binding : %s %s" % (eqs, val) + # bind all vars in the eqset to obj + for name in eqs: + del self.equisets[name] + self.bound[name] = val + self.unbound.remove(eqs) + + def _merge(self, eqs1, eqs2): + print "unbound variables binding : %s %s" % (eqs1, eqs2) + if eqs1 == eqs2: return + # merge two equisets into one + neweqs = eqs1 | eqs2 + # let's reassign everybody to eqs1 + for name in neweqs: + self.equisets[name] = neweqs + self.unbound.remove(eqs1) + self.unbound.remove(eqs2) + self.unbound.add(neweqs) + + #-- UNIFY ------------------------------------------ + + def unify(self, x, y): + #FIXME in case of failure, the store state is not + # properly restored ... + print "unify %s with %s" % (x,y) + # do the memoization work + if (x,y) in self.unify_memo: raise Circularity(x,y) + self.unify_memo.add((x, y)) + # dispatch to the apropriate unifier + try: + if x not in self.bound and y not in self.bound: + if x != y: + if type(x) is Var: + self.bind(x,y) + else: + self.bind(y,x) + elif x in self.bound and y in self.bound: + self._unify_bound(x,y) + elif x in self.bound: + self.bind(x,y) + else: + self.bind(y,x) + except AlreadyBound: + raise UnificationFailure(x, y) + + def _unify_bound(self, x, y): + print "unify bound %s %s" % (x, y) + vx, vy = (self.bound[x], self.bound[y]) + if not _unifiable(vx, vy): raise UnificationFailure(x, y) + elif type(vx) in [list, set] and isinstance(vy, type(vx)): + self._unify_iterable(x, y) + elif type(vx) is dict and isinstance(vy, type(vx)): + self._unify_mapping(x, y) + else: + raise UnificationFailure(x, y) + + def _unify_iterable(self, x, y): + print "unify sequences %s %s" % (x, y) + vx, vy = (self.bound[x], self.bound[y]) + idx, top = (0, len(vx)) + while (idx < top): + self.unify(vx[idx], vy[idx]) + idx += 1 + + def _unify_mapping(self, x, y): + print "unify mappings %s %s" % (x, y) + vx, vy = (self.bound[x], self.bound[y]) + for xk in vx.keys(): + self.unify(vx[xk], vy[xk]) + +#-- Unifiability checks--------------------------------------- +#-- +#-- quite costly & could be merged back in unify +#-- FIXME : memoize _iterable + +def _iterable(thing): + return type(thing) in [list, set] + +def _mapping(thing): + return type(thing) is dict + +def _unifiable(term1, term2): + """Checks wether two terms can be unified""" + print "unifiable ? %s %s" % (term1, term2) + if _iterable(term1): + if _iterable(term2): + return _iter_unifiable(term1, term2) + return False + if _mapping(term1) and _mapping(term2): + return _mapping_unifiable(term1, term2) + if not(isinstance(term1, Var) or isinstance(term2, Var)): + return term1 == term2 # same 'atomic' object + return True + +def _iter_unifiable(c1, c2): + """Checks wether two iterables can be unified""" + print "unifiable sequences ? %s %s" % (c1, c2) + if len(c1) != len(c2): return False + idx, top = (0, len(c1)) + while(idx < top): + if not _unifiable(c1[idx], c2[idx]): + return False + idx += 1 + return True + +def _mapping_unifiable(m1, m2): + """Checks wether two mappings can be unified""" + print "unifiable mappings ? %s %s" % (m1, m2) + if len(m1) != len(m2): return False + if m1.keys() != m2.keys(): return False + v1, v2 = (m1.items(), m2.items()) + v1.sort() + v2.sort() + return _iter_unifiable([e[1] for e in v1], + [e[1] for e in v2]) + +#-- Some utilities ----------------------------------------------- +#-- +#-- the global store +_store = Store() + +#-- global accessor functions +def bind(var, val): + return _store.bind(var, val) + +def unify(var1, var2): + return _store.unify(var1, var2) + +def bound(): + return _store.bound.keys() + +def unbound(): + res = [] + for cluster in _store.unbound: + res.append('='.join([str(var) for var in cluster])) + return res From nico at codespeak.net Thu Jan 19 12:02:36 2006 From: nico at codespeak.net (nico at codespeak.net) Date: Thu, 19 Jan 2006 12:02:36 +0100 (CET) Subject: [pypy-svn] r22421 - pypy/dist/pypy/doc Message-ID: <20060119110236.7860D27DC2@code1.codespeak.net> Author: nico Date: Thu Jan 19 12:02:35 2006 New Revision: 22421 Modified: pypy/dist/pypy/doc/constraints-and-logic.txt Log: added list of possible testbeds Modified: pypy/dist/pypy/doc/constraints-and-logic.txt ============================================================================== --- pypy/dist/pypy/doc/constraints-and-logic.txt (original) +++ pypy/dist/pypy/doc/constraints-and-logic.txt Thu Jan 19 12:02:35 2006 @@ -327,3 +327,27 @@ * Adapt the parser/lexer to the syntactic requirements + + +Ideas of testbeds +================= + +try to use logic related features in all of these python projects to +make sure of the design and implementation of logic in pypy. + +- rdflib.sparql an implementation of W3C's SPARQL language coupled + with redfoot. + http://rdflib.net/2005/09/10/rdflib-2.2.2/html/public/rdflib.sparql-module.html + +- cwm a rule-based engine for rdf inferencing + http://www.w3.org/2000/10/swap/doc/cwm.html + +- pychinko an implementation of RETE algorithm for rule-based systems + http://www.mindswap.org/~katz/pychinko/ + http://en.wikipedia.org/wiki/Rete_algorithm + +- pylint: using constraints to generate design-related comments and + warnings is done in SOUL or doable with pyontology + http://www.logilab.org/projects/pylint + +- pyontology. DFKI's OWL manipulation tool (see svn) \ No newline at end of file From auc at codespeak.net Thu Jan 19 12:12:14 2006 From: auc at codespeak.net (auc at codespeak.net) Date: Thu, 19 Jan 2006 12:12:14 +0100 (CET) Subject: [pypy-svn] r22423 - pypy/dist/pypy/lib/logic Message-ID: <20060119111214.C7F7127DC3@code1.codespeak.net> Author: auc Date: Thu Jan 19 12:12:13 2006 New Revision: 22423 Modified: pypy/dist/pypy/lib/logic/test_unification.py pypy/dist/pypy/lib/logic/unification.py Log: displace memoization stuff small cleanup Modified: pypy/dist/pypy/lib/logic/test_unification.py ============================================================================== --- pypy/dist/pypy/lib/logic/test_unification.py (original) +++ pypy/dist/pypy/lib/logic/test_unification.py Thu Jan 19 12:12:13 2006 @@ -75,7 +75,7 @@ u.var('a'), u.var('b')) u.bind(x, [y]) u.bind(y, [x]) - raises(u.Circularity, u.unify, x, y) + raises(u.UnificationFailure, u.unify, x, y) u.bind(z, [1, w]) u.bind(w, [z, 2]) raises(u.UnificationFailure, u.unify, z, w) Modified: pypy/dist/pypy/lib/logic/unification.py ============================================================================== --- pypy/dist/pypy/lib/logic/unification.py (original) +++ pypy/dist/pypy/lib/logic/unification.py Thu Jan 19 12:12:13 2006 @@ -4,7 +4,7 @@ #TODO : # * ensure that the store is intact after a failure -# (maybe with some anti-bind ops) +# (make unify atomic) # * provide a way to copy the store to a fresh one # (clone operator) # After reading more of the "book", I see some more ops @@ -33,7 +33,8 @@ return self.__str__() def __eq__(self, thing): - return type(thing) == Var and self.name == thing.name + return isinstance(thing, Var) \ + and self.name == thing.name def __hash__(self): return self.name.__hash__() @@ -64,12 +65,7 @@ def __str__(self): return "%s %s can't be unified" % (self.var1, self.var2) - -class Circularity(UnificationFailure): - def __str__(self): - return "cycle : % %s not unifiable" % (self.var1, - self.var2) - + #----------- Variable Exceptions------------------------- class NotAVariable(VariableException): def __str__(self): @@ -95,9 +91,6 @@ self.equisets = {} self.unbound = set() self.bound = {} - # memoizer for unify (avoids infinite loops when - # one wants to unify vars with cycles) - self.unify_memo = set() def add_unbound(self, var): # register globally @@ -172,14 +165,12 @@ #FIXME in case of failure, the store state is not # properly restored ... print "unify %s with %s" % (x,y) - # do the memoization work - if (x,y) in self.unify_memo: raise Circularity(x,y) - self.unify_memo.add((x, y)) + if not _unifiable(x, y): raise UnificationFailure(x, y) # dispatch to the apropriate unifier try: if x not in self.bound and y not in self.bound: if x != y: - if type(x) is Var: + if isinstance(x, Var): self.bind(x,y) else: self.bind(y,x) @@ -195,8 +186,7 @@ def _unify_bound(self, x, y): print "unify bound %s %s" % (x, y) vx, vy = (self.bound[x], self.bound[y]) - if not _unifiable(vx, vy): raise UnificationFailure(x, y) - elif type(vx) in [list, set] and isinstance(vy, type(vx)): + if type(vx) in [list, set] and isinstance(vy, type(vx)): self._unify_iterable(x, y) elif type(vx) is dict and isinstance(vy, type(vx)): self._unify_mapping(x, y) @@ -227,13 +217,21 @@ def _mapping(thing): return type(thing) is dict - + +_unifiable_memo = set() + def _unifiable(term1, term2): + _unifiable_memo = set() + return _really_unifiable(term1, term2) + +def _really_unifiable(term1, term2): """Checks wether two terms can be unified""" + if ((id(term1), id(term2))) in _unifiable_memo: return False + _unifiable_memo.add((id(term1), id(term2))) print "unifiable ? %s %s" % (term1, term2) if _iterable(term1): if _iterable(term2): - return _iter_unifiable(term1, term2) + return _iterable_unifiable(term1, term2) return False if _mapping(term1) and _mapping(term2): return _mapping_unifiable(term1, term2) @@ -241,13 +239,13 @@ return term1 == term2 # same 'atomic' object return True -def _iter_unifiable(c1, c2): +def _iterable_unifiable(c1, c2): """Checks wether two iterables can be unified""" print "unifiable sequences ? %s %s" % (c1, c2) if len(c1) != len(c2): return False idx, top = (0, len(c1)) while(idx < top): - if not _unifiable(c1[idx], c2[idx]): + if not _really_unifiable(c1[idx], c2[idx]): return False idx += 1 return True @@ -260,8 +258,8 @@ v1, v2 = (m1.items(), m2.items()) v1.sort() v2.sort() - return _iter_unifiable([e[1] for e in v1], - [e[1] for e in v2]) + return _iterable_unifiable([e[1] for e in v1], + [e[1] for e in v2]) #-- Some utilities ----------------------------------------------- #-- From ac at codespeak.net Thu Jan 19 12:27:31 2006 From: ac at codespeak.net (ac at codespeak.net) Date: Thu, 19 Jan 2006 12:27:31 +0100 (CET) Subject: [pypy-svn] r22424 - pypy/branch/arre-experiments/pypy/translator/tool Message-ID: <20060119112731.72D1527DB6@code1.codespeak.net> Author: ac Date: Thu Jan 19 12:27:31 2006 New Revision: 22424 Modified: pypy/branch/arre-experiments/pypy/translator/tool/cbuild.py Log: Oops, do not enable debug option for everyone. Modified: pypy/branch/arre-experiments/pypy/translator/tool/cbuild.py ============================================================================== --- pypy/branch/arre-experiments/pypy/translator/tool/cbuild.py (original) +++ pypy/branch/arre-experiments/pypy/translator/tool/cbuild.py Thu Jan 19 12:27:31 2006 @@ -271,15 +271,14 @@ self.libraries.append('m') if 'pthread' not in self.libraries: self.libraries.append('pthread') - self.compile_extra += ['-g', '-O2', '-pthread'] - self.link_extra += ['-g', '-pthread'] + self.compile_extra += ['-O2', '-pthread'] + self.link_extra += ['-pthread'] if sys.platform == 'darwin': if '/sw/include' not in self.include_dirs: self.include_dirs.append('/sw/include') if '/sw/lib' not in self.library_dirs: self.library_dirs.append('/sw/lib') - self.compile_extra += ['-g', '-O2'] - self.link_extra += ['-g'] + self.compile_extra += ['-O2'] if outputfilename is None: self.outputfilename = py.path.local(cfilenames[0]).new(ext=ext) From ericvrp at codespeak.net Thu Jan 19 13:06:46 2006 From: ericvrp at codespeak.net (ericvrp at codespeak.net) Date: Thu, 19 Jan 2006 13:06:46 +0100 (CET) Subject: [pypy-svn] r22427 - in pypy/dist/pypy/translator/llvm: module test Message-ID: <20060119120646.0707727DC2@code1.codespeak.net> Author: ericvrp Date: Thu Jan 19 13:06:41 2006 New Revision: 22427 Modified: pypy/dist/pypy/translator/llvm/module/raisingop.h pypy/dist/pypy/translator/llvm/test/test_exc_operation.py Log: Made possible exception raising ops use long's instead of int's. (for 64 bit architectures) Skipping multiply with overflow test because I don't know why it fails. Modified: pypy/dist/pypy/translator/llvm/module/raisingop.h ============================================================================== --- pypy/dist/pypy/translator/llvm/module/raisingop.h (original) +++ pypy/dist/pypy/translator/llvm/module/raisingop.h Thu Jan 19 13:06:41 2006 @@ -1,6 +1,8 @@ /* expanded macros of genc's c/int.h */ +#define WORD long +#define UWORD unsigned long -int pypyop_int_neg_ovf(int x) { +WORD pypyop_int_neg_ovf(WORD x) { long r = -x; if (x >= 0 || x != -x){ return r; @@ -10,8 +12,8 @@ } } -int pypyop_int_abs_ovf(int x) { - int r = x >= 0 ? x : -x; +WORD pypyop_int_abs_ovf(WORD x) { + WORD r = x >= 0 ? x : -x; if (x >= 0 || x != -x) { return r; } else { @@ -20,8 +22,8 @@ } } -int pypyop_int_add_ovf(int x, int y) { - int r = x + y; +WORD pypyop_int_add_ovf(WORD x, WORD y) { + WORD r = x + y; if ((r^x) >= 0 || (r^y) >= 0) { return r; @@ -31,8 +33,8 @@ } } -int pypyop_int_sub_ovf(int x, int y) { - int r = x - y; +WORD pypyop_int_sub_ovf(WORD x, WORD y) { + WORD r = x - y; if ((r^x) >= 0 || (r^(~y)) >= 0) { return r; @@ -44,7 +46,7 @@ return r; } -int _pypyop_int_mul_ovf(long a, long b, long *longprod) { +WORD _pypyop_int_mul_ovf(long a, long b, long *longprod) { double doubled_longprod; /* (double)longprod */ double doubleprod; /* (double)a * (double)b */ @@ -75,11 +77,11 @@ } } -int pypyop_int_mul_ovf(int x, int y) { +WORD pypyop_int_mul_ovf(WORD x, WORD y) { long r; #ifndef HAVE_LONG_LONG - + if (_pypyop_int_mul_ovf(x, y, &r)) { return r; } else { @@ -98,8 +100,8 @@ #endif } -int pypyop_int_lshift_ovf_val(int x, int y) { - int r; +WORD pypyop_int_lshift_ovf_val(WORD x, WORD y) { + WORD r; if (y < 0) { raisePyExc_ValueError("negative shift count"); @@ -115,7 +117,7 @@ return r; } -int pypyop_int_rshift_val(int x, int y) { +WORD pypyop_int_rshift_val(WORD x, WORD y) { if (y < 0) { raisePyExc_ValueError("negative shift count"); return 0; @@ -143,7 +145,7 @@ return xdivy; } -int pypyop_int_floordiv_ovf_zer(int x, int y) { +WORD pypyop_int_floordiv_ovf_zer(WORD x, WORD y) { if (y) { if (y == -1 && x < 0 && ((unsigned long) x << 1) == 0) { raisePyExc_OverflowError("integer division"); @@ -157,7 +159,7 @@ } } -int pypyop_int_floordiv_zer(int x, int y) { +WORD pypyop_int_floordiv_zer(WORD x, WORD y) { if (y) { return _pypyop_divmod_adj(x, y, NULL); } else { @@ -166,7 +168,7 @@ } } -unsigned int pypyop_uint_floordiv_zer(unsigned int x, unsigned int y) { +UWORD pypyop_uint_floordiv_zer(UWORD x, UWORD y) { if (y) { return x / y; } else { @@ -175,7 +177,7 @@ } } -int pypyop_int_mod_ovf_zer(int x, int y) { +WORD pypyop_int_mod_ovf_zer(WORD x, WORD y) { long r; if (y) { if (y == -1 && x < 0 && ((unsigned long) x << 1) == 0) { @@ -192,7 +194,7 @@ } } -int pypyop_int_mod_zer(int x, int y) { +WORD pypyop_int_mod_zer(WORD x, WORD y) { long r; if (y) { _pypyop_divmod_adj(x, y, &r); @@ -203,7 +205,7 @@ } } -unsigned int pypyop_uint_mod_zer(unsigned int x, unsigned int y) { +UWORD pypyop_uint_mod_zer(UWORD x, UWORD y) { unsigned long r; if (y) { _pypyop_divmod_adj(x, y, &r); Modified: pypy/dist/pypy/translator/llvm/test/test_exc_operation.py ============================================================================== --- pypy/dist/pypy/translator/llvm/test/test_exc_operation.py (original) +++ pypy/dist/pypy/translator/llvm/test/test_exc_operation.py Thu Jan 19 13:06:41 2006 @@ -4,9 +4,6 @@ from pypy.rpython.rarithmetic import r_uint, ovfcheck, ovfcheck_lshift from pypy.translator.test import snippet -if sys.maxint != 2**31-1: - py.test.skip("WIP") - def test_zerodiv_int(): def zerodiv_int(n): try: @@ -172,6 +169,8 @@ assert f(-sys.maxint) == 123 def test_int_mul_ovf(): + if sys.maxint != 2**31-1: + py.test.skip("WIP on 64 bit architectures") def mul_func(i): try: return ovfcheck(i * 100) From mwh at codespeak.net Thu Jan 19 13:57:07 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Thu, 19 Jan 2006 13:57:07 +0100 (CET) Subject: [pypy-svn] r22437 - in pypy/dist/pypy: annotation rpython/memory rpython/memory/test Message-ID: <20060119125707.D2CAD27DC3@code1.codespeak.net> Author: mwh Date: Thu Jan 19 13:57:05 2006 New Revision: 22437 Modified: pypy/dist/pypy/annotation/binaryop.py pypy/dist/pypy/annotation/builtin.py pypy/dist/pypy/annotation/model.py pypy/dist/pypy/rpython/memory/lladdress.py pypy/dist/pypy/rpython/memory/test/test_address.py Log: make the new-ish offsetof be annotated as a SomeOffset() not a SomeInteger() Modified: pypy/dist/pypy/annotation/binaryop.py ============================================================================== --- pypy/dist/pypy/annotation/binaryop.py (original) +++ pypy/dist/pypy/annotation/binaryop.py Thu Jan 19 13:57:05 2006 @@ -11,7 +11,7 @@ from pypy.annotation.model import SomeTuple, SomeImpossibleValue, s_ImpossibleValue from pypy.annotation.model import SomeInstance, SomeBuiltin, SomeIterator from pypy.annotation.model import SomePBC, SomeSlice, SomeFloat, s_None -from pypy.annotation.model import SomeExternalObject +from pypy.annotation.model import SomeExternalObject, SomeOffset from pypy.annotation.model import SomeAddress, SomeTypedAddressAccess from pypy.annotation.model import unionof, UnionError, set, missing_operation, TLS from pypy.annotation.model import add_knowntypedata, merge_knowntypedata @@ -733,3 +733,6 @@ def union((s_obj, s_addr)): raise UnionError, "union of address and anything else makes no sense" +class __extend__(pairtype(SomeOffset, SomeOffset)): + def add((s_off1, s_off2)): + return SomeOffset() Modified: pypy/dist/pypy/annotation/builtin.py ============================================================================== --- pypy/dist/pypy/annotation/builtin.py (original) +++ pypy/dist/pypy/annotation/builtin.py Thu Jan 19 13:57:05 2006 @@ -8,7 +8,7 @@ from pypy.annotation.model import SomeUnicodeCodePoint, SomeAddress from pypy.annotation.model import SomeFloat, unionof from pypy.annotation.model import SomePBC, SomeInstance, SomeDict -from pypy.annotation.model import SomeExternalObject +from pypy.annotation.model import SomeExternalObject, SomeOffset from pypy.annotation.model import annotation_to_lltype, lltype_to_annotation from pypy.annotation.model import add_knowntypedata from pypy.annotation.model import s_ImpossibleValue @@ -505,7 +505,7 @@ # offsetof/sizeof def offsetof(TYPE, fldname): - return SomeInteger() + return SomeOffset() BUILTIN_ANALYZERS[lladdress.offsetof] = offsetof Modified: pypy/dist/pypy/annotation/model.py ============================================================================== --- pypy/dist/pypy/annotation/model.py (original) +++ pypy/dist/pypy/annotation/model.py Thu Jan 19 13:57:05 2006 @@ -452,6 +452,11 @@ def can_be_none(self): return False +class SomeOffset(SomeObject): + immutable = True + def can_be_none(self): + return False + # The following class is used to annotate the intermediate value that # appears in expressions of the form: Modified: pypy/dist/pypy/rpython/memory/lladdress.py ============================================================================== --- pypy/dist/pypy/rpython/memory/lladdress.py (original) +++ pypy/dist/pypy/rpython/memory/lladdress.py Thu Jan 19 13:57:05 2006 @@ -132,7 +132,7 @@ def annotation(self): from pypy.annotation import model - return model.SomeInteger() + return model.SomeOffset() def __repr__(self): return "" % (self.TYPE, self.fldname) Modified: pypy/dist/pypy/rpython/memory/test/test_address.py ============================================================================== --- pypy/dist/pypy/rpython/memory/test/test_address.py (original) +++ pypy/dist/pypy/rpython/memory/test/test_address.py Thu Jan 19 13:57:05 2006 @@ -120,13 +120,13 @@ return offsetof(S, 'x') + offsetof(S, 'y') a = RPythonAnnotator() s = a.build_types(f, []) - assert s.knowntype == int + assert isinstance(s, annmodel.SomeOffset) coff = offsetof(S, 'y') def f(): return coff a = RPythonAnnotator() s = a.build_types(f, []) - assert s.knowntype == int + assert isinstance(s, annmodel.SomeOffset) class TestAddressRTyping(object): def test_null(self): From ericvrp at codespeak.net Thu Jan 19 14:00:03 2006 From: ericvrp at codespeak.net (ericvrp at codespeak.net) Date: Thu, 19 Jan 2006 14:00:03 +0100 (CET) Subject: [pypy-svn] r22438 - in pypy/dist/pypy/translator/llvm: . module Message-ID: <20060119130003.2F61F27DC7@code1.codespeak.net> Author: ericvrp Date: Thu Jan 19 13:59:54 2006 New Revision: 22438 Modified: pypy/dist/pypy/translator/llvm/extfuncnode.py pypy/dist/pypy/translator/llvm/module/support.py pypy/dist/pypy/translator/llvm/opwriter.py Log: small fixes for ia64 Modified: pypy/dist/pypy/translator/llvm/extfuncnode.py ============================================================================== --- pypy/dist/pypy/translator/llvm/extfuncnode.py (original) +++ pypy/dist/pypy/translator/llvm/extfuncnode.py Thu Jan 19 13:59:54 2006 @@ -14,6 +14,7 @@ "%LL_os_isatty" : ExtFuncSig("int", None), "%LL_stack_too_big" : ExtFuncSig("int", None), "%LL_os_lseek" : ExtFuncSig("int", None), + "%LL_os_write" : ExtFuncSig(None, ["int", None]), "%LL_thread_acquirelock" : ExtFuncSig("int", [None, "int"]), "%LL_thread_start" : ExtFuncSig(None, ["sbyte*", "sbyte*"]), } Modified: pypy/dist/pypy/translator/llvm/module/support.py ============================================================================== --- pypy/dist/pypy/translator/llvm/module/support.py (original) +++ pypy/dist/pypy/translator/llvm/module/support.py Thu Jan 19 13:59:54 2006 @@ -41,16 +41,16 @@ ret %RPyString* %rpy } -internal fastcc int %pypyop_int_abs(int %x) { +internal fastcc WORD %pypyop_int_abs(WORD %x) { block0: - %cond1 = setge int %x, 0 + %cond1 = setge WORD %x, 0 br bool %cond1, label %return_block, label %block1 block1: - %x2 = sub int 0, %x + %x2 = sub WORD 0, %x br label %return_block return_block: - %result = phi int [%x, %block0], [%x2, %block1] - ret int %result + %result = phi WORD [%x, %block0], [%x2, %block1] + ret WORD %result } internal fastcc long %pypyop_llong_abs(long %x) { Modified: pypy/dist/pypy/translator/llvm/opwriter.py ============================================================================== --- pypy/dist/pypy/translator/llvm/opwriter.py (original) +++ pypy/dist/pypy/translator/llvm/opwriter.py Thu Jan 19 13:59:54 2006 @@ -192,8 +192,9 @@ opr.argrefs[0], -1) def uint_invert(self, opr): + from sys import maxint self.codewriter.binaryop("xor", opr.retref, opr.argtypes[0], - opr.argrefs[0], str((1L<<32) - 1)) + opr.argrefs[0], str(maxint*2+1)) def binaryop(self, opr): assert len(opr.argrefs) == 2 From ac at codespeak.net Thu Jan 19 14:28:03 2006 From: ac at codespeak.net (ac at codespeak.net) Date: Thu, 19 Jan 2006 14:28:03 +0100 (CET) Subject: [pypy-svn] r22439 - pypy/branch/arre-experiments/pypy/interpreter Message-ID: <20060119132803.E648827DC4@code1.codespeak.net> Author: ac Date: Thu Jan 19 14:28:03 2006 New Revision: 22439 Modified: pypy/branch/arre-experiments/pypy/interpreter/argument.py pypy/branch/arre-experiments/pypy/interpreter/function.py Log: A bugfix and a tweak. Modified: pypy/branch/arre-experiments/pypy/interpreter/argument.py ============================================================================== --- pypy/branch/arre-experiments/pypy/interpreter/argument.py (original) +++ pypy/branch/arre-experiments/pypy/interpreter/argument.py Thu Jan 19 14:28:03 2006 @@ -197,9 +197,9 @@ scope_w[i + blindargs] = self.valuestack.top(self.nargs - 1 - i) if has_vararg: if blindargs > co_argcount: - startarg_w = extravarargs + stararg_w = extravarargs for i in range(self.nargs): - startarg_w.append(self.valuestack.top(self.nargs - 1 - i)) + stararg_w.append(self.valuestack.top(self.nargs - 1 - i)) else: stararg_w = [None] * (self.nargs + blindargs - co_argcount) for i in range(co_argcount - blindargs, self.nargs): Modified: pypy/branch/arre-experiments/pypy/interpreter/function.py ============================================================================== --- pypy/branch/arre-experiments/pypy/interpreter/function.py (original) +++ pypy/branch/arre-experiments/pypy/interpreter/function.py Thu Jan 19 14:28:03 2006 @@ -37,8 +37,7 @@ self.closure) sig = self.code.signature() # XXX start of hack for performance - if (isinstance(frame, PyFrame) and - frame.setfastscope is PyFrame.setfastscope): + if frame.setfastscope is PyFrame.setfastscope: args_matched = args.parse_into_scope(frame.fastlocals_w, self.name, sig, self.defs_w) frame.init_cells(args_matched) From ericvrp at codespeak.net Thu Jan 19 15:25:46 2006 From: ericvrp at codespeak.net (ericvrp at codespeak.net) Date: Thu, 19 Jan 2006 15:25:46 +0100 (CET) Subject: [pypy-svn] r22441 - pypy/dist/pypy/translator/llvm/module Message-ID: <20060119142546.E04B627DC4@code1.codespeak.net> Author: ericvrp Date: Thu Jan 19 15:25:43 2006 New Revision: 22441 Modified: pypy/dist/pypy/translator/llvm/module/support.py Log: some handcoded llvm for function that require long->int wrappers, but which are not automatically generated because they are not external functions. Now standalones for demo/ricards.py and demo/bpnn.py can be created. Theywill not run though! ("Fatal error during initialization: out of memory") Modified: pypy/dist/pypy/translator/llvm/module/support.py ============================================================================== --- pypy/dist/pypy/translator/llvm/module/support.py (original) +++ pypy/dist/pypy/translator/llvm/module/support.py Thu Jan 19 15:25:43 2006 @@ -88,6 +88,18 @@ ret void } +internal fastcc int %pypy_entry_point(%RPyListOfString* %argv) { + %result = call long %pypy_entry_point(%RPyListOfString* %argv) + %tmp = cast long %result to int + ret int %tmp +} + +internal fastcc void %pypy__RPyListOfString_SetItem__listPtr_Signed_rpy_stringPtr(%RPyListOfString* %l_1, int %index_0, %RPyString* %newstring_0) { + %index_0_long = cast int %index_0 to long + call void %pypy__RPyListOfString_SetItem__listPtr_Signed_rpy_stringPtr(%RPyListOfString* %l_1, long %index_0_long, %RPyString* %newstring_0) + ret void +} + """ def write_raise_exc(c_name, exc_repr, codewriter): From auc at codespeak.net Thu Jan 19 15:46:35 2006 From: auc at codespeak.net (auc at codespeak.net) Date: Thu, 19 Jan 2006 15:46:35 +0100 (CET) Subject: [pypy-svn] r22442 - pypy/dist/pypy/lib/logic Message-ID: <20060119144635.64F5E27DC7@code1.codespeak.net> Author: auc Date: Thu Jan 19 15:46:33 2006 New Revision: 22442 Modified: pypy/dist/pypy/lib/logic/test_unification.py pypy/dist/pypy/lib/logic/unification.py Log: prepare ground for atomic unify Modified: pypy/dist/pypy/lib/logic/test_unification.py ============================================================================== --- pypy/dist/pypy/lib/logic/test_unification.py (original) +++ pypy/dist/pypy/lib/logic/test_unification.py Thu Jan 19 15:46:33 2006 @@ -6,35 +6,32 @@ def setup_method(self, meth): u._store = u.Store() + def test_already_in_store(self): + x = u.var('x') + raises(u.AlreadyInStore, u.var, 'x') + + def test_already_bound(self): + x = u.var('x') + u.bind(x, 42) + raises(u.AlreadyBound, u.bind, x, 42) + def test_bind_var_var(self): x = u.var('x') y = u.var('y') z = u.var('z') u.bind(x, z) - assert u._store.unbound == set([u.EqSet([y]), - u.EqSet([x,z])]) - assert u.bound() == [] + assert x.val == u.EqSet([x, z]) + assert y.val == u.EqSet([y]) + assert z.val == u.EqSet([x, z]) def test_bind_var_val(self): - y = u.var('y') - u.bind(y, 42) - assert u.unbound() == [] - assert u._store.bound == {y:42} - x = u.var('x') - z = u.var('z') + x, y, z = u.var('x'), u.var('y'), u.var('z') u.bind(x, z) + u.bind(y, 42) u.bind(z, 3.14) - assert u.unbound() == [] - assert u._store.bound == {x:3.14, y:42, z:3.14} - - def test_already_in_store(self): - x = u.var('x') - raises(u.AlreadyInStore, u.var, 'x') - - def test_already_bound(self): - x = u.var('x') - u.bind(x, 42) - raises(u.AlreadyBound, u.bind, x, 42) + assert x.val == 3.14 + assert y.val == 42 + assert z.val == 3.14 def test_unify_lists_success(self): x,y,z,w = (u.var('x'), u.var('y'), @@ -42,8 +39,10 @@ u.bind(x, [42, z]) u.bind(y, [w, 44]) u.unify(x, y) - assert u._store.bound == {y: [w, 44], x: [42, z], - z: 44, w: 42} + assert x.val == [42, z] + assert y.val == [w, 44] + assert z.val == 44 + assert w.val == 42 def test_unify_dicts_success(self): x,y,z,w = (u.var('x'), u.var('y'), @@ -51,9 +50,10 @@ u.bind(x, {1:42, 2:z}) u.bind(y, {1:w, 2:44}) u.unify(x, y) - assert u._store.bound == {y: {1:w, 2:44}, - x: {1:42, 2:z}, - z: 44, w: 42} + assert x.val == {1:42, 2:z} + assert y.val == {1:w, 2:44} + assert z.val == 44 + assert w.val == 42 def test_unify_failure(self): x,y,z = u.var('x'), u.var('y'), u.var('z') Modified: pypy/dist/pypy/lib/logic/unification.py ============================================================================== --- pypy/dist/pypy/lib/logic/unification.py (original) +++ pypy/dist/pypy/lib/logic/unification.py Thu Jan 19 15:46:33 2006 @@ -16,17 +16,36 @@ """An equivalence set for variables""" pass +class NoValue: + pass + class Var(object): def __init__(self, name, store): self.name = name self.store = store - store.add_unbound(self) + # top-level 'commited' binding + self.val = NoValue + # when updated in a 'transaction', keep track + # of our initial value (for abort cases) + self.previous = None + + def begin(self): + self.previous = self.val + + def commit(self): + self.previous = None + + def abort(self): + self.val = self.previous + + def is_bound(self): + return not isinstance(self.val, EqSet) \ + and self.val != NoValue def __str__(self): - if self in self.store.bound.keys(): - return "%s = %s" % (self.name, - self.store.bound[self]) + if self.is_bound(): + return "%s = %s" % (self.name, self.val) return "%s" % self.name def __repr__(self): @@ -40,7 +59,9 @@ return self.name.__hash__() def var(name): - return Var(name, _store) + v = Var(name, _store) + _store.add_unbound(v) + return v #----------- Store Exceptions ---------------------------- class VariableException(Exception): @@ -90,7 +111,10 @@ self.vars = set() self.equisets = {} self.unbound = set() - self.bound = {} + #self.bound = {} + # the transaction flag helps manage the change-list + # for variables + self.in_transaction = False def add_unbound(self, var): # register globally @@ -99,9 +123,9 @@ print "adding %s to the store" % var self.vars.add(var) # put into new singleton equiv. set - eqset = EqSet([var]) - self.equisets[var] = eqset - self.unbound.add(eqset) + var.val = EqSet([var]) + #self.equisets[var] = eqset + #self.unbound.add(eqset) #-- BIND ------------------------------------------- @@ -113,51 +137,34 @@ assert(isinstance(var, Var) and (var in self.vars)) if var == val: return - if self._both_are_vars(var, val): - if self._both_are_bound(var, val): + if _both_are_vars(var, val): + if _both_are_bound(var, val): raise AlreadyBound(var.name) - if self.bound.has_key(var): # 2. + if var.is_bound(): # 2b. var is bound, not var self.bind(val, var) - elif self.bound.has_key(val): # 2. - self._bind(self.equisets[var], - self.bound[val]) - else: # 1. - self._merge(self.equisets[var], - self.equisets[val]) - else: # 3. - if self.bound.has_key(var): + elif val.is_bound(): # 2a.val is bound, not val + self._bind(var.val, val.val) + else: # 1. both are unbound + self._merge(var.val, val.val) + else: # 3. val is really a value + if var.is_bound(): raise AlreadyBound(var.name) - self._bind(self.equisets[var], val) + self._bind(var.val, val) - def _both_are_vars(self, v1, v2): - try: - return v1 in self.vars and v2 in self.vars - except: - return False - - def _both_are_bound(self, v1, v2): - return self.bound.has_key(v1) and \ - self.bound.has_key(v2) def _bind(self, eqs, val): print "variable - value binding : %s %s" % (eqs, val) # bind all vars in the eqset to obj - for name in eqs: - del self.equisets[name] - self.bound[name] = val - self.unbound.remove(eqs) + for var in eqs: + var.val = val def _merge(self, eqs1, eqs2): print "unbound variables binding : %s %s" % (eqs1, eqs2) if eqs1 == eqs2: return # merge two equisets into one neweqs = eqs1 | eqs2 - # let's reassign everybody to eqs1 - for name in neweqs: - self.equisets[name] = neweqs - self.unbound.remove(eqs1) - self.unbound.remove(eqs2) - self.unbound.add(neweqs) + # let's reassign everybody to neweqs + self._bind(neweqs, neweqs) #-- UNIFY ------------------------------------------ @@ -167,25 +174,30 @@ print "unify %s with %s" % (x,y) if not _unifiable(x, y): raise UnificationFailure(x, y) # dispatch to the apropriate unifier + if not isinstance(x, Var): + return self._unify_var_val(y, x) + if not isinstance(y, Var): + return self._unify_var_val(x, y) try: - if x not in self.bound and y not in self.bound: - if x != y: - if isinstance(x, Var): - self.bind(x,y) - else: - self.bind(y,x) - elif x in self.bound and y in self.bound: + if x.is_bound() and y.is_bound(): self._unify_bound(x,y) - elif x in self.bound: + elif x.isbound(): self.bind(x,y) else: self.bind(y,x) except AlreadyBound: raise UnificationFailure(x, y) + + def _unify_var_val(self, x, y): + if not x.is_bound(): + if x != y: + self.bind(x, y) + else: + raise UnificationFailure(x ,y) def _unify_bound(self, x, y): print "unify bound %s %s" % (x, y) - vx, vy = (self.bound[x], self.bound[y]) + vx, vy = (x.val, y.val) if type(vx) in [list, set] and isinstance(vy, type(vx)): self._unify_iterable(x, y) elif type(vx) is dict and isinstance(vy, type(vx)): @@ -195,7 +207,7 @@ def _unify_iterable(self, x, y): print "unify sequences %s %s" % (x, y) - vx, vy = (self.bound[x], self.bound[y]) + vx, vy = (x.val, y.val) idx, top = (0, len(vx)) while (idx < top): self.unify(vx[idx], vy[idx]) @@ -203,14 +215,13 @@ def _unify_mapping(self, x, y): print "unify mappings %s %s" % (x, y) - vx, vy = (self.bound[x], self.bound[y]) + vx, vy = (x.val, y.val) for xk in vx.keys(): self.unify(vx[xk], vy[xk]) #-- Unifiability checks--------------------------------------- #-- #-- quite costly & could be merged back in unify -#-- FIXME : memoize _iterable def _iterable(thing): return type(thing) in [list, set] @@ -218,6 +229,7 @@ def _mapping(thing): return type(thing) is dict +# memoizer for _unifiable _unifiable_memo = set() def _unifiable(term1, term2): @@ -262,6 +274,13 @@ [e[1] for e in v2]) #-- Some utilities ----------------------------------------------- + +def _both_are_vars(v1, v2): + return isinstance(v1, Var) and isinstance(v2, Var) + +def _both_are_bound(v1, v2): + return v1.is_bound() and v2.is_bound() + #-- #-- the global store _store = Store() From ericvrp at codespeak.net Thu Jan 19 15:47:38 2006 From: ericvrp at codespeak.net (ericvrp at codespeak.net) Date: Thu, 19 Jan 2006 15:47:38 +0100 (CET) Subject: [pypy-svn] r22443 - in pypy/dist/pypy/translator/llvm: . module Message-ID: <20060119144738.00F8627DC0@code1.codespeak.net> Author: ericvrp Date: Thu Jan 19 15:47:32 2006 New Revision: 22443 Modified: pypy/dist/pypy/translator/llvm/genllvm.py pypy/dist/pypy/translator/llvm/module/support.py Log: argh Modified: pypy/dist/pypy/translator/llvm/genllvm.py ============================================================================== --- pypy/dist/pypy/translator/llvm/genllvm.py (original) +++ pypy/dist/pypy/translator/llvm/genllvm.py Thu Jan 19 15:47:32 2006 @@ -10,7 +10,7 @@ from pypy.translator.llvm.codewriter import CodeWriter from pypy.translator.llvm import extfuncnode from pypy.translator.llvm.module.support import \ - extdeclarations, extfunctions, write_raise_exc + extdeclarations, extfunctions, extfunctions_standalone, write_raise_exc from pypy.translator.llvm.node import LLVMNode from pypy.translator.llvm.externs2ll import setup_externs, generate_llfile from pypy.translator.llvm.gc import GcPolicy @@ -151,6 +151,8 @@ codewriter.header_comment('External Function Implementation') codewriter.write_lines(self.llexterns_functions) codewriter.write_lines(self._set_wordsize(extfunctions)) + if self.standalone: + codewriter.write_lines(self._set_wordsize(extfunctions_standalone)) self.write_extern_impls(codewriter) self.write_setup_impl(codewriter) Modified: pypy/dist/pypy/translator/llvm/module/support.py ============================================================================== --- pypy/dist/pypy/translator/llvm/module/support.py (original) +++ pypy/dist/pypy/translator/llvm/module/support.py Thu Jan 19 15:47:32 2006 @@ -88,12 +88,6 @@ ret void } -internal fastcc int %pypy_entry_point(%RPyListOfString* %argv) { - %result = call long %pypy_entry_point(%RPyListOfString* %argv) - %tmp = cast long %result to int - ret int %tmp -} - internal fastcc void %pypy__RPyListOfString_SetItem__listPtr_Signed_rpy_stringPtr(%RPyListOfString* %l_1, int %index_0, %RPyString* %newstring_0) { %index_0_long = cast int %index_0 to long call void %pypy__RPyListOfString_SetItem__listPtr_Signed_rpy_stringPtr(%RPyListOfString* %l_1, long %index_0_long, %RPyString* %newstring_0) @@ -102,6 +96,18 @@ """ +extfunctions_standalone = """ +""" +if maxint != 2**31-1: + extfunctions_standalone += """ +internal fastcc int %pypy_entry_point(%RPyListOfString* %argv) { + %result = call long %pypy_entry_point(%RPyListOfString* %argv) + %tmp = cast long %result to int + ret int %tmp +} + +""" + def write_raise_exc(c_name, exc_repr, codewriter): l = """ From auc at codespeak.net Thu Jan 19 16:00:25 2006 From: auc at codespeak.net (auc at codespeak.net) Date: Thu, 19 Jan 2006 16:00:25 +0100 (CET) Subject: [pypy-svn] r22444 - pypy/dist/pypy/lib/logic Message-ID: <20060119150025.E498727DC8@code1.codespeak.net> Author: auc Date: Thu Jan 19 16:00:24 2006 New Revision: 22444 Modified: pypy/dist/pypy/lib/logic/unification.py Log: cleanup Modified: pypy/dist/pypy/lib/logic/unification.py ============================================================================== --- pypy/dist/pypy/lib/logic/unification.py (original) +++ pypy/dist/pypy/lib/logic/unification.py Thu Jan 19 16:00:24 2006 @@ -105,13 +105,7 @@ def __init__(self): # set of all known vars - # var->equivalence set mapping for unbound vars - # set of equisets (clusters of unbound variables) - # var->objects bindings self.vars = set() - self.equisets = {} - self.unbound = set() - #self.bound = {} # the transaction flag helps manage the change-list # for variables self.in_transaction = False @@ -124,8 +118,6 @@ self.vars.add(var) # put into new singleton equiv. set var.val = EqSet([var]) - #self.equisets[var] = eqset - #self.unbound.add(eqset) #-- BIND ------------------------------------------- @@ -174,26 +166,23 @@ print "unify %s with %s" % (x,y) if not _unifiable(x, y): raise UnificationFailure(x, y) # dispatch to the apropriate unifier - if not isinstance(x, Var): + if not x in self.vars: + if not y in self.vars: + if x != y: raise UnificationFailure(x, y) return self._unify_var_val(y, x) - if not isinstance(y, Var): + if not y in self.vars: return self._unify_var_val(x, y) - try: - if x.is_bound() and y.is_bound(): - self._unify_bound(x,y) - elif x.isbound(): - self.bind(x,y) - else: - self.bind(y,x) - except AlreadyBound: - raise UnificationFailure(x, y) + if _both_are_bound(x, y): + self._unify_bound(x,y) + elif x.isbound(): + self.bind(x,y) + else: + self.bind(y,x) def _unify_var_val(self, x, y): - if not x.is_bound(): - if x != y: - self.bind(x, y) - else: - raise UnificationFailure(x ,y) + if x.is_bound(): raise UnificationFailure(x ,y) + if x != y: + self.bind(x, y) def _unify_bound(self, x, y): print "unify bound %s %s" % (x, y) From ericvrp at codespeak.net Thu Jan 19 16:07:03 2006 From: ericvrp at codespeak.net (ericvrp at codespeak.net) Date: Thu, 19 Jan 2006 16:07:03 +0100 (CET) Subject: [pypy-svn] r22445 - in pypy/dist/pypy/translator/llvm: . test Message-ID: <20060119150703.647EF27DC2@code1.codespeak.net> Author: ericvrp Date: Thu Jan 19 16:06:59 2006 New Revision: 22445 Modified: pypy/dist/pypy/translator/llvm/opwriter.py pypy/dist/pypy/translator/llvm/test/test_lladdresses.py Log: Fixes for raw memory arithmetic. Fix for test this gives misalignment warnings on ia64. ("python(17687): unaligned access to 0x6000000000a3f05a, ip=0x2000000001b14c00") Modified: pypy/dist/pypy/translator/llvm/opwriter.py ============================================================================== --- pypy/dist/pypy/translator/llvm/opwriter.py (original) +++ pypy/dist/pypy/translator/llvm/opwriter.py Thu Jan 19 16:06:59 2006 @@ -457,7 +457,7 @@ self.codewriter.getelementptr(incr_addr, addr_type, cast_addr, - [("int", arg_incr)], + [(self.word, arg_incr)], getptr=False) cast_addr = incr_addr self.codewriter.store(argtype_value, arg_value, cast_addr) @@ -479,7 +479,7 @@ self.codewriter.getelementptr(incr_addr, addr_type, cast_addr, - [("int", arg_incr)], + [(self.word, arg_incr)], getptr=False) cast_addr = incr_addr Modified: pypy/dist/pypy/translator/llvm/test/test_lladdresses.py ============================================================================== --- pypy/dist/pypy/translator/llvm/test/test_lladdresses.py (original) +++ pypy/dist/pypy/translator/llvm/test/test_lladdresses.py Thu Jan 19 16:06:59 2006 @@ -75,13 +75,13 @@ def f(): addr = raw_malloc(100) addr.signed[0] = 12 - (addr + 10).signed[0] = 42 - (addr + 20).char[0] = "a" + (addr + 16).signed[0] = 42 + (addr + 32).char[0] = "a" addr1 = raw_malloc(100) raw_memcopy(addr, addr1, 100) result = addr1.signed[0] == 12 - result += (addr1 + 10).signed[0] == 42 - result += (addr1 + 20).char[0] == "a" + result += (addr1 + 16).signed[0] == 42 + result += (addr1 + 32).char[0] == "a" return result fc = compile(f, []) res = fc() From auc at codespeak.net Thu Jan 19 17:11:56 2006 From: auc at codespeak.net (auc at codespeak.net) Date: Thu, 19 Jan 2006 17:11:56 +0100 (CET) Subject: [pypy-svn] r22450 - pypy/dist/pypy/lib/logic Message-ID: <20060119161156.C83C827DC9@code1.codespeak.net> Author: auc Date: Thu Jan 19 17:11:54 2006 New Revision: 22450 Modified: pypy/dist/pypy/lib/logic/test_unification.py pypy/dist/pypy/lib/logic/unification.py Log: unify made atomic Modified: pypy/dist/pypy/lib/logic/test_unification.py ============================================================================== --- pypy/dist/pypy/lib/logic/test_unification.py (original) +++ pypy/dist/pypy/lib/logic/test_unification.py Thu Jan 19 17:11:54 2006 @@ -60,6 +60,10 @@ u.bind(x, [42, z]) u.bind(y, [z, 44]) raises(u.UnificationFailure, u.unify, x, y) + # check store consistency + assert x.val == [42, z] + assert y.val == [z, 44] + assert z.val == u.EqSet([z]) def test_unify_failure2(self): x,y,z,w = (u.var('x'), u.var('y'), @@ -68,6 +72,11 @@ u.bind(y, [w, 44]) u.bind(z, w) raises(u.UnificationFailure, u.unify, x, y) + # check store consistency + assert x.val == [42, z] + assert y.val == [w, 44] + assert z.val == u.EqSet([z,w]) + assert w.val == u.EqSet([z,w]) def test_unify_circular(self): x, y, z, w, a, b = (u.var('x'), u.var('y'), @@ -82,4 +91,12 @@ u.bind(a, {1:42, 2:b}) u.bind(b, {1:a, 2:42}) raises(u.UnificationFailure, u.unify, a, b) + # check store consistency + assert x.val == [y] + assert y.val == [x] + assert z.val == [1, w] + assert w.val == [z, 2] + assert a.val == {1:42, 2:b} + assert b.val == {1:a, 2:42} + Modified: pypy/dist/pypy/lib/logic/unification.py ============================================================================== --- pypy/dist/pypy/lib/logic/unification.py (original) +++ pypy/dist/pypy/lib/logic/unification.py Thu Jan 19 17:11:54 2006 @@ -3,8 +3,9 @@ # crude and buggy ... #TODO : -# * ensure that the store is intact after a failure -# (make unify atomic) +# * turn Var into some dataflow-ish thing +# * ensure that the store supports concurrent access +# (using the implicit blocking provided by dataflow vars) # * provide a way to copy the store to a fresh one # (clone operator) # After reading more of the "book", I see some more ops @@ -12,7 +13,7 @@ # space ... #----------- Variables ---------------------------------- -class EqSet(frozenset): +class EqSet(set): """An equivalence set for variables""" pass @@ -25,23 +26,33 @@ self.name = name self.store = store # top-level 'commited' binding - self.val = NoValue + self._val = NoValue # when updated in a 'transaction', keep track # of our initial value (for abort cases) self.previous = None + self.changed = False - def begin(self): - self.previous = self.val + def is_bound(self): + return not isinstance(self.val, EqSet) \ + and self.val != NoValue def commit(self): - self.previous = None + self.changed = False def abort(self): self.val = self.previous + self.changed = False - def is_bound(self): - return not isinstance(self.val, EqSet) \ - and self.val != NoValue + def set_val(self, val): + if self.store.in_transaction: + if not self.changed: + self.previous = self._val + self.changed = True + print "in transaction, %s <- %s" % (self.name, val) + self._val = val + def get_val(self): + return self._val + val = property(get_val, set_val) def __str__(self): if self.is_bound(): @@ -106,8 +117,6 @@ def __init__(self): # set of all known vars self.vars = set() - # the transaction flag helps manage the change-list - # for variables self.in_transaction = False def add_unbound(self, var): @@ -154,13 +163,29 @@ print "unbound variables binding : %s %s" % (eqs1, eqs2) if eqs1 == eqs2: return # merge two equisets into one - neweqs = eqs1 | eqs2 + eqs1 |= eqs2 # let's reassign everybody to neweqs - self._bind(neweqs, neweqs) + self._bind(eqs1, eqs1) #-- UNIFY ------------------------------------------ def unify(self, x, y): + self.in_transaction = True + try: + try: + self._really_unify(x, y) + for var in self.vars: + if var.changed: + var.commit() + except: + for var in self.vars: + if var.changed: + var.abort() + raise + finally: + self.in_transaction = False + + def _really_unify(self, x, y): #FIXME in case of failure, the store state is not # properly restored ... print "unify %s with %s" % (x,y) @@ -169,10 +194,10 @@ if not x in self.vars: if not y in self.vars: if x != y: raise UnificationFailure(x, y) - return self._unify_var_val(y, x) - if not y in self.vars: - return self._unify_var_val(x, y) - if _both_are_bound(x, y): + self._unify_var_val(y, x) + elif not y in self.vars: + self._unify_var_val(x, y) + elif _both_are_bound(x, y): self._unify_bound(x,y) elif x.isbound(): self.bind(x,y) @@ -180,7 +205,7 @@ self.bind(y,x) def _unify_var_val(self, x, y): - if x.is_bound(): raise UnificationFailure(x ,y) + if x.is_bound(): raise UnificationFailure(x, y) if x != y: self.bind(x, y) @@ -199,14 +224,14 @@ vx, vy = (x.val, y.val) idx, top = (0, len(vx)) while (idx < top): - self.unify(vx[idx], vy[idx]) + self._really_unify(vx[idx], vy[idx]) idx += 1 def _unify_mapping(self, x, y): print "unify mappings %s %s" % (x, y) vx, vy = (x.val, y.val) for xk in vx.keys(): - self.unify(vx[xk], vy[xk]) + self._really_unify(vx[xk], vy[xk]) #-- Unifiability checks--------------------------------------- #-- From ac at codespeak.net Thu Jan 19 17:38:54 2006 From: ac at codespeak.net (ac at codespeak.net) Date: Thu, 19 Jan 2006 17:38:54 +0100 (CET) Subject: [pypy-svn] r22452 - in pypy/branch/arre-experiments/pypy: interpreter interpreter/astcompiler interpreter/test objspace/flow objspace/flow/test objspace/std tool/test Message-ID: <20060119163854.BABD927DC9@code1.codespeak.net> Author: ac Date: Thu Jan 19 17:38:53 2006 New Revision: 22452 Modified: pypy/branch/arre-experiments/pypy/interpreter/astcompiler/pyassem.py pypy/branch/arre-experiments/pypy/interpreter/baseobjspace.py pypy/branch/arre-experiments/pypy/interpreter/gateway.py pypy/branch/arre-experiments/pypy/interpreter/pycode.py pypy/branch/arre-experiments/pypy/interpreter/pycompiler.py pypy/branch/arre-experiments/pypy/interpreter/test/test_eval.py pypy/branch/arre-experiments/pypy/interpreter/test/test_function.py pypy/branch/arre-experiments/pypy/objspace/flow/objspace.py pypy/branch/arre-experiments/pypy/objspace/flow/test/test_framestate.py pypy/branch/arre-experiments/pypy/objspace/std/marshal_impl.py pypy/branch/arre-experiments/pypy/tool/test/test_pytestsupport.py Log: Refactor construction of PyCode objects. Modified: pypy/branch/arre-experiments/pypy/interpreter/astcompiler/pyassem.py ============================================================================== --- pypy/branch/arre-experiments/pypy/interpreter/astcompiler/pyassem.py (original) +++ pypy/branch/arre-experiments/pypy/interpreter/astcompiler/pyassem.py Thu Jan 19 17:38:53 2006 @@ -832,19 +832,18 @@ argcount = argcount - 1 # was return new.code, now we just return the parameters and let # the caller create the code object - # XXX _code_new_w itself is not really annotable - return PyCode(self.space)._code_new_w( argcount, nlocals, - self.stacksize, self.flags, - self.lnotab.getCode(), - self.getConsts(), - self.names, - self.varnames, - self.filename, self.name, - self.firstline, - self.lnotab.getTable(), - self.freevars, - self.cellvars - ) + return PyCode( self.space, argcount, nlocals, + self.stacksize, self.flags, + self.lnotab.getCode(), + self.getConsts(), + self.names, + self.varnames, + self.filename, self.name, + self.firstline, + self.lnotab.getTable(), + self.freevars, + self.cellvars + ) def getConsts(self): """Return a tuple for the const slot of the code object Modified: pypy/branch/arre-experiments/pypy/interpreter/baseobjspace.py ============================================================================== --- pypy/branch/arre-experiments/pypy/interpreter/baseobjspace.py (original) +++ pypy/branch/arre-experiments/pypy/interpreter/baseobjspace.py Thu Jan 19 17:38:53 2006 @@ -577,7 +577,7 @@ if isinstance(expression, str): expression = compile(expression, '?', 'eval') if isinstance(expression, types.CodeType): - expression = PyCode(self)._from_code(expression) + expression = PyCode._from_code(self, expression) if not isinstance(expression, PyCode): raise TypeError, 'space.eval(): expected a string, code or PyCode object' return expression.exec_code(self, w_globals, w_locals) @@ -589,7 +589,7 @@ if isinstance(statement, str): statement = compile(statement, '?', 'exec') if isinstance(statement, types.CodeType): - statement = PyCode(self)._from_code(statement) + statement = PyCode._from_code(self, statement) if not isinstance(statement, PyCode): raise TypeError, 'space.exec_(): expected a string, code or PyCode object' w_key = self.wrap('__builtins__') Modified: pypy/branch/arre-experiments/pypy/interpreter/gateway.py ============================================================================== --- pypy/branch/arre-experiments/pypy/interpreter/gateway.py (original) +++ pypy/branch/arre-experiments/pypy/interpreter/gateway.py Thu Jan 19 17:38:53 2006 @@ -694,7 +694,7 @@ def build_applevel_dict(self, space): "NOT_RPYTHON" from pypy.interpreter.pycode import PyCode - pycode = PyCode(space)._from_code(self.code, hidden_applevel=self.hidden_applevel) + pycode = PyCode._from_code(space, self.code, hidden_applevel=self.hidden_applevel) w_glob = space.newdict([]) space.setitem(w_glob, space.wrap('__name__'), space.wrap('__builtin__')) space.exec_(pycode, w_glob, w_glob) Modified: pypy/branch/arre-experiments/pypy/interpreter/pycode.py ============================================================================== --- pypy/branch/arre-experiments/pypy/interpreter/pycode.py (original) +++ pypy/branch/arre-experiments/pypy/interpreter/pycode.py Thu Jan 19 17:38:53 2006 @@ -4,7 +4,8 @@ The bytecode interpreter itself is implemented by the PyFrame class. """ -import dis +import dis, imp, struct, types + from pypy.interpreter import eval from pypy.interpreter.error import OperationError from pypy.interpreter.gateway import NoneNotWrapped @@ -45,6 +46,7 @@ kwargname = None return argnames, varargname, kwargname +cpython_magic, = struct.unpack("lineno mapping - self.hidden_applevel = False - self.do_fastcall = -1 - - def _code_new( self, argcount, nlocals, stacksize, flags, - code, consts, names, varnames, filename, - name, firstlineno, lnotab, freevars, cellvars, - hidden_applevel=False, from_cpython=False): - """Initialize a new code objects from parameters from new.code""" - # simply try to suck in all attributes we know of - # with a lot of boring asserts to enforce type knowledge - # XXX get rid of that ASAP with a real compiler! - import types - x = argcount; assert isinstance(x, int) - self.co_argcount = x - x = nlocals; assert isinstance(x, int) - self.co_nlocals = x - x = stacksize; assert isinstance(x, int) - self.co_stacksize = x - x = flags; assert isinstance(x, int) - self.co_flags = x - x = code; assert isinstance(x, str) - self.co_code = x - #self.co_consts = - x = names; assert isinstance(x, tuple) - self.co_names = [ str(n) for n in x ] - x = varnames; assert isinstance(x, tuple) - self.co_varnames = [ str(n) for n in x ] - x = freevars; assert isinstance(x, tuple) - self.co_freevars = [ str(n) for n in x ] - x = cellvars; assert isinstance(x, tuple) - self.co_cellvars = [ str(n) for n in x ] - x = filename; assert isinstance(x, str) - self.co_filename = x - x = name; assert isinstance(x, str) - self.co_name = x - x = firstlineno; assert isinstance(x, int) - self.co_firstlineno = x - x = lnotab; assert isinstance(x, str) - self.co_lnotab = x - # recursively _from_code()-ify the code objects in code.co_consts - space = self.space - newconsts_w = [] - for const in consts: - if isinstance(const, types.CodeType): # from stable compiler - const = PyCode(space)._from_code(const, hidden_applevel=hidden_applevel, - from_cpython=from_cpython) - newconsts_w.append(space.wrap(const)) - self.co_consts_w = newconsts_w - # stick the underlying CPython magic value, if the code object - # comes from there - if from_cpython: - import imp, struct - magic, = struct.unpack(" Author: auc Date: Thu Jan 19 18:09:16 2006 New Revision: 22456 Modified: pypy/dist/pypy/lib/logic/test_unification.py pypy/dist/pypy/lib/logic/unification.py Log: TODO update Modified: pypy/dist/pypy/lib/logic/test_unification.py ============================================================================== --- pypy/dist/pypy/lib/logic/test_unification.py (original) +++ pypy/dist/pypy/lib/logic/test_unification.py Thu Jan 19 18:09:16 2006 @@ -71,7 +71,9 @@ u.bind(x, [42, z]) u.bind(y, [w, 44]) u.bind(z, w) + assert u._store.in_transaction == False raises(u.UnificationFailure, u.unify, x, y) + assert u._store.in_transaction == False # check store consistency assert x.val == [42, z] assert y.val == [w, 44] Modified: pypy/dist/pypy/lib/logic/unification.py ============================================================================== --- pypy/dist/pypy/lib/logic/unification.py (original) +++ pypy/dist/pypy/lib/logic/unification.py Thu Jan 19 18:09:16 2006 @@ -1,16 +1,15 @@ -# Oz unification in Python 2.4 +# Oz-like unification of dataflow variables in Python 2.4 # within a single assignment store -# crude and buggy ... +# crude ... #TODO : -# * turn Var into some dataflow-ish thing +# * understand this : +# http://www.mozart-oz.org/papers/abstracts/ProgrammingConstraintServices.html +# * turn Var into some dataflow-ish thing (as far as Python allows) # * ensure that the store supports concurrent access # (using the implicit blocking provided by dataflow vars) -# * provide a way to copy the store to a fresh one -# (clone operator) -# After reading more of the "book", I see some more ops -# are needed for the store to be part of a computation -# space ... +# * add entailment checks +# * add constraints support #----------- Variables ---------------------------------- class EqSet(set): From mwh at codespeak.net Thu Jan 19 18:18:02 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Thu, 19 Jan 2006 18:18:02 +0100 (CET) Subject: [pypy-svn] r22457 - in pypy/dist/pypy: annotation rpython rpython/lltypesystem rpython/memory rpython/memory/test Message-ID: <20060119171802.5E9DA27DC9@code1.codespeak.net> Author: mwh Date: Thu Jan 19 18:17:59 2006 New Revision: 22457 Modified: pypy/dist/pypy/annotation/model.py pypy/dist/pypy/rpython/lltypesystem/lltype.py pypy/dist/pypy/rpython/memory/lladdress.py pypy/dist/pypy/rpython/memory/test/test_address.py pypy/dist/pypy/rpython/objectmodel.py pypy/dist/pypy/rpython/raddress.py pypy/dist/pypy/rpython/rbuiltin.py Log: allow rtyping of calls to the offsetof function. Modified: pypy/dist/pypy/annotation/model.py ============================================================================== --- pypy/dist/pypy/annotation/model.py (original) +++ pypy/dist/pypy/annotation/model.py Thu Jan 19 18:17:59 2006 @@ -522,6 +522,7 @@ (SomeChar(), lltype.Char), (SomeUnicodeCodePoint(), lltype.UniChar), (SomeAddress(), lladdress.Address), + (SomeOffset(), lladdress.Offset), ] def annotation_to_lltype(s_val, info=None): Modified: pypy/dist/pypy/rpython/lltypesystem/lltype.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/lltype.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/lltype.py Thu Jan 19 18:17:59 2006 @@ -1,6 +1,7 @@ import weakref import py from pypy.rpython.rarithmetic import r_uint, r_ulonglong, r_longlong +from pypy.rpython.objectmodel import Symbolic from pypy.tool.uid import Hashable from pypy.tool.tls import tlsobject from types import NoneType @@ -500,6 +501,8 @@ if tp is unicode: assert len(val) == 1 return UniChar + if issubclass(tp, Symbolic): + return val.lltype() raise TypeError("typeOf(%r object)" % (tp.__name__,)) class InvalidCast(TypeError): Modified: pypy/dist/pypy/rpython/memory/lladdress.py ============================================================================== --- pypy/dist/pypy/rpython/memory/lladdress.py (original) +++ pypy/dist/pypy/rpython/memory/lladdress.py Thu Jan 19 18:17:59 2006 @@ -134,9 +134,14 @@ from pypy.annotation import model return model.SomeOffset() + def lltype(self): + return Offset + def __repr__(self): return "" % (self.TYPE, self.fldname) +Offset = lltype.Primitive("Offset", OffsetOf(lltype.Void, '')) + def sizeof(TYPE, n=None): pass Modified: pypy/dist/pypy/rpython/memory/test/test_address.py ============================================================================== --- pypy/dist/pypy/rpython/memory/test/test_address.py (original) +++ pypy/dist/pypy/rpython/memory/test/test_address.py Thu Jan 19 18:17:59 2006 @@ -217,6 +217,27 @@ graph = graphof(a.translator, f) assert graph.startblock.operations[0].result.concretetype == Address + + def test_simple_offsetof(self): + from pypy.rpython.lltypesystem import lltype + from pypy.rpython.memory.lladdress import offsetof + S = lltype.GcStruct('S', ('x', lltype.Bool), ('y', lltype.Signed)) + def f(): + return offsetof(S, 'x') + offsetof(S, 'y') + a = RPythonAnnotator() + s = a.build_types(f, []) + rtyper = RPythonTyper(a) + rtyper.specialize() #does not raise + + coff = offsetof(S, 'y') + def f(): + return coff + a = RPythonAnnotator() + s = a.build_types(f, []) + rtyper = RPythonTyper(a) + rtyper.specialize() #does not raise + + class TestAddressInLLInterp(object): def test_null(self): def f(): Modified: pypy/dist/pypy/rpython/objectmodel.py ============================================================================== --- pypy/dist/pypy/rpython/objectmodel.py (original) +++ pypy/dist/pypy/rpython/objectmodel.py Thu Jan 19 18:17:59 2006 @@ -8,6 +8,9 @@ def annotation(self): return None + def lltype(self): + return None + import new import weakref Modified: pypy/dist/pypy/rpython/raddress.py ============================================================================== --- pypy/dist/pypy/rpython/raddress.py (original) +++ pypy/dist/pypy/rpython/raddress.py Thu Jan 19 18:17:59 2006 @@ -1,7 +1,7 @@ # rtyping of memory address operations from pypy.annotation.pairtype import pairtype from pypy.annotation import model as annmodel -from pypy.rpython.memory.lladdress import NULL, Address +from pypy.rpython.memory.lladdress import NULL, Address, Offset from pypy.rpython.rmodel import Repr, IntegerRepr from pypy.rpython.lltypesystem import lltype @@ -110,3 +110,20 @@ v_addr1, v_addr2 = hop.inputargs(Address, Address) return hop.genop('adr_ge', [v_addr1, v_addr2], resulttype=lltype.Bool) + +class __extend__(annmodel.SomeOffset): + def rtyper_makerepr(self, rtyper): + return offset_repr + + def rtyper_makekey(self): + return self.__class__, + +class OffsetRepr(Repr): + lowleveltype = Offset + +offset_repr = OffsetRepr() + +class __extend__(pairtype(OffsetRepr, OffsetRepr)): + def rtype_add((r_offset1, r_offset2), hop): + v_offset1, v_offset2 = hop.inputargs(Offset, Offset) + return hop.genop('offset_add', [v_offset1, v_offset2], resulttype=Offset) Modified: pypy/dist/pypy/rpython/rbuiltin.py ============================================================================== --- pypy/dist/pypy/rpython/rbuiltin.py (original) +++ pypy/dist/pypy/rpython/rbuiltin.py Thu Jan 19 18:17:59 2006 @@ -378,6 +378,13 @@ BUILTIN_TYPER[lladdress.raw_free] = rtype_raw_free BUILTIN_TYPER[lladdress.raw_memcopy] = rtype_raw_memcopy +def rtype_offsetof(hop): + from pypy.rpython.memory.lladdress import Offset + TYPE, field = hop.inputargs(lltype.Void, lltype.Void) + return hop.inputconst(Offset, lladdress.offsetof(TYPE.value, field.value)) + +BUILTIN_TYPER[lladdress.offsetof] = rtype_offsetof + # _________________________________________________________________ # non-gc objects From rxe at codespeak.net Thu Jan 19 19:48:16 2006 From: rxe at codespeak.net (rxe at codespeak.net) Date: Thu, 19 Jan 2006 19:48:16 +0100 (CET) Subject: [pypy-svn] r22459 - pypy/dist/pypy/translator/llvm/test Message-ID: <20060119184816.DBA1827DC5@code1.codespeak.net> Author: rxe Date: Thu Jan 19 19:48:15 2006 New Revision: 22459 Modified: pypy/dist/pypy/translator/llvm/test/test_lladdresses.py Log: Exploring a bit at how flavoured mallocs could work in llvm with separate policies - not much success - postponed until sprint. Modified: pypy/dist/pypy/translator/llvm/test/test_lladdresses.py ============================================================================== --- pypy/dist/pypy/translator/llvm/test/test_lladdresses.py (original) +++ pypy/dist/pypy/translator/llvm/test/test_lladdresses.py Thu Jan 19 19:48:15 2006 @@ -116,6 +116,20 @@ fn = compile(f, [int]) assert fn(1) == 2 +def test_flavored_varmalloc_raw(): + py.test.skip("flavored_malloc not working?") + A = lltype.Array(lltype.Signed) + VARS = lltype.GcStruct('test', ('a', lltype.Signed), ('b', A)) + def f(x, y): + #s = lltype.flavored_malloc('gc', VARS, x) + s = lltype.malloc(VARS, n=x, flavor='gc') + s.a = 42 + s.b[0] = y * 2 + return s.b[0] - s.a + + fn = compile(f, [int, int]) + assert fn(2, 24) == 6 + def test_flavored_malloc_alloca(): class A(object): _alloc_flavor_ = "stack" @@ -128,3 +142,4 @@ return result fn = compile(f, [int]) assert fn(1) == 2 + From auc at codespeak.net Fri Jan 20 10:01:42 2006 From: auc at codespeak.net (auc at codespeak.net) Date: Fri, 20 Jan 2006 10:01:42 +0100 (CET) Subject: [pypy-svn] r22465 - pypy/dist/pypy/lib/logic Message-ID: <20060120090142.3463627DC7@code1.codespeak.net> Author: auc Date: Fri Jan 20 10:01:40 2006 New Revision: 22465 Added: pypy/dist/pypy/lib/logic/variable.py Modified: pypy/dist/pypy/lib/logic/unification.py Log: add a lot of documentation (from CTM) move Var class into variable.py Modified: pypy/dist/pypy/lib/logic/unification.py ============================================================================== --- pypy/dist/pypy/lib/logic/unification.py (original) +++ pypy/dist/pypy/lib/logic/unification.py Fri Jan 20 10:01:40 2006 @@ -1,83 +1,126 @@ -# Oz-like unification of dataflow variables in Python 2.4 -# within a single assignment store -# crude ... +## Oz-like unification of dataflow variables in Python 2.4 +## within a single assignment store +## crude ... + +## Unification over dataflow variables in Oz is a powerful mechanism +## which is the basis of relational, logic and constraint programming. +## Please read the stuff in variable.py to understand the nature of +## dataflow variables. +## +## Unification in Oz (almost verbatim from CTM*) +## ============================================= +## +## Unification as put in Oz is a very powerful operation. It is +## provided through the '=' operator. +## +## Binding a variable to a value is a special case of an operation +## called unification. The unification = makes the +## partial values and equal, if possible, by adding +## zero or more bindings to the store. For example, f(X Y)=f(1 2) does +## two bindings: X=1 and Y=2. If the two terms cannot be made equal, +## then an exception is raised. Unification exists because of partial +## values; if there would be only complete values, then it would have +## no meaning. +## +## Unification adds information to the single-assignment store (a set +## of dataflow variables, where each variable is either unbound or +## bound to some other store entity). +## +## Example : if the store has X=foo(A) and Y=foo(25), then doing X=Y +## will bind A to 25. +## +## *failure* is the exception raised when unification cannot +## happen. For instance (unification of two records) : +## +## person(name:X1 age:23) +## person(name:"Auc" age:32) +## +## will raise it. The fail statement raises this exception. +## +## It is symetric : X=Y means the same as Y=X +## It can create cyclic structures, as in : X=person(grandfather:X) +## It can bind cyclic structures : +## +## X=f(a:X b:_) +## Y=f(a:_ b:Y) +## X=Y +## +## creates a two cycles structure writable as : X=f(a:X b:X) +## +## Unification algorithm : +## +## operation unify(x, y) that unify two partial values x and y in the +## store st. +## +## The Store consists of a set of k variables : x1, ... xk that are +## partitioned as follows: +## +## * set of unbound variables that are equal (also called equivalence +## sets of variables). The variables in each set are equal to each +## other but not to any other variables. +## +## * variables bound to a number, record or procedure (also called +## determined variables). +## +## Example : st = {x1=foo(a:x2), x2=25, x3=x4=x5, x6, x7=x8} that has +## 8 variables. It has three equivalence sets : +## {x3,x4,x5},{x6},{x7,x8}. It has two determined variables x1 and x2. +## +## The primitive bind operation : unification is defined in terms of a +## primitive bind operation on the store st. The operation binds all +## variables in an equivalence set: +## +## * bind(ES,) binds all variables in the equivalence set ES to the +## number or record . For example, the operation +## bind({x7,x8},foo(a:x2)) binds x7 and x8, which no longer are in +## an equivalence set. +## +## * bind(ES1,ES2) merges the equivalence set ES1 with the equivalence +## set ES2. +## +## Algorithm unify(x,y) +## +## 1. if x is in the equivalence set ESx and y is in the equivalence +## set ESy, then do bind(ESx,ESy). It is a noop if ESx == ESy. +## +## 2. if x is in the equivalence set ESx and y is determined**, then do +## bind(ESx,y) +## +## 3. if y is in the equivalence set ESy and y is determined, then do +## bind(ESy,x) +## +## 4. if x is bound to l(l1:x1,...,ln:xn) and y is bound to +## l'(l'1:y1,...,l'm:ym) with l!=l' or {l1,...,ln}!={l'1,...,l'm}, +## then raise a failure exception +## +## 5. if x is bound to l(l1:x1,...,ln:xn) and y is bound to +## l(l1:y1,...,ln:yn), then for i from 1 to n do unify(xi,yi). +## +## +## With cycles : the above algorithm does not handle unification of +## partial values with cycles. For example, on x=f(a:x) and y=f(a:y) a +## call to unify(x,y) results in the recursive call unify(x,y), which +## is identical to the original call, looping forever. A simple fix +## is to make sure that unify(x,y) is called at most once for each +## possible pair of two variables (x,y); We do this through +## memoization of called pairs. + +## * CTM : Concepts, Techniques and Models of Computer Programming +## ** determined == bound #TODO : # * understand this : # http://www.mozart-oz.org/papers/abstracts/ProgrammingConstraintServices.html +# * support '_' as shown above # * turn Var into some dataflow-ish thing (as far as Python allows) # * ensure that the store supports concurrent access # (using the implicit blocking provided by dataflow vars) # * add entailment checks # * add constraints support -#----------- Variables ---------------------------------- -class EqSet(set): - """An equivalence set for variables""" - pass - -class NoValue: - pass - -class Var(object): - - def __init__(self, name, store): - self.name = name - self.store = store - # top-level 'commited' binding - self._val = NoValue - # when updated in a 'transaction', keep track - # of our initial value (for abort cases) - self.previous = None - self.changed = False - - def is_bound(self): - return not isinstance(self.val, EqSet) \ - and self.val != NoValue - - def commit(self): - self.changed = False - - def abort(self): - self.val = self.previous - self.changed = False - - def set_val(self, val): - if self.store.in_transaction: - if not self.changed: - self.previous = self._val - self.changed = True - print "in transaction, %s <- %s" % (self.name, val) - self._val = val - def get_val(self): - return self._val - val = property(get_val, set_val) - - def __str__(self): - if self.is_bound(): - return "%s = %s" % (self.name, self.val) - return "%s" % self.name - - def __repr__(self): - return self.__str__() - - def __eq__(self, thing): - return isinstance(thing, Var) \ - and self.name == thing.name - - def __hash__(self): - return self.name.__hash__() - -def var(name): - v = Var(name, _store) - _store.add_unbound(v) - return v +from variable import EqSet, Var, VariableException, NotAVariable #----------- Store Exceptions ---------------------------- -class VariableException(Exception): - def __init__(self, name): - self.name = name - class UnboundVariable(VariableException): def __str__(self): return "%s has no value yet" % self.name @@ -97,11 +140,6 @@ return "%s %s can't be unified" % (self.var1, self.var2) -#----------- Variable Exceptions------------------------- -class NotAVariable(VariableException): - def __str__(self): - return "%s is not a variable" % self.name - #----------- Store ------------------------------------ class Store(object): """The Store consists of a set of k variables @@ -299,6 +337,11 @@ _store = Store() #-- global accessor functions +def var(name): + v = Var(name, _store) + _store.add_unbound(v) + return v + def bind(var, val): return _store.bind(var, val) Added: pypy/dist/pypy/lib/logic/variable.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/lib/logic/variable.py Fri Jan 20 10:01:40 2006 @@ -0,0 +1,72 @@ +#----------- Exceptions --------------------------------- +class VariableException(Exception): + def __init__(self, name): + self.name = name + +class NotAVariable(VariableException): + def __str__(self): + return "%s is not a variable" % self.name + +#----------- Variables ---------------------------------- +class EqSet(set): + """An equivalence set for variables""" + pass + +class NoValue: + pass + +class Var(object): + + def __init__(self, name, store): + self.name = name + self.store = store + # top-level 'commited' binding + self._val = NoValue + # when updated in a 'transaction', keep track + # of our initial value (for abort cases) + self.previous = None + self.changed = False + + def is_bound(self): + return not isinstance(self.val, EqSet) \ + and self.val != NoValue + + def commit(self): + self.changed = False + + def abort(self): + self.val = self.previous + self.changed = False + + def set_val(self, val): + if self.store.in_transaction: + if not self.changed: + self.previous = self._val + self.changed = True + print "in transaction, %s <- %s" % (self.name, val) + self._val = val + def get_val(self): + return self._val + val = property(get_val, set_val) + + def __str__(self): + if self.is_bound(): + return "%s = %s" % (self.name, self.val) + return "%s" % self.name + + def __repr__(self): + return self.__str__() + + def __eq__(self, thing): + return isinstance(thing, Var) \ + and self.name == thing.name + + def __hash__(self): + return self.name.__hash__() + +def var(name): + v = Var(name, _store) + _store.add_unbound(v) + return v + + From ericvrp at codespeak.net Fri Jan 20 11:14:06 2006 From: ericvrp at codespeak.net (ericvrp at codespeak.net) Date: Fri, 20 Jan 2006 11:14:06 +0100 (CET) Subject: [pypy-svn] r22466 - in pypy/dist/pypy/translator/js: . test Message-ID: <20060120101406.F284627DC4@code1.codespeak.net> Author: ericvrp Date: Fri Jan 20 11:14:04 2006 New Revision: 22466 Modified: pypy/dist/pypy/translator/js/codewriter.py pypy/dist/pypy/translator/js/database.py pypy/dist/pypy/translator/js/optimize.py pypy/dist/pypy/translator/js/opwriter.py pypy/dist/pypy/translator/js/test/test_genllvm.py pypy/dist/pypy/translator/js/test/test_typed.py Log: Fixed genjs casts, 6 more tests pass. Added optimization for char to string and int to string. Modified: pypy/dist/pypy/translator/js/codewriter.py ============================================================================== --- pypy/dist/pypy/translator/js/codewriter.py (original) +++ pypy/dist/pypy/translator/js/codewriter.py Fri Jan 20 11:14:04 2006 @@ -280,6 +280,10 @@ self.append("%(targetvar)s = 0.0 + %(fromvar)s" % locals()) elif targettype in ('bool',): self.append("%(targetvar)s = %(fromvar)s == 0" % locals()) + elif targettype in ('ubyte',): + self.append("%(targetvar)s = %(fromvar)s.charCodeAt(0)" % locals()) + elif targettype in ('sbyte',): + self.append("%(targetvar)s = String.fromCharCode(%(fromvar)s)" % locals()) else: self.comment("next line should be: %(targetvar)s = cast %(fromtype)s %(fromvar)s to %(targettype)s" % locals()) self.append("%(targetvar)s = %(fromvar)s" % locals()) Modified: pypy/dist/pypy/translator/js/database.py ============================================================================== --- pypy/dist/pypy/translator/js/database.py (original) +++ pypy/dist/pypy/translator/js/database.py Fri Jan 20 11:14:04 2006 @@ -231,12 +231,7 @@ return repr def char_to_str(self, value): - x = ord(value) - if x >= 128: - r = "cast (ubyte %s to sbyte)" % x - else: - r = str(x) - return r + return "String.fromCharCode(%s)" % ord(value) def primitive_to_str(self, type_, value): if type_ is lltype.Bool: Modified: pypy/dist/pypy/translator/js/optimize.py ============================================================================== --- pypy/dist/pypy/translator/js/optimize.py (original) +++ pypy/dist/pypy/translator/js/optimize.py Fri Jan 20 11:14:04 2006 @@ -4,6 +4,8 @@ 'll_stritem_nonneg__rpy_stringPtr_Signed', 'll_stritem__rpy_stringPtr_Signed', 'll_streq__rpy_stringPtr_rpy_stringPtr', + 'll_str__IntegerR_SignedConst_Signed', + 'll_chr2str__Char', 'll_issubclass__object_vtablePtr_object_vtablePtr' #TODO ] @@ -34,6 +36,12 @@ return '%s = (%s == %s) || (%s && %s && %s.chars == %s.chars)' %\ (targetvar, s0,s1, s0,s1, s0,s1) + elif funcname == 'll_str__IntegerR_SignedConst_Signed': + return '%s = new Object({hash:0, chars:%s + ""})' % (targetvar, params[0]) + + elif funcname == 'll_chr2str__Char': + return '%s = new Object({hash:0, chars:%s})' % (targetvar, params[0]) + return '%s = %s(%s)' % (targetvar, funcname, ', '.join(params)) def optimize_filesize(filename): Modified: pypy/dist/pypy/translator/js/opwriter.py ============================================================================== --- pypy/dist/pypy/translator/js/opwriter.py (original) +++ pypy/dist/pypy/translator/js/opwriter.py Fri Jan 20 11:14:04 2006 @@ -76,6 +76,9 @@ self.block = block def write_operation(self, op): + #log(str(op)) + #self.codewriter.comment(str(op)) + invoke = op.opname.startswith('invoke:') if invoke: self.invoke(op) @@ -158,14 +161,8 @@ name = self.char_operations[op.opname] assert len(op.args) == 2 res = self.db.repr_arg(op.result) - if True: - c1 = self.db.repr_arg(op.args[0]) - c2 = self.db.repr_arg(op.args[1]) - else: - c1 = self.db.repr_tmpvar() - c2 = self.db.repr_tmpvar() - self.codewriter.cast(c1, "sbyte", self.db.repr_arg(op.args[0]), "ubyte") - self.codewriter.cast(c2, "sbyte", self.db.repr_arg(op.args[1]), "ubyte") + c1 = self.db.repr_arg(op.args[0]) + c2 = self.db.repr_arg(op.args[1]) self.codewriter.binaryop(name, res, c1, c2) def cast_char_to_int(self, op): Modified: pypy/dist/pypy/translator/js/test/test_genllvm.py ============================================================================== --- pypy/dist/pypy/translator/js/test/test_genllvm.py (original) +++ pypy/dist/pypy/translator/js/test/test_genllvm.py Fri Jan 20 11:14:04 2006 @@ -288,7 +288,7 @@ f = compile_function(string_simple, [int]) assert f(0) -def DONTtest_string_simple_ops(): #issue with casts +def test_string_simple_ops(): def string_simple_ops(i): res = 0 s = str(i) @@ -301,21 +301,21 @@ f = compile_function(string_simple_ops, [int]) assert f(5) == ord('5') + 2 -def DONTtest_string_getitem1(): #issue with cast sbyte to ubyte +def test_string_getitem1(): l = "Hello, World" def string_getitem1(i): return ord(l[i]) f = compile_function(string_getitem1, [int]) assert f(0) == ord("H") -def DONTtest_string_getitem2(): #issue with cast sbyte to ubyte +def test_string_getitem2(): def string_test(i): l = "Hello, World" return ord(l[i]) f = compile_function(string_test, [int]) assert f(0) == ord("H") -def DONTtest_list_of_string(): #issue with casts +def test_list_of_string(): a = ["hello", "world"] def string_simple(i, j, k, l): s = a[i][j] + a[k][l] Modified: pypy/dist/pypy/translator/js/test/test_typed.py ============================================================================== --- pypy/dist/pypy/translator/js/test/test_typed.py (original) +++ pypy/dist/pypy/translator/js/test/test_typed.py Fri Jan 20 11:14:04 2006 @@ -75,7 +75,7 @@ result = fn() assert result == 4 -def DONTtest_str_compare(): #issue with skipped/incorrect cast sbyte -> ubyte +def test_str_compare(): def testfn_eq(i, j): s1 = ['one', 'two'] s2 = ['one', 'two', 'o', 'on', 'twos', 'foobar'] @@ -305,7 +305,7 @@ for ii in range(255): assert f(ii) == ii -def DONTtest_char_comparisons(): #issue with illegal cast syntax +def test_char_comparisons(): def comps(v): x = chr(v) res = 0 From mwh at codespeak.net Fri Jan 20 12:11:51 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Fri, 20 Jan 2006 12:11:51 +0100 (CET) Subject: [pypy-svn] r22467 - in pypy/dist/pypy/rpython/memory: . test Message-ID: <20060120111151.7BEF927DCC@code1.codespeak.net> Author: mwh Date: Fri Jan 20 12:11:50 2006 New Revision: 22467 Modified: pypy/dist/pypy/rpython/memory/lladdress.py pypy/dist/pypy/rpython/memory/test/test_address.py Log: check that you can actually _call_ these offset manipulations. implement offset addition, in a very type-anal way. Modified: pypy/dist/pypy/rpython/memory/lladdress.py ============================================================================== --- pypy/dist/pypy/rpython/memory/lladdress.py (original) +++ pypy/dist/pypy/rpython/memory/lladdress.py Fri Jan 20 12:11:50 2006 @@ -126,9 +126,9 @@ class OffsetOf(Symbolic): - def __init__(self, TYPE, fldname): + def __init__(self, TYPE, *fldnames): self.TYPE = TYPE - self.fldname = fldname + self.fldnames = fldnames def annotation(self): from pypy.annotation import model @@ -138,9 +138,18 @@ return Offset def __repr__(self): - return "" % (self.TYPE, self.fldname) + return "" % (self.TYPE, self.fldnames) -Offset = lltype.Primitive("Offset", OffsetOf(lltype.Void, '')) + def __add__(self, other): + if not isinstance(other, OffsetOf): + return NotImplemented + t = self.TYPE + for f in self.fldnames: + t = t._flds[f] + assert t == other.TYPE + return OffsetOf(self.TYPE, *(self.fldnames + other.fldnames)) + +Offset = lltype.Primitive("Offset", OffsetOf(lltype.Void)) def sizeof(TYPE, n=None): pass Modified: pypy/dist/pypy/rpython/memory/test/test_address.py ============================================================================== --- pypy/dist/pypy/rpython/memory/test/test_address.py (original) +++ pypy/dist/pypy/rpython/memory/test/test_address.py Fri Jan 20 12:11:50 2006 @@ -117,17 +117,41 @@ from pypy.rpython.memory.lladdress import offsetof S = lltype.GcStruct('S', ('x', lltype.Bool), ('y', lltype.Signed)) def f(): - return offsetof(S, 'x') + offsetof(S, 'y') + return offsetof(S, 'x') + f() a = RPythonAnnotator() s = a.build_types(f, []) assert isinstance(s, annmodel.SomeOffset) + coff = offsetof(S, 'y') def f(): return coff + f() + a = RPythonAnnotator() + s = a.build_types(f, []) + assert isinstance(s, annmodel.SomeOffset) + + def test_offset_addition(self): + from pypy.rpython.lltypesystem import lltype + from pypy.rpython.memory.lladdress import offsetof + S = lltype.Struct('S', ('x', lltype.Bool), ('y', lltype.Signed)) + T = lltype.GcStruct('T', ('r', lltype.Float), ('s1', S), ('s2', S)) + def f(): + return offsetof(T, 's1') + offsetof(S, 'x') + f() a = RPythonAnnotator() s = a.build_types(f, []) assert isinstance(s, annmodel.SomeOffset) + coff = offsetof(T, 's2') + offsetof(S, 'y') + def f(): + return coff + f() + a = RPythonAnnotator() + s = a.build_types(f, []) + assert isinstance(s, annmodel.SomeOffset) + + class TestAddressRTyping(object): def test_null(self): def f(): @@ -223,7 +247,8 @@ from pypy.rpython.memory.lladdress import offsetof S = lltype.GcStruct('S', ('x', lltype.Bool), ('y', lltype.Signed)) def f(): - return offsetof(S, 'x') + offsetof(S, 'y') + return offsetof(S, 'x') + f() a = RPythonAnnotator() s = a.build_types(f, []) rtyper = RPythonTyper(a) @@ -232,6 +257,29 @@ coff = offsetof(S, 'y') def f(): return coff + f() + a = RPythonAnnotator() + s = a.build_types(f, []) + rtyper = RPythonTyper(a) + rtyper.specialize() #does not raise + + def test_offset_addition(self): + from pypy.rpython.lltypesystem import lltype + from pypy.rpython.memory.lladdress import offsetof + S = lltype.Struct('S', ('x', lltype.Bool), ('y', lltype.Signed)) + T = lltype.GcStruct('T', ('r', lltype.Float), ('s1', S), ('s2', S)) + def f(): + return offsetof(T, 's1') + offsetof(S, 'x') + f() + a = RPythonAnnotator() + s = a.build_types(f, []) + rtyper = RPythonTyper(a) + rtyper.specialize() #does not raise + + coff = offsetof(T, 's2') + offsetof(S, 'y') + def f(): + return coff + f() a = RPythonAnnotator() s = a.build_types(f, []) rtyper = RPythonTyper(a) From auc at codespeak.net Fri Jan 20 17:17:28 2006 From: auc at codespeak.net (auc at codespeak.net) Date: Fri, 20 Jan 2006 17:17:28 +0100 (CET) Subject: [pypy-svn] r22470 - pypy/dist/pypy/lib/logic Message-ID: <20060120161728.7D5A627DCC@code1.codespeak.net> Author: auc Date: Fri Jan 20 17:17:25 2006 New Revision: 22470 Added: pypy/dist/pypy/lib/logic/test_variable.py Modified: pypy/dist/pypy/lib/logic/unification.py pypy/dist/pypy/lib/logic/variable.py Log: add blocking read primitive Added: pypy/dist/pypy/lib/logic/test_variable.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/lib/logic/test_variable.py Fri Jan 20 17:17:25 2006 @@ -0,0 +1,56 @@ +import unification as u +from threading import Thread + +class Consumer(Thread): + + def give_var(self, var): + self.var = var + + def run(self): + val = self.var.get() + +class NConsumer(Thread): + + def give_vars(self, vars_): + self.vars = vars_ + + def run(self): + val = [var.get() for var in self.vars] + +class TestVariable: + + def setup_method(self, meth): + u._store = u.Store() + + def test_one_thread_reading_one_var(self): + cons = Consumer() + x = u.var('x') + cons.give_var(x) + cons.start() + u.bind(x, 42) + cons.join() + assert cons.var.val == 42 + + def test_many_threads_reading_one_var(self): + conss = [Consumer() for i in range(10)] + x = u.var('x') + for cons in conss: + cons.give_var(x) + cons.start() + u.bind(x, 42) + for cons in conss: + cons.join() + assert cons.var.val == 42 + + def test_many_thread_reading_many_var(self): + conss = [NConsumer() for i in range(10)] + vars_ = [u.var(str(i)) for i in range(10)] + for cons in conss: + cons.give_vars(vars_) + cons.start() + for var in vars_: + u.bind(var, var.name) + for cons in conss: + cons.join() + for i in range(10): + assert vars_[i].val == str(i) Modified: pypy/dist/pypy/lib/logic/unification.py ============================================================================== --- pypy/dist/pypy/lib/logic/unification.py (original) +++ pypy/dist/pypy/lib/logic/unification.py Fri Jan 20 17:17:25 2006 @@ -178,14 +178,14 @@ if _both_are_vars(var, val): if _both_are_bound(var, val): raise AlreadyBound(var.name) - if var.is_bound(): # 2b. var is bound, not var + if var._is_bound(): # 2b. var is bound, not var self.bind(val, var) - elif val.is_bound(): # 2a.val is bound, not val + elif val._is_bound(): # 2a.val is bound, not val self._bind(var.val, val.val) else: # 1. both are unbound self._merge(var.val, val.val) else: # 3. val is really a value - if var.is_bound(): + if var._is_bound(): raise AlreadyBound(var.name) self._bind(var.val, val) @@ -213,11 +213,11 @@ self._really_unify(x, y) for var in self.vars: if var.changed: - var.commit() + var._commit() except: for var in self.vars: if var.changed: - var.abort() + var._abort() raise finally: self.in_transaction = False @@ -242,7 +242,7 @@ self.bind(y,x) def _unify_var_val(self, x, y): - if x.is_bound(): raise UnificationFailure(x, y) + if x._is_bound(): raise UnificationFailure(x, y) if x != y: self.bind(x, y) @@ -330,7 +330,7 @@ return isinstance(v1, Var) and isinstance(v2, Var) def _both_are_bound(v1, v2): - return v1.is_bound() and v2.is_bound() + return v1._is_bound() and v2._is_bound() #-- #-- the global store @@ -349,10 +349,9 @@ return _store.unify(var1, var2) def bound(): - return _store.bound.keys() + return map(_store.vars, + lambda v: v.is_bound()) def unbound(): - res = [] - for cluster in _store.unbound: - res.append('='.join([str(var) for var in cluster])) - return res + return map(_store.vars, + lambda v: not v.is_bound()) Modified: pypy/dist/pypy/lib/logic/variable.py ============================================================================== --- pypy/dist/pypy/lib/logic/variable.py (original) +++ pypy/dist/pypy/lib/logic/variable.py Fri Jan 20 17:17:25 2006 @@ -1,3 +1,7 @@ +# First cut at representing Oz dataflow variable + +import threading + #----------- Exceptions --------------------------------- class VariableException(Exception): def __init__(self, name): @@ -10,7 +14,12 @@ #----------- Variables ---------------------------------- class EqSet(set): """An equivalence set for variables""" - pass + +## def __str__(self): +## if len(self) == 0: +## return '' +## for var in self: +## '='.join(var.name) class NoValue: pass @@ -26,28 +35,39 @@ # of our initial value (for abort cases) self.previous = None self.changed = False + # a condition variable for concurrent access + self.mutex = threading.Lock() + self.value_condition = threading.Condition(self.mutex) - def is_bound(self): - return not isinstance(self.val, EqSet) \ - and self.val != NoValue + # for consumption by the global store - def commit(self): + def _is_bound(self): + return not isinstance(self._val, EqSet) \ + and self._val != NoValue + + # 'transaction' support + + def _commit(self): self.changed = False - def abort(self): + def _abort(self): self.val = self.previous self.changed = False - def set_val(self, val): + # value accessors + def _set_val(self, val): + self.value_condition.acquire() if self.store.in_transaction: if not self.changed: self.previous = self._val self.changed = True - print "in transaction, %s <- %s" % (self.name, val) self._val = val - def get_val(self): + self.value_condition.notifyAll() + self.value_condition.release() + + def _get_val(self): return self._val - val = property(get_val, set_val) + val = property(_get_val, _set_val) def __str__(self): if self.is_bound(): @@ -64,9 +84,21 @@ def __hash__(self): return self.name.__hash__() -def var(name): - v = Var(name, _store) - _store.add_unbound(v) - return v - + #---- Concurrent public ops -------------------------- + def is_bound(self): + self.mutex.acquire() + res = self._is_bound() + self.mutex.release() + return res + + # should be used by threads that want to block on + # unbound variables + def get(self): + try: + self.value_condition.acquire() + while not self._is_bound(): + self.value_condition.wait() + return self.val + finally: + self.value_condition.release() From ericvrp at codespeak.net Sat Jan 21 10:07:33 2006 From: ericvrp at codespeak.net (ericvrp at codespeak.net) Date: Sat, 21 Jan 2006 10:07:33 +0100 (CET) Subject: [pypy-svn] r22474 - in pypy/dist/pypy/translator/js: . test Message-ID: <20060121090733.C43DF27DCC@code1.codespeak.net> Author: ericvrp Date: Sat Jan 21 10:07:32 2006 New Revision: 22474 Modified: pypy/dist/pypy/translator/js/codewriter.py pypy/dist/pypy/translator/js/funcnode.py pypy/dist/pypy/translator/js/optimize.py pypy/dist/pypy/translator/js/test/test_merge_if_blocks.py pypy/dist/pypy/translator/js/test/test_stackless.py Log: * Support for merge if blocks. * Cleanup stackless output when calling native javascript function. Modified: pypy/dist/pypy/translator/js/codewriter.py ============================================================================== --- pypy/dist/pypy/translator/js/codewriter.py (original) +++ pypy/dist/pypy/translator/js/codewriter.py Sat Jan 21 10:07:32 2006 @@ -134,12 +134,14 @@ self.append('}') self.skip_closeblock() - def switch(self, cond, defaultdest, value_labels): - if not defaultdest: - raise TypeError('switches must have a default case.') - labels = ','.join(["'%s':%s" % (value, label) for value, label in value_labels]) - #XXX for performance this Object should become global - self.append("if (!(block = {%s}[%s])) block = %s" % (labels, cond, defaultdest)) + def switch(self, cond, mapping, default_code): + #XXX the performance of this code can be improved by: + # 1. not using eval + # 2. storing the object in a global variable + labels = ','.join(["%s:%s" % (exitcase, code) for exitcase, code in mapping]) + self.append("var switch_code = {%s}[%s]" % (labels, cond)) + self.append("if (!switch_code) switch_code=%s" % default_code) + self.append('eval(switch_code)') def openfunc(self, decl, funcnode, blocks): self.decl = decl @@ -202,10 +204,10 @@ if not exceptions: assert no_exception is None - if self.js.stackless: + optimized, code = optimize_call('%s = %s(%s)' % (targetvar, functionref, args)) + if self.js.stackless and not optimized: self.append("slp_stack_depth++") - self.append( optimize_call('%s = %s(%s)' % (targetvar, functionref, args)) ) - if self.js.stackless: + self.append(code) self.append("slp_stack_depth--") selfdecl = self.decl.split('(')[0] usedvars = ', '.join(self._usedvars.keys()) @@ -224,17 +226,21 @@ self.append('case %d:' % self._resume_blocknum) self.indent_more() self._resume_blocknum += 1 + else: #not stackless + self.append(code) else: assert no_exception is not None no_exception_label, no_exception_exit = no_exception self.append('try {') self.indent_more() - if self.js.stackless: + optimized, code = optimize_call('%s = %s(%s)' % (targetvar, functionref, args)) + if self.js.stackless and not optimized: self.comment('TODO: XXX stackless in combination with exceptions handling') self.append("slp_stack_depth++") - self.append( optimize_call('%s = %s(%s)' % (targetvar, functionref, args)) ) - if self.js.stackless: + self.append(code) self.append("slp_stack_depth--") #XXX we don't actually get here when an exception occurs! + else: + self.append(code) self._phi(no_exception_exit) self._goto_block(no_exception_label) self.indent_less() Modified: pypy/dist/pypy/translator/js/funcnode.py ============================================================================== --- pypy/dist/pypy/translator/js/funcnode.py (original) +++ pypy/dist/pypy/translator/js/funcnode.py Sat Jan 21 10:07:32 2006 @@ -99,7 +99,7 @@ (lltype.Signed, lltype.Unsigned, lltype.SignedLongLong, lltype.UnsignedLongLong, lltype.Char, lltype.UniChar): defaultlink = None - value_labels = [] + mapping = [] for link in block.exits: if link.exitcase is 'default': defaultlink = link @@ -108,10 +108,30 @@ exitcase = link.llexitcase if block.exitswitch.concretetype in [lltype.Char, lltype.UniChar]: exitcase = ord(exitcase) - value_labels.append( (exitcase, - self.blockindex[link.target]) ) + exitcase = "'%s'" % exitcase - codewriter.switch(cond, self.blockindex[defaultlink.target], value_labels) + try: + targetvar = link.target.inputargs[0] + targetvar_value = self.db.repr_arg(link.args[0]) + s = "%s=%s;" % (targetvar, targetvar_value) + except: + s = '' + targetblock = self.blockindex[link.target] + + code = "'%sblock=%s'" % (s, targetblock) + mapping.append( (exitcase, code) ) + + try: + default_targetvar = defaultlink.target.inputargs[0] + default_targetvar_value = self.db.repr_arg(defaultlink.args[0]) + s = "%s=%s;" % (default_targetvar, default_targetvar_value) + except: + s = '' + default_targetblock = self.blockindex[defaultlink.target] + default_code = "'%sblock=%s'" % (s, default_targetblock) + if block.exitswitch.concretetype == lltype.Char: + cond = "%s.charCodeAt(0)" % cond + codewriter.switch(cond, mapping, default_code) else: raise BranchException("exitswitch type '%s' not supported" % Modified: pypy/dist/pypy/translator/js/optimize.py ============================================================================== --- pypy/dist/pypy/translator/js/optimize.py (original) +++ pypy/dist/pypy/translator/js/optimize.py Sat Jan 21 10:07:32 2006 @@ -6,7 +6,8 @@ 'll_streq__rpy_stringPtr_rpy_stringPtr', 'll_str__IntegerR_SignedConst_Signed', 'll_chr2str__Char', - 'll_issubclass__object_vtablePtr_object_vtablePtr' #TODO + 'll_issubclass__object_vtablePtr_object_vtablePtr', + 'll_str__FloatR_FloatConst_Float', ] @@ -16,33 +17,34 @@ params = [param.strip() for param in params[:-1].split(',')] if funcname == 'll_strlen__rpy_stringPtr': - return '%s = %s.chars.length' % (targetvar, params[0]) + return True, '%s = %s.chars.length' % (targetvar, params[0]) elif funcname == 'll_strconcat__rpy_stringPtr_rpy_stringPtr': #XXX javascript of ll_strconcat__rpy_stringPtr_rpy_stringPtr actually does not work, FIX IT! # by outcommenting this code end running js/test/test_genllvm.py -k test_simple_chars p = '%s.chars' % '.chars + '.join(params) - return '%s = new Object({hash:0, chars:%s})' % (targetvar, p) + return True, '%s = new Object({hash:0, chars:%s})' % (targetvar, p) elif funcname == 'll_stritem_nonneg__rpy_stringPtr_Signed': - return '%s = %s.chars[%s]' % (targetvar, params[0], params[1]) + return True, '%s = %s.chars[%s]' % (targetvar, params[0], params[1]) elif funcname == 'll_stritem__rpy_stringPtr_Signed': s, i = params - return '%s = %s.chars[%s >= 0 ? %s : %s + %s.chars.length]' % (targetvar, s, i, i, i, s) + return True, '%s = %s.chars[%s >= 0 ? %s : %s + %s.chars.length]' % (targetvar, s, i, i, i, s) elif funcname == 'll_streq__rpy_stringPtr_rpy_stringPtr': s0, s1 = params - return '%s = (%s == %s) || (%s && %s && %s.chars == %s.chars)' %\ + return True, '%s = (%s == %s) || (%s && %s && %s.chars == %s.chars)' %\ (targetvar, s0,s1, s0,s1, s0,s1) - elif funcname == 'll_str__IntegerR_SignedConst_Signed': - return '%s = new Object({hash:0, chars:%s + ""})' % (targetvar, params[0]) + elif funcname in ('ll_str__IntegerR_SignedConst_Signed', + 'll_str__FloatR_FloatConst_Float'): + return True, '%s = new Object({hash:0, chars:%s + ""})' % (targetvar, params[0]) elif funcname == 'll_chr2str__Char': - return '%s = new Object({hash:0, chars:%s})' % (targetvar, params[0]) + return True, '%s = new Object({hash:0, chars:%s})' % (targetvar, params[0]) - return '%s = %s(%s)' % (targetvar, funcname, ', '.join(params)) + return False, '%s = %s(%s)' % (targetvar, funcname, ', '.join(params)) def optimize_filesize(filename): f = open(filename, "r") Modified: pypy/dist/pypy/translator/js/test/test_merge_if_blocks.py ============================================================================== --- pypy/dist/pypy/translator/js/test/test_merge_if_blocks.py (original) +++ pypy/dist/pypy/translator/js/test/test_merge_if_blocks.py Sat Jan 21 10:07:32 2006 @@ -15,19 +15,17 @@ assert simple(i) == merge_if_blocks_simple(i) def test_merge_if_blocks_basic(): - py.test.skip("merge_if_block failing because link exit values are not used") def merge_if_blocks_basic(i): if i == 5: return 1005 elif i == 8: return 1008 return 2222 - basic = compile_function(merge_if_blocks_basic , [int], view=True) + basic = compile_function(merge_if_blocks_basic , [int]) for i in range(-20,20): assert basic(i) == merge_if_blocks_basic(i) def test_merge_if_blocks_chr(): - py.test.skip("merge_if_block failing because link exit values are not used") def merge_if_blocks_chr(i): c = chr(i) if c == '\x05': @@ -40,7 +38,6 @@ assert basic(i) == merge_if_blocks_chr(i) def test_merge_if_blocks_uni(): - py.test.skip("merge_if_block failing because link exit values are not used") def merge_if_blocks_uni(i): c = unichr(i) if c == u'\x05': @@ -54,7 +51,6 @@ def test_merge_if_blocks_many(): - py.test.skip("merge_if_block failing because link exit values are not used") def merge_if_blocks_many(i): if i == 0: return 1000 Modified: pypy/dist/pypy/translator/js/test/test_stackless.py ============================================================================== --- pypy/dist/pypy/translator/js/test/test_stackless.py (original) +++ pypy/dist/pypy/translator/js/test/test_stackless.py Sat Jan 21 10:07:32 2006 @@ -68,8 +68,6 @@ assert data.strip() == '100' def test_stackless_arguments(): - py.test.skip("stackless feature incomplete (empty Object mallocs)") - def f(n, d, t): if n > 0: res = f(n-1, d, t) From cfbolz at codespeak.net Sat Jan 21 12:57:38 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Sat, 21 Jan 2006 12:57:38 +0100 (CET) Subject: [pypy-svn] r22477 - pypy/extradoc/sprintinfo/mallorca Message-ID: <20060121115738.8071627DC6@code1.codespeak.net> Author: cfbolz Date: Sat Jan 21 12:57:36 2006 New Revision: 22477 Modified: pypy/extradoc/sprintinfo/mallorca/people.txt Log: my dates Modified: pypy/extradoc/sprintinfo/mallorca/people.txt ============================================================================== --- pypy/extradoc/sprintinfo/mallorca/people.txt (original) +++ pypy/extradoc/sprintinfo/mallorca/people.txt Sat Jan 21 12:57:36 2006 @@ -14,7 +14,7 @@ Michael Hudson 22/1 - 30/1 Hostal Terminus Samuele Pedroni 22/1 - 30/1 Hotel Colon Eric van Riet Paap 22/1 - 28/1 casa M. -Carl Friedrich Bolz ? ? +Carl Friedrich Bolz 21/1 - 29/1 flat Anders Chrigstroem 22/1 - 30/1 Hotel Colon Christian Tismer 22/1 - 30/1 ? Gerald Klix 22/1 - 30/1 ? From ale at codespeak.net Sat Jan 21 13:06:24 2006 From: ale at codespeak.net (ale at codespeak.net) Date: Sat, 21 Jan 2006 13:06:24 +0100 (CET) Subject: [pypy-svn] r22478 - pypy/extradoc/sprintinfo/mallorca Message-ID: <20060121120624.3D50427DD7@code1.codespeak.net> Author: ale Date: Sat Jan 21 13:06:23 2006 New Revision: 22478 Modified: pypy/extradoc/sprintinfo/mallorca/people.txt Log: I cant be at the sprint. I will be sprinting with Logilab in Paris instead. Have a nice sprint. Modified: pypy/extradoc/sprintinfo/mallorca/people.txt ============================================================================== --- pypy/extradoc/sprintinfo/mallorca/people.txt (original) +++ pypy/extradoc/sprintinfo/mallorca/people.txt Sat Jan 21 13:06:23 2006 @@ -20,7 +20,6 @@ Gerald Klix 22/1 - 30/1 ? Stephan Diehl 22/1 - 28/1 ? Jacob Hallen ? ? -Anders Lehmann 22/1 -29/1 Hotel Almudainal Holger Krekel ? ? Richard Emslie 22/1 - 29/1 catalonia hotel Andrew Thompson 22/1 - 29/1 Hotel Terminus From lac at codespeak.net Sun Jan 22 02:52:27 2006 From: lac at codespeak.net (lac at codespeak.net) Date: Sun, 22 Jan 2006 02:52:27 +0100 (CET) Subject: [pypy-svn] r22485 - pypy/extradoc/pypy.org Message-ID: <20060122015227.1C50527DB6@code1.codespeak.net> Author: lac Date: Sun Jan 22 02:52:26 2006 New Revision: 22485 Modified: pypy/extradoc/pypy.org/index.txt Log: typo Modified: pypy/extradoc/pypy.org/index.txt ============================================================================== --- pypy/extradoc/pypy.org/index.txt (original) +++ pypy/extradoc/pypy.org/index.txt Sun Jan 22 02:52:26 2006 @@ -8,7 +8,7 @@ PyPy EU project description (004779) -------------------------------------- -The PyPy project have been an ongoing Open Source Python language +The PyPy project has been an ongoing Open Source Python language implementation since 2003. In December 2004 PyPy recieved EU-funding within the Framework Programme 6, second call for proposals ("Open development platforms and services" IST). From ale at codespeak.net Sun Jan 22 13:44:04 2006 From: ale at codespeak.net (ale at codespeak.net) Date: Sun, 22 Jan 2006 13:44:04 +0100 (CET) Subject: [pypy-svn] r22486 - in pypy/dist/pypy/lib/pyontology: . test Message-ID: <20060122124404.1827D27DB4@code1.codespeak.net> Author: ale Date: Sun Jan 22 13:44:03 2006 New Revision: 22486 Modified: pypy/dist/pypy/lib/pyontology/pyontology.py pypy/dist/pypy/lib/pyontology/test/test_ontology.py Log: Added a test for sameas -> changes of the sameasconstraint got rid of get_values Modified: pypy/dist/pypy/lib/pyontology/pyontology.py ============================================================================== --- pypy/dist/pypy/lib/pyontology/pyontology.py (original) +++ pypy/dist/pypy/lib/pyontology/pyontology.py Sun Jan 22 13:44:03 2006 @@ -1,6 +1,5 @@ from rdflib import Graph, URIRef, BNode, Literal from logilab.constraint import Repository, Solver -from logilab.constraint.fd import BinaryExpression from logilab.constraint.fd import FiniteDomain as fd from logilab.constraint.propagation import AbstractDomain, AbstractConstraint, ConsistencyFailure import sys @@ -279,7 +278,7 @@ self.constraints.append(cls.constraint) def first(self, s, var): - avar = self.make_var(None, var) + avar = self.make_var(ClassDomain, var) svar = self.make_var(List, s) vals = [] vals += self.variables[svar].getValues() @@ -410,16 +409,16 @@ self.constraints.append(constrain) #XXX need to change this -## def distinctMembers(self, s, var): -## res = self.get_list(var) -## self.constraints.append(AllDistinct([self.make_var(ClassDomain, y) for y in res])) -## return res + def distinctMembers(self, s, var): + s_var = self.make_var(AllDifferent, s) + var_var = self.make_var(List, var) + constrain = AllDifferentConstraint(s_var, var_var) + self.constraints.append(constrain) def sameAs(self, s, var): s_var = self.make_var(None, s) var_var = self.make_var(None, var) - constrain = BinaryExpression([s_var, var_var], - "%s == %s" %(s_var, var_var)) + constrain = SameasConstraint(s_var, var_var) self.constraints.append(constrain) def hasValue(self, s, var): @@ -510,18 +509,6 @@ else: return 1 -def get_values(dom, domains, attr = 'getValues'): - res = [] - if type(dom) == Literal: - return [dom] - for val in getattr(dom, attr)(): - res.append(val) - if type(val) == tuple: - val = val[0] - if val in domains.keys(): - res.extend(get_values(domains[val], domains, attr)) - return res - class SubClassConstraint(AbstractConstraint): cost=1 @@ -551,10 +538,10 @@ def narrow(self, domains): subdom = domains[self.variable] superdom = domains[self.object] - bases = get_values(superdom, domains, 'getBases') - subdom.bases += [bas for bas in bases if bas not in subdom.bases] - vals1 = get_values(superdom, domains, 'getValues') - vals2 = get_values(subdom, domains, 'getValues') +## bases = get_values(superdom, domains, 'getBases') +## subdom.bases += [bas for bas in bases if bas not in subdom.bases] + vals1 = superdom.getValues() + vals2 = subdom.getValues() for i in vals1: if i in vals2: raise ConsistencyFailure() @@ -592,7 +579,7 @@ def narrow(self, domains): propdom = domains[self.variable] domaindom = domains[self.object] - newdomain = get_values(domaindom, domains, 'getValues') +[self.object] + newdomain = domaindom.getValues() +[self.object] domain = [] olddomain = propdom.domain if olddomain: @@ -612,7 +599,7 @@ def narrow(self, domains): subdom = domains[self.variable] superdom = domains[self.object] - vals = get_values(superdom, domains, 'getValues') + vals = superdom.getValues() for val in subdom.getValues(): if not val in vals: vals.append(val) @@ -623,7 +610,7 @@ def narrow(self, domains): subdom = domains[self.variable] superdom = domains[self.object] - vals = get_values(superdom, domains, 'getValues') + vals = superdom.getValues() for val in subdom.getValues(): if not val in vals: raise ConsistencyFailure("Value not in prescribed range") @@ -725,6 +712,35 @@ else: return 0 +class SameasConstraint(SubClassConstraint): + + def narrow(self, domains): + if self.variable == self.object: + return 1 + else: + for dom in domains.values(): + vals = dom.getValues() + if isinstance(dom, Property): + val = Linkeddict(vals) + if self.variable in val.keys() and not self.object in val.keys(): + vals +=[(self.object,v) for v in val[self.variable]] + dom.setValues(vals) + elif not self.variable in val.keys() and self.object in val.keys(): + vals +=[(self.variable,v) for v in val[self.object]] + dom.setValues(vals) + elif self.variable in val.keys() and self.object in val.keys(): + if not val[self.object] == val[self.variable]: + raise ConsistencyFailure("Sameas failure: The two individuals (%s, %s) \ + has different values for property %r"%(self.variable, self.object, dom)) + else: + if self.variable in vals and not self.object in vals: + vals.append(self.object) + elif not self.variable in vals and self.object in vals: + vals.append(self.variable) + else: + continue + dom.setValues(vals) + return 0 class ListConstraint(OwlConstraint): """Contraint: all values must be distinct""" @@ -747,7 +763,7 @@ class RestrictionConstraint(OwlConstraint): - cost = 90 + cost = 70 def narrow(self, domains): prop = domains[self.variable].property Modified: pypy/dist/pypy/lib/pyontology/test/test_ontology.py ============================================================================== --- pypy/dist/pypy/lib/pyontology/test/test_ontology.py (original) +++ pypy/dist/pypy/lib/pyontology/test/test_ontology.py Sun Jan 22 13:44:03 2006 @@ -116,7 +116,6 @@ def test_ObjectProperty(): sub = URIRef('a') - pred = URIRef('type') obj = URIRef(namespaces['owl']+'#ObjectProperty') O = Ontology() O.type(sub, obj) @@ -478,3 +477,39 @@ O.differentFrom(cls, cls) O.type(cls, namespaces['owl']+'#Thing') py.test.raises(ConsistencyFailure, O.consistency, 3) + +def test_sameas(): + O = Ontology() + cls = BNode('anon') + own1 = BNode('liist1') + own2 = BNode('liist2') + O.sameAs(cls, own1) + O.sameAs(own1, own2) + O.sameAs(cls, own2) + O.type(cls, namespaces['owl']+'#Thing') + O.type(own1, namespaces['owl']+'#Thing') + O.type(own2, namespaces['owl']+'#Thing') + sub = URIRef('a') + obj = URIRef(namespaces['owl']+'#ObjectProperty') + O.type(sub, obj) + O.variables[O.make_var(None,sub)].setValues([(cls,'1')]) + O.consistency(3) +# assert len(O.rep._constraints) == 4 + assert ('_liist1','1') in O.rep._domains[O.make_var(None,sub)].getValues() + +def test_sameasconsistency(): + O = Ontology() + cls = BNode('anon') + own1 = BNode('liist1') + O.sameAs(cls, own1) + O.type(cls, namespaces['owl']+'#Thing') + O.type(own1, namespaces['owl']+'#Thing') + sub = URIRef('a') + obj = URIRef(namespaces['owl']+'#ObjectProperty') + O.type(sub, obj) + O.variables[O.make_var(None,sub)].setValues([(cls,'1'), (own1,'2')]) + print O.variables + for dom in O.variables.values() :print type(dom) + py.test.raises(ConsistencyFailure, O.consistency, 3) + + From ale at codespeak.net Mon Jan 23 08:28:39 2006 From: ale at codespeak.net (ale at codespeak.net) Date: Mon, 23 Jan 2006 08:28:39 +0100 (CET) Subject: [pypy-svn] r22495 - pypy/extradoc/sprintinfo/mallorca Message-ID: <20060123072839.AB95927DC7@code1.codespeak.net> Author: ale Date: Mon Jan 23 08:28:37 2006 New Revision: 22495 Added: pypy/extradoc/sprintinfo/mallorca/Paris_mini_sprint.txt Log: (adim, nico, ale) Some topics that we want to consider during the minisprint in Paris. Added: pypy/extradoc/sprintinfo/mallorca/Paris_mini_sprint.txt ============================================================================== --- (empty file) +++ pypy/extradoc/sprintinfo/mallorca/Paris_mini_sprint.txt Mon Jan 23 08:28:37 2006 @@ -0,0 +1,14 @@ +Mini Sprint at Logilab +----------------- + +Possible topics +------------ + + + Finish exposing the Grammar at application level + + Work on the constraint solving engine + Make an application level package without scheduling (awaits + exposing of stackless features at applevel) + + From ale at codespeak.net Mon Jan 23 09:18:47 2006 From: ale at codespeak.net (ale at codespeak.net) Date: Mon, 23 Jan 2006 09:18:47 +0100 (CET) Subject: [pypy-svn] r22496 - pypy/extradoc/sprintinfo/mallorca Message-ID: <20060123081847.E74B127DBD@code1.codespeak.net> Author: ale Date: Mon Jan 23 09:18:46 2006 New Revision: 22496 Modified: pypy/extradoc/sprintinfo/mallorca/Paris_mini_sprint.txt Log: (auc, adim, nico, ale) Detailing the goals for the week Modified: pypy/extradoc/sprintinfo/mallorca/Paris_mini_sprint.txt ============================================================================== --- pypy/extradoc/sprintinfo/mallorca/Paris_mini_sprint.txt (original) +++ pypy/extradoc/sprintinfo/mallorca/Paris_mini_sprint.txt Mon Jan 23 09:18:46 2006 @@ -8,7 +8,11 @@ Finish exposing the Grammar at application level Work on the constraint solving engine - Make an application level package without scheduling (awaits - exposing of stackless features at applevel) + + The goal of the week is to have a first version of a Oz like "computation space". + This means that we need to advance the "store" already in the repository, and make + the current implementation of "variables" safe for concurrency. + + Write a document detailing the goals for WP10 (Aspects and contracts) From auc at codespeak.net Mon Jan 23 10:12:35 2006 From: auc at codespeak.net (auc at codespeak.net) Date: Mon, 23 Jan 2006 10:12:35 +0100 (CET) Subject: [pypy-svn] r22497 - pypy/dist/pypy/lib/logic Message-ID: <20060123091235.875A427DCC@code1.codespeak.net> Author: auc Date: Mon Jan 23 10:12:33 2006 New Revision: 22497 Modified: pypy/dist/pypy/lib/logic/test_unification.py pypy/dist/pypy/lib/logic/test_variable.py pypy/dist/pypy/lib/logic/unification.py pypy/dist/pypy/lib/logic/variable.py Log: checks against same variable creation Modified: pypy/dist/pypy/lib/logic/test_unification.py ============================================================================== --- pypy/dist/pypy/lib/logic/test_unification.py (original) +++ pypy/dist/pypy/lib/logic/test_unification.py Mon Jan 23 10:12:33 2006 @@ -1,14 +1,16 @@ import unification as u +import variable as v from py.test import raises, skip +from threading import Thread class TestUnification: def setup_method(self, meth): u._store = u.Store() - def test_already_in_store(self): - x = u.var('x') - raises(u.AlreadyInStore, u.var, 'x') +## def test_already_in_store(self): +## x = u.var('x') +## raises(u.AlreadyInStore, u.var, 'x') def test_already_bound(self): x = u.var('x') @@ -102,3 +104,26 @@ assert b.val == {1:a, 2:42} + def test_threads_creating_vars(self): + def create_var(*args): + x = u.var('x') + + def create_var2(*args): + raises(v.AlreadyExists, u.var, 'x') + + t1, t2 = (FunThread(create_var), + FunThread(create_var2)) + t1.start() + t2.start() + + +class FunThread(Thread): + + def __init__(self, fun, *args): + Thread.__init__(self) + self.fun = fun + self.args = args + + def run(self): + self.fun(self.args) + Modified: pypy/dist/pypy/lib/logic/test_variable.py ============================================================================== --- pypy/dist/pypy/lib/logic/test_variable.py (original) +++ pypy/dist/pypy/lib/logic/test_variable.py Mon Jan 23 10:12:33 2006 @@ -1,4 +1,6 @@ +from py.test import raises import unification as u +import variable as v from threading import Thread class Consumer(Thread): @@ -22,6 +24,11 @@ def setup_method(self, meth): u._store = u.Store() + +## def test_no_same_name(self): +## x = u.var('x') +## raises(v.AlreadyExists, u.var, 'x') + def test_one_thread_reading_one_var(self): cons = Consumer() x = u.var('x') Modified: pypy/dist/pypy/lib/logic/unification.py ============================================================================== --- pypy/dist/pypy/lib/logic/unification.py (original) +++ pypy/dist/pypy/lib/logic/unification.py Mon Jan 23 10:12:33 2006 @@ -10,7 +10,7 @@ ## Unification in Oz (almost verbatim from CTM*) ## ============================================= ## -## Unification as put in Oz is a very powerful operation. It is +## Unification as put in Oz is "a very powerful operation". It is ## provided through the '=' operator. ## ## Binding a variable to a value is a special case of an operation @@ -152,8 +152,9 @@ (also called determined variables).""" def __init__(self): - # set of all known vars + # mapping of names to vars (all of them) self.vars = set() + self.names = set() self.in_transaction = False def add_unbound(self, var): @@ -162,6 +163,7 @@ raise AlreadyInStore(var.name) print "adding %s to the store" % var self.vars.add(var) + self.names.add(var.name) # put into new singleton equiv. set var.val = EqSet([var]) Modified: pypy/dist/pypy/lib/logic/variable.py ============================================================================== --- pypy/dist/pypy/lib/logic/variable.py (original) +++ pypy/dist/pypy/lib/logic/variable.py Mon Jan 23 10:12:33 2006 @@ -11,6 +11,10 @@ def __str__(self): return "%s is not a variable" % self.name +class AlreadyExists(VariableException): + def __str__(self): + return "%s already exists" % self.name + #----------- Variables ---------------------------------- class EqSet(set): """An equivalence set for variables""" @@ -27,6 +31,8 @@ class Var(object): def __init__(self, name, store): + if name in store.names: + raise AlreadyExists(name) self.name = name self.store = store # top-level 'commited' binding From cfbolz at codespeak.net Mon Jan 23 10:54:24 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Mon, 23 Jan 2006 10:54:24 +0100 (CET) Subject: [pypy-svn] r22499 - pypy/extradoc/sprintinfo/mallorca Message-ID: <20060123095424.6F71727DBD@code1.codespeak.net> Author: cfbolz Date: Mon Jan 23 10:54:23 2006 New Revision: 22499 Added: pypy/extradoc/sprintinfo/mallorca/planning.txt Log: (all present): results of the first planning session Added: pypy/extradoc/sprintinfo/mallorca/planning.txt ============================================================================== --- (empty file) +++ pypy/extradoc/sprintinfo/mallorca/planning.txt Mon Jan 23 10:54:23 2006 @@ -0,0 +1,63 @@ +================== +Mallorca planning +================== + + +People present +============================= + +Armin Rigo +Michael Hudson +Samuele Pedroni +Eric van Riet Paap +Carl Friedrich Bolz +Anders Chrigstroem +Christian Tismer +Gerald Klix +Stephan Diehl +Richard Emslie +Beatrice Duering + +People supposedly present +========================= + +Andrew Thompson +Jacob Hallen + + +Tasks +====== + + - JIT: Armin + + - stackless application level interface + + - l3interp/offsetof/sizeof work: Michael, Arre, (Samuele) + + - integration of the garbage collection toolkit with the + offsetof work done: Carl Friedrich, Richard + + - improving the way we interface with external functions. + (also think about how exceptions fit in there) + + - tutorial about the translator: Armin, Gerald, Stephan, + Eric, Christian + + +later: +====== + + - our threading implementation is still incredibly + slow, we need to work on that. + + - refactor argument handling and calls in the object space + + - look into how the LLVM JIT works + + - transformations to be more clever with checking for + exceptions + + - socket module + already improved but still far from complete + + From rxe at codespeak.net Mon Jan 23 11:12:57 2006 From: rxe at codespeak.net (rxe at codespeak.net) Date: Mon, 23 Jan 2006 11:12:57 +0100 (CET) Subject: [pypy-svn] r22500 - pypy/dist/pypy/doc Message-ID: <20060123101257.E991427DCE@code1.codespeak.net> Author: rxe Date: Mon Jan 23 11:12:56 2006 New Revision: 22500 Modified: pypy/dist/pypy/doc/garbage_collection.txt Log: Typso Modified: pypy/dist/pypy/doc/garbage_collection.txt ============================================================================== --- pypy/dist/pypy/doc/garbage_collection.txt (original) +++ pypy/dist/pypy/doc/garbage_collection.txt Mon Jan 23 11:12:56 2006 @@ -74,7 +74,7 @@ reads a signed integer from the point in memory that is ``index`` signed integers distant from the ``addr`` - ``addr.signed[offset] = value``: + ``addr.signed[index] = value``: writes the signed integer ``value`` to the point in memory that is ``index`` signed integers distant from the ``addr`` @@ -146,7 +146,7 @@ the type ``varsize_offset_to_length(typeid)`` --> offset: - returns _the offset to the length (number of items) of the variable + returns the offset to the length (number of items) of the variable sized type ``varsize_offsets_to_gcpointers_in_var_part(typeid)`` --> list of offsets: @@ -307,7 +307,7 @@ Deferred reference counting is a reference counting algorithm that tries to reduce the overhead of reference counting at the expense of the immediacy of a regular refcounting implementation. To achieve this the refcounts of an -objects count only the references from other objects on the heap, not +object only count the references from other objects on the heap, not references from the stack. If the refcount of an object reaches zero it can not be freed immediately (since it could be reachable from the stack). Instead it is added to a zero-refcount-list. When collection occurs the refcounts of From auc at codespeak.net Mon Jan 23 11:30:56 2006 From: auc at codespeak.net (auc at codespeak.net) Date: Mon, 23 Jan 2006 11:30:56 +0100 (CET) Subject: [pypy-svn] r22501 - pypy/dist/pypy/lib/logic Message-ID: <20060123103056.90E1927DCC@code1.codespeak.net> Author: auc Date: Mon Jan 23 11:30:54 2006 New Revision: 22501 Modified: pypy/dist/pypy/lib/logic/test_unification.py pypy/dist/pypy/lib/logic/unification.py Log: check concurrent binding (auc, ale) Modified: pypy/dist/pypy/lib/logic/test_unification.py ============================================================================== --- pypy/dist/pypy/lib/logic/test_unification.py (original) +++ pypy/dist/pypy/lib/logic/test_unification.py Mon Jan 23 11:30:54 2006 @@ -3,14 +3,24 @@ from py.test import raises, skip from threading import Thread +class FunThread(Thread): + + def __init__(self, fun, *args): + Thread.__init__(self) + self.fun = fun + self.args = args + + def run(self): + self.fun(self, *self.args) + class TestUnification: def setup_method(self, meth): u._store = u.Store() -## def test_already_in_store(self): -## x = u.var('x') -## raises(u.AlreadyInStore, u.var, 'x') + def test_already_in_store(self): + x = u.var('x') + raises(v.AlreadyExists, u.var, 'x') def test_already_bound(self): x = u.var('x') @@ -105,25 +115,50 @@ def test_threads_creating_vars(self): - def create_var(*args): + def create_var(thread, *args): x = u.var('x') - def create_var2(*args): + def create_var2(thread, *args): raises(v.AlreadyExists, u.var, 'x') t1, t2 = (FunThread(create_var), FunThread(create_var2)) t1.start() t2.start() + + + def test_threads_binding_vars(self): + def do_stuff(thread, var, val): + thread.raised = False + try: + u.bind(var, val) + except u.AlreadyBound: + thread.raised = True + + x = u.var('x') + vars_ = [] + for nvar in range(1000): + v = u.var('x-'+str(nvar)) + u.bind(x, v) + vars_.append(v) + + for var in u._store.vars: + assert var.val == x.val + + t1, t2 = (FunThread(do_stuff, x, 42), + FunThread(do_stuff, x, 43)) + t1.start() + t2.start() + t1.join() + t2.join() + #check that every var is really bound to 42 or 43 + for var in u._store.vars: + assert var.val == x.val + assert (t2.raised and not t1.raised) or \ + (t1.raised and not t2.raised) + -class FunThread(Thread): - def __init__(self, fun, *args): - Thread.__init__(self) - self.fun = fun - self.args = args - def run(self): - self.fun(self.args) Modified: pypy/dist/pypy/lib/logic/unification.py ============================================================================== --- pypy/dist/pypy/lib/logic/unification.py (original) +++ pypy/dist/pypy/lib/logic/unification.py Mon Jan 23 11:30:54 2006 @@ -118,6 +118,8 @@ # * add entailment checks # * add constraints support +import threading + from variable import EqSet, Var, VariableException, NotAVariable #----------- Store Exceptions ---------------------------- @@ -156,6 +158,7 @@ self.vars = set() self.names = set() self.in_transaction = False + self.lock = threading.Lock() def add_unbound(self, var): # register globally @@ -174,6 +177,7 @@ 2. (unbound)Variable/(bound)Variable or 3. (unbound)Variable/Value binding """ + self.lock.acquire() assert(isinstance(var, Var) and (var in self.vars)) if var == val: return @@ -190,16 +194,17 @@ if var._is_bound(): raise AlreadyBound(var.name) self._bind(var.val, val) + self.lock.release() def _bind(self, eqs, val): - print "variable - value binding : %s %s" % (eqs, val) + # print "variable - value binding : %s %s" % (eqs, val) # bind all vars in the eqset to obj for var in eqs: var.val = val def _merge(self, eqs1, eqs2): - print "unbound variables binding : %s %s" % (eqs1, eqs2) + # print "unbound variables binding : %s %s" % (eqs1, eqs2) if eqs1 == eqs2: return # merge two equisets into one eqs1 |= eqs2 From mwh at codespeak.net Mon Jan 23 12:14:44 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Mon, 23 Jan 2006 12:14:44 +0100 (CET) Subject: [pypy-svn] r22508 - in pypy/dist/pypy/rpython: l3interp l3interp/test memory Message-ID: <20060123111444.73F9327DDD@code1.codespeak.net> Author: mwh Date: Mon Jan 23 12:14:39 2006 New Revision: 22508 Modified: pypy/dist/pypy/rpython/l3interp/convertgraph.py pypy/dist/pypy/rpython/l3interp/l3interp.py pypy/dist/pypy/rpython/l3interp/model.py pypy/dist/pypy/rpython/l3interp/test/test_convert.py pypy/dist/pypy/rpython/memory/lladdress.py Log: (arre, mwh) A test that getfield of an int on a constant struct can be converted to an L3 graph that can be l3interped -- and almost nothing else :) Modified: pypy/dist/pypy/rpython/l3interp/convertgraph.py ============================================================================== --- pypy/dist/pypy/rpython/l3interp/convertgraph.py (original) +++ pypy/dist/pypy/rpython/l3interp/convertgraph.py Mon Jan 23 12:14:39 2006 @@ -3,6 +3,7 @@ from pypy.rpython.l3interp.model import Op from pypy.objspace.flow import model as flowmodel from pypy.rpython.lltypesystem import lltype +from pypy.rpython.memory.lladdress import OffsetOf, fakeaddress class LL2L3Converter(object): @@ -32,6 +33,22 @@ else: return 'ptr' +def getaccesskind(T): + assert isinstance(T, lltype.LowLevelType) + if isinstance(T, lltype.Primitive): + if T == lltype.Float: + return 'dbl' + if T == lltype.Signed: + return 'int' + if T == lltype.Char: + return 'char' + elif T == lltype.Void: + raise Exception("accessing a Void value?") + else: + raise Exception("don't know how to acess %s value"%T) + else: + return 'ptr' + def convert_block(block, memo): if block in memo: return memo[block] @@ -41,7 +58,8 @@ 'ptr': 0} constants = {'int': [], 'dbl': [], - 'ptr': []} + 'ptr': [], + 'offset':[]} var2stack = {} def push(v): @@ -54,11 +72,15 @@ kind = getkind(v.concretetype) if isinstance(v, flowmodel.Constant): clist = constants[kind] + if kind == 'ptr': + value = fakeaddress(v.value) + else: + value = v.value try: - res = clist.index(v.value) + res = clist.index(value) except ValueError: res = len(clist) - clist.append(v.value) + clist.append(value) return res else: position = var2stack[v] @@ -87,9 +109,25 @@ return l3block for spaceop in block.operations: - insns.append(model.very_low_level_opcode[spaceop.opname]) - for v in spaceop.args: - insns.append(get(v)) + if spaceop.opname == 'getfield': + opname = spaceop.opname + '_' + \ + getaccesskind(spaceop.result.concretetype) + insns.append(model.very_low_level_opcode[opname]) + v0, v1 = spaceop.args + insns.append(get(v0)) + + offset = OffsetOf(v0.concretetype, v1.value) + clist = constants['offset'] + try: + res = clist.index(offset) + except ValueError: + res = len(clist) + clist.append(offset) + insns.append(res) + else: + insns.append(model.very_low_level_opcode[spaceop.opname]) + for v in spaceop.args: + insns.append(get(v)) if spaceop.result.concretetype is not lltype.Void: push(spaceop.result) @@ -127,5 +165,6 @@ if constants['int']: l3block.constants_int = constants['int'] if constants['dbl']: l3block.constants_dbl = constants['dbl'] if constants['ptr']: l3block.constants_ptr = constants['ptr'] + if constants['offset']: l3block.constants_offset = constants['offset'] return l3block Modified: pypy/dist/pypy/rpython/l3interp/l3interp.py ============================================================================== --- pypy/dist/pypy/rpython/l3interp/l3interp.py (original) +++ pypy/dist/pypy/rpython/l3interp/l3interp.py Mon Jan 23 12:14:39 2006 @@ -114,6 +114,11 @@ if op >= 0: return self.block.constants_ptr[op] else: return self.stack_ptr[op] + def getoffset(self): + op = self.nextop() + assert op >= 0 + return self.block.constants_offset[op] + def restorestacks(self): del self.stack_int[self.base_int:] del self.stack_dbl[self.base_dbl:] @@ -164,6 +169,11 @@ else: self.stack_int.append(0) + def op_getfield_int(self): + p = self.getptr() + o = self.getoffset() + self.stack_int.append((p + o).signed[0]) + def op_direct_call(self): block = self.block assert block.called_graphs is not None Modified: pypy/dist/pypy/rpython/l3interp/model.py ============================================================================== --- pypy/dist/pypy/rpython/l3interp/model.py (original) +++ pypy/dist/pypy/rpython/l3interp/model.py Mon Jan 23 12:14:39 2006 @@ -41,7 +41,8 @@ 'getarrayitem', 'getarraysize', 'getarraysubstruct', 'setarrayitem', #struct operations: - 'getfield', 'getsubstruct', 'setfield', + 'getfield_int', 'getfield_char', 'getfield_dbl', 'getfield_ptr', + 'getsubstruct', 'setfield', #integer operations: 'int_abs', 'int_abs_ovf', 'int_add', 'int_add_ovf', 'int_and', @@ -104,6 +105,7 @@ constants_int=None, constants_dbl=None, constants_ptr=None, + constants_offset=None, called_graphs=None): self.insns = insns self.exit0 = exit0 @@ -111,6 +113,7 @@ self.constants_int = constants_int self.constants_dbl = constants_dbl self.constants_ptr = constants_ptr + self.constants_offset = constants_offset self.called_graphs = called_graphs class Link(object): Modified: pypy/dist/pypy/rpython/l3interp/test/test_convert.py ============================================================================== --- pypy/dist/pypy/rpython/l3interp/test/test_convert.py (original) +++ pypy/dist/pypy/rpython/l3interp/test/test_convert.py Mon Jan 23 12:14:39 2006 @@ -40,4 +40,24 @@ assert isinstance(result, l3interp.L3Integer) assert result.intval == 1 - +def test_convert_getfield(): + class C: + def __init__(self, x): + self.x = x + one = C(1) + two = C(2) + + def f(n): + if n: + return one.x + else: + return two.x + l3graph = l3ify(f, [int]) + result = l3interp.l3interpret(l3graph, [3], [], []) + assert isinstance(result, l3interp.L3Integer) + assert result.intval == 1 + + result = l3interp.l3interpret(l3graph, [0], [], []) + assert isinstance(result, l3interp.L3Integer) + assert result.intval == 2 + Modified: pypy/dist/pypy/rpython/memory/lladdress.py ============================================================================== --- pypy/dist/pypy/rpython/memory/lladdress.py (original) +++ pypy/dist/pypy/rpython/memory/lladdress.py Mon Jan 23 12:14:39 2006 @@ -160,3 +160,33 @@ def itemoffsetof(TYPE, n=None): pass + +class fakeaddress(object): + def __init__(self, ob, offset=None): + self.ob = ob + if offset is None: + self.offset = OffsetOf(self.ob._TYPE) + else: + self.offset = offset + + def __add__(self, other): + if not isinstance(other, OffsetOf): + return NotImplemented + return fakeaddress(self.ob, self.offset + other) + +class _fakeaccessor(object): + def __init__(self, addr): + self.addr = addr + def __getitem__(self, index): + assert index == 0 + ob = self.addr.ob + for n in self.addr.offset.fldnames: + ob = getattr(ob, n) + # XXX will need to do pointers differently! + assert lltype.typeOf(ob) == self.TYPE + return ob + +class _signed_fakeaccessor(_fakeaccessor): + TYPE = lltype.Signed + +fakeaddress.signed = property(_signed_fakeaccessor) From mwh at codespeak.net Mon Jan 23 12:21:06 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Mon, 23 Jan 2006 12:21:06 +0100 (CET) Subject: [pypy-svn] r22509 - in pypy/dist/pypy/rpython/l3interp: . test Message-ID: <20060123112106.674F527DC0@code1.codespeak.net> Author: mwh Date: Mon Jan 23 12:21:04 2006 New Revision: 22509 Modified: pypy/dist/pypy/rpython/l3interp/l3interp.py pypy/dist/pypy/rpython/l3interp/test/test_convert.py Log: disable some of last check in as it broke the translation of the l3 interpreter. Modified: pypy/dist/pypy/rpython/l3interp/l3interp.py ============================================================================== --- pypy/dist/pypy/rpython/l3interp/l3interp.py (original) +++ pypy/dist/pypy/rpython/l3interp/l3interp.py Mon Jan 23 12:21:04 2006 @@ -169,7 +169,7 @@ else: self.stack_int.append(0) - def op_getfield_int(self): + def later_op_getfield_int(self): p = self.getptr() o = self.getoffset() self.stack_int.append((p + o).signed[0]) Modified: pypy/dist/pypy/rpython/l3interp/test/test_convert.py ============================================================================== --- pypy/dist/pypy/rpython/l3interp/test/test_convert.py (original) +++ pypy/dist/pypy/rpython/l3interp/test/test_convert.py Mon Jan 23 12:21:04 2006 @@ -40,7 +40,7 @@ assert isinstance(result, l3interp.L3Integer) assert result.intval == 1 -def test_convert_getfield(): +def dont_test_convert_getfield(): class C: def __init__(self, x): self.x = x From ac at codespeak.net Mon Jan 23 13:23:06 2006 From: ac at codespeak.net (ac at codespeak.net) Date: Mon, 23 Jan 2006 13:23:06 +0100 (CET) Subject: [pypy-svn] r22510 - in pypy/dist: . pypy/annotation pypy/interpreter pypy/interpreter/astcompiler pypy/interpreter/test pypy/objspace/flow pypy/objspace/flow/test pypy/objspace/std pypy/rpython pypy/tool/test Message-ID: <20060123122306.57D1927B84@code1.codespeak.net> Author: ac Date: Mon Jan 23 13:23:03 2006 New Revision: 22510 Modified: pypy/dist/ (props changed) pypy/dist/pypy/annotation/bookkeeper.py pypy/dist/pypy/annotation/description.py pypy/dist/pypy/interpreter/argument.py pypy/dist/pypy/interpreter/astcompiler/pyassem.py pypy/dist/pypy/interpreter/baseobjspace.py pypy/dist/pypy/interpreter/function.py pypy/dist/pypy/interpreter/gateway.py pypy/dist/pypy/interpreter/miscutils.py pypy/dist/pypy/interpreter/nestedscope.py pypy/dist/pypy/interpreter/pycode.py pypy/dist/pypy/interpreter/pycompiler.py pypy/dist/pypy/interpreter/pyframe.py pypy/dist/pypy/interpreter/pyopcode.py pypy/dist/pypy/interpreter/test/test_eval.py pypy/dist/pypy/interpreter/test/test_function.py pypy/dist/pypy/objspace/flow/objspace.py pypy/dist/pypy/objspace/flow/test/test_framestate.py pypy/dist/pypy/objspace/std/marshal_impl.py pypy/dist/pypy/rpython/callparse.py pypy/dist/pypy/tool/test/test_pytestsupport.py Log: Integrate work from my branch with experiments on arguments passing. Merged revisions 21927,22069,22111,22227,22230,22265,22398,22401,22424,22439,22452 via svnmerge from svn+ssh://codespeak.net/svn/pypy/branch/arre-experiments ........ r21927 | ac | 2006-01-11 13:35:06 +0100 (Wed, 11 Jan 2006) | 1 line Have Arguments.prepend() not copy data from self. ........ r22069 | ac | 2006-01-12 19:54:24 +0100 (Thu, 12 Jan 2006) | 1 line Slight refactoring. ........ r22111 | ac | 2006-01-13 14:39:32 +0100 (Fri, 13 Jan 2006) | 3 lines Initialized merge tracking via "svnmerge" with revisions "1-21923" from svn+ssh://codespeak.net/svn/pypy/dist ........ r22227 | ac | 2006-01-16 17:29:58 +0100 (Mon, 16 Jan 2006) | 1 line Specialcase positional arguments on valuestack. ........ r22230 | ac | 2006-01-16 18:13:37 +0100 (Mon, 16 Jan 2006) | 1 line Fix some typos. ........ r22265 | ac | 2006-01-18 00:54:39 +0100 (Wed, 18 Jan 2006) | 4 lines Some nice progress on the Arguments handling. the microbench for object creation is now down to a factor of ~20 ........ r22398 | ac | 2006-01-18 17:24:29 +0100 (Wed, 18 Jan 2006) | 3 lines Some more tweeking. Possibly a regression performance-wise. ........ r22401 | ac | 2006-01-18 23:02:58 +0100 (Wed, 18 Jan 2006) | 1 line Bugfixes. ........ r22424 | ac | 2006-01-19 12:27:31 +0100 (Thu, 19 Jan 2006) | 1 line Oops, do not enable debug option for everyone. ........ r22439 | ac | 2006-01-19 14:28:03 +0100 (Thu, 19 Jan 2006) | 1 line A bugfix and a tweak. ........ r22452 | ac | 2006-01-19 17:38:53 +0100 (Thu, 19 Jan 2006) | 1 line Refactor construction of PyCode objects. ........ Modified: pypy/dist/pypy/annotation/bookkeeper.py ============================================================================== --- pypy/dist/pypy/annotation/bookkeeper.py (original) +++ pypy/dist/pypy/annotation/bookkeeper.py Mon Jan 23 13:23:03 2006 @@ -634,6 +634,7 @@ For the Arguments class: if it really needs other operations, it means that the call pattern is too complex for R-Python. """ + w_tuple = SomeTuple def newtuple(self, items_s): return SomeTuple(items_s) @@ -648,6 +649,13 @@ return s_obj.items raise CallPatternTooComplex, "'*' argument must be SomeTuple" + def is_w(self, one, other): + return one is other + + def type(self, item): + return type(item) + + class CallPatternTooComplex(Exception): pass Modified: pypy/dist/pypy/annotation/description.py ============================================================================== --- pypy/dist/pypy/annotation/description.py (original) +++ pypy/dist/pypy/annotation/description.py Mon Jan 23 13:23:03 2006 @@ -202,7 +202,7 @@ try: inputcells = args.match_signature(signature, defs_s) except ArgErr, e: - raise TypeError, "signature mismatch: %s" % e.getmsg(args, self.name) + raise TypeError, "signature mismatch: %s" % e.getmsg(self.name) return inputcells def specialize(self, inputcells): Modified: pypy/dist/pypy/interpreter/argument.py ============================================================================== --- pypy/dist/pypy/interpreter/argument.py (original) +++ pypy/dist/pypy/interpreter/argument.py Mon Jan 23 13:23:03 2006 @@ -3,9 +3,239 @@ """ from pypy.interpreter.error import OperationError +import os +class AbstractArguments: -class Arguments: + def parse(self, fnname, signature, defaults_w=[]): + """Parse args and kwargs to initialize a frame + according to the signature of code object. + """ + try: + return self.match_signature(signature, defaults_w) + except ArgErr, e: + raise OperationError(self.space.w_TypeError, + self.space.wrap(e.getmsg(fnname))) + + def parse_into_scope(self, scope_w, fnname, signature, defaults_w=[]): + """Parse args and kwargs to initialize a frame + according to the signature of code object. + Store the argumentvalues into scope_w. + scope_w must be big enough for signature. + """ + argnames, varargname, kwargname = signature + has_vararg = varargname is not None + has_kwarg = kwargname is not None + try: + return self._match_signature(scope_w, argnames, has_vararg, + has_kwarg, defaults_w, 0, None) + except ArgErr, e: + raise OperationError(self.space.w_TypeError, + self.space.wrap(e.getmsg(fnname))) + + def frompacked(space, w_args=None, w_kwds=None): + """Convenience static method to build an Arguments + from a wrapped sequence and a wrapped dictionary.""" + return Arguments(space, [], w_stararg=w_args, w_starstararg=w_kwds) + frompacked = staticmethod(frompacked) + + def fromshape(space, (shape_cnt,shape_keys,shape_star,shape_stst), data_w): + args_w = data_w[:shape_cnt] + p = shape_cnt + kwds_w = {} + for i in range(len(shape_keys)): + kwds_w[shape_keys[i]] = data_w[p] + p += 1 + if shape_star: + w_star = data_w[p] + p += 1 + else: + w_star = None + if shape_stst: + w_starstar = data_w[p] + p += 1 + else: + w_starstar = None + return Arguments(space, args_w, kwds_w, w_star, w_starstar) + fromshape = staticmethod(fromshape) + + def prepend(self, w_firstarg): + "Return a new Arguments with a new argument inserted first." + return ArgumentsPrepended(self, w_firstarg) + + def match_signature(self, signature, defaults_w): + """Parse args and kwargs according to the signature of a code object, + or raise an ArgErr in case of failure. + """ + argnames, varargname, kwargname = signature + scopelen = len(argnames) + has_vararg = varargname is not None + has_kwarg = kwargname is not None + if has_vararg: + scopelen += 1 + if has_kwarg: + scopelen += 1 + scope_w = [None] * scopelen + self._match_signature(scope_w, argnames, has_vararg, has_kwarg, defaults_w, 0, None) + return scope_w + +class ArgumentsPrepended(AbstractArguments): + def __init__(self, args, w_firstarg): + self.space = args.space + self.args = args + self.w_firstarg = w_firstarg + + def firstarg(self): + "Return the first argument for inspection." + return self.w_firstarg + + def __repr__(self): + return 'ArgumentsPrepended(%r, %r)' % (self.args, self.w_firstarg) + + def has_keywords(self): + return self.args.has_keywords() + + def unpack(self): + arguments_w, kwds_w = self.args.unpack() + return ([self.w_firstarg] + arguments_w), kwds_w + + def fixedunpack(self, argcount): + if argcount <= 0: + raise ValueError, "too many arguments (%d expected)" % argcount # XXX: Incorrect + return [self.w_firstarg] + self.args.fixedunpack(argcount - 1) + + def _rawshape(self, nextra=0): + return self.args._rawshape(nextra + 1) + + def _match_signature(self, scope_w, argnames, has_vararg=False, has_kwarg=False, defaults_w=[], blindargs=0, extravarargs=None): + """Parse args and kwargs according to the signature of a code object, + or raise an ArgErr in case of failure. + Return the number of arguments filled in. + """ + if blindargs < len(argnames): + scope_w[blindargs] = self.w_firstarg + else: + if extravarargs is None: + extravarargs = [ self.w_firstarg ] + else: + extravarargs.append(self.w_firstarg) + return self.args._match_signature(scope_w, argnames, has_vararg, + has_kwarg, defaults_w, + blindargs + 1, extravarargs) + + def flatten(self): + (shape_cnt, shape_keys, shape_star, shape_stst), data_w = self.args.flatten() + data_w.insert(0, self.w_firstarg) + return (shape_cnt + 1, shape_keys, shape_star, shape_stst), data_w + + def num_args(self): + return self.args.num_args() + 1 + + def num_kwds(self): + return self.args.num_kwds() + +class ArgumentsFromValuestack(AbstractArguments): + """ + Collects the arguments of a fuction call as stored on a PyFrame valuestack. + + Only for the case of purely positional arguments, for now. + """ + + def __init__(self, space, valuestack, nargs=0): + self.space = space + self.valuestack = valuestack + self.nargs = nargs + + def firstarg(self): + if self.nargs <= 0: + return None + return self.valuestack.top(self.nargs - 1) + + def __repr__(self): + return 'ArgumentsFromValuestack(%r, %r)' % (self.valuestack, self.nargs) + + def has_keywords(self): + return False + + def unpack(self): + args_w = [None] * self.nargs + for i in range(self.nargs): + args_w[i] = self.valuestack.top(self.nargs - 1 - i) + return args_w, {} + + def fixedunpack(self, argcount): + if self.nargs > argcount: + raise ValueError, "too many arguments (%d expected)" % argcount + elif self.nargs < argcount: + raise ValueError, "not enough arguments (%d expected)" % argcount + data_w = [None] * self.nargs + for i in range(self.nargs): + data_w[i] = self.valuestack.top(nargs - 1 - i) + return data_w + + def _rawshape(self, nextra=0): + return nextra + self.nargs, (), False, False + + def _match_signature(self, scope_w, argnames, has_vararg=False, has_kwarg=False, defaults_w=[], blindargs=0, extravarargs=None): + """Parse args and kwargs according to the signature of a code object, + or raise an ArgErr in case of failure. + Return the number of arguments filled in. + """ + co_argcount = len(argnames) + if blindargs + self.nargs + len(defaults_w) < co_argcount: + raise ArgErrCount(blindargs + self.nargs , 0, + (co_argcount, has_vararg, has_kwarg), + defaults_w, co_argcount - blindargs - + self.nargs - len(defaults_w)) + if blindargs + self.nargs > co_argcount and not has_vararg: + raise ArgErrCount(blindargs + self.nargs, 0, + (co_argcount, has_vararg, has_kwarg), + defaults_w, 0) + + if blindargs + self.nargs >= co_argcount: + for i in range(co_argcount - blindargs): + scope_w[i + blindargs] = self.valuestack.top(self.nargs - 1 - i) + if has_vararg: + if blindargs > co_argcount: + stararg_w = extravarargs + for i in range(self.nargs): + stararg_w.append(self.valuestack.top(self.nargs - 1 - i)) + else: + stararg_w = [None] * (self.nargs + blindargs - co_argcount) + for i in range(co_argcount - blindargs, self.nargs): + stararg_w[i - co_argcount + blindargs] = self.valuestack.top(self.nargs - 1 - i) + scope_w[co_argcount] = self.space.newtuple(stararg_w) + co_argcount += 1 + else: + for i in range(self.nargs): + scope_w[i + blindargs] = self.valuestack.top(self.nargs - 1 - i) + ndefaults = len(defaults_w) + missing = co_argcount - self.nargs - blindargs + first_default = ndefaults - missing + for i in range(missing): + scope_w[self.nargs + blindargs + i] = defaults_w[first_default + i] + if has_vararg: + scope_w[co_argcount] = self.space.newtuple([]) + co_argcount += 1 + + if has_kwarg: + scope_w[co_argcount] = self.space.newdict([]) + co_argcount += 1 + return co_argcount + + def flatten(self): + data_w = [None] * self.nargs + for i in range(self.nargs): + data_w[i] = self.valuestack.top(nargs - 1 - i) + return nextra + self.nargs, (), False, False, data_w + + def num_args(self): + return self.nargs + + def num_kwds(self): + return 0 + +class Arguments(AbstractArguments): """ Collects the arguments of a function call. @@ -24,12 +254,14 @@ self.w_stararg = w_stararg self.w_starstararg = w_starstararg - def frompacked(space, w_args=None, w_kwds=None): - """Convenience static method to build an Arguments - from a wrapped sequence and a wrapped dictionary.""" - return Arguments(space, [], w_stararg=w_args, w_starstararg=w_kwds) - frompacked = staticmethod(frompacked) - + def num_args(self): + self._unpack() + return len(self.arguments_w) + + def num_kwds(self): + self._unpack() + return len(self.kwds_w) + def __repr__(self): if self.w_starstararg is not None: return 'Arguments(%s, %s, %s, %s)' % (self.arguments_w, @@ -50,6 +282,11 @@ def unpack(self): "Return a ([w1,w2...], {'kw':w3...}) pair." + self._unpack() + return self.arguments_w, self.kwds_w + + def _unpack(self): + "unpack the *arg and **kwd into w_arguments and kwds_w" # --- unpack the * argument now --- if self.w_stararg is not None: self.arguments_w += self.space.unpackiterable(self.w_stararg) @@ -82,14 +319,6 @@ d[key] = space.getitem(w_starstararg, w_key) self.kwds_w = d self.w_starstararg = None - return self.arguments_w, self.kwds_w - - def prepend(self, w_firstarg): - "Return a new Arguments with a new argument inserted first." - args = Arguments(self.space, [w_firstarg] + self.arguments_w, - self.kwds_w, self.w_stararg, self.w_starstararg) - args.blind_arguments = self.blind_arguments + 1 - return args def has_keywords(self): return bool(self.kwds_w) or (self.w_starstararg is not None and @@ -126,28 +355,13 @@ ### Parsing for function calls ### - def parse(self, fnname, signature, defaults_w=[]): - """Parse args and kwargs to initialize a frame - according to the signature of code object. - """ - space = self.space - # If w_stararg is not exactly a tuple, unpack it now: - # self.match_signature() assumes that it can use it directly for - # a matching *arg in the callee's signature. - if self.w_stararg is not None: - if not space.is_w(space.type(self.w_stararg), space.w_tuple): - self.unpack() - try: - return self.match_signature(signature, defaults_w) - except ArgErr, e: - raise OperationError(space.w_TypeError, - space.wrap(e.getmsg(self, fnname))) - - def match_signature(self, signature, defaults_w=[]): + def _match_signature(self, scope_w, argnames, has_vararg=False, + has_kwarg=False, defaults_w=[], blindargs=0, + extravarargs=None): """Parse args and kwargs according to the signature of a code object, or raise an ArgErr in case of failure. + Return the number of arguments filled in. """ - argnames, varargname, kwargname = signature # # args_w = list of the normal actual parameters, wrapped # kwds_w = real dictionary {'keyword': wrapped parameter} @@ -158,20 +372,35 @@ if self.w_stararg is not None: # There is a case where we don't have to unpack() a w_stararg: # if it matches exactly a *arg in the signature. - if len(self.arguments_w) == co_argcount and varargname is not None: + if (len(self.arguments_w) + blindargs == co_argcount and + has_vararg and + self.space.is_w(self.space.type(self.w_stararg), + self.space.w_tuple)): pass else: - self.unpack() # sets self.w_stararg to None + self._unpack() # sets self.w_stararg to None # always unpack the ** arguments if self.w_starstararg is not None: - self.unpack() + self._unpack() args_w = self.arguments_w kwds_w = self.kwds_w - + num_kwds = 0 + if kwds_w is not None: + num_kwds = len(kwds_w) + # put as many positional input arguments into place as available - scope_w = args_w[:co_argcount] - input_argcount = len(scope_w) + if blindargs >= co_argcount: + input_argcount = co_argcount + elif len(args_w) + blindargs > co_argcount: + for i in range(co_argcount - blindargs): + scope_w[i + blindargs] = args_w[i] + input_argcount = co_argcount + next_arg = co_argcount - blindargs + else: + for i in range(len(args_w)): + scope_w[i + blindargs] = args_w[i] + input_argcount = len(args_w) + blindargs # check that no keyword argument conflicts with these # note that for this purpose we ignore the first blind_arguments, @@ -194,43 +423,54 @@ for i in range(input_argcount, co_argcount): name = argnames[i] if name in remainingkwds_w: - scope_w.append(remainingkwds_w[name]) + scope_w[i] = remainingkwds_w[name] del remainingkwds_w[name] elif i >= def_first: - scope_w.append(defaults_w[i-def_first]) + scope_w[i] = defaults_w[i-def_first] else: # error: not enough arguments. Don't signal it immediately # because it might be related to a problem with */** or # keyword arguments, which will be checked for below. missing += 1 + # collect extra positional arguments into the *vararg - if varargname is not None: + if has_vararg: if self.w_stararg is None: # common case - if len(args_w) > co_argcount: # check required by rpython - starargs_w = args_w[co_argcount:] + args_left = co_argcount - blindargs + if args_left < 0: # check required by rpython + starargs_w = extravarargs + if len(args_w): + starargs_w.extend(args_w) + elif len(args_w) > args_left: + starargs_w = args_w[args_left:] else: starargs_w = [] - scope_w.append(self.space.newtuple(starargs_w)) + scope_w[co_argcount] = self.space.newtuple(starargs_w) else: # shortcut for the non-unpack() case above - scope_w.append(self.w_stararg) - elif len(args_w) > co_argcount: - raise ArgErrCount(signature, defaults_w, 0) + scope_w[co_argcount] = self.w_stararg + elif len(args_w) + blindargs > co_argcount: + raise ArgErrCount(len(args_w) + blindargs, num_kwds, + (co_argcount, has_vararg, has_kwarg), + defaults_w, 0) # collect extra keyword arguments into the **kwarg - if kwargname is not None: + if has_kwarg: w_kwds = self.space.newdict([]) if remainingkwds_w: for key, w_value in remainingkwds_w.items(): self.space.setitem(w_kwds, self.space.wrap(key), w_value) - scope_w.append(w_kwds) + scope_w[co_argcount + has_vararg] = w_kwds elif remainingkwds_w: raise ArgErrUnknownKwds(remainingkwds_w) if missing: - raise ArgErrCount(signature, defaults_w, missing) - return scope_w + raise ArgErrCount(len(args_w) + blindargs, num_kwds, + (co_argcount, has_vararg, has_kwarg), + defaults_w, missing) + return co_argcount + has_vararg + has_kwarg + ### Argument <-> list of w_objects together with "shape" information def _rawshape(self, nextra=0): @@ -253,26 +493,6 @@ data_w.append(self.w_starstararg) return (shape_cnt, shape_keys, shape_star, shape_stst), data_w - def fromshape(space, (shape_cnt,shape_keys,shape_star,shape_stst), data_w): - args_w = data_w[:shape_cnt] - p = shape_cnt - kwds_w = {} - for i in range(len(shape_keys)): - kwds_w[shape_keys[i]] = data_w[p] - p += 1 - if shape_star: - w_star = data_w[p] - p += 1 - else: - w_star = None - if shape_stst: - w_starstar = data_w[p] - p += 1 - else: - w_starstar = None - return Arguments(space, args_w, kwds_w, w_star, w_starstar) - fromshape = staticmethod(fromshape) - def rawshape(args, nextra=0): return args._rawshape(nextra) @@ -284,45 +504,48 @@ class ArgErr(Exception): - def getmsg(self, args, fnname): + def getmsg(self, fnname): raise NotImplementedError class ArgErrCount(ArgErr): - def __init__(self, signature, defaults_w, missing_args): + def __init__(self, nargs, nkwds, signature, defaults_w, missing_args): self.signature = signature - self.defaults_w = defaults_w + self.num_defaults = len(defaults_w) self.missing_args = missing_args - - def getmsg(self, args, fnname): - argnames, varargname, kwargname = self.signature - args_w, kwds_w = args.unpack() - if kwargname is not None or (kwds_w and self.defaults_w): + self.num_args = nargs + self.num_kwds = nkwds + + def getmsg(self, fnname): + args = None + num_args, has_vararg, has_kwarg = self.signature + #args_w, kwds_w = args.unpack() + if has_kwarg or (self.num_kwds and self.num_defaults): msg2 = "non-keyword " if self.missing_args: - required_args = len(argnames) - len(self.defaults_w) + required_args = num_args - self.num_defaults nargs = required_args - self.missing_args else: - nargs = len(args_w) + nargs = self.num_args else: msg2 = "" - nargs = len(args_w) + len(kwds_w) - n = len(argnames) + nargs = self.num_args + self.num_kwds + n = num_args if n == 0: msg = "%s() takes no %sargument (%d given)" % ( fnname, msg2, nargs) else: - defcount = len(self.defaults_w) - if defcount == 0 and varargname is None: + defcount = self.num_defaults + if defcount == 0 and has_vararg: msg1 = "exactly" elif not self.missing_args: msg1 = "at most" else: msg1 = "at least" n -= defcount - if not kwds_w: # msg "f() takes at least X non-keyword args" + if not self.num_kwds: # msg "f() takes at least X non-keyword args" msg2 = "" # is confusing if no kwd arg actually provided if n == 1: plural = "" @@ -342,7 +565,7 @@ def __init__(self, argname): self.argname = argname - def getmsg(self, args, fnname): + def getmsg(self, fnname): msg = "%s() got multiple values for keyword argument '%s'" % ( fnname, self.argname) @@ -351,16 +574,18 @@ class ArgErrUnknownKwds(ArgErr): def __init__(self, kwds_w): - self.kwds_w = kwds_w + self.kwd_name = '' + self.num_kwds = len(kwds_w) + if self.num_kwds == 1: + self.kwd_name = kwds_w.keys()[0] - def getmsg(self, args, fnname): - kwds_w = self.kwds_w - if len(kwds_w) == 1: + def getmsg(self, fnname): + if self.num_kwds == 1: msg = "%s() got an unexpected keyword argument '%s'" % ( fnname, - kwds_w.keys()[0]) + self.kwd_name) else: msg = "%s() got %d unexpected keyword arguments" % ( fnname, - len(kwds_w)) + self.num_kwds) return msg Modified: pypy/dist/pypy/interpreter/astcompiler/pyassem.py ============================================================================== --- pypy/dist/pypy/interpreter/astcompiler/pyassem.py (original) +++ pypy/dist/pypy/interpreter/astcompiler/pyassem.py Mon Jan 23 13:23:03 2006 @@ -832,19 +832,18 @@ argcount = argcount - 1 # was return new.code, now we just return the parameters and let # the caller create the code object - # XXX _code_new_w itself is not really annotable - return PyCode(self.space)._code_new_w( argcount, nlocals, - self.stacksize, self.flags, - self.lnotab.getCode(), - self.getConsts(), - self.names, - self.varnames, - self.filename, self.name, - self.firstline, - self.lnotab.getTable(), - self.freevars, - self.cellvars - ) + return PyCode( self.space, argcount, nlocals, + self.stacksize, self.flags, + self.lnotab.getCode(), + self.getConsts(), + self.names, + self.varnames, + self.filename, self.name, + self.firstline, + self.lnotab.getTable(), + self.freevars, + self.cellvars + ) def getConsts(self): """Return a tuple for the const slot of the code object Modified: pypy/dist/pypy/interpreter/baseobjspace.py ============================================================================== --- pypy/dist/pypy/interpreter/baseobjspace.py (original) +++ pypy/dist/pypy/interpreter/baseobjspace.py Mon Jan 23 13:23:03 2006 @@ -577,7 +577,7 @@ if isinstance(expression, str): expression = compile(expression, '?', 'eval') if isinstance(expression, types.CodeType): - expression = PyCode(self)._from_code(expression) + expression = PyCode._from_code(self, expression) if not isinstance(expression, PyCode): raise TypeError, 'space.eval(): expected a string, code or PyCode object' return expression.exec_code(self, w_globals, w_locals) @@ -589,7 +589,7 @@ if isinstance(statement, str): statement = compile(statement, '?', 'exec') if isinstance(statement, types.CodeType): - statement = PyCode(self)._from_code(statement) + statement = PyCode._from_code(self, statement) if not isinstance(statement, PyCode): raise TypeError, 'space.exec_(): expected a string, code or PyCode object' w_key = self.wrap('__builtins__') Modified: pypy/dist/pypy/interpreter/function.py ============================================================================== --- pypy/dist/pypy/interpreter/function.py (original) +++ pypy/dist/pypy/interpreter/function.py Mon Jan 23 13:23:03 2006 @@ -9,6 +9,7 @@ from pypy.interpreter.error import OperationError from pypy.interpreter.baseobjspace import Wrappable from pypy.interpreter.eval import Code +from pypy.interpreter.pyframe import PyFrame class Function(Wrappable): """A function is a code object captured with some environment: @@ -32,10 +33,18 @@ return "" % self.name def call_args(self, args): - scope_w = args.parse(self.name, self.code.signature(), self.defs_w) frame = self.code.create_frame(self.space, self.w_func_globals, - self.closure) - frame.setfastscope(scope_w) + self.closure) + sig = self.code.signature() + # XXX start of hack for performance + if frame.setfastscope is PyFrame.setfastscope: + args_matched = args.parse_into_scope(frame.fastlocals_w, self.name, + sig, self.defs_w) + frame.init_cells(args_matched) + # XXX end of hack for performance + else: + scope_w = args.parse(self.name, sig, self.defs_w) + frame.setfastscope(scope_w) return frame.run() def getdict(self): Modified: pypy/dist/pypy/interpreter/gateway.py ============================================================================== --- pypy/dist/pypy/interpreter/gateway.py (original) +++ pypy/dist/pypy/interpreter/gateway.py Mon Jan 23 13:23:03 2006 @@ -312,9 +312,14 @@ del d['setfastscope_UWS_%s' % label] self.miniglobals['OperationError'] = OperationError + self.miniglobals['os'] = os source = """if 1: def _run_UWS_%s(self): - return self.behavior(%s) + try: + return self.behavior(%s) + except MemoryError: + os.write(2, 'Fail in _run() of ' + self.b_name + '\\n') + raise \n""" % (label, ','.join(self.run_args)) exec compile2(source) in self.miniglobals, d d['_run'] = d['_run_UWS_%s' % label] @@ -327,6 +332,7 @@ def create(self, space, code, w_globals): newframe = frame_cls(space, code, w_globals) newframe.behavior = self.behavior + newframe.b_name = self.b_name return newframe MyBuiltinFrameFactory.__name__ = 'BuiltinFrameFactory_UwS_%s' % label @@ -339,6 +345,7 @@ factory = frame_uw_factory_cls() factory.behavior = func + factory.b_name = func.__name__ return factory @@ -687,7 +694,7 @@ def build_applevel_dict(self, space): "NOT_RPYTHON" from pypy.interpreter.pycode import PyCode - pycode = PyCode(space)._from_code(self.code, hidden_applevel=self.hidden_applevel) + pycode = PyCode._from_code(space, self.code, hidden_applevel=self.hidden_applevel) w_glob = space.newdict([]) space.setitem(w_glob, space.wrap('__name__'), space.wrap('__builtin__')) space.exec_(pycode, w_glob, w_glob) Modified: pypy/dist/pypy/interpreter/miscutils.py ============================================================================== --- pypy/dist/pypy/interpreter/miscutils.py (original) +++ pypy/dist/pypy/interpreter/miscutils.py Mon Jan 23 13:23:03 2006 @@ -33,6 +33,10 @@ def pop(self): return self.items.pop() + def drop(self, n): + if n > 0: + del self.items[-n:] + def top(self, position=0): """'position' is 0 for the top of the stack, 1 for the item below, and so on. It must not be negative.""" @@ -84,6 +88,12 @@ self.ptr = ptr return ret + def drop(self, n): + while n > 0: + n -= 1 + self.ptr -= 1 + self.items[self.ptr] = None + def top(self, position=0): # for a fixed stack, we assume correct indices return self.items[self.ptr + ~position] Modified: pypy/dist/pypy/interpreter/nestedscope.py ============================================================================== --- pypy/dist/pypy/interpreter/nestedscope.py (original) +++ pypy/dist/pypy/interpreter/nestedscope.py Mon Jan 23 13:23:03 2006 @@ -99,8 +99,7 @@ else: cell.set(w_value) - def setfastscope(self, scope_w): - PyInterpFrame.setfastscope(self, scope_w) + def init_cells(self, num_vars): if self.pycode.co_cellvars: # the first few cell vars could shadow already-set arguments, # in the same order as they appear in co_varnames @@ -109,10 +108,10 @@ cellvars = code.co_cellvars next = 0 nextname = cellvars[0] - for i in range(len(scope_w)): + for i in range(num_vars): if argvars[i] == nextname: # argument i has the same name as the next cell var - w_value = scope_w[i] + w_value = self.fastlocals_w[i] self.cells[next] = Cell(w_value) next += 1 try: Modified: pypy/dist/pypy/interpreter/pycode.py ============================================================================== --- pypy/dist/pypy/interpreter/pycode.py (original) +++ pypy/dist/pypy/interpreter/pycode.py Mon Jan 23 13:23:03 2006 @@ -4,7 +4,8 @@ The bytecode interpreter itself is implemented by the PyFrame class. """ -import dis +import dis, imp, struct, types + from pypy.interpreter import eval from pypy.interpreter.error import OperationError from pypy.interpreter.gateway import NoneNotWrapped @@ -45,6 +46,7 @@ kwargname = None return argnames, varargname, kwargname +cpython_magic, = struct.unpack("lineno mapping - self.hidden_applevel = False - self.do_fastcall = -1 - - def _code_new( self, argcount, nlocals, stacksize, flags, - code, consts, names, varnames, filename, - name, firstlineno, lnotab, freevars, cellvars, - hidden_applevel=False, from_cpython=False): - """Initialize a new code objects from parameters from new.code""" - # simply try to suck in all attributes we know of - # with a lot of boring asserts to enforce type knowledge - # XXX get rid of that ASAP with a real compiler! - import types - x = argcount; assert isinstance(x, int) - self.co_argcount = x - x = nlocals; assert isinstance(x, int) - self.co_nlocals = x - x = stacksize; assert isinstance(x, int) - self.co_stacksize = x - x = flags; assert isinstance(x, int) - self.co_flags = x - x = code; assert isinstance(x, str) - self.co_code = x - #self.co_consts = - x = names; assert isinstance(x, tuple) - self.co_names = [ str(n) for n in x ] - x = varnames; assert isinstance(x, tuple) - self.co_varnames = [ str(n) for n in x ] - x = freevars; assert isinstance(x, tuple) - self.co_freevars = [ str(n) for n in x ] - x = cellvars; assert isinstance(x, tuple) - self.co_cellvars = [ str(n) for n in x ] - x = filename; assert isinstance(x, str) - self.co_filename = x - x = name; assert isinstance(x, str) - self.co_name = x - x = firstlineno; assert isinstance(x, int) - self.co_firstlineno = x - x = lnotab; assert isinstance(x, str) - self.co_lnotab = x - # recursively _from_code()-ify the code objects in code.co_consts - space = self.space - newconsts_w = [] - for const in consts: - if isinstance(const, types.CodeType): # from stable compiler - const = PyCode(space)._from_code(const, hidden_applevel=hidden_applevel, - from_cpython=from_cpython) - newconsts_w.append(space.wrap(const)) - self.co_consts_w = newconsts_w - # stick the underlying CPython magic value, if the code object - # comes from there - if from_cpython: - import imp, struct - magic, = struct.unpack(" len(self.fastlocals_w): raise ValueError, "new fastscope is longer than the allocated area" self.fastlocals_w[:len(scope_w)] = scope_w - + self.init_cells(len(scope_w)) + + def init_cells(self, numvars): + """Initialize cellvars from self.fastlocals_w + This is overridden in PyNestedScopeFrame""" + pass + def getclosure(self): return None Modified: pypy/dist/pypy/interpreter/pyopcode.py ============================================================================== --- pypy/dist/pypy/interpreter/pyopcode.py (original) +++ pypy/dist/pypy/interpreter/pyopcode.py Mon Jan 23 13:23:03 2006 @@ -9,7 +9,7 @@ from pypy.interpreter import gateway, function, eval from pypy.interpreter import pyframe, pytraceback from pypy.interpreter.miscutils import InitializedClass -from pypy.interpreter.argument import Arguments +from pypy.interpreter.argument import Arguments, ArgumentsFromValuestack from pypy.interpreter.pycode import PyCode from pypy.tool.sourcetools import func_with_new_name from pypy.rpython.objectmodel import we_are_translated @@ -647,13 +647,14 @@ w_key = f.valuestack.pop() key = f.space.str_w(w_key) keywords[key] = w_value - arguments = [f.valuestack.pop() for i in range(n_arguments)] - arguments.reverse() + arguments = [None] * n_arguments + for i in range(n_arguments - 1, -1, -1): + arguments[i] = f.valuestack.pop() args = Arguments(f.space, arguments, keywords, w_star, w_starstar) w_function = f.valuestack.pop() w_result = f.space.call_args(w_function, args) f.valuestack.push(w_result) - + def CALL_FUNCTION(f, oparg): # XXX start of hack for performance if oparg == 0: # 0 arg, 0 keyword arg @@ -678,6 +679,17 @@ w_function = f.valuestack.pop() w_result = f.space.call_function(w_function, w_arg1, w_arg2, w_arg3) f.valuestack.push(w_result) + elif (oparg >> 8) & 0xff == 0: + # Only positional arguments + nargs = oparg & 0xff + args = ArgumentsFromValuestack(f.space, f.valuestack, nargs) + w_function = f.valuestack.top(nargs) + try: + w_result = f.space.call_args(w_function, args) + finally: + args.valuestack = None + f.valuestack.drop(nargs + 1) + f.valuestack.push(w_result) # XXX end of hack for performance else: # general case Modified: pypy/dist/pypy/interpreter/test/test_eval.py ============================================================================== --- pypy/dist/pypy/interpreter/test/test_eval.py (original) +++ pypy/dist/pypy/interpreter/test/test_eval.py Mon Jan 23 13:23:03 2006 @@ -8,7 +8,7 @@ def setup_method(self, method): def c(x, y, *args): pass - code = PyCode(self.space)._from_code(c.func_code) + code = PyCode._from_code(self.space, c.func_code) class ConcreteFastscopeFrame(Frame): Modified: pypy/dist/pypy/interpreter/test/test_function.py ============================================================================== --- pypy/dist/pypy/interpreter/test/test_function.py (original) +++ pypy/dist/pypy/interpreter/test/test_function.py Mon Jan 23 13:23:03 2006 @@ -223,7 +223,7 @@ def setup_method(self, method): def c(self, bar): return bar - code = PyCode(self.space)._from_code(c.func_code) + code = PyCode._from_code(self.space, c.func_code) self.fn = Function(self.space, code, self.space.newdict([])) def test_get(self): @@ -250,7 +250,7 @@ space = self.space # Create some function for this test only def m(self): return self - func = Function(space, PyCode(self.space)._from_code(m.func_code), + func = Function(space, PyCode._from_code(self.space, m.func_code), space.newdict([])) # Some shorthands obj1 = space.wrap(23) Modified: pypy/dist/pypy/objspace/flow/objspace.py ============================================================================== --- pypy/dist/pypy/objspace/flow/objspace.py (original) +++ pypy/dist/pypy/objspace/flow/objspace.py Mon Jan 23 13:23:03 2006 @@ -225,7 +225,7 @@ if func.func_doc and func.func_doc.lstrip().startswith('NOT_RPYTHON'): raise Exception, "%r is tagged as NOT_RPYTHON" % (func,) code = func.func_code - code = PyCode(self)._from_code(code) + code = PyCode._from_code(self, code) if func.func_closure is None: closure = None else: Modified: pypy/dist/pypy/objspace/flow/test/test_framestate.py ============================================================================== --- pypy/dist/pypy/objspace/flow/test/test_framestate.py (original) +++ pypy/dist/pypy/objspace/flow/test/test_framestate.py Mon Jan 23 13:23:03 2006 @@ -18,7 +18,7 @@ except AttributeError: pass code = func.func_code - code = PyCode(self.space)._from_code(code) + code = PyCode._from_code(self.space, code) w_globals = Constant({}) # space.newdict([]) frame = code.create_frame(space, w_globals) Modified: pypy/dist/pypy/objspace/std/marshal_impl.py ============================================================================== --- pypy/dist/pypy/objspace/std/marshal_impl.py (original) +++ pypy/dist/pypy/objspace/std/marshal_impl.py Mon Jan 23 13:23:03 2006 @@ -410,22 +410,24 @@ return res def unmarshal_pycode(space, u, tc): - code = PyCode(space) - code.co_argcount = u.get_int() - code.co_nlocals = u.get_int() - code.co_stacksize = u.get_int() - code.co_flags = u.get_int() - code.co_code = unmarshal_str(u) + argcount = u.get_int() + nlocals = u.get_int() + stacksize = u.get_int() + flags = u.get_int() + code = unmarshal_str(u) u.start(TYPE_TUPLE) - code.co_consts_w = u.get_list_w() - code.co_names = unmarshal_strlist(u, TYPE_TUPLE) - code.co_varnames = unmarshal_strlist(u, TYPE_TUPLE) - code.co_freevars = unmarshal_strlist(u, TYPE_TUPLE) - code.co_cellvars = unmarshal_strlist(u, TYPE_TUPLE) - code.co_filename = unmarshal_str(u) - code.co_name = unmarshal_str(u) - code.co_firstlineno = u.get_int() - code.co_lnotab = unmarshal_str(u) + consts_w = u.get_list_w() + names = unmarshal_strlist(u, TYPE_TUPLE) + varnames = unmarshal_strlist(u, TYPE_TUPLE) + freevars = unmarshal_strlist(u, TYPE_TUPLE) + cellvars = unmarshal_strlist(u, TYPE_TUPLE) + filename = unmarshal_str(u) + name = unmarshal_str(u) + firstlineno = u.get_int() + lnotab = unmarshal_str(u) + code = PyCode._code_new_w(space, argcount, nlocals, stacksize, flags, + code, consts_w, names, varnames, filename, + name, firstlineno, lnotab, freevars, cellvars) return space.wrap(code) register(TYPE_CODE, unmarshal_pycode) Modified: pypy/dist/pypy/rpython/callparse.py ============================================================================== --- pypy/dist/pypy/rpython/callparse.py (original) +++ pypy/dist/pypy/rpython/callparse.py Mon Jan 23 13:23:03 2006 @@ -8,28 +8,6 @@ pass -# for parsing call arguments -class RPythonCallsSpace: - """Pseudo Object Space providing almost no real operation. - For the Arguments class: if it really needs other operations, it means - that the call pattern is too complex for R-Python. - """ - def newtuple(self, items): - return NewTupleHolder(items) - - def newdict(self, stuff): - raise CallPatternTooComplex, "'**' argument" - - def unpackiterable(self, it, expected_length=None): - if it.is_tuple(): - items = it.items() - if (expected_length is not None and - expected_length != len(items)): - raise ValueError - return items - raise CallPatternTooComplex, "'*' argument must be a tuple" - - def getrinputs(rtyper, graph): """Return the list of reprs of the input arguments to the 'graph'.""" return [rtyper.bindingrepr(v) for v in graph.getargs()] @@ -69,7 +47,7 @@ try: holders = arguments.match_signature(signature, defs_h) except ArgErr, e: - raise TyperError, "signature mismatch: %s" % e.getmsg(arguments, graph.name) + raise TyperError, "signature mismatch: %s" % e.getmsg(graph.name) assert len(holders) == len(rinputs), "argument parsing mismatch" vlist = [] @@ -171,3 +149,32 @@ r_tup, v_tuple = self.holder.access(hop) v = r_tup.getitem(hop, v_tuple, index) return hop.llops.convertvar(v, r_tup.items_r[index], repr) + +# for parsing call arguments +class RPythonCallsSpace: + """Pseudo Object Space providing almost no real operation. + For the Arguments class: if it really needs other operations, it means + that the call pattern is too complex for R-Python. + """ + w_tuple = NewTupleHolder + def newtuple(self, items): + return NewTupleHolder(items) + + def newdict(self, stuff): + raise CallPatternTooComplex, "'**' argument" + + def unpackiterable(self, it, expected_length=None): + if it.is_tuple(): + items = it.items() + if (expected_length is not None and + expected_length != len(items)): + raise ValueError + return items + raise CallPatternTooComplex, "'*' argument must be a tuple" + + def is_w(self, one, other): + return one is other + + def type(self, item): + return type(item) + Modified: pypy/dist/pypy/tool/test/test_pytestsupport.py ============================================================================== --- pypy/dist/pypy/tool/test/test_pytestsupport.py (original) +++ pypy/dist/pypy/tool/test/test_pytestsupport.py Mon Jan 23 13:23:03 2006 @@ -13,7 +13,7 @@ def test_AppFrame(space): import sys - co = PyCode(space)._from_code(somefunc.func_code) + co = PyCode._from_code(space, somefunc.func_code) pyframe = PyFrame(space, co, space.newdict([]), None) runner = AppFrame(pyframe) exprinfo.run("f = lambda x: x+1", runner) From auc at codespeak.net Mon Jan 23 13:31:33 2006 From: auc at codespeak.net (auc at codespeak.net) Date: Mon, 23 Jan 2006 13:31:33 +0100 (CET) Subject: [pypy-svn] r22511 - pypy/dist/pypy/lib/logic Message-ID: <20060123123133.2DECC27DBD@code1.codespeak.net> Author: auc Date: Mon Jan 23 13:31:31 2006 New Revision: 22511 Modified: pypy/dist/pypy/lib/logic/test_unification.py pypy/dist/pypy/lib/logic/unification.py Log: check * concurrent unification * values unification Modified: pypy/dist/pypy/lib/logic/test_unification.py ============================================================================== --- pypy/dist/pypy/lib/logic/test_unification.py (original) +++ pypy/dist/pypy/lib/logic/test_unification.py Mon Jan 23 13:31:31 2006 @@ -45,6 +45,21 @@ assert y.val == 42 assert z.val == 3.14 + def test_unify_same(self): + x,y,z = (u.var('x'), u.var('y'), u.var('z')) + u.bind(x, [42, z]) + u.bind(y, [z, 42]) + u.unify(x, y) + assert z.val == 42 + + def test_unify_values(self): + x, y = u.var('x'), u.var('y') + u.bind(x, [1, 2, 3]) + u.bind(y, [1, 2, 3]) + u.unify(x, y) + assert x.val == [1, 2, 3] + assert y.val == [1, 2, 3] + def test_unify_lists_success(self): x,y,z,w = (u.var('x'), u.var('y'), u.var('z'), u.var('w')) @@ -159,6 +174,32 @@ (t1.raised and not t2.raised) + def test_threads_unifying_vars(self): + x, y, z = u.var('x'), u.var('y'), u.var('z') + l1 = range(999) + l2 = range(999) + l1[-1] = z + l2[0] = z + l2[-1] = 0 + u.bind(x, l1) + u.bind(y, l2) + def do_unify(thread, v1, v2): + thread.raised = False + print thread + try: + u.unify(v1, v2) + except u.UnificationFailure: + thread.raised = True + print thread + t1, t2 = (FunThread(do_unify, x, y), + FunThread(do_unify, x, y)) + t1.start() + t2.start() + t1.join() + t2.join() + assert z.val == 0 + assert (t2.raised and not t1.raised) or \ + (t1.raised and not t2.raised) Modified: pypy/dist/pypy/lib/logic/unification.py ============================================================================== --- pypy/dist/pypy/lib/logic/unification.py (original) +++ pypy/dist/pypy/lib/logic/unification.py Mon Jan 23 13:31:31 2006 @@ -230,14 +230,13 @@ self.in_transaction = False def _really_unify(self, x, y): - #FIXME in case of failure, the store state is not - # properly restored ... - print "unify %s with %s" % (x,y) + # print "unify %s with %s" % (x,y) if not _unifiable(x, y): raise UnificationFailure(x, y) # dispatch to the apropriate unifier if not x in self.vars: if not y in self.vars: if x != y: raise UnificationFailure(x, y) + else: return self._unify_var_val(y, x) elif not y in self.vars: self._unify_var_val(x, y) @@ -249,12 +248,15 @@ self.bind(y,x) def _unify_var_val(self, x, y): - if x._is_bound(): raise UnificationFailure(x, y) - if x != y: - self.bind(x, y) + #if x._is_bound(): raise UnificationFailure(x, y) + if x.val != y: + try: + self.bind(x, y) + except AlreadyBound: + raise UnificationFailure(x, y) def _unify_bound(self, x, y): - print "unify bound %s %s" % (x, y) + # print "unify bound %s %s" % (x, y) vx, vy = (x.val, y.val) if type(vx) in [list, set] and isinstance(vy, type(vx)): self._unify_iterable(x, y) @@ -264,7 +266,7 @@ raise UnificationFailure(x, y) def _unify_iterable(self, x, y): - print "unify sequences %s %s" % (x, y) + # print "unify sequences %s %s" % (x, y) vx, vy = (x.val, y.val) idx, top = (0, len(vx)) while (idx < top): @@ -272,7 +274,7 @@ idx += 1 def _unify_mapping(self, x, y): - print "unify mappings %s %s" % (x, y) + # print "unify mappings %s %s" % (x, y) vx, vy = (x.val, y.val) for xk in vx.keys(): self._really_unify(vx[xk], vy[xk]) @@ -298,7 +300,7 @@ """Checks wether two terms can be unified""" if ((id(term1), id(term2))) in _unifiable_memo: return False _unifiable_memo.add((id(term1), id(term2))) - print "unifiable ? %s %s" % (term1, term2) + # print "unifiable ? %s %s" % (term1, term2) if _iterable(term1): if _iterable(term2): return _iterable_unifiable(term1, term2) @@ -311,7 +313,7 @@ def _iterable_unifiable(c1, c2): """Checks wether two iterables can be unified""" - print "unifiable sequences ? %s %s" % (c1, c2) + # print "unifiable sequences ? %s %s" % (c1, c2) if len(c1) != len(c2): return False idx, top = (0, len(c1)) while(idx < top): @@ -322,7 +324,7 @@ def _mapping_unifiable(m1, m2): """Checks wether two mappings can be unified""" - print "unifiable mappings ? %s %s" % (m1, m2) + # print "unifiable mappings ? %s %s" % (m1, m2) if len(m1) != len(m2): return False if m1.keys() != m2.keys(): return False v1, v2 = (m1.items(), m2.items()) From bea at codespeak.net Mon Jan 23 14:04:53 2006 From: bea at codespeak.net (bea at codespeak.net) Date: Mon, 23 Jan 2006 14:04:53 +0100 (CET) Subject: [pypy-svn] r22514 - pypy/extradoc/planning Message-ID: <20060123130453.ADD8527DC6@code1.codespeak.net> Author: bea Date: Mon Jan 23 14:04:52 2006 New Revision: 22514 Modified: pypy/extradoc/planning/sprint-planning.txt Log: Added the idea of the parallell sprint in february, focusing on wp 9 and 10, arranged by Logilab and DFKI (Nicolas) Possible joint sprint with OZ people and Roel Wuyrts (our reviewer from ULB) Modified: pypy/extradoc/planning/sprint-planning.txt ============================================================================== --- pypy/extradoc/planning/sprint-planning.txt (original) +++ pypy/extradoc/planning/sprint-planning.txt Mon Jan 23 14:04:52 2006 @@ -24,6 +24,18 @@ Primary organizer: Holger and Michael +February/Europe +------------------- + +Time & location: ? parallell with PyCon and PyCon sprint (24th February-1 March) +Brussels or Saarbruecken? + +Nature: joint sprint with OZ community and Roel Wuyts/ULB attending ? +DFKI and Logilab primarily since other partners are sprinting +at PyCon and also do the Malaga conference. + +Primary organizer: Nicolas Chauvat + April/Tokyo ------------- From pedronis at codespeak.net Mon Jan 23 14:32:29 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Mon, 23 Jan 2006 14:32:29 +0100 (CET) Subject: [pypy-svn] r22517 - pypy/extradoc/sprintinfo/mallorca Message-ID: <20060123133229.0452727B60@code1.codespeak.net> Author: pedronis Date: Mon Jan 23 14:32:28 2006 New Revision: 22517 Modified: pypy/extradoc/sprintinfo/mallorca/planning.txt Log: more planning and pairing. Added idea about ctypes inspired C glue. Modified: pypy/extradoc/sprintinfo/mallorca/planning.txt ============================================================================== --- pypy/extradoc/sprintinfo/mallorca/planning.txt (original) +++ pypy/extradoc/sprintinfo/mallorca/planning.txt Mon Jan 23 14:32:28 2006 @@ -30,7 +30,7 @@ - JIT: Armin - - stackless application level interface + - stackless application level interface: Christian, Eric - l3interp/offsetof/sizeof work: Michael, Arre, (Samuele) @@ -38,7 +38,10 @@ offsetof work done: Carl Friedrich, Richard - improving the way we interface with external functions. - (also think about how exceptions fit in there) + (also think about how exceptions fit in there): + add static ctypes functionality into RPython itself + Gerald, Stephan (Samuele) + - tutorial about the translator: Armin, Gerald, Stephan, Eric, Christian From mwh at codespeak.net Mon Jan 23 14:42:38 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Mon, 23 Jan 2006 14:42:38 +0100 (CET) Subject: [pypy-svn] r22518 - in pypy/dist/pypy: annotation rpython rpython/lltypesystem rpython/memory rpython/memory/test translator/c Message-ID: <20060123134238.573D227DC4@code1.codespeak.net> Author: mwh Date: Mon Jan 23 14:42:29 2006 New Revision: 22518 Added: pypy/dist/pypy/rpython/lltypesystem/llmemory.py (contents, props changed) Modified: pypy/dist/pypy/annotation/binaryop.py pypy/dist/pypy/annotation/bookkeeper.py pypy/dist/pypy/annotation/builtin.py pypy/dist/pypy/annotation/model.py pypy/dist/pypy/rpython/llinterp.py pypy/dist/pypy/rpython/memory/gc.py pypy/dist/pypy/rpython/memory/lladdress.py pypy/dist/pypy/rpython/memory/lltypelayout.py pypy/dist/pypy/rpython/memory/lltypesimulation.py pypy/dist/pypy/rpython/memory/test/test_address.py pypy/dist/pypy/rpython/memory/test/test_lltypesimulation.py pypy/dist/pypy/rpython/raddress.py pypy/dist/pypy/rpython/rbuiltin.py pypy/dist/pypy/translator/c/primitive.py Log: (arre, mwh) Move Address and OffsetOf and a few other bits from rpython.memory.lladdress into the new file rpython.lltypesystem.llmemory. This probably breaks l3interp as it currently is in the repository, but we're working there anyway. Modified: pypy/dist/pypy/annotation/binaryop.py ============================================================================== --- pypy/dist/pypy/annotation/binaryop.py (original) +++ pypy/dist/pypy/annotation/binaryop.py Mon Jan 23 14:42:29 2006 @@ -736,3 +736,7 @@ class __extend__(pairtype(SomeOffset, SomeOffset)): def add((s_off1, s_off2)): return SomeOffset() + +class __extend__(pairtype(SomeAddress, SomeOffset)): + def add((s_addr, s_off)): + return SomeAddress() Modified: pypy/dist/pypy/annotation/bookkeeper.py ============================================================================== --- pypy/dist/pypy/annotation/bookkeeper.py (original) +++ pypy/dist/pypy/annotation/bookkeeper.py Mon Jan 23 14:42:29 2006 @@ -20,7 +20,7 @@ from pypy.rpython.rarithmetic import r_int, r_uint, r_ulonglong, r_longlong from pypy.rpython.objectmodel import r_dict, Symbolic from pypy.tool.algo.unionfind import UnionFind -from pypy.rpython.lltypesystem import lltype +from pypy.rpython.lltypesystem import lltype, llmemory from pypy.rpython.ootypesystem import ootype from pypy.rpython.memory import lladdress @@ -361,6 +361,8 @@ elif isinstance(x, lladdress.address): assert x is lladdress.NULL result= SomeAddress(is_null=True) + elif isinstance(x, llmemory.fakeaddress): + result = SomeAddress() elif isinstance(x, ootype._static_meth): result = SomeOOStaticMeth(ootype.typeOf(x)) elif isinstance(x, ootype._class): Modified: pypy/dist/pypy/annotation/builtin.py ============================================================================== --- pypy/dist/pypy/annotation/builtin.py (original) +++ pypy/dist/pypy/annotation/builtin.py Mon Jan 23 14:42:29 2006 @@ -504,10 +504,12 @@ #_________________________________ # offsetof/sizeof +from pypy.rpython.lltypesystem import llmemory + def offsetof(TYPE, fldname): return SomeOffset() -BUILTIN_ANALYZERS[lladdress.offsetof] = offsetof +BUILTIN_ANALYZERS[llmemory.offsetof] = offsetof #_________________________________ Modified: pypy/dist/pypy/annotation/model.py ============================================================================== --- pypy/dist/pypy/annotation/model.py (original) +++ pypy/dist/pypy/annotation/model.py Mon Jan 23 14:42:29 2006 @@ -442,7 +442,7 @@ # ____________________________________________________________ # memory addresses -from pypy.rpython.memory import lladdress +from pypy.rpython.lltypesystem import llmemory class SomeAddress(SomeObject): immutable = True @@ -521,8 +521,8 @@ (SomeFloat(), lltype.Float), (SomeChar(), lltype.Char), (SomeUnicodeCodePoint(), lltype.UniChar), - (SomeAddress(), lladdress.Address), - (SomeOffset(), lladdress.Offset), + (SomeAddress(), llmemory.Address), + (SomeOffset(), llmemory.Offset), ] def annotation_to_lltype(s_val, info=None): Modified: pypy/dist/pypy/rpython/llinterp.py ============================================================================== --- pypy/dist/pypy/rpython/llinterp.py (original) +++ pypy/dist/pypy/rpython/llinterp.py Mon Jan 23 14:42:29 2006 @@ -1,6 +1,6 @@ from pypy.objspace.flow.model import FunctionGraph, Constant, Variable, c_last_exception from pypy.rpython.rarithmetic import intmask, r_uint, ovfcheck, r_longlong, r_ulonglong -from pypy.rpython.lltypesystem import lltype +from pypy.rpython.lltypesystem import lltype, llmemory from pypy.rpython.memory import lladdress from pypy.rpython.ootypesystem import ootype from pypy.rpython.objectmodel import FREED_OBJECT @@ -543,12 +543,12 @@ return lladdress.raw_malloc(size) def op_raw_free(self, addr): - assert self.llt.typeOf(addr) == lladdress.Address + assert self.llt.typeOf(addr) == llmemory.Address lladdress.raw_free(addr) def op_raw_memcopy(self, fromaddr, toaddr, size): - assert self.llt.typeOf(fromaddr) == lladdress.Address - assert self.llt.typeOf(toaddr) == lladdress.Address + assert self.llt.typeOf(fromaddr) == llmemory.Address + assert self.llt.typeOf(toaddr) == llmemory.Address lladdress.raw_memcopy(fromaddr, toaddr, size) def op_raw_load(self, addr, typ, offset): Added: pypy/dist/pypy/rpython/lltypesystem/llmemory.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/rpython/lltypesystem/llmemory.py Mon Jan 23 14:42:29 2006 @@ -0,0 +1,79 @@ +# this file contains the definitions and most extremely faked +# implementations of things relating to the description of the layout +# of objects in memeory. + +# sizeof, offsetof + +from pypy.rpython.objectmodel import Symbolic +from pypy.rpython.lltypesystem import lltype + +class OffsetOf(Symbolic): + + def __init__(self, TYPE, *fldnames): + self.TYPE = TYPE + self.fldnames = fldnames + + def annotation(self): + from pypy.annotation import model + return model.SomeOffset() + + def lltype(self): + return Offset + + def __repr__(self): + return "" % (self.TYPE, self.fldnames) + + def __add__(self, other): + if not isinstance(other, OffsetOf): + return NotImplemented + t = self.TYPE + for f in self.fldnames: + t = t._flds[f] + assert t == other.TYPE + return OffsetOf(self.TYPE, *(self.fldnames + other.fldnames)) + +Offset = lltype.Primitive("Offset", OffsetOf(lltype.Void)) + +def sizeof(TYPE, n=None): + pass + +def offsetof(TYPE, fldname): + assert fldname in TYPE._flds + return OffsetOf(TYPE, fldname) + +def itemoffsetof(TYPE, n=None): + pass + +class fakeaddress(object): + def __init__(self, ob, offset=None): + self.ob = ob + if offset is None: + self.offset = OffsetOf(lltype.typeOf(self.ob)) + else: + self.offset = offset + + def __add__(self, other): + if not isinstance(other, OffsetOf): + return NotImplemented + return fakeaddress(self.ob, self.offset + other) + +class _fakeaccessor(object): + def __init__(self, addr): + self.addr = addr + def __getitem__(self, index): + assert index == 0 + ob = self.addr.ob + for n in self.addr.offset.fldnames: + ob = getattr(ob, n) + # XXX will need to do pointers differently! + assert lltype.typeOf(ob) == self.TYPE + return ob + +class _signed_fakeaccessor(_fakeaccessor): + TYPE = lltype.Signed + +fakeaddress.signed = property(_signed_fakeaccessor) + +Address = lltype.Primitive("Address", fakeaddress(None)) + +fakeaddress._TYPE = Address Modified: pypy/dist/pypy/rpython/memory/gc.py ============================================================================== --- pypy/dist/pypy/rpython/memory/gc.py (original) +++ pypy/dist/pypy/rpython/memory/gc.py Mon Jan 23 14:42:29 2006 @@ -1,8 +1,8 @@ from pypy.rpython.memory.lladdress import raw_malloc, raw_free, raw_memcopy -from pypy.rpython.memory.lladdress import NULL, address, Address +from pypy.rpython.memory.lladdress import NULL, address from pypy.rpython.memory.support import AddressLinkedList from pypy.rpython.memory import lltypesimulation -from pypy.rpython.lltypesystem import lltype +from pypy.rpython.lltypesystem import lltype, llmemory from pypy.rpython.objectmodel import free_non_gc_object import sys @@ -27,7 +27,7 @@ gc_interface = { "malloc": lltype.FuncType((lltype.Signed, lltype.Signed), lltype.Signed), "collect": lltype.FuncType((), lltype.Void), - "write_barrier": lltype.FuncType((Address, ) * 3, lltype.Void), + "write_barrier": lltype.FuncType((llmemory.Address, ) * 3, lltype.Void), } def dummy_get_roots1(): Modified: pypy/dist/pypy/rpython/memory/lladdress.py ============================================================================== --- pypy/dist/pypy/rpython/memory/lladdress.py (original) +++ pypy/dist/pypy/rpython/memory/lladdress.py Mon Jan 23 14:42:29 2006 @@ -1,7 +1,8 @@ import struct from pypy.rpython.memory.simulator import MemorySimulator, MemorySimulatorError from pypy.rpython.rarithmetic import r_uint - +from pypy.rpython.lltypesystem import llmemory +from pypy.rpython.lltypesystem import lltype class address(object): def __new__(cls, intaddress=0): @@ -108,85 +109,10 @@ def get_py_object(address): return simulator.get_py_object(address.intaddress) - -from pypy.rpython.lltypesystem import lltype -Address = lltype.Primitive("Address", NULL) - -address._TYPE = Address +address._TYPE = llmemory.Address supported_access_types = {"signed": lltype.Signed, "unsigned": lltype.Unsigned, "char": lltype.Char, - "address": Address, + "address": llmemory.Address, } - -# sizeof, offsetof - -from pypy.rpython.objectmodel import Symbolic - -class OffsetOf(Symbolic): - - def __init__(self, TYPE, *fldnames): - self.TYPE = TYPE - self.fldnames = fldnames - - def annotation(self): - from pypy.annotation import model - return model.SomeOffset() - - def lltype(self): - return Offset - - def __repr__(self): - return "" % (self.TYPE, self.fldnames) - - def __add__(self, other): - if not isinstance(other, OffsetOf): - return NotImplemented - t = self.TYPE - for f in self.fldnames: - t = t._flds[f] - assert t == other.TYPE - return OffsetOf(self.TYPE, *(self.fldnames + other.fldnames)) - -Offset = lltype.Primitive("Offset", OffsetOf(lltype.Void)) - -def sizeof(TYPE, n=None): - pass - -def offsetof(TYPE, fldname): - assert fldname in TYPE._flds - return OffsetOf(TYPE, fldname) - -def itemoffsetof(TYPE, n=None): - pass - -class fakeaddress(object): - def __init__(self, ob, offset=None): - self.ob = ob - if offset is None: - self.offset = OffsetOf(self.ob._TYPE) - else: - self.offset = offset - - def __add__(self, other): - if not isinstance(other, OffsetOf): - return NotImplemented - return fakeaddress(self.ob, self.offset + other) - -class _fakeaccessor(object): - def __init__(self, addr): - self.addr = addr - def __getitem__(self, index): - assert index == 0 - ob = self.addr.ob - for n in self.addr.offset.fldnames: - ob = getattr(ob, n) - # XXX will need to do pointers differently! - assert lltype.typeOf(ob) == self.TYPE - return ob - -class _signed_fakeaccessor(_fakeaccessor): - TYPE = lltype.Signed - -fakeaddress.signed = property(_signed_fakeaccessor) Modified: pypy/dist/pypy/rpython/memory/lltypelayout.py ============================================================================== --- pypy/dist/pypy/rpython/memory/lltypelayout.py (original) +++ pypy/dist/pypy/rpython/memory/lltypelayout.py Mon Jan 23 14:42:29 2006 @@ -1,5 +1,4 @@ -from pypy.rpython.lltypesystem import lltype -from pypy.rpython.memory import lladdress +from pypy.rpython.lltypesystem import lltype, llmemory import struct @@ -8,7 +7,7 @@ lltype.Char: "c", lltype.Bool: "B", lltype.Float: "d", - lladdress.Address: "P", + llmemory.Address: "P", } Modified: pypy/dist/pypy/rpython/memory/lltypesimulation.py ============================================================================== --- pypy/dist/pypy/rpython/memory/lltypesimulation.py (original) +++ pypy/dist/pypy/rpython/memory/lltypesimulation.py Mon Jan 23 14:42:29 2006 @@ -4,7 +4,7 @@ from pypy.rpython.memory.lltypelayout import get_variable_size, sizeof from pypy.rpython.memory.lltypelayout import primitive_to_fmt from pypy.rpython.memory import lladdress -from pypy.rpython.lltypesystem import lltype +from pypy.rpython.lltypesystem import lltype, llmemory log = py.log.Producer("lltypesim") @@ -15,7 +15,7 @@ return simulatorptr(lltype.Ptr(T), address) elif T == lltype.Bool: return bool(address._load(primitive_to_fmt[T])[0]) - elif T == lladdress.Address: + elif T == llmemory.Address: return (self._address + offset).address[0] elif isinstance(T, lltype.Primitive): return address._load(primitive_to_fmt[T])[0] @@ -63,7 +63,7 @@ if isinstance(T, lltype.Primitive): if T == lltype.Void: return None - elif T == lladdress.Address: + elif T == llmemory.Address: return (self._address + offset).address[0] res = (self._address + offset)._load(primitive_to_fmt[T])[0] if T == lltype.Bool: @@ -92,7 +92,7 @@ if isinstance(T, lltype.Primitive): if T == lltype.Void: return - if T == lladdress.Address: + if T == llmemory.Address: (self._address + offset).address[0] = value else: (self._address + offset)._store(primitive_to_fmt[T], Modified: pypy/dist/pypy/rpython/memory/test/test_address.py ============================================================================== --- pypy/dist/pypy/rpython/memory/test/test_address.py (original) +++ pypy/dist/pypy/rpython/memory/test/test_address.py Mon Jan 23 14:42:29 2006 @@ -9,7 +9,7 @@ from pypy.rpython.memory.lladdress import address, NULL from pypy.rpython.memory.lladdress import raw_malloc, raw_free, raw_memcopy from pypy.rpython.memory.lladdress import get_py_object, get_address_of_object -from pypy.rpython.memory.lladdress import Address +from pypy.rpython.lltypesystem.llmemory import Address from pypy.rpython.memory.simulator import MemorySimulatorError from pypy.rpython.memory.test.test_llinterpsim import interpret @@ -114,7 +114,7 @@ def test_simple_offsetof(self): from pypy.rpython.lltypesystem import lltype - from pypy.rpython.memory.lladdress import offsetof + from pypy.rpython.lltypesystem.llmemory import offsetof S = lltype.GcStruct('S', ('x', lltype.Bool), ('y', lltype.Signed)) def f(): return offsetof(S, 'x') @@ -133,7 +133,7 @@ def test_offset_addition(self): from pypy.rpython.lltypesystem import lltype - from pypy.rpython.memory.lladdress import offsetof + from pypy.rpython.lltypesystem.llmemory import offsetof S = lltype.Struct('S', ('x', lltype.Bool), ('y', lltype.Signed)) T = lltype.GcStruct('T', ('r', lltype.Float), ('s1', S), ('s2', S)) def f(): @@ -244,7 +244,7 @@ def test_simple_offsetof(self): from pypy.rpython.lltypesystem import lltype - from pypy.rpython.memory.lladdress import offsetof + from pypy.rpython.lltypesystem.llmemory import offsetof S = lltype.GcStruct('S', ('x', lltype.Bool), ('y', lltype.Signed)) def f(): return offsetof(S, 'x') @@ -265,7 +265,7 @@ def test_offset_addition(self): from pypy.rpython.lltypesystem import lltype - from pypy.rpython.memory.lladdress import offsetof + from pypy.rpython.lltypesystem.llmemory import offsetof S = lltype.Struct('S', ('x', lltype.Bool), ('y', lltype.Signed)) T = lltype.GcStruct('T', ('r', lltype.Float), ('s1', S), ('s2', S)) def f(): Modified: pypy/dist/pypy/rpython/memory/test/test_lltypesimulation.py ============================================================================== --- pypy/dist/pypy/rpython/memory/test/test_lltypesimulation.py (original) +++ pypy/dist/pypy/rpython/memory/test/test_lltypesimulation.py Mon Jan 23 14:42:29 2006 @@ -331,7 +331,7 @@ assert not s0.a != s0.a def test_struct_with_address(): - S = lltype.GcStruct("s", ('a', lladdress.Address)) + S = lltype.GcStruct("s", ('a', llmemory.Address)) s = malloc(S) s.a = lladdress.NULL assert s.a == lladdress.NULL Modified: pypy/dist/pypy/rpython/raddress.py ============================================================================== --- pypy/dist/pypy/rpython/raddress.py (original) +++ pypy/dist/pypy/rpython/raddress.py Mon Jan 23 14:42:29 2006 @@ -1,7 +1,8 @@ # rtyping of memory address operations from pypy.annotation.pairtype import pairtype from pypy.annotation import model as annmodel -from pypy.rpython.memory.lladdress import NULL, Address, Offset +from pypy.rpython.memory.lladdress import NULL, address +from pypy.rpython.lltypesystem.llmemory import Address, Offset from pypy.rpython.rmodel import Repr, IntegerRepr from pypy.rpython.lltypesystem import lltype @@ -23,8 +24,8 @@ lowleveltype = Address def convert_const(self, value): - assert value is NULL - return NULL + assert not isinstance(value, address) or value is NULL + return value def rtype_getattr(self, hop): v_access = hop.inputarg(address_repr, 0) Modified: pypy/dist/pypy/rpython/rbuiltin.py ============================================================================== --- pypy/dist/pypy/rpython/rbuiltin.py (original) +++ pypy/dist/pypy/rpython/rbuiltin.py Mon Jan 23 14:42:29 2006 @@ -361,17 +361,18 @@ # memory addresses from pypy.rpython.memory import lladdress +from pypy.rpython.lltypesystem import llmemory def rtype_raw_malloc(hop): v_size, = hop.inputargs(lltype.Signed) - return hop.genop('raw_malloc', [v_size], resulttype=lladdress.Address) + return hop.genop('raw_malloc', [v_size], resulttype=llmemory.Address) def rtype_raw_free(hop): - v_addr, = hop.inputargs(lladdress.Address) + v_addr, = hop.inputargs(llmemory.Address) return hop.genop('raw_free', [v_addr]) def rtype_raw_memcopy(hop): - v_list = hop.inputargs(lladdress.Address, lladdress.Address, lltype.Signed) + v_list = hop.inputargs(llmemory.Address, llmemory.Address, lltype.Signed) return hop.genop('raw_memcopy', v_list) BUILTIN_TYPER[lladdress.raw_malloc] = rtype_raw_malloc @@ -379,11 +380,11 @@ BUILTIN_TYPER[lladdress.raw_memcopy] = rtype_raw_memcopy def rtype_offsetof(hop): - from pypy.rpython.memory.lladdress import Offset TYPE, field = hop.inputargs(lltype.Void, lltype.Void) - return hop.inputconst(Offset, lladdress.offsetof(TYPE.value, field.value)) + return hop.inputconst(llmemory.Offset, + llmemory.offsetof(TYPE.value, field.value)) -BUILTIN_TYPER[lladdress.offsetof] = rtype_offsetof +BUILTIN_TYPER[llmemory.offsetof] = rtype_offsetof # _________________________________________________________________ # non-gc objects Modified: pypy/dist/pypy/translator/c/primitive.py ============================================================================== --- pypy/dist/pypy/translator/c/primitive.py (original) +++ pypy/dist/pypy/translator/c/primitive.py Mon Jan 23 14:42:29 2006 @@ -1,6 +1,7 @@ import sys from pypy.rpython.lltypesystem.lltype import * -from pypy.rpython.memory.lladdress import Address, NULL +from pypy.rpython.lltypesystem.llmemory import Address +from pypy.rpython.memory.lladdress import NULL # ____________________________________________________________ # @@ -53,6 +54,7 @@ return '%d' % ord(value) def name_address(value): + # XXX this will need to change! assert value == NULL return 'NULL' From stephan at codespeak.net Mon Jan 23 14:49:39 2006 From: stephan at codespeak.net (stephan at codespeak.net) Date: Mon, 23 Jan 2006 14:49:39 +0100 (CET) Subject: [pypy-svn] r22519 - pypy/dist/demo Message-ID: <20060123134939.B583527DC4@code1.codespeak.net> Author: stephan Date: Mon Jan 23 14:49:38 2006 New Revision: 22519 Added: pypy/dist/demo/test_ignore.txt Log: just testing subversion account Added: pypy/dist/demo/test_ignore.txt ============================================================================== From stephan at codespeak.net Mon Jan 23 14:50:23 2006 From: stephan at codespeak.net (stephan at codespeak.net) Date: Mon, 23 Jan 2006 14:50:23 +0100 (CET) Subject: [pypy-svn] r22520 - pypy/dist/demo Message-ID: <20060123135023.470BC27DC4@code1.codespeak.net> Author: stephan Date: Mon Jan 23 14:50:21 2006 New Revision: 22520 Removed: pypy/dist/demo/test_ignore.txt Log: From rxe at codespeak.net Mon Jan 23 14:53:00 2006 From: rxe at codespeak.net (rxe at codespeak.net) Date: Mon, 23 Jan 2006 14:53:00 +0100 (CET) Subject: [pypy-svn] r22521 - in pypy/dist/pypy/rpython/memory: . test Message-ID: <20060123135300.4F74F27DC4@code1.codespeak.net> Author: rxe Date: Mon Jan 23 14:52:58 2006 New Revision: 22521 Modified: pypy/dist/pypy/rpython/memory/lladdress.py pypy/dist/pypy/rpython/memory/lltypelayout.py pypy/dist/pypy/rpython/memory/test/test_address.py Log: (cfbolz, rxe) Added convert_offset_to_int() and fixed up address.__add__ to deal with OffsetsOf. Modified: pypy/dist/pypy/rpython/memory/lladdress.py ============================================================================== --- pypy/dist/pypy/rpython/memory/lladdress.py (original) +++ pypy/dist/pypy/rpython/memory/lladdress.py Mon Jan 23 14:52:58 2006 @@ -5,6 +5,8 @@ from pypy.rpython.lltypesystem import lltype class address(object): + #XXX should be _address! + def __new__(cls, intaddress=0): if intaddress == 0: null = cls.__dict__.get("NULL") @@ -22,13 +24,18 @@ return self.intaddress def __add__(self, offset): - assert isinstance(offset, int) - return address(self.intaddress + offset) + from pypy.rpython.memory.lltypelayout import convert_offset_to_int + if isinstance(offset, int): + return address(self.intaddress + offset) + else: + assert isinstance(offset, llmemory.OffsetOf) + return address(self.intaddress + convert_offset_to_int(offset)) def __sub__(self, other): if isinstance(other, int): return address(self.intaddress - other) else: + assert isinstance(other, address) return self.intaddress - other.intaddress def __cmp__(self, other): Modified: pypy/dist/pypy/rpython/memory/lltypelayout.py ============================================================================== --- pypy/dist/pypy/rpython/memory/lltypelayout.py (original) +++ pypy/dist/pypy/rpython/memory/lltypelayout.py Mon Jan 23 14:52:58 2006 @@ -87,7 +87,14 @@ else: return fixedsize + i * varsize - +def convert_offset_to_int(offset): + TYPE = offset.TYPE + res = 0 + for fld in offset.fldnames: + layout = get_layout(TYPE) + res += layout[fld] + TYPE = getattr(TYPE, fld) + return res # _____________________________________________________________________________ # the following functions are used to find contained pointers Modified: pypy/dist/pypy/rpython/memory/test/test_address.py ============================================================================== --- pypy/dist/pypy/rpython/memory/test/test_address.py (original) +++ pypy/dist/pypy/rpython/memory/test/test_address.py Mon Jan 23 14:52:58 2006 @@ -12,6 +12,7 @@ from pypy.rpython.lltypesystem.llmemory import Address from pypy.rpython.memory.simulator import MemorySimulatorError from pypy.rpython.memory.test.test_llinterpsim import interpret +from pypy.rpython.lltypesystem import lltype class TestAddressAnnotation(object): def test_null(self): @@ -411,3 +412,11 @@ assert get_py_object(addr1.address[0])(0) == 1 assert (addr1 + 10).signed[0] == 42 assert (addr1 + 20).char[0] == "a" + + def test_add_offsetofs(self): + from pypy.rpython.lltypesystem.llmemory import offsetof + S = lltype.GcStruct("struct", ('a', lltype.Signed), ('b', lltype.Signed)) + addr = raw_malloc(100) + (addr + offsetof(S, 'b')).signed[0] = 42 + assert (addr + offsetof(S, 'b')).signed[0] == 42 + From mwh at codespeak.net Mon Jan 23 15:05:47 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Mon, 23 Jan 2006 15:05:47 +0100 (CET) Subject: [pypy-svn] r22522 - pypy/dist/pypy/rpython Message-ID: <20060123140547.DE9E127B84@code1.codespeak.net> Author: mwh Date: Mon Jan 23 15:05:46 2006 New Revision: 22522 Modified: pypy/dist/pypy/rpython/raddress.py Log: rtype the addition of an address and an offset Modified: pypy/dist/pypy/rpython/raddress.py ============================================================================== --- pypy/dist/pypy/rpython/raddress.py (original) +++ pypy/dist/pypy/rpython/raddress.py Mon Jan 23 15:05:46 2006 @@ -128,3 +128,8 @@ def rtype_add((r_offset1, r_offset2), hop): v_offset1, v_offset2 = hop.inputargs(Offset, Offset) return hop.genop('offset_add', [v_offset1, v_offset2], resulttype=Offset) + +class __extend__(pairtype(AddressRepr, OffsetRepr)): + def rtype_add((r_offset1, r_offset2), hop): + v_offset1, v_offset2 = hop.inputargs(Address, Offset) + return hop.genop('adr_offset_add', [v_offset1, v_offset2], resulttype=Address) From mwh at codespeak.net Mon Jan 23 15:24:46 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Mon, 23 Jan 2006 15:24:46 +0100 (CET) Subject: [pypy-svn] r22524 - in pypy/dist/pypy: rpython/l3interp rpython/l3interp/test translator/c translator/c/src Message-ID: <20060123142446.400AD27DC4@code1.codespeak.net> Author: mwh Date: Mon Jan 23 15:24:41 2006 New Revision: 22524 Modified: pypy/dist/pypy/rpython/l3interp/convertgraph.py pypy/dist/pypy/rpython/l3interp/l3interp.py pypy/dist/pypy/rpython/l3interp/model.py pypy/dist/pypy/rpython/l3interp/test/test_convert.py pypy/dist/pypy/translator/c/database.py pypy/dist/pypy/translator/c/primitive.py pypy/dist/pypy/translator/c/src/address.h Log: (arre, mwh) some support in genc for offsets. reenable the test from test_convert and the getfield_int l3 operation. Modified: pypy/dist/pypy/rpython/l3interp/convertgraph.py ============================================================================== --- pypy/dist/pypy/rpython/l3interp/convertgraph.py (original) +++ pypy/dist/pypy/rpython/l3interp/convertgraph.py Mon Jan 23 15:24:41 2006 @@ -3,7 +3,7 @@ from pypy.rpython.l3interp.model import Op from pypy.objspace.flow import model as flowmodel from pypy.rpython.lltypesystem import lltype -from pypy.rpython.memory.lladdress import OffsetOf, fakeaddress +from pypy.rpython.lltypesystem.llmemory import OffsetOf, fakeaddress class LL2L3Converter(object): Modified: pypy/dist/pypy/rpython/l3interp/l3interp.py ============================================================================== --- pypy/dist/pypy/rpython/l3interp/l3interp.py (original) +++ pypy/dist/pypy/rpython/l3interp/l3interp.py Mon Jan 23 15:24:41 2006 @@ -2,6 +2,8 @@ from pypy.rpython.memory import lladdress from pypy.rpython.rarithmetic import r_uint from pypy.interpreter.miscutils import InitializedClass +from pypy.rpython.lltypesystem.llmemory import fakeaddress, OffsetOf +from pypy.rpython.lltypesystem import lltype class L3Exception(Exception): pass @@ -40,11 +42,20 @@ return L3Pointer(args_ptr.pop()) raise AssertionError("stacks corrupted") +constant_offset = OffsetOf(lltype.Void) +constant_fakeaddress = fakeaddress(None) + class L3Frame(object): def __init__(self, graph, stack_int, stack_dbl, stack_ptr): self.graph = graph self.block = self.graph.startblock + if self.block.constants_ptr is None: + self.block.constants_ptr = [constant_fakeaddress] + self.block.constants_ptr = None + if self.block.constants_offset is None: + self.block.constants_offset = [constant_offset] + self.block.constants_offset = None self.i = 0 self.stack_int = stack_int self.stack_dbl = stack_dbl @@ -140,7 +151,7 @@ self.stack_dbl.append(x) raise L3Return - def op_ptr_return(self): + def op_adr_return(self): x = self.getptr() self.restorestacks() self.stack_ptr.append(x) @@ -169,11 +180,14 @@ else: self.stack_int.append(0) - def later_op_getfield_int(self): + def op_getfield_int(self): p = self.getptr() o = self.getoffset() self.stack_int.append((p + o).signed[0]) + def op_flavored_malloc(self): + self.stack_ptr.append(constant_fakeaddress) + def op_direct_call(self): block = self.block assert block.called_graphs is not None Modified: pypy/dist/pypy/rpython/l3interp/model.py ============================================================================== --- pypy/dist/pypy/rpython/l3interp/model.py (original) +++ pypy/dist/pypy/rpython/l3interp/model.py Mon Jan 23 15:24:41 2006 @@ -1,6 +1,5 @@ -from pypy.rpython.memory import lladdress from pypy.objspace.flow import model as flowmodel -from pypy.rpython.lltypesystem import lltype +from pypy.rpython.lltypesystem import lltype, llmemory very_low_level_ops = [ 'nop', @@ -87,7 +86,7 @@ primitives = [lltype.Signed, lltype.Unsigned, lltype.Float, lltype.Char, - lltype.UniChar, lladdress.Address, lltype.Void] + lltype.UniChar, llmemory.Address, lltype.Void] primitive_to_number = {} for i, p in enumerate(primitives): @@ -113,6 +112,7 @@ self.constants_int = constants_int self.constants_dbl = constants_dbl self.constants_ptr = constants_ptr + self.constants_offset = constants_offset self.called_graphs = called_graphs Modified: pypy/dist/pypy/rpython/l3interp/test/test_convert.py ============================================================================== --- pypy/dist/pypy/rpython/l3interp/test/test_convert.py (original) +++ pypy/dist/pypy/rpython/l3interp/test/test_convert.py Mon Jan 23 15:24:41 2006 @@ -40,7 +40,7 @@ assert isinstance(result, l3interp.L3Integer) assert result.intval == 1 -def dont_test_convert_getfield(): +def test_convert_getfield(): class C: def __init__(self, x): self.x = x Modified: pypy/dist/pypy/translator/c/database.py ============================================================================== --- pypy/dist/pypy/translator/c/database.py (original) +++ pypy/dist/pypy/translator/c/database.py Mon Jan 23 15:24:41 2006 @@ -2,6 +2,8 @@ Primitive, Ptr, typeOf, RuntimeTypeInfo, \ Struct, Array, FuncType, PyObject, Void, \ ContainerType, OpaqueType +from pypy.rpython.lltypesystem.llmemory import Offset, Address +from pypy.rpython.memory.lladdress import NULL from pypy.translator.c.primitive import PrimitiveName, PrimitiveType from pypy.translator.c.primitive import PrimitiveErrorValue from pypy.translator.c.node import StructDefNode, ArrayDefNode @@ -119,7 +121,19 @@ raise Exception("don't know about %r" % (T,)) else: T = typeOf(obj) - if isinstance(T, Primitive): + if T is Offset: + assert len(obj.fldnames) <= 1 + if len(obj.fldnames) == 0: + return '0 /*offsetof*/' + else: + return 'offsetof(%s, %s)'%( + self.gettype(obj.TYPE), obj.fldnames[0]) + elif T is Address and obj is not NULL: + if obj.ob is None: + return 'NULL' + else: + return self.get(obj.ob) + elif isinstance(T, Primitive): return PrimitiveName[T](obj) elif isinstance(T, Ptr): if obj: # test if the ptr is non-NULL Modified: pypy/dist/pypy/translator/c/primitive.py ============================================================================== --- pypy/dist/pypy/translator/c/primitive.py (original) +++ pypy/dist/pypy/translator/c/primitive.py Mon Jan 23 15:24:41 2006 @@ -1,6 +1,6 @@ import sys from pypy.rpython.lltypesystem.lltype import * -from pypy.rpython.lltypesystem.llmemory import Address +from pypy.rpython.lltypesystem.llmemory import Address, Offset from pypy.rpython.memory.lladdress import NULL # ____________________________________________________________ @@ -58,7 +58,6 @@ assert value == NULL return 'NULL' - PrimitiveName = { Signed: name_signed, SignedLongLong: name_signedlonglong, @@ -83,6 +82,7 @@ Bool: 'char @', Void: 'void @', Address: 'void* @', + Offset: 'long @', } PrimitiveErrorValue = { @@ -96,4 +96,5 @@ Bool: '((char) -1)', Void: '/* error */', Address: 'NULL', + Offset: '0', } Modified: pypy/dist/pypy/translator/c/src/address.h ============================================================================== --- pypy/dist/pypy/translator/c/src/address.h (original) +++ pypy/dist/pypy/translator/c/src/address.h Mon Jan 23 15:24:41 2006 @@ -8,6 +8,7 @@ #define OP_ADR_DELTA(x,y,r,err) r = ((char *)(x) - (char *)(y)) #define OP_ADR_SUB(x,y,r,err) r = ((char *)(x) - (y)) #define OP_ADR_ADD(x,y,r,err) r = ((char *)(x) + (y)) +#define OP_ADR_OFFSET_ADD(x,y,r,err) r = ((char *)(x) + (y)) #define OP_ADR_EQ(x,y,r,err) r = ((x) == (y)) #define OP_ADR_NE(x,y,r,err) r = ((x) != (y)) @@ -30,3 +31,4 @@ #define OP_RAW_FREE(x,r,err) free(x); #define OP_RAW_MEMCOPY(x,y,size,r,err) memcpy(y,x,size); + From rxe at codespeak.net Mon Jan 23 15:51:30 2006 From: rxe at codespeak.net (rxe at codespeak.net) Date: Mon, 23 Jan 2006 15:51:30 +0100 (CET) Subject: [pypy-svn] r22525 - in pypy/dist/pypy/rpython/memory: . test Message-ID: <20060123145130.72D3B27DC4@code1.codespeak.net> Author: rxe Date: Mon Jan 23 15:51:28 2006 New Revision: 22525 Modified: pypy/dist/pypy/rpython/memory/lladdress.py pypy/dist/pypy/rpython/memory/test/test_address.py Log: (cfbolz, rxe): made addr.signed[0] = offset convert offset to an int correctly Modified: pypy/dist/pypy/rpython/memory/lladdress.py ============================================================================== --- pypy/dist/pypy/rpython/memory/lladdress.py (original) +++ pypy/dist/pypy/rpython/memory/lladdress.py Mon Jan 23 15:51:28 2006 @@ -48,6 +48,9 @@ return simulator.getstruct(fmt, self.intaddress) def _store(self, fmt, *values): + # XXX annoyance: suddenly an OffsetOf changes into a Signed?! + if len(values) == 1 and isinstance(values[0], llmemory.OffsetOf): + values = [convert_offset_to_int(values[0])] simulator.setstruct(fmt, self.intaddress, *values) def __nonzero__(self): @@ -67,12 +70,19 @@ def __setitem__(self, offset, value): simulator.setstruct(self.format, self.intaddress + offset * self.size, self.convert_to(value)) - + + class _signed_accessor(_accessor): format = "l" size = struct.calcsize("l") convert_from = int - convert_to = int + + def convert_to(self, offset): + from pypy.rpython.memory.lltypelayout import convert_offset_to_int + # XXX same annoyance as in _store + if isinstance(offset, llmemory.OffsetOf): + return convert_offset_to_int(offset) + return int(offset) class _unsigned_accessor(_accessor): format = "L" Modified: pypy/dist/pypy/rpython/memory/test/test_address.py ============================================================================== --- pypy/dist/pypy/rpython/memory/test/test_address.py (original) +++ pypy/dist/pypy/rpython/memory/test/test_address.py Mon Jan 23 15:51:28 2006 @@ -419,4 +419,6 @@ addr = raw_malloc(100) (addr + offsetof(S, 'b')).signed[0] = 42 assert (addr + offsetof(S, 'b')).signed[0] == 42 - + addr.signed[5] = offsetof(S, 'b') + offset = addr.signed[5] + assert (addr + offset).signed[0] == 42 From stephan at codespeak.net Mon Jan 23 15:54:32 2006 From: stephan at codespeak.net (stephan at codespeak.net) Date: Mon, 23 Jan 2006 15:54:32 +0100 (CET) Subject: [pypy-svn] r22526 - in pypy/dist/pypy/rpython/rctypes: . test Message-ID: <20060123145432.79B2627DC4@code1.codespeak.net> Author: stephan Date: Mon Jan 23 15:54:30 2006 New Revision: 22526 Added: pypy/dist/pypy/rpython/rctypes/ pypy/dist/pypy/rpython/rctypes/__init__.py pypy/dist/pypy/rpython/rctypes/interface.py pypy/dist/pypy/rpython/rctypes/test/ pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py Log: added first tests for rctypes implementation. Only DUMMY code at the moment. Added: pypy/dist/pypy/rpython/rctypes/__init__.py ============================================================================== Added: pypy/dist/pypy/rpython/rctypes/interface.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/rpython/rctypes/interface.py Mon Jan 23 15:54:30 2006 @@ -0,0 +1,3 @@ +from ctypes import * + + Added: pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py Mon Jan 23 15:54:30 2006 @@ -0,0 +1,22 @@ +import py.test + +def setup_module(mod): + try: + import ctypes + except ImportError: + py.test.skip("this test needs ctypes installed") + +class Test_rctypes: + def test_simple(self): + from pypy.rpython.rctypes.interface import windll, c_char_p + + import sys + if sys.platform == 'win32': + py.test.raises(WindowsError,"windll.LoadLibrary('kernel42.dll')") + mylib = windll.LoadLibrary('kernel32.dll') + gcl = mylib.GetCommandLineA + gcl.restype = c_char_p + def tst(): + return gcl() + res = tst() + assert isinstance(res, str) From stephan at codespeak.net Mon Jan 23 16:35:54 2006 From: stephan at codespeak.net (stephan at codespeak.net) Date: Mon, 23 Jan 2006 16:35:54 +0100 (CET) Subject: [pypy-svn] r22532 - in pypy/dist/pypy/rpython/rctypes: . test Message-ID: <20060123153554.E757227DC4@code1.codespeak.net> Author: stephan Date: Mon Jan 23 16:35:52 2006 New Revision: 22532 Modified: pypy/dist/pypy/rpython/rctypes/interface.py pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py Log: extended the rctype test to something usable on Linux and Windows Modified: pypy/dist/pypy/rpython/rctypes/interface.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/interface.py (original) +++ pypy/dist/pypy/rpython/rctypes/interface.py Mon Jan 23 16:35:52 2006 @@ -1,3 +1,47 @@ from ctypes import * +""" +Loading +------- +windll.LoadLibrary() (<- Windows calling conventions) +cdll.LoadLibrary() (<- Unix calling conventions) + +Types +----- + +c_char +c_byte +c_ubyte +c_short +c_ushort +c_int +c_uint +c_long +c_ulong +c_longlong +c_ulonglong +c_float +c_double +c_char_p +c_wchar_p +c_void_p + +Function Interface +------------------ + +somefunc.restype = c_char +somefunc.argtypes = [c_char,c_int] + +Structure +--------- + +class POINT(Structure): + _fields_ = [("x", c_int), + ("y", c_int)] + +Arrays +------ + +TenPointsArray = POINT * 10 +""" Modified: pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py Mon Jan 23 16:35:52 2006 @@ -8,15 +8,22 @@ class Test_rctypes: def test_simple(self): - from pypy.rpython.rctypes.interface import windll, c_char_p + from pypy.rpython.rctypes.interface import cdll, c_char_p, c_int import sys if sys.platform == 'win32': - py.test.raises(WindowsError,"windll.LoadLibrary('kernel42.dll')") - mylib = windll.LoadLibrary('kernel32.dll') - gcl = mylib.GetCommandLineA - gcl.restype = c_char_p - def tst(): - return gcl() - res = tst() - assert isinstance(res, str) + mylib = cdll.LoadLibrary('msvcrt.dll') + elif sys.platform == 'linux2': + mylib = cdll.LoadLibrary('libc') + else: + py.test.skip("don't know how to load the c lib for %s" % + sys.platform) + + atoi = mylib.atoi + atoi.restype = c_int + atoi.argstype = [c_char_p] + def o_atoi(a): + return atoi(a) + + res = o_atoi('42') + assert res == 42 From ludal at codespeak.net Mon Jan 23 16:42:55 2006 From: ludal at codespeak.net (ludal at codespeak.net) Date: Mon, 23 Jan 2006 16:42:55 +0100 (CET) Subject: [pypy-svn] r22533 - in pypy/branch/ast-experiments/pypy: interpreter/pyparser interpreter/pyparser/test translator/goal Message-ID: <20060123154255.3BBD827DC4@code1.codespeak.net> Author: ludal Date: Mon Jan 23 16:42:50 2006 New Revision: 22533 Added: pypy/branch/ast-experiments/pypy/translator/goal/targetebnfparser.py (contents, props changed) Modified: pypy/branch/ast-experiments/pypy/interpreter/pyparser/ebnfgrammar.py pypy/branch/ast-experiments/pypy/interpreter/pyparser/ebnflexer.py pypy/branch/ast-experiments/pypy/interpreter/pyparser/ebnfparse.py pypy/branch/ast-experiments/pypy/interpreter/pyparser/grammar.py pypy/branch/ast-experiments/pypy/interpreter/pyparser/pysymbol.py pypy/branch/ast-experiments/pypy/interpreter/pyparser/pythonlexer.py pypy/branch/ast-experiments/pypy/interpreter/pyparser/pythonutil.py pypy/branch/ast-experiments/pypy/interpreter/pyparser/pytoken.py pypy/branch/ast-experiments/pypy/interpreter/pyparser/test/test_astcompiler.py pypy/branch/ast-experiments/pypy/interpreter/pyparser/tuplebuilder.py Log: (adim,ludal) a new (not yet annotatable) EBNFParser that will build grammar parsers at runtime Modified: pypy/branch/ast-experiments/pypy/interpreter/pyparser/ebnfgrammar.py ============================================================================== --- pypy/branch/ast-experiments/pypy/interpreter/pyparser/ebnfgrammar.py (original) +++ pypy/branch/ast-experiments/pypy/interpreter/pyparser/ebnfgrammar.py Mon Jan 23 16:42:50 2006 @@ -17,16 +17,15 @@ _count += 1 sym_map[val] = name sym_rmap[name] = val + globals()[name] = val return val tok_map = {} tok_rmap = {} -def g_add_token( **kwargs ): +def g_add_token(sym, name): global _count - assert len(kwargs) == 1 - sym, name = kwargs.popitem() if name in tok_rmap: return tok_rmap[name] val = _count @@ -35,13 +34,18 @@ tok_rmap[name] = val sym_map[val] = sym sym_rmap[sym] = val + globals()[sym] = val return val -g_add_token( EOF='EOF' ) + + +g_add_token('EOF', 'EOF') + def grammar_grammar(): - """NOT RPYTHON (mostly because of g_add_token I suppose) + """ + (mostly because of g_add_token I suppose) Builds the grammar for the grammar file Here's the description of the grammar's grammar :: @@ -51,7 +55,7 @@ alternative: sequence ( '|' sequence )+ star: '*' | '+' - sequence: (SYMBOL star? | STRING | option | group star? )+ + sequence: (SYMBOL star? | STRING | option | group )+ option: '[' alternative ']' group: '(' alternative ')' star? """ @@ -59,12 +63,12 @@ S = g_add_symbol T = g_add_token # star: '*' | '+' - star = Alternative( S("star"), [Token(T(TOK_STAR='*')), Token(T(TOK_ADD='+'))] ) + star = Alternative( S("star"), [Token(T('TOK_STAR', '*')), Token(T('TOK_ADD', '+'))] ) star_opt = KleeneStar ( S("star_opt"), 0, 1, rule=star ) # rule: SYMBOL ':' alternative - symbol = Sequence( S("symbol"), [Token(T(TOK_SYMBOL='SYMBOL')), star_opt] ) - symboldef = Token( T(TOK_SYMDEF="SYMDEF") ) + symbol = Sequence( S("symbol"), [Token(T('TOK_SYMBOL', 'SYMBOL')), star_opt] ) + symboldef = Token( T('TOK_SYMDEF', 'SYMDEF') ) alternative = Sequence( S("alternative"), []) rule = Sequence( S("rule"), [symboldef, alternative] ) @@ -73,19 +77,19 @@ # alternative: sequence ( '|' sequence )* sequence = KleeneStar( S("sequence"), 1 ) - seq_cont_list = Sequence( S("seq_cont_list"), [Token(T(TOK_BAR='|')), sequence] ) + seq_cont_list = Sequence( S("seq_cont_list"), [Token(T('TOK_BAR', '|')), sequence] ) sequence_cont = KleeneStar( S("sequence_cont"),0, rule=seq_cont_list ) alternative.args = [ sequence, sequence_cont ] # option: '[' alternative ']' - option = Sequence( S("option"), [Token(T(TOK_LBRACKET='[')), alternative, Token(T(TOK_RBRACKET=']'))] ) + option = Sequence( S("option"), [Token(T('TOK_LBRACKET', '[')), alternative, Token(T('TOK_RBRACKET', ']'))] ) # group: '(' alternative ')' - group = Sequence( S("group"), [Token(T(TOK_LPAR='(')), alternative, Token(T(TOK_RPAR=')')), star_opt] ) + group = Sequence( S("group"), [Token(T('TOK_LPAR', '(')), alternative, Token(T('TOK_RPAR', ')')), star_opt] ) # sequence: (SYMBOL | STRING | option | group )+ - string = Token(T(TOK_STRING='STRING')) + string = Token(T('TOK_STRING', 'STRING')) alt = Alternative( S("sequence_alt"), [symbol, string, option, group] ) sequence.args = [ alt ] @@ -106,4 +110,4 @@ del _value del grammar_grammar del g_add_symbol -del g_add_token +# del g_add_token Modified: pypy/branch/ast-experiments/pypy/interpreter/pyparser/ebnflexer.py ============================================================================== --- pypy/branch/ast-experiments/pypy/interpreter/pyparser/ebnflexer.py (original) +++ pypy/branch/ast-experiments/pypy/interpreter/pyparser/ebnflexer.py Mon Jan 23 16:42:50 2006 @@ -3,7 +3,7 @@ analyser in grammar.py """ -from grammar import TokenSource, Token +from grammar import TokenSource, Token, AbstractContext from ebnfgrammar import * @@ -15,6 +15,12 @@ idx+=1 return idx + +class GrammarSourceContext(AbstractContext): + def __init__(self, pos, peek): + self.pos = pos + self.peek = peek + class GrammarSource(TokenSource): """Fully RPython - see targetebnflexer.py The grammar tokenizer @@ -25,8 +31,8 @@ SYMBOL: a rule symbol usually appeary right of a SYMDEF tokens: '[', ']', '(' ,')', '*', '+', '|' """ - def __init__(self, inpstring ): - TokenSource.__init__(self) + def __init__(self, inpstring): + # TokenSource.__init__(self) self.input = inpstring self.pos = 0 self.begin = 0 @@ -36,7 +42,7 @@ def context(self): """returns an opaque context object, used to backtrack to a well known position in the parser""" - return self.pos, self._peeked + return GrammarSourceContext( self.pos, self._peeked ) def offset(self, ctx=None): """Returns the current parsing position from the start @@ -44,14 +50,17 @@ if ctx is None: return self.pos else: - assert type(ctx)==int - return ctx + assert isinstance(ctx, GrammarSourceContext) + return ctx.pos def restore(self, ctx): """restore the context provided by context()""" - self.pos, self._peeked = ctx + assert isinstance( ctx, GrammarSourceContext ) + self.pos = ctx.pos + self._peeked = ctx.peek + - def current_line(self): + def current_linesource(self): pos = idx = self.begin inp = self.input end = len(inp) Modified: pypy/branch/ast-experiments/pypy/interpreter/pyparser/ebnfparse.py ============================================================================== --- pypy/branch/ast-experiments/pypy/interpreter/pyparser/ebnfparse.py (original) +++ pypy/branch/ast-experiments/pypy/interpreter/pyparser/ebnfparse.py Mon Jan 23 16:42:50 2006 @@ -1,6 +1,7 @@ #!/usr/bin/env python -from grammar import BaseGrammarBuilder, Alternative, Sequence, Token, \ - KleeneStar, GrammarElement, build_first_sets, EmptyToken +from grammar import BaseGrammarBuilder, Alternative, Sequence, Token +from grammar import GrammarProxy, KleeneStar, GrammarElement, build_first_sets +from grammar import EmptyToken, AbstractBuilder, AbstractContext from ebnflexer import GrammarSource import ebnfgrammar from ebnfgrammar import GRAMMAR_GRAMMAR, sym_map @@ -8,8 +9,32 @@ import pytoken import pysymbol -import re -py_name = re.compile(r"[a-zA-Z_][a-zA-Z0-9_]*", re.M) + +ORDA = ord("A") +ORDZ = ord("Z") +ORDa = ord("a") +ORDz = ord("z") +ORD0 = ord("0") +ORD9 = ord("9") +ORD_ = ord("_") + +def is_py_name( name ): + if len(name)<1: + return False + v = ord(name[0]) + if not (ORDA <= v <= ORDZ or + ORDa <= v <= ORDz or v == ORD_ ): + return False + for c in name: + v = ord(c) + if not (ORDA <= v <= ORDZ or + ORDa <= v <= ORDz or + ORD0 <= v <= ORD9 or + v == ORD_ ): + return False + return True + + punct=['>=', '<>', '!=', '<', '>', '<=', '==', '\\*=', '//=', '%=', '^=', '<<=', '\\*\\*=', '\\', '=', @@ -19,6 +44,8 @@ '@', '\\[', '\\]', '`', '\\{', '\\}'] + + TERMINALS = [ 'NAME', 'NUMBER', 'STRING', 'NEWLINE', 'ENDMARKER', 'INDENT', 'DEDENT' ] @@ -46,7 +73,7 @@ """ ctx = source.context() tk = source.next() - if tk.codename==self.codename: + if tk.codename == self.codename: if tk.value not in self.keywords: ret = builder.token( tk.codename, tk.value, source ) return self.debug_return( ret, tk.codename, tk.value ) @@ -57,7 +84,7 @@ """special case of match token for tokens which are really keywords """ if not isinstance(other, Token): - raise RuntimeError("Unexpected token type %r" % other) + raise RuntimeError("Unexpected token type") if other is EmptyToken: return False if other.codename != self.codename: @@ -75,7 +102,8 @@ # we do a pass through the variables to detect # terminal symbols from non terminals for r in self.items: - for i,a in enumerate(r.args): + for i in range(len(r.args)): + a = r.args[i] if a.codename in self.rules: assert isinstance(a,Token) r.args[i] = self.rules[a.codename] @@ -148,7 +176,7 @@ value = node.value tokencode = pytoken.tok_punct.get( value, None ) if tokencode is None: - if not py_name.match( value ): + if not is_py_name( value ): raise RuntimeError("Unknown STRING value ('%s')" % value ) # assume a keyword tok = Token( pytoken.NAME, value ) @@ -177,6 +205,160 @@ raise RuntimeError("Unknown Visitor for %r" % node.name) + +class EBNFBuilder(AbstractBuilder): + """Build a grammar tree""" + def __init__(self, rules=None, debug=0, symbols=None ): + if symbols is None: + symbols = pysymbol.SymbolMapper() + AbstractBuilder.__init__(self, rules, debug, symbols) + self.rule_stack = [] + self.root_rules = {} + self.keywords = [] + self.seqcounts = [] # number of items in the current sequence + self.altcounts = [] # number of sequence in the current alternative + self.curaltcount = 0 + self.curseqcount = 0 + self.current_subrule = 0 + self.current_rule = -1 + + def new_symbol(self): + current_rule_name = self.symbols.sym_name.get(self.current_rule,"x") + rule_name = ":" + current_rule_name + "_%d" % self.current_subrule + self.current_subrule += 1 + symval = self.symbols.add_anon_symbol( rule_name ) + return symval + + def get_symbolcode(self, name ): + codename = self.symbols.sym_values.get( name, -1 ) + if codename == -1: + codename = self.symbols.add_symbol( name ) + return codename + + def get_rule( self, name ): + codename = self.get_symbolcode( name ) + if codename in self.root_rules: + return self.root_rules[codename] + proxy = GrammarProxy( codename ) + self.root_rules[codename] = proxy + return proxy + + def context(self): + """Return an opaque context object""" + return None + + def restore(self, ctx): + """Accept an opaque context object""" + assert False, "Not supported" + + def alternative(self, rule, source): +# print " alternative", rule.display(level=0,symbols=ebnfgrammar.sym_map) + return True + + def pop_rules( self, count ): + offset = len(self.rule_stack)-count + assert offset>=0 + rules = self.rule_stack[offset:] + del self.rule_stack[offset:] + return rules + + def sequence(self, rule, source, elts_number): +# print " sequence", rule.display(level=0,symbols=ebnfgrammar.sym_map) + _rule = rule.codename + if _rule == ebnfgrammar.sequence: +# print " -sequence", self.curaltcount, self.curseqcount + if self.curseqcount==1: + self.curseqcount = 0 + self.curaltcount += 1 + return True + rules = self.pop_rules(self.curseqcount) + new_rule = Sequence( self.new_symbol(), rules ) + self.rule_stack.append( new_rule ) + self.curseqcount = 0 + self.curaltcount += 1 + elif _rule == ebnfgrammar.alternative: +# print " -alternative", self.curaltcount, self.curseqcount + if self.curaltcount == 1: + self.curaltcount = 0 + return True + rules = self.pop_rules(self.curaltcount) + new_rule = Alternative( self.new_symbol(), rules ) + self.rule_stack.append( new_rule ) + self.curaltcount = 0 + elif _rule == ebnfgrammar.group: +# print " -group", self.curaltcount, self.curseqcount + self.curseqcount += 1 + elif _rule == ebnfgrammar.option: +# print " -option", self.curaltcount, self.curseqcount + self.curseqcount += 1 + elif _rule == ebnfgrammar.rule: +# print " -rule", self.curaltcount, self.curseqcount + assert len(self.rule_stack)==1 + old_rule = self.rule_stack[0] + del self.rule_stack[0] + old_rule.codename = self.current_rule + self.root_rules[self.current_rule] = old_rule + self.current_subrule = 0 + return True + + def token(self, name, value, source): +# print "token", name, value + if name == ebnfgrammar.TOK_STRING: + self.handle_TOK_STRING( name, value ) + self.curseqcount += 1 + elif name == ebnfgrammar.TOK_SYMDEF: + self.current_rule = self.get_symbolcode( value ) + elif name == ebnfgrammar.TOK_SYMBOL: + rule = self.get_rule( value ) + self.rule_stack.append( rule ) + self.curseqcount += 1 + elif name == ebnfgrammar.TOK_STAR: + top = self.rule_stack[-1] + rule = KleeneStar( self.new_symbol(), _min=0, rule=top) + self.rule_stack[-1] = rule + elif name == ebnfgrammar.TOK_ADD: + top = self.rule_stack[-1] + rule = KleeneStar( self.new_symbol(), _min=1, rule=top) + self.rule_stack[-1] = rule + elif name == ebnfgrammar.TOK_BAR: + assert self.curseqcount == 0 + elif name == ebnfgrammar.TOK_LPAR: + self.altcounts.append( self.curaltcount ) + self.seqcounts.append( self.curseqcount ) + self.curseqcount = 0 + self.curaltcount = 0 + elif name == ebnfgrammar.TOK_RPAR: + assert self.curaltcount == 0 + self.curaltcount = self.altcounts.pop() + self.curseqcount = self.seqcounts.pop() + elif name == ebnfgrammar.TOK_LBRACKET: + self.altcounts.append( self.curaltcount ) + self.seqcounts.append( self.curseqcount ) + self.curseqcount = 0 + self.curaltcount = 0 + elif name == ebnfgrammar.TOK_RBRACKET: + assert self.curaltcount == 0 + assert self.curseqcount == 0 + self.curaltcount = self.altcounts.pop() + self.curseqcount = self.seqcounts.pop() + return True + + def handle_TOK_STRING( self, name, value ): + try: + tokencode = pytoken.tok_punct[value] + except KeyError: + if not is_py_name(value): + raise RuntimeError("Unknown STRING value ('%s')" % value) + # assume a keyword + tok = Token(pytoken.NAME, value) + if value not in self.keywords: + self.keywords.append(value) + else: + # punctuation + tok = Token(tokencode, None) + self.rule_stack.append(tok) + + class EBNFVisitor(AbstractSyntaxVisitor): def __init__(self): @@ -190,7 +372,8 @@ self.symbols = pysymbol.SymbolMapper( pysymbol._cpython_symbols.sym_name ) def new_symbol(self): - rule_name = ":%s_%s" % (self.current_rule, self.current_subrule) + current_rule_name = self.symbols.sym_name.get(self.current_rule,"x") + rule_name = ":" + self.current_rule + "_" + str(self.current_subrule) self.current_subrule += 1 symval = self.symbols.add_anon_symbol( rule_name ) return symval @@ -241,25 +424,24 @@ node.visit(vis) return vis + def parse_grammar_text(txt): """parses a grammar input stream : file-like object representing the grammar to parse """ source = GrammarSource(txt) - builder = BaseGrammarBuilder() + builder = EBNFBuilder(pysymbol._cpython_symbols) result = GRAMMAR_GRAMMAR.match(source, builder) - node = builder.stack[-1] - vis = EBNFVisitor() - node.visit(vis) - return vis + return builder def target_parse_grammar_text(txt): vis = parse_grammar_text(txt) # do nothing + return None -from pprint import pprint -if __name__ == "__main__": +def main_build(): + from pprint import pprint grambuild = parse_grammar(file('data/Grammar2.3')) for i,r in enumerate(grambuild.items): print "% 3d : %s" % (i, r) @@ -267,3 +449,9 @@ pprint(grambuild.tokens) print "|".join(grambuild.tokens.keys() ) +def main_build(): + import sys + return parse_grammar_text( file(sys.argv[-1]).read() ) + +if __name__ == "__main__": + result = main_build() Modified: pypy/branch/ast-experiments/pypy/interpreter/pyparser/grammar.py ============================================================================== --- pypy/branch/ast-experiments/pypy/interpreter/pyparser/grammar.py (original) +++ pypy/branch/ast-experiments/pypy/interpreter/pyparser/grammar.py Mon Jan 23 16:42:50 2006 @@ -10,10 +10,13 @@ try: from pypy.interpreter.baseobjspace import Wrappable from pypy.interpreter.pyparser.pytoken import NULLTOKEN + from pypy.interpreter.pyparser.pysymbol import SymbolMapper except ImportError: # allows standalone testing Wrappable = object - NULLTOKEN = None + NULLTOKEN = -1 # None + from pysymbol import SymbolMapper + from syntaxtree import SyntaxNode, TempSyntaxNode, TokenNode @@ -28,6 +31,7 @@ else: return "["+str(codename)+"]" + #### Abstract interface for a lexer/tokenizer class TokenSource(object): """Abstract base class for a source tokenizer""" @@ -50,7 +54,7 @@ of the context""" return -1 - def current_line(self): + def current_linesource(self): """Returns the current line""" return "" @@ -102,7 +106,7 @@ class AbstractBuilder(object): """Abstract base class for builder objects""" - def __init__(self, rules=None, debug=0, symbols={} ): + def __init__(self, symbols, rules=None, debug=0 ): # a dictionary of grammar rules for debug/reference if rules is not None: self.rules = rules @@ -110,7 +114,9 @@ self.rules = {} # This attribute is here for convenience self.debug = debug - self.symbols = symbols # mapping from codename to symbols + # mapping from codename to symbols + assert isinstance( symbols, SymbolMapper ) + self.symbols = symbols def context(self): """Return an opaque context object""" @@ -143,6 +149,8 @@ class BaseGrammarBuilder(AbstractBuilder): """Base/default class for a builder""" def __init__(self, rules=None, debug=0, symbols={} ): + if rules is None: + rules = SymbolMapper() AbstractBuilder.__init__(self, rules, debug, symbols ) # stacks contain different objects depending on the builder class # to be RPython they should not be defined in the base class @@ -355,6 +363,23 @@ +class GrammarProxy(GrammarElement): + def __init__(self, rule_name ): + GrammarElement.__init__(self, -1) + self.rule_name = rule_name + self.object = None + + def display(self, level=0, symbols={}): + """Helper function used to represent the grammar. + mostly used for debugging the grammar itself""" + name = symbols.get(self.rule_name,str(self.rule_name)) + repr = "Proxy("+name + if self.object: + repr+=","+self.object.display(level=1,symbols=symbols) + repr += ")" + return repr + + class Alternative(GrammarElement): """Represents an alternative in a grammar rule (as in S -> A | B | C)""" @@ -553,7 +578,7 @@ """ if DEBUG > 1: print "try kle:", self.display(0,builder.symbols) - ctx = 0 + ctx = None bctx = None if self.min: ctx = source.context() @@ -625,8 +650,8 @@ class Token(GrammarElement): """Represents a Token in a grammar rule (a lexer token)""" - def __init__( self, codename, value = None): - GrammarElement.__init__( self, codename ) + def __init__(self, codename, value=None): + GrammarElement.__init__(self, codename) self.value = value self.first_set = [self] # self.first_set = {self: 1} @@ -673,7 +698,7 @@ the comparison algorithm is similar to the one in match() """ if not isinstance(other, Token): - raise RuntimeError("Unexpected token type %r" % other) + raise RuntimeError("Unexpected token type") if other is EmptyToken: return False res = other.codename == self.codename and self.value in [None, other.value] Modified: pypy/branch/ast-experiments/pypy/interpreter/pyparser/pysymbol.py ============================================================================== --- pypy/branch/ast-experiments/pypy/interpreter/pyparser/pysymbol.py (original) +++ pypy/branch/ast-experiments/pypy/interpreter/pyparser/pysymbol.py Mon Jan 23 16:42:50 2006 @@ -27,7 +27,7 @@ self._count = _count def add_symbol( self, sym ): - assert type(sym)==str + # assert isinstance(sym, str) if not sym in self.sym_values: self._count += 1 val = self._count @@ -37,7 +37,7 @@ return self.sym_values[ sym ] def add_anon_symbol( self, sym ): - assert type(sym)==str + # assert isinstance(sym, str) if not sym in self.sym_values: self._anoncount -= 1 val = self._anoncount @@ -48,7 +48,7 @@ def __getitem__(self, sym ): """NOT RPYTHON""" - assert type(sym)==str + # assert isinstance(sym, str) return self.sym_values[ sym ] @@ -65,7 +65,7 @@ """Update the symbol module according to rules in PythonParser instance : parser""" for rule in parser.rules: - _cpython_symbols.add_symbol( rule ) + _cpython_symbols.add_symbol(rule) # There is no symbol in this module until the grammar is loaded # once loaded the grammar parser will fill the mappings with the Modified: pypy/branch/ast-experiments/pypy/interpreter/pyparser/pythonlexer.py ============================================================================== --- pypy/branch/ast-experiments/pypy/interpreter/pyparser/pythonlexer.py (original) +++ pypy/branch/ast-experiments/pypy/interpreter/pyparser/pythonlexer.py Mon Jan 23 16:42:50 2006 @@ -5,7 +5,7 @@ import sys from codeop import PyCF_DONT_IMPLY_DEDENT -from pypy.interpreter.pyparser.grammar import TokenSource, Token +from pypy.interpreter.pyparser.grammar import TokenSource, Token, AbstractContext from pypy.interpreter.pyparser.error import SyntaxError import pytoken from pytoken import NEWLINE @@ -293,6 +293,11 @@ #print '----------------------------------------- pyparser/pythonlexer.py' return token_list + +class PythonSourceContext(AbstractContext): + def __init__(self, pos ): + self.pos = pos + class PythonSource(TokenSource): """This source uses Jonathan's tokenizer""" def __init__(self, strings, flags=0): @@ -317,7 +322,7 @@ self._offset = pos return tok - def current_line(self): + def current_linesource(self): """Returns the current line being parsed""" return self._current_line @@ -327,11 +332,12 @@ def context(self): """Returns an opaque context object for later restore""" - return self.stack_pos + return PythonSourceContext(self.stack_pos) def restore(self, ctx): """Restores a context""" - self.stack_pos = ctx + assert isinstance(ctx, PythonSourceContext) + self.stack_pos = ctx.pos def peek(self): """returns next token without consuming it""" Modified: pypy/branch/ast-experiments/pypy/interpreter/pyparser/pythonutil.py ============================================================================== --- pypy/branch/ast-experiments/pypy/interpreter/pyparser/pythonutil.py (original) +++ pypy/branch/ast-experiments/pypy/interpreter/pyparser/pythonutil.py Mon Jan 23 16:42:50 2006 @@ -6,6 +6,7 @@ from tuplebuilder import TupleBuilder from astbuilder import AstBuilder from pypy.interpreter.pyparser import pysymbol +import pysymbol PYTHON_PARSER = pythonparse.PYTHON_PARSER TARGET_DICT = { @@ -60,7 +61,7 @@ tuples (StackElement is only a wrapper class around these tuples) """ - builder = TupleBuilder(PYTHON_PARSER.rules, lineno=False) + builder = TupleBuilder(pysymbol._cpython_symbols, PYTHON_PARSER.rules, lineno=False) if space is not None: builder.space = space target_rule = TARGET_DICT[mode] Modified: pypy/branch/ast-experiments/pypy/interpreter/pyparser/pytoken.py ============================================================================== --- pypy/branch/ast-experiments/pypy/interpreter/pyparser/pytoken.py (original) +++ pypy/branch/ast-experiments/pypy/interpreter/pyparser/pytoken.py Mon Jan 23 16:42:50 2006 @@ -5,21 +5,21 @@ N_TOKENS = 0 -tok_name = {} -tok_values = {} +# This is used to replace None +NULLTOKEN = -1 + +tok_name = {-1 : 'NULLTOKEN'} +tok_values = {'NULLTOKEN' : -1} -def add_token(name, value=None): +def add_token(name): global N_TOKENS - if value is None: - value = N_TOKENS - N_TOKENS += 1 + value = N_TOKENS + N_TOKENS += 1 _g = globals() _g[name] = value tok_name[value] = name tok_values[name] = value -# This is used to replace None -add_token( 'NULLTOKEN', -1 ) # For compatibility, this produces the same constant values as Python 2.4. add_token( 'ENDMARKER' ) Modified: pypy/branch/ast-experiments/pypy/interpreter/pyparser/test/test_astcompiler.py ============================================================================== --- pypy/branch/ast-experiments/pypy/interpreter/pyparser/test/test_astcompiler.py (original) +++ pypy/branch/ast-experiments/pypy/interpreter/pyparser/test/test_astcompiler.py Mon Jan 23 16:42:50 2006 @@ -3,6 +3,7 @@ from pypy.interpreter.pyparser.astbuilder import AstBuilder from pypy.interpreter.pyparser.tuplebuilder import TupleBuilder from pypy.interpreter.pycode import PyCode +from pypy.interpreter.pyparser.pysymbol import _cpython_symbols import py.test def setup_module(mod): @@ -82,7 +83,7 @@ def compile_with_testcompiler(expr, target='exec', space=FakeSpace()): target2 = TARGET_DICT['exec'] # xxx exec: single not really tested - builder = TupleBuilder() + builder = TupleBuilder(_cpython_symbols) PYTHON_PARSER.parse_source(expr, target2, builder) tuples = builder.stack[-1].as_tuple(True) from pypy.interpreter.stablecompiler import transformer, pycodegen, misc @@ -221,7 +222,7 @@ for snippet_name in LIBSTUFF: filepath = os.path.join(os.path.dirname(__file__), '../../../lib', snippet_name) source = file(filepath).read() - yield check_compile, source, 'exec' + yield check_compile, source, 'exec' def test_single_inputs(): Modified: pypy/branch/ast-experiments/pypy/interpreter/pyparser/tuplebuilder.py ============================================================================== --- pypy/branch/ast-experiments/pypy/interpreter/pyparser/tuplebuilder.py (original) +++ pypy/branch/ast-experiments/pypy/interpreter/pyparser/tuplebuilder.py Mon Jan 23 16:42:50 2006 @@ -1,6 +1,7 @@ from grammar import AbstractBuilder, AbstractContext from pytoken import tok_name, tok_rpunct, NEWLINE, INDENT, DEDENT, ENDMARKER +import pysymbol class StackElement: """wraps TupleBuilder's tuples""" @@ -57,8 +58,8 @@ class TupleBuilder(AbstractBuilder): """A builder that directly produce the AST""" - def __init__(self, rules=None, debug=0, lineno=True): - AbstractBuilder.__init__(self, rules, debug) + def __init__(self, symbols, rules=None, debug=0, lineno=True): + AbstractBuilder.__init__(self, symbols, rules, debug) # This attribute is here for convenience self.source_encoding = None self.lineno = lineno Added: pypy/branch/ast-experiments/pypy/translator/goal/targetebnfparser.py ============================================================================== --- (empty file) +++ pypy/branch/ast-experiments/pypy/translator/goal/targetebnfparser.py Mon Jan 23 16:42:50 2006 @@ -0,0 +1,36 @@ +from pypy.interpreter.pyparser.ebnfparse import target_parse_grammar_text +from pypy.annotation import policy as annpolicy + + +entry_point = target_parse_grammar_text + +# _____ Define and setup target ___ + +def target(*args): + policy = annpolicy.AnnotatorPolicy() + policy.allow_someobjects = False + return entry_point, [str]# , policy + +def get_llinterp_args(): + return [1] + +# _____ Run translated _____ +def run(c_entry_point): + import sys + NBC=100 + import time + src = file("../../interpreter/pyparser/data/Grammar2.4").read() + print "Translated:" + t1 = time.time() + for i in range(NBC): + c_entry_point( src ) + t2 = time.time() + print "%8.5f sec/loop" % (float(t2-t1)/NBC) + print "CPython:" + t1 = time.time() + for i in range(NBC): + entry_point( src ) + t2 = time.time() + print "%8.5f sec/loop" % (float(t2-t1)/NBC) + + From gromit at codespeak.net Mon Jan 23 16:56:52 2006 From: gromit at codespeak.net (gromit at codespeak.net) Date: Mon, 23 Jan 2006 16:56:52 +0100 (CET) Subject: [pypy-svn] r22534 - pypy/dist/pypy/rpython/rctypes/test Message-ID: <20060123155652.F3CCA27DC4@code1.codespeak.net> Author: gromit Date: Mon Jan 23 16:56:51 2006 New Revision: 22534 Modified: pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py Log: FIX: Got teh linux DLL importing stuff right Modified: pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py Mon Jan 23 16:56:51 2006 @@ -14,7 +14,7 @@ if sys.platform == 'win32': mylib = cdll.LoadLibrary('msvcrt.dll') elif sys.platform == 'linux2': - mylib = cdll.LoadLibrary('libc') + mylib = cdll.LoadLibrary('libc.so.6') else: py.test.skip("don't know how to load the c lib for %s" % sys.platform) From mwh at codespeak.net Mon Jan 23 17:51:03 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Mon, 23 Jan 2006 17:51:03 +0100 (CET) Subject: [pypy-svn] r22536 - in pypy/dist/pypy: rpython/l3interp rpython/l3interp/test translator/c Message-ID: <20060123165103.D025D27B84@code1.codespeak.net> Author: mwh Date: Mon Jan 23 17:51:00 2006 New Revision: 22536 Modified: pypy/dist/pypy/rpython/l3interp/l3interp.py pypy/dist/pypy/rpython/l3interp/test/test_convert.py pypy/dist/pypy/rpython/l3interp/test/test_l3interp.py pypy/dist/pypy/translator/c/database.py Log: (arre, mwh) Actually test the translation of a non-trivial use of offsets. This meant fixing the c generation, and finding that really weird things happen if you let a translationcontext go away before graphs that it made. Modified: pypy/dist/pypy/rpython/l3interp/l3interp.py ============================================================================== --- pypy/dist/pypy/rpython/l3interp/l3interp.py (original) +++ pypy/dist/pypy/rpython/l3interp/l3interp.py Mon Jan 23 17:51:00 2006 @@ -50,6 +50,10 @@ def __init__(self, graph, stack_int, stack_dbl, stack_ptr): self.graph = graph self.block = self.graph.startblock + # XXX aaaaaaaargh! + if self.block.constants_int is None: + self.block.constants_int = [0] + self.block.constants_int = None if self.block.constants_ptr is None: self.block.constants_ptr = [constant_fakeaddress] self.block.constants_ptr = None Modified: pypy/dist/pypy/rpython/l3interp/test/test_convert.py ============================================================================== --- pypy/dist/pypy/rpython/l3interp/test/test_convert.py (original) +++ pypy/dist/pypy/rpython/l3interp/test/test_convert.py Mon Jan 23 17:51:00 2006 @@ -7,7 +7,14 @@ t.buildannotator().build_types(f, inputargs) t.buildrtyper().specialize() conv = convertgraph.LL2L3Converter() - return conv.convert_graph(t.graphs[0]) + g = conv.convert_graph(t.graphs[0]) + # XXX this vile, vile hack prevents the TranslationContext from + # being deallocated which leads to the vtables of certain + # important types (like object) going away, which is generally + # very, very confusing. + # XXX work out why, fix it + g.keepthislotalive = t + return g def test_convert_add(): Modified: pypy/dist/pypy/rpython/l3interp/test/test_l3interp.py ============================================================================== --- pypy/dist/pypy/rpython/l3interp/test/test_l3interp.py (original) +++ pypy/dist/pypy/rpython/l3interp/test/test_l3interp.py Mon Jan 23 17:51:00 2006 @@ -121,3 +121,33 @@ fn = translate(eval_call, [int]) assert fn(4) == 7 assert fn(0) == 3 + +#---------------------------------------------------------------------- + +from pypy.rpython.l3interp.test.test_convert import l3ify + +def test_getfield(): + class C: + def __init__(self, x): + self.x = x + one = C(1) + two = C(2) + + def f(n): + if n: + return one.x + else: + return two.x + + l3graph = l3ify(f, [int]) + + def entry_point(x): + value = l3interp.l3interpret(l3graph, [x], [], []) + assert isinstance(value, l3interp.L3Integer) + return value.intval + + fn = translate(entry_point, [int]) + + assert fn(3) == f(3) + assert fn(0) == f(0) + Modified: pypy/dist/pypy/translator/c/database.py ============================================================================== --- pypy/dist/pypy/translator/c/database.py (original) +++ pypy/dist/pypy/translator/c/database.py Mon Jan 23 17:51:00 2006 @@ -126,8 +126,10 @@ if len(obj.fldnames) == 0: return '0 /*offsetof*/' else: - return 'offsetof(%s, %s)'%( - self.gettype(obj.TYPE), obj.fldnames[0]) + structnode = self.gettypedefnode(obj.TYPE.TO) + return 'offsetof(struct %s, %s)'%( + structnode.name, + structnode.c_struct_field_name(obj.fldnames[0])) elif T is Address and obj is not NULL: if obj.ob is None: return 'NULL' From auc at codespeak.net Mon Jan 23 17:56:55 2006 From: auc at codespeak.net (auc at codespeak.net) Date: Mon, 23 Jan 2006 17:56:55 +0100 (CET) Subject: [pypy-svn] r22537 - pypy/dist/pypy/lib/logic Message-ID: <20060123165655.9AEB027B84@code1.codespeak.net> Author: auc Date: Mon Jan 23 17:56:53 2006 New Revision: 22537 Added: pypy/dist/pypy/lib/logic/constraint.py Modified: pypy/dist/pypy/lib/logic/test_unification.py pypy/dist/pypy/lib/logic/unification.py pypy/dist/pypy/lib/logic/variable.py Log: (auc, ale) * many more misc. checks * add constraint stuff Added: pypy/dist/pypy/lib/logic/constraint.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/lib/logic/constraint.py Mon Jan 23 17:56:53 2006 @@ -0,0 +1,369 @@ +# a) new requirement : be able to postpone asking fo the +# values of the domain + +class ConsistencyFailure(Exception): + """The repository is not in a consistent state""" + pass + + + +#-- Domains -------------------------------------------- + +class AbstractDomain(object): + """Implements the functionnality related to the changed flag. + Can be used as a starting point for concrete domains""" + + #__implements__ = DomainInterface + def __init__(self): + self.__changed = 0 + + def reset_flags(self): + self.__changed = 0 + + def has_changed(self): + return self.__changed + + def _value_removed(self): + """The implementation of remove_value should call this method""" + self.__changed = 1 + if self.size() == 0: + raise ConsistencyFailure() + + +class FiniteDomain(AbstractDomain): + """ + Variable Domain with a finite set of possible values + """ + + _copy_count = 0 + _write_count = 0 + + def __init__(self, values): + """values is a list of values in the domain + This class uses a dictionnary to make sure that there are + no duplicate values""" + AbstractDomain.__init__(self) + if isinstance(values,FiniteDomain): + # do a copy on write + self._cow = True + values._cow = True + FiniteDomain._copy_count += 1 + self._values = values._values + else: + # don't check this there (a) + #assert len(values) > 0 + self.set_values(values) + + ##self.getValues = self._values.keys + + def set_values(self, values): + self._cow = False + FiniteDomain._write_count += 1 + self._values = set(values) + + def remove_value(self, value): + """Remove value of domain and check for consistency""" +## print "removing", value, "from", self._values.keys() + if self._cow: + self.set_values(self._values) + del self._values[value] + self._value_removed() + + def remove_values(self, values): + """Remove values of domain and check for consistency""" + if self._cow: + self.set_values(self._values) + if values: +## print "removing", values, "from", self._values.keys() + for val in values : + self._values.remove(val) + self._value_removed() + __delitem__ = remove_value + + def size(self): + """computes the size of a finite domain""" + return len(self._values) + __len__ = size + + def get_values(self): + """return all the values in the domain""" + return self._values + + def __iter__(self): + return iter(self._values) + + def copy(self): + """clone the domain""" + return FiniteDomain(self) + + def __repr__(self): + return '' % str(self.get_values()) + + def __eq__(self, other): + if other is None: return False + return self._values == other._values + + def intersection(self, other): + if other is None: return self.get_values() + return self.get_values() & other.get_values() + +#-- Constraints ------------------------------------------ + +class AbstractConstraint(object): + #__implements__ = ConstraintInterface + + def __init__(self, variables): + """variables is a list of variables which appear in the formula""" + self._variables = variables + + def affectedVariables(self): + """ Return a list of all variables affected by this constraint """ + return self._variabl + + def isVariableRelevant(self, variable): + return variable in self._variables + + def estimateCost(self, domains): + """Return an estimate of the cost of the narrowing of the constraint""" + return reduce(operator.mul, + [domains[var].size() for var in self._variables]) + + +class BasicConstraint(object): + """A BasicConstraint, which is never queued by the Repository + A BasicConstraint affects only one variable, and will be entailed + on the first call to narrow()""" + + def __init__(self, variable, reference, operator): + """variables is a list of variables on which + the constraint is applied""" + self._variable = variable + self._reference = reference + self._operator = operator + + def __repr__(self): + return '<%s %s %s>'% (self.__class__, self._variable, self._reference) + + def isVariableRelevant(self, variable): + return variable == self._variable + + def estimateCost(self, domains): + return 0 # get in the first place in the queue + + def affectedVariables(self): + return [self._variable] + + def getVariable(self): + return self._variable + + def narrow(self, domains): + domain = domains[self._variable] + operator = self._operator + ref = self._reference + try: + for val in domain.get_values() : + if not operator(val, ref) : + domain.remove_value(val) + except ConsistencyFailure: + raise ConsistencyFailure('inconsistency while applying %s' % \ + repr(self)) + return 1 + + + +class Expression(AbstractConstraint): + """A constraint represented as a python expression.""" + _FILTER_CACHE = {} + + def __init__(self, variables, formula, type='fd.Expression'): + """variables is a list of variables which appear in the formula + formula is a python expression that will be evaluated as a boolean""" + AbstractConstraint.__init__(self, variables) + self.formula = formula + self.type = type + try: + self.filterFunc = Expression._FILTER_CACHE[formula] + except KeyError: + self.filterFunc = eval('lambda %s: %s' % \ + (','.join(variables), formula), {}, {}) + Expression._FILTER_CACHE[formula] = self.filterFunc + + def _init_result_cache(self): + """key = (variable,value), value = [has_success,has_failure]""" + result_cache = {} + for var_name in self._variables: + result_cache[var_name] = {} + return result_cache + + + def _assign_values(self, domains): + variables = [] + kwargs = {} + for variable in self._variables: + domain = domains[variable] + values = domain.get_values() + variables.append((domain.size(), [variable, values, 0, len(values)])) + kwargs[variable] = values[0] + # sort variables to instanciate those with fewer possible values first + variables.sort() + + go_on = 1 + while go_on: + yield kwargs + # try to instanciate the next variable + for size, curr in variables: + if (curr[2] + 1) < curr[-1]: + curr[2] += 1 + kwargs[curr[0]] = curr[1][curr[2]] + break + else: + curr[2] = 0 + kwargs[curr[0]] = curr[1][0] + else: + # it's over + go_on = 0 + + + def narrow(self, domains): + """generic narrowing algorithm for n-ary expressions""" + maybe_entailed = 1 + ffunc = self.filterFunc + result_cache = self._init_result_cache() + for kwargs in self._assign_values(domains): + if maybe_entailed: + for var, val in kwargs.iteritems(): + if val not in result_cache[var]: + break + else: + continue + if ffunc(**kwargs): + for var, val in kwargs.items(): + result_cache[var][val] = 1 + else: + maybe_entailed = 0 + + try: + for var, keep in result_cache.iteritems(): + domain = domains[var] + domain.remove_values([val for val in domain if val not in keep]) + + except ConsistencyFailure: + raise ConsistencyFailure('Inconsistency while applying %s' % \ + repr(self)) + except KeyError: + # There are no more value in result_cache + pass + + return maybe_entailed + + def __repr__(self): + return '<%s "%s">' % (self.type, self.formula) + +class BinaryExpression(Expression): + """A binary constraint represented as a python expression + + This implementation uses a narrowing algorithm optimized for + binary constraints.""" + + def __init__(self, variables, formula, type = 'fd.BinaryExpression'): + assert len(variables) == 2 + Expression.__init__(self, variables, formula, type) + + def narrow(self, domains): + """specialized narrowing algorithm for binary expressions + Runs much faster than the generic version""" + maybe_entailed = 1 + var1 = self._variables[0] + dom1 = domains[var1] + values1 = dom1.get_values() + var2 = self._variables[1] + dom2 = domains[var2] + values2 = dom2.get_values() + ffunc = self.filterFunc + if dom2.size() < dom1.size(): + var1, var2 = var2, var1 + dom1, dom2 = dom2, dom1 + values1, values2 = values2, values1 + + kwargs = {} + keep1 = {} + keep2 = {} + maybe_entailed = 1 + try: + # iterate for all values + for val1 in values1: + kwargs[var1] = val1 + for val2 in values2: + kwargs[var2] = val2 + if val1 in keep1 and val2 in keep2 and maybe_entailed == 0: + continue + if ffunc(**kwargs): + keep1[val1] = 1 + keep2[val2] = 1 + else: + maybe_entailed = 0 + + dom1.remove_values([val for val in values1 if val not in keep1]) + dom2.remove_values([val for val in values2 if val not in keep2]) + + except ConsistencyFailure: + raise ConsistencyFailure('Inconsistency while applying %s' % \ + repr(self)) + except Exception: + print self, kwargs + raise + return maybe_entailed + + +def make_expression(variables, formula, constraint_type=None): + """create a new constraint of type Expression or BinaryExpression + The chosen class depends on the number of variables in the constraint""" + # encode unicode + vars = [] + for var in variables: + if type(var) == type(u''): + vars.append(var.encode()) + else: + vars.append(var) + if len(vars) == 2: + if constraint_type is not None: + return BinaryExpression(vars, formula, constraint_type) + else: + return BinaryExpression(vars, formula) + + else: + if constraint_type is not None: + return Expression(vars, formula, constraint_type) + else: + return Expression(vars, formula) + + +class Equals(BasicConstraint): + """A basic constraint variable == constant value""" + def __init__(self, variable, reference): + BasicConstraint.__init__(self, variable, reference, operator.eq) + +class NotEquals(BasicConstraint): + """A basic constraint variable != constant value""" + def __init__(self, variable, reference): + BasicConstraint.__init__(self, variable, reference, operator.ne) + +class LesserThan(BasicConstraint): + """A basic constraint variable < constant value""" + def __init__(self, variable, reference): + BasicConstraint.__init__(self, variable, reference, operator.lt) + +class LesserOrEqual(BasicConstraint): + """A basic constraint variable <= constant value""" + def __init__(self, variable, reference): + BasicConstraint.__init__(self, variable, reference, operator.le) + +class GreaterThan(BasicConstraint): + """A basic constraint variable > constant value""" + def __init__(self, variable, reference): + BasicConstraint.__init__(self, variable, reference, operator.gt) + +class GreaterOrEqual(BasicConstraint): + """A basic constraint variable >= constant value""" + def __init__(self, variable, reference): + BasicConstraint.__init__(self, variable, reference, operator.ge) Modified: pypy/dist/pypy/lib/logic/test_unification.py ============================================================================== --- pypy/dist/pypy/lib/logic/test_unification.py (original) +++ pypy/dist/pypy/lib/logic/test_unification.py Mon Jan 23 17:56:53 2006 @@ -1,5 +1,6 @@ import unification as u import variable as v +from constraint import FiniteDomain from py.test import raises, skip from threading import Thread @@ -46,11 +47,23 @@ assert z.val == 3.14 def test_unify_same(self): - x,y,z = (u.var('x'), u.var('y'), u.var('z')) + x,y,z,w = (u.var('x'), u.var('y'), + u.var('z'), u.var('w')) u.bind(x, [42, z]) u.bind(y, [z, 42]) + u.bind(w, [z, 43]) + raises(u.UnificationFailure, u.unify, x, w) + u.unify(x, y) + assert z.val == 42 + + def test_double_unification(self): + x, y, z = (u.var('x'), u.var('y'), + u.var('z')) + u.bind(x, 42) + u.bind(y, z) u.unify(x, y) assert z.val == 42 + #raises(u.UnificationFailure, u.unify, x, y) def test_unify_values(self): x, y = u.var('x'), u.var('y') @@ -199,7 +212,43 @@ t2.start() t1.join() t2.join() - assert z.val == 0 + print "Z", z assert (t2.raised and not t1.raised) or \ (t1.raised and not t2.raised) + assert z.val == 0 + + def test_set_var_domain(self): + x = u.var('x') + u.set_domain(x, [1, 3, 5]) + assert x.dom == FiniteDomain([1, 3, 5]) + assert u._store.domains[x] == FiniteDomain([1, 3, 5]) + + def test_bind_with_domain(self): + x = u.var('x') + u.set_domain(x, [1, 2, 3]) + raises(u.OutOfDomain, u.bind, x, 42) + u.bind(x, 3) + assert x.val == 3 + + def test_bind_with_incompatible_domains(self): + x, y = u.var('x'), u.var('y') + u.set_domain(x, [1, 2]) + u.set_domain(y, [3, 4]) + raises(u.IncompatibleDomains, u.bind, x, y) + u.set_domain(y, [2, 4]) + u.bind(x, y) + # check x and y are in the same equiv. set + assert x.val == y.val + + + def test_unify_with_domains(self): + x,y,z = u.var('x'), u.var('y'), u.var('z') + u.bind(x, [42, z]) + u.bind(y, [z, 42]) + u.set_domain(z, [1, 2, 3]) + raises(u.UnificationFailure, u.unify, x, y) + u.set_domain(z, [41, 42, 43]) + u.unify(x, y) + assert z.val == 42 + assert z.dom == FiniteDomain([41, 42, 43]) Modified: pypy/dist/pypy/lib/logic/unification.py ============================================================================== --- pypy/dist/pypy/lib/logic/unification.py (original) +++ pypy/dist/pypy/lib/logic/unification.py Mon Jan 23 17:56:53 2006 @@ -121,6 +121,7 @@ import threading from variable import EqSet, Var, VariableException, NotAVariable +from constraint import FiniteDomain #----------- Store Exceptions ---------------------------- class UnboundVariable(VariableException): @@ -135,13 +136,27 @@ def __str__(self): return "%s already in store" % self.name +class OutOfDomain(VariableException): + def __str__(self): + return "value not in domain of %s" % self.name + class UnificationFailure(Exception): + def __init__(self, var1, var2, cause=None): + self.var1, self.var2 = (var1, var2) + self.cause = cause + def __str__(self): + diag = "%s %s can't be unified" + if self.cause: + diag += " because %s" % self.cause + return diag % (self.var1, self.var2) + +class IncompatibleDomains(Exception): def __init__(self, var1, var2): self.var1, self.var2 = (var1, var2) def __str__(self): - return "%s %s can't be unified" % (self.var1, - self.var2) - + return "%s %s have incompatible domains" % \ + (self.var1, self.var2) + #----------- Store ------------------------------------ class Store(object): """The Store consists of a set of k variables @@ -157,6 +172,11 @@ # mapping of names to vars (all of them) self.vars = set() self.names = set() + # mapping of vars to domains + self.domains = {} + # mapping of names to constraints (all...) + self.contraints = {} + # consistency-preserving stuff self.in_transaction = False self.lock = threading.Lock() @@ -170,6 +190,15 @@ # put into new singleton equiv. set var.val = EqSet([var]) + #-- Bind var to domain -------------------- + + def set_domain(self, var, dom): + assert(isinstance(var, Var) and (var in self.vars)) + if var.is_bound(): + raise AlreadyBound + var.dom = FiniteDomain(dom) + self.domains[var] = var.dom + #-- BIND ------------------------------------------- def bind(self, var, val): @@ -177,39 +206,54 @@ 2. (unbound)Variable/(bound)Variable or 3. (unbound)Variable/Value binding """ - self.lock.acquire() - assert(isinstance(var, Var) and (var in self.vars)) - if var == val: - return - if _both_are_vars(var, val): - if _both_are_bound(var, val): - raise AlreadyBound(var.name) - if var._is_bound(): # 2b. var is bound, not var - self.bind(val, var) - elif val._is_bound(): # 2a.val is bound, not val - self._bind(var.val, val.val) - else: # 1. both are unbound - self._merge(var.val, val.val) - else: # 3. val is really a value - if var._is_bound(): - raise AlreadyBound(var.name) - self._bind(var.val, val) - self.lock.release() + try: + self.lock.acquire() + assert(isinstance(var, Var) and (var in self.vars)) + if var == val: + return + if _both_are_vars(var, val): + if _both_are_bound(var, val): + raise AlreadyBound(var.name) + if var._is_bound(): # 2b. var is bound, not var + self.bind(val, var) + elif val._is_bound(): # 2a.var is bound, not val + self._bind(var.val, val.val) + else: # 1. both are unbound + self._merge(var, val) + else: # 3. val is really a value + if var._is_bound(): + raise AlreadyBound(var.name) + self._bind(var.val, val) + finally: + self.lock.release() def _bind(self, eqs, val): # print "variable - value binding : %s %s" % (eqs, val) - # bind all vars in the eqset to obj + # bind all vars in the eqset to val for var in eqs: + if var.dom != None: + if val not in var.dom.get_values(): + # undo the half-done binding + for v in eqs: + v.val = eqs + raise OutOfDomain(var) var.val = val - def _merge(self, eqs1, eqs2): + def _merge(self, v1, v2): + for v in v1.val: + if not _compatible_domains(v, v2.val): + raise IncompatibleDomains(v1, v2) + self._really_merge(v1.val, v2.val) + + def _really_merge(self, eqs1, eqs2): # print "unbound variables binding : %s %s" % (eqs1, eqs2) if eqs1 == eqs2: return # merge two equisets into one eqs1 |= eqs2 - # let's reassign everybody to neweqs - self._bind(eqs1, eqs1) + # let's reassign everybody to the merged eq + for var in eqs1: + var.val = eqs1 #-- UNIFY ------------------------------------------ @@ -221,11 +265,13 @@ for var in self.vars: if var.changed: var._commit() - except: + except Exception, cause: for var in self.vars: if var.changed: var._abort() - raise + if isinstance(cause, UnificationFailure): + raise + raise UnificationFailure(x, y, cause) finally: self.in_transaction = False @@ -242,7 +288,7 @@ self._unify_var_val(x, y) elif _both_are_bound(x, y): self._unify_bound(x,y) - elif x.isbound(): + elif x._is_bound(): self.bind(x,y) else: self.bind(y,x) @@ -279,6 +325,20 @@ for xk in vx.keys(): self._really_unify(vx[xk], vy[xk]) + +def _compatible_domains(var, eqs): + """check that the domain of var is compatible + with the domains of the vars in the eqs + """ + if var.dom == None: return True + empty = set() + for v in eqs: + if v.dom == None: continue + if v.dom.intersection(var.dom) == empty: + return False + return True + + #-- Unifiability checks--------------------------------------- #-- #-- quite costly & could be merged back in unify @@ -293,6 +353,7 @@ _unifiable_memo = set() def _unifiable(term1, term2): + global _unifiable_memo _unifiable_memo = set() return _really_unifiable(term1, term2) @@ -351,6 +412,9 @@ _store.add_unbound(v) return v +def set_domain(var, dom): + return _store.set_domain(var, dom) + def bind(var, val): return _store.bind(var, val) Modified: pypy/dist/pypy/lib/logic/variable.py ============================================================================== --- pypy/dist/pypy/lib/logic/variable.py (original) +++ pypy/dist/pypy/lib/logic/variable.py Mon Jan 23 17:56:53 2006 @@ -37,6 +37,8 @@ self.store = store # top-level 'commited' binding self._val = NoValue + # domain + self.dom = None # when updated in a 'transaction', keep track # of our initial value (for abort cases) self.previous = None @@ -93,9 +95,11 @@ #---- Concurrent public ops -------------------------- def is_bound(self): - self.mutex.acquire() - res = self._is_bound() - self.mutex.release() + try: + self.mutex.acquire() + res = self._is_bound() + finally: + self.mutex.release() return res # should be used by threads that want to block on From mwh at codespeak.net Mon Jan 23 18:07:05 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Mon, 23 Jan 2006 18:07:05 +0100 (CET) Subject: [pypy-svn] r22538 - pypy/dist/pypy/rpython/l3interp/test Message-ID: <20060123170705.6753727DC0@code1.codespeak.net> Author: mwh Date: Mon Jan 23 18:07:03 2006 New Revision: 22538 Modified: pypy/dist/pypy/rpython/l3interp/test/test_l3interp.py Log: (arre, mwh) use isolate and boehm in the translated l3interp tests. Modified: pypy/dist/pypy/rpython/l3interp/test/test_l3interp.py ============================================================================== --- pypy/dist/pypy/rpython/l3interp/test/test_l3interp.py (original) +++ pypy/dist/pypy/rpython/l3interp/test/test_l3interp.py Mon Jan 23 18:07:03 2006 @@ -5,7 +5,15 @@ from pypy.translator.translator import TranslationContext from pypy.annotation import policy +def setup_module(mod): + mod._cleanups = [] + +def teardown_module(mod): + while mod._cleanups: + mod._cleanups.pop()() + def translate(func, inputargs): + from pypy.translator.c.gc import BoehmGcPolicy t = TranslationContext() pol = policy.AnnotatorPolicy() pol.allow_someobjects = False @@ -14,10 +22,11 @@ from pypy.translator.tool.cbuild import skip_missing_compiler from pypy.translator.c import genc - builder = genc.CExtModuleBuilder(t, func) + builder = genc.CExtModuleBuilder(t, func, gcpolicy=BoehmGcPolicy) builder.generate_source() skip_missing_compiler(builder.compile) - builder.import_module() + builder.isolated_import() + _cleanups.append(builder.cleanup) return builder.get_entry_point() From ludal at codespeak.net Mon Jan 23 18:14:24 2006 From: ludal at codespeak.net (ludal at codespeak.net) Date: Mon, 23 Jan 2006 18:14:24 +0100 (CET) Subject: [pypy-svn] r22540 - in pypy/branch/ast-experiments/pypy/interpreter/pyparser: . test Message-ID: <20060123171424.ED99627DC0@code1.codespeak.net> Author: ludal Date: Mon Jan 23 18:14:22 2006 New Revision: 22540 Modified: pypy/branch/ast-experiments/pypy/interpreter/pyparser/ebnfparse.py pypy/branch/ast-experiments/pypy/interpreter/pyparser/grammar.py pypy/branch/ast-experiments/pypy/interpreter/pyparser/pythonparse.py pypy/branch/ast-experiments/pypy/interpreter/pyparser/test/test_astbuilder.py Log: (ludal,adim) debugging and first attempt to replace old with new grammar parser Modified: pypy/branch/ast-experiments/pypy/interpreter/pyparser/ebnfparse.py ============================================================================== --- pypy/branch/ast-experiments/pypy/interpreter/pyparser/ebnfparse.py (original) +++ pypy/branch/ast-experiments/pypy/interpreter/pyparser/ebnfparse.py Mon Jan 23 18:14:22 2006 @@ -124,7 +124,7 @@ def ebnf_handle_alternative(self, node): items = [node.nodes[0].visit(self)] - items += node.nodes[1].visit(self) + items += node.nodes[1].visit(self) if len(items) == 1 and not items[0].is_root(): return items[0] alt = Alternative(self.new_symbol(), items) @@ -203,7 +203,7 @@ def handle_unknown( self, node ): raise RuntimeError("Unknown Visitor for %r" % node.name) - + class EBNFBuilder(AbstractBuilder): @@ -221,6 +221,8 @@ self.curseqcount = 0 self.current_subrule = 0 self.current_rule = -1 + self.all_rules = [] + self.tokens = {} def new_symbol(self): current_rule_name = self.symbols.sym_name.get(self.current_rule,"x") @@ -229,6 +231,35 @@ symval = self.symbols.add_anon_symbol( rule_name ) return symval + def new_rule(self, rule): + self.all_rules.append(rule) + return rule + + def resolve_rules(self): + """Remove GrammarProxy objects""" + to_be_deleted = {} + for rule in self.all_rules: + for i, arg in enumerate(rule.args): + if isinstance(arg, GrammarProxy): + real_rule = self.root_rules[arg.codename] + if isinstance(real_rule, GrammarProxy): + # If we still have a GrammarProxy associated to this codename + # this means we have encountered a terminal symbol + to_be_deleted[ arg.codename ] = True + rule.args[i] = self.get_token( arg.codename ) + #print arg, "-> Token(",arg.rule_name,")" + else: + #print arg, "->", real_rule + rule.args[i] = real_rule + for codename in to_be_deleted.keys(): + del self.root_rules[codename] + + def get_token(self, codename ): + if codename in self.tokens: + return self.tokens[codename] + token = self.tokens[codename] = Token(codename) + return token + def get_symbolcode(self, name ): codename = self.symbols.sym_values.get( name, -1 ) if codename == -1: @@ -236,10 +267,13 @@ return codename def get_rule( self, name ): + tokencode = pytoken.tok_values.get( name, -1 ) + if tokencode>=0: + return self.get_token( tokencode ) codename = self.get_symbolcode( name ) if codename in self.root_rules: return self.root_rules[codename] - proxy = GrammarProxy( codename ) + proxy = GrammarProxy( name, codename ) self.root_rules[codename] = proxy return proxy @@ -250,11 +284,11 @@ def restore(self, ctx): """Accept an opaque context object""" assert False, "Not supported" - + def alternative(self, rule, source): # print " alternative", rule.display(level=0,symbols=ebnfgrammar.sym_map) return True - + def pop_rules( self, count ): offset = len(self.rule_stack)-count assert offset>=0 @@ -272,7 +306,7 @@ self.curaltcount += 1 return True rules = self.pop_rules(self.curseqcount) - new_rule = Sequence( self.new_symbol(), rules ) + new_rule = self.new_rule(Sequence( self.new_symbol(), rules )) self.rule_stack.append( new_rule ) self.curseqcount = 0 self.curaltcount += 1 @@ -282,7 +316,7 @@ self.curaltcount = 0 return True rules = self.pop_rules(self.curaltcount) - new_rule = Alternative( self.new_symbol(), rules ) + new_rule = self.new_rule(Alternative( self.new_symbol(), rules )) self.rule_stack.append( new_rule ) self.curaltcount = 0 elif _rule == ebnfgrammar.group: @@ -314,11 +348,11 @@ self.curseqcount += 1 elif name == ebnfgrammar.TOK_STAR: top = self.rule_stack[-1] - rule = KleeneStar( self.new_symbol(), _min=0, rule=top) + rule = self.new_rule(KleeneStar( self.new_symbol(), _min=0, rule=top)) self.rule_stack[-1] = rule elif name == ebnfgrammar.TOK_ADD: top = self.rule_stack[-1] - rule = KleeneStar( self.new_symbol(), _min=1, rule=top) + rule = self.new_rule(KleeneStar( self.new_symbol(), _min=1, rule=top)) self.rule_stack[-1] = rule elif name == ebnfgrammar.TOK_BAR: assert self.curseqcount == 0 @@ -424,7 +458,6 @@ node.visit(vis) return vis - def parse_grammar_text(txt): """parses a grammar input @@ -441,7 +474,7 @@ return None def main_build(): - from pprint import pprint + from pprint import pprint grambuild = parse_grammar(file('data/Grammar2.3')) for i,r in enumerate(grambuild.items): print "% 3d : %s" % (i, r) Modified: pypy/branch/ast-experiments/pypy/interpreter/pyparser/grammar.py ============================================================================== --- pypy/branch/ast-experiments/pypy/interpreter/pyparser/grammar.py (original) +++ pypy/branch/ast-experiments/pypy/interpreter/pyparser/grammar.py Mon Jan 23 18:14:22 2006 @@ -215,7 +215,7 @@ """Base parser class""" symbols = {} # dirty trick to provide a symbols mapping while printing (and not putting it in every object) - + def __init__(self, codename): # the rule name #assert type(codename)==int @@ -311,10 +311,26 @@ pass def __str__(self): - return self.display(0, GrammarElement.symbols ) + # XXX: remove me after debug + symbols = {} + import pytoken + import pysymbol + symbols.update( pysymbol._cpython_symbols.sym_name ) + symbols.update( pytoken.tok_name ) + + return self.display(0, symbols ) +# return self.display(0, GrammarElement.symbols ) def __repr__(self): - return self.display(0, GrammarElement.symbols ) + # XXX: remove me after debug + symbols = {} + import pytoken + import pysymbol + symbols.update( pysymbol._cpython_symbols.sym_name ) + symbols.update( pytoken.tok_name ) + + return self.display(0, symbols ) +# return self.display(0, GrammarElement.symbols ) def display(self, level=0, symbols={}): """Helper function used to represent the grammar. @@ -364,15 +380,15 @@ class GrammarProxy(GrammarElement): - def __init__(self, rule_name ): - GrammarElement.__init__(self, -1) + def __init__(self, rule_name, codename=-1 ): + GrammarElement.__init__(self, codename ) self.rule_name = rule_name self.object = None def display(self, level=0, symbols={}): """Helper function used to represent the grammar. mostly used for debugging the grammar itself""" - name = symbols.get(self.rule_name,str(self.rule_name)) + name = symbols.get(self.codename, self.rule_name) repr = "Proxy("+name if self.object: repr+=","+self.object.display(level=1,symbols=symbols) Modified: pypy/branch/ast-experiments/pypy/interpreter/pyparser/pythonparse.py ============================================================================== --- pypy/branch/ast-experiments/pypy/interpreter/pyparser/pythonparse.py (original) +++ pypy/branch/ast-experiments/pypy/interpreter/pyparser/pythonparse.py Mon Jan 23 18:14:22 2006 @@ -20,9 +20,9 @@ class PythonParser(object): """Wrapper class for python grammar""" - def __init__(self, grammar_builder): - self.items = grammar_builder.items - self.rules = grammar_builder.rules + def __init__(self, rules, items ): + self.items = items + self.rules = rules # Build first sets for each rule (including anonymous ones) grammar.build_first_sets(self.items) @@ -125,11 +125,21 @@ grammar.DEBUG = 0 gram = ebnfparse.parse_grammar( file(fname) ) grammar.DEBUG = level - parser = PythonParser( gram ) + parser = PythonParser( gram.rules, gram.items ) return parser +def python_grammar_dyn(fname): + """Loads the grammar using the 'dynamic' rpython parser""" + _grammar_file = file(fname) + ebnfbuilder = ebnfparse.parse_grammar_text( file(fname).read() ) + ebnfbuilder.resolve_rules() + parser = PythonParser( ebnfbuilder.root_rules, ebnfbuilder.all_rules ) + return parser + + debug_print( "Loading grammar %s" % PYTHON_GRAMMAR ) PYTHON_PARSER = python_grammar( PYTHON_GRAMMAR ) +PYTHON_PARSER_DYN = python_grammar_dyn( PYTHON_GRAMMAR ) def reload_grammar(version): """helper function to test with pypy different grammars""" Modified: pypy/branch/ast-experiments/pypy/interpreter/pyparser/test/test_astbuilder.py ============================================================================== --- pypy/branch/ast-experiments/pypy/interpreter/pyparser/test/test_astbuilder.py (original) +++ pypy/branch/ast-experiments/pypy/interpreter/pyparser/test/test_astbuilder.py Mon Jan 23 18:14:22 2006 @@ -1,6 +1,6 @@ import os -from pypy.interpreter.pyparser.pythonparse import PYTHON_PARSER +from pypy.interpreter.pyparser.pythonparse import PYTHON_PARSER_DYN as PYTHON_PARSER from pypy.interpreter.pyparser.astbuilder import AstBuilder from pypy.interpreter.pyparser.pythonutil import ast_from_input from pypy.interpreter.stablecompiler.transformer import Transformer From mwh at codespeak.net Mon Jan 23 18:30:18 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Mon, 23 Jan 2006 18:30:18 +0100 (CET) Subject: [pypy-svn] r22543 - in pypy/dist/pypy/rpython/l3interp: . test Message-ID: <20060123173018.9E98D27DC7@code1.codespeak.net> Author: mwh Date: Mon Jan 23 18:30:16 2006 New Revision: 22543 Modified: pypy/dist/pypy/rpython/l3interp/convertgraph.py pypy/dist/pypy/rpython/l3interp/l3interp.py pypy/dist/pypy/rpython/l3interp/test/test_convert.py Log: (arre, mwh) Actual, as opposed to pretend, support for void returns in the l3interpreter and converter. Modified: pypy/dist/pypy/rpython/l3interp/convertgraph.py ============================================================================== --- pypy/dist/pypy/rpython/l3interp/convertgraph.py (original) +++ pypy/dist/pypy/rpython/l3interp/convertgraph.py Mon Jan 23 18:30:16 2006 @@ -87,7 +87,8 @@ return position - stacksizes[kind] # < 0 for v in block.inputargs: - push(v) + if v.concretetype is not lltype.Void: + push(v) insns = [] l3block = model.Block(insns) @@ -136,8 +137,9 @@ 'dbl': [], 'ptr': []} for v in link.args: - kind = getkind(v.concretetype) - targetregs[kind].append(get(v)) + if v.concretetype is not lltype.Void: + kind = getkind(v.concretetype) + targetregs[kind].append(get(v)) return model.Link(convert_block(link.target, memo), targetregs['int'] or None, targetregs['dbl'] or None, Modified: pypy/dist/pypy/rpython/l3interp/l3interp.py ============================================================================== --- pypy/dist/pypy/rpython/l3interp/l3interp.py (original) +++ pypy/dist/pypy/rpython/l3interp/l3interp.py Mon Jan 23 18:30:16 2006 @@ -24,6 +24,8 @@ def __init__(self, ptrval): self.ptrval = ptrval +class L3Nothing(L3Value): + pass def l3interpret(graph, args_int, args_dbl, args_ptr): assert len(args_int) == graph.nargs_int @@ -40,6 +42,8 @@ return L3Double(args_dbl.pop()) if nint == 0 and ndbl == 0 and nptr == 1: return L3Pointer(args_ptr.pop()) + if nint == 0 and ndbl == 0 and nptr == 0: + return L3Nothing() raise AssertionError("stacks corrupted") constant_offset = OffsetOf(lltype.Void) Modified: pypy/dist/pypy/rpython/l3interp/test/test_convert.py ============================================================================== --- pypy/dist/pypy/rpython/l3interp/test/test_convert.py (original) +++ pypy/dist/pypy/rpython/l3interp/test/test_convert.py Mon Jan 23 18:30:16 2006 @@ -46,6 +46,14 @@ result = l3interp.l3interpret(l3graph, [0], [], []) assert isinstance(result, l3interp.L3Integer) assert result.intval == 1 + +def test_void_return(): + def f(): + pass + l3graph = l3ify(f, []) + result = l3interp.l3interpret(l3graph, [], [], []) + assert isinstance(result, l3interp.L3Nothing) + def test_convert_getfield(): class C: From gromit at codespeak.net Mon Jan 23 18:37:03 2006 From: gromit at codespeak.net (gromit at codespeak.net) Date: Mon, 23 Jan 2006 18:37:03 +0100 (CET) Subject: [pypy-svn] r22544 - pypy/dist/pypy/rpython/rctypes/test Message-ID: <20060123173703.8EE9F27DD8@code1.codespeak.net> Author: gromit Date: Mon Jan 23 18:37:02 2006 New Revision: 22544 Modified: pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py Log: ADD: (gromit,stephan) A rtyping testcase for atoi. Modified: pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py Mon Jan 23 18:37:02 2006 @@ -5,25 +5,37 @@ import ctypes except ImportError: py.test.skip("this test needs ctypes installed") - -class Test_rctypes: - def test_simple(self): - from pypy.rpython.rctypes.interface import cdll, c_char_p, c_int - + else: import sys + from pypy.rpython.rctypes.interface import cdll, c_char_p, c_int if sys.platform == 'win32': mylib = cdll.LoadLibrary('msvcrt.dll') elif sys.platform == 'linux2': mylib = cdll.LoadLibrary('libc.so.6') else: py.test.skip("don't know how to load the c lib for %s" % - sys.platform) + sys.platform) atoi = mylib.atoi atoi.restype = c_int atoi.argstype = [c_char_p] def o_atoi(a): return atoi(a) + mod.o_atoi = o_atoi + + +class Test_rctypes: + + from pypy.annotation.annrpython import RPythonAnnotator + + def test_simple(self): + res = o_atoi('42') assert res == 42 + + def inprogress_test_annotate_simple(self): + a = self.RPythonAnnotator() + s = a.build_types(o_atoi, [str]) + # result should be an integer + assert s.knowntype == int From cfbolz at codespeak.net Mon Jan 23 18:50:00 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Mon, 23 Jan 2006 18:50:00 +0100 (CET) Subject: [pypy-svn] r22545 - in pypy/dist/pypy: annotation rpython rpython/lltypesystem translator/c Message-ID: <20060123175000.C312B27DDB@code1.codespeak.net> Author: cfbolz Date: Mon Jan 23 18:49:56 2006 New Revision: 22545 Modified: pypy/dist/pypy/annotation/model.py pypy/dist/pypy/rpython/lltypesystem/llmemory.py pypy/dist/pypy/rpython/raddress.py pypy/dist/pypy/rpython/rbuiltin.py pypy/dist/pypy/translator/c/database.py pypy/dist/pypy/translator/c/primitive.py Log: (arigo, cfbolz): maybe intermediate checkin removed the Offset Primitive and replaced it by Signed everywhere. Modified: pypy/dist/pypy/annotation/model.py ============================================================================== --- pypy/dist/pypy/annotation/model.py (original) +++ pypy/dist/pypy/annotation/model.py Mon Jan 23 18:49:56 2006 @@ -514,6 +514,7 @@ annotation_to_ll_map = [ (s_None, lltype.Void), # also matches SomeImpossibleValue() (SomeBool(), lltype.Bool), + (SomeOffset(), lltype.Signed), (SomeInteger(), lltype.Signed), (SomeInteger(size=2), lltype.SignedLongLong), (SomeInteger(nonneg=True, unsigned=True), lltype.Unsigned), @@ -522,7 +523,6 @@ (SomeChar(), lltype.Char), (SomeUnicodeCodePoint(), lltype.UniChar), (SomeAddress(), llmemory.Address), - (SomeOffset(), llmemory.Offset), ] def annotation_to_lltype(s_val, info=None): Modified: pypy/dist/pypy/rpython/lltypesystem/llmemory.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/llmemory.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/llmemory.py Mon Jan 23 18:49:56 2006 @@ -18,7 +18,7 @@ return model.SomeOffset() def lltype(self): - return Offset + return lltype.Signed def __repr__(self): return "" % (self.TYPE, self.fldnames) @@ -32,8 +32,6 @@ assert t == other.TYPE return OffsetOf(self.TYPE, *(self.fldnames + other.fldnames)) -Offset = lltype.Primitive("Offset", OffsetOf(lltype.Void)) - def sizeof(TYPE, n=None): pass Modified: pypy/dist/pypy/rpython/raddress.py ============================================================================== --- pypy/dist/pypy/rpython/raddress.py (original) +++ pypy/dist/pypy/rpython/raddress.py Mon Jan 23 18:49:56 2006 @@ -2,7 +2,7 @@ from pypy.annotation.pairtype import pairtype from pypy.annotation import model as annmodel from pypy.rpython.memory.lladdress import NULL, address -from pypy.rpython.lltypesystem.llmemory import Address, Offset +from pypy.rpython.lltypesystem.llmemory import Address from pypy.rpython.rmodel import Repr, IntegerRepr from pypy.rpython.lltypesystem import lltype @@ -120,16 +120,16 @@ return self.__class__, class OffsetRepr(Repr): - lowleveltype = Offset + lowleveltype = lltype.Signed offset_repr = OffsetRepr() class __extend__(pairtype(OffsetRepr, OffsetRepr)): def rtype_add((r_offset1, r_offset2), hop): - v_offset1, v_offset2 = hop.inputargs(Offset, Offset) - return hop.genop('offset_add', [v_offset1, v_offset2], resulttype=Offset) + v_offset1, v_offset2 = hop.inputargs(offset_repr, offset_repr) + return hop.genop('offset_add', [v_offset1, v_offset2], resulttype=lltype.Signed) class __extend__(pairtype(AddressRepr, OffsetRepr)): def rtype_add((r_offset1, r_offset2), hop): - v_offset1, v_offset2 = hop.inputargs(Address, Offset) + v_offset1, v_offset2 = hop.inputargs(Address, offset_repr) return hop.genop('adr_offset_add', [v_offset1, v_offset2], resulttype=Address) Modified: pypy/dist/pypy/rpython/rbuiltin.py ============================================================================== --- pypy/dist/pypy/rpython/rbuiltin.py (original) +++ pypy/dist/pypy/rpython/rbuiltin.py Mon Jan 23 18:49:56 2006 @@ -381,7 +381,7 @@ def rtype_offsetof(hop): TYPE, field = hop.inputargs(lltype.Void, lltype.Void) - return hop.inputconst(llmemory.Offset, + return hop.inputconst(lltype.Signed, llmemory.offsetof(TYPE.value, field.value)) BUILTIN_TYPER[llmemory.offsetof] = rtype_offsetof Modified: pypy/dist/pypy/translator/c/database.py ============================================================================== --- pypy/dist/pypy/translator/c/database.py (original) +++ pypy/dist/pypy/translator/c/database.py Mon Jan 23 18:49:56 2006 @@ -2,7 +2,7 @@ Primitive, Ptr, typeOf, RuntimeTypeInfo, \ Struct, Array, FuncType, PyObject, Void, \ ContainerType, OpaqueType -from pypy.rpython.lltypesystem.llmemory import Offset, Address +from pypy.rpython.lltypesystem.llmemory import Address from pypy.rpython.memory.lladdress import NULL from pypy.translator.c.primitive import PrimitiveName, PrimitiveType from pypy.translator.c.primitive import PrimitiveErrorValue @@ -121,22 +121,8 @@ raise Exception("don't know about %r" % (T,)) else: T = typeOf(obj) - if T is Offset: - assert len(obj.fldnames) <= 1 - if len(obj.fldnames) == 0: - return '0 /*offsetof*/' - else: - structnode = self.gettypedefnode(obj.TYPE.TO) - return 'offsetof(struct %s, %s)'%( - structnode.name, - structnode.c_struct_field_name(obj.fldnames[0])) - elif T is Address and obj is not NULL: - if obj.ob is None: - return 'NULL' - else: - return self.get(obj.ob) - elif isinstance(T, Primitive): - return PrimitiveName[T](obj) + if isinstance(T, Primitive): + return PrimitiveName[T](obj, self) elif isinstance(T, Ptr): if obj: # test if the ptr is non-NULL node = self.getcontainernode(obj._obj) Modified: pypy/dist/pypy/translator/c/primitive.py ============================================================================== --- pypy/dist/pypy/translator/c/primitive.py (original) +++ pypy/dist/pypy/translator/c/primitive.py Mon Jan 23 18:49:56 2006 @@ -1,33 +1,43 @@ import sys from pypy.rpython.lltypesystem.lltype import * -from pypy.rpython.lltypesystem.llmemory import Address, Offset +from pypy.rpython.lltypesystem.llmemory import Address, OffsetOf, fakeaddress from pypy.rpython.memory.lladdress import NULL # ____________________________________________________________ # # Primitives -def name_signed(value): +def name_signed(value, db): + if isinstance(value, Symbolic): + assert isinstance(value, OffsetOf) + assert len(value.fldnames) <= 1 + if len(value.fldnames) == 0: + return '0 /*offsetof*/' + else: + structnode = db.gettypedefnode(value.TYPE.TO) + return 'offsetof(struct %s, %s)'%( + structnode.name, + structnode.c_struct_field_name(value.fldnames[0])) if value == -sys.maxint-1: # blame C return '(-%dL-1L)' % sys.maxint else: return '%dL' % value -def name_unsigned(value): +def name_unsigned(value, db): assert value >= 0 return '%dUL' % value -def name_unsignedlonglong(value): +def name_unsignedlonglong(value, db): assert value >= 0 return '%dULL' % value -def name_signedlonglong(value): +def name_signedlonglong(value, db): return '%dLL' % value def isinf(x): return x != 0.0 and x / 2 == x -def name_float(value): +def name_float(value, db): if isinf(value): if value > 0: return '(Py_HUGE_VAL)' @@ -36,27 +46,31 @@ else: return repr(value) -def name_char(value): +def name_char(value, db): assert type(value) is str and len(value) == 1 if ' ' <= value < '\x7f': return "'%s'" % (value.replace("\\", r"\\").replace("'", r"\'"),) else: return '%d' % ord(value) -def name_bool(value): +def name_bool(value, db): return '%d' % value -def name_void(value): +def name_void(value, db): return '/* nothing */' -def name_unichar(value): +def name_unichar(value, db): assert type(value) is unicode and len(value) == 1 return '%d' % ord(value) -def name_address(value): - # XXX this will need to change! - assert value == NULL - return 'NULL' +def name_address(value, db): + if value is NULL: + return 'NULL' + assert isinstance(value, fakeaddress) + if value.ob is None: + return 'NULL' + else: + return db.get(value.ob) PrimitiveName = { Signed: name_signed, @@ -82,7 +96,6 @@ Bool: 'char @', Void: 'void @', Address: 'void* @', - Offset: 'long @', } PrimitiveErrorValue = { @@ -96,5 +109,4 @@ Bool: '((char) -1)', Void: '/* error */', Address: 'NULL', - Offset: '0', } From mwh at codespeak.net Mon Jan 23 19:27:33 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Mon, 23 Jan 2006 19:27:33 +0100 (CET) Subject: [pypy-svn] r22546 - in pypy/dist/pypy/rpython/l3interp: . test Message-ID: <20060123182733.63D2027DC6@code1.codespeak.net> Author: mwh Date: Mon Jan 23 19:27:30 2006 New Revision: 22546 Modified: pypy/dist/pypy/rpython/l3interp/convertgraph.py pypy/dist/pypy/rpython/l3interp/l3interp.py pypy/dist/pypy/rpython/l3interp/model.py pypy/dist/pypy/rpython/l3interp/test/test_l3interp.py Log: (arre,mwh) A test and support for the l3 operation 'setfield_int'. We were *extremely* surprised that this passed when it did :) Modified: pypy/dist/pypy/rpython/l3interp/convertgraph.py ============================================================================== --- pypy/dist/pypy/rpython/l3interp/convertgraph.py (original) +++ pypy/dist/pypy/rpython/l3interp/convertgraph.py Mon Jan 23 19:27:30 2006 @@ -125,6 +125,22 @@ res = len(clist) clist.append(offset) insns.append(res) + elif spaceop.opname == 'setfield': + v0, v1, v2 = spaceop.args + opname = spaceop.opname + '_' + \ + getaccesskind(v2.concretetype) + insns.append(model.very_low_level_opcode[opname]) + insns.append(get(v0)) + + offset = OffsetOf(v0.concretetype, v1.value) + clist = constants['offset'] + try: + res = clist.index(offset) + except ValueError: + res = len(clist) + clist.append(offset) + insns.append(res) + insns.append(get(v2)) else: insns.append(model.very_low_level_opcode[spaceop.opname]) for v in spaceop.args: Modified: pypy/dist/pypy/rpython/l3interp/l3interp.py ============================================================================== --- pypy/dist/pypy/rpython/l3interp/l3interp.py (original) +++ pypy/dist/pypy/rpython/l3interp/l3interp.py Mon Jan 23 19:27:30 2006 @@ -193,6 +193,12 @@ o = self.getoffset() self.stack_int.append((p + o).signed[0]) + def op_setfield_int(self): + p = self.getptr() + o = self.getoffset() + v = self.getint() + (p + o).signed[0] = v + def op_flavored_malloc(self): self.stack_ptr.append(constant_fakeaddress) Modified: pypy/dist/pypy/rpython/l3interp/model.py ============================================================================== --- pypy/dist/pypy/rpython/l3interp/model.py (original) +++ pypy/dist/pypy/rpython/l3interp/model.py Mon Jan 23 19:27:30 2006 @@ -41,7 +41,8 @@ #struct operations: 'getfield_int', 'getfield_char', 'getfield_dbl', 'getfield_ptr', - 'getsubstruct', 'setfield', + 'getsubstruct', + 'setfield_int', 'setfield_char', 'setfield_dbl', 'setfield_ptr', #integer operations: 'int_abs', 'int_abs_ovf', 'int_add', 'int_add_ovf', 'int_and', Modified: pypy/dist/pypy/rpython/l3interp/test/test_l3interp.py ============================================================================== --- pypy/dist/pypy/rpython/l3interp/test/test_l3interp.py (original) +++ pypy/dist/pypy/rpython/l3interp/test/test_l3interp.py Mon Jan 23 19:27:30 2006 @@ -155,8 +155,42 @@ assert isinstance(value, l3interp.L3Integer) return value.intval + assert entry_point(3) == f(3) + assert entry_point(0) == f(0) + fn = translate(entry_point, [int]) assert fn(3) == f(3) assert fn(0) == f(0) + +def test_setfield(): + class C: + def __init__(self, x): + self.x = x + c = C(1) + + def getorset(n): + if n: + c.x = n + return 0 + else: + return c.x + + getorsetgraph = l3ify(getorset, [int]) + + def entry_point(x): + value = l3interp.l3interpret(getorsetgraph, [x], [], []) + assert isinstance(value, l3interp.L3Integer) + return value.intval + + assert entry_point(-3) == 0 + assert entry_point(0) == -3 + + fn = translate(entry_point, [int]) + + assert fn(-3) == 0 + assert fn(0) == -3 + + + From cfbolz at codespeak.net Mon Jan 23 19:28:17 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Mon, 23 Jan 2006 19:28:17 +0100 (CET) Subject: [pypy-svn] r22547 - in pypy/dist/pypy: rpython translator/c/src Message-ID: <20060123182817.53CB827DC6@code1.codespeak.net> Author: cfbolz Date: Mon Jan 23 19:28:13 2006 New Revision: 22547 Modified: pypy/dist/pypy/rpython/llinterp.py pypy/dist/pypy/rpython/raddress.py pypy/dist/pypy/translator/c/src/address.h Log: cfbolz, arigo and samuele around replace some of the operations that are obsole now that Offsets are Signeds on l2 level with the regular int version Modified: pypy/dist/pypy/rpython/llinterp.py ============================================================================== --- pypy/dist/pypy/rpython/llinterp.py (original) +++ pypy/dist/pypy/rpython/llinterp.py Mon Jan 23 19:28:13 2006 @@ -662,6 +662,14 @@ return func(x, y) """ % locals()).compile() + op_original_int_add = op_int_add + + def op_int_add(self, x, y): + if isinstance(x, llmemory.OffsetOf): + return x + y + else: + return self.op_original_int_add(x, y) + def op_unichar_eq(self, x, y): assert isinstance(x, unicode) and len(x) == 1 assert isinstance(y, unicode) and len(y) == 1 Modified: pypy/dist/pypy/rpython/raddress.py ============================================================================== --- pypy/dist/pypy/rpython/raddress.py (original) +++ pypy/dist/pypy/rpython/raddress.py Mon Jan 23 19:28:13 2006 @@ -127,9 +127,9 @@ class __extend__(pairtype(OffsetRepr, OffsetRepr)): def rtype_add((r_offset1, r_offset2), hop): v_offset1, v_offset2 = hop.inputargs(offset_repr, offset_repr) - return hop.genop('offset_add', [v_offset1, v_offset2], resulttype=lltype.Signed) + return hop.genop('int_add', [v_offset1, v_offset2], resulttype=lltype.Signed) class __extend__(pairtype(AddressRepr, OffsetRepr)): def rtype_add((r_offset1, r_offset2), hop): v_offset1, v_offset2 = hop.inputargs(Address, offset_repr) - return hop.genop('adr_offset_add', [v_offset1, v_offset2], resulttype=Address) + return hop.genop('adr_add', [v_offset1, v_offset2], resulttype=Address) Modified: pypy/dist/pypy/translator/c/src/address.h ============================================================================== --- pypy/dist/pypy/translator/c/src/address.h (original) +++ pypy/dist/pypy/translator/c/src/address.h Mon Jan 23 19:28:13 2006 @@ -8,7 +8,6 @@ #define OP_ADR_DELTA(x,y,r,err) r = ((char *)(x) - (char *)(y)) #define OP_ADR_SUB(x,y,r,err) r = ((char *)(x) - (y)) #define OP_ADR_ADD(x,y,r,err) r = ((char *)(x) + (y)) -#define OP_ADR_OFFSET_ADD(x,y,r,err) r = ((char *)(x) + (y)) #define OP_ADR_EQ(x,y,r,err) r = ((x) == (y)) #define OP_ADR_NE(x,y,r,err) r = ((x) != (y)) From ericvrp at codespeak.net Mon Jan 23 19:28:23 2006 From: ericvrp at codespeak.net (ericvrp at codespeak.net) Date: Mon, 23 Jan 2006 19:28:23 +0100 (CET) Subject: [pypy-svn] r22548 - in pypy/dist/pypy/rpython: . module Message-ID: <20060123182823.7EF8D27DD8@code1.codespeak.net> Author: ericvrp Date: Mon Jan 23 19:28:19 2006 New Revision: 22548 Added: pypy/dist/pypy/rpython/module/ll_js.py pypy/dist/pypy/rpython/rjs.py Modified: pypy/dist/pypy/rpython/extfunctable.py Log: Added to be able to call native javascript (external functions). Currently only eval() is supported and it will probably stay this way, at least until performance becomes a problem. Modified: pypy/dist/pypy/rpython/extfunctable.py ============================================================================== --- pypy/dist/pypy/rpython/extfunctable.py (original) +++ pypy/dist/pypy/rpython/extfunctable.py Mon Jan 23 19:28:19 2006 @@ -231,6 +231,11 @@ 'll_stackless/switch')) # ___________________________________________________________ +# javascript +from pypy.rpython import rjs +declare(rjs.jseval, str, 'll_js/jseval') + +# ___________________________________________________________ # the exceptions that can be implicitely raised by some operations standardexceptions = { TypeError : True, Added: pypy/dist/pypy/rpython/module/ll_js.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/rpython/module/ll_js.py Mon Jan 23 19:28:19 2006 @@ -0,0 +1,5 @@ +from pypy.rpython import rjs + +def ll_js_jseval(s): + return rjs.jseval(s) +ll_js_jseval.suggested_primitive = True Added: pypy/dist/pypy/rpython/rjs.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/rpython/rjs.py Mon Jan 23 19:28:19 2006 @@ -0,0 +1,7 @@ +""" +This file defines utilities for calling native javascript code in an +RPython-compliant way, intended for use by genjs. +""" + +def jseval(): + raise NotImplementedError("only works in translated versions") From cfbolz at codespeak.net Mon Jan 23 19:29:12 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Mon, 23 Jan 2006 19:29:12 +0100 (CET) Subject: [pypy-svn] r22549 - pypy/dist/pypy/rpython/memory Message-ID: <20060123182912.48C9427DC6@code1.codespeak.net> Author: cfbolz Date: Mon Jan 23 19:29:10 2006 New Revision: 22549 Modified: pypy/dist/pypy/rpython/memory/lladdress.py pypy/dist/pypy/rpython/memory/lltypelayout.py Log: (cfbolz, rxe) use offsetof in the GCs where appropriate Modified: pypy/dist/pypy/rpython/memory/lladdress.py ============================================================================== --- pypy/dist/pypy/rpython/memory/lladdress.py (original) +++ pypy/dist/pypy/rpython/memory/lladdress.py Mon Jan 23 19:29:10 2006 @@ -49,6 +49,7 @@ def _store(self, fmt, *values): # XXX annoyance: suddenly an OffsetOf changes into a Signed?! + from pypy.rpython.memory.lltypelayout import convert_offset_to_int if len(values) == 1 and isinstance(values[0], llmemory.OffsetOf): values = [convert_offset_to_int(values[0])] simulator.setstruct(fmt, self.intaddress, *values) Modified: pypy/dist/pypy/rpython/memory/lltypelayout.py ============================================================================== --- pypy/dist/pypy/rpython/memory/lltypelayout.py (original) +++ pypy/dist/pypy/rpython/memory/lltypelayout.py Mon Jan 23 19:29:10 2006 @@ -102,15 +102,14 @@ def offsets_to_gc_pointers(TYPE): if isinstance(TYPE, lltype.Struct): offsets = [] - layout = get_layout(TYPE) for name in TYPE._names: FIELD = getattr(TYPE, name) if (isinstance(FIELD, lltype.Ptr) and FIELD._needsgc() and FIELD.TO is not lltype.PyObject): - offsets.append(layout[name]) + offsets.append(llmemory.offsetof(TYPE, name)) elif isinstance(FIELD, lltype.Struct): suboffsets = offsets_to_gc_pointers(FIELD) - offsets += [s + layout[name] for s in suboffsets] + offsets += [s + llmemory.offsetof(TYPE, name) for s in suboffsets] return offsets return [] @@ -124,7 +123,7 @@ def varsize_offsets_to_gcpointers_in_var_part(TYPE): if isinstance(TYPE, lltype.Array): if isinstance(TYPE.OF, lltype.Ptr): - return [0] + return [llmemory.OffsetOf(TYPE.OF)] elif isinstance(TYPE.OF, lltype.Struct): return offsets_to_gc_pointers(TYPE.OF) return [] From ericvrp at codespeak.net Mon Jan 23 19:30:46 2006 From: ericvrp at codespeak.net (ericvrp at codespeak.net) Date: Mon, 23 Jan 2006 19:30:46 +0100 (CET) Subject: [pypy-svn] r22550 - in pypy/dist/pypy/translator/js: . test Message-ID: <20060123183046.102E927B68@code1.codespeak.net> Author: ericvrp Date: Mon Jan 23 19:30:43 2006 New Revision: 22550 Added: pypy/dist/pypy/translator/js/test/test_jseval.py Modified: pypy/dist/pypy/translator/js/optimize.py Log: Implementation and tests for jseval external function. Modified: pypy/dist/pypy/translator/js/optimize.py ============================================================================== --- pypy/dist/pypy/translator/js/optimize.py (original) +++ pypy/dist/pypy/translator/js/optimize.py Mon Jan 23 19:30:43 2006 @@ -4,10 +4,14 @@ 'll_stritem_nonneg__rpy_stringPtr_Signed', 'll_stritem__rpy_stringPtr_Signed', 'll_streq__rpy_stringPtr_rpy_stringPtr', - 'll_str__IntegerR_SignedConst_Signed', 'll_chr2str__Char', - 'll_issubclass__object_vtablePtr_object_vtablePtr', + 'll_str__IntegerR_SignedConst_Signed', 'll_str__FloatR_FloatConst_Float', + + #'ll_issubclass__object_vtablePtr_object_vtablePtr', + + #externals... + 'll_js_jseval__rpy_stringPtr', ] @@ -37,12 +41,16 @@ return True, '%s = (%s == %s) || (%s && %s && %s.chars == %s.chars)' %\ (targetvar, s0,s1, s0,s1, s0,s1) + elif funcname == 'll_chr2str__Char': + return True, '%s = new Object({hash:0, chars:%s})' % (targetvar, params[0]) + elif funcname in ('ll_str__IntegerR_SignedConst_Signed', 'll_str__FloatR_FloatConst_Float'): return True, '%s = new Object({hash:0, chars:%s + ""})' % (targetvar, params[0]) - elif funcname == 'll_chr2str__Char': - return True, '%s = new Object({hash:0, chars:%s})' % (targetvar, params[0]) + #externals... + elif funcname == 'll_js_jseval__rpy_stringPtr': + return True, '%s = eval(%s)' % (targetvar, params[0]) return False, '%s = %s(%s)' % (targetvar, funcname, ', '.join(params)) Added: pypy/dist/pypy/translator/js/test/test_jseval.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/js/test/test_jseval.py Mon Jan 23 19:30:43 2006 @@ -0,0 +1,12 @@ +from __future__ import division +import py + +from pypy.translator.js.test.runtest import compile_function +from pypy.rpython.rjs import jseval + +def test_jseval1(): + def jseval1(s): + return jseval(s) + jseval1_fn = compile_function(jseval1, [str]) + e = "4+7" + assert jseval1_fn(e) == eval(e) From mwh at codespeak.net Mon Jan 23 19:43:13 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Mon, 23 Jan 2006 19:43:13 +0100 (CET) Subject: [pypy-svn] r22551 - in pypy/dist/pypy/rpython: l3interp l3interp/test lltypesystem Message-ID: <20060123184313.D09C027B68@code1.codespeak.net> Author: mwh Date: Mon Jan 23 19:43:06 2006 New Revision: 22551 Modified: pypy/dist/pypy/rpython/l3interp/l3interp.py pypy/dist/pypy/rpython/l3interp/test/test_convert.py pypy/dist/pypy/rpython/l3interp/test/test_l3interp.py pypy/dist/pypy/rpython/lltypesystem/llmemory.py Log: (arre,mwh) Note a deficiency in l3interp.convertgraph(). A test and implementation for getfield_ptr and what was necessary to l3interp this on the llmemory style of address. Modified: pypy/dist/pypy/rpython/l3interp/l3interp.py ============================================================================== --- pypy/dist/pypy/rpython/l3interp/l3interp.py (original) +++ pypy/dist/pypy/rpython/l3interp/l3interp.py Mon Jan 23 19:43:06 2006 @@ -193,6 +193,11 @@ o = self.getoffset() self.stack_int.append((p + o).signed[0]) + def op_getfield_ptr(self): + p = self.getptr() + o = self.getoffset() + self.stack_ptr.append((p + o).address[0]) + def op_setfield_int(self): p = self.getptr() o = self.getoffset() Modified: pypy/dist/pypy/rpython/l3interp/test/test_convert.py ============================================================================== --- pypy/dist/pypy/rpython/l3interp/test/test_convert.py (original) +++ pypy/dist/pypy/rpython/l3interp/test/test_convert.py Mon Jan 23 19:43:06 2006 @@ -76,3 +76,13 @@ assert isinstance(result, l3interp.L3Integer) assert result.intval == 2 +def dont_test_call(): + def f(): + return 2 + def g(): + return f() + l3graph = l3ify(g, []) + result = l3interp.l3interpret(l3graph, [], [], []) + assert isinstance(result, l3interp.L3Integer) + assert result.intval == g() + Modified: pypy/dist/pypy/rpython/l3interp/test/test_l3interp.py ============================================================================== --- pypy/dist/pypy/rpython/l3interp/test/test_l3interp.py (original) +++ pypy/dist/pypy/rpython/l3interp/test/test_l3interp.py Mon Jan 23 19:43:06 2006 @@ -194,3 +194,47 @@ +def test_getfield_complex(): + class C: + def __init__(self, x): + self.x = x + one = C(1) + two = C(2) + + class D: + def __init__(self, c1, c2): + self.c1 = c1 + self.c2 = c2 + + d1 = D(one, two) + d2 = D(two, one) + + def f(n, m): + if n: + if m: + return d1.c1.x + else: + return d1.c2.x + else: + if m: + return d2.c1.x + else: + return d2.c2.x + + l3graph = l3ify(f, [int, int]) + + def entry_point(x, y): + value = l3interp.l3interpret(l3graph, [x, y], [], []) + assert isinstance(value, l3interp.L3Integer) + return value.intval + + for x in 0, 1: + for y in 0, 1: + assert entry_point(x, y) == f(x, y) + + fn = translate(entry_point, [int, int]) + + for x in 0, 1: + for y in 0, 1: + assert fn(x, y) == f(x, y) + Modified: pypy/dist/pypy/rpython/lltypesystem/llmemory.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/llmemory.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/llmemory.py Mon Jan 23 19:43:06 2006 @@ -63,14 +63,36 @@ ob = self.addr.ob for n in self.addr.offset.fldnames: ob = getattr(ob, n) - # XXX will need to do pointers differently! - assert lltype.typeOf(ob) == self.TYPE - return ob + return self.vet(ob) + + def __setitem__(self, index, value): + assert index == 0 + ob = self.addr.ob + for n in self.addr.offset.fldnames[:-1]: + ob = getattr(ob, n) + fld = self.addr.offset.fldnames[-1] + assert lltype.typeOf(value) == self.TYPE + assert lltype.typeOf(ob).TO._flds[fld] == self.TYPE + setattr(ob, fld, value) class _signed_fakeaccessor(_fakeaccessor): TYPE = lltype.Signed + def vet(self, value): + assert lltype.typeOf(value) == self.TYPE + return value + +class _address_fakeaccessor(_fakeaccessor): + TYPE = None + + def vet(self, value): + # XXX is this the right check for "Ptr-ness" ? + assert isinstance(value, lltype._ptr) + return fakeaddress(value) + + fakeaddress.signed = property(_signed_fakeaccessor) +fakeaddress.address = property(_address_fakeaccessor) Address = lltype.Primitive("Address", fakeaddress(None)) From adim at codespeak.net Tue Jan 24 09:52:26 2006 From: adim at codespeak.net (adim at codespeak.net) Date: Tue, 24 Jan 2006 09:52:26 +0100 (CET) Subject: [pypy-svn] r22556 - pypy/branch/ast-experiments/pypy/interpreter/astcompiler Message-ID: <20060124085226.991D427B84@code1.codespeak.net> Author: adim Date: Tue Jan 24 09:52:24 2006 New Revision: 22556 Modified: pypy/branch/ast-experiments/pypy/interpreter/astcompiler/ast.py pypy/branch/ast-experiments/pypy/interpreter/astcompiler/astgen.py Log: added insert_after and insert_before methods on appropriate Node classes (exposed those methods at applevel) Modified: pypy/branch/ast-experiments/pypy/interpreter/astcompiler/ast.py ============================================================================== --- pypy/branch/ast-experiments/pypy/interpreter/astcompiler/ast.py (original) +++ pypy/branch/ast-experiments/pypy/interpreter/astcompiler/ast.py Tue Jan 24 09:52:24 2006 @@ -291,6 +291,15 @@ del self.nodes[:] for w_itm in space.unpackiterable(w_arg): self.nodes.append( space.interp_w(Node, w_itm)) + def descr_insert_after(space, self, node, w_added_nodes): + added_nodes = [space.interp_w(Node, w_node) for w_node in space.unpackiterable(w_added_nodes)] + index = self.nodes.index(node) + 1 + self.nodes[index:index] = added_nodes + + def descr_insert_before(space, self, node, w_added_nodes): + added_nodes = [space.interp_w(Node, w_node) for w_node in space.unpackiterable(w_added_nodes)] + index = self.nodes.index(node) + self.nodes[index:index] = added_nodes def descr_And_new(space, w_subtype, w_nodes, lineno=-1): self = space.allocate_instance(And, w_subtype) @@ -308,6 +317,8 @@ __new__ = interp2app(descr_And_new, unwrap_spec=[ObjSpace, W_Root, W_Root, int]), accept=interp2app(descr_And_accept, unwrap_spec=[ObjSpace, W_Root, W_Root] ), nodes=GetSetProperty(And.fget_nodes, And.fset_nodes ), + insert_after=interp2app(And.descr_insert_after.im_func, unwrap_spec=[ObjSpace, And, Node, W_Root]), + insert_before=interp2app(And.descr_insert_before.im_func, unwrap_spec=[ObjSpace, And, Node, W_Root]), ) class AssAttr(Node): @@ -426,6 +437,15 @@ del self.nodes[:] for w_itm in space.unpackiterable(w_arg): self.nodes.append( space.interp_w(Node, w_itm)) + def descr_insert_after(space, self, node, w_added_nodes): + added_nodes = [space.interp_w(Node, w_node) for w_node in space.unpackiterable(w_added_nodes)] + index = self.nodes.index(node) + 1 + self.nodes[index:index] = added_nodes + + def descr_insert_before(space, self, node, w_added_nodes): + added_nodes = [space.interp_w(Node, w_node) for w_node in space.unpackiterable(w_added_nodes)] + index = self.nodes.index(node) + self.nodes[index:index] = added_nodes def descr_AssList_new(space, w_subtype, w_nodes, lineno=-1): self = space.allocate_instance(AssList, w_subtype) @@ -443,6 +463,8 @@ __new__ = interp2app(descr_AssList_new, unwrap_spec=[ObjSpace, W_Root, W_Root, int]), accept=interp2app(descr_AssList_accept, unwrap_spec=[ObjSpace, W_Root, W_Root] ), nodes=GetSetProperty(AssList.fget_nodes, AssList.fset_nodes ), + insert_after=interp2app(AssList.descr_insert_after.im_func, unwrap_spec=[ObjSpace, AssList, Node, W_Root]), + insert_before=interp2app(AssList.descr_insert_before.im_func, unwrap_spec=[ObjSpace, AssList, Node, W_Root]), ) class AssName(Node): @@ -534,6 +556,15 @@ del self.nodes[:] for w_itm in space.unpackiterable(w_arg): self.nodes.append( space.interp_w(Node, w_itm)) + def descr_insert_after(space, self, node, w_added_nodes): + added_nodes = [space.interp_w(Node, w_node) for w_node in space.unpackiterable(w_added_nodes)] + index = self.nodes.index(node) + 1 + self.nodes[index:index] = added_nodes + + def descr_insert_before(space, self, node, w_added_nodes): + added_nodes = [space.interp_w(Node, w_node) for w_node in space.unpackiterable(w_added_nodes)] + index = self.nodes.index(node) + self.nodes[index:index] = added_nodes def descr_AssTuple_new(space, w_subtype, w_nodes, lineno=-1): self = space.allocate_instance(AssTuple, w_subtype) @@ -551,6 +582,8 @@ __new__ = interp2app(descr_AssTuple_new, unwrap_spec=[ObjSpace, W_Root, W_Root, int]), accept=interp2app(descr_AssTuple_accept, unwrap_spec=[ObjSpace, W_Root, W_Root] ), nodes=GetSetProperty(AssTuple.fget_nodes, AssTuple.fset_nodes ), + insert_after=interp2app(AssTuple.descr_insert_after.im_func, unwrap_spec=[ObjSpace, AssTuple, Node, W_Root]), + insert_before=interp2app(AssTuple.descr_insert_before.im_func, unwrap_spec=[ObjSpace, AssTuple, Node, W_Root]), ) class Assert(Node): @@ -643,6 +676,15 @@ del self.nodes[:] for w_itm in space.unpackiterable(w_arg): self.nodes.append( space.interp_w(Node, w_itm)) + def descr_insert_after(space, self, node, w_added_nodes): + added_nodes = [space.interp_w(Node, w_node) for w_node in space.unpackiterable(w_added_nodes)] + index = self.nodes.index(node) + 1 + self.nodes[index:index] = added_nodes + + def descr_insert_before(space, self, node, w_added_nodes): + added_nodes = [space.interp_w(Node, w_node) for w_node in space.unpackiterable(w_added_nodes)] + index = self.nodes.index(node) + self.nodes[index:index] = added_nodes def fget_expr( space, self): return space.wrap(self.expr) def fset_expr( space, self, w_arg): @@ -666,6 +708,8 @@ __new__ = interp2app(descr_Assign_new, unwrap_spec=[ObjSpace, W_Root, W_Root, W_Root, int]), accept=interp2app(descr_Assign_accept, unwrap_spec=[ObjSpace, W_Root, W_Root] ), nodes=GetSetProperty(Assign.fget_nodes, Assign.fset_nodes ), + insert_after=interp2app(Assign.descr_insert_after.im_func, unwrap_spec=[ObjSpace, Assign, Node, W_Root]), + insert_before=interp2app(Assign.descr_insert_before.im_func, unwrap_spec=[ObjSpace, Assign, Node, W_Root]), expr=GetSetProperty(Assign.fget_expr, Assign.fset_expr ), ) @@ -859,6 +903,15 @@ del self.nodes[:] for w_itm in space.unpackiterable(w_arg): self.nodes.append( space.interp_w(Node, w_itm)) + def descr_insert_after(space, self, node, w_added_nodes): + added_nodes = [space.interp_w(Node, w_node) for w_node in space.unpackiterable(w_added_nodes)] + index = self.nodes.index(node) + 1 + self.nodes[index:index] = added_nodes + + def descr_insert_before(space, self, node, w_added_nodes): + added_nodes = [space.interp_w(Node, w_node) for w_node in space.unpackiterable(w_added_nodes)] + index = self.nodes.index(node) + self.nodes[index:index] = added_nodes def descr_Bitand_new(space, w_subtype, w_nodes, lineno=-1): self = space.allocate_instance(Bitand, w_subtype) @@ -876,6 +929,8 @@ __new__ = interp2app(descr_Bitand_new, unwrap_spec=[ObjSpace, W_Root, W_Root, int]), accept=interp2app(descr_Bitand_accept, unwrap_spec=[ObjSpace, W_Root, W_Root] ), nodes=GetSetProperty(Bitand.fget_nodes, Bitand.fset_nodes ), + insert_after=interp2app(Bitand.descr_insert_after.im_func, unwrap_spec=[ObjSpace, Bitand, Node, W_Root]), + insert_before=interp2app(Bitand.descr_insert_before.im_func, unwrap_spec=[ObjSpace, Bitand, Node, W_Root]), ) class Bitor(BitOp): @@ -904,6 +959,15 @@ del self.nodes[:] for w_itm in space.unpackiterable(w_arg): self.nodes.append( space.interp_w(Node, w_itm)) + def descr_insert_after(space, self, node, w_added_nodes): + added_nodes = [space.interp_w(Node, w_node) for w_node in space.unpackiterable(w_added_nodes)] + index = self.nodes.index(node) + 1 + self.nodes[index:index] = added_nodes + + def descr_insert_before(space, self, node, w_added_nodes): + added_nodes = [space.interp_w(Node, w_node) for w_node in space.unpackiterable(w_added_nodes)] + index = self.nodes.index(node) + self.nodes[index:index] = added_nodes def descr_Bitor_new(space, w_subtype, w_nodes, lineno=-1): self = space.allocate_instance(Bitor, w_subtype) @@ -921,6 +985,8 @@ __new__ = interp2app(descr_Bitor_new, unwrap_spec=[ObjSpace, W_Root, W_Root, int]), accept=interp2app(descr_Bitor_accept, unwrap_spec=[ObjSpace, W_Root, W_Root] ), nodes=GetSetProperty(Bitor.fget_nodes, Bitor.fset_nodes ), + insert_after=interp2app(Bitor.descr_insert_after.im_func, unwrap_spec=[ObjSpace, Bitor, Node, W_Root]), + insert_before=interp2app(Bitor.descr_insert_before.im_func, unwrap_spec=[ObjSpace, Bitor, Node, W_Root]), ) class Bitxor(BitOp): @@ -949,6 +1015,15 @@ del self.nodes[:] for w_itm in space.unpackiterable(w_arg): self.nodes.append( space.interp_w(Node, w_itm)) + def descr_insert_after(space, self, node, w_added_nodes): + added_nodes = [space.interp_w(Node, w_node) for w_node in space.unpackiterable(w_added_nodes)] + index = self.nodes.index(node) + 1 + self.nodes[index:index] = added_nodes + + def descr_insert_before(space, self, node, w_added_nodes): + added_nodes = [space.interp_w(Node, w_node) for w_node in space.unpackiterable(w_added_nodes)] + index = self.nodes.index(node) + self.nodes[index:index] = added_nodes def descr_Bitxor_new(space, w_subtype, w_nodes, lineno=-1): self = space.allocate_instance(Bitxor, w_subtype) @@ -966,6 +1041,8 @@ __new__ = interp2app(descr_Bitxor_new, unwrap_spec=[ObjSpace, W_Root, W_Root, int]), accept=interp2app(descr_Bitxor_accept, unwrap_spec=[ObjSpace, W_Root, W_Root] ), nodes=GetSetProperty(Bitxor.fget_nodes, Bitxor.fset_nodes ), + insert_after=interp2app(Bitxor.descr_insert_after.im_func, unwrap_spec=[ObjSpace, Bitxor, Node, W_Root]), + insert_before=interp2app(Bitxor.descr_insert_before.im_func, unwrap_spec=[ObjSpace, Bitxor, Node, W_Root]), ) class Break(Node): @@ -1341,6 +1418,15 @@ del self.nodes[:] for w_itm in space.unpackiterable(w_arg): self.nodes.append( space.interp_w(Node, w_itm)) + def descr_insert_after(space, self, node, w_added_nodes): + added_nodes = [space.interp_w(Node, w_node) for w_node in space.unpackiterable(w_added_nodes)] + index = self.nodes.index(node) + 1 + self.nodes[index:index] = added_nodes + + def descr_insert_before(space, self, node, w_added_nodes): + added_nodes = [space.interp_w(Node, w_node) for w_node in space.unpackiterable(w_added_nodes)] + index = self.nodes.index(node) + self.nodes[index:index] = added_nodes def descr_Decorators_new(space, w_subtype, w_nodes, lineno=-1): self = space.allocate_instance(Decorators, w_subtype) @@ -1358,6 +1444,8 @@ __new__ = interp2app(descr_Decorators_new, unwrap_spec=[ObjSpace, W_Root, W_Root, int]), accept=interp2app(descr_Decorators_accept, unwrap_spec=[ObjSpace, W_Root, W_Root] ), nodes=GetSetProperty(Decorators.fget_nodes, Decorators.fset_nodes ), + insert_after=interp2app(Decorators.descr_insert_after.im_func, unwrap_spec=[ObjSpace, Decorators, Node, W_Root]), + insert_before=interp2app(Decorators.descr_insert_before.im_func, unwrap_spec=[ObjSpace, Decorators, Node, W_Root]), ) class Dict(Node): @@ -2645,6 +2733,15 @@ del self.nodes[:] for w_itm in space.unpackiterable(w_arg): self.nodes.append( space.interp_w(Node, w_itm)) + def descr_insert_after(space, self, node, w_added_nodes): + added_nodes = [space.interp_w(Node, w_node) for w_node in space.unpackiterable(w_added_nodes)] + index = self.nodes.index(node) + 1 + self.nodes[index:index] = added_nodes + + def descr_insert_before(space, self, node, w_added_nodes): + added_nodes = [space.interp_w(Node, w_node) for w_node in space.unpackiterable(w_added_nodes)] + index = self.nodes.index(node) + self.nodes[index:index] = added_nodes def descr_List_new(space, w_subtype, w_nodes, lineno=-1): self = space.allocate_instance(List, w_subtype) @@ -2662,6 +2759,8 @@ __new__ = interp2app(descr_List_new, unwrap_spec=[ObjSpace, W_Root, W_Root, int]), accept=interp2app(descr_List_accept, unwrap_spec=[ObjSpace, W_Root, W_Root] ), nodes=GetSetProperty(List.fget_nodes, List.fset_nodes ), + insert_after=interp2app(List.descr_insert_after.im_func, unwrap_spec=[ObjSpace, List, Node, W_Root]), + insert_before=interp2app(List.descr_insert_before.im_func, unwrap_spec=[ObjSpace, List, Node, W_Root]), ) class ListComp(Node): @@ -3118,6 +3217,15 @@ del self.nodes[:] for w_itm in space.unpackiterable(w_arg): self.nodes.append( space.interp_w(Node, w_itm)) + def descr_insert_after(space, self, node, w_added_nodes): + added_nodes = [space.interp_w(Node, w_node) for w_node in space.unpackiterable(w_added_nodes)] + index = self.nodes.index(node) + 1 + self.nodes[index:index] = added_nodes + + def descr_insert_before(space, self, node, w_added_nodes): + added_nodes = [space.interp_w(Node, w_node) for w_node in space.unpackiterable(w_added_nodes)] + index = self.nodes.index(node) + self.nodes[index:index] = added_nodes def descr_Or_new(space, w_subtype, w_nodes, lineno=-1): self = space.allocate_instance(Or, w_subtype) @@ -3135,6 +3243,8 @@ __new__ = interp2app(descr_Or_new, unwrap_spec=[ObjSpace, W_Root, W_Root, int]), accept=interp2app(descr_Or_accept, unwrap_spec=[ObjSpace, W_Root, W_Root] ), nodes=GetSetProperty(Or.fget_nodes, Or.fset_nodes ), + insert_after=interp2app(Or.descr_insert_after.im_func, unwrap_spec=[ObjSpace, Or, Node, W_Root]), + insert_before=interp2app(Or.descr_insert_before.im_func, unwrap_spec=[ObjSpace, Or, Node, W_Root]), ) class Pass(Node): @@ -3251,6 +3361,15 @@ del self.nodes[:] for w_itm in space.unpackiterable(w_arg): self.nodes.append( space.interp_w(Node, w_itm)) + def descr_insert_after(space, self, node, w_added_nodes): + added_nodes = [space.interp_w(Node, w_node) for w_node in space.unpackiterable(w_added_nodes)] + index = self.nodes.index(node) + 1 + self.nodes[index:index] = added_nodes + + def descr_insert_before(space, self, node, w_added_nodes): + added_nodes = [space.interp_w(Node, w_node) for w_node in space.unpackiterable(w_added_nodes)] + index = self.nodes.index(node) + self.nodes[index:index] = added_nodes def fget_dest( space, self): if self.dest is None: return space.w_None @@ -3277,6 +3396,8 @@ __new__ = interp2app(descr_Print_new, unwrap_spec=[ObjSpace, W_Root, W_Root, W_Root, int]), accept=interp2app(descr_Print_accept, unwrap_spec=[ObjSpace, W_Root, W_Root] ), nodes=GetSetProperty(Print.fget_nodes, Print.fset_nodes ), + insert_after=interp2app(Print.descr_insert_after.im_func, unwrap_spec=[ObjSpace, Print, Node, W_Root]), + insert_before=interp2app(Print.descr_insert_before.im_func, unwrap_spec=[ObjSpace, Print, Node, W_Root]), dest=GetSetProperty(Print.fget_dest, Print.fset_dest ), ) @@ -3312,6 +3433,15 @@ del self.nodes[:] for w_itm in space.unpackiterable(w_arg): self.nodes.append( space.interp_w(Node, w_itm)) + def descr_insert_after(space, self, node, w_added_nodes): + added_nodes = [space.interp_w(Node, w_node) for w_node in space.unpackiterable(w_added_nodes)] + index = self.nodes.index(node) + 1 + self.nodes[index:index] = added_nodes + + def descr_insert_before(space, self, node, w_added_nodes): + added_nodes = [space.interp_w(Node, w_node) for w_node in space.unpackiterable(w_added_nodes)] + index = self.nodes.index(node) + self.nodes[index:index] = added_nodes def fget_dest( space, self): if self.dest is None: return space.w_None @@ -3338,6 +3468,8 @@ __new__ = interp2app(descr_Printnl_new, unwrap_spec=[ObjSpace, W_Root, W_Root, W_Root, int]), accept=interp2app(descr_Printnl_accept, unwrap_spec=[ObjSpace, W_Root, W_Root] ), nodes=GetSetProperty(Printnl.fget_nodes, Printnl.fset_nodes ), + insert_after=interp2app(Printnl.descr_insert_after.im_func, unwrap_spec=[ObjSpace, Printnl, Node, W_Root]), + insert_before=interp2app(Printnl.descr_insert_before.im_func, unwrap_spec=[ObjSpace, Printnl, Node, W_Root]), dest=GetSetProperty(Printnl.fget_dest, Printnl.fset_dest ), ) @@ -3622,6 +3754,15 @@ del self.nodes[:] for w_itm in space.unpackiterable(w_arg): self.nodes.append( space.interp_w(Node, w_itm)) + def descr_insert_after(space, self, node, w_added_nodes): + added_nodes = [space.interp_w(Node, w_node) for w_node in space.unpackiterable(w_added_nodes)] + index = self.nodes.index(node) + 1 + self.nodes[index:index] = added_nodes + + def descr_insert_before(space, self, node, w_added_nodes): + added_nodes = [space.interp_w(Node, w_node) for w_node in space.unpackiterable(w_added_nodes)] + index = self.nodes.index(node) + self.nodes[index:index] = added_nodes def descr_Sliceobj_new(space, w_subtype, w_nodes, lineno=-1): self = space.allocate_instance(Sliceobj, w_subtype) @@ -3639,6 +3780,8 @@ __new__ = interp2app(descr_Sliceobj_new, unwrap_spec=[ObjSpace, W_Root, W_Root, int]), accept=interp2app(descr_Sliceobj_accept, unwrap_spec=[ObjSpace, W_Root, W_Root] ), nodes=GetSetProperty(Sliceobj.fget_nodes, Sliceobj.fset_nodes ), + insert_after=interp2app(Sliceobj.descr_insert_after.im_func, unwrap_spec=[ObjSpace, Sliceobj, Node, W_Root]), + insert_before=interp2app(Sliceobj.descr_insert_before.im_func, unwrap_spec=[ObjSpace, Sliceobj, Node, W_Root]), ) class Stmt(Node): @@ -3667,6 +3810,15 @@ del self.nodes[:] for w_itm in space.unpackiterable(w_arg): self.nodes.append( space.interp_w(Node, w_itm)) + def descr_insert_after(space, self, node, w_added_nodes): + added_nodes = [space.interp_w(Node, w_node) for w_node in space.unpackiterable(w_added_nodes)] + index = self.nodes.index(node) + 1 + self.nodes[index:index] = added_nodes + + def descr_insert_before(space, self, node, w_added_nodes): + added_nodes = [space.interp_w(Node, w_node) for w_node in space.unpackiterable(w_added_nodes)] + index = self.nodes.index(node) + self.nodes[index:index] = added_nodes def descr_Stmt_new(space, w_subtype, w_nodes, lineno=-1): self = space.allocate_instance(Stmt, w_subtype) @@ -3684,6 +3836,8 @@ __new__ = interp2app(descr_Stmt_new, unwrap_spec=[ObjSpace, W_Root, W_Root, int]), accept=interp2app(descr_Stmt_accept, unwrap_spec=[ObjSpace, W_Root, W_Root] ), nodes=GetSetProperty(Stmt.fget_nodes, Stmt.fset_nodes ), + insert_after=interp2app(Stmt.descr_insert_after.im_func, unwrap_spec=[ObjSpace, Stmt, Node, W_Root]), + insert_before=interp2app(Stmt.descr_insert_before.im_func, unwrap_spec=[ObjSpace, Stmt, Node, W_Root]), ) class Sub(BinaryOp): @@ -3976,6 +4130,15 @@ del self.nodes[:] for w_itm in space.unpackiterable(w_arg): self.nodes.append( space.interp_w(Node, w_itm)) + def descr_insert_after(space, self, node, w_added_nodes): + added_nodes = [space.interp_w(Node, w_node) for w_node in space.unpackiterable(w_added_nodes)] + index = self.nodes.index(node) + 1 + self.nodes[index:index] = added_nodes + + def descr_insert_before(space, self, node, w_added_nodes): + added_nodes = [space.interp_w(Node, w_node) for w_node in space.unpackiterable(w_added_nodes)] + index = self.nodes.index(node) + self.nodes[index:index] = added_nodes def descr_Tuple_new(space, w_subtype, w_nodes, lineno=-1): self = space.allocate_instance(Tuple, w_subtype) @@ -3993,6 +4156,8 @@ __new__ = interp2app(descr_Tuple_new, unwrap_spec=[ObjSpace, W_Root, W_Root, int]), accept=interp2app(descr_Tuple_accept, unwrap_spec=[ObjSpace, W_Root, W_Root] ), nodes=GetSetProperty(Tuple.fget_nodes, Tuple.fset_nodes ), + insert_after=interp2app(Tuple.descr_insert_after.im_func, unwrap_spec=[ObjSpace, Tuple, Node, W_Root]), + insert_before=interp2app(Tuple.descr_insert_before.im_func, unwrap_spec=[ObjSpace, Tuple, Node, W_Root]), ) class UnaryAdd(UnaryOp): Modified: pypy/branch/ast-experiments/pypy/interpreter/astcompiler/astgen.py ============================================================================== --- pypy/branch/ast-experiments/pypy/interpreter/astcompiler/astgen.py (original) +++ pypy/branch/ast-experiments/pypy/interpreter/astcompiler/astgen.py Tue Jan 24 09:52:24 2006 @@ -294,6 +294,18 @@ print >> buf, " return visitor.visit%s(self)" % self.name + def _gen_insertnodes_func(self, buf): + print >> buf, " def descr_insert_after(space, self, node, w_added_nodes):" + print >> buf, " added_nodes = [space.interp_w(Node, w_node) for w_node in space.unpackiterable(w_added_nodes)]" + print >> buf, " index = self.nodes.index(node) + 1" + print >> buf, " self.nodes[index:index] = added_nodes" + print >> buf + print >> buf, " def descr_insert_before(space, self, node, w_added_nodes):" + print >> buf, " added_nodes = [space.interp_w(Node, w_node) for w_node in space.unpackiterable(w_added_nodes)]" + print >> buf, " index = self.nodes.index(node)" + print >> buf, " self.nodes[index:index] = added_nodes" + + def _gen_fget_func(self, buf, attr, prop ): # FGET print >> buf, " def fget_%s( space, self):" % attr @@ -345,6 +357,8 @@ if "fset_%s" % attr not in self.additional_methods: self._gen_fset_func( buf, attr, prop ) + if prop[attr] == P_NESTED and attr == 'nodes': + self._gen_insertnodes_func(buf) def _gen_typedef(self, buf): initargs = [strip_default(arg.strip()) @@ -364,6 +378,9 @@ print >> buf, " accept=interp2app(descr_%s_accept, unwrap_spec=[ObjSpace, W_Root, W_Root] )," % self.name for attr in self.argnames: print >> buf, " %s=GetSetProperty(%s.fget_%s, %s.fset_%s )," % (attr,self.name,attr,self.name,attr) + if self.argprops[attr] == P_NESTED and attr == "nodes": + print >> buf, " insert_after=interp2app(%s.descr_insert_after.im_func, unwrap_spec=[ObjSpace, %s, Node, W_Root])," % (self.name, self.name) + print >> buf, " insert_before=interp2app(%s.descr_insert_before.im_func, unwrap_spec=[ObjSpace, %s, Node, W_Root])," % (self.name, self.name) print >> buf, " )" From pedronis at codespeak.net Tue Jan 24 10:11:15 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Tue, 24 Jan 2006 10:11:15 +0100 (CET) Subject: [pypy-svn] r22558 - pypy/extradoc/sprintinfo/mallorca Message-ID: <20060124091115.B749C27DB8@code1.codespeak.net> Author: pedronis Date: Tue Jan 24 10:11:14 2006 New Revision: 22558 Modified: pypy/extradoc/sprintinfo/mallorca/planning.txt Log: planning session. Modified: pypy/extradoc/sprintinfo/mallorca/planning.txt ============================================================================== --- pypy/extradoc/sprintinfo/mallorca/planning.txt (original) +++ pypy/extradoc/sprintinfo/mallorca/planning.txt Tue Jan 24 10:11:14 2006 @@ -28,24 +28,22 @@ Tasks ====== - - JIT: Armin + - JIT: Armin, Carl. Graph generating ops - stackless application level interface: Christian, Eric + Design thinking. Coding could start today. - - l3interp/offsetof/sizeof work: Michael, Arre, (Samuele) + - l3interp/offsetof/sizeof work: Michael, Arre + Progress. Next sizeof... - - integration of the garbage collection toolkit with the + (- integration of the garbage collection toolkit with the offsetof work done: Carl Friedrich, Richard + Offsetof refactoring finished. Continuing with sizeof ...) - improving the way we interface with external functions. (also think about how exceptions fit in there): add static ctypes functionality into RPython itself - Gerald, Stephan (Samuele) - - - - tutorial about the translator: Armin, Gerald, Stephan, - Eric, Christian - + Gerald, Stephan, Samuele later: ====== From auc at codespeak.net Tue Jan 24 10:25:34 2006 From: auc at codespeak.net (auc at codespeak.net) Date: Tue, 24 Jan 2006 10:25:34 +0100 (CET) Subject: [pypy-svn] r22560 - pypy/dist/pypy/lib/logic Message-ID: <20060124092534.6559D27DB8@code1.codespeak.net> Author: auc Date: Tue Jan 24 10:25:32 2006 New Revision: 22560 Modified: pypy/dist/pypy/lib/logic/test_unification.py pypy/dist/pypy/lib/logic/test_variable.py pypy/dist/pypy/lib/logic/unification.py Log: * remove bogus test * more comments in code * bugs fixed (double successful unification must be ok) Modified: pypy/dist/pypy/lib/logic/test_unification.py ============================================================================== --- pypy/dist/pypy/lib/logic/test_unification.py (original) +++ pypy/dist/pypy/lib/logic/test_unification.py Tue Jan 24 10:25:32 2006 @@ -63,7 +63,9 @@ u.bind(y, z) u.unify(x, y) assert z.val == 42 - #raises(u.UnificationFailure, u.unify, x, y) + u.unify(x, y) + assert (z.val == x.val) and (x.val == y.val) + def test_unify_values(self): x, y = u.var('x'), u.var('y') @@ -187,37 +189,6 @@ (t1.raised and not t2.raised) - def test_threads_unifying_vars(self): - x, y, z = u.var('x'), u.var('y'), u.var('z') - l1 = range(999) - l2 = range(999) - l1[-1] = z - l2[0] = z - l2[-1] = 0 - u.bind(x, l1) - u.bind(y, l2) - - def do_unify(thread, v1, v2): - thread.raised = False - print thread - try: - u.unify(v1, v2) - except u.UnificationFailure: - thread.raised = True - print thread - - t1, t2 = (FunThread(do_unify, x, y), - FunThread(do_unify, x, y)) - t1.start() - t2.start() - t1.join() - t2.join() - print "Z", z - assert (t2.raised and not t1.raised) or \ - (t1.raised and not t2.raised) - assert z.val == 0 - - def test_set_var_domain(self): x = u.var('x') u.set_domain(x, [1, 3, 5]) Modified: pypy/dist/pypy/lib/logic/test_variable.py ============================================================================== --- pypy/dist/pypy/lib/logic/test_variable.py (original) +++ pypy/dist/pypy/lib/logic/test_variable.py Tue Jan 24 10:25:32 2006 @@ -25,9 +25,9 @@ u._store = u.Store() -## def test_no_same_name(self): -## x = u.var('x') -## raises(v.AlreadyExists, u.var, 'x') + def test_no_same_name(self): + x = u.var('x') + raises(v.AlreadyExists, u.var, 'x') def test_one_thread_reading_one_var(self): cons = Consumer() Modified: pypy/dist/pypy/lib/logic/unification.py ============================================================================== --- pypy/dist/pypy/lib/logic/unification.py (original) +++ pypy/dist/pypy/lib/logic/unification.py Tue Jan 24 10:25:32 2006 @@ -178,7 +178,7 @@ self.contraints = {} # consistency-preserving stuff self.in_transaction = False - self.lock = threading.Lock() + self.lock = threading.RLock() def add_unbound(self, var): # register globally @@ -221,6 +221,7 @@ else: # 1. both are unbound self._merge(var, val) else: # 3. val is really a value + print "%s, is that you ?" % var if var._is_bound(): raise AlreadyBound(var.name) self._bind(var.val, val) @@ -278,13 +279,15 @@ def _really_unify(self, x, y): # print "unify %s with %s" % (x,y) if not _unifiable(x, y): raise UnificationFailure(x, y) - # dispatch to the apropriate unifier if not x in self.vars: if not y in self.vars: + # duh ! x & y not vars if x != y: raise UnificationFailure(x, y) else: return + # same call, reverse args. order self._unify_var_val(y, x) elif not y in self.vars: + # x is Var, y a value self._unify_var_val(x, y) elif _both_are_bound(x, y): self._unify_bound(x,y) @@ -294,7 +297,6 @@ self.bind(y,x) def _unify_var_val(self, x, y): - #if x._is_bound(): raise UnificationFailure(x, y) if x.val != y: try: self.bind(x, y) @@ -309,10 +311,11 @@ elif type(vx) is dict and isinstance(vy, type(vx)): self._unify_mapping(x, y) else: - raise UnificationFailure(x, y) + if vx != vy: + raise UnificationFailure(x, y) def _unify_iterable(self, x, y): - # print "unify sequences %s %s" % (x, y) + print "unify sequences %s %s" % (x, y) vx, vy = (x.val, y.val) idx, top = (0, len(vx)) while (idx < top): From bea at codespeak.net Tue Jan 24 10:34:02 2006 From: bea at codespeak.net (bea at codespeak.net) Date: Tue, 24 Jan 2006 10:34:02 +0100 (CET) Subject: [pypy-svn] r22562 - pypy/extradoc/sprintinfo/tokyo Message-ID: <20060124093402.CC49527DB9@code1.codespeak.net> Author: bea Date: Tue Jan 24 10:34:01 2006 New Revision: 22562 Added: pypy/extradoc/sprintinfo/tokyo/ pypy/extradoc/sprintinfo/tokyo/people.txt Log: start people file for tokyo - please fill out names for now Added: pypy/extradoc/sprintinfo/tokyo/people.txt ============================================================================== --- (empty file) +++ pypy/extradoc/sprintinfo/tokyo/people.txt Tue Jan 24 10:34:01 2006 @@ -0,0 +1,32 @@ +People coming to the Tokyo sprint 2006 +================================================== + +People who have a ``?`` in their arrive/depart or accomodation +column are known to be coming but there are no details +available yet from them. + +==================== ============== ===================== + Name Arrive/Depart Accomodation +==================== ============== ===================== +Beatrice D?ring + + +==================== ============== ===================== + +People on the following list were present at previous sprints: + +==================== ============== ===================== + Name Arrive/Depart Accomodation +==================== ============== ===================== +Ludovic Aubry +Adrien Di Mascio +Laura Creighton +Lene Wagner +Anders Lehmann +Niklaus Haldimann +Johan Hahn +Amaury Forgeot d'Arc +Valentino Volonghi +Boris Feigin +Bert Freudenberg +==================== ============== ===================== From ericvrp at codespeak.net Tue Jan 24 10:47:47 2006 From: ericvrp at codespeak.net (ericvrp at codespeak.net) Date: Tue, 24 Jan 2006 10:47:47 +0100 (CET) Subject: [pypy-svn] r22565 - in pypy/dist/pypy/translator/llvm: . test Message-ID: <20060124094747.1B1B127DB6@code1.codespeak.net> Author: ericvrp Date: Tue Jan 24 10:47:46 2006 New Revision: 22565 Modified: pypy/dist/pypy/translator/llvm/database.py pypy/dist/pypy/translator/llvm/test/test_lladdresses.py Log: Fixes to genllvm for WIP of lladdresses Modified: pypy/dist/pypy/translator/llvm/database.py ============================================================================== --- pypy/dist/pypy/translator/llvm/database.py (original) +++ pypy/dist/pypy/translator/llvm/database.py Tue Jan 24 10:47:46 2006 @@ -12,8 +12,8 @@ OpaqueTypeNode, ExtOpaqueTypeNode from pypy.rpython.lltypesystem import lltype from pypy.objspace.flow.model import Constant, Variable -from pypy.rpython.memory.lladdress import Address, NULL - +from pypy.rpython.memory.lladdress import address, NULL + log = log.database class Database(object): @@ -38,7 +38,7 @@ lltype.Void: "void", lltype.UnsignedLongLong: "ulong", lltype.SignedLongLong: "long", - Address: "sbyte*"} + address: "sbyte*"} # 32 bit platform if sys.maxint == 2**31-1: @@ -362,7 +362,7 @@ repr = str(ord(value)) elif type_ is lltype.Float: repr = self.float_to_str(value) - elif type_ is Address: + elif type_ is address: assert value == NULL repr = 'null' else: Modified: pypy/dist/pypy/translator/llvm/test/test_lladdresses.py ============================================================================== --- pypy/dist/pypy/translator/llvm/test/test_lladdresses.py (original) +++ pypy/dist/pypy/translator/llvm/test/test_lladdresses.py Tue Jan 24 10:47:46 2006 @@ -4,6 +4,8 @@ from pypy.rpython.objectmodel import free_non_gc_object import py +py.test.skip("WIP") + def test_null(): def f(): return NULL - NULL From cfbolz at codespeak.net Tue Jan 24 10:51:25 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Tue, 24 Jan 2006 10:51:25 +0100 (CET) Subject: [pypy-svn] r22566 - pypy/dist/pypy/doc/discussion Message-ID: <20060124095125.AA11D27DB8@code1.codespeak.net> Author: cfbolz Date: Tue Jan 24 10:51:24 2006 New Revision: 22566 Modified: pypy/dist/pypy/doc/discussion/draft-jit-ideas.txt Log: (cfbolz, arigo) Designed the functions that generate operations. Modified: pypy/dist/pypy/doc/discussion/draft-jit-ideas.txt ============================================================================== --- pypy/dist/pypy/doc/discussion/draft-jit-ideas.txt (original) +++ pypy/dist/pypy/doc/discussion/draft-jit-ideas.txt Tue Jan 24 10:51:24 2006 @@ -90,6 +90,24 @@ replace all the explicit building of Graph, Block, Link, Operation and Variable with calls to such operations. +Functions: + +* gengraph() -> (opaque start link object) + +* genblock([list-of-CONCRETE_TYPES-of-inputargs]) -> (opaque block object) + +* genop(block, opname, [list-of-vars], RESULT_CONCRETE_TYPE) -> (result var) + +* genconst(block, llvalue) -> (result var) + +* genlink(block, exitcase-or-None, [list-of-vars]) -> (opaque link object) + +* genreturnlink(block, return-var) -> (opaque link object) + +* closeblock(block, exitswitch-var-or-None) + +* closelink(link, block) + L3 interpreter ~~~~~~~~~~~~~~~~~~~ From adim at codespeak.net Tue Jan 24 10:55:57 2006 From: adim at codespeak.net (adim at codespeak.net) Date: Tue, 24 Jan 2006 10:55:57 +0100 (CET) Subject: [pypy-svn] r22570 - pypy/branch/ast-experiments/pypy/module/recparser/hooksamples Message-ID: <20060124095557.E004A27DB8@code1.codespeak.net> Author: adim Date: Tue Jan 24 10:55:56 2006 New Revision: 22570 Modified: pypy/branch/ast-experiments/pypy/module/recparser/hooksamples/tracer.py Log: updated assign tracer example with insert_before / insert_after Modified: pypy/branch/ast-experiments/pypy/module/recparser/hooksamples/tracer.py ============================================================================== --- pypy/branch/ast-experiments/pypy/module/recparser/hooksamples/tracer.py (original) +++ pypy/branch/ast-experiments/pypy/module/recparser/hooksamples/tracer.py Tue Jan 24 10:55:56 2006 @@ -6,7 +6,16 @@ """ from parser import ASTPrintnl, ASTConst, ASTName -from parser import install_compiler_hook +from parser import install_compiler_hook, source2ast + +BEFORE_LOG_SOURCE = """if '%s' in locals() or '%s' in globals(): + print '(before) %s <--', locals().get('%s', globals().get('%s', '')) +""" +AFTER_LOG_SOURCE = "print '(after) %s <--', %s" + +def get_statements(source): + module = source2ast(source) + return module.node.nodes class Tracer: def visitModule(self, module): @@ -23,11 +32,10 @@ def visitAssign(self, assign): stmt = assign.parent varname = assign.nodes[0].name - lognode = ASTPrintnl([ASTConst('%s <--' % varname), ASTName(varname)], None) - index = stmt.nodes.index(assign) - newstmts = stmt.nodes - newstmts.insert(index + 1, lognode) - stmt.nodes = newstmts + before_stmts = get_statements(BEFORE_LOG_SOURCE % ((varname,) * 5)) + after_stmts = get_statements(AFTER_LOG_SOURCE % (varname, varname)) + stmt.insert_before(assign, before_stmts) + stmt.insert_after(assign, after_stmts) return assign def __getattr__(self, attrname): From gromit at codespeak.net Tue Jan 24 11:02:27 2006 From: gromit at codespeak.net (gromit at codespeak.net) Date: Tue, 24 Jan 2006 11:02:27 +0100 (CET) Subject: [pypy-svn] r22571 - pypy/dist/pypy/annotation Message-ID: <20060124100227.044DF27DBA@code1.codespeak.net> Author: gromit Date: Tue Jan 24 11:02:26 2006 New Revision: 22571 Modified: pypy/dist/pypy/annotation/bookkeeper.py Log: ADD: (gromit,stephan) Added a case to Bookkeeper.immutablevalue that annotates rcypte's type descriptor classes. Modified: pypy/dist/pypy/annotation/bookkeeper.py ============================================================================== --- pypy/dist/pypy/annotation/bookkeeper.py (original) +++ pypy/dist/pypy/annotation/bookkeeper.py Tue Jan 24 11:02:26 2006 @@ -354,6 +354,8 @@ elif ishashable(x) and x in BUILTIN_ANALYZERS: _module = getattr(x,"__module__","unknown") result = SomeBuiltin(BUILTIN_ANALYZERS[x], methodname="%s.%s" % (_module, x.__name__)) + elif hasattr(x, "compute_result_annotation"): + result = SomeBuiltin(x.compute_result_annotation, "?" ) elif tp in EXTERNAL_TYPE_ANALYZERS: result = SomeExternalObject(tp) elif isinstance(x, lltype._ptr): From gromit at codespeak.net Tue Jan 24 11:04:06 2006 From: gromit at codespeak.net (gromit at codespeak.net) Date: Tue, 24 Jan 2006 11:04:06 +0100 (CET) Subject: [pypy-svn] r22572 - pypy/dist/pypy/rpython/rctypes Message-ID: <20060124100406.67A6927B4C@code1.codespeak.net> Author: gromit Date: Tue Jan 24 11:04:04 2006 New Revision: 22572 Modified: pypy/dist/pypy/rpython/rctypes/interface.py Log: ADD: (gromit,stephan) Use teh rctypes objects from our implementation module. Modified: pypy/dist/pypy/rpython/rctypes/interface.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/interface.py (original) +++ pypy/dist/pypy/rpython/rctypes/interface.py Tue Jan 24 11:04:04 2006 @@ -1,4 +1,9 @@ -from ctypes import * +from ctypes import _DLLS +from implementation import RCDLL as CDLL, RWinDLL as WinDLL, c_int, c_char_p + +cdll = _DLLS( CDLL ) +windll = _DLLS( WinDLL ) + """ Loading From gromit at codespeak.net Tue Jan 24 11:06:21 2006 From: gromit at codespeak.net (gromit at codespeak.net) Date: Tue, 24 Jan 2006 11:06:21 +0100 (CET) Subject: [pypy-svn] r22573 - pypy/dist/pypy/rpython/rctypes Message-ID: <20060124100621.2F12027DB5@code1.codespeak.net> Author: gromit Date: Tue Jan 24 11:06:19 2006 New Revision: 22573 Added: pypy/dist/pypy/rpython/rctypes/implementation.py Log: ADD: (gromit,stephan) Added an implementation module. Added: pypy/dist/pypy/rpython/rctypes/implementation.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/rpython/rctypes/implementation.py Tue Jan 24 11:06:19 2006 @@ -0,0 +1,46 @@ +""" +The rctypes implementaion is contained here. +""" + +from ctypes import * +from ctypes import _FUNCFLAG_CDECL, _FUNCFLAG_STDCALL +from pypy.annotation.model import SomeInteger + +c_int.annotator_type = SomeInteger() + + +class FunctionPointerAnnotation(object): + + def compute_result_annotation(self, *args_s): + """ + Answer the annotation of the external function's result + """ + return self.restype.annotator_type + + +class RCDLL(CDLL): + """ + This is the restricted version of ctypes' CDLL class. + """ + + class _CdeclFuncPtr(FunctionPointerAnnotation, CDLL._CdeclFuncPtr): + """ + A simple extension of ctypes function pointers that + implements a simple interface to the anotator. + """ + _flags_ = _FUNCFLAG_CDECL + + + +class RWinDLL(WinDLL): + """ + This is the restricted version of ctypes' WINDLL class + """ + + class _StdcallFuncPtr(FunctionPointerAnnotation, WinDLL._StdcallFuncPtr): + """ + A simple extension of ctypes function pointers that + implements a simple interface to the anotator. + """ + _flags_ = _FUNCFLAG_STDCALL + From ericvrp at codespeak.net Tue Jan 24 11:07:31 2006 From: ericvrp at codespeak.net (ericvrp at codespeak.net) Date: Tue, 24 Jan 2006 11:07:31 +0100 (CET) Subject: [pypy-svn] r22574 - pypy/dist/pypy/translator/js/test Message-ID: <20060124100731.780A127DB5@code1.codespeak.net> Author: ericvrp Date: Tue Jan 24 11:07:29 2006 New Revision: 22574 Modified: pypy/dist/pypy/translator/js/test/test_genllvm.py Log: genjs fix for pow inaccuracy on Internet Explorer Modified: pypy/dist/pypy/translator/js/test/test_genllvm.py ============================================================================== --- pypy/dist/pypy/translator/js/test/test_genllvm.py (original) +++ pypy/dist/pypy/translator/js/test/test_genllvm.py Tue Jan 24 11:07:29 2006 @@ -70,7 +70,7 @@ x += x ** 1 x += x ** 2 x += i + 1 * i // i - 1 - return x + return int(x+0.0001) f = compile_function(ops, [int]) assert f(1) == ops(1) assert f(2) == ops(2) @@ -89,7 +89,7 @@ x += x ** 1 x += x ** 2 x += i + 1 * i // i - 1 - return x + return int(x+0.0001) f = compile_function(ops, [r_uint]) assert f(1) == ops(1) assert f(2) == ops(2) From ericvrp at codespeak.net Tue Jan 24 11:15:01 2006 From: ericvrp at codespeak.net (ericvrp at codespeak.net) Date: Tue, 24 Jan 2006 11:15:01 +0100 (CET) Subject: [pypy-svn] r22575 - pypy/dist/pypy/translator/js/test Message-ID: <20060124101501.EB5D627DB5@code1.codespeak.net> Author: ericvrp Date: Tue Jan 24 11:15:01 2006 New Revision: 22575 Modified: pypy/dist/pypy/translator/js/test/browsertest.py pypy/dist/pypy/translator/js/test/test_stackless.py Log: genjs used Python 'raise' statement by accident. InternetExplorer was not too happy with this. Now using 'throw' consistantly. Modified: pypy/dist/pypy/translator/js/test/browsertest.py ============================================================================== --- pypy/dist/pypy/translator/js/test/browsertest.py (original) +++ pypy/dist/pypy/translator/js/test/browsertest.py Tue Jan 24 11:15:01 2006 @@ -24,9 +24,9 @@ result = %(jstestcase)s; } catch (e) { try { - result = "raise Exception('" + e.toSource() + "')"; + result = "throw Exception('" + e.toSource() + "')"; } catch (dummy) { - result = "raise Exception('unknown')"; + result = "throw Exception('unknown')"; } } Modified: pypy/dist/pypy/translator/js/test/test_stackless.py ============================================================================== --- pypy/dist/pypy/translator/js/test/test_stackless.py (original) +++ pypy/dist/pypy/translator/js/test/test_stackless.py Tue Jan 24 11:15:01 2006 @@ -68,6 +68,7 @@ assert data.strip() == '100' def test_stackless_arguments(): + py.test.skip("[Object object] unknown failure") def f(n, d, t): if n > 0: res = f(n-1, d, t) From auc at codespeak.net Tue Jan 24 11:24:02 2006 From: auc at codespeak.net (auc at codespeak.net) Date: Tue, 24 Jan 2006 11:24:02 +0100 (CET) Subject: [pypy-svn] r22577 - pypy/dist/pypy/lib/logic Message-ID: <20060124102402.DF5A127DB6@code1.codespeak.net> Author: auc Date: Tue Jan 24 11:23:58 2006 New Revision: 22577 Modified: pypy/dist/pypy/lib/logic/constraint.py pypy/dist/pypy/lib/logic/test_unification.py pypy/dist/pypy/lib/logic/unification.py Log: (auc, ale) * add constraints to the store * twists to constraints to make them accept our variables Modified: pypy/dist/pypy/lib/logic/constraint.py ============================================================================== --- pypy/dist/pypy/lib/logic/constraint.py (original) +++ pypy/dist/pypy/lib/logic/constraint.py Tue Jan 24 11:23:58 2006 @@ -1,11 +1,16 @@ # a) new requirement : be able to postpone asking fo the # values of the domain +#-- Exceptions --------------------------------------- + class ConsistencyFailure(Exception): """The repository is not in a consistent state""" pass - +class DomainlessVariables(Exception): + """A constraint can't be defined on variables + without a domain""" + pass #-- Domains -------------------------------------------- @@ -110,15 +115,17 @@ #-- Constraints ------------------------------------------ class AbstractConstraint(object): - #__implements__ = ConstraintInterface def __init__(self, variables): """variables is a list of variables which appear in the formula""" + for var in variables: + if var.dom is None: + raise DomainlessVariables self._variables = variables def affectedVariables(self): """ Return a list of all variables affected by this constraint """ - return self._variabl + return self._variables def isVariableRelevant(self, variable): return variable in self._variables @@ -170,6 +177,14 @@ return 1 +def make_lambda_head(vars): + var_ids = ','.join([var.name for var in vars]) + return 'lambda ' + var_ids + ':' + +def expand_expr_template(expr, vars): + for var in vars: + expr.replace(var.name, var.name + '.val') + return expr class Expression(AbstractConstraint): """A constraint represented as a python expression.""" @@ -184,8 +199,8 @@ try: self.filterFunc = Expression._FILTER_CACHE[formula] except KeyError: - self.filterFunc = eval('lambda %s: %s' % \ - (','.join(variables), formula), {}, {}) + self.filterFunc = eval(make_lambda_head(variables) \ + + expand_expr_template(formula, variables), {}, {}) Expression._FILTER_CACHE[formula] = self.filterFunc def _init_result_cache(self): Modified: pypy/dist/pypy/lib/logic/test_unification.py ============================================================================== --- pypy/dist/pypy/lib/logic/test_unification.py (original) +++ pypy/dist/pypy/lib/logic/test_unification.py Tue Jan 24 11:23:58 2006 @@ -1,6 +1,6 @@ import unification as u import variable as v -from constraint import FiniteDomain +import constraint as c from py.test import raises, skip from threading import Thread @@ -192,8 +192,7 @@ def test_set_var_domain(self): x = u.var('x') u.set_domain(x, [1, 3, 5]) - assert x.dom == FiniteDomain([1, 3, 5]) - assert u._store.domains[x] == FiniteDomain([1, 3, 5]) + assert x.dom == c.FiniteDomain([1, 3, 5]) def test_bind_with_domain(self): x = u.var('x') @@ -222,4 +221,15 @@ u.set_domain(z, [41, 42, 43]) u.unify(x, y) assert z.val == 42 - assert z.dom == FiniteDomain([41, 42, 43]) + assert z.dom == c.FiniteDomain([41, 42, 43]) + + def test_add_constraint(self): + x,y,z = u.var('x'), u.var('y'), u.var('z') + raises(c.DomainlessVariables, + c.Expression, [x, y, z], 'x == y + z') + x.dom = c.FiniteDomain([1, 2]) + y.dom = c.FiniteDomain([2, 3]) + z.dom = c.FiniteDomain([3, 4]) + k = c.Expression([x, y, z], 'x == y + z') + u.add_constraint(k) + assert k in u._store.constraints Modified: pypy/dist/pypy/lib/logic/unification.py ============================================================================== --- pypy/dist/pypy/lib/logic/unification.py (original) +++ pypy/dist/pypy/lib/logic/unification.py Tue Jan 24 11:23:58 2006 @@ -172,10 +172,8 @@ # mapping of names to vars (all of them) self.vars = set() self.names = set() - # mapping of vars to domains - self.domains = {} - # mapping of names to constraints (all...) - self.contraints = {} + # set of all constraints + self.constraints = set() # consistency-preserving stuff self.in_transaction = False self.lock = threading.RLock() @@ -197,7 +195,11 @@ if var.is_bound(): raise AlreadyBound var.dom = FiniteDomain(dom) - self.domains[var] = var.dom + + #-- Add constraints ------------------------- + + def add_constraint(self, constraint): + self.constraints.add(constraint) #-- BIND ------------------------------------------- @@ -418,6 +420,9 @@ def set_domain(var, dom): return _store.set_domain(var, dom) +def add_constraint(constraint): + return _store.add_constraint(constraint) + def bind(var, val): return _store.bind(var, val) From cfbolz at codespeak.net Tue Jan 24 12:01:28 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Tue, 24 Jan 2006 12:01:28 +0100 (CET) Subject: [pypy-svn] r22579 - in pypy/dist/pypy: doc/discussion rpython rpython/test Message-ID: <20060124110128.22BD127DB8@code1.codespeak.net> Author: cfbolz Date: Tue Jan 24 12:01:24 2006 New Revision: 22579 Added: pypy/dist/pypy/rpython/rgenop.py pypy/dist/pypy/rpython/test/test_rgenop.py Modified: pypy/dist/pypy/doc/discussion/draft-jit-ideas.txt Log: (arigo, cfbolz): updated discussion, implemented the operation-generating functions with low level graphs + test Modified: pypy/dist/pypy/doc/discussion/draft-jit-ideas.txt ============================================================================== --- pypy/dist/pypy/doc/discussion/draft-jit-ideas.txt (original) +++ pypy/dist/pypy/doc/discussion/draft-jit-ideas.txt Tue Jan 24 12:01:24 2006 @@ -92,21 +92,27 @@ Functions: -* gengraph() -> (opaque start link object) +* newgraph(name) -> (opaque start link object) -* genblock([list-of-CONCRETE_TYPES-of-inputargs]) -> (opaque block object) +* newblock() -> (opaque block object) + +* geninputarg(block, CONCRETE_TYPE) -> (input var) * genop(block, opname, [list-of-vars], RESULT_CONCRETE_TYPE) -> (result var) * genconst(block, llvalue) -> (result var) -* genlink(block, exitcase-or-None, [list-of-vars]) -> (opaque link object) +* newlink(block, [list-of-vars]) -> (opaque link object) + +* newreturnlink(block, return-var) -> (opaque link object) + +* closeblock1(block, link) -* genreturnlink(block, return-var) -> (opaque link object) +* closeblock2(block, exitswitch-var, false_link, true_link) -* closeblock(block, exitswitch-var-or-None) +* closelink(link, targetblock) -* closelink(link, block) +* runlink(link, [list-of-llvalues]) L3 interpreter Added: pypy/dist/pypy/rpython/rgenop.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/rpython/rgenop.py Tue Jan 24 12:01:24 2006 @@ -0,0 +1,96 @@ +""" +Functions that generate flow graphs and operations. +The functions below produce L2 graphs, but they define an interface +that can be used to produce any other kind of graph. +""" + +from pypy.rpython.lltypesystem import lltype +from pypy.objspace.flow import model as flowmodel + + +def newgraph(name): + startblock_to_throw_away = flowmodel.Block([]) # grumble + return flowmodel.FunctionGraph(name, startblock_to_throw_away) + +def newblock(): + return flowmodel.Block([]) + +def geninputarg(block, CONCRETE_TYPE): + v = flowmodel.Variable() + v.concretetype = CONCRETE_TYPE + block.inputargs.append(v) + return v + +def genop(block, opname, vars, RESULT_TYPE): + v = flowmodel.Variable() + v.concretetype = RESULT_TYPE + op = flowmodel.SpaceOperation(opname, vars, v) + block.operations.append(op) + return v + +def genconst(block, llvalue): + v = flowmodel.Constant(llvalue) + v.concretetype = lltype.typeOf(llvalue) + return v + +def newlink(block, vars): + return flowmodel.Link(vars, None) + +def newreturnlink(block, var): + v = flowmodel.Variable() + v.concretetype = var.concretetype + pseudoreturnblock = flowmodel.Block([v]) + pseudoreturnblock.operations = () + return flowmodel.Link([var], pseudoreturnblock) + +def closeblock1(block, link): + block.closeblock(link) + +def closeblock2(block, exitswitch, false_link, true_link): + block.exitswitch = exitswitch + false_link.exitcase = False + false_link.llexitcase = False + true_link.exitcase = True + true_link.llexitcase = True + block.closeblock(false_link, true_link) + +def closelink(link, targetblock): + if isinstance(link, flowmodel.Link): + assert link.target is None, "%r already closed" % (link,) + assert ([v.concretetype for v in link.args] == + [v.concretetype for v in targetblock.inputargs]) + link.target = targetblock + elif isinstance(link, flowmodel.FunctionGraph): + graph = link + graph.startblock = targetblock + targetblock.isstartblock = True + else: + raise TypeError + +def _patchgraph(graph): + returntype = None + for link in graph.iterlinks(): + if link.target.operations == (): + assert len(link.args) == 1 # for now + if returntype is None: + returntype = link.target.inputargs[0].concretetype + else: + assert returntype == link.target.inputargs[0].concretetype + link.target = graph.returnblock + if returntype is None: + returntype = lltype.Void + graph.returnblock.inputargs[0].concretetype = returntype + +class PseudoRTyper(object): + def __init__(self): + from pypy.rpython.typesystem import LowLevelTypeSystem + self.type_system = LowLevelTypeSystem.instance + +def runlink(startlink, args): + from pypy.rpython.llinterp import LLInterpreter + assert isinstance(startlink, flowmodel.FunctionGraph), "XXX" + graph = startlink # for now + _patchgraph(graph) + flowmodel.checkgraph(graph) + llinterp = LLInterpreter(PseudoRTyper()) + return llinterp.eval_graph(graph, args) Added: pypy/dist/pypy/rpython/test/test_rgenop.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/rpython/test/test_rgenop.py Tue Jan 24 12:01:24 2006 @@ -0,0 +1,15 @@ +from pypy.rpython.rgenop import * +from pypy.rpython.lltypesystem.lltype import * + + +def test_square(): + startlink = newgraph("square") + block = newblock() + v0 = geninputarg(block, Signed) + v1 = genop(block, 'int_mul', [v0, v0], Signed) + link = newreturnlink(block, v1) + closeblock1(block, link) + closelink(startlink, block) + + res = runlink(startlink, [17]) + assert res == 289 From auc at codespeak.net Tue Jan 24 12:04:46 2006 From: auc at codespeak.net (auc at codespeak.net) Date: Tue, 24 Jan 2006 12:04:46 +0100 (CET) Subject: [pypy-svn] r22580 - pypy/dist/pypy/lib/logic Message-ID: <20060124110446.5987627DB8@code1.codespeak.net> Author: auc Date: Tue Jan 24 12:04:44 2006 New Revision: 22580 Modified: pypy/dist/pypy/lib/logic/constraint.py pypy/dist/pypy/lib/logic/test_unification.py Log: (auc, ale) * update constraints to cope with our vars * check that narrow work on n-ary expressions Modified: pypy/dist/pypy/lib/logic/constraint.py ============================================================================== --- pypy/dist/pypy/lib/logic/constraint.py (original) +++ pypy/dist/pypy/lib/logic/constraint.py Tue Jan 24 12:04:44 2006 @@ -91,8 +91,9 @@ __len__ = size def get_values(self): - """return all the values in the domain""" - return self._values + """return all the values in the domain + in an indexable sequence""" + return list(self._values) def __iter__(self): return iter(self._values) @@ -118,9 +119,11 @@ def __init__(self, variables): """variables is a list of variables which appear in the formula""" + self._names_to_vars = {} for var in variables: if var.dom is None: raise DomainlessVariables + self._names_to_vars[var.name] = var self._variables = variables def affectedVariables(self): @@ -207,18 +210,18 @@ """key = (variable,value), value = [has_success,has_failure]""" result_cache = {} for var_name in self._variables: - result_cache[var_name] = {} + result_cache[var_name.name] = {} return result_cache - def _assign_values(self, domains): + def _assign_values(self): variables = [] kwargs = {} for variable in self._variables: - domain = domains[variable] + domain = variable.dom values = domain.get_values() variables.append((domain.size(), [variable, values, 0, len(values)])) - kwargs[variable] = values[0] + kwargs[variable.name] = values[0] # sort variables to instanciate those with fewer possible values first variables.sort() @@ -229,22 +232,23 @@ for size, curr in variables: if (curr[2] + 1) < curr[-1]: curr[2] += 1 - kwargs[curr[0]] = curr[1][curr[2]] + kwargs[curr[0].name] = curr[1][curr[2]] break else: curr[2] = 0 - kwargs[curr[0]] = curr[1][0] + kwargs[curr[0].name] = curr[1][0] else: # it's over go_on = 0 - def narrow(self, domains): + def narrow(self): + # removed domain arg. (auc, ale) """generic narrowing algorithm for n-ary expressions""" maybe_entailed = 1 ffunc = self.filterFunc result_cache = self._init_result_cache() - for kwargs in self._assign_values(domains): + for kwargs in self._assign_values(): if maybe_entailed: for var, val in kwargs.iteritems(): if val not in result_cache[var]: @@ -259,7 +263,7 @@ try: for var, keep in result_cache.iteritems(): - domain = domains[var] + domain = self._names_to_vars[var].dom domain.remove_values([val for val in domain if val not in keep]) except ConsistencyFailure: Modified: pypy/dist/pypy/lib/logic/test_unification.py ============================================================================== --- pypy/dist/pypy/lib/logic/test_unification.py (original) +++ pypy/dist/pypy/lib/logic/test_unification.py Tue Jan 24 12:04:44 2006 @@ -233,3 +233,12 @@ k = c.Expression([x, y, z], 'x == y + z') u.add_constraint(k) assert k in u._store.constraints + + def test_narrowing_domains(self): + x,y,z = u.var('x'), u.var('y'), u.var('z') + x.dom = c.FiniteDomain([1, 2]) + y.dom = c.FiniteDomain([2, 3]) + z.dom = c.FiniteDomain([3, 4]) + k = c.Expression([x, y, z], 'x == y + z') + raises(c.ConsistencyFailure, k.narrow) + From cfbolz at codespeak.net Tue Jan 24 12:08:52 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Tue, 24 Jan 2006 12:08:52 +0100 (CET) Subject: [pypy-svn] r22581 - pypy/dist/pypy/rpython/test Message-ID: <20060124110852.4B97E27DB9@code1.codespeak.net> Author: cfbolz Date: Tue Jan 24 12:08:50 2006 New Revision: 22581 Modified: pypy/dist/pypy/rpython/test/test_rgenop.py Log: (arigo, cfbolz): another test for the new graph-generating functions that unfortunately didn't catch any bugs. Modified: pypy/dist/pypy/rpython/test/test_rgenop.py ============================================================================== --- pypy/dist/pypy/rpython/test/test_rgenop.py (original) +++ pypy/dist/pypy/rpython/test/test_rgenop.py Tue Jan 24 12:08:50 2006 @@ -3,6 +3,7 @@ def test_square(): + """def square(v0): return v0*v0""" startlink = newgraph("square") block = newblock() v0 = geninputarg(block, Signed) @@ -13,3 +14,26 @@ res = runlink(startlink, [17]) assert res == 289 + +def test_if(): + """ + def f(v0): + if v0 < 0: + return 0 + else: + return v0 + """ + startlink = newgraph("if") + block = newblock() + v0 = geninputarg(block, Signed) + const0 = genconst(block, 0) + v1 = genop(block, 'int_lt', [v0, const0], Bool) + true_link = newreturnlink(block, const0) + false_link = newreturnlink(block, v0) + closeblock2(block, v1, false_link, true_link) + closelink(startlink, block) + + res = runlink(startlink, [-1]) + assert res == 0 + res = runlink(startlink, [42]) + assert res == 42 From ericvrp at codespeak.net Tue Jan 24 13:13:14 2006 From: ericvrp at codespeak.net (ericvrp at codespeak.net) Date: Tue, 24 Jan 2006 13:13:14 +0100 (CET) Subject: [pypy-svn] r22582 - in pypy/dist/pypy/translator/llvm: . test Message-ID: <20060124121314.785D527DB6@code1.codespeak.net> Author: ericvrp Date: Tue Jan 24 13:13:13 2006 New Revision: 22582 Modified: pypy/dist/pypy/translator/llvm/database.py pypy/dist/pypy/translator/llvm/test/test_lladdresses.py Log: Fix for fix Modified: pypy/dist/pypy/translator/llvm/database.py ============================================================================== --- pypy/dist/pypy/translator/llvm/database.py (original) +++ pypy/dist/pypy/translator/llvm/database.py Tue Jan 24 13:13:13 2006 @@ -10,9 +10,9 @@ VoidArrayNode, ArrayTypeNode, VoidArrayTypeNode from pypy.translator.llvm.opaquenode import OpaqueNode, ExtOpaqueNode, \ OpaqueTypeNode, ExtOpaqueTypeNode -from pypy.rpython.lltypesystem import lltype +from pypy.rpython.lltypesystem import lltype, llmemory from pypy.objspace.flow.model import Constant, Variable -from pypy.rpython.memory.lladdress import address, NULL +from pypy.rpython.memory.lladdress import NULL log = log.database @@ -38,7 +38,7 @@ lltype.Void: "void", lltype.UnsignedLongLong: "ulong", lltype.SignedLongLong: "long", - address: "sbyte*"} + llmemory.Address: "sbyte*"} # 32 bit platform if sys.maxint == 2**31-1: @@ -362,7 +362,7 @@ repr = str(ord(value)) elif type_ is lltype.Float: repr = self.float_to_str(value) - elif type_ is address: + elif type_ is llmemory.Address: assert value == NULL repr = 'null' else: Modified: pypy/dist/pypy/translator/llvm/test/test_lladdresses.py ============================================================================== --- pypy/dist/pypy/translator/llvm/test/test_lladdresses.py (original) +++ pypy/dist/pypy/translator/llvm/test/test_lladdresses.py Tue Jan 24 13:13:13 2006 @@ -4,7 +4,6 @@ from pypy.rpython.objectmodel import free_non_gc_object import py -py.test.skip("WIP") def test_null(): def f(): From ludal at codespeak.net Tue Jan 24 13:18:02 2006 From: ludal at codespeak.net (ludal at codespeak.net) Date: Tue, 24 Jan 2006 13:18:02 +0100 (CET) Subject: [pypy-svn] r22583 - pypy/branch/ast-experiments/pypy/interpreter/pyparser Message-ID: <20060124121802.9AD0027DB6@code1.codespeak.net> Author: ludal Date: Tue Jan 24 13:18:01 2006 New Revision: 22583 Modified: pypy/branch/ast-experiments/pypy/interpreter/pyparser/ebnfparse.py Log: bugfixes Modified: pypy/branch/ast-experiments/pypy/interpreter/pyparser/ebnfparse.py ============================================================================== --- pypy/branch/ast-experiments/pypy/interpreter/pyparser/ebnfparse.py (original) +++ pypy/branch/ast-experiments/pypy/interpreter/pyparser/ebnfparse.py Tue Jan 24 13:18:01 2006 @@ -27,14 +27,14 @@ return False for c in name: v = ord(c) - if not (ORDA <= v <= ORDZ or + if not (ORDA <= v <= ORDZ or ORDa <= v <= ORDz or ORD0 <= v <= ORD9 or v == ORD_ ): return False return True - - + + punct=['>=', '<>', '!=', '<', '>', '<=', '==', '\\*=', '//=', '%=', '^=', '<<=', '\\*\\*=', '\\', '=', @@ -79,7 +79,7 @@ return self.debug_return( ret, tk.codename, tk.value ) source.restore( ctx ) return 0 - + def match_token(self, other): """special case of match token for tokens which are really keywords """ @@ -214,7 +214,6 @@ AbstractBuilder.__init__(self, rules, debug, symbols) self.rule_stack = [] self.root_rules = {} - self.keywords = [] self.seqcounts = [] # number of items in the current sequence self.altcounts = [] # number of sequence in the current alternative self.curaltcount = 0 @@ -223,8 +222,12 @@ self.current_rule = -1 self.all_rules = [] self.tokens = {} + self.keywords = [] + self.tokens[pytoken.NAME] = NameToken(keywords=self.keywords) def new_symbol(self): + """Allocate and return a new (anonymous) grammar symbol whose + name is based on the current grammar rule being parsed""" current_rule_name = self.symbols.sym_name.get(self.current_rule,"x") rule_name = ":" + current_rule_name + "_%d" % self.current_subrule self.current_subrule += 1 @@ -232,6 +235,7 @@ return symval def new_rule(self, rule): + """A simple helper method that registers a new rule as 'known'""" self.all_rules.append(rule) return rule @@ -255,6 +259,7 @@ del self.root_rules[codename] def get_token(self, codename ): + """Returns a new or existing token""" if codename in self.tokens: return self.tokens[codename] token = self.tokens[codename] = Token(codename) @@ -324,17 +329,26 @@ self.curseqcount += 1 elif _rule == ebnfgrammar.option: # print " -option", self.curaltcount, self.curseqcount + # pops the last alternative + rules = self.pop_rules( 1 ) + new_rule = self.new_rule(KleeneStar( self.new_symbol(), _min=0, _max=1, rule=rules[0] )) + self.rule_stack.append( new_rule ) self.curseqcount += 1 elif _rule == ebnfgrammar.rule: # print " -rule", self.curaltcount, self.curseqcount assert len(self.rule_stack)==1 old_rule = self.rule_stack[0] del self.rule_stack[0] - old_rule.codename = self.current_rule + if isinstance(old_rule,Token): + # Wrap a token into an alternative + old_rule = self.new_rule(Alternative( self.current_rule, [old_rule] )) + else: + # Make sure we use the codename from the named rule + old_rule.codename = self.current_rule self.root_rules[self.current_rule] = old_rule self.current_subrule = 0 return True - + def token(self, name, value, source): # print "token", name, value if name == ebnfgrammar.TOK_STRING: From arigo at codespeak.net Tue Jan 24 13:30:07 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 24 Jan 2006 13:30:07 +0100 (CET) Subject: [pypy-svn] r22584 - pypy/dist/pypy/tool Message-ID: <20060124123007.66C5527DB6@code1.codespeak.net> Author: arigo Date: Tue Jan 24 13:30:06 2006 New Revision: 22584 Modified: pypy/dist/pypy/tool/fixeol Log: fix for fixeol. Modified: pypy/dist/pypy/tool/fixeol ============================================================================== --- pypy/dist/pypy/tool/fixeol (original) +++ pypy/dist/pypy/tool/fixeol Tue Jan 24 13:30:06 2006 @@ -43,6 +43,7 @@ "by fixing the EOL style.") print "*"*30 #return False + path.write(data, mode='wb') return True return True From arigo at codespeak.net Tue Jan 24 13:31:33 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 24 Jan 2006 13:31:33 +0100 (CET) Subject: [pypy-svn] r22585 - in pypy/dist/pypy: annotation doc doc/statistic doc/weekly interpreter/pyparser jit/test lib/logic module/_socket/rpython module/recparser/test rpython rpython/module rpython/ootypesystem rpython/ootypesystem/test rpython/test tool tool/pytest/run-script translator translator/asm translator/asm/i386gen translator/backendopt translator/backendopt/test translator/c/test translator/goal translator/js translator/js/test translator/llvm/test translator/microbench translator/pickle translator/test Message-ID: <20060124123133.0E15127DB8@code1.codespeak.net> Author: arigo Date: Tue Jan 24 13:31:23 2006 New Revision: 22585 Modified: pypy/dist/pypy/annotation/description.py (props changed) pypy/dist/pypy/doc/constraints-and-logic.txt (props changed) pypy/dist/pypy/doc/index-report.txt (props changed) pypy/dist/pypy/doc/statistic/rebin.py (props changed) pypy/dist/pypy/doc/weekly/summary-2005-12-09.txt (props changed) pypy/dist/pypy/doc/weekly/summary-2005-12-16.txt (props changed) pypy/dist/pypy/interpreter/pyparser/ebnfgrammar.py (props changed) pypy/dist/pypy/jit/test/__init__.py (props changed) pypy/dist/pypy/jit/test/test_jit_tl.py (props changed) pypy/dist/pypy/lib/logic/ (props changed) pypy/dist/pypy/lib/logic/constraint.py (props changed) pypy/dist/pypy/lib/logic/test_unification.py (props changed) pypy/dist/pypy/lib/logic/test_variable.py (props changed) pypy/dist/pypy/lib/logic/unification.py (props changed) pypy/dist/pypy/lib/logic/variable.py (props changed) pypy/dist/pypy/module/_socket/rpython/rsocket.py (contents, props changed) pypy/dist/pypy/module/recparser/test/test_compilehooks.py (props changed) pypy/dist/pypy/rpython/module/ll_js.py (props changed) pypy/dist/pypy/rpython/ootypesystem/rbuiltin.py (props changed) pypy/dist/pypy/rpython/ootypesystem/test/test_oopbc.py (props changed) pypy/dist/pypy/rpython/rgenop.py (props changed) pypy/dist/pypy/rpython/rjs.py (props changed) pypy/dist/pypy/rpython/test/test_rgenop.py (props changed) pypy/dist/pypy/tool/importfun.py (props changed) pypy/dist/pypy/tool/pytest/run-script/ (props changed) pypy/dist/pypy/tool/release_dates.py (props changed) pypy/dist/pypy/tool/statistic_irc_log.py (props changed) pypy/dist/pypy/tool/statistic_over_time.py (props changed) pypy/dist/pypy/translator/asm/i386gen/__init__.py (contents, props changed) pypy/dist/pypy/translator/asm/i386gen/i386_assembler.py (contents, props changed) pypy/dist/pypy/translator/asm/model.py (props changed) pypy/dist/pypy/translator/asm/simulator.py (contents, props changed) pypy/dist/pypy/translator/backendopt/escape.py (props changed) pypy/dist/pypy/translator/backendopt/merge_if_blocks.py (props changed) pypy/dist/pypy/translator/backendopt/test/test_escape.py (props changed) pypy/dist/pypy/translator/backendopt/test/test_merge_if_blocks.py (props changed) pypy/dist/pypy/translator/c/test/test_coroutine.py (contents, props changed) pypy/dist/pypy/translator/goal/targetebnflexer.py (props changed) pypy/dist/pypy/translator/goal/targetrecursivestandalone.py (props changed) pypy/dist/pypy/translator/interactive.py (props changed) pypy/dist/pypy/translator/js/conftest.py (props changed) pypy/dist/pypy/translator/js/optimize.py (props changed) pypy/dist/pypy/translator/js/support.py (props changed) pypy/dist/pypy/translator/js/test/browsertest.py (props changed) pypy/dist/pypy/translator/js/test/test_jseval.py (props changed) pypy/dist/pypy/translator/js/test/test_merge_if_blocks.py (props changed) pypy/dist/pypy/translator/llvm/test/test_merge_if_blocks.py (props changed) pypy/dist/pypy/translator/microbench/ (props changed) pypy/dist/pypy/translator/microbench/microbench.py (props changed) pypy/dist/pypy/translator/microbench/test_count1.py (props changed) pypy/dist/pypy/translator/microbench/test_create1.py (props changed) pypy/dist/pypy/translator/pickle/genpickle.py (contents, props changed) pypy/dist/pypy/translator/test/test_geninterp.py (contents, props changed) pypy/dist/pypy/translator/test/test_interactive.py (props changed) pypy/dist/pypy/translator/test/test_rpystone.py (contents, props changed) Log: fixeol Modified: pypy/dist/pypy/module/_socket/rpython/rsocket.py ============================================================================== --- pypy/dist/pypy/module/_socket/rpython/rsocket.py (original) +++ pypy/dist/pypy/module/_socket/rpython/rsocket.py Tue Jan 24 13:31:23 2006 @@ -1,55 +1,55 @@ -""" -Helper file for Python equivalents of socket specific calls. -""" - -import socket - -# HACK: We have to prevent GC to collect the socket object we create within this -# module. Because socket.close() is called on GC this can lead to strange -# effects in corner cases where file descriptors are reused. -socket_cache = {} -keep_sockets_alive = [] - -class ADDRINFO(object): - # a simulated addrinfo structure from C, i.e. a chained list - # returned by getaddrinfo() - def __init__(self, host, port, family, socktype, proto, flags): - addrinfo = socket.getaddrinfo(host, port, - family, socktype, proto, flags) - self._entries = iter(addrinfo) - - def nextinfo(self): - try: - info = self._entries.next() - except StopIteration: - return [0] * 8 - - return info[:-1] + info[-1] - - def free(self): - pass - -def getaddrinfo(host, port, family, socktype, proto, flags): - return ADDRINFO(host, port, family, socktype, proto, flags) - -def newsocket(family, type, protocol): - s = socket.socket(family, type, protocol) - fileno = s.fileno() - if socket_cache.has_key(fileno): - keep_sockets_alive.append(socket_cache[fileno]) - socket_cache[fileno] = s - return fileno - -def connect(fd, sockname, family): - s = socket_cache[fd] - if family == socket.AF_INET: - s.connect(sockname[:2]) - elif family == socket.AF_INET6: - s.connect(sockname) - -def getpeername(fd): - s = socket_cache[fd] - return s.getpeername() - -def freesockname(sockname): - pass +""" +Helper file for Python equivalents of socket specific calls. +""" + +import socket + +# HACK: We have to prevent GC to collect the socket object we create within this +# module. Because socket.close() is called on GC this can lead to strange +# effects in corner cases where file descriptors are reused. +socket_cache = {} +keep_sockets_alive = [] + +class ADDRINFO(object): + # a simulated addrinfo structure from C, i.e. a chained list + # returned by getaddrinfo() + def __init__(self, host, port, family, socktype, proto, flags): + addrinfo = socket.getaddrinfo(host, port, + family, socktype, proto, flags) + self._entries = iter(addrinfo) + + def nextinfo(self): + try: + info = self._entries.next() + except StopIteration: + return [0] * 8 + + return info[:-1] + info[-1] + + def free(self): + pass + +def getaddrinfo(host, port, family, socktype, proto, flags): + return ADDRINFO(host, port, family, socktype, proto, flags) + +def newsocket(family, type, protocol): + s = socket.socket(family, type, protocol) + fileno = s.fileno() + if socket_cache.has_key(fileno): + keep_sockets_alive.append(socket_cache[fileno]) + socket_cache[fileno] = s + return fileno + +def connect(fd, sockname, family): + s = socket_cache[fd] + if family == socket.AF_INET: + s.connect(sockname[:2]) + elif family == socket.AF_INET6: + s.connect(sockname) + +def getpeername(fd): + s = socket_cache[fd] + return s.getpeername() + +def freesockname(sockname): + pass Modified: pypy/dist/pypy/translator/asm/i386gen/__init__.py ============================================================================== --- pypy/dist/pypy/translator/asm/i386gen/__init__.py (original) +++ pypy/dist/pypy/translator/asm/i386gen/__init__.py Tue Jan 24 13:31:23 2006 @@ -1 +1 @@ -#thats all +#thats all Modified: pypy/dist/pypy/translator/asm/i386gen/i386_assembler.py ============================================================================== --- pypy/dist/pypy/translator/asm/i386gen/i386_assembler.py (original) +++ pypy/dist/pypy/translator/asm/i386gen/i386_assembler.py Tue Jan 24 13:31:23 2006 @@ -1,44 +1,44 @@ -"""i386 Basic assembler... -Designed to mirror the PPC assembler system, operands added as required. - -Current system needs to assemble given, code, link into python, and return python -callabale. (Stub routine currently given). - -""" - -class i386Assembler: - - def __init__(self): - self._opcodes=[] - - def __getattr__(self,attr): - def func(*args): - return self.op(attr,args) - return func - - def op(self,opcode,*args): - self._opcodes.append((opcode,args)) - - def Make_func(cls,assembler,input='ii',output='i'): - return lambda x,y:x+y+1 - - Make_func=classmethod(Make_func) - - def dump(self): - l=1000 - for op in self._opcodes: - print '>>%d :%s' %(l,str(op)) - l+=1 - -make_func=i386Assembler.Make_func - - -if __name__=='__main__': - a=i386Assembler() - a.op('mov','ax,''bx') - - a.mov('spi','esi') - print a._opcodes - a.dump() - - +"""i386 Basic assembler... +Designed to mirror the PPC assembler system, operands added as required. + +Current system needs to assemble given, code, link into python, and return python +callabale. (Stub routine currently given). + +""" + +class i386Assembler: + + def __init__(self): + self._opcodes=[] + + def __getattr__(self,attr): + def func(*args): + return self.op(attr,args) + return func + + def op(self,opcode,*args): + self._opcodes.append((opcode,args)) + + def Make_func(cls,assembler,input='ii',output='i'): + return lambda x,y:x+y+1 + + Make_func=classmethod(Make_func) + + def dump(self): + l=1000 + for op in self._opcodes: + print '>>%d :%s' %(l,str(op)) + l+=1 + +make_func=i386Assembler.Make_func + + +if __name__=='__main__': + a=i386Assembler() + a.op('mov','ax,''bx') + + a.mov('spi','esi') + print a._opcodes + a.dump() + + Modified: pypy/dist/pypy/translator/asm/simulator.py ============================================================================== --- pypy/dist/pypy/translator/asm/simulator.py (original) +++ pypy/dist/pypy/translator/asm/simulator.py Tue Jan 24 13:31:23 2006 @@ -1,217 +1,217 @@ -""" IRM Simulator """ -import autopath -from pypy.rpython.llinterp import LLFrame -#from pypy.translator.asm.infregmachine import Instruction - -""" -Notes on the register allocation algorithm: - - -To convert our IRM to an FRM machine, we must perform some swapping of the registers. This is in effect -'paging', we are only allowed to perform XCHG operations on the slow (memory) registers, while we can do -anything with our fast (CPU registers). - -There are various algorithms available, including the Linear Scan Algorithm (google this), but for now we -have decided to implement a simple, but (hopefully) reasonably effective last-recently-used algortithm. - -Regardless of the swap algorithm , at each stage we must keep track of which IRM register is held in which -FRM register. Our original test-suite simply gave the register usages, and checked the swap/usage sequence. - -We need to rebuild the suite, checking the register map at EACH stage of the process. Fiddly, but important! - -We need some notation: - -IRMxxx denotes an Infinite Register Machine that will use at most xxx registers - -FRMxxx.yyy denotes a finite-register machine, with xxx fast registers, and a total of yyy registers. - -""" - - -def regmap(regperm): - """answer a map IRM notation -> current FRM notation""" - map={} - for reg in range(1,len(regperm)): - #print reg,map,regperm - map[reg]=regperm.index(reg) - return map - -def maxRegister(commands): - pool=[] - for cmd in commands: - if not isinstance(cmd,str): - pool+=cmd.registers_used() - if pool==[]: - return 1 - return max(pool) - - -def TranslateProgram(commands,nreg): - """answer this program into one which only uses nreg fast registers""" - totreg=maxRegister(commands) - assert nreg>=3 ,'Some commands may use 3 registers!!!!' - newprog=[] - pipe=[] - - # this must be as big as the total number of registers+1 (we start - # at index 1) - old2new=range(0,totreg+1) - - - for cmd in commands: - # if we use any registers, we must possibly swap first, and - # then remap - if isinstance(cmd,str) or cmd.name in ('J','JT','JF'): - # label or jump so pass through - newprog.append(cmd) - else: - # so now remap the registers! - - regused=cmd.registers_used() - t2p=[old2new[x] for x in regused] - for reg in regused: - goingin=regmap(old2new)[reg] - if goingin>nreg: - if pipe[-1] not in t2p: - index=-1 - elif pipe[-2] not in t2p: - index=-2 - else: - assert pipe[-3]!=goingin #this must be true for nreg>=3 - index=-3 - #now swap to end of pipe, so code as before works. - pipe[index],pipe[-1]=pipe[-1],pipe[index] - goingout=pipe[-1] - newprog.append(Instruction('EXCH',(goingin,goingout))) - old2new[goingout],old2new[goingin] = old2new[goingin],old2new[goingout] - val=goingout - else: - val=goingin - pipe=[val]+pipe - - if len(pipe)>nreg: - pipe.pop() #this value fell out of the pipe - assert len(pipe)<=nreg - #now we can emit the command with registers remapped - rm=regmap(old2new) - newprog.append(cmd.renumber(rm)) - return newprog - - - -class Machine: - - def RunProgram(cls,commands,args=[],tracing=False): - nreg=maxRegister(commands) - machine=Machine(nreg,args) - machine._tracing = tracing - ip=0 - if tracing: - print 'args', args - while not machine.stopped(): - if ip>=len(commands): - return None - cmd=commands[ip] - if isinstance(cmd,str): - pass - elif cmd.name=='J': - ip=commands.index(cmd.arguments[0]) - elif cmd.name=='JT': - c = machine.creg() - assert c is not None - if c: - ip=commands.index(cmd.arguments[0]) - else: - machine.op(cmd.name,*cmd.arguments) - ip+=1 - if tracing: - print 'ret', machine._retval - return machine._retval - RunProgram=classmethod(RunProgram) - - - def __init__(self,nreg,args): - self._nreg=nreg - self._args=args - self._stopped=False - self._creg=None - self._tracing = False - self._registers=[None for x in range(nreg+1)] - - def creg(self): - return self._creg - - def registers(self): - return self._registers[1:] - - def register(self, reg): - v = self._registers[reg] - assert v is not None - return v - - def stopped(self): - return self._stopped - - def op(self,opcode,*operands): - if self._tracing: - args = [] - for arg in operands: - if isinstance(arg, int): - args.append('r%s=%s'%(arg, self._registers[arg])) - else: - args.append(arg) - print opcode, ', '.join(map(str, args)) - #will want to trap later to defer unimplemented to the LLInterpreter... - m = getattr(self,opcode,None) - if m is not None: - m(*operands) - else: - self.llop(opcode, *operands) - - def RETPYTHON(self,reg): - self._stopped=True - self._retval=self.register(reg) - - def LIA(self,destination,argindex): - self._registers[destination]=self._args[argindex.value] - - def LOAD(self,destination,immed): - self._registers[destination]=immed.value - - def MOV(self,destination,source): - self._registers[destination]=self.register(source) - - def EXCH(self,destination,source): - #self._registers[destination],self._registers[source]=self.register(source),self.register(destination) - self._registers[destination],self._registers[source]=self._registers[source],self._registers[destination] - - - def int_gt(self,rega,regb): - self._creg = self.register(rega) > self.register(regb) - - def int_lt(self,rega,regb): - self._creg = self.register(rega) < self.register(regb) - - def int_ge(self,rega,regb): - self._creg = self.register(rega) >= self.register(regb) - - def int_le(self,rega,regb): - self._creg = self.register(rega) <= self.register(regb) - - def int_eq(self,rega,regb): - self._creg = self.register(rega) == self.register(regb) - - def int_ne(self,rega,regb): - self._creg = self.register(rega) != self.register(regb) - - - def llop(self, opcode, destination, *sources): - sourcevalues = [] - for r in sources: - sourcevalues.append(self.register(r)) - self._registers[destination] = LLFrame.__dict__['op_'+opcode](None, *sourcevalues) - - - - - +""" IRM Simulator """ +import autopath +from pypy.rpython.llinterp import LLFrame +#from pypy.translator.asm.infregmachine import Instruction + +""" +Notes on the register allocation algorithm: + + +To convert our IRM to an FRM machine, we must perform some swapping of the registers. This is in effect +'paging', we are only allowed to perform XCHG operations on the slow (memory) registers, while we can do +anything with our fast (CPU registers). + +There are various algorithms available, including the Linear Scan Algorithm (google this), but for now we +have decided to implement a simple, but (hopefully) reasonably effective last-recently-used algortithm. + +Regardless of the swap algorithm , at each stage we must keep track of which IRM register is held in which +FRM register. Our original test-suite simply gave the register usages, and checked the swap/usage sequence. + +We need to rebuild the suite, checking the register map at EACH stage of the process. Fiddly, but important! + +We need some notation: + +IRMxxx denotes an Infinite Register Machine that will use at most xxx registers + +FRMxxx.yyy denotes a finite-register machine, with xxx fast registers, and a total of yyy registers. + +""" + + +def regmap(regperm): + """answer a map IRM notation -> current FRM notation""" + map={} + for reg in range(1,len(regperm)): + #print reg,map,regperm + map[reg]=regperm.index(reg) + return map + +def maxRegister(commands): + pool=[] + for cmd in commands: + if not isinstance(cmd,str): + pool+=cmd.registers_used() + if pool==[]: + return 1 + return max(pool) + + +def TranslateProgram(commands,nreg): + """answer this program into one which only uses nreg fast registers""" + totreg=maxRegister(commands) + assert nreg>=3 ,'Some commands may use 3 registers!!!!' + newprog=[] + pipe=[] + + # this must be as big as the total number of registers+1 (we start + # at index 1) + old2new=range(0,totreg+1) + + + for cmd in commands: + # if we use any registers, we must possibly swap first, and + # then remap + if isinstance(cmd,str) or cmd.name in ('J','JT','JF'): + # label or jump so pass through + newprog.append(cmd) + else: + # so now remap the registers! + + regused=cmd.registers_used() + t2p=[old2new[x] for x in regused] + for reg in regused: + goingin=regmap(old2new)[reg] + if goingin>nreg: + if pipe[-1] not in t2p: + index=-1 + elif pipe[-2] not in t2p: + index=-2 + else: + assert pipe[-3]!=goingin #this must be true for nreg>=3 + index=-3 + #now swap to end of pipe, so code as before works. + pipe[index],pipe[-1]=pipe[-1],pipe[index] + goingout=pipe[-1] + newprog.append(Instruction('EXCH',(goingin,goingout))) + old2new[goingout],old2new[goingin] = old2new[goingin],old2new[goingout] + val=goingout + else: + val=goingin + pipe=[val]+pipe + + if len(pipe)>nreg: + pipe.pop() #this value fell out of the pipe + assert len(pipe)<=nreg + #now we can emit the command with registers remapped + rm=regmap(old2new) + newprog.append(cmd.renumber(rm)) + return newprog + + + +class Machine: + + def RunProgram(cls,commands,args=[],tracing=False): + nreg=maxRegister(commands) + machine=Machine(nreg,args) + machine._tracing = tracing + ip=0 + if tracing: + print 'args', args + while not machine.stopped(): + if ip>=len(commands): + return None + cmd=commands[ip] + if isinstance(cmd,str): + pass + elif cmd.name=='J': + ip=commands.index(cmd.arguments[0]) + elif cmd.name=='JT': + c = machine.creg() + assert c is not None + if c: + ip=commands.index(cmd.arguments[0]) + else: + machine.op(cmd.name,*cmd.arguments) + ip+=1 + if tracing: + print 'ret', machine._retval + return machine._retval + RunProgram=classmethod(RunProgram) + + + def __init__(self,nreg,args): + self._nreg=nreg + self._args=args + self._stopped=False + self._creg=None + self._tracing = False + self._registers=[None for x in range(nreg+1)] + + def creg(self): + return self._creg + + def registers(self): + return self._registers[1:] + + def register(self, reg): + v = self._registers[reg] + assert v is not None + return v + + def stopped(self): + return self._stopped + + def op(self,opcode,*operands): + if self._tracing: + args = [] + for arg in operands: + if isinstance(arg, int): + args.append('r%s=%s'%(arg, self._registers[arg])) + else: + args.append(arg) + print opcode, ', '.join(map(str, args)) + #will want to trap later to defer unimplemented to the LLInterpreter... + m = getattr(self,opcode,None) + if m is not None: + m(*operands) + else: + self.llop(opcode, *operands) + + def RETPYTHON(self,reg): + self._stopped=True + self._retval=self.register(reg) + + def LIA(self,destination,argindex): + self._registers[destination]=self._args[argindex.value] + + def LOAD(self,destination,immed): + self._registers[destination]=immed.value + + def MOV(self,destination,source): + self._registers[destination]=self.register(source) + + def EXCH(self,destination,source): + #self._registers[destination],self._registers[source]=self.register(source),self.register(destination) + self._registers[destination],self._registers[source]=self._registers[source],self._registers[destination] + + + def int_gt(self,rega,regb): + self._creg = self.register(rega) > self.register(regb) + + def int_lt(self,rega,regb): + self._creg = self.register(rega) < self.register(regb) + + def int_ge(self,rega,regb): + self._creg = self.register(rega) >= self.register(regb) + + def int_le(self,rega,regb): + self._creg = self.register(rega) <= self.register(regb) + + def int_eq(self,rega,regb): + self._creg = self.register(rega) == self.register(regb) + + def int_ne(self,rega,regb): + self._creg = self.register(rega) != self.register(regb) + + + def llop(self, opcode, destination, *sources): + sourcevalues = [] + for r in sources: + sourcevalues.append(self.register(r)) + self._registers[destination] = LLFrame.__dict__['op_'+opcode](None, *sourcevalues) + + + + + Modified: pypy/dist/pypy/translator/c/test/test_coroutine.py ============================================================================== --- pypy/dist/pypy/translator/c/test/test_coroutine.py (original) +++ pypy/dist/pypy/translator/c/test/test_coroutine.py Tue Jan 24 13:31:23 2006 @@ -1,245 +1,245 @@ -""" -minimalistic coroutine implementation -""" - -import os -import py -from pypy.rpython.rstack import yield_current_frame_to_caller - -def wrap_stackless_function(fn): - from pypy.translator.translator import TranslationContext - from pypy.translator.c.genc import CStandaloneBuilder - from pypy.annotation.model import SomeList, SomeString - from pypy.annotation.listdef import ListDef - from pypy.translator.backendopt.all import backend_optimizations - - def entry_point(argv): - os.write(1, str(fn())) - return 0 - - s_list_of_strings = SomeList(ListDef(None, SomeString())) - s_list_of_strings.listdef.resize() - t = TranslationContext() - t.buildannotator().build_types(entry_point, [s_list_of_strings]) - #t.view() - t.buildrtyper().specialize() - backend_optimizations(t) - cbuilder = CStandaloneBuilder(t, entry_point) - cbuilder.stackless = True - cbuilder.generate_source() - cbuilder.compile() - return cbuilder.cmdexec('') - -# ____________________________________________________________ - -""" -Trying to build the simplest possible coroutine interface. - -A coroutine is a tiny wrapper around a frame, or better -to say a one-shot continuation. This continuation is -resumed whenever we switch to the coroutine. On depart, -the coroutine is updated with its current state, that is, -the continuation is replaced. To avoid confusion with -general continuations, we are naming them as 'frame' -in the code. By frame, we are referring to the toplevel -frame as a placeholder for the whole structure appended -to it. This might be a chain of frames, or even a special -stack structure, when we implement 'hard switching'. The -abstraction layer should make this invisible. - -The 'seed' of coroutines is actually the special function -yield_current_frame_to_caller(). It is, in a sense, able -to return twice. When yield_current_frame_to_caller() is -reached, it creates a resumable frame and returns it to the -caller of the current function. This frame serves as the -entry point to the coroutine. - -On every entry to the coroutine, the return value of the -point where we left off is the continuation of the caller. -We need to update the caller's frame with it. -This is not necessarily the caller which created ourself. -We are therefore keeping track of the current coroutine. - -The update sequence during a switch to a coroutine is: - -- save the return value (caller's continuation) in the - calling coroutine, which is still 'current' -- change current to ourself (the callee) -- invalidate our continuation by setting it to None. -""" - - -class CoState(object): - pass - -costate = CoState() - -class CoroutineDamage(SystemError): - pass - -class Coroutine(object): - - def __init__(self): - self.frame = None - - def bind(self, thunk): - if self.frame is not None: - raise CoroutineDamage - self.frame = self._bind(thunk) - - def _bind(self, thunk): - binder = costate.current - costate.last.frame = yield_current_frame_to_caller() - thunk.call() - if binder.frame is None: - binder = costate.main - costate.last, costate.current = costate.current, binder - frame, binder.frame = binder.frame, None - return frame - - def switch(self): - if self.frame is None: - raise CoroutineDamage - costate.last, costate.current = costate.current, self - frame, self.frame = self.frame, None - costate.last.frame = frame.switch() - -costate.current = costate.last = costate.main = Coroutine() - -def output(stuff): - os.write(2, stuff + '\n') - -def test_coroutine(): - - def g(lst, coros): - coro_f, coro_g, coro_h = coros - lst.append(2) - output('g appended 2') - coro_h.switch() - lst.append(5) - output('g appended 5') - - def h(lst, coros): - coro_f, coro_g, coro_h = coros - lst.append(3) - output('h appended 3') - coro_f.switch() - lst.append(7) - output('h appended 7') - - class T: - def __init__(self, func, arg1, arg2): - self.func = func - self.arg1 = arg1 - self.arg2 = arg2 - def call(self): - self.func(self.arg1, self.arg2) - - def f(): - lst = [1] - coro_f = costate.main - coro_g = Coroutine() - coro_h = Coroutine() - coros = [coro_f, coro_g, coro_h] - thunk_g = T(g, lst, coros) - output('binding g after f set 1') - coro_g.bind(thunk_g) - thunk_h = T(h, lst, coros) - output('binding h after f set 1') - coro_h.bind(thunk_h) - output('switching to g') - coro_g.switch() - lst.append(4) - output('f appended 4') - coro_g.switch() - lst.append(6) - output('f appended 6') - coro_h.switch() - lst.append(8) - output('f appended 8') - n = 0 - for i in lst: - n = n*10 + i - return n - - data = wrap_stackless_function(f) - assert int(data.strip()) == 12345678 - -def test_coroutine2(): - - class TBase: - def call(self): - pass - - class T(TBase): - def __init__(self, func, arg1, arg2): - self.func = func - self.arg1 = arg1 - self.arg2 = arg2 - def call(self): - self.res = self.func(self.arg1, self.arg2) - - class T1(TBase): - def __init__(self, func, arg1): - self.func = func - self.arg1 = arg1 - def call(self): - self.res = self.func(self.arg1) - - def g(lst, coros): - coro_f1, coro_g, coro_h = coros - lst.append(2) - output('g appended 2') - coro_h.switch() - lst.append(5) - output('g appended 5') - output('exiting g') - - def h(lst, coros): - coro_f1, coro_g, coro_h = coros - lst.append(3) - output('h appended 3') - coro_f1.switch() - lst.append(7) - output('h appended 7') - output('exiting h') - - def f1(coro_f1): - lst = [1] - coro_g = Coroutine() - coro_h = Coroutine() - coros = [coro_f1, coro_g, coro_h] - thunk_g = T(g, lst, coros) - output('binding g after f1 set 1') - coro_g.bind(thunk_g) - thunk_h = T(h, lst, coros) - output('binding h after f1 set 1') - coro_h.bind(thunk_h) - output('switching to g') - coro_g.switch() - lst.append(4) - output('f1 appended 4') - coro_g.switch() - lst.append(6) - output('f1 appended 6') - coro_h.switch() - lst.append(8) - output('f1 appended 8') - n = 0 - for i in lst: - n = n*10 + i - output('exiting f1') - return n - - def f(): - coro_f = costate.main - coro_f1 = Coroutine() - thunk_f1 = T1(f1, coro_f1) - output('binding f1 after f set 1') - coro_f1.bind(thunk_f1) - coro_f1.switch() - output('return to main :-(') - return thunk_f1.res - - data = wrap_stackless_function(f) - assert int(data.strip()) == 12345678 +""" +minimalistic coroutine implementation +""" + +import os +import py +from pypy.rpython.rstack import yield_current_frame_to_caller + +def wrap_stackless_function(fn): + from pypy.translator.translator import TranslationContext + from pypy.translator.c.genc import CStandaloneBuilder + from pypy.annotation.model import SomeList, SomeString + from pypy.annotation.listdef import ListDef + from pypy.translator.backendopt.all import backend_optimizations + + def entry_point(argv): + os.write(1, str(fn())) + return 0 + + s_list_of_strings = SomeList(ListDef(None, SomeString())) + s_list_of_strings.listdef.resize() + t = TranslationContext() + t.buildannotator().build_types(entry_point, [s_list_of_strings]) + #t.view() + t.buildrtyper().specialize() + backend_optimizations(t) + cbuilder = CStandaloneBuilder(t, entry_point) + cbuilder.stackless = True + cbuilder.generate_source() + cbuilder.compile() + return cbuilder.cmdexec('') + +# ____________________________________________________________ + +""" +Trying to build the simplest possible coroutine interface. + +A coroutine is a tiny wrapper around a frame, or better +to say a one-shot continuation. This continuation is +resumed whenever we switch to the coroutine. On depart, +the coroutine is updated with its current state, that is, +the continuation is replaced. To avoid confusion with +general continuations, we are naming them as 'frame' +in the code. By frame, we are referring to the toplevel +frame as a placeholder for the whole structure appended +to it. This might be a chain of frames, or even a special +stack structure, when we implement 'hard switching'. The +abstraction layer should make this invisible. + +The 'seed' of coroutines is actually the special function +yield_current_frame_to_caller(). It is, in a sense, able +to return twice. When yield_current_frame_to_caller() is +reached, it creates a resumable frame and returns it to the +caller of the current function. This frame serves as the +entry point to the coroutine. + +On every entry to the coroutine, the return value of the +point where we left off is the continuation of the caller. +We need to update the caller's frame with it. +This is not necessarily the caller which created ourself. +We are therefore keeping track of the current coroutine. + +The update sequence during a switch to a coroutine is: + +- save the return value (caller's continuation) in the + calling coroutine, which is still 'current' +- change current to ourself (the callee) +- invalidate our continuation by setting it to None. +""" + + +class CoState(object): + pass + +costate = CoState() + +class CoroutineDamage(SystemError): + pass + +class Coroutine(object): + + def __init__(self): + self.frame = None + + def bind(self, thunk): + if self.frame is not None: + raise CoroutineDamage + self.frame = self._bind(thunk) + + def _bind(self, thunk): + binder = costate.current + costate.last.frame = yield_current_frame_to_caller() + thunk.call() + if binder.frame is None: + binder = costate.main + costate.last, costate.current = costate.current, binder + frame, binder.frame = binder.frame, None + return frame + + def switch(self): + if self.frame is None: + raise CoroutineDamage + costate.last, costate.current = costate.current, self + frame, self.frame = self.frame, None + costate.last.frame = frame.switch() + +costate.current = costate.last = costate.main = Coroutine() + +def output(stuff): + os.write(2, stuff + '\n') + +def test_coroutine(): + + def g(lst, coros): + coro_f, coro_g, coro_h = coros + lst.append(2) + output('g appended 2') + coro_h.switch() + lst.append(5) + output('g appended 5') + + def h(lst, coros): + coro_f, coro_g, coro_h = coros + lst.append(3) + output('h appended 3') + coro_f.switch() + lst.append(7) + output('h appended 7') + + class T: + def __init__(self, func, arg1, arg2): + self.func = func + self.arg1 = arg1 + self.arg2 = arg2 + def call(self): + self.func(self.arg1, self.arg2) + + def f(): + lst = [1] + coro_f = costate.main + coro_g = Coroutine() + coro_h = Coroutine() + coros = [coro_f, coro_g, coro_h] + thunk_g = T(g, lst, coros) + output('binding g after f set 1') + coro_g.bind(thunk_g) + thunk_h = T(h, lst, coros) + output('binding h after f set 1') + coro_h.bind(thunk_h) + output('switching to g') + coro_g.switch() + lst.append(4) + output('f appended 4') + coro_g.switch() + lst.append(6) + output('f appended 6') + coro_h.switch() + lst.append(8) + output('f appended 8') + n = 0 + for i in lst: + n = n*10 + i + return n + + data = wrap_stackless_function(f) + assert int(data.strip()) == 12345678 + +def test_coroutine2(): + + class TBase: + def call(self): + pass + + class T(TBase): + def __init__(self, func, arg1, arg2): + self.func = func + self.arg1 = arg1 + self.arg2 = arg2 + def call(self): + self.res = self.func(self.arg1, self.arg2) + + class T1(TBase): + def __init__(self, func, arg1): + self.func = func + self.arg1 = arg1 + def call(self): + self.res = self.func(self.arg1) + + def g(lst, coros): + coro_f1, coro_g, coro_h = coros + lst.append(2) + output('g appended 2') + coro_h.switch() + lst.append(5) + output('g appended 5') + output('exiting g') + + def h(lst, coros): + coro_f1, coro_g, coro_h = coros + lst.append(3) + output('h appended 3') + coro_f1.switch() + lst.append(7) + output('h appended 7') + output('exiting h') + + def f1(coro_f1): + lst = [1] + coro_g = Coroutine() + coro_h = Coroutine() + coros = [coro_f1, coro_g, coro_h] + thunk_g = T(g, lst, coros) + output('binding g after f1 set 1') + coro_g.bind(thunk_g) + thunk_h = T(h, lst, coros) + output('binding h after f1 set 1') + coro_h.bind(thunk_h) + output('switching to g') + coro_g.switch() + lst.append(4) + output('f1 appended 4') + coro_g.switch() + lst.append(6) + output('f1 appended 6') + coro_h.switch() + lst.append(8) + output('f1 appended 8') + n = 0 + for i in lst: + n = n*10 + i + output('exiting f1') + return n + + def f(): + coro_f = costate.main + coro_f1 = Coroutine() + thunk_f1 = T1(f1, coro_f1) + output('binding f1 after f set 1') + coro_f1.bind(thunk_f1) + coro_f1.switch() + output('return to main :-(') + return thunk_f1.res + + data = wrap_stackless_function(f) + assert int(data.strip()) == 12345678 Modified: pypy/dist/pypy/translator/pickle/genpickle.py ============================================================================== --- pypy/dist/pypy/translator/pickle/genpickle.py (original) +++ pypy/dist/pypy/translator/pickle/genpickle.py Tue Jan 24 13:31:23 2006 @@ -1,793 +1,793 @@ -""" -Generate a Python source file from the flowmodel. -The purpose is to create something that allows -to restart code generation after flowing and maybe -annotation. - -The generated source appeared to be way too large -for the CPython compiler. Therefore, we cut the -source into pieces and compile them seperately. -""" -from __future__ import generators, division, nested_scopes -import __future__ -all_feature_names = __future__.all_feature_names -import os, sys, new, __builtin__ - -from pypy.translator.gensupp import uniquemodulename, NameManager -from pypy.translator.gensupp import builtin_base -from pypy.rpython.rarithmetic import r_int, r_uint -from pypy.objspace.flow.model import Variable, Constant, SpaceOperation -from pypy.objspace.flow.model import FunctionGraph, Block, Link, Atom -from pypy.objspace.flow.flowcontext import SpamBlock, EggBlock -from pypy.annotation.model import SomeInteger, SomeObject, SomeChar, SomeBool -from pypy.annotation.model import SomeList, SomeString, SomeTuple -from pypy.annotation.unaryop import SomeInstance -from pypy.interpreter.baseobjspace import ObjSpace -from pypy.translator.pickle import slotted - -from pickle import whichmodule, PicklingError -from copy_reg import _reconstructor - -import pickle - -from types import * -import types, weakref - -class AlreadyCreated(Exception): pass - -# ____________________________________________________________ - - -#XXX Hack: This float is supposed to overflow to inf -OVERFLOWED_FLOAT = float("1e10000000000000000000000000000000") -#XXX Hack: and this one to underflow to -inf -OVERFLOWED_FLOATM = float("-1e10000000000000000000000000000000") - -class GenPickle: - - def __init__(self, translator, writer = None): - self.translator = translator - self.writer = writer - self.initcode = [] - self.produce = self._produce() - self.produce( - 'from __future__ import %s\n' % ', '.join(all_feature_names) + - 'import new, types, sys', - ) - self.picklenames = {} # memoize objects - for name in all_feature_names + "new types sys".split(): - self.memoize(globals()[name], name) - self.memoize((), '()') - self.namespace = NameManager() - self.uniquename = self.namespace.uniquename - self.namespace.make_reserved_names('None False True') - self.namespace.make_reserved_names('new types sys') - self.namespace.make_reserved_names(' '.join(all_feature_names)) - self.namespace.make_reserved_names('result') # result dict - self.result = {} - self.simple_const_types = { - int: repr, - long: repr, - float: self.save_float, - str: repr, - unicode: repr, - type(None): repr, - bool: repr, - } - self.typecache = {} # hold types vs. nameof methods - # we distinguish between the "user program" and other stuff. - # "user program" will never use save_global. - self.domains = ( - 'pypy.objspace.std.', - 'pypy.objspace.descroperation', - 'pypy._cache.', - 'pypy.interpreter.', - 'pypy.module.', - 'pypy.translator.test.', - '__main__', - ) - self.shortnames = { - Variable: 'V', - Constant: 'C', - Block: 'B', - SpamBlock: 'SB', - EggBlock: 'EB', - Link: 'L', - FunctionGraph: 'FG', - SomeInteger: 'sI', - SomeObject: 'sO', - SomeChar: 'sC', - SomeBool: 'sB', - SomeList: 'sL', - SomeString: 'sS', - SomeTuple: 'sT', - SomeInstance: 'sIn', - } - self.inline_instances = { - SpaceOperation: True, - } - - def save_float(self, fl): - if fl == OVERFLOWED_FLOAT: - return 'float("1e10000000000000000000000000000000")' - elif fl == OVERFLOWED_FLOATM: - return 'float("-1e10000000000000000000000000000000")' - return repr(fl) - - def pickle(self, *args, **kwds): - for obj in args: - self.nameof(obj) - for obj in kwds.values(): - self.nameof(obj) - self.result.update(kwds) - - def finish(self): - self.pickle(self.result) - self.produce('result = %s' % self.nameof(self.result)) - if self.writer: - self.writer.close() - - def memoize(self, obj, name): - self.picklenames[id(obj)] = name - return name - - def memoize_unique(self, obj, basename): - if id(obj) in self.picklenames: - raise AlreadyCreated - return self.memoize(obj, self.uniquename(basename)) - - def _produce(self): - writer = self.writer - down = 1234 - cnt = [0, 0] # text, calls - self.last_progress = '' - if writer: - write = writer.write - else: - write = self.initcode.append - def produce(text): - write(text+'\n') - cnt[0] += len(text) + 1 - cnt[1] += 1 - if cnt[1] == down: - cnt[1] = 0 - self.progress("%d" % cnt[0]) - return produce - - def progress(self, txt): - back = '\x08' * len(self.last_progress) - self.last_progress = txt+' ' # soft space - print back+txt, - - def nameof(self, obj): - try: - try: - return self.picklenames[id(obj)] - except KeyError: - typ = type(obj) - return self.simple_const_types[typ](obj) - except KeyError: - try: - try: - meth = self.typecache[typ] - except KeyError: - obj_builtin_base = builtin_base(obj) - if (obj_builtin_base in (object,) + tuple( - self.simple_const_types.keys()) and - typ is not obj_builtin_base): - # assume it's a user defined thingy - meth = self.nameof_instance - else: - for cls in typ.__mro__: - meth = getattr(self, 'nameof_' + ''.join( - [ c for c in cls.__name__ - if c.isalpha() or c == '_'] ), None) - if meth: - break - else: - raise Exception, "nameof(%r)" % (obj,) - self.typecache[typ] = meth - name = meth(obj) - except AlreadyCreated: - name = self.picklenames[id(obj)] - return name - except Exception, e: - self.problem = e, obj - raise - - def nameofargs(self, tup, plain_tuple = False): - """ a string with the nameofs, concatenated """ - # see if we can build a compact representation - ret = ', '.join([self.nameof(arg) for arg in tup]) - if plain_tuple and len(tup) == 1: - ret += ',' - if len(ret) <= 90: - return ret - ret = '\n ' + ',\n '.join( - [self.nameof(arg) for arg in tup]) + ',\n ' - return ret - - def nameof_object(self, value): - if type(value) is not object: - raise Exception, "nameof(%r): type %s not object" % ( - value, type(value).__name__) - name = self.memoize_unique(value, 'g_object') - self.produce('%s = object()' % name) - return name - - def nameof_module(self, value): - # all allowed here, we reproduce ourselves - if self.is_app_domain(value.__name__): - name = self.memoize_unique(value, 'gmod_%s' % value.__name__) - self.produce('%s = new.module(%r)\n' - 'sys.modules[%r] = %s'% ( - name, value.__name__, value.__name__, name) ) - def initmodule(): - names = value.__dict__.keys() - names.sort() - for k in names: - try: - v = value.__dict__[k] - nv = self.nameof(v) - yield '%s.%s = %s' % (name, k, nv) - except PicklingError: - pass - for line in initmodule(): - self.produce(line) - else: - name = self.memoize_unique(value, value.__name__) - self.produce('%s = __import__(%r)' % (name, value.__name__,)) - return name - - def skipped_function(self, func, reason=None, _dummydict={}): - # Generates a placeholder for missing functions - # that raises an exception when called. - # The original code object is retained in an - # attribute '_skipped_code' - skipname = 'gskippedfunc_' + func.__name__ - funcname = func.__name__ - # need to handle this specially - if id(func) in self.picklenames: - raise AlreadyCreated - # generate code object before the skipped func (reads better) - func_code = getattr(func, 'func_code', None) # maybe builtin - self.nameof(func_code) - if reason: - text = 'skipped: %r, see _skipped_code attr: %s' % ( - reason, funcname) - else: - text = 'skipped, see _skipped_code attr: %s' % funcname - def dummy(*args, **kwds): - raise NotImplementedError, text - _dummydict['__builtins__'] = __builtins__ - skippedfunc = new.function(dummy.func_code, _dummydict, skipname, (), - dummy.func_closure) - skippedfunc._skipped_code = func_code - name = self.nameof(skippedfunc) - return self.memoize(func, name) - - def nameof_staticmethod(self, sm): - # XXX XXX XXXX - func = sm.__get__(42.5) - functionname = self.nameof(func) - name = self.memoize_unique(sm, 'gsm_' + func.__name__) - self.produce('%s = staticmethod(%s)' % (name, functionname)) - return name - - def nameof_instancemethod(self, meth): - func = self.nameof(meth.im_func) - typ = self.nameof(meth.im_class) - if meth.im_self is None: - # no error checking here - name = self.memoize_unique(meth, 'gmeth_' + func) - self.produce('%s = %s.%s' % (name, typ, meth.__name__)) - else: - ob = self.nameof(meth.im_self) - name = self.memoize_unique(meth, 'gumeth_'+ func) - self.produce('%s = new.instancemethod(%s, %s, %s)' % ( - name, func, ob, typ)) - return name +""" +Generate a Python source file from the flowmodel. +The purpose is to create something that allows +to restart code generation after flowing and maybe +annotation. + +The generated source appeared to be way too large +for the CPython compiler. Therefore, we cut the +source into pieces and compile them seperately. +""" +from __future__ import generators, division, nested_scopes +import __future__ +all_feature_names = __future__.all_feature_names +import os, sys, new, __builtin__ + +from pypy.translator.gensupp import uniquemodulename, NameManager +from pypy.translator.gensupp import builtin_base +from pypy.rpython.rarithmetic import r_int, r_uint +from pypy.objspace.flow.model import Variable, Constant, SpaceOperation +from pypy.objspace.flow.model import FunctionGraph, Block, Link, Atom +from pypy.objspace.flow.flowcontext import SpamBlock, EggBlock +from pypy.annotation.model import SomeInteger, SomeObject, SomeChar, SomeBool +from pypy.annotation.model import SomeList, SomeString, SomeTuple +from pypy.annotation.unaryop import SomeInstance +from pypy.interpreter.baseobjspace import ObjSpace +from pypy.translator.pickle import slotted + +from pickle import whichmodule, PicklingError +from copy_reg import _reconstructor + +import pickle + +from types import * +import types, weakref + +class AlreadyCreated(Exception): pass + +# ____________________________________________________________ + + +#XXX Hack: This float is supposed to overflow to inf +OVERFLOWED_FLOAT = float("1e10000000000000000000000000000000") +#XXX Hack: and this one to underflow to -inf +OVERFLOWED_FLOATM = float("-1e10000000000000000000000000000000") + +class GenPickle: + + def __init__(self, translator, writer = None): + self.translator = translator + self.writer = writer + self.initcode = [] + self.produce = self._produce() + self.produce( + 'from __future__ import %s\n' % ', '.join(all_feature_names) + + 'import new, types, sys', + ) + self.picklenames = {} # memoize objects + for name in all_feature_names + "new types sys".split(): + self.memoize(globals()[name], name) + self.memoize((), '()') + self.namespace = NameManager() + self.uniquename = self.namespace.uniquename + self.namespace.make_reserved_names('None False True') + self.namespace.make_reserved_names('new types sys') + self.namespace.make_reserved_names(' '.join(all_feature_names)) + self.namespace.make_reserved_names('result') # result dict + self.result = {} + self.simple_const_types = { + int: repr, + long: repr, + float: self.save_float, + str: repr, + unicode: repr, + type(None): repr, + bool: repr, + } + self.typecache = {} # hold types vs. nameof methods + # we distinguish between the "user program" and other stuff. + # "user program" will never use save_global. + self.domains = ( + 'pypy.objspace.std.', + 'pypy.objspace.descroperation', + 'pypy._cache.', + 'pypy.interpreter.', + 'pypy.module.', + 'pypy.translator.test.', + '__main__', + ) + self.shortnames = { + Variable: 'V', + Constant: 'C', + Block: 'B', + SpamBlock: 'SB', + EggBlock: 'EB', + Link: 'L', + FunctionGraph: 'FG', + SomeInteger: 'sI', + SomeObject: 'sO', + SomeChar: 'sC', + SomeBool: 'sB', + SomeList: 'sL', + SomeString: 'sS', + SomeTuple: 'sT', + SomeInstance: 'sIn', + } + self.inline_instances = { + SpaceOperation: True, + } + + def save_float(self, fl): + if fl == OVERFLOWED_FLOAT: + return 'float("1e10000000000000000000000000000000")' + elif fl == OVERFLOWED_FLOATM: + return 'float("-1e10000000000000000000000000000000")' + return repr(fl) + + def pickle(self, *args, **kwds): + for obj in args: + self.nameof(obj) + for obj in kwds.values(): + self.nameof(obj) + self.result.update(kwds) + + def finish(self): + self.pickle(self.result) + self.produce('result = %s' % self.nameof(self.result)) + if self.writer: + self.writer.close() + + def memoize(self, obj, name): + self.picklenames[id(obj)] = name + return name + + def memoize_unique(self, obj, basename): + if id(obj) in self.picklenames: + raise AlreadyCreated + return self.memoize(obj, self.uniquename(basename)) + + def _produce(self): + writer = self.writer + down = 1234 + cnt = [0, 0] # text, calls + self.last_progress = '' + if writer: + write = writer.write + else: + write = self.initcode.append + def produce(text): + write(text+'\n') + cnt[0] += len(text) + 1 + cnt[1] += 1 + if cnt[1] == down: + cnt[1] = 0 + self.progress("%d" % cnt[0]) + return produce + + def progress(self, txt): + back = '\x08' * len(self.last_progress) + self.last_progress = txt+' ' # soft space + print back+txt, + + def nameof(self, obj): + try: + try: + return self.picklenames[id(obj)] + except KeyError: + typ = type(obj) + return self.simple_const_types[typ](obj) + except KeyError: + try: + try: + meth = self.typecache[typ] + except KeyError: + obj_builtin_base = builtin_base(obj) + if (obj_builtin_base in (object,) + tuple( + self.simple_const_types.keys()) and + typ is not obj_builtin_base): + # assume it's a user defined thingy + meth = self.nameof_instance + else: + for cls in typ.__mro__: + meth = getattr(self, 'nameof_' + ''.join( + [ c for c in cls.__name__ + if c.isalpha() or c == '_'] ), None) + if meth: + break + else: + raise Exception, "nameof(%r)" % (obj,) + self.typecache[typ] = meth + name = meth(obj) + except AlreadyCreated: + name = self.picklenames[id(obj)] + return name + except Exception, e: + self.problem = e, obj + raise + + def nameofargs(self, tup, plain_tuple = False): + """ a string with the nameofs, concatenated """ + # see if we can build a compact representation + ret = ', '.join([self.nameof(arg) for arg in tup]) + if plain_tuple and len(tup) == 1: + ret += ',' + if len(ret) <= 90: + return ret + ret = '\n ' + ',\n '.join( + [self.nameof(arg) for arg in tup]) + ',\n ' + return ret + + def nameof_object(self, value): + if type(value) is not object: + raise Exception, "nameof(%r): type %s not object" % ( + value, type(value).__name__) + name = self.memoize_unique(value, 'g_object') + self.produce('%s = object()' % name) + return name + + def nameof_module(self, value): + # all allowed here, we reproduce ourselves + if self.is_app_domain(value.__name__): + name = self.memoize_unique(value, 'gmod_%s' % value.__name__) + self.produce('%s = new.module(%r)\n' + 'sys.modules[%r] = %s'% ( + name, value.__name__, value.__name__, name) ) + def initmodule(): + names = value.__dict__.keys() + names.sort() + for k in names: + try: + v = value.__dict__[k] + nv = self.nameof(v) + yield '%s.%s = %s' % (name, k, nv) + except PicklingError: + pass + for line in initmodule(): + self.produce(line) + else: + name = self.memoize_unique(value, value.__name__) + self.produce('%s = __import__(%r)' % (name, value.__name__,)) + return name + + def skipped_function(self, func, reason=None, _dummydict={}): + # Generates a placeholder for missing functions + # that raises an exception when called. + # The original code object is retained in an + # attribute '_skipped_code' + skipname = 'gskippedfunc_' + func.__name__ + funcname = func.__name__ + # need to handle this specially + if id(func) in self.picklenames: + raise AlreadyCreated + # generate code object before the skipped func (reads better) + func_code = getattr(func, 'func_code', None) # maybe builtin + self.nameof(func_code) + if reason: + text = 'skipped: %r, see _skipped_code attr: %s' % ( + reason, funcname) + else: + text = 'skipped, see _skipped_code attr: %s' % funcname + def dummy(*args, **kwds): + raise NotImplementedError, text + _dummydict['__builtins__'] = __builtins__ + skippedfunc = new.function(dummy.func_code, _dummydict, skipname, (), + dummy.func_closure) + skippedfunc._skipped_code = func_code + name = self.nameof(skippedfunc) + return self.memoize(func, name) + + def nameof_staticmethod(self, sm): + # XXX XXX XXXX + func = sm.__get__(42.5) + functionname = self.nameof(func) + name = self.memoize_unique(sm, 'gsm_' + func.__name__) + self.produce('%s = staticmethod(%s)' % (name, functionname)) + return name + + def nameof_instancemethod(self, meth): + func = self.nameof(meth.im_func) + typ = self.nameof(meth.im_class) + if meth.im_self is None: + # no error checking here + name = self.memoize_unique(meth, 'gmeth_' + func) + self.produce('%s = %s.%s' % (name, typ, meth.__name__)) + else: + ob = self.nameof(meth.im_self) + name = self.memoize_unique(meth, 'gumeth_'+ func) + self.produce('%s = new.instancemethod(%s, %s, %s)' % ( + name, func, ob, typ)) + return name nameof_method = nameof_instancemethod # when run on top of PyPy - - def should_translate_attr(self, pbc, attr): - ann = self.translator.annotator - if ann: - classdef = ann.getuserclasses().get(pbc.__class__) - else: - classdef = None - ignore = getattr(pbc.__class__, 'NOT_RPYTHON_ATTRIBUTES', []) - if attr in ignore: - return False - if classdef: - return classdef.about_attribute(attr) is not None - # by default, render if we don't know anything - return True - - def nameof_builtin_function_or_method(self, func): - if func.__self__ is None: - # builtin function - # where does it come from? Python2.2 doesn't have func.__module__ - for modname, module in sys.modules.items(): - # here we don't ignore extension modules, but it must be - # a builtin module - if not module: continue - if hasattr(module, '__file__'): - fname = module.__file__.lower() - pyendings = '.py', '.pyc', '.pyo' - if [fname.endswith(ending) for ending in pyendings]: - continue - if func is getattr(module, func.__name__, None): - break - else: - #raise Exception, '%r not found in any built-in module' % (func,) - return self.skipped_function( - func, 'not found in any built-in module') - name = self.memoize_unique(func, 'gbltin_' + func.__name__) - if modname == '__builtin__': - self.produce('%s = %s' % (name, func.__name__)) - else: - modname = self.nameof(module) - self.produce('%s = %s.%s' % (name, modname, func.__name__)) - else: - # builtin (bound) method - selfname = self.nameof(func.__self__) - name = self.memoize_unique(func, 'gbltinmethod_' + func.__name__) - self.produce('%s = %s.%s' % (name, selfname, func.__name__)) - return name - - def nameof_classobj(self, cls): - if cls.__doc__ and cls.__doc__.lstrip().startswith('NOT_RPYTHON'): - raise PicklingError, "%r should never be reached" % (cls,) - - try: - return self.save_global(cls) - except PicklingError, e: - pass - - metaclass = "type" - if issubclass(cls, Exception): - # if cls.__module__ == 'exceptions': - # don't rely on this, py.magic redefines AssertionError - if getattr(__builtin__, cls.__name__, None) is cls: - name = self.memoize_unique(cls, 'gexc_' + cls.__name__) - self.produce('%s = %s' % (name, cls.__name__)) - return name - if not isinstance(cls, type): - assert type(cls) is ClassType - metaclass = "types.ClassType" - - basenames = [self.nameof(base) for base in cls.__bases__] - def initclassobj(): - content = cls.__dict__.items() - content.sort() - ignore = getattr(cls, 'NOT_RPYTHON_ATTRIBUTES', []) - isapp = self.is_app_domain(cls.__module__) - for key, value in content: - if key.startswith('__'): - if key in ['__module__', '__doc__', '__dict__', '__slots__', - '__weakref__', '__repr__', '__metaclass__']: - continue - # XXX some __NAMES__ are important... nicer solution sought - #raise Exception, "unexpected name %r in class %s"%(key, cls) - if isapp: - if (isinstance(value, staticmethod) and value.__get__(1) not in - self.translator.flowgraphs and self.translator.frozen): - continue - if isinstance(value, classmethod): - doc = value.__get__(cls).__doc__ - if doc and doc.lstrip().startswith("NOT_RPYTHON"): - continue - if (isinstance(value, FunctionType) and value not in - self.translator.flowgraphs and self.translator.frozen): - continue - if key in ignore: - continue - if type(value) in self.descriptor_filter: - continue # this gets computed - - yield '%s.%s = %s' % (name, key, self.nameof(value)) - - baseargs = ", ".join(basenames) - if baseargs: - baseargs = '(%s)' % baseargs - name = self.memoize_unique(cls, 'gcls_' + cls.__name__) - ini = 'class %s%s:\n __metaclass__ = %s' % (name, baseargs, metaclass) - if '__slots__' in cls.__dict__: - ini += '\n __slots__ = %r' % cls.__slots__ - self.produce(ini) - self.produce('%s.__name__ = %r' % (name, cls.__name__)) - self.produce('%s.__module__ = %r' % (name, cls.__module__)) - for line in initclassobj(): - self.produce(line) - return name - - nameof_class = nameof_classobj # for Python 2.2 - - typename_mapping = { - InstanceType: 'types.InstanceType', - type(None): 'type(None)', - CodeType: 'types.CodeType', - type(sys): 'type(new)', - - r_int: 'r_int', - r_uint: 'r_uint', - - # XXX more hacks - # type 'builtin_function_or_method': - type(len): 'type(len)', - # type 'method_descriptor': - type(type.__reduce__): 'type(type.__reduce__)', - # type 'wrapper_descriptor': - type(type(None).__repr__): 'type(type(None).__repr__)', - # type 'getset_descriptor': - type(type.__dict__['__dict__']): "type(type.__dict__['__dict__'])", - # type 'member_descriptor': - type(type.__dict__['__basicsize__']): "type(type.__dict__['__basicsize__'])", - # type 'instancemethod': - type(Exception().__init__): 'type(Exception().__init__)', - # type 'listiterator': - type(iter([])): 'type(iter([]))', - } - descriptor_filter = {} - for _key in typename_mapping.keys(): - if _key.__name__.endswith('descriptor'): - descriptor_filter[_key] = True - del _key - - def nameof_type(self, cls): - if cls.__module__ != '__builtin__': - return self.nameof_classobj(cls) # user-defined type - name = self.memoize_unique(cls, 'gtype_%s' % cls.__name__) - if getattr(__builtin__, cls.__name__, None) is cls: - expr = cls.__name__ # type available from __builtin__ - elif cls in types.__dict__.values(): - for key, value in types.__dict__.items(): - if value is cls: - break - self.produce('from types import %s as %s' % ( - key, name)) - return name - elif cls in weakref.__dict__.values(): - for key, value in weakref.__dict__.items(): - if value is cls: - break - self.produce('from weakref import %s as %s' % ( - key, name)) - return name - else: - expr = self.typename_mapping[cls] - self.produce('%s = %s' % (name, expr)) - return name - - def nameof_tuple(self, tup): - chunk = 20 - # first create all arguments - for i in range(0, len(tup), chunk): - self.nameofargs(tup[i:i+chunk], True) - # see if someone else created us meanwhile - name = self.memoize_unique(tup, 'T%d' % len(tup)) - argstr = self.nameofargs(tup[:chunk], True) - self.produce('%s = (%s)' % (name, argstr)) - for i in range(chunk, len(tup), chunk): - argstr = self.nameofargs(tup[i:i+chunk], True) - self.produce('%s += (%s)' % (name, argstr) ) - return name - - def nameof_list(self, lis): - chunk = 20 - def initlist(): - for i in range(0, len(lis), chunk): - argstr = self.nameofargs(lis[i:i+chunk]) - yield '%s += [%s]' % (name, argstr) - name = self.memoize_unique(lis, 'L%d' % len(lis)) - self.produce('%s = []' % name) - for line in initlist(): - self.produce(line) - return name - - def is_app_domain(self, modname, exclude=()): - for domain in self.domains: - if domain.endswith('.') and modname.startswith(domain): - # handle subpaths - return True - if modname == domain and modname not in exclude: - # handle exact module names - return True - return False - - def nameof_dict(self, dic): - if '__name__' in dic: - module = dic['__name__'] - try: - if type(module) is str and self.is_app_domain(module): - raise ImportError - __import__(module) - mod = sys.modules[module] - except (ImportError, KeyError, TypeError): - pass - else: - if dic is mod.__dict__ and not self.is_app_domain(module): - dictname = module.split('.')[-1] + '__dict__' - dictname = self.memoize_unique(dic, dictname) - self.produce('from %s import __dict__ as %s' % ( - module, dictname) ) - return dictname - def initdict(): - keys = dic.keys() - keys.sort() - told = False - for k in keys: - try: - nk, nv = self.nameof(k), self.nameof(dic[k]) - yield '%s[%s] = %s' % (name, nk, nv) - except PicklingError: - pass - else: - # some sanity check - if type(k) is int: - if k in self.picklenames: - print ('WARNING: this dict most likely contains ' - 'the id of some object!!') - print 'name of object: %s' % self.picklenames[k] - elif k == id(dic[k]): - print ('WARNING: this dict most likely contains ' - 'the id of one of it\'s objects!!') - if not told: - print dic - told = True - name = self.memoize_unique(dic, 'D%d' % len(dic)) - self.produce('%s = {}' % name) - for line in initdict(): - self.produce(line) - return name - - # strange prebuilt instances below, don't look too closely - # XXX oh well. - def nameof_member_descriptor(self, md): - cls = self.nameof(md.__objclass__) - name = self.memoize_unique(md, 'gdescriptor_%s_%s' % ( - md.__objclass__.__name__, md.__name__)) - self.produce('%s = %s.__dict__[%r]' % (name, cls, md.__name__)) - return name - nameof_getset_descriptor = nameof_member_descriptor - nameof_method_descriptor = nameof_member_descriptor - nameof_wrapper_descriptor = nameof_member_descriptor - - def nameof_instance(self, instance): - if isinstance(instance, Atom): - # cannot reconstruct this, it *must* be - # the one from model - return self.save_global(instance) - def initinstance(): - if hasattr(instance, '__setstate__'): - # the instance knows what to do - args = self.nameof(restorestate) - yield '%s.__setstate__(%s)' % (name, args) - return - elif type(restorestate) is tuple: - setstate = self.nameof(_set) - argstr = self.nameofargs(restorestate) - yield '%s(%s, %s)' % (setstate, name, argstr) - return - assert type(restorestate) is dict, ( - "%s has no dict and no __setstate__" % name) - content = restorestate.items() - content.sort() - attrs = [] - for key, value in content: - if self.should_translate_attr(instance, key): - if hasattr(value, '__doc__'): - doc = value.__doc__ - if type(doc) is str and doc.lstrip().startswith('NOT_RPYTHON'): - continue - attrs.append( (key, self.nameof(value)) ) - for k, v in attrs: - yield '%s.%s = %s' % (name, k, v) - - klass = instance.__class__ - cls = self.nameof(klass) - if hasattr(klass, '__base__'): - base_class = builtin_base(instance) - base = self.nameof(base_class) - else: - base_class = None - base = cls - if klass in self.inline_instances: - immediate = True - else: - if klass in self.shortnames: - name = self.memoize_unique(instance, self.shortnames[klass]) - else: - name = self.memoize_unique(instance, 'ginst_' + klass.__name__) - immediate = False - if hasattr(instance, '__reduce_ex__'): - try: - reduced = instance.__reduce_ex__() - except TypeError: - # oops! slots and no __getstate__? - if not (hasattr(instance, '__slots__') - and not hasattr(instance, '__getstate__') ): - print "PROBLEM:", instance - raise - assert not hasattr(instance, '__dict__'), ('wrong assumptions' - ' about __slots__ in %s instance without __setstate__,' - ' please update %s' % (klass.__name__, __name__) ) - restorestate = _get(instance) - restorer = _rec - restoreargs = klass, - else: - restorer = reduced[0] - restoreargs = reduced[1] - if restorer is _reconstructor: - restorer = _rec - if restoreargs[1:] == (object, None): - restoreargs = restoreargs[:1] - if len(reduced) > 2: - restorestate = reduced[2] - else: - restorestate = None - restorename = self.nameof(restorer) - # ignore possible dict, handled later by initinstance filtering - # in other cases, we expect that the class knows what to pickle. - else: - restoreargs = (base, cls) - restorename = '%s.__new__' % base - if hasattr(instance, '__getstate__'): - restorestate = instance.__getstate__() - else: - restorestate = instance.__dict__ - restoreargstr = self.nameofargs(restoreargs) - if immediate: - assert restorestate is None - return '%s(%s)' % (restorename, restoreargstr) - if isinstance(klass, type): - self.produce('%s = %s(%s)' % (name, restorename, restoreargstr)) - else: - self.produce('%s = new.instance(%s)' % (name, cls)) - if restorestate is not None: - for line in initinstance(): - self.produce(line) - return name - - def save_global(self, obj): - # this is almost similar to pickle.py - name = obj.__name__ - module = getattr(obj, "__module__", None) - if module is None: - module = whichmodule(obj, name) - if self.is_app_domain(module): - # not allowed to import this - raise PicklingError('%s belongs to the user program' % - name) - try: - __import__(module) - mod = sys.modules[module] - klass = getattr(mod, name) - except (ImportError, KeyError, AttributeError): - raise PicklingError( - "Can't pickle %r: it's not found as %s.%s" % - (obj, module, name)) - else: - if klass is not obj: - raise PicklingError( - "Can't pickle %r: it's not the same object as %s.%s" % - (obj, module, name)) - # from here we do our own stuff - restorename = self.memoize_unique(obj, obj.__name__) - if restorename != obj.__name__: - self.produce('from %s import %s as %s' % ( - module, obj.__name__, restorename) ) - else: - self.produce('from %s import %s' % ( - module, obj.__name__) ) - return restorename - - def nameof_function(self, func): - # look for skipped functions - if self.translator.frozen: - if func not in self.translator.flowgraphs: - # see if this is in translator's domain - module = whichmodule(func, func.__name__) - if self.is_app_domain(module, exclude=['__main__']): - # see if this buddy has been skipped in another save, before - if not hasattr(func, '_skipped_code'): - return self.skipped_function(func, - 'not found in translator\'s flowgraphs') - else: - if (func.func_doc and - func.func_doc.lstrip().startswith('NOT_RPYTHON')): - return self.skipped_function(func, 'tagged as NOT_RPYTHON') - try: - return self.save_global(func) - except PicklingError: - pass - args = (func.func_code, func.func_globals, func.func_name, - func.func_defaults, func.func_closure) - argstr = self.nameofargs(args) - if hasattr(func, '_skipped_code'): - name = self.memoize_unique(func, func.__name__) - else: - name = self.memoize_unique(func, 'gfunc_' + func.__name__) - self.produce('%s = new.function(%s)' % (name, argstr) ) - if func.__dict__: - def initfunction(): - items = func.__dict__.items() - items.sort() - for k, v in items: - try: - yield '%s.%s = %s' % (name, k, self.nameof(v)) - except PicklingError: - pass - for line in initfunction(): - self.produce(line) - return name - - def nameof_cell(self, cel): - # no need to name cells. Their contents is what is shared. - obj = break_cell(cel) - return '%s(%s)' % (self.nameof(make_cell), self.nameof(obj)) - - def nameof_property(self, prop): - argstr = self.nameofargs((prop.fget, prop.fset, prop.fdel, - prop.__doc__)) - name = self.memoize_unique(prop, 'gprop_') - self.produce('%s = property(%s)' % (name, argstr) ) - return name - - def nameof_code(self, code): - args = (code.co_argcount, code.co_nlocals, code.co_stacksize, - code.co_flags, code.co_code, code.co_consts, code.co_names, - code.co_varnames, code.co_filename, code.co_name, - code.co_firstlineno, code.co_lnotab, code.co_freevars, - code.co_cellvars) - argstr = self.nameofargs(args) - name = self.memoize_unique(code, 'gcode_' + code.co_name) - self.produce('%s = new.code(%s)' % (name, argstr)) - return name - - def nameof_file(self, fil): - if fil is sys.stdin: return "sys.stdin" - if fil is sys.stdout: return "sys.stdout" - if fil is sys.stderr: return "sys.stderr" - raise Exception, 'Cannot translate an already-open file: %r' % (fil,) - - def nameof_methodwrapper(self, wp): - # this object should be enhanced in CPython! - msg = '%r: method %s of unknown object cannot be reconstructed' % ( - wp, wp.__name__) - return self.skipped_function(wp, msg) - - def nameof_weakref(self, value): - # no need to name weakrefs. Their contents is what is weakref'ed. - # obtain the ref'ed object by calling - obj = value() - return '%s(%s)' % (self.nameof(type(value)), self.nameof(obj)) - - -def make_cell(obj): - def func(): - return obj - return func.func_closure[0] - -def break_cell(cel): - obj = None - def func(): - return obj - args = (func.func_code, func.func_globals, func.func_name, - func.func_defaults, (cel,)) - func = new.function(*args) - return func() - -# some shortcuts, to make the pickle smaller - -def _rec(klass, base=object, state=None): - return _reconstructor(klass, base, state) - -def _get(obj): - return slotted.__getstate__(obj) - -def _set(obj, *args): - slotted.__setstate__(obj, args) - -__all__ = ['GenPickle'] + + def should_translate_attr(self, pbc, attr): + ann = self.translator.annotator + if ann: + classdef = ann.getuserclasses().get(pbc.__class__) + else: + classdef = None + ignore = getattr(pbc.__class__, 'NOT_RPYTHON_ATTRIBUTES', []) + if attr in ignore: + return False + if classdef: + return classdef.about_attribute(attr) is not None + # by default, render if we don't know anything + return True + + def nameof_builtin_function_or_method(self, func): + if func.__self__ is None: + # builtin function + # where does it come from? Python2.2 doesn't have func.__module__ + for modname, module in sys.modules.items(): + # here we don't ignore extension modules, but it must be + # a builtin module + if not module: continue + if hasattr(module, '__file__'): + fname = module.__file__.lower() + pyendings = '.py', '.pyc', '.pyo' + if [fname.endswith(ending) for ending in pyendings]: + continue + if func is getattr(module, func.__name__, None): + break + else: + #raise Exception, '%r not found in any built-in module' % (func,) + return self.skipped_function( + func, 'not found in any built-in module') + name = self.memoize_unique(func, 'gbltin_' + func.__name__) + if modname == '__builtin__': + self.produce('%s = %s' % (name, func.__name__)) + else: + modname = self.nameof(module) + self.produce('%s = %s.%s' % (name, modname, func.__name__)) + else: + # builtin (bound) method + selfname = self.nameof(func.__self__) + name = self.memoize_unique(func, 'gbltinmethod_' + func.__name__) + self.produce('%s = %s.%s' % (name, selfname, func.__name__)) + return name + + def nameof_classobj(self, cls): + if cls.__doc__ and cls.__doc__.lstrip().startswith('NOT_RPYTHON'): + raise PicklingError, "%r should never be reached" % (cls,) + + try: + return self.save_global(cls) + except PicklingError, e: + pass + + metaclass = "type" + if issubclass(cls, Exception): + # if cls.__module__ == 'exceptions': + # don't rely on this, py.magic redefines AssertionError + if getattr(__builtin__, cls.__name__, None) is cls: + name = self.memoize_unique(cls, 'gexc_' + cls.__name__) + self.produce('%s = %s' % (name, cls.__name__)) + return name + if not isinstance(cls, type): + assert type(cls) is ClassType + metaclass = "types.ClassType" + + basenames = [self.nameof(base) for base in cls.__bases__] + def initclassobj(): + content = cls.__dict__.items() + content.sort() + ignore = getattr(cls, 'NOT_RPYTHON_ATTRIBUTES', []) + isapp = self.is_app_domain(cls.__module__) + for key, value in content: + if key.startswith('__'): + if key in ['__module__', '__doc__', '__dict__', '__slots__', + '__weakref__', '__repr__', '__metaclass__']: + continue + # XXX some __NAMES__ are important... nicer solution sought + #raise Exception, "unexpected name %r in class %s"%(key, cls) + if isapp: + if (isinstance(value, staticmethod) and value.__get__(1) not in + self.translator.flowgraphs and self.translator.frozen): + continue + if isinstance(value, classmethod): + doc = value.__get__(cls).__doc__ + if doc and doc.lstrip().startswith("NOT_RPYTHON"): + continue + if (isinstance(value, FunctionType) and value not in + self.translator.flowgraphs and self.translator.frozen): + continue + if key in ignore: + continue + if type(value) in self.descriptor_filter: + continue # this gets computed + + yield '%s.%s = %s' % (name, key, self.nameof(value)) + + baseargs = ", ".join(basenames) + if baseargs: + baseargs = '(%s)' % baseargs + name = self.memoize_unique(cls, 'gcls_' + cls.__name__) + ini = 'class %s%s:\n __metaclass__ = %s' % (name, baseargs, metaclass) + if '__slots__' in cls.__dict__: + ini += '\n __slots__ = %r' % cls.__slots__ + self.produce(ini) + self.produce('%s.__name__ = %r' % (name, cls.__name__)) + self.produce('%s.__module__ = %r' % (name, cls.__module__)) + for line in initclassobj(): + self.produce(line) + return name + + nameof_class = nameof_classobj # for Python 2.2 + + typename_mapping = { + InstanceType: 'types.InstanceType', + type(None): 'type(None)', + CodeType: 'types.CodeType', + type(sys): 'type(new)', + + r_int: 'r_int', + r_uint: 'r_uint', + + # XXX more hacks + # type 'builtin_function_or_method': + type(len): 'type(len)', + # type 'method_descriptor': + type(type.__reduce__): 'type(type.__reduce__)', + # type 'wrapper_descriptor': + type(type(None).__repr__): 'type(type(None).__repr__)', + # type 'getset_descriptor': + type(type.__dict__['__dict__']): "type(type.__dict__['__dict__'])", + # type 'member_descriptor': + type(type.__dict__['__basicsize__']): "type(type.__dict__['__basicsize__'])", + # type 'instancemethod': + type(Exception().__init__): 'type(Exception().__init__)', + # type 'listiterator': + type(iter([])): 'type(iter([]))', + } + descriptor_filter = {} + for _key in typename_mapping.keys(): + if _key.__name__.endswith('descriptor'): + descriptor_filter[_key] = True + del _key + + def nameof_type(self, cls): + if cls.__module__ != '__builtin__': + return self.nameof_classobj(cls) # user-defined type + name = self.memoize_unique(cls, 'gtype_%s' % cls.__name__) + if getattr(__builtin__, cls.__name__, None) is cls: + expr = cls.__name__ # type available from __builtin__ + elif cls in types.__dict__.values(): + for key, value in types.__dict__.items(): + if value is cls: + break + self.produce('from types import %s as %s' % ( + key, name)) + return name + elif cls in weakref.__dict__.values(): + for key, value in weakref.__dict__.items(): + if value is cls: + break + self.produce('from weakref import %s as %s' % ( + key, name)) + return name + else: + expr = self.typename_mapping[cls] + self.produce('%s = %s' % (name, expr)) + return name + + def nameof_tuple(self, tup): + chunk = 20 + # first create all arguments + for i in range(0, len(tup), chunk): + self.nameofargs(tup[i:i+chunk], True) + # see if someone else created us meanwhile + name = self.memoize_unique(tup, 'T%d' % len(tup)) + argstr = self.nameofargs(tup[:chunk], True) + self.produce('%s = (%s)' % (name, argstr)) + for i in range(chunk, len(tup), chunk): + argstr = self.nameofargs(tup[i:i+chunk], True) + self.produce('%s += (%s)' % (name, argstr) ) + return name + + def nameof_list(self, lis): + chunk = 20 + def initlist(): + for i in range(0, len(lis), chunk): + argstr = self.nameofargs(lis[i:i+chunk]) + yield '%s += [%s]' % (name, argstr) + name = self.memoize_unique(lis, 'L%d' % len(lis)) + self.produce('%s = []' % name) + for line in initlist(): + self.produce(line) + return name + + def is_app_domain(self, modname, exclude=()): + for domain in self.domains: + if domain.endswith('.') and modname.startswith(domain): + # handle subpaths + return True + if modname == domain and modname not in exclude: + # handle exact module names + return True + return False + + def nameof_dict(self, dic): + if '__name__' in dic: + module = dic['__name__'] + try: + if type(module) is str and self.is_app_domain(module): + raise ImportError + __import__(module) + mod = sys.modules[module] + except (ImportError, KeyError, TypeError): + pass + else: + if dic is mod.__dict__ and not self.is_app_domain(module): + dictname = module.split('.')[-1] + '__dict__' + dictname = self.memoize_unique(dic, dictname) + self.produce('from %s import __dict__ as %s' % ( + module, dictname) ) + return dictname + def initdict(): + keys = dic.keys() + keys.sort() + told = False + for k in keys: + try: + nk, nv = self.nameof(k), self.nameof(dic[k]) + yield '%s[%s] = %s' % (name, nk, nv) + except PicklingError: + pass + else: + # some sanity check + if type(k) is int: + if k in self.picklenames: + print ('WARNING: this dict most likely contains ' + 'the id of some object!!') + print 'name of object: %s' % self.picklenames[k] + elif k == id(dic[k]): + print ('WARNING: this dict most likely contains ' + 'the id of one of it\'s objects!!') + if not told: + print dic + told = True + name = self.memoize_unique(dic, 'D%d' % len(dic)) + self.produce('%s = {}' % name) + for line in initdict(): + self.produce(line) + return name + + # strange prebuilt instances below, don't look too closely + # XXX oh well. + def nameof_member_descriptor(self, md): + cls = self.nameof(md.__objclass__) + name = self.memoize_unique(md, 'gdescriptor_%s_%s' % ( + md.__objclass__.__name__, md.__name__)) + self.produce('%s = %s.__dict__[%r]' % (name, cls, md.__name__)) + return name + nameof_getset_descriptor = nameof_member_descriptor + nameof_method_descriptor = nameof_member_descriptor + nameof_wrapper_descriptor = nameof_member_descriptor + + def nameof_instance(self, instance): + if isinstance(instance, Atom): + # cannot reconstruct this, it *must* be + # the one from model + return self.save_global(instance) + def initinstance(): + if hasattr(instance, '__setstate__'): + # the instance knows what to do + args = self.nameof(restorestate) + yield '%s.__setstate__(%s)' % (name, args) + return + elif type(restorestate) is tuple: + setstate = self.nameof(_set) + argstr = self.nameofargs(restorestate) + yield '%s(%s, %s)' % (setstate, name, argstr) + return + assert type(restorestate) is dict, ( + "%s has no dict and no __setstate__" % name) + content = restorestate.items() + content.sort() + attrs = [] + for key, value in content: + if self.should_translate_attr(instance, key): + if hasattr(value, '__doc__'): + doc = value.__doc__ + if type(doc) is str and doc.lstrip().startswith('NOT_RPYTHON'): + continue + attrs.append( (key, self.nameof(value)) ) + for k, v in attrs: + yield '%s.%s = %s' % (name, k, v) + + klass = instance.__class__ + cls = self.nameof(klass) + if hasattr(klass, '__base__'): + base_class = builtin_base(instance) + base = self.nameof(base_class) + else: + base_class = None + base = cls + if klass in self.inline_instances: + immediate = True + else: + if klass in self.shortnames: + name = self.memoize_unique(instance, self.shortnames[klass]) + else: + name = self.memoize_unique(instance, 'ginst_' + klass.__name__) + immediate = False + if hasattr(instance, '__reduce_ex__'): + try: + reduced = instance.__reduce_ex__() + except TypeError: + # oops! slots and no __getstate__? + if not (hasattr(instance, '__slots__') + and not hasattr(instance, '__getstate__') ): + print "PROBLEM:", instance + raise + assert not hasattr(instance, '__dict__'), ('wrong assumptions' + ' about __slots__ in %s instance without __setstate__,' + ' please update %s' % (klass.__name__, __name__) ) + restorestate = _get(instance) + restorer = _rec + restoreargs = klass, + else: + restorer = reduced[0] + restoreargs = reduced[1] + if restorer is _reconstructor: + restorer = _rec + if restoreargs[1:] == (object, None): + restoreargs = restoreargs[:1] + if len(reduced) > 2: + restorestate = reduced[2] + else: + restorestate = None + restorename = self.nameof(restorer) + # ignore possible dict, handled later by initinstance filtering + # in other cases, we expect that the class knows what to pickle. + else: + restoreargs = (base, cls) + restorename = '%s.__new__' % base + if hasattr(instance, '__getstate__'): + restorestate = instance.__getstate__() + else: + restorestate = instance.__dict__ + restoreargstr = self.nameofargs(restoreargs) + if immediate: + assert restorestate is None + return '%s(%s)' % (restorename, restoreargstr) + if isinstance(klass, type): + self.produce('%s = %s(%s)' % (name, restorename, restoreargstr)) + else: + self.produce('%s = new.instance(%s)' % (name, cls)) + if restorestate is not None: + for line in initinstance(): + self.produce(line) + return name + + def save_global(self, obj): + # this is almost similar to pickle.py + name = obj.__name__ + module = getattr(obj, "__module__", None) + if module is None: + module = whichmodule(obj, name) + if self.is_app_domain(module): + # not allowed to import this + raise PicklingError('%s belongs to the user program' % + name) + try: + __import__(module) + mod = sys.modules[module] + klass = getattr(mod, name) + except (ImportError, KeyError, AttributeError): + raise PicklingError( + "Can't pickle %r: it's not found as %s.%s" % + (obj, module, name)) + else: + if klass is not obj: + raise PicklingError( + "Can't pickle %r: it's not the same object as %s.%s" % + (obj, module, name)) + # from here we do our own stuff + restorename = self.memoize_unique(obj, obj.__name__) + if restorename != obj.__name__: + self.produce('from %s import %s as %s' % ( + module, obj.__name__, restorename) ) + else: + self.produce('from %s import %s' % ( + module, obj.__name__) ) + return restorename + + def nameof_function(self, func): + # look for skipped functions + if self.translator.frozen: + if func not in self.translator.flowgraphs: + # see if this is in translator's domain + module = whichmodule(func, func.__name__) + if self.is_app_domain(module, exclude=['__main__']): + # see if this buddy has been skipped in another save, before + if not hasattr(func, '_skipped_code'): + return self.skipped_function(func, + 'not found in translator\'s flowgraphs') + else: + if (func.func_doc and + func.func_doc.lstrip().startswith('NOT_RPYTHON')): + return self.skipped_function(func, 'tagged as NOT_RPYTHON') + try: + return self.save_global(func) + except PicklingError: + pass + args = (func.func_code, func.func_globals, func.func_name, + func.func_defaults, func.func_closure) + argstr = self.nameofargs(args) + if hasattr(func, '_skipped_code'): + name = self.memoize_unique(func, func.__name__) + else: + name = self.memoize_unique(func, 'gfunc_' + func.__name__) + self.produce('%s = new.function(%s)' % (name, argstr) ) + if func.__dict__: + def initfunction(): + items = func.__dict__.items() + items.sort() + for k, v in items: + try: + yield '%s.%s = %s' % (name, k, self.nameof(v)) + except PicklingError: + pass + for line in initfunction(): + self.produce(line) + return name + + def nameof_cell(self, cel): + # no need to name cells. Their contents is what is shared. + obj = break_cell(cel) + return '%s(%s)' % (self.nameof(make_cell), self.nameof(obj)) + + def nameof_property(self, prop): + argstr = self.nameofargs((prop.fget, prop.fset, prop.fdel, + prop.__doc__)) + name = self.memoize_unique(prop, 'gprop_') + self.produce('%s = property(%s)' % (name, argstr) ) + return name + + def nameof_code(self, code): + args = (code.co_argcount, code.co_nlocals, code.co_stacksize, + code.co_flags, code.co_code, code.co_consts, code.co_names, + code.co_varnames, code.co_filename, code.co_name, + code.co_firstlineno, code.co_lnotab, code.co_freevars, + code.co_cellvars) + argstr = self.nameofargs(args) + name = self.memoize_unique(code, 'gcode_' + code.co_name) + self.produce('%s = new.code(%s)' % (name, argstr)) + return name + + def nameof_file(self, fil): + if fil is sys.stdin: return "sys.stdin" + if fil is sys.stdout: return "sys.stdout" + if fil is sys.stderr: return "sys.stderr" + raise Exception, 'Cannot translate an already-open file: %r' % (fil,) + + def nameof_methodwrapper(self, wp): + # this object should be enhanced in CPython! + msg = '%r: method %s of unknown object cannot be reconstructed' % ( + wp, wp.__name__) + return self.skipped_function(wp, msg) + + def nameof_weakref(self, value): + # no need to name weakrefs. Their contents is what is weakref'ed. + # obtain the ref'ed object by calling + obj = value() + return '%s(%s)' % (self.nameof(type(value)), self.nameof(obj)) + + +def make_cell(obj): + def func(): + return obj + return func.func_closure[0] + +def break_cell(cel): + obj = None + def func(): + return obj + args = (func.func_code, func.func_globals, func.func_name, + func.func_defaults, (cel,)) + func = new.function(*args) + return func() + +# some shortcuts, to make the pickle smaller + +def _rec(klass, base=object, state=None): + return _reconstructor(klass, base, state) + +def _get(obj): + return slotted.__getstate__(obj) + +def _set(obj, *args): + slotted.__setstate__(obj, args) + +__all__ = ['GenPickle'] Modified: pypy/dist/pypy/translator/test/test_geninterp.py ============================================================================== --- pypy/dist/pypy/translator/test/test_geninterp.py (original) +++ pypy/dist/pypy/translator/test/test_geninterp.py Tue Jan 24 13:31:23 2006 @@ -1,273 +1,273 @@ -""" -Description -_____________________________ - -This test is almost a copy of test_genc.py -The setup code is slightly different: -Instead of compiling single functions from -snippets.py, almost all of snippets is translated, -up to the point where they are untranslatable. -snippets has been slightly re-ordered for that. - -The idea was to create a couple of tests without much -extra work, in a sense derived from the test_genc. - -A problem with that is, that the tests actually should -be run at application level. The test code checks real -Python values,so we have to do tricks to unwrap things. -This is limited: -Some tests cannot work, since they mutate their arguments. -Some tests operate with un-unwrappable things. -Those are disabled for the moment by an 'needapp_' prefix. - -XXX think about a way to produce more tests from a common -XXX basis. Should we write generators for such tests like this? -""" -import autopath -import py -from pypy.tool.udir import udir -from pypy.objspace.flow.model import * -from pypy.translator.tool.cbuild import make_module_from_c -from pypy.translator.tool.cbuild import skip_missing_compiler -from pypy.translator.geninterplevel import translate_as_module -from pypy.translator.test import snippet -from pypy.interpreter.error import OperationError -from py.code import Source - -class TestGenRpyTestCase: - objspacename = 'std' - - snippet_ad = """if 1: - def import_func(): - import copy_reg - return copy_reg._reconstructor.func_code.co_name - - def import_sys_func(): - import sys - return sys.__name__ -""" - - def setup_class(cls): - # simply compile snippets just once - src = str(Source(snippet)) - # truncate non-compilable stuff for now: - p = src.index('Non compilable Functions') - src = src[:p] + '\n' - # put our ad into snippet - exec cls.snippet_ad in snippet.__dict__ - src += cls.snippet_ad - # just in case of trouble, we produce a tempfile - ini, newsrc = translate_as_module(src, tmpname = str( - udir.join("_geninterp_test.py"))) - cls.w_glob = ini(cls.space) - - def build_interpfunc(self, func, *morefuncs): - # we ignore morefuncs, since they live in snippets - space = self.space - func = space.getitem(self.w_glob, space.wrap(func.__name__)) - def wrapunwrap(*args): - w_args = space.wrap(args) - try: - w_res = space.call(func, w_args) - except OperationError, e: - w_typ = e.w_type - # XXX how to unwrap an exception? - name = space.unwrap(space.getattr(w_typ, space.wrap('__name__'))) - exc = __builtins__[name] - raise exc - return space.unwrap(w_res) - return wrapunwrap - - # small addition to see whether imports look fine - def test_import(self): - import copy_reg - impfunc = self.build_interpfunc(snippet.import_func) - assert impfunc() == '_reconstructor' - - def test_import_sys(self): - impfunc = self.build_interpfunc(snippet.import_sys_func) - assert impfunc() == 'sys' - - def test_simple_func(self): - cfunc = self.build_interpfunc(snippet.simple_func) - assert cfunc(1) == 2 - - def test_while_func(self): - while_func = self.build_interpfunc(snippet.while_func) - assert while_func(10) == 55 - - def test_nested_whiles(self): - nested_whiles = self.build_interpfunc(snippet.nested_whiles) - assert nested_whiles(111, 114) == ( - '...!...!...!...!...!') - - def test_poor_man_range(self): - poor_man_range = self.build_interpfunc(snippet.poor_man_range) - assert poor_man_range(10) == range(10) - - def poor_man_rev_range(self): - poor_man_rev_range = self.build_interpfunc(snippet.poor_man_rev_range) - assert poor_man_rev_range(10) == range(9,-1,-1) - - def test_simple_id(self): - #we just want to see, if renaming of parameter works correctly - #if the first branch is the end branch - simple_id = self.build_interpfunc(snippet.simple_id) - assert simple_id(9) == 9 - - def test_branch_id(self): - branch_id = self.build_interpfunc(snippet.branch_id) - assert branch_id(1, 2, 3) == 2 - assert branch_id(0, 2, 3) == 3 - - def test_int_id(self): - int_id = self.build_interpfunc(snippet.int_id) - assert int_id(3) == 3 - - def dont_test_attrs(self): - attrs = self.build_interpfunc(snippet.attrs) - assert attrs() == 9 - - def test_builtinusage(self): - fun = self.build_interpfunc(snippet.builtinusage) - assert fun() == 4 - - def xpensive_test_sieve(self): - sieve = self.build_interpfunc(snippet.sieve_of_eratosthenes) - assert sieve() == 1028 - - def test_slice(self): - half = self.build_interpfunc(snippet.half_of_n) - assert half(10) == 5 - - def test_poly_branch(self): - poly_branch = self.build_interpfunc(snippet.poly_branch) - assert poly_branch(10) == [1,2,3]*2 - assert poly_branch(0) == ['a','b','c']*2 - - def test_and(self): - sand = self.build_interpfunc(snippet.s_and) - assert sand(5, 6) == "yes" - assert sand(5, 0) == "no" - assert sand(0, 6) == "no" - assert sand(0, 0) == "no" - - def test_yast(self): - yast = self.build_interpfunc(snippet.yast) - assert yast([1000,100,10,1]) == 1111 - assert yast(range(100)) == (99*100)/2 - - def test_with_init(self): - with_init = self.build_interpfunc(snippet.with_init) - assert with_init(0) == 0 - assert with_init(-100) == -100 - - def test_with_more_init(self): - with_more_init = self.build_interpfunc(snippet.with_more_init) - assert with_more_init(10, False) == -10 - assert with_more_init(20, True) == 20 - - def needapp_test_global_instance(self): - global_instance = self.build_interpfunc(snippet.global_instance) - assert global_instance() == 42 - - def needapp_test_global_newstyle_instance(self): - global_newstyle_instance = self.build_interpfunc(snippet.global_newstyle_instance) - assert global_newstyle_instance().a == 1 - - def needapp_test_global_recursive_list(self): - global_recursive_list = self.build_interpfunc(snippet.global_recursive_list) - lst = global_recursive_list() - assert len(lst) == 1 - assert lst[0] is lst - -## def test_global_badinit(self): -## global_badinit = self.build_interpfunc(snippet.global_badinit) -## self.assertEquals(global_badinit(), 1) - - def test_multiple_inheritance(self): - multiple_inheritance = self.build_interpfunc(snippet.multiple_inheritance) - assert multiple_inheritance() == 1+2+3+4 - - def test_call_star_args(self): - call_star_args = self.build_interpfunc(snippet.call_star_args) - assert call_star_args(42) == 52 - - def test_call_default_args(self): - call_default_args = self.build_interpfunc(snippet.call_default_args) - assert call_default_args(42) == 111+42+3 - - def test_call_default_and_star_args(self): - call_default_and_star_args = self.build_interpfunc( - snippet.call_default_and_star_args) - assert call_default_and_star_args(42) == ( - (111+42+3+0, -1000-2000-3000+2)) - - def test_call_with_star(self): - call_with_star = self.build_interpfunc(snippet.call_with_star) - assert call_with_star(()) == -15L - assert call_with_star((4,)) == -13L - assert call_with_star((4,7)) == -9L - assert call_with_star([]) == -15L - assert call_with_star([4]) == -13L - assert call_with_star([4,7]) == -9L - raises(TypeError, call_with_star, (4,7,12)) - raises(TypeError, call_with_star, [4,7,12,63]) - raises(TypeError, call_with_star, 521) - - def test_call_with_keyword(self): - call_with_keyword = self.build_interpfunc(snippet.call_with_keyword) - assert call_with_keyword(100) == 82 - - def test_call_very_complex(self): - call_very_complex = self.build_interpfunc(snippet.call_very_complex, - snippet.default_args) - assert call_very_complex(5, (3,), {}) == -12 - assert call_very_complex(5, (), {'y': 3}) == -12 - raises(TypeError, call_very_complex, 5, (3,), {'y': 4}) - - def test_finallys(self): - finallys = self.build_interpfunc(snippet.finallys) - assert finallys(['hello']) == 8 - assert finallys('X') == 8 - assert finallys([]) == 6 - assert finallys('XY') == 6 - - def needapp_test_finally2(self): - finally2 = self.build_interpfunc(snippet.finally2) - lst = range(10) - finally2(lst, 5) - assert lst == [0,1,2,3,4, 6, 6,7,8, 'done'] - dic = {} - raises(KeyError, finally2, dic, "won't find this key") - assert dic == {-1: 'done'} - - def test_bare_raise(self): - bare_raise = self.build_interpfunc(snippet.bare_raise) - assert bare_raise(range(0, 100, 10), False) == 50 - assert bare_raise(range(0, 100, 10), True) == 50 - raises(IndexError, bare_raise, range(0, 30, 10), False) - assert bare_raise(range(0, 30, 10), True) == None - - def needapp_test_get_set_del_slice(self): - fn = self.build_interpfunc(snippet.get_set_del_slice) - l = list('abcdefghij') - result = fn(l) - assert l == [3, 'c', 8, 11, 'h', 9] - assert result == ([3, 'c'], [9], [11, 'h']) - - def test_do_try_raise_choose(self): - fn = self.build_interpfunc(snippet.do_try_raise_choose) - result = fn() - assert result == [-1,0,1,2] - - - def test_t_isinstance(self): - fn = self.build_interpfunc(snippet.t_isinstance) - result = fn(1, 2) - assert result == True - - def test_t_issubclass(self): - fn = self.build_interpfunc(snippet.t_issubclass) - result = fn(1, 2) - assert result == True +""" +Description +_____________________________ + +This test is almost a copy of test_genc.py +The setup code is slightly different: +Instead of compiling single functions from +snippets.py, almost all of snippets is translated, +up to the point where they are untranslatable. +snippets has been slightly re-ordered for that. + +The idea was to create a couple of tests without much +extra work, in a sense derived from the test_genc. + +A problem with that is, that the tests actually should +be run at application level. The test code checks real +Python values,so we have to do tricks to unwrap things. +This is limited: +Some tests cannot work, since they mutate their arguments. +Some tests operate with un-unwrappable things. +Those are disabled for the moment by an 'needapp_' prefix. + +XXX think about a way to produce more tests from a common +XXX basis. Should we write generators for such tests like this? +""" +import autopath +import py +from pypy.tool.udir import udir +from pypy.objspace.flow.model import * +from pypy.translator.tool.cbuild import make_module_from_c +from pypy.translator.tool.cbuild import skip_missing_compiler +from pypy.translator.geninterplevel import translate_as_module +from pypy.translator.test import snippet +from pypy.interpreter.error import OperationError +from py.code import Source + +class TestGenRpyTestCase: + objspacename = 'std' + + snippet_ad = """if 1: + def import_func(): + import copy_reg + return copy_reg._reconstructor.func_code.co_name + + def import_sys_func(): + import sys + return sys.__name__ +""" + + def setup_class(cls): + # simply compile snippets just once + src = str(Source(snippet)) + # truncate non-compilable stuff for now: + p = src.index('Non compilable Functions') + src = src[:p] + '\n' + # put our ad into snippet + exec cls.snippet_ad in snippet.__dict__ + src += cls.snippet_ad + # just in case of trouble, we produce a tempfile + ini, newsrc = translate_as_module(src, tmpname = str( + udir.join("_geninterp_test.py"))) + cls.w_glob = ini(cls.space) + + def build_interpfunc(self, func, *morefuncs): + # we ignore morefuncs, since they live in snippets + space = self.space + func = space.getitem(self.w_glob, space.wrap(func.__name__)) + def wrapunwrap(*args): + w_args = space.wrap(args) + try: + w_res = space.call(func, w_args) + except OperationError, e: + w_typ = e.w_type + # XXX how to unwrap an exception? + name = space.unwrap(space.getattr(w_typ, space.wrap('__name__'))) + exc = __builtins__[name] + raise exc + return space.unwrap(w_res) + return wrapunwrap + + # small addition to see whether imports look fine + def test_import(self): + import copy_reg + impfunc = self.build_interpfunc(snippet.import_func) + assert impfunc() == '_reconstructor' + + def test_import_sys(self): + impfunc = self.build_interpfunc(snippet.import_sys_func) + assert impfunc() == 'sys' + + def test_simple_func(self): + cfunc = self.build_interpfunc(snippet.simple_func) + assert cfunc(1) == 2 + + def test_while_func(self): + while_func = self.build_interpfunc(snippet.while_func) + assert while_func(10) == 55 + + def test_nested_whiles(self): + nested_whiles = self.build_interpfunc(snippet.nested_whiles) + assert nested_whiles(111, 114) == ( + '...!...!...!...!...!') + + def test_poor_man_range(self): + poor_man_range = self.build_interpfunc(snippet.poor_man_range) + assert poor_man_range(10) == range(10) + + def poor_man_rev_range(self): + poor_man_rev_range = self.build_interpfunc(snippet.poor_man_rev_range) + assert poor_man_rev_range(10) == range(9,-1,-1) + + def test_simple_id(self): + #we just want to see, if renaming of parameter works correctly + #if the first branch is the end branch + simple_id = self.build_interpfunc(snippet.simple_id) + assert simple_id(9) == 9 + + def test_branch_id(self): + branch_id = self.build_interpfunc(snippet.branch_id) + assert branch_id(1, 2, 3) == 2 + assert branch_id(0, 2, 3) == 3 + + def test_int_id(self): + int_id = self.build_interpfunc(snippet.int_id) + assert int_id(3) == 3 + + def dont_test_attrs(self): + attrs = self.build_interpfunc(snippet.attrs) + assert attrs() == 9 + + def test_builtinusage(self): + fun = self.build_interpfunc(snippet.builtinusage) + assert fun() == 4 + + def xpensive_test_sieve(self): + sieve = self.build_interpfunc(snippet.sieve_of_eratosthenes) + assert sieve() == 1028 + + def test_slice(self): + half = self.build_interpfunc(snippet.half_of_n) + assert half(10) == 5 + + def test_poly_branch(self): + poly_branch = self.build_interpfunc(snippet.poly_branch) + assert poly_branch(10) == [1,2,3]*2 + assert poly_branch(0) == ['a','b','c']*2 + + def test_and(self): + sand = self.build_interpfunc(snippet.s_and) + assert sand(5, 6) == "yes" + assert sand(5, 0) == "no" + assert sand(0, 6) == "no" + assert sand(0, 0) == "no" + + def test_yast(self): + yast = self.build_interpfunc(snippet.yast) + assert yast([1000,100,10,1]) == 1111 + assert yast(range(100)) == (99*100)/2 + + def test_with_init(self): + with_init = self.build_interpfunc(snippet.with_init) + assert with_init(0) == 0 + assert with_init(-100) == -100 + + def test_with_more_init(self): + with_more_init = self.build_interpfunc(snippet.with_more_init) + assert with_more_init(10, False) == -10 + assert with_more_init(20, True) == 20 + + def needapp_test_global_instance(self): + global_instance = self.build_interpfunc(snippet.global_instance) + assert global_instance() == 42 + + def needapp_test_global_newstyle_instance(self): + global_newstyle_instance = self.build_interpfunc(snippet.global_newstyle_instance) + assert global_newstyle_instance().a == 1 + + def needapp_test_global_recursive_list(self): + global_recursive_list = self.build_interpfunc(snippet.global_recursive_list) + lst = global_recursive_list() + assert len(lst) == 1 + assert lst[0] is lst + +## def test_global_badinit(self): +## global_badinit = self.build_interpfunc(snippet.global_badinit) +## self.assertEquals(global_badinit(), 1) + + def test_multiple_inheritance(self): + multiple_inheritance = self.build_interpfunc(snippet.multiple_inheritance) + assert multiple_inheritance() == 1+2+3+4 + + def test_call_star_args(self): + call_star_args = self.build_interpfunc(snippet.call_star_args) + assert call_star_args(42) == 52 + + def test_call_default_args(self): + call_default_args = self.build_interpfunc(snippet.call_default_args) + assert call_default_args(42) == 111+42+3 + + def test_call_default_and_star_args(self): + call_default_and_star_args = self.build_interpfunc( + snippet.call_default_and_star_args) + assert call_default_and_star_args(42) == ( + (111+42+3+0, -1000-2000-3000+2)) + + def test_call_with_star(self): + call_with_star = self.build_interpfunc(snippet.call_with_star) + assert call_with_star(()) == -15L + assert call_with_star((4,)) == -13L + assert call_with_star((4,7)) == -9L + assert call_with_star([]) == -15L + assert call_with_star([4]) == -13L + assert call_with_star([4,7]) == -9L + raises(TypeError, call_with_star, (4,7,12)) + raises(TypeError, call_with_star, [4,7,12,63]) + raises(TypeError, call_with_star, 521) + + def test_call_with_keyword(self): + call_with_keyword = self.build_interpfunc(snippet.call_with_keyword) + assert call_with_keyword(100) == 82 + + def test_call_very_complex(self): + call_very_complex = self.build_interpfunc(snippet.call_very_complex, + snippet.default_args) + assert call_very_complex(5, (3,), {}) == -12 + assert call_very_complex(5, (), {'y': 3}) == -12 + raises(TypeError, call_very_complex, 5, (3,), {'y': 4}) + + def test_finallys(self): + finallys = self.build_interpfunc(snippet.finallys) + assert finallys(['hello']) == 8 + assert finallys('X') == 8 + assert finallys([]) == 6 + assert finallys('XY') == 6 + + def needapp_test_finally2(self): + finally2 = self.build_interpfunc(snippet.finally2) + lst = range(10) + finally2(lst, 5) + assert lst == [0,1,2,3,4, 6, 6,7,8, 'done'] + dic = {} + raises(KeyError, finally2, dic, "won't find this key") + assert dic == {-1: 'done'} + + def test_bare_raise(self): + bare_raise = self.build_interpfunc(snippet.bare_raise) + assert bare_raise(range(0, 100, 10), False) == 50 + assert bare_raise(range(0, 100, 10), True) == 50 + raises(IndexError, bare_raise, range(0, 30, 10), False) + assert bare_raise(range(0, 30, 10), True) == None + + def needapp_test_get_set_del_slice(self): + fn = self.build_interpfunc(snippet.get_set_del_slice) + l = list('abcdefghij') + result = fn(l) + assert l == [3, 'c', 8, 11, 'h', 9] + assert result == ([3, 'c'], [9], [11, 'h']) + + def test_do_try_raise_choose(self): + fn = self.build_interpfunc(snippet.do_try_raise_choose) + result = fn() + assert result == [-1,0,1,2] + + + def test_t_isinstance(self): + fn = self.build_interpfunc(snippet.t_isinstance) + result = fn(1, 2) + assert result == True + + def test_t_issubclass(self): + fn = self.build_interpfunc(snippet.t_issubclass) + result = fn(1, 2) + assert result == True Modified: pypy/dist/pypy/translator/test/test_rpystone.py ============================================================================== --- pypy/dist/pypy/translator/test/test_rpystone.py (original) +++ pypy/dist/pypy/translator/test/test_rpystone.py Tue Jan 24 13:31:23 2006 @@ -1,20 +1,20 @@ -from pypy.translator.geninterplevel import translate_as_module, __file__ as __ -from pypy.objspace.std import Space -import os -fname = os.path.join(os.path.dirname(__), "test", "rpystone.py") -src = file(fname).read() -init, ign = translate_as_module(src)#, tmpname="/tmp/look.py") - -LOOPS = 25 - -def test_rpystone(): - space = Space() - modic = init(space) - entry = space.getitem(modic, space.wrap("entrypoint")) - # warm-up,to get everything translated - space.call(entry, space.newtuple([space.wrap(-1)])) - # now this is the real one - space.call(entry, space.newtuple([space.wrap(LOOPS)])) - -if __name__ == "__main__": +from pypy.translator.geninterplevel import translate_as_module, __file__ as __ +from pypy.objspace.std import Space +import os +fname = os.path.join(os.path.dirname(__), "test", "rpystone.py") +src = file(fname).read() +init, ign = translate_as_module(src)#, tmpname="/tmp/look.py") + +LOOPS = 25 + +def test_rpystone(): + space = Space() + modic = init(space) + entry = space.getitem(modic, space.wrap("entrypoint")) + # warm-up,to get everything translated + space.call(entry, space.newtuple([space.wrap(-1)])) + # now this is the real one + space.call(entry, space.newtuple([space.wrap(LOOPS)])) + +if __name__ == "__main__": test_rpystone() \ No newline at end of file From cfbolz at codespeak.net Tue Jan 24 13:36:56 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Tue, 24 Jan 2006 13:36:56 +0100 (CET) Subject: [pypy-svn] r22587 - pypy/dist/pypy/rpython/test Message-ID: <20060124123656.1295927DB6@code1.codespeak.net> Author: cfbolz Date: Tue Jan 24 13:36:54 2006 New Revision: 22587 Modified: pypy/dist/pypy/rpython/test/test_rgenop.py Log: (arigo, cfbolz) yet another test that didn't find bugs but was annoying to write :-( Modified: pypy/dist/pypy/rpython/test/test_rgenop.py ============================================================================== --- pypy/dist/pypy/rpython/test/test_rgenop.py (original) +++ pypy/dist/pypy/rpython/test/test_rgenop.py Tue Jan 24 13:36:54 2006 @@ -37,3 +37,40 @@ assert res == 0 res = runlink(startlink, [42]) assert res == 42 + +def test_loop(): + """ + def f(v0): + i = 1 + result = 1 + while i <= v0: + result *= i + i += 1 + return result + """ + startlink = newgraph("loop") + block = newblock() + v0 = geninputarg(block, Signed) + const1 = genconst(block, 1) + link = newlink(block, [const1, const1, v0]) + closeblock1(block, link) + closelink(startlink, block) + loopblock = newblock() + result0 = geninputarg(loopblock, Signed) + i0 = geninputarg(loopblock, Signed) + v1 = geninputarg(loopblock, Signed) + closelink(link, loopblock) + const1 = genconst(block, 1) + result1 = genop(loopblock, 'int_mul', [result0, i0], Signed) + i1 = genop(loopblock, 'int_add', [i0, const1], Signed) + v2 = genop(loopblock, 'int_le', [i1, v1], Bool) + false_link = newreturnlink(loopblock, result1) + true_link = newlink(loopblock, [result1, i1, v1]) + closelink(true_link, loopblock) + closeblock2(loopblock, v2, false_link, true_link) + res = runlink(startlink, [0]) + assert res == 1 + res = runlink(startlink, [1]) + assert res == 1 + res = runlink(startlink, [7]) + assert res == 5040 From arigo at codespeak.net Tue Jan 24 13:45:39 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 24 Jan 2006 13:45:39 +0100 (CET) Subject: [pypy-svn] r22588 - pypy/dist/pypy/translator/c Message-ID: <20060124124539.0B0BB27DB8@code1.codespeak.net> Author: arigo Date: Tue Jan 24 13:45:34 2006 New Revision: 22588 Modified: pypy/dist/pypy/translator/c/stackless.py Log: fix an import. Modified: pypy/dist/pypy/translator/c/stackless.py ============================================================================== --- pypy/dist/pypy/translator/c/stackless.py (original) +++ pypy/dist/pypy/translator/c/stackless.py Tue Jan 24 13:45:34 2006 @@ -7,7 +7,7 @@ import py from pypy.objspace.flow.model import Variable from pypy.rpython.lltypesystem import lltype -from pypy.rpython.memory.lladdress import Address +from pypy.rpython.lltypesystem.llmemory import Address from pypy.rpython.module import ll_stack, ll_stackless from pypy.translator.c.support import cdecl from pypy.translator.c.funcgen import FunctionCodeGenerator From auc at codespeak.net Tue Jan 24 14:12:44 2006 From: auc at codespeak.net (auc at codespeak.net) Date: Tue, 24 Jan 2006 14:12:44 +0100 (CET) Subject: [pypy-svn] r22589 - pypy/dist/pypy/lib/logic Message-ID: <20060124131244.F3EE027DB8@code1.codespeak.net> Author: auc Date: Tue Jan 24 14:12:41 2006 New Revision: 22589 Modified: pypy/dist/pypy/lib/logic/constraint.py pypy/dist/pypy/lib/logic/test_unification.py pypy/dist/pypy/lib/logic/unification.py pypy/dist/pypy/lib/logic/variable.py Log: (auc, ale) * add satisfiability check on constraints within the store * more tests Modified: pypy/dist/pypy/lib/logic/constraint.py ============================================================================== --- pypy/dist/pypy/lib/logic/constraint.py (original) +++ pypy/dist/pypy/lib/logic/constraint.py Tue Jan 24 14:12:41 2006 @@ -124,6 +124,7 @@ if var.dom is None: raise DomainlessVariables self._names_to_vars[var.name] = var + var.add_constraint(self) self._variables = variables def affectedVariables(self): Modified: pypy/dist/pypy/lib/logic/test_unification.py ============================================================================== --- pypy/dist/pypy/lib/logic/test_unification.py (original) +++ pypy/dist/pypy/lib/logic/test_unification.py Tue Jan 24 14:12:41 2006 @@ -234,11 +234,90 @@ u.add_constraint(k) assert k in u._store.constraints - def test_narrowing_domains(self): + def test_narrowing_domains_failure(self): x,y,z = u.var('x'), u.var('y'), u.var('z') x.dom = c.FiniteDomain([1, 2]) y.dom = c.FiniteDomain([2, 3]) z.dom = c.FiniteDomain([3, 4]) k = c.Expression([x, y, z], 'x == y + z') raises(c.ConsistencyFailure, k.narrow) + + def test_narrowing_domains_success(self): + x,y,z = u.var('x'), u.var('y'), u.var('z') + x.dom = c.FiniteDomain([1, 2, 5]) + y.dom = c.FiniteDomain([2, 3]) + z.dom = c.FiniteDomain([3, 4]) + k = c.Expression([x, y, z], 'x == y + z') + k.narrow() + assert x.dom == c.FiniteDomain([5]) + assert y.dom == c.FiniteDomain([2]) + assert z.dom == c.FiniteDomain([3]) + + def test_store_satisfiable_success(self): + x,y,z = u.var('x'), u.var('y'), u.var('z') + x.dom = c.FiniteDomain([1, 2, 5]) + y.dom = c.FiniteDomain([2, 3]) + z.dom = c.FiniteDomain([3, 4]) + k = c.Expression([x, y, z], 'x == y + z') + u.add_constraint(k) + assert u.satisfiable(k) == True + assert x.dom == c.FiniteDomain([1, 2, 5]) + assert y.dom == c.FiniteDomain([2, 3]) + assert z.dom == c.FiniteDomain([3, 4]) + + def test_store_satisfiable_failure(self): + x,y,z = u.var('x'), u.var('y'), u.var('z') + x.dom = c.FiniteDomain([1, 2]) + y.dom = c.FiniteDomain([2, 3]) + z.dom = c.FiniteDomain([3, 4]) + k = c.Expression([x, y, z], 'x == y + z') + u.add_constraint(k) + assert u.satisfiable(k) == False + assert x.dom == c.FiniteDomain([1, 2]) + assert y.dom == c.FiniteDomain([2, 3]) + assert z.dom == c.FiniteDomain([3, 4]) + + def test_satisfy_many_const_success(self): + x,y,z,w = (u.var('x'), u.var('y'), + u.var('z'), u.var('w')) + x.dom = c.FiniteDomain([1, 2, 5]) + y.dom = c.FiniteDomain([2, 3]) + z.dom = c.FiniteDomain([3, 4]) + w.dom = c.FiniteDomain([1, 4, 5]) + k1 = c.Expression([x, y, z], 'x == y + z') + k2 = c.Expression([z, w], 'z < w') + u.add_constraint(k1) + u.add_constraint(k2) + assert u.satisfiable(k1) == True + assert x.dom == c.FiniteDomain([1, 2, 5]) + assert y.dom == c.FiniteDomain([2, 3]) + assert z.dom == c.FiniteDomain([3, 4]) + assert w.dom == c.FiniteDomain([1, 4, 5]) + assert u.satisfiable(k2) == True + assert x.dom == c.FiniteDomain([1, 2, 5]) + assert y.dom == c.FiniteDomain([2, 3]) + assert z.dom == c.FiniteDomain([3, 4]) + assert w.dom == c.FiniteDomain([1, 4, 5]) + + def test_satisfy_many_const_failure(self): + x,y,z,w = (u.var('x'), u.var('y'), + u.var('z'), u.var('w')) + x.dom = c.FiniteDomain([1, 2, 5]) + y.dom = c.FiniteDomain([2, 3]) + z.dom = c.FiniteDomain([3, 4]) + w.dom = c.FiniteDomain([1]) + k1 = c.Expression([x, y, z], 'x == y + z') + k2 = c.Expression([z, w], 'z < w') + u.add_constraint(k1) + u.add_constraint(k2) + assert u.satisfiable(k1) == False + assert x.dom == c.FiniteDomain([1, 2, 5]) + assert y.dom == c.FiniteDomain([2, 3]) + assert z.dom == c.FiniteDomain([3, 4]) + assert w.dom == c.FiniteDomain([1]) + assert u.satisfiable(k2) == False + assert x.dom == c.FiniteDomain([1, 2, 5]) + assert y.dom == c.FiniteDomain([2, 3]) + assert z.dom == c.FiniteDomain([3, 4]) + assert w.dom == c.FiniteDomain([1]) Modified: pypy/dist/pypy/lib/logic/unification.py ============================================================================== --- pypy/dist/pypy/lib/logic/unification.py (original) +++ pypy/dist/pypy/lib/logic/unification.py Tue Jan 24 14:12:41 2006 @@ -121,7 +121,7 @@ import threading from variable import EqSet, Var, VariableException, NotAVariable -from constraint import FiniteDomain +from constraint import FiniteDomain, ConsistencyFailure #----------- Store Exceptions ---------------------------- class UnboundVariable(VariableException): @@ -196,11 +196,47 @@ raise AlreadyBound var.dom = FiniteDomain(dom) - #-- Add constraints ------------------------- + #-- Constraints ------------------------- def add_constraint(self, constraint): self.constraints.add(constraint) + def satisfiable(self, constraint): + """ * satisfiable (k) checks that the constraint k + can be satisfied wrt its variable domains + and other constraints on these variables + * does NOT mutate the store + """ + # Satisfiability of one constraints entails + # satisfiability of the transitive closure + # of all constraints associated with the vars + # of our given constraint. + # We make a copy of the domains + # then traverse the constraints & attached vars + # to collect all (in)directly affected vars + # then compute narrow() on all (in)directly + # affected constraints. + assert constraint in self.constraints + varset = set() + constset = set() + compute_dependant_vars(constraint, varset, constset) + old_domains = {} + for var in varset: + old_domains[var] = FiniteDomain(var.dom) + + def restore_domains(domains): + for var, dom in domains.items(): + var.dom = dom + + for const in constset: + try: + const.narrow() + except ConsistencyFailure: + restore_domains(old_domains) + return False + restore_domains(old_domains) + return True + #-- BIND ------------------------------------------- def bind(self, var, val): @@ -344,6 +380,19 @@ return True +def compute_dependant_vars(constraint, varset, + constset): + if constraint in constset: return + constset.add(constraint) + for var in constraint.affectedVariables(): + varset.add(var) + dep_consts = var.constraints + for const in dep_consts: + if const in constset: + continue + compute_dependant_vars(const, varset, constset) + + #-- Unifiability checks--------------------------------------- #-- #-- quite costly & could be merged back in unify @@ -423,6 +472,9 @@ def add_constraint(constraint): return _store.add_constraint(constraint) +def satisfiable(constraint): + return _store.satisfiable(constraint) + def bind(var, val): return _store.bind(var, val) Modified: pypy/dist/pypy/lib/logic/variable.py ============================================================================== --- pypy/dist/pypy/lib/logic/variable.py (original) +++ pypy/dist/pypy/lib/logic/variable.py Tue Jan 24 14:12:41 2006 @@ -39,6 +39,8 @@ self._val = NoValue # domain self.dom = None + # constraints + self.constraints = set() # when updated in a 'transaction', keep track # of our initial value (for abort cases) self.previous = None @@ -92,6 +94,10 @@ def __hash__(self): return self.name.__hash__() + + def add_constraint(self, constraint): + self.constraints.add(constraint) + #---- Concurrent public ops -------------------------- def is_bound(self): From ac at codespeak.net Tue Jan 24 14:22:58 2006 From: ac at codespeak.net (ac at codespeak.net) Date: Tue, 24 Jan 2006 14:22:58 +0100 (CET) Subject: [pypy-svn] r22590 - in pypy/dist/pypy: annotation rpython rpython/l3interp rpython/l3interp/test rpython/lltypesystem rpython/lltypesystem/test rpython/memory translator/c Message-ID: <20060124132258.8D39F27B5C@code1.codespeak.net> Author: ac Date: Tue Jan 24 14:22:57 2006 New Revision: 22590 Added: pypy/dist/pypy/rpython/lltypesystem/test/ (props changed) pypy/dist/pypy/rpython/lltypesystem/test/test_llmemory.py (contents, props changed) Modified: pypy/dist/pypy/annotation/binaryop.py pypy/dist/pypy/rpython/l3interp/convertgraph.py pypy/dist/pypy/rpython/l3interp/l3interp.py pypy/dist/pypy/rpython/l3interp/model.py pypy/dist/pypy/rpython/l3interp/test/test_convert.py pypy/dist/pypy/rpython/l3interp/test/test_l3interp.py pypy/dist/pypy/rpython/lltypesystem/llmemory.py pypy/dist/pypy/rpython/memory/lladdress.py pypy/dist/pypy/rpython/memory/lltypelayout.py pypy/dist/pypy/rpython/raddress.py pypy/dist/pypy/translator/c/primitive.py Log: (mwh, arre) Implemented getarrayitem and in the process refactored Offsets. Modified: pypy/dist/pypy/annotation/binaryop.py ============================================================================== --- pypy/dist/pypy/annotation/binaryop.py (original) +++ pypy/dist/pypy/annotation/binaryop.py Tue Jan 24 14:22:57 2006 @@ -737,6 +737,10 @@ def add((s_off1, s_off2)): return SomeOffset() +class __extend__(pairtype(SomeOffset, SomeInteger)): + def mul((s_off1, s_off2)): + return SomeOffset() + class __extend__(pairtype(SomeAddress, SomeOffset)): def add((s_addr, s_off)): return SomeAddress() Modified: pypy/dist/pypy/rpython/l3interp/convertgraph.py ============================================================================== --- pypy/dist/pypy/rpython/l3interp/convertgraph.py (original) +++ pypy/dist/pypy/rpython/l3interp/convertgraph.py Tue Jan 24 14:22:57 2006 @@ -3,7 +3,8 @@ from pypy.rpython.l3interp.model import Op from pypy.objspace.flow import model as flowmodel from pypy.rpython.lltypesystem import lltype -from pypy.rpython.lltypesystem.llmemory import OffsetOf, fakeaddress +from pypy.rpython.lltypesystem.llmemory import FieldOffset, \ + ItemOffset, ArrayItemsOffset, fakeaddress class LL2L3Converter(object): @@ -86,6 +87,15 @@ position = var2stack[v] return position - stacksizes[kind] # < 0 + def getoffset(offset): + clist = constants['offset'] + try: + res = clist.index(offset) + except ValueError: + res = len(clist) + clist.append(offset) + return res + for v in block.inputargs: if v.concretetype is not lltype.Void: push(v) @@ -117,14 +127,8 @@ v0, v1 = spaceop.args insns.append(get(v0)) - offset = OffsetOf(v0.concretetype, v1.value) - clist = constants['offset'] - try: - res = clist.index(offset) - except ValueError: - res = len(clist) - clist.append(offset) - insns.append(res) + offset = FieldOffset(v0.concretetype, v1.value) + insns.append(getoffset(offset)) elif spaceop.opname == 'setfield': v0, v1, v2 = spaceop.args opname = spaceop.opname + '_' + \ @@ -132,15 +136,22 @@ insns.append(model.very_low_level_opcode[opname]) insns.append(get(v0)) - offset = OffsetOf(v0.concretetype, v1.value) - clist = constants['offset'] - try: - res = clist.index(offset) - except ValueError: - res = len(clist) - clist.append(offset) - insns.append(res) + offset = FieldOffset(v0.concretetype, v1.value) + insns.append(getoffset(offset)) insns.append(get(v2)) + elif spaceop.opname == 'getarrayitem': + opname = spaceop.opname + '_' + \ + getaccesskind(spaceop.result.concretetype) + insns.append(model.very_low_level_opcode[opname]) + v0, v1 = spaceop.args + insns.append(get(v0)) + insns.append(get(v1)) + + offset = ArrayItemsOffset(v0.concretetype) + insns.append(getoffset(offset)) + + offset = ItemOffset(spaceop.result.concretetype) + insns.append(getoffset(offset)) else: insns.append(model.very_low_level_opcode[spaceop.opname]) for v in spaceop.args: Modified: pypy/dist/pypy/rpython/l3interp/l3interp.py ============================================================================== --- pypy/dist/pypy/rpython/l3interp/l3interp.py (original) +++ pypy/dist/pypy/rpython/l3interp/l3interp.py Tue Jan 24 14:22:57 2006 @@ -2,7 +2,7 @@ from pypy.rpython.memory import lladdress from pypy.rpython.rarithmetic import r_uint from pypy.interpreter.miscutils import InitializedClass -from pypy.rpython.lltypesystem.llmemory import fakeaddress, OffsetOf +from pypy.rpython.lltypesystem.llmemory import fakeaddress, AddressOffset from pypy.rpython.lltypesystem import lltype class L3Exception(Exception): @@ -46,7 +46,7 @@ return L3Nothing() raise AssertionError("stacks corrupted") -constant_offset = OffsetOf(lltype.Void) +constant_offset = AddressOffset() constant_fakeaddress = fakeaddress(None) class L3Frame(object): @@ -203,6 +203,14 @@ o = self.getoffset() v = self.getint() (p + o).signed[0] = v + + def op_getarrayitem_int(self): + a = self.getptr() + i = self.getint() + items_offset = self.getoffset() + s = self.getoffset() + v = (a + items_offset + s * i).signed[0] + self.stack_int.append(v) def op_flavored_malloc(self): self.stack_ptr.append(constant_fakeaddress) Modified: pypy/dist/pypy/rpython/l3interp/model.py ============================================================================== --- pypy/dist/pypy/rpython/l3interp/model.py (original) +++ pypy/dist/pypy/rpython/l3interp/model.py Tue Jan 24 14:22:57 2006 @@ -37,7 +37,8 @@ 'float_ne', 'float_neg', 'float_sub', 'float_truediv', #array operations: - 'getarrayitem', 'getarraysize', 'getarraysubstruct', 'setarrayitem', + 'getarrayitem_int', 'getarrayitem_ptr', + 'getarraysize', 'getarraysubstruct', 'setarrayitem', #struct operations: 'getfield_int', 'getfield_char', 'getfield_dbl', 'getfield_ptr', Modified: pypy/dist/pypy/rpython/l3interp/test/test_convert.py ============================================================================== --- pypy/dist/pypy/rpython/l3interp/test/test_convert.py (original) +++ pypy/dist/pypy/rpython/l3interp/test/test_convert.py Tue Jan 24 14:22:57 2006 @@ -76,6 +76,29 @@ assert isinstance(result, l3interp.L3Integer) assert result.intval == 2 + +def test_convert_getitem(): + from pypy.rpython.lltypesystem import lltype + + A = lltype.GcArray(lltype.Signed) + a = lltype.malloc(A, 3) + a[0] = 1 + a[1] = 2 + a[2] = 3 + + + def f(n): + return a[n] + + l3graph = l3ify(f, [int]) + result = l3interp.l3interpret(l3graph, [0], [], []) + assert isinstance(result, l3interp.L3Integer) + assert result.intval == 1 + + result = l3interp.l3interpret(l3graph, [1], [], []) + assert isinstance(result, l3interp.L3Integer) + assert result.intval == 2 + def dont_test_call(): def f(): return 2 Modified: pypy/dist/pypy/rpython/l3interp/test/test_l3interp.py ============================================================================== --- pypy/dist/pypy/rpython/l3interp/test/test_l3interp.py (original) +++ pypy/dist/pypy/rpython/l3interp/test/test_l3interp.py Tue Jan 24 14:22:57 2006 @@ -238,3 +238,31 @@ for y in 0, 1: assert fn(x, y) == f(x, y) + +def test_getitem(): + from pypy.rpython.lltypesystem import lltype + + A = lltype.GcArray(lltype.Signed) + a = lltype.malloc(A, 3) + a[0] = 1 + a[1] = 2 + a[2] = 3 + + + def f(n): + return a[n] + + l3graph = l3ify(f, [int]) + def entry_point(x): + value = l3interp.l3interpret(l3graph, [x], [], []) + assert isinstance(value, l3interp.L3Integer) + return value.intval + + for arg in 0,1,2: + assert entry_point(arg) == f(arg) + + fn = translate(entry_point, [int]) + + for arg in 0,1,2: + assert fn(arg) == f(arg) + Modified: pypy/dist/pypy/rpython/lltypesystem/llmemory.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/llmemory.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/llmemory.py Tue Jan 24 14:22:57 2006 @@ -7,11 +7,7 @@ from pypy.rpython.objectmodel import Symbolic from pypy.rpython.lltypesystem import lltype -class OffsetOf(Symbolic): - - def __init__(self, TYPE, *fldnames): - self.TYPE = TYPE - self.fldnames = fldnames +class AddressOffset(Symbolic): def annotation(self): from pypy.annotation import model @@ -20,72 +16,140 @@ def lltype(self): return lltype.Signed + def __add__(self, other): + return CompositeOffset(self, other) + + +class ItemOffset(AddressOffset): + + def __init__(self, TYPE, repeat=1): + self.TYPE = TYPE + self.repeat = repeat + def __repr__(self): - return "" % (self.TYPE, self.fldnames) + return "" % (self.TYPE, self.repeat) - def __add__(self, other): - if not isinstance(other, OffsetOf): + def __mul__(self, other): + if not isinstance(other, int): return NotImplemented - t = self.TYPE - for f in self.fldnames: - t = t._flds[f] - assert t == other.TYPE - return OffsetOf(self.TYPE, *(self.fldnames + other.fldnames)) + return ItemOffset(self.TYPE, self.repeat * other) + + def get(self, ob): + return ob[self.repeat] + + def set(self, ob, value): + ob[self.repeat] = value + + +class FieldOffset(AddressOffset): + + def __init__(self, TYPE, fldname): + self.TYPE = TYPE + self.fldname = fldname + + def __repr__(self): + return "" % (self.TYPE, self.fldname) + + def get(self, ob): + return getattr(ob, self.fldname) + + def set(self, ob, value): + setattr(ob, self.fldname, value) + + +class CompositeOffset(AddressOffset): + + def __init__(self, first, second): + self.first = first + self.second = second + + def __repr__(self): + return '< %r + %r >'%(self.first, self.second) + + def get(self, ob): + return self.second.get(self.first.get(ob)) + + def set(self, ob, value): + return self.second.set(self.first.get(ob), value) + + +class ArrayItemsOffset(AddressOffset): + + def __init__(self, TYPE): + self.TYPE = TYPE + + def __repr__(self): + return '< ArrayItemsOffset >' + + def get(self, ob): + return ob + + def set(self, ob, value): + raise Exception("can't assign to an array's items") + def sizeof(TYPE, n=None): pass def offsetof(TYPE, fldname): assert fldname in TYPE._flds - return OffsetOf(TYPE, fldname) + return FieldOffset(TYPE, fldname) -def itemoffsetof(TYPE, n=None): - pass +# ------------------------------------------------------------- class fakeaddress(object): def __init__(self, ob, offset=None): self.ob = ob - if offset is None: - self.offset = OffsetOf(lltype.typeOf(self.ob)) - else: - self.offset = offset + self.offset = offset def __add__(self, other): - if not isinstance(other, OffsetOf): - return NotImplemented - return fakeaddress(self.ob, self.offset + other) - + if isinstance(other, AddressOffset): + if self.offset is None: + offset = other + else: + offset = self.offset + other + return fakeaddress(self.ob, offset) + return NotImplemented + + def get(self): + if self.offset is None: + return self.ob + else: + return self.offset.get(self.ob) + + def set(self, value): + if self.offset is None: + raise Exception("can't assign to whole object") + else: + self.offset.set(self.ob, value) + +# XXX the indexing in code like +# addr.signed[0] = v +# is just silly. remove it. + class _fakeaccessor(object): def __init__(self, addr): self.addr = addr def __getitem__(self, index): assert index == 0 - ob = self.addr.ob - for n in self.addr.offset.fldnames: - ob = getattr(ob, n) - return self.vet(ob) + return self.convert(self.addr.get()) def __setitem__(self, index, value): assert index == 0 - ob = self.addr.ob - for n in self.addr.offset.fldnames[:-1]: - ob = getattr(ob, n) - fld = self.addr.offset.fldnames[-1] - assert lltype.typeOf(value) == self.TYPE - assert lltype.typeOf(ob).TO._flds[fld] == self.TYPE - setattr(ob, fld, value) + self.addr.set(value) + class _signed_fakeaccessor(_fakeaccessor): TYPE = lltype.Signed - def vet(self, value): + def convert(self, value): assert lltype.typeOf(value) == self.TYPE return value class _address_fakeaccessor(_fakeaccessor): TYPE = None - def vet(self, value): + def convert(self, value): # XXX is this the right check for "Ptr-ness" ? assert isinstance(value, lltype._ptr) return fakeaddress(value) Added: pypy/dist/pypy/rpython/lltypesystem/test/test_llmemory.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/rpython/lltypesystem/test/test_llmemory.py Tue Jan 24 14:22:57 2006 @@ -0,0 +1,35 @@ +from pypy.rpython.lltypesystem.llmemory import * + +def test_simple(): + class C: + def __init__(self, x): + self.x = x + c = C(1) + a = fakeaddress(c) + assert a.get() is c + b = a + FieldOffset('dummy', 'x') + assert b.get() == 1 + b.set(2) + assert c.x == 2 + +def test_composite(): + class C: + def __init__(self, x): + self.x = x + c = C(C(3)) + a = fakeaddress(c) + assert a.get() is c + b = a + FieldOffset('dummy', 'x') + FieldOffset('dummy', 'x') + assert b.get() == 3 + b.set(2) + assert c.x.x == 2 + +def test_array(): + x = [2, 3, 5, 7, 11] + a = fakeaddress(x) + # is there a way to ensure that we add the ArrayItemsOffset ? + b = a + ArrayItemsOffset('dummy') + ItemOffset('dummy')*3 + assert b.get() == x[3] + b.set(14) + assert x[3] == 14 + Modified: pypy/dist/pypy/rpython/memory/lladdress.py ============================================================================== --- pypy/dist/pypy/rpython/memory/lladdress.py (original) +++ pypy/dist/pypy/rpython/memory/lladdress.py Tue Jan 24 14:22:57 2006 @@ -28,7 +28,7 @@ if isinstance(offset, int): return address(self.intaddress + offset) else: - assert isinstance(offset, llmemory.OffsetOf) + assert isinstance(offset, llmemory.AddressOffset) return address(self.intaddress + convert_offset_to_int(offset)) def __sub__(self, other): @@ -50,7 +50,7 @@ def _store(self, fmt, *values): # XXX annoyance: suddenly an OffsetOf changes into a Signed?! from pypy.rpython.memory.lltypelayout import convert_offset_to_int - if len(values) == 1 and isinstance(values[0], llmemory.OffsetOf): + if len(values) == 1 and isinstance(values[0], llmemory.AddressOffset): values = [convert_offset_to_int(values[0])] simulator.setstruct(fmt, self.intaddress, *values) @@ -81,7 +81,7 @@ def convert_to(self, offset): from pypy.rpython.memory.lltypelayout import convert_offset_to_int # XXX same annoyance as in _store - if isinstance(offset, llmemory.OffsetOf): + if isinstance(offset, llmemory.AddressOffset): return convert_offset_to_int(offset) return int(offset) Modified: pypy/dist/pypy/rpython/memory/lltypelayout.py ============================================================================== --- pypy/dist/pypy/rpython/memory/lltypelayout.py (original) +++ pypy/dist/pypy/rpython/memory/lltypelayout.py Tue Jan 24 14:22:57 2006 @@ -88,13 +88,21 @@ return fixedsize + i * varsize def convert_offset_to_int(offset): - TYPE = offset.TYPE - res = 0 - for fld in offset.fldnames: - layout = get_layout(TYPE) - res += layout[fld] - TYPE = getattr(TYPE, fld) - return res + if isinstance(offset, llmemory.FieldOffset): + layout = get_layout(offset.TYPE) + return layout[offset.fldname] + elif isinstance(offset, llmemory.CompositeOffset): + return convert_offset_to_int(offset.first) + \ + convert_offset_to_int(offset.second) + elif type(offset) == llmemory.AddressOffset: + return 0 + elif isinstance(offset, llmemory.ItemOffset): + return sizeof(offset.TYPE) * offset.repeat + elif isinstance(offset, llmemory.ArrayItemsOffset): + return get_fixed_size(lltype.Signed) + else: + raise Exception("unknown offset type %r"%offset) + # _____________________________________________________________________________ # the following functions are used to find contained pointers Modified: pypy/dist/pypy/rpython/raddress.py ============================================================================== --- pypy/dist/pypy/rpython/raddress.py (original) +++ pypy/dist/pypy/rpython/raddress.py Tue Jan 24 14:22:57 2006 @@ -129,6 +129,11 @@ v_offset1, v_offset2 = hop.inputargs(offset_repr, offset_repr) return hop.genop('int_add', [v_offset1, v_offset2], resulttype=lltype.Signed) +class __extend__(pairtype(OffsetRepr, IntegerRepr)): + def rtype_mul((r_offset, r_int), hop): + v_offset, v_int = hop.inputargs(r_offset, r_int) + return hop.genop('int_mul', [v_offset, v_int], resulttype=lltype.Signed) + class __extend__(pairtype(AddressRepr, OffsetRepr)): def rtype_add((r_offset1, r_offset2), hop): v_offset1, v_offset2 = hop.inputargs(Address, offset_repr) Modified: pypy/dist/pypy/translator/c/primitive.py ============================================================================== --- pypy/dist/pypy/translator/c/primitive.py (original) +++ pypy/dist/pypy/translator/c/primitive.py Tue Jan 24 14:22:57 2006 @@ -1,6 +1,7 @@ import sys from pypy.rpython.lltypesystem.lltype import * -from pypy.rpython.lltypesystem.llmemory import Address, OffsetOf, fakeaddress +from pypy.rpython.lltypesystem.llmemory import Address, fakeaddress, \ + AddressOffset, ItemOffset, ArrayItemsOffset, FieldOffset from pypy.rpython.memory.lladdress import NULL # ____________________________________________________________ @@ -9,15 +10,21 @@ def name_signed(value, db): if isinstance(value, Symbolic): - assert isinstance(value, OffsetOf) - assert len(value.fldnames) <= 1 - if len(value.fldnames) == 0: - return '0 /*offsetof*/' - else: + if isinstance(value, FieldOffset): structnode = db.gettypedefnode(value.TYPE.TO) - return 'offsetof(struct %s, %s)'%( - structnode.name, - structnode.c_struct_field_name(value.fldnames[0])) + return 'offsetof(%s, %s)'%( + db.gettype(value.TYPE.TO).replace('@', ''), + structnode.c_struct_field_name(value.fldname)) + elif isinstance(value, ItemOffset): + return '(sizeof(%s) * %s)'%( + db.gettype(value.TYPE).replace('@', ''), value.repeat) + elif isinstance(value, ArrayItemsOffset): + return 'offsetof(%s, items)'%( + db.gettype(value.TYPE.TO).replace('@', '')) + elif type(value) == AddressOffset: + return '0' + else: + raise Exception("unimplemented symbolic %r"%value) if value == -sys.maxint-1: # blame C return '(-%dL-1L)' % sys.maxint else: From arigo at codespeak.net Tue Jan 24 14:30:10 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 24 Jan 2006 14:30:10 +0100 (CET) Subject: [pypy-svn] r22591 - in pypy/dist/pypy: rpython translator/c Message-ID: <20060124133010.3695927DB8@code1.codespeak.net> Author: arigo Date: Tue Jan 24 14:30:09 2006 New Revision: 22591 Modified: pypy/dist/pypy/rpython/rlist.py pypy/dist/pypy/translator/c/extfunc.py Log: fix a compiler warning: rlist.LIST_OF_STR was kind of broken, remove it and hack differently. Modified: pypy/dist/pypy/rpython/rlist.py ============================================================================== --- pypy/dist/pypy/rpython/rlist.py (original) +++ pypy/dist/pypy/rpython/rlist.py Tue Jan 24 14:30:09 2006 @@ -1028,14 +1028,3 @@ raise StopIteration iter.index = index + 1 return l.ll_items()[index] - -# ___________________________________________________________ - -LIST_OF_STR = GcStruct("list", - ("length", Signed), - ("items", Ptr(GcArray(Ptr(rstr.STR)))), - adtmeths = { - "ll_newlist": ll_newlist, - "ll_length": ll_length, - "ll_items": ll_items, - }) Modified: pypy/dist/pypy/translator/c/extfunc.py ============================================================================== --- pypy/dist/pypy/translator/c/extfunc.py (original) +++ pypy/dist/pypy/translator/c/extfunc.py Tue Jan 24 14:30:09 2006 @@ -3,7 +3,7 @@ from pypy.rpython.lltypesystem import lltype from pypy.translator.c.support import cdecl from pypy.rpython.rstr import STR -from pypy.rpython import rlist +from pypy.rpython import rlist, rstr from pypy.rpython.module import ll_os, ll_time, ll_math, ll_strtod from pypy.rpython.module import ll_stackless, ll_stack from pypy.module.thread.rpython import ll_thread @@ -84,11 +84,18 @@ #______________________________________________________ +def find_list_of_str(rtyper): + for r in rtyper.reprs.itervalues(): + if isinstance(r, rlist.ListRepr) and r.item_repr is rstr.string_repr: + return r.lowleveltype.TO + return None def predeclare_common_types(db, rtyper): # Common types yield ('RPyString', STR) - yield ('RPyListOfString', rlist.LIST_OF_STR) + LIST_OF_STR = find_list_of_str(rtyper) + if LIST_OF_STR is not None: + yield ('RPyListOfString', LIST_OF_STR) yield ('RPyFREXP_RESULT', ll_math.FREXP_RESULT) yield ('RPyMODF_RESULT', ll_math.MODF_RESULT) yield ('RPySTAT_RESULT', ll_os.STAT_RESULT) @@ -107,15 +114,17 @@ # Such result should be only from C code # returned directly as results - p = lltype.Ptr(rlist.LIST_OF_STR) - - def _RPyListOfString_New(length=lltype.Signed): - return rlist.LIST_OF_STR.ll_newlist(length) - - def _RPyListOfString_SetItem(l=p, - index=lltype.Signed, - newstring=lltype.Ptr(STR)): - rlist.ll_setitem_nonneg(rlist.dum_nocheck, l, index, newstring) + LIST_OF_STR = find_list_of_str(rtyper) + if LIST_OF_STR is not None: + p = lltype.Ptr(LIST_OF_STR) + + def _RPyListOfString_New(length=lltype.Signed): + return LIST_OF_STR.ll_newlist(length) + + def _RPyListOfString_SetItem(l=p, + index=lltype.Signed, + newstring=lltype.Ptr(STR)): + rlist.ll_setitem_nonneg(rlist.dum_nocheck, l, index, newstring) for fname, f in locals().items(): if isinstance(f, types.FunctionType): From ac at codespeak.net Tue Jan 24 14:53:54 2006 From: ac at codespeak.net (ac at codespeak.net) Date: Tue, 24 Jan 2006 14:53:54 +0100 (CET) Subject: [pypy-svn] r22592 - in pypy/dist/pypy/rpython/l3interp: . test Message-ID: <20060124135354.E48DF27DB8@code1.codespeak.net> Author: ac Date: Tue Jan 24 14:53:54 2006 New Revision: 22592 Modified: pypy/dist/pypy/rpython/l3interp/convertgraph.py pypy/dist/pypy/rpython/l3interp/l3interp.py pypy/dist/pypy/rpython/l3interp/model.py pypy/dist/pypy/rpython/l3interp/test/test_l3interp.py Log: (mwh, arre) Implement setarrayitem + tests. Modified: pypy/dist/pypy/rpython/l3interp/convertgraph.py ============================================================================== --- pypy/dist/pypy/rpython/l3interp/convertgraph.py (original) +++ pypy/dist/pypy/rpython/l3interp/convertgraph.py Tue Jan 24 14:53:54 2006 @@ -152,6 +152,20 @@ offset = ItemOffset(spaceop.result.concretetype) insns.append(getoffset(offset)) + elif spaceop.opname == 'setarrayitem': + array, index, value = spaceop.args + opname = spaceop.opname + '_' + \ + getaccesskind(value.concretetype) + insns.append(model.very_low_level_opcode[opname]) + insns.append(get(array)) + insns.append(get(index)) + + offset = ArrayItemsOffset(array.concretetype) + insns.append(getoffset(offset)) + + offset = ItemOffset(value.concretetype) + insns.append(getoffset(offset)) + insns.append(get(value)) else: insns.append(model.very_low_level_opcode[spaceop.opname]) for v in spaceop.args: Modified: pypy/dist/pypy/rpython/l3interp/l3interp.py ============================================================================== --- pypy/dist/pypy/rpython/l3interp/l3interp.py (original) +++ pypy/dist/pypy/rpython/l3interp/l3interp.py Tue Jan 24 14:53:54 2006 @@ -212,6 +212,30 @@ v = (a + items_offset + s * i).signed[0] self.stack_int.append(v) + def op_getarrayitem_ptr(self): + a = self.getptr() + i = self.getint() + items_offset = self.getoffset() + s = self.getoffset() + v = (a + items_offset + s * i).address[0] + self.stack_ptr.append(v) + + def op_setarrayitem_int(self): + a = self.getptr() + i = self.getint() + items_offset = self.getoffset() + s = self.getoffset() + v = self.getint() + (a + items_offset + s * i).signed[0] = v + + def op_setarrayitem_ptr(self): + a = self.getptr() + i = self.getint() + items_offset = self.getoffset() + s = self.getoffset() + v = self.getptr() + (a + items_offset + s * i).address[0] = v + def op_flavored_malloc(self): self.stack_ptr.append(constant_fakeaddress) Modified: pypy/dist/pypy/rpython/l3interp/model.py ============================================================================== --- pypy/dist/pypy/rpython/l3interp/model.py (original) +++ pypy/dist/pypy/rpython/l3interp/model.py Tue Jan 24 14:53:54 2006 @@ -38,7 +38,8 @@ #array operations: 'getarrayitem_int', 'getarrayitem_ptr', - 'getarraysize', 'getarraysubstruct', 'setarrayitem', + 'getarraysize', 'getarraysubstruct', + 'setarrayitem_int', ' setarrayitem_ptr', #struct operations: 'getfield_int', 'getfield_char', 'getfield_dbl', 'getfield_ptr', Modified: pypy/dist/pypy/rpython/l3interp/test/test_l3interp.py ============================================================================== --- pypy/dist/pypy/rpython/l3interp/test/test_l3interp.py (original) +++ pypy/dist/pypy/rpython/l3interp/test/test_l3interp.py Tue Jan 24 14:53:54 2006 @@ -189,8 +189,8 @@ fn = translate(entry_point, [int]) - assert fn(-3) == 0 - assert fn(0) == -3 + assert fn(9) == 0 + assert fn(0) == 9 @@ -266,3 +266,71 @@ for arg in 0,1,2: assert fn(arg) == f(arg) +def test_setitem(): + from pypy.rpython.lltypesystem import lltype + + A = lltype.GcArray(lltype.Signed) + a = lltype.malloc(A, 3) + a[0] = 1 + a[1] = 2 + a[2] = 3 + + + def f(index, value): + if value: + a[index] = value + return 0 + return a[index] + + l3graph = l3ify(f, [int, int]) + def entry_point(x, y): + value = l3interp.l3interpret(l3graph, [x, y], [], []) + assert isinstance(value, l3interp.L3Integer) + return value.intval + + entry_point(1, 42) + assert entry_point(1, 0) == 42 + + fn = translate(entry_point, [int, int]) + + entry_point(1, 65) + assert entry_point(1, 0) == 65 + +def test_getsetitem_complex(): + from pypy.rpython.lltypesystem import lltype + + A = lltype.GcArray(lltype.Signed) + AA = lltype.GcArray(lltype.Ptr(A)) + a = lltype.malloc(AA, 3) + a[0] = lltype.malloc(A, 1) + a[1] = lltype.malloc(A, 2) + a[2] = lltype.malloc(A, 3) + a[0][0] = 1 + + a[1][0] = 2 + a[1][1] = 3 + + a[2][0] = 4 + a[2][1] = 5 + a[2][2] = 6 + + def f(x, y, value): + if value: + a[x][y] = value + return 0 + return a[x][y] + + l3graph = l3ify(f, [int, int, int]) + def entry_point(x, y, value): + value = l3interp.l3interpret(l3graph, [x, y, value], [], []) + assert isinstance(value, l3interp.L3Integer) + return value.intval + + for x,y,value in (0,0,0), (0,0,9), (0,0,0), (2,1,0), (2,1,14), (2,1,0): + assert entry_point(x, y, value) == f(x, y, value) + + + fn = translate(entry_point, [int, int, int]) + + for x,y,value in (0,0,0), (0,0,11), (0,0,0), (2,1,0), (2,1,26), (2,1,0): + assert fn(x, y, value) == f(x, y, value) From auc at codespeak.net Tue Jan 24 14:55:32 2006 From: auc at codespeak.net (auc at codespeak.net) Date: Tue, 24 Jan 2006 14:55:32 +0100 (CET) Subject: [pypy-svn] r22593 - pypy/dist/pypy/lib/logic Message-ID: <20060124135532.B16B427DB8@code1.codespeak.net> Author: auc Date: Tue Jan 24 14:55:30 2006 New Revision: 22593 Modified: pypy/dist/pypy/lib/logic/test_unification.py pypy/dist/pypy/lib/logic/unification.py Log: (auc, ale) * add satisfiability check returning narrowed domains * more tests Modified: pypy/dist/pypy/lib/logic/test_unification.py ============================================================================== --- pypy/dist/pypy/lib/logic/test_unification.py (original) +++ pypy/dist/pypy/lib/logic/test_unification.py Tue Jan 24 14:55:30 2006 @@ -298,6 +298,17 @@ assert y.dom == c.FiniteDomain([2, 3]) assert z.dom == c.FiniteDomain([3, 4]) assert w.dom == c.FiniteDomain([1, 4, 5]) + narrowed_doms = u.get_satisfying_domains(k1) + assert narrowed_doms == {x:c.FiniteDomain([5]), + y:c.FiniteDomain([2]), + z:c.FiniteDomain([3]), + w:c.FiniteDomain([4, 5])} + narrowed_doms = u.get_satisfying_domains(k2) + assert narrowed_doms == {x:c.FiniteDomain([5]), + y:c.FiniteDomain([2]), + z:c.FiniteDomain([3]), + w:c.FiniteDomain([4, 5])} + def test_satisfy_many_const_failure(self): x,y,z,w = (u.var('x'), u.var('y'), @@ -320,4 +331,9 @@ assert y.dom == c.FiniteDomain([2, 3]) assert z.dom == c.FiniteDomain([3, 4]) assert w.dom == c.FiniteDomain([1]) + narrowed_doms = u.get_satisfying_domains(k1) + assert narrowed_doms == {} + narrowed_doms = u.get_satisfying_domains(k2) + assert narrowed_doms == {} + Modified: pypy/dist/pypy/lib/logic/unification.py ============================================================================== --- pypy/dist/pypy/lib/logic/unification.py (original) +++ pypy/dist/pypy/lib/logic/unification.py Tue Jan 24 14:55:30 2006 @@ -220,14 +220,8 @@ varset = set() constset = set() compute_dependant_vars(constraint, varset, constset) - old_domains = {} - for var in varset: - old_domains[var] = FiniteDomain(var.dom) - - def restore_domains(domains): - for var, dom in domains.items(): - var.dom = dom - + old_domains = collect_domains(varset) + for const in constset: try: const.narrow() @@ -237,6 +231,27 @@ restore_domains(old_domains) return True + + def get_satisfying_domains(self, constraint): + assert constraint in self.constraints + varset = set() + constset = set() + compute_dependant_vars(constraint, varset, constset) + old_domains = collect_domains(varset) + + for const in constset: + try: + const.narrow() + except ConsistencyFailure: + restore_domains(old_domains) + return {} + narrowed_domains = collect_domains(varset) + restore_domains(old_domains) + return narrowed_domains + + def satisfy(constraint): + pass + #-- BIND ------------------------------------------- def bind(self, var, val): @@ -393,6 +408,25 @@ compute_dependant_vars(const, varset, constset) +#-- collect / restore utilities for domains + +def collect_domains(varset): + """makes a copy of domains of a set of vars + into a var -> dom mapping + """ + dom = {} + for var in varset: + dom[var] = FiniteDomain(var.dom) + return dom + +def restore_domains(domains): + """sets the domain of the vars in the domains mapping + to their (previous) value + """ + for var, dom in domains.items(): + var.dom = dom + + #-- Unifiability checks--------------------------------------- #-- #-- quite costly & could be merged back in unify @@ -475,6 +509,9 @@ def satisfiable(constraint): return _store.satisfiable(constraint) +def get_satisfying_domains(constraint): + return _store.get_satisfying_domains(constraint) + def bind(var, val): return _store.bind(var, val) From gromit at codespeak.net Tue Jan 24 14:58:28 2006 From: gromit at codespeak.net (gromit at codespeak.net) Date: Tue, 24 Jan 2006 14:58:28 +0100 (CET) Subject: [pypy-svn] r22594 - in pypy/dist/pypy/rpython/rctypes: . test Message-ID: <20060124135828.D4FD327B5C@code1.codespeak.net> Author: gromit Date: Tue Jan 24 14:58:26 2006 New Revision: 22594 Modified: pypy/dist/pypy/rpython/rctypes/implementation.py pypy/dist/pypy/rpython/rctypes/interface.py pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py Log: (stephan,gromit) ADD: Specialization support, some additional types at the interface. Modified: pypy/dist/pypy/rpython/rctypes/implementation.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/implementation.py (original) +++ pypy/dist/pypy/rpython/rctypes/implementation.py Tue Jan 24 14:58:26 2006 @@ -5,11 +5,14 @@ from ctypes import * from ctypes import _FUNCFLAG_CDECL, _FUNCFLAG_STDCALL from pypy.annotation.model import SomeInteger +from pypy.rpython.lltypesystem.lltype import Signed + c_int.annotator_type = SomeInteger() +c_int.ll_type = Signed -class FunctionPointerAnnotation(object): +class FunctionPointerTranslation(object): def compute_result_annotation(self, *args_s): """ @@ -17,13 +20,20 @@ """ return self.restype.annotator_type + def __hash__(self): + return id(self) + + def specialize(self, hop): + return hop.llops.gencapicall(self.__name__, hop.args_v[1:], + resulttype=self.restype.ll_type, _callable=None) + class RCDLL(CDLL): """ This is the restricted version of ctypes' CDLL class. """ - class _CdeclFuncPtr(FunctionPointerAnnotation, CDLL._CdeclFuncPtr): + class _CdeclFuncPtr(FunctionPointerTranslation, CDLL._CdeclFuncPtr): """ A simple extension of ctypes function pointers that implements a simple interface to the anotator. @@ -37,7 +47,7 @@ This is the restricted version of ctypes' WINDLL class """ - class _StdcallFuncPtr(FunctionPointerAnnotation, WinDLL._StdcallFuncPtr): + class _StdcallFuncPtr(FunctionPointerTranslation, WinDLL._StdcallFuncPtr): """ A simple extension of ctypes function pointers that implements a simple interface to the anotator. Modified: pypy/dist/pypy/rpython/rctypes/interface.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/interface.py (original) +++ pypy/dist/pypy/rpython/rctypes/interface.py Tue Jan 24 14:58:26 2006 @@ -1,5 +1,5 @@ from ctypes import _DLLS -from implementation import RCDLL as CDLL, RWinDLL as WinDLL, c_int, c_char_p +from implementation import RCDLL as CDLL, RWinDLL as WinDLL, c_int, c_char_p, c_char, POINTER cdll = _DLLS( CDLL ) windll = _DLLS( WinDLL ) Modified: pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py Tue Jan 24 14:58:26 2006 @@ -7,7 +7,7 @@ py.test.skip("this test needs ctypes installed") else: import sys - from pypy.rpython.rctypes.interface import cdll, c_char_p, c_int + from pypy.rpython.rctypes.interface import cdll, c_char_p, c_int, c_char, POINTER if sys.platform == 'win32': mylib = cdll.LoadLibrary('msvcrt.dll') elif sys.platform == 'linux2': @@ -19,6 +19,7 @@ atoi = mylib.atoi atoi.restype = c_int atoi.argstype = [c_char_p] + atoi.argstype = [POINTER(c_char)] def o_atoi(a): return atoi(a) mod.o_atoi = o_atoi @@ -27,6 +28,7 @@ class Test_rctypes: from pypy.annotation.annrpython import RPythonAnnotator + from pypy.translator.translator import TranslationContext def test_simple(self): @@ -34,8 +36,17 @@ res = o_atoi('42') assert res == 42 - def inprogress_test_annotate_simple(self): + def test_annotate_simple(self): a = self.RPythonAnnotator() s = a.build_types(o_atoi, [str]) # result should be an integer assert s.knowntype == int + + def test_specialize_simple(self): + t = self.TranslationContext() + a = t.buildannotator() + s = a.build_types(o_atoi, [str]) + # result should be an integer + assert s.knowntype == int + t.buildrtyper().specialize() + #d#t.view() From auc at codespeak.net Tue Jan 24 15:22:06 2006 From: auc at codespeak.net (auc at codespeak.net) Date: Tue, 24 Jan 2006 15:22:06 +0100 (CET) Subject: [pypy-svn] r22595 - pypy/dist/pypy/lib/logic Message-ID: <20060124142206.63E9427DB6@code1.codespeak.net> Author: auc Date: Tue Jan 24 15:22:00 2006 New Revision: 22595 Modified: pypy/dist/pypy/lib/logic/constraint.py pypy/dist/pypy/lib/logic/test_unification.py pypy/dist/pypy/lib/logic/unification.py Log: (auc, ale) * add satisfy method on (store, constraint) which updates the store (narrows domains) * more tests Modified: pypy/dist/pypy/lib/logic/constraint.py ============================================================================== --- pypy/dist/pypy/lib/logic/constraint.py (original) +++ pypy/dist/pypy/lib/logic/constraint.py Tue Jan 24 15:22:00 2006 @@ -111,7 +111,7 @@ def intersection(self, other): if other is None: return self.get_values() - return self.get_values() & other.get_values() + return self._values & other._values #-- Constraints ------------------------------------------ Modified: pypy/dist/pypy/lib/logic/test_unification.py ============================================================================== --- pypy/dist/pypy/lib/logic/test_unification.py (original) +++ pypy/dist/pypy/lib/logic/test_unification.py Tue Jan 24 15:22:00 2006 @@ -277,7 +277,7 @@ assert y.dom == c.FiniteDomain([2, 3]) assert z.dom == c.FiniteDomain([3, 4]) - def test_satisfy_many_const_success(self): + def test_satisfiable_many_const_success(self): x,y,z,w = (u.var('x'), u.var('y'), u.var('z'), u.var('w')) x.dom = c.FiniteDomain([1, 2, 5]) @@ -310,7 +310,7 @@ w:c.FiniteDomain([4, 5])} - def test_satisfy_many_const_failure(self): + def test_satisfiable_many_const_failure(self): x,y,z,w = (u.var('x'), u.var('y'), u.var('z'), u.var('w')) x.dom = c.FiniteDomain([1, 2, 5]) @@ -335,5 +335,42 @@ assert narrowed_doms == {} narrowed_doms = u.get_satisfying_domains(k2) assert narrowed_doms == {} + + def test_satisfy_many_const_failure(self): + x,y,z,w = (u.var('x'), u.var('y'), + u.var('z'), u.var('w')) + x.dom = c.FiniteDomain([1, 2, 5]) + y.dom = c.FiniteDomain([2, 3]) + z.dom = c.FiniteDomain([3, 4]) + w.dom = c.FiniteDomain([1]) + k1 = c.Expression([x, y, z], 'x == y + z') + k2 = c.Expression([z, w], 'z < w') + u.add_constraint(k1) + u.add_constraint(k2) + raises(u.ConsistencyFailure, u.satisfy, k1) + assert x.dom == c.FiniteDomain([1, 2, 5]) + assert y.dom == c.FiniteDomain([2, 3]) + assert z.dom == c.FiniteDomain([3, 4]) + assert w.dom == c.FiniteDomain([1]) + raises(u.ConsistencyFailure, u.satisfy, k2) + assert x.dom == c.FiniteDomain([1, 2, 5]) + assert y.dom == c.FiniteDomain([2, 3]) + assert z.dom == c.FiniteDomain([3, 4]) + assert w.dom == c.FiniteDomain([1]) - + def test_satisfy_many_const_success(self): + x,y,z,w = (u.var('x'), u.var('y'), + u.var('z'), u.var('w')) + x.dom = c.FiniteDomain([1, 2, 5]) + y.dom = c.FiniteDomain([2, 3]) + z.dom = c.FiniteDomain([3, 4]) + w.dom = c.FiniteDomain([1, 4, 5]) + k1 = c.Expression([x, y, z], 'x == y + z') + k2 = c.Expression([z, w], 'z < w') + u.add_constraint(k1) + u.add_constraint(k2) + u.satisfy(k2) + assert x.dom == c.FiniteDomain([5]) + assert y.dom == c.FiniteDomain([2]) + assert z.dom == c.FiniteDomain([3]) + assert w.dom == c.FiniteDomain([4, 5]) Modified: pypy/dist/pypy/lib/logic/unification.py ============================================================================== --- pypy/dist/pypy/lib/logic/unification.py (original) +++ pypy/dist/pypy/lib/logic/unification.py Tue Jan 24 15:22:00 2006 @@ -249,9 +249,21 @@ restore_domains(old_domains) return narrowed_domains - def satisfy(constraint): - pass + def satisfy(self, constraint): + assert constraint in self.constraints + varset = set() + constset = set() + compute_dependant_vars(constraint, varset, constset) + old_domains = collect_domains(varset) + for const in constset: + try: + const.narrow() + except ConsistencyFailure: + restore_domains(old_domains) + raise + + #-- BIND ------------------------------------------- def bind(self, var, val): @@ -512,6 +524,9 @@ def get_satisfying_domains(constraint): return _store.get_satisfying_domains(constraint) +def satisfy(constraint): + return _store.satisfy(constraint) + def bind(var, val): return _store.bind(var, val) From gromit at codespeak.net Tue Jan 24 15:32:01 2006 From: gromit at codespeak.net (gromit at codespeak.net) Date: Tue, 24 Jan 2006 15:32:01 +0100 (CET) Subject: [pypy-svn] r22596 - pypy/dist/pypy/annotation Message-ID: <20060124143201.0994A27DB5@code1.codespeak.net> Author: gromit Date: Tue Jan 24 15:31:59 2006 New Revision: 22596 Modified: pypy/dist/pypy/annotation/bookkeeper.py Log: FIX: (stephan,gromit) ctypes integration. Modified: pypy/dist/pypy/annotation/bookkeeper.py ============================================================================== --- pypy/dist/pypy/annotation/bookkeeper.py (original) +++ pypy/dist/pypy/annotation/bookkeeper.py Tue Jan 24 15:31:59 2006 @@ -355,7 +355,7 @@ _module = getattr(x,"__module__","unknown") result = SomeBuiltin(BUILTIN_ANALYZERS[x], methodname="%s.%s" % (_module, x.__name__)) elif hasattr(x, "compute_result_annotation"): - result = SomeBuiltin(x.compute_result_annotation, "?" ) + result = SomeBuiltin(x.compute_result_annotation, methodname=x.__name__) elif tp in EXTERNAL_TYPE_ANALYZERS: result = SomeExternalObject(tp) elif isinstance(x, lltype._ptr): From gromit at codespeak.net Tue Jan 24 15:33:33 2006 From: gromit at codespeak.net (gromit at codespeak.net) Date: Tue, 24 Jan 2006 15:33:33 +0100 (CET) Subject: [pypy-svn] r22597 - pypy/dist/pypy/rpython Message-ID: <20060124143333.5A6FF27DB6@code1.codespeak.net> Author: gromit Date: Tue Jan 24 15:33:31 2006 New Revision: 22597 Modified: pypy/dist/pypy/rpython/rbuiltin.py Log: ADD: (stephan,gromit) Add support for ctypes. Modified: pypy/dist/pypy/rpython/rbuiltin.py ============================================================================== --- pypy/dist/pypy/rpython/rbuiltin.py (original) +++ pypy/dist/pypy/rpython/rbuiltin.py Tue Jan 24 15:33:31 2006 @@ -58,8 +58,11 @@ bltintyper = rtyper.type_system.rbuiltin.\ BUILTIN_TYPER[self.builtinfunc] except KeyError: - raise TyperError("don't know about built-in function %r" % ( - self.builtinfunc,)) + if hasattr(self.builtinfunc,"specialize"): + bltintyper = self.builtinfunc.specialize + else: + raise TyperError("don't know about built-in function %r" % ( + self.builtinfunc,)) hop2 = hop.copy() hop2.r_s_popfirstarg() return bltintyper(hop2) From auc at codespeak.net Tue Jan 24 15:57:32 2006 From: auc at codespeak.net (auc at codespeak.net) Date: Tue, 24 Jan 2006 15:57:32 +0100 (CET) Subject: [pypy-svn] r22598 - pypy/dist/pypy/lib/logic Message-ID: <20060124145732.8849527DB5@code1.codespeak.net> Author: auc Date: Tue Jan 24 15:57:31 2006 New Revision: 22598 Added: pypy/dist/pypy/lib/logic/computationspace.py Log: (auc, ale) add computation space 'manifesto' Added: pypy/dist/pypy/lib/logic/computationspace.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/lib/logic/computationspace.py Tue Jan 24 15:57:31 2006 @@ -0,0 +1,105 @@ +## The Constraint-based computation model +## -------------------------------------- + +## A computation space collects together basic constraints (in fact +## variable domains) and propagators (aka constraints), and puts them +## inside an encapsulation boundary. A propagator is simply a thread that +## can add new basic constraints to the store. A computation space is +## always created inside a parent space; it can see the constraints of +## its parent. + +## Basic constraints and propagators +## --------------------------------- + +## Basic constraints are constraints that are directly represented in the +## single-assignment store. An example of a basic constraint is a binding +## of a dataflow variable, such as x = person(age:y). This gives partial +## information about x. (SEE CHAPTER 2) The binding represents an +## equality constraint between the dataflow variable and a partial value. + +## Here, we extend the store with a new kind of basic constraint, namely +## membership in a finite domain. A finite domain is a finite set of +## integers. The new basic constraint is an equation of the form x in D, +## where D is a finite domain. This gives partial information about +## x. This basic constraint is added to the store by the statement +## x::D. Note that the finite domain constraint x in {n} is equivalent to +## the binding x=n. + +## Adding the constraints x::D1, x::D2, ..., x::Dn restricts the domain +## of x to the intersection of the n domains, provided that the latter is +## nonempty. Adding a constraint with an empty domain for a variable +## would result in an inconsistent store. + +## The single-assignment store seen previously can be considered as a +## constraint store. The basic constraints it represent are all of the +## form x=y, where x and y are partial values. We call these constraints +## 'bindings', althhough they can do more than simply bind a variable to +## a value. The general operation of binding two partial values together +## is called unification. + +## What are all the complete values to which a fresh variable can be +## bound ? This is not an obvious question since we can add cyclic +## bindings to the store, e.g X=foo(X). It turns out that variable can be +## bound to rational trees. A rational tree generalizes the finite trees +## to allow a certain kind of infinite trees that can be stored in a +## finite space. A rational tree can be represented as a root directed +## graph. Unfolding the graph to remove cycles and sharing yields the +## tree. For example the graph X=foo(X) represents the tree +## X=foo(foo(...)). + +## When a variable is initially declared, it can potentially be bound to +## any rational tree. Each basic constraint (i.e each binding) that we +## add to the store restricts the set of rational trees to which the +## variable can be bound. For example, adding the constraint +## x=person(age:y) restricts the possible bindings of x, namely to +## rational trees whose root is a record of label person with one field +## whose feature name is age. The single-assignment store therefore +## implements constraints over rational trees. + +## The set of complete values to which a variable can be bound is called +## its domain. We talk here about constraints over two domains: rational +## trees and finite domains (the Mozart system implements finite sets of +## integer, fine-grained records and real interval). + +## A propagator is a thread that continually observes the store and +## occasionally adds new basic constraints to the store. Each time a +## variable's domain is changes into the store, the propagators that use +## that variable must be given a chance to execute, so they can propagate +## new partial information ot variables. Wainting for a domain change is +## fine-grained variant of waiting for determinacy. + +## The execution of these propagators must be order-independant. + +## Programming searches with computation spaces +## -------------------------------------------- + +## We outline how computation spaces are used to implement search and +## distribution strategies. A search strategy defines how the search tree +## is explored (ie breadth-first,...). A distribution strategy defines +## the shape and content of the search tree, i.e how many alternatives +## exist at a node and what constraint is added for each +## alternative. Computation spaces can be used to program search +## strategies and distribution strategies independant of each other. How +## it is done : + +## * create the space with the correct program inside. This program +## defines all the variable and constraints in the space. + +## * let the program run into the space. Variables and propagators are +## created. All propagators execute until no more information can be +## added to the store in this manner. The space eventually reaches +## stability. + +## * during the space's execution, the computation inside the space can +## decide to create a choice point. The decision of which constraint to +## add for each alternative defines the distribution strategy. One of +## the space's thread will suspend when the choice point is created. + +## * when the space has become stable, execution continues outside the +## space, to decide what to do next. There are different possibilities +## depending on whether or not a choice point has been created in that +## space. If there is none, then execution can stop and return with a +## solution. If there is one, then the search strategy decides which +## alternative to choose and commits to that alternative. + + From ac at codespeak.net Tue Jan 24 16:03:11 2006 From: ac at codespeak.net (ac at codespeak.net) Date: Tue, 24 Jan 2006 16:03:11 +0100 (CET) Subject: [pypy-svn] r22599 - pypy/dist/pypy/rpython/lltypesystem Message-ID: <20060124150311.69B7027DB9@code1.codespeak.net> Author: ac Date: Tue Jan 24 16:03:11 2006 New Revision: 22599 Modified: pypy/dist/pypy/rpython/lltypesystem/llmemory.py Log: (mwh, arre) Implement sketches of sizeof and itemoffsetof. These can not be reasonably annotated and rtyped. Modified: pypy/dist/pypy/rpython/lltypesystem/llmemory.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/llmemory.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/llmemory.py Tue Jan 24 16:03:11 2006 @@ -89,12 +89,24 @@ def sizeof(TYPE, n=None): - pass - + if n is None: + assert not TYPE._is_varsize() + return ItemOffset(TYPE) + else: + if isinstance(TYPE, Array): + return itemoffsetof(TYPE, n) + elif isinstance(TYPE, Struct): + return FieldOffset(TYPE, TYPE._arrayfld) + \ + itemoffsetof(TYPE._flds[TYPE._arrayfld], n) + else: + raise Exception("don't know how to take the size of a %r"%TYPE) + def offsetof(TYPE, fldname): assert fldname in TYPE._flds return FieldOffset(TYPE, fldname) +def itemoffsetof(TYPE, n=0): + return ArrayItemsOffset(TYPE) + ItemOffset(TYPE.OF) * n # ------------------------------------------------------------- class fakeaddress(object): From stephan at codespeak.net Tue Jan 24 16:32:11 2006 From: stephan at codespeak.net (stephan at codespeak.net) Date: Tue, 24 Jan 2006 16:32:11 +0100 (CET) Subject: [pypy-svn] r22600 - in pypy/dist/pypy: rpython/rctypes rpython/rctypes/test translator/c Message-ID: <20060124153211.4160C27DB5@code1.codespeak.net> Author: stephan Date: Tue Jan 24 16:32:08 2006 New Revision: 22600 Modified: pypy/dist/pypy/rpython/rctypes/implementation.py pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py pypy/dist/pypy/translator/c/external.py Log: intermediate checkin for rpython/rctypes Modified: pypy/dist/pypy/rpython/rctypes/implementation.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/implementation.py (original) +++ pypy/dist/pypy/rpython/rctypes/implementation.py Tue Jan 24 16:32:08 2006 @@ -24,8 +24,12 @@ return id(self) def specialize(self, hop): - return hop.llops.gencapicall(self.__name__, hop.args_v[1:], - resulttype=self.restype.ll_type, _callable=None) + return hop.llops.gencapicall(self.__name__, hop.args_v, + resulttype=self.restype.ll_type, _callable=None, + convert_params=self.convert_params) + + def convert_params(self, backend, param_info_list): + raise NotImplementedError class RCDLL(CDLL): Modified: pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py Tue Jan 24 16:32:08 2006 @@ -1,4 +1,8 @@ import py.test +from pypy.annotation.annrpython import RPythonAnnotator +from pypy.translator.translator import TranslationContext +from pypy.translator.c.test.test_genc import compile + def setup_module(mod): try: @@ -27,9 +31,6 @@ class Test_rctypes: - from pypy.annotation.annrpython import RPythonAnnotator - from pypy.translator.translator import TranslationContext - def test_simple(self): @@ -37,16 +38,22 @@ assert res == 42 def test_annotate_simple(self): - a = self.RPythonAnnotator() + a = RPythonAnnotator() s = a.build_types(o_atoi, [str]) # result should be an integer assert s.knowntype == int def test_specialize_simple(self): - t = self.TranslationContext() + t = TranslationContext() a = t.buildannotator() s = a.build_types(o_atoi, [str]) # result should be an integer assert s.knowntype == int t.buildrtyper().specialize() #d#t.view() + + def x_test_compile_simple(self): + fn = compile(o_atoi, [str]) + res = fn("42") + assert res == 42 + Modified: pypy/dist/pypy/translator/c/external.py ============================================================================== --- pypy/dist/pypy/translator/c/external.py (original) +++ pypy/dist/pypy/translator/c/external.py Tue Jan 24 16:32:08 2006 @@ -31,7 +31,11 @@ yield '%s;' % cdecl(self.resulttypename, 'result') def cfunction_body(self): - call = '%s(%s)' % (self.fnptr._name, ', '.join(self.argnames())) + try: + convert_params = self.fnptr.convert_params + except AttributeError: + convert_params = lambda backend, args: [arg for _,arg in args] + call = '%s(%s)' % (self.fnptr._name, ', '.join(convert_params("c", zip(self.FUNCTYPE.ARGS, self.argnames())))) if self.FUNCTYPE.RESULT is not Void: yield 'result = %s;' % call yield 'if (PyErr_Occurred()) RPyConvertExceptionFromCPython();' From cfbolz at codespeak.net Tue Jan 24 16:56:44 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Tue, 24 Jan 2006 16:56:44 +0100 (CET) Subject: [pypy-svn] r22602 - pypy/dist/pypy/jit Message-ID: <20060124155644.D8A3F27DB5@code1.codespeak.net> Author: cfbolz Date: Tue Jan 24 16:56:43 2006 New Revision: 22602 Modified: pypy/dist/pypy/jit/llabstractinterp.py Log: (cfbolz, arigo) Started to convert llabstractinterp in the direction required by the operation-generating functions -- mostly, by removing the introspection of the generated blocks that llabstractinterp used to do all over the place. Modified: pypy/dist/pypy/jit/llabstractinterp.py ============================================================================== --- pypy/dist/pypy/jit/llabstractinterp.py (original) +++ pypy/dist/pypy/jit/llabstractinterp.py Tue Jan 24 16:56:43 2006 @@ -161,7 +161,6 @@ def __init__(self, policy=best_policy): self.graphs = [] self.graphstates = {} # {origgraph: {BlockState: GraphState}} - self.pendingstates = {} # {Link-or-GraphState: next-BlockState} self.blocks = {} # {BlockState.key(): list-of-LLBlockStates} self.policy = policy @@ -189,9 +188,8 @@ graphstate = self.graphstates[origgraph][frozenstate] except KeyError: d = self.graphstates.setdefault(origgraph, {}) - graphstate = GraphState(self, origgraph, inputstate, n=len(d)) + graphstate = GraphState(self, origgraph, inputstate, frozenstate, n=len(d)) d[frozenstate] = graphstate - self.pendingstates[graphstate] = frozenstate #print "SCHEDULE_GRAPH", graphstate return graphstate @@ -199,9 +197,7 @@ #print "SCHEDULE", args_a, origblock frozenstate = self.schedule_getstate(inputstate) args_v = inputstate.getruntimevars() - newlink = Link(args_v, None) - self.pendingstates[newlink] = frozenstate - return newlink + return LinkState(args_v, frozenstate) def schedule_getstate(self, inputstate): # NOTA BENE: copyblocks can get shared between different copygraphs! @@ -252,10 +248,22 @@ raise HintError("cannot trace the origin of %r" % (a,)) +class LinkState(object): + """Wrapper for a residual link and the frozen state that it should go to.""" + + def __init__(self, args_v, frozenstate): + self.args_v = args_v + self.link = Link(args_v, None) + self.frozenstate = frozenstate + + def settarget(self, block): + self.link.settarget(block) + + class GraphState(object): """Entry state of a graph.""" - def __init__(self, interp, origgraph, inputstate, n): + def __init__(self, interp, origgraph, inputstate, frozenstate, n): self.interp = interp self.origgraph = origgraph name = '%s_%d' % (origgraph.name, n) @@ -269,9 +277,10 @@ self.copygraph.exceptblock.inputargs[1])]: if hasattr(orig_v, 'concretetype'): copy_v.concretetype = orig_v.concretetype - # The 'args' attribute is needed by try_to_complete(), - # which looks for it on either a GraphState or a Link - self.args = inputstate.getruntimevars() + # The 'args_v' attribute is needed by try_to_complete(), + # which looks for it on either a GraphState or a LinkState + self.args_v = inputstate.getruntimevars() + self.frozenstate = frozenstate #self.a_return = None self.state = "before" @@ -295,14 +304,14 @@ def try_to_complete(self): graph = self.copygraph interp = self.interp - pending = [self] + pending = [self] # pending list of a GraphState and many LinkStates seen = {} did_any_generalization = False # follow all possible links, forcing the blocks along the way to be # computed while pending: - next = pending.pop() - state = interp.pendingstates[next] + next = pending.pop() # a GraphState or a LinkState + state = next.frozenstate # debugging: print the current call stack print @@ -342,26 +351,26 @@ # that links with different *fixed* constants don't interfere # with each other. - key = tuple(state.enum_fixed_constants(next.args)) + key = tuple(state.enum_fixed_constants(next.args_v)) try: - block = state.copyblocks[key] + block, blockargs, blockexits = state.copyblocks[key] except KeyError: if interp.policy.const_propagate: - # originally, propagate all constants from next.args + # originally, propagate all constants from next.args_v # optimistically to the new block - initial_key = next.args + initial_key = next.args_v else: # don't propagate anything more than required ('fixed') initial_key = key print 'flowin() with initial key', initial_key - block = self.flowin(state, initial_key) - state.copyblocks[key] = block + block, blockargs, blockexits = self.flowin(state, initial_key) + state.copyblocks[key] = block, blockargs, blockexits else: # check if the tentative constants of the existing block # are compatible with the ones specified by the new link merged_key = [] recompute = False - for c1, c2 in zip(block.inputargs, next.args): + for c1, c2 in zip(blockargs, next.args_v): if isinstance(c1, Constant) and c1 != c2: # incompatibility merged_key.append(None) # force a Variable @@ -370,24 +379,25 @@ merged_key.append(c1) # unmodified if recompute: print 'flowin() merged as', merged_key - block = self.flowin(state, merged_key) - state.copyblocks[key] = block + block, blockargs, blockexits = self.flowin(state, merged_key) + state.copyblocks[key] = block, blockargs, blockexits did_any_generalization = True next.settarget(block) - for link in block.exits: - if link.target is None or link.target.operations != (): - if link not in seen: - seen[link] = True - pending.append(link) + for ls in blockexits: + if ls.frozenstate is not None: + if ls not in seen: + seen[ls] = True + pending.append(ls) else: - # link.target is a return or except block; make sure - # that it is really the one from 'graph' -- by patching - # 'graph' if necessary. - if len(link.target.inputargs) == 1: + # 'ls' is the linkstate of a link to a return or except + # block; make sure that it is really the one from 'graph' + # -- by patching 'graph' if necessary. + # XXX don't use ls.link.target! + if len(ls.link.target.inputargs) == 1: #self.a_return = state.args_a[0] - graph.returnblock = link.target + graph.returnblock = ls.link.target elif len(link.target.inputargs) == 2: - graph.exceptblock = link.target + graph.exceptblock = ls.link.target else: raise Exception("uh?") @@ -444,7 +454,9 @@ target = self.copygraph.exceptblock args_v = [builder.binding(v).forcevarorconst(builder) for v in origblock.inputargs] - raise InsertNextLink(Link(args_v, target)) + ls = LinkState(args_v, frozenstate=None) + ls.settarget(target) + raise InsertNextLink(ls) else: # finishing a handle_call_inlining(): link back to # the parent, passing the return value @@ -473,7 +485,7 @@ except InsertNextLink, e: # the current operation forces a jump to another block - newlinks = [e.link] + newlinkstates = [e.linkstate] else: # normal path: build exit links and schedule their target for @@ -491,19 +503,19 @@ else: links = [link for link in origblock.exits if link.llexitcase == v.value] - newlinks = [] + newlinkstates = [] for origlink in links: args_a = [builder.binding(v) for v in origlink.args] nextinputstate = LLBlockState(builder.runningstate.back, args_a, origlink.target) - newlink = self.interp.schedule(nextinputstate) + newlinkstate = self.interp.schedule(nextinputstate) if newexitswitch is not None: - newlink.exitcase = origlink.exitcase - newlink.llexitcase = origlink.llexitcase - newlinks.append(newlink) + newlinkstate.link.exitcase = origlink.exitcase #XXX change later + newlinkstate.link.llexitcase = origlink.llexitcase + newlinkstates.append(newlinkstate) - newblock = builder.buildblock(newexitswitch, newlinks) - return newblock + newblock = builder.buildblock(newexitswitch, newlinkstates) + return newblock, builder.newinputargs, newlinkstates class BlockBuilder(object): @@ -519,11 +531,11 @@ self.bindings = self.runningstate.getbindings() self.residual_operations = [] - def buildblock(self, newexitswitch, newlinks): + def buildblock(self, newexitswitch, newlinkstates): b = Block(self.newinputargs) b.operations = self.residual_operations b.exitswitch = newexitswitch - b.closeblock(*newlinks) + b.closeblock(*[ls.link for ls in newlinkstates]) return b def binding(self, v): @@ -894,8 +906,8 @@ class InsertNextLink(Exception): - def __init__(self, link): - self.link = link + def __init__(self, linkstate): + self.linkstate = linkstate class RestartCompleting(Exception): pass From gromit at codespeak.net Tue Jan 24 17:06:28 2006 From: gromit at codespeak.net (gromit at codespeak.net) Date: Tue, 24 Jan 2006 17:06:28 +0100 (CET) Subject: [pypy-svn] r22603 - in pypy/dist/pypy/rpython/rctypes: . test Message-ID: <20060124160628.B8F5627DB6@code1.codespeak.net> Author: gromit Date: Tue Jan 24 17:06:26 2006 New Revision: 22603 Modified: pypy/dist/pypy/rpython/rctypes/implementation.py pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py Log: CHG: (stephan,gromit) Made c code generation work Modified: pypy/dist/pypy/rpython/rctypes/implementation.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/implementation.py (original) +++ pypy/dist/pypy/rpython/rctypes/implementation.py Tue Jan 24 17:06:26 2006 @@ -10,7 +10,14 @@ c_int.annotator_type = SomeInteger() c_int.ll_type = Signed - +c_char_p.annotator_type = None +c_char_p.ll_type = None +c_char_p.wrap_arg = staticmethod( + lambda ll_type, arg_name: "RPyString_AsString(%s)" % arg_name ) +POINTER( c_char ).annotator_type = None +POINTER( c_char ).ll_type = None +POINTER( c_char ).wrap_arg = staticmethod( + lambda ll_type, arg_name: "RPyString_AsString(%s)" % arg_name ) class FunctionPointerTranslation(object): @@ -29,7 +36,15 @@ convert_params=self.convert_params) def convert_params(self, backend, param_info_list): - raise NotImplementedError + assert "c" == backend.lower() + assert self.argtypes is not None + answer = [] + for ctype_type, (ll_type, arg_name) in zip(self.argtypes, param_info_list): + if ll_type == ctype_type.ll_type: + answer.append(arg_name) + else: + answer.append(ctype_type.wrap_arg(ll_type, arg_name)) + return answer class RCDLL(CDLL): Modified: pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py Tue Jan 24 17:06:26 2006 @@ -22,8 +22,8 @@ atoi = mylib.atoi atoi.restype = c_int - atoi.argstype = [c_char_p] - atoi.argstype = [POINTER(c_char)] + atoi.argtypes = [c_char_p] + atoi.argtypes = [POINTER(c_char)] def o_atoi(a): return atoi(a) mod.o_atoi = o_atoi @@ -52,7 +52,7 @@ t.buildrtyper().specialize() #d#t.view() - def x_test_compile_simple(self): + def test_compile_simple(self): fn = compile(o_atoi, [str]) res = fn("42") assert res == 42 From adim at codespeak.net Tue Jan 24 17:28:51 2006 From: adim at codespeak.net (adim at codespeak.net) Date: Tue, 24 Jan 2006 17:28:51 +0100 (CET) Subject: [pypy-svn] r22604 - pypy/branch/ast-experiments/pypy/module/recparser/hooksamples Message-ID: <20060124162851.8070F27DBC@code1.codespeak.net> Author: adim Date: Tue Jan 24 17:28:50 2006 New Revision: 22604 Modified: pypy/branch/ast-experiments/pypy/module/recparser/hooksamples/tracer.py Log: oops, forgot to checkin tracer update Modified: pypy/branch/ast-experiments/pypy/module/recparser/hooksamples/tracer.py ============================================================================== --- pypy/branch/ast-experiments/pypy/module/recparser/hooksamples/tracer.py (original) +++ pypy/branch/ast-experiments/pypy/module/recparser/hooksamples/tracer.py Tue Jan 24 17:28:50 2006 @@ -5,7 +5,7 @@ XXX: crashes on everything else than simple assignment (AssAttr, etc.) """ -from parser import ASTPrintnl, ASTConst, ASTName +from parser import ASTPrintnl, ASTConst, ASTName, ASTAssign from parser import install_compiler_hook, source2ast BEFORE_LOG_SOURCE = """if '%s' in locals() or '%s' in globals(): @@ -29,15 +29,18 @@ child.accept(self) return node - def visitAssign(self, assign): + def visitAssName(self, assname): + assign = assname + while not isinstance(assign, ASTAssign): + assign = assign.parent stmt = assign.parent - varname = assign.nodes[0].name + varname = assname.name before_stmts = get_statements(BEFORE_LOG_SOURCE % ((varname,) * 5)) after_stmts = get_statements(AFTER_LOG_SOURCE % (varname, varname)) stmt.insert_before(assign, before_stmts) stmt.insert_after(assign, after_stmts) - return assign - + return assname + def __getattr__(self, attrname): if attrname.startswith('visit'): return self.default From auc at codespeak.net Tue Jan 24 17:36:04 2006 From: auc at codespeak.net (auc at codespeak.net) Date: Tue, 24 Jan 2006 17:36:04 +0100 (CET) Subject: [pypy-svn] r22605 - pypy/dist/pypy/lib/logic Message-ID: <20060124163604.563B827DBC@code1.codespeak.net> Author: auc Date: Tue Jan 24 17:36:02 2006 New Revision: 22605 Modified: pypy/dist/pypy/lib/logic/computationspace.py Log: (auc, ale) * more of the manifesto * dummy computation space Modified: pypy/dist/pypy/lib/logic/computationspace.py ============================================================================== --- pypy/dist/pypy/lib/logic/computationspace.py (original) +++ pypy/dist/pypy/lib/logic/computationspace.py Tue Jan 24 17:36:02 2006 @@ -1,5 +1,5 @@ ## The Constraint-based computation model -## -------------------------------------- +## ====================================== ## A computation space collects together basic constraints (in fact ## variable domains) and propagators (aka constraints), and puts them @@ -102,4 +102,93 @@ ## solution. If there is one, then the search strategy decides which ## alternative to choose and commits to that alternative. +## Refinement +## ============ + +## a computation space is made of : + +## * a single assignment store +## * a thread store +## * a mutable store + +## operations are : + +## * newspace (creation) +## * wait_stable +## * choose +## * ask +## * commit +## * clone +## * inject +## * merge + +## Newspace +## -------- + +## newspace p : when given a one-argument procedure p, creates a new +## computation space and returns a reference to it. In this space, a +## fresh root variable r and a new thread are created, and p(r) is +## invoked in the thread. + +## There is always a top-level computatio space where threads may +## interact with the 'external' world. + +## Wait_stable +## ----------- + +## Waits until the current space becomes stable. + +## Choose +## ------ + +## Y=choose(N) waits until te current space becomes stable, blocks the +## current thread, and then creates a choice point with N alternatives in +## the current space. The blocked choose call waits for an alternative to +## be chosen by a commit operation on the space. The choose call only +## defines how many alternatives tere are; it does not specify what toi +## do for an alternative.Eventually, choose continues its execution with +## Y=I when alternative I (1= Author: gromit Date: Tue Jan 24 17:50:38 2006 New Revision: 22606 Modified: pypy/dist/pypy/rpython/rctypes/implementation.py pypy/dist/pypy/rpython/rctypes/interface.py Log: FIX: Don't import windows stuff on linux. Modified: pypy/dist/pypy/rpython/rctypes/implementation.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/implementation.py (original) +++ pypy/dist/pypy/rpython/rctypes/implementation.py Tue Jan 24 17:50:38 2006 @@ -2,8 +2,11 @@ The rctypes implementaion is contained here. """ +import sys from ctypes import * -from ctypes import _FUNCFLAG_CDECL, _FUNCFLAG_STDCALL +from ctypes import _FUNCFLAG_CDECL +if sys.platform == "win32": + from ctypes import _FUNCFLAG_STDCALL from pypy.annotation.model import SomeInteger from pypy.rpython.lltypesystem.lltype import Signed @@ -61,15 +64,16 @@ -class RWinDLL(WinDLL): - """ - This is the restricted version of ctypes' WINDLL class - """ - - class _StdcallFuncPtr(FunctionPointerTranslation, WinDLL._StdcallFuncPtr): +if sys.platform == "win32": + class RWinDLL(WinDLL): """ - A simple extension of ctypes function pointers that - implements a simple interface to the anotator. + This is the restricted version of ctypes' WINDLL class """ - _flags_ = _FUNCFLAG_STDCALL + + class _StdcallFuncPtr(FunctionPointerTranslation, WinDLL._StdcallFuncPtr): + """ + A simple extension of ctypes function pointers that + implements a simple interface to the anotator. + """ + _flags_ = _FUNCFLAG_STDCALL Modified: pypy/dist/pypy/rpython/rctypes/interface.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/interface.py (original) +++ pypy/dist/pypy/rpython/rctypes/interface.py Tue Jan 24 17:50:38 2006 @@ -1,8 +1,13 @@ from ctypes import _DLLS -from implementation import RCDLL as CDLL, RWinDLL as WinDLL, c_int, c_char_p, c_char, POINTER +from implementation import RCDLL as CDLL, c_int, c_char_p, c_char, POINTER +try: + from implementation import RWinDLL as WinDLL +except ImportError: + WinDLL = None cdll = _DLLS( CDLL ) -windll = _DLLS( WinDLL ) +if WinDLL: + windll = _DLLS( WinDLL ) """ From tismer at codespeak.net Tue Jan 24 17:57:34 2006 From: tismer at codespeak.net (tismer at codespeak.net) Date: Tue, 24 Jan 2006 17:57:34 +0100 (CET) Subject: [pypy-svn] r22607 - pypy/extradoc/sprintinfo/tokyo Message-ID: <20060124165734.0834427DBD@code1.codespeak.net> Author: tismer Date: Tue Jan 24 17:57:32 2006 New Revision: 22607 Modified: pypy/extradoc/sprintinfo/tokyo/people.txt Log: added olde tismer Modified: pypy/extradoc/sprintinfo/tokyo/people.txt ============================================================================== --- pypy/extradoc/sprintinfo/tokyo/people.txt (original) +++ pypy/extradoc/sprintinfo/tokyo/people.txt Tue Jan 24 17:57:32 2006 @@ -9,6 +9,7 @@ Name Arrive/Depart Accomodation ==================== ============== ===================== Beatrice D?ring +Christian Tismer ==================== ============== ===================== From cfbolz at codespeak.net Tue Jan 24 18:02:05 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Tue, 24 Jan 2006 18:02:05 +0100 (CET) Subject: [pypy-svn] r22608 - in pypy/dist/pypy: doc/discussion rpython rpython/test Message-ID: <20060124170205.BEC4227DBC@code1.codespeak.net> Author: cfbolz Date: Tue Jan 24 18:02:02 2006 New Revision: 22608 Modified: pypy/dist/pypy/doc/discussion/draft-jit-ideas.txt pypy/dist/pypy/rpython/rgenop.py pypy/dist/pypy/rpython/test/test_rgenop.py Log: (cfbolz, samuele, arigo) Yet another iteration of the genop() functions. Modified: pypy/dist/pypy/doc/discussion/draft-jit-ideas.txt ============================================================================== --- pypy/dist/pypy/doc/discussion/draft-jit-ideas.txt (original) +++ pypy/dist/pypy/doc/discussion/draft-jit-ideas.txt Tue Jan 24 18:02:02 2006 @@ -92,8 +92,6 @@ Functions: -* newgraph(name) -> (opaque start link object) - * newblock() -> (opaque block object) * geninputarg(block, CONCRETE_TYPE) -> (input var) @@ -102,17 +100,15 @@ * genconst(block, llvalue) -> (result var) -* newlink(block, [list-of-vars]) -> (opaque link object) - -* newreturnlink(block, return-var) -> (opaque link object) +* closeblock1(block) -> link -* closeblock1(block, link) +* closeblock2(block, exitswitch-var) -> (false_link, true_link) -* closeblock2(block, exitswitch-var, false_link, true_link) +* closelink(link, [link-vars], targetblock) -* closelink(link, targetblock) +* closereturnlink(link, return-var) -* runlink(link, [list-of-llvalues]) +* runblock(block, [list-of-llvalues]) L3 interpreter Modified: pypy/dist/pypy/rpython/rgenop.py ============================================================================== --- pypy/dist/pypy/rpython/rgenop.py (original) +++ pypy/dist/pypy/rpython/rgenop.py Tue Jan 24 18:02:02 2006 @@ -8,10 +8,6 @@ from pypy.objspace.flow import model as flowmodel -def newgraph(name): - startblock_to_throw_away = flowmodel.Block([]) # grumble - return flowmodel.FunctionGraph(name, startblock_to_throw_away) - def newblock(): return flowmodel.Block([]) @@ -33,32 +29,27 @@ v.concretetype = lltype.typeOf(llvalue) return v -def newlink(block, vars): - return flowmodel.Link(vars, None) - -def newreturnlink(block, var): - v = flowmodel.Variable() - v.concretetype = var.concretetype - pseudoreturnblock = flowmodel.Block([v]) - pseudoreturnblock.operations = () - return flowmodel.Link([var], pseudoreturnblock) - -def closeblock1(block, link): +def closeblock1(block): + link = flowmodel.Link([], None) block.closeblock(link) + return link -def closeblock2(block, exitswitch, false_link, true_link): +def closeblock2(block, exitswitch): block.exitswitch = exitswitch + false_link = flowmodel.Link([], None) false_link.exitcase = False false_link.llexitcase = False + true_link = flowmodel.Link([], None) true_link.exitcase = True true_link.llexitcase = True block.closeblock(false_link, true_link) + return false_link, true_link -def closelink(link, targetblock): +def closelink(link, vars, targetblock): if isinstance(link, flowmodel.Link): - assert link.target is None, "%r already closed" % (link,) - assert ([v.concretetype for v in link.args] == + assert ([v.concretetype for v in vars] == [v.concretetype for v in targetblock.inputargs]) + link.args[:] = vars link.target = targetblock elif isinstance(link, flowmodel.FunctionGraph): graph = link @@ -67,6 +58,13 @@ else: raise TypeError +def closereturnlink(link, returnvar): + v = flowmodel.Variable() + v.concretetype = returnvar.concretetype + pseudoreturnblock = flowmodel.Block([v]) + pseudoreturnblock.operations = () + closelink(link, [returnvar], pseudoreturnblock) + def _patchgraph(graph): returntype = None for link in graph.iterlinks(): @@ -86,10 +84,9 @@ from pypy.rpython.typesystem import LowLevelTypeSystem self.type_system = LowLevelTypeSystem.instance -def runlink(startlink, args): +def runblock(block, args): from pypy.rpython.llinterp import LLInterpreter - assert isinstance(startlink, flowmodel.FunctionGraph), "XXX" - graph = startlink # for now + graph = flowmodel.FunctionGraph('?', block) _patchgraph(graph) flowmodel.checkgraph(graph) llinterp = LLInterpreter(PseudoRTyper()) Modified: pypy/dist/pypy/rpython/test/test_rgenop.py ============================================================================== --- pypy/dist/pypy/rpython/test/test_rgenop.py (original) +++ pypy/dist/pypy/rpython/test/test_rgenop.py Tue Jan 24 18:02:02 2006 @@ -4,15 +4,13 @@ def test_square(): """def square(v0): return v0*v0""" - startlink = newgraph("square") block = newblock() v0 = geninputarg(block, Signed) v1 = genop(block, 'int_mul', [v0, v0], Signed) - link = newreturnlink(block, v1) - closeblock1(block, link) - closelink(startlink, block) + link = closeblock1(block) + closereturnlink(link, v1) - res = runlink(startlink, [17]) + res = runblock(block, [17]) assert res == 289 def test_if(): @@ -23,19 +21,17 @@ else: return v0 """ - startlink = newgraph("if") block = newblock() v0 = geninputarg(block, Signed) const0 = genconst(block, 0) v1 = genop(block, 'int_lt', [v0, const0], Bool) - true_link = newreturnlink(block, const0) - false_link = newreturnlink(block, v0) - closeblock2(block, v1, false_link, true_link) - closelink(startlink, block) + false_link, true_link = closeblock2(block, v1) + closereturnlink(true_link, const0) + closereturnlink(false_link, v0) - res = runlink(startlink, [-1]) + res = runblock(block, [-1]) assert res == 0 - res = runlink(startlink, [42]) + res = runblock(block, [42]) assert res == 42 def test_loop(): @@ -48,29 +44,26 @@ i += 1 return result """ - startlink = newgraph("loop") block = newblock() v0 = geninputarg(block, Signed) const1 = genconst(block, 1) - link = newlink(block, [const1, const1, v0]) - closeblock1(block, link) - closelink(startlink, block) + link = closeblock1(block) loopblock = newblock() result0 = geninputarg(loopblock, Signed) i0 = geninputarg(loopblock, Signed) v1 = geninputarg(loopblock, Signed) - closelink(link, loopblock) + closelink(link, [const1, const1, v0], loopblock) const1 = genconst(block, 1) result1 = genop(loopblock, 'int_mul', [result0, i0], Signed) i1 = genop(loopblock, 'int_add', [i0, const1], Signed) v2 = genop(loopblock, 'int_le', [i1, v1], Bool) - false_link = newreturnlink(loopblock, result1) - true_link = newlink(loopblock, [result1, i1, v1]) - closelink(true_link, loopblock) - closeblock2(loopblock, v2, false_link, true_link) - res = runlink(startlink, [0]) + false_link, true_link = closeblock2(loopblock, v2) + closereturnlink(false_link, result1) + closelink(true_link, [result1, i1, v1], loopblock) + + res = runblock(block, [0]) assert res == 1 - res = runlink(startlink, [1]) + res = runblock(block, [1]) assert res == 1 - res = runlink(startlink, [7]) + res = runblock(block, [7]) assert res == 5040 From tismer at codespeak.net Tue Jan 24 18:05:24 2006 From: tismer at codespeak.net (tismer at codespeak.net) Date: Tue, 24 Jan 2006 18:05:24 +0100 (CET) Subject: [pypy-svn] r22610 - pypy/dist/pypy/translator/c/test Message-ID: <20060124170524.8A35427DBD@code1.codespeak.net> Author: tismer Date: Tue Jan 24 18:05:23 2006 New Revision: 22610 Modified: pypy/dist/pypy/translator/c/test/test_coroutine.py Log: (Eric, Jacob, Christian) some refinements to coroutines, as they will be used in the stackless module most probably. Modified: pypy/dist/pypy/translator/c/test/test_coroutine.py ============================================================================== --- pypy/dist/pypy/translator/c/test/test_coroutine.py (original) +++ pypy/dist/pypy/translator/c/test/test_coroutine.py Tue Jan 24 18:05:23 2006 @@ -70,9 +70,10 @@ class CoState(object): - pass + def __init__(self): + self.last = self.current = self.main = Coroutine() -costate = CoState() +costate = None class CoroutineDamage(SystemError): pass @@ -81,6 +82,10 @@ def __init__(self): self.frame = None + if costate is None: + self.parent = self + else: + self.parent = costate.main def bind(self, thunk): if self.frame is not None: @@ -88,23 +93,27 @@ self.frame = self._bind(thunk) def _bind(self, thunk): - binder = costate.current + self.parent = costate.current costate.last.frame = yield_current_frame_to_caller() thunk.call() - if binder.frame is None: - binder = costate.main - costate.last, costate.current = costate.current, binder - frame, binder.frame = binder.frame, None - return frame + if self.parent.frame is None: + self.parent = costate.main + return self._update_state(self.parent) def switch(self): if self.frame is None: raise CoroutineDamage - costate.last, costate.current = costate.current, self - frame, self.frame = self.frame, None - costate.last.frame = frame.switch() + costate.last.frame = self._update_state(self).switch() + # note that last gets updated before assignment! + + def _update_state(new): + costate.last, costate.current = costate.current, new + frame, new.frame = new.frame, None + return frame + _update_state = staticmethod(_update_state) + +costate = CoState() -costate.current = costate.last = costate.main = Coroutine() def output(stuff): os.write(2, stuff + '\n') From cfbolz at codespeak.net Tue Jan 24 18:11:20 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Tue, 24 Jan 2006 18:11:20 +0100 (CET) Subject: [pypy-svn] r22613 - pypy/dist/pypy/jit Message-ID: <20060124171120.C784E27DBC@code1.codespeak.net> Author: cfbolz Date: Tue Jan 24 18:11:19 2006 New Revision: 22613 Modified: pypy/dist/pypy/jit/llabstractinterp.py Log: (cfbolz, arigo) Create blocks directly without Constants in their inputargs. Modified: pypy/dist/pypy/jit/llabstractinterp.py ============================================================================== --- pypy/dist/pypy/jit/llabstractinterp.py (original) +++ pypy/dist/pypy/jit/llabstractinterp.py Tue Jan 24 18:11:19 2006 @@ -253,10 +253,17 @@ def __init__(self, args_v, frozenstate): self.args_v = args_v - self.link = Link(args_v, None) + self.link = Link([], None) self.frozenstate = frozenstate - def settarget(self, block): + def settarget(self, block, blockargs): + args = [] + for v1, v2 in zip(self.args_v, blockargs): + if isinstance(v2, Constant): + assert v1 == v2 + else: + args.append(v1) + self.link.args = args self.link.settarget(block) @@ -284,7 +291,7 @@ #self.a_return = None self.state = "before" - def settarget(self, block): + def settarget(self, block, blockargs): block.isstartblock = True self.copygraph.startblock = block @@ -382,7 +389,7 @@ block, blockargs, blockexits = self.flowin(state, merged_key) state.copyblocks[key] = block, blockargs, blockexits did_any_generalization = True - next.settarget(block) + next.settarget(block, blockargs) for ls in blockexits: if ls.frozenstate is not None: if ls not in seen: @@ -404,8 +411,6 @@ if did_any_generalization: raise RestartCompleting - remove_constant_inputargs(graph) - # the graph should be complete now; sanity-check try: checkgraph(graph) @@ -455,7 +460,7 @@ args_v = [builder.binding(v).forcevarorconst(builder) for v in origblock.inputargs] ls = LinkState(args_v, frozenstate=None) - ls.settarget(target) + ls.settarget(target, target.inputargs) raise InsertNextLink(ls) else: # finishing a handle_call_inlining(): link back to @@ -532,7 +537,7 @@ self.residual_operations = [] def buildblock(self, newexitswitch, newlinkstates): - b = Block(self.newinputargs) + b = Block([v for v in self.newinputargs if isinstance(v, Variable)]) b.operations = self.residual_operations b.exitswitch = newexitswitch b.closeblock(*[ls.link for ls in newlinkstates]) @@ -934,22 +939,3 @@ if op.result in used: result.append(op.result) return result - -def remove_constant_inputargs(graph): - # for simplicity, the logic in GraphState produces graphs that can - # pass constants from one block to the next explicitly, via a - # link.args -> block.inputargs. Remove them now. - for link in graph.iterlinks(): - i = 0 - for v in link.target.inputargs: - if isinstance(v, Constant): - del link.args[i] - else: - i += 1 - for block in graph.iterblocks(): - i = 0 - for v in block.inputargs[:]: - if isinstance(v, Constant): - del block.inputargs[i] - else: - i += 1 From arigo at codespeak.net Tue Jan 24 18:19:00 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 24 Jan 2006 18:19:00 +0100 (CET) Subject: [pypy-svn] r22614 - pypy/dist/pypy/rpython Message-ID: <20060124171900.819E127B58@code1.codespeak.net> Author: arigo Date: Tue Jan 24 18:18:59 2006 New Revision: 22614 Modified: pypy/dist/pypy/rpython/llinterp.py Log: ImportError in the llinterp. Modified: pypy/dist/pypy/rpython/llinterp.py ============================================================================== --- pypy/dist/pypy/rpython/llinterp.py (original) +++ pypy/dist/pypy/rpython/llinterp.py Tue Jan 24 18:18:59 2006 @@ -665,11 +665,19 @@ op_original_int_add = op_int_add def op_int_add(self, x, y): - if isinstance(x, llmemory.OffsetOf): + if isinstance(x, llmemory.AddressOffset): return x + y else: return self.op_original_int_add(x, y) - + + op_original_int_mul = op_int_mul + + def op_int_mul(self, x, y): + if isinstance(x, llmemory.AddressOffset): + return x * y + else: + return self.op_original_int_mul(x, y) + def op_unichar_eq(self, x, y): assert isinstance(x, unicode) and len(x) == 1 assert isinstance(y, unicode) and len(y) == 1 From ac at codespeak.net Tue Jan 24 18:40:58 2006 From: ac at codespeak.net (ac at codespeak.net) Date: Tue, 24 Jan 2006 18:40:58 +0100 (CET) Subject: [pypy-svn] r22616 - in pypy/dist/pypy: annotation rpython rpython/l3interp rpython/l3interp/test translator/c/src Message-ID: <20060124174058.75E2527DB6@code1.codespeak.net> Author: ac Date: Tue Jan 24 18:40:57 2006 New Revision: 22616 Modified: pypy/dist/pypy/annotation/builtin.py pypy/dist/pypy/rpython/l3interp/convertgraph.py pypy/dist/pypy/rpython/l3interp/l3interp.py pypy/dist/pypy/rpython/l3interp/test/test_l3interp.py pypy/dist/pypy/rpython/rbuiltin.py pypy/dist/pypy/translator/c/src/mem.h Log: (arre,mwh) Refactor l3interp.convertgraph.convert_block to mostly be methods on a BlockConverter class. A test and a very naive implementation depending very very much on the Boehm GC for the l3 operation 'malloc'. Modified: pypy/dist/pypy/annotation/builtin.py ============================================================================== --- pypy/dist/pypy/annotation/builtin.py (original) +++ pypy/dist/pypy/annotation/builtin.py Tue Jan 24 18:40:57 2006 @@ -511,6 +511,11 @@ BUILTIN_ANALYZERS[llmemory.offsetof] = offsetof +from pypy.rpython.l3interp import l3interp +def l3malloc(size): + return SomeAddress() + +BUILTIN_ANALYZERS[l3interp.malloc] = l3malloc #_________________________________ # external functions Modified: pypy/dist/pypy/rpython/l3interp/convertgraph.py ============================================================================== --- pypy/dist/pypy/rpython/l3interp/convertgraph.py (original) +++ pypy/dist/pypy/rpython/l3interp/convertgraph.py Tue Jan 24 18:40:57 2006 @@ -49,30 +49,36 @@ raise Exception("don't know how to acess %s value"%T) else: return 'ptr' - def convert_block(block, memo): if block in memo: return memo[block] + converter = BlockConverter(block, memo) + return converter.convert() + +class BlockConverter: - stacksizes = {'int': 0, - 'dbl': 0, - 'ptr': 0} - constants = {'int': [], - 'dbl': [], - 'ptr': [], - 'offset':[]} - var2stack = {} + def __init__(self, block, memo): + self.block = block + self.memo = memo + self.stacksizes = {'int': 0, + 'dbl': 0, + 'ptr': 0} + self.constants = {'int': [], + 'dbl': [], + 'ptr': [], + 'offset':[]} + self.var2stack = {} - def push(v): + def push(self, v): kind = getkind(v.concretetype) - position = stacksizes[kind] - stacksizes[kind] += 1 - var2stack[v] = position + position = self.stacksizes[kind] + self.stacksizes[kind] += 1 + self.var2stack[v] = position - def get(v): + def get(self, v): kind = getkind(v.concretetype) if isinstance(v, flowmodel.Constant): - clist = constants[kind] + clist = self.constants[kind] if kind == 'ptr': value = fakeaddress(v.value) else: @@ -84,11 +90,11 @@ clist.append(value) return res else: - position = var2stack[v] - return position - stacksizes[kind] # < 0 + position = self.var2stack[v] + return position - self.stacksizes[kind] # < 0 - def getoffset(offset): - clist = constants['offset'] + def getoffset(self, offset): + clist = self.constants['offset'] try: res = clist.index(offset) except ValueError: @@ -96,118 +102,142 @@ clist.append(offset) return res - for v in block.inputargs: - if v.concretetype is not lltype.Void: - push(v) - - insns = [] - l3block = model.Block(insns) - memo[block] = l3block - - if block.operations == (): - if len(block.inputargs) == 1: # return block - if block.inputargs[0].concretetype is lltype.Void: - l3block.insns.append(Op.void_return) - else: - kind = getkind(block.inputargs[0].concretetype) - l3block.insns.append(model.very_low_level_opcode[ - {'int': 'int_return', - 'dbl': 'float_return', - 'ptr': 'adr_return'}[kind]]) - l3block.insns.append(-1) - else: - raise NotImplementedError("except block") - return l3block - - for spaceop in block.operations: - if spaceop.opname == 'getfield': - opname = spaceop.opname + '_' + \ - getaccesskind(spaceop.result.concretetype) - insns.append(model.very_low_level_opcode[opname]) - v0, v1 = spaceop.args - insns.append(get(v0)) - - offset = FieldOffset(v0.concretetype, v1.value) - insns.append(getoffset(offset)) - elif spaceop.opname == 'setfield': - v0, v1, v2 = spaceop.args - opname = spaceop.opname + '_' + \ - getaccesskind(v2.concretetype) - insns.append(model.very_low_level_opcode[opname]) - insns.append(get(v0)) - - offset = FieldOffset(v0.concretetype, v1.value) - insns.append(getoffset(offset)) - insns.append(get(v2)) - elif spaceop.opname == 'getarrayitem': - opname = spaceop.opname + '_' + \ - getaccesskind(spaceop.result.concretetype) - insns.append(model.very_low_level_opcode[opname]) - v0, v1 = spaceop.args - insns.append(get(v0)) - insns.append(get(v1)) - - offset = ArrayItemsOffset(v0.concretetype) - insns.append(getoffset(offset)) - - offset = ItemOffset(spaceop.result.concretetype) - insns.append(getoffset(offset)) - elif spaceop.opname == 'setarrayitem': - array, index, value = spaceop.args - opname = spaceop.opname + '_' + \ - getaccesskind(value.concretetype) - insns.append(model.very_low_level_opcode[opname]) - insns.append(get(array)) - insns.append(get(index)) - - offset = ArrayItemsOffset(array.concretetype) - insns.append(getoffset(offset)) - - offset = ItemOffset(value.concretetype) - insns.append(getoffset(offset)) - insns.append(get(value)) - else: - insns.append(model.very_low_level_opcode[spaceop.opname]) - for v in spaceop.args: - insns.append(get(v)) - if spaceop.result.concretetype is not lltype.Void: - push(spaceop.result) - - def convert_link(link): + def convert_link(self, link): targetregs = {'int': [], 'dbl': [], 'ptr': []} for v in link.args: if v.concretetype is not lltype.Void: kind = getkind(v.concretetype) - targetregs[kind].append(get(v)) - return model.Link(convert_block(link.target, memo), + targetregs[kind].append(self.get(v)) + return model.Link(convert_block(link.target, self.memo), targetregs['int'] or None, targetregs['dbl'] or None, targetregs['ptr'] or None) - if block.exitswitch is None: - insns.append(Op.jump) - link, = block.exits - l3block.exit0 = convert_link(link) - - elif block.exitswitch != flowmodel.Constant(flowmodel.last_exception): - link0, link1 = block.exits - if link0.exitcase: - link0, link1 = link1, link0 - assert not link0.exitcase - assert link1.exitcase - insns.append(Op.jump_cond) - insns.append(get(block.exitswitch)) - l3block.exit0 = convert_link(link0) - l3block.exit1 = convert_link(link1) + def convert(self, memo=None): + block = self.block + for v in block.inputargs: + if v.concretetype is not lltype.Void: + self.push(v) + + self.insns = [] + l3block = model.Block(self.insns) + self.memo[block] = l3block + + if block.operations == (): + if len(block.inputargs) == 1: # return block + if block.inputargs[0].concretetype is lltype.Void: + l3block.insns.append(Op.void_return) + else: + kind = getkind(block.inputargs[0].concretetype) + l3block.insns.append(model.very_low_level_opcode[ + {'int': 'int_return', + 'dbl': 'float_return', + 'ptr': 'adr_return'}[kind]]) + l3block.insns.append(-1) + else: + raise NotImplementedError("except block") + return l3block - else: - raise NotImplementedError("exceptions") + for spaceop in block.operations: + getattr(self, 'convert_'+spaceop.opname, + self.default_convert)(spaceop) + + if block.exitswitch is None: + self.insns.append(Op.jump) + link, = block.exits + l3block.exit0 = self.convert_link(link) + + elif block.exitswitch != flowmodel.Constant(flowmodel.last_exception): + link0, link1 = block.exits + if link0.exitcase: + link0, link1 = link1, link0 + assert not link0.exitcase + assert link1.exitcase + self.insns.append(Op.jump_cond) + self.insns.append(self.get(block.exitswitch)) + l3block.exit0 = self.convert_link(link0) + l3block.exit1 = self.convert_link(link1) + + else: + raise NotImplementedError("exceptions") + + if self.constants['int']: l3block.constants_int = self.constants['int'] + if self.constants['dbl']: l3block.constants_dbl = self.constants['dbl'] + if self.constants['ptr']: l3block.constants_ptr = self.constants['ptr'] + if self.constants['offset']: + l3block.constants_offset = self.constants['offset'] + + return l3block + + def default_convert(self, spaceop): + self.insns.append(model.very_low_level_opcode[spaceop.opname]) + for v in spaceop.args: + self.insns.append(self.get(v)) + if spaceop.result.concretetype is not lltype.Void: + self.push(spaceop.result) - if constants['int']: l3block.constants_int = constants['int'] - if constants['dbl']: l3block.constants_dbl = constants['dbl'] - if constants['ptr']: l3block.constants_ptr = constants['ptr'] - if constants['offset']: l3block.constants_offset = constants['offset'] + def convert_getfield(self, spaceop): + opname = spaceop.opname + '_' + \ + getaccesskind(spaceop.result.concretetype) + self.insns.append(model.very_low_level_opcode[opname]) + v0, v1 = spaceop.args + self.insns.append(self.get(v0)) + + offset = FieldOffset(v0.concretetype, v1.value) + self.insns.append(self.getoffset(offset)) + self.push(spaceop.result) + + def convert_setfield(self, spaceop): + v0, v1, v2 = spaceop.args + opname = spaceop.opname + '_' + \ + getaccesskind(v2.concretetype) + self.insns.append(model.very_low_level_opcode[opname]) + self.insns.append(self.get(v0)) + + offset = FieldOffset(v0.concretetype, v1.value) + self.insns.append(self.getoffset(offset)) + self.insns.append(self.get(v2)) + + def convert_getarrayitem(self, spaceop): + opname = spaceop.opname + '_' + \ + getaccesskind(spaceop.result.concretetype) + self.insns.append(model.very_low_level_opcode[opname]) + v0, v1 = spaceop.args + self.insns.append(self.get(v0)) + self.insns.append(self.get(v1)) + + offset = ArrayItemsOffset(v0.concretetype) + self.insns.append(self.getoffset(offset)) + + offset = ItemOffset(spaceop.result.concretetype) + self.insns.append(self.getoffset(offset)) + self.push(spaceop.result) + + def convert_setarrayitem(self, spaceop): + array, index, value = spaceop.args + opname = spaceop.opname + '_' + \ + getaccesskind(value.concretetype) + self.insns.append(model.very_low_level_opcode[opname]) + self.insns.append(self.get(array)) + self.insns.append(self.get(index)) + + offset = ArrayItemsOffset(array.concretetype) + self.insns.append(self.getoffset(offset)) + + offset = ItemOffset(value.concretetype) + self.insns.append(self.getoffset(offset)) + self.insns.append(self.get(value)) + + def convert_malloc(self, spaceop): + type, = spaceop.args + self.insns.append(Op.malloc) + self.insns.append(self.getoffset(ItemOffset(type.value))) + self.push(spaceop.result) + + def convert_malloc_varsize(self, spaceop): + TYPE, nitems = spaceop.args + + raise NotImplementedError - return l3block Modified: pypy/dist/pypy/rpython/l3interp/l3interp.py ============================================================================== --- pypy/dist/pypy/rpython/l3interp/l3interp.py (original) +++ pypy/dist/pypy/rpython/l3interp/l3interp.py Tue Jan 24 18:40:57 2006 @@ -252,8 +252,16 @@ frame = L3Frame(graph, self.stack_int, self.stack_dbl, self.stack_ptr) frame.execute() + def op_malloc(self): + size = self.getoffset() + self.stack_ptr.append(malloc(size)) + # ____________________________________________________________ +# XXX should this live here? +def malloc(size): + return fakeaddress(lltype.malloc(size.TYPE)) + class L3Return(Exception): pass Modified: pypy/dist/pypy/rpython/l3interp/test/test_l3interp.py ============================================================================== --- pypy/dist/pypy/rpython/l3interp/test/test_l3interp.py (original) +++ pypy/dist/pypy/rpython/l3interp/test/test_l3interp.py Tue Jan 24 18:40:57 2006 @@ -4,6 +4,7 @@ from pypy.translator.c.test.test_genc import compile from pypy.translator.translator import TranslationContext from pypy.annotation import policy +from pypy.rpython.lltypesystem import lltype def setup_module(mod): mod._cleanups = [] @@ -240,7 +241,6 @@ def test_getitem(): - from pypy.rpython.lltypesystem import lltype A = lltype.GcArray(lltype.Signed) a = lltype.malloc(A, 3) @@ -267,7 +267,6 @@ assert fn(arg) == f(arg) def test_setitem(): - from pypy.rpython.lltypesystem import lltype A = lltype.GcArray(lltype.Signed) a = lltype.malloc(A, 3) @@ -297,7 +296,6 @@ assert entry_point(1, 0) == 65 def test_getsetitem_complex(): - from pypy.rpython.lltypesystem import lltype A = lltype.GcArray(lltype.Signed) AA = lltype.GcArray(lltype.Ptr(A)) @@ -334,3 +332,25 @@ for x,y,value in (0,0,0), (0,0,11), (0,0,0), (2,1,0), (2,1,26), (2,1,0): assert fn(x, y, value) == f(x, y, value) + + +def test_malloc(): + S = lltype.GcStruct("S", ('x',lltype.Signed)) + + def f(n): + s = lltype.malloc(S) + s.x = n + return s.x + + l3graph = l3ify(f, [int]) + def entry_point(x): + value = l3interp.l3interpret(l3graph, [x], [], []) + assert isinstance(value, l3interp.L3Integer) + return value.intval + + assert entry_point(5) == f(5) + + fn = translate(entry_point, [int]) + + assert fn(9) == f(9) + Modified: pypy/dist/pypy/rpython/rbuiltin.py ============================================================================== --- pypy/dist/pypy/rpython/rbuiltin.py (original) +++ pypy/dist/pypy/rpython/rbuiltin.py Tue Jan 24 18:40:57 2006 @@ -389,6 +389,18 @@ BUILTIN_TYPER[llmemory.offsetof] = rtype_offsetof +# XXX this next little bit is a monstrous hack. the Real Thing awaits +# some kind of proper GC integration +from pypy.rpython.l3interp import l3interp +from pypy.rpython.raddress import offset_repr + +def rtype_l3malloc(hop): + v_list = hop.inputargs(offset_repr) + return hop.genop("call_boehm_gc_alloc", v_list, + resulttype=llmemory.Address) + +BUILTIN_TYPER[l3interp.malloc] = rtype_l3malloc + # _________________________________________________________________ # non-gc objects Modified: pypy/dist/pypy/translator/c/src/mem.h ============================================================================== --- pypy/dist/pypy/translator/c/src/mem.h (original) +++ pypy/dist/pypy/translator/c/src/mem.h Tue Jan 24 18:40:57 2006 @@ -75,6 +75,13 @@ memset((void*) r, 0, size); \ } +/* as we said in rbuiltin.py: +# XXX this next little bit is a monstrous hack. the Real Thing awaits +# some kind of proper GC integration +if GC integration has happened and this junk is still here, please delete it :) +*/ +#define OP_CALL_BOEHM_GC_ALLOC(size, r, err) OP_BOEHM_ZERO_MALLOC(size, r, 0, 0, err) + #undef PUSH_ALIVE #define PUSH_ALIVE(obj) From cfbolz at codespeak.net Tue Jan 24 18:58:13 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Tue, 24 Jan 2006 18:58:13 +0100 (CET) Subject: [pypy-svn] r22618 - pypy/dist/pypy/jit Message-ID: <20060124175813.27A6427DB6@code1.codespeak.net> Author: cfbolz Date: Tue Jan 24 18:58:11 2006 New Revision: 22618 Modified: pypy/dist/pypy/jit/llabstractinterp.py Log: More moving of stuff around to make it look like it could use the genop() functions. Modified: pypy/dist/pypy/jit/llabstractinterp.py ============================================================================== --- pypy/dist/pypy/jit/llabstractinterp.py (original) +++ pypy/dist/pypy/jit/llabstractinterp.py Tue Jan 24 18:58:11 2006 @@ -251,10 +251,13 @@ class LinkState(object): """Wrapper for a residual link and the frozen state that it should go to.""" + exitcase = None + llexitcase = None + def __init__(self, args_v, frozenstate): self.args_v = args_v - self.link = Link([], None) self.frozenstate = frozenstate + self.link = None def settarget(self, block, blockargs): args = [] @@ -396,17 +399,28 @@ seen[ls] = True pending.append(ls) else: - # 'ls' is the linkstate of a link to a return or except - # block; make sure that it is really the one from 'graph' - # -- by patching 'graph' if necessary. # XXX don't use ls.link.target! - if len(ls.link.target.inputargs) == 1: - #self.a_return = state.args_a[0] - graph.returnblock = ls.link.target - elif len(link.target.inputargs) == 2: - graph.exceptblock = ls.link.target + if ls.link is None or ls.link.target is None: + # resolve the LinkState to go to the return + # or except block + if len(ls.args_v) == 1: + target = graph.returnblock + elif len(ls.args_v) == 2: + target = graph.exceptblock + else: + raise Exception("uh?") + ls.settarget(target, target.inputargs) else: - raise Exception("uh?") + # the LinkState is already going to a return or except + # block; make sure that it is really the one from + # 'graph' -- by patching 'graph' if necessary. + if len(ls.link.target.inputargs) == 1: + #self.a_return = state.args_a[0] + graph.returnblock = ls.link.target + elif len(link.target.inputargs) == 2: + graph.exceptblock = ls.link.target + else: + raise Exception("uh?") if did_any_generalization: raise RestartCompleting @@ -453,14 +467,9 @@ # copies of return and except blocks are *normal* blocks # currently; they are linked to the official return or # except block of the copygraph. - if len(origblock.inputargs) == 1: - target = self.copygraph.returnblock - else: - target = self.copygraph.exceptblock args_v = [builder.binding(v).forcevarorconst(builder) for v in origblock.inputargs] ls = LinkState(args_v, frozenstate=None) - ls.settarget(target, target.inputargs) raise InsertNextLink(ls) else: # finishing a handle_call_inlining(): link back to @@ -515,8 +524,8 @@ args_a, origlink.target) newlinkstate = self.interp.schedule(nextinputstate) if newexitswitch is not None: - newlinkstate.link.exitcase = origlink.exitcase #XXX change later - newlinkstate.link.llexitcase = origlink.llexitcase + newlinkstate.exitcase = origlink.exitcase + newlinkstate.llexitcase = origlink.llexitcase newlinkstates.append(newlinkstate) newblock = builder.buildblock(newexitswitch, newlinkstates) @@ -540,7 +549,14 @@ b = Block([v for v in self.newinputargs if isinstance(v, Variable)]) b.operations = self.residual_operations b.exitswitch = newexitswitch - b.closeblock(*[ls.link for ls in newlinkstates]) + exits = [] + for ls in newlinkstates: + link = Link([], None) + link.exitcase = ls.exitcase + link.llexitcase = ls.llexitcase + ls.link = link + exits.append(link) + b.closeblock(*exits) return b def binding(self, v): From gromit at codespeak.net Tue Jan 24 19:22:55 2006 From: gromit at codespeak.net (gromit at codespeak.net) Date: Tue, 24 Jan 2006 19:22:55 +0100 (CET) Subject: [pypy-svn] r22619 - in pypy/dist/pypy/rpython/rctypes: . test Message-ID: <20060124182255.A175E27DB8@code1.codespeak.net> Author: gromit Date: Tue Jan 24 19:22:53 2006 New Revision: 22619 Modified: pypy/dist/pypy/rpython/rctypes/interface.py pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py Log: ADD: (stephan, gromit) Exposing ctypes structures and a c test module. Modified: pypy/dist/pypy/rpython/rctypes/interface.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/interface.py (original) +++ pypy/dist/pypy/rpython/rctypes/interface.py Tue Jan 24 19:22:53 2006 @@ -1,5 +1,5 @@ from ctypes import _DLLS -from implementation import RCDLL as CDLL, c_int, c_char_p, c_char, POINTER +from implementation import RCDLL as CDLL, c_int, c_char_p, c_char, POINTER, Structure, byref try: from implementation import RWinDLL as WinDLL except ImportError: Modified: pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py Tue Jan 24 19:22:53 2006 @@ -2,6 +2,8 @@ from pypy.annotation.annrpython import RPythonAnnotator from pypy.translator.translator import TranslationContext from pypy.translator.c.test.test_genc import compile +from pypy.translator.tool.cbuild import compile_c_module +import sys def setup_module(mod): @@ -10,8 +12,7 @@ except ImportError: py.test.skip("this test needs ctypes installed") else: - import sys - from pypy.rpython.rctypes.interface import cdll, c_char_p, c_int, c_char, POINTER + from pypy.rpython.rctypes.interface import cdll, c_char_p, c_int, c_char, POINTER, Structure, byref if sys.platform == 'win32': mylib = cdll.LoadLibrary('msvcrt.dll') elif sys.platform == 'linux2': @@ -27,13 +28,17 @@ def o_atoi(a): return atoi(a) mod.o_atoi = o_atoi + mod.cdll = cdll + class tagpoint(Structure): + _fields_ = [("x", c_int), + ("y", c_int)] + mod.tagpoint = tagpoint + mod.byref = byref class Test_rctypes: def test_simple(self): - - res = o_atoi('42') assert res == 42 @@ -57,3 +62,29 @@ res = fn("42") assert res == 42 + +class Test_structure: + + def setup_class(cls): + compile_c_module( [ py.path.local( "_rctypes_test.c" ) ], "_rctypes_test" ) + + def test_simple_as_extension_module(self): + import _rctypes_test as t0 + import _rctypes_test as t1 + assert t1 is t0 + assert "_rctypes_test" in sys.modules + + def test_simple(self): + if sys.platform == "win32": + dll = cdll.LoadLibrary( "_rctypes_test.pyd" ) + else: + dll = cdll.LoadLibrary( "_rctypes_test.so" ) + in_point = tagpoint() + in_point.x = 42 + in_point.y = 17 + out_point = tagpoint() + assert in_point.x + in_point.y == dll._testfunc_byval( in_point, byref( out_point ) ) + assert out_point.x == 42 + assert out_point.y == 17 + + From tismer at codespeak.net Tue Jan 24 19:39:06 2006 From: tismer at codespeak.net (tismer at codespeak.net) Date: Tue, 24 Jan 2006 19:39:06 +0100 (CET) Subject: [pypy-svn] r22620 - pypy/dist/pypy/translator/goal Message-ID: <20060124183906.95C1727DBF@code1.codespeak.net> Author: tismer Date: Tue Jan 24 19:39:04 2006 New Revision: 22620 Added: pypy/dist/pypy/translator/goal/targetpypy_stacklesstest.py (contents, props changed) Log: a version of targetpypystandalone that is suitable for testing stackless Added: pypy/dist/pypy/translator/goal/targetpypy_stacklesstest.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/goal/targetpypy_stacklesstest.py Tue Jan 24 19:39:04 2006 @@ -0,0 +1,119 @@ +""" +This is a hacked version of targetpypystandalone.py +with options set and optimized for testing stackless +on application level. +The problem is, that with the standard interpreter, +we have no choice for testing but to build everything. +Trying to reduce time by disabling all optimizations. +""" +import os, sys +from pypy.objspace.std.objspace import StdObjSpace +# XXX from pypy.annotation.model import * +# since we are execfile()'ed this would pull some +# weird objects into the globals, which we would try to pickle. +from pypy.annotation.model import SomeList, SomeString +from pypy.annotation.listdef import ListDef +from pypy.interpreter import gateway +from pypy.interpreter.error import OperationError +from pypy.translator.goal.ann_override import PyPyAnnotatorPolicy + +# WARNING: this requires the annotator. +# There is no easy way to build all caches manually, +# but the annotator can do it for us for free. + +try: + this_dir = os.path.dirname(__file__) +except NameError: + this_dir = os.path.dirname(sys.argv[0]) + +def debug(msg): + os.write(2, "debug: " + msg + '\n') + +# __________ Entry point __________ + +def entry_point(argv): + debug("entry point starting") + for arg in argv: + debug(" argv -> " + arg) + try: + w_executable = space.wrap(argv[0]) + w_argv = space.newlist([space.wrap(s) for s in argv[1:]]) + w_exitcode = space.call_function(w_entry_point, w_executable, w_argv) + # try to pull it all in + ## from pypy.interpreter import main, interactive, error + ## con = interactive.PyPyConsole(space) + ## con.interact() + except OperationError, e: + debug("OperationError:") + debug(" operror-type: " + e.w_type.getname(space, '?')) + debug(" operror-value: " + space.str_w(space.str(e.w_value))) + return 1 + return space.int_w(w_exitcode) + +# _____ Define and setup target ___ + +# for now this will do for option handling + +take_options = True + +stackless_options = { + 'text': True, + 'lowmem': True, + } + +def opt_parser(): + import py + defl = {'thread': False} + defl.update(stackless_options) + parser = py.compat.optparse.OptionParser(usage="target PyPy standalone", add_help_option=False) + parser.set_defaults(**defl) + parser.add_option("--thread", action="store_true", dest="thread", help="enable threading") + return parser + +def print_help(): + opt_parser().print_help() + + +def target(driver, args): + options = driver.options + driver.disable(["backendopt"]) + + tgt_options, _ = opt_parser().parse_args(args) + + translate_pypy.log_options(tgt_options, "target PyPy options in effect") + + options.thread = tgt_options.thread + + global space, w_entry_point + + geninterp = not getattr(options, 'lowmem', False) + + # obscure hack to stuff the translation options into the translated PyPy + import pypy.module.sys + wrapstr = 'space.wrap(%r)' % (options.__dict__) + pypy.module.sys.Module.interpleveldefs['pypy_translation_info'] = wrapstr + + # disable translation of the whole of classobjinterp.py + StdObjSpace.setup_old_style_classes = lambda self: None + + usemodules = ['stackless'] + if options.thread: + usemodules.append('thread') + + space = StdObjSpace(nofaking=True, + compiler="ast", # interpreter/astcompiler + translating=True, + usemodules=usemodules, + geninterp=geninterp) + # manually imports app_main.py + filename = os.path.join(this_dir, 'app_main.py') + w_dict = space.newdict([]) + space.exec_(open(filename).read(), w_dict, w_dict) + w_entry_point = space.getitem(w_dict, space.wrap('entry_point')) + + # sanity-check: call the entry point + res = entry_point(["pypy", "app_basic_example.py"]) + assert res == 0 + + return entry_point, None, PyPyAnnotatorPolicy(single_space = space) + From cfbolz at codespeak.net Tue Jan 24 22:45:49 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Tue, 24 Jan 2006 22:45:49 +0100 (CET) Subject: [pypy-svn] r22621 - in pypy/dist/pypy/rpython/rctypes: . test Message-ID: <20060124214549.0ADA327B5B@code1.codespeak.net> Author: cfbolz Date: Tue Jan 24 22:45:20 2006 New Revision: 22621 Modified: pypy/dist/pypy/rpython/rctypes/implementation.py pypy/dist/pypy/rpython/rctypes/interface.py pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py Log: follow PEP 8 (and the style used in PyPy in general -- admittedly it is not part of coding-guide.txt) Modified: pypy/dist/pypy/rpython/rctypes/implementation.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/implementation.py (original) +++ pypy/dist/pypy/rpython/rctypes/implementation.py Tue Jan 24 22:45:20 2006 @@ -16,11 +16,11 @@ c_char_p.annotator_type = None c_char_p.ll_type = None c_char_p.wrap_arg = staticmethod( - lambda ll_type, arg_name: "RPyString_AsString(%s)" % arg_name ) -POINTER( c_char ).annotator_type = None -POINTER( c_char ).ll_type = None -POINTER( c_char ).wrap_arg = staticmethod( - lambda ll_type, arg_name: "RPyString_AsString(%s)" % arg_name ) + lambda ll_type, arg_name: "RPyString_AsString(%s)" % arg_name) +POINTER(c_char).annotator_type = None +POINTER(c_char).ll_type = None +POINTER(c_char).wrap_arg = staticmethod( + lambda ll_type, arg_name: "RPyString_AsString(%s)" % arg_name) class FunctionPointerTranslation(object): Modified: pypy/dist/pypy/rpython/rctypes/interface.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/interface.py (original) +++ pypy/dist/pypy/rpython/rctypes/interface.py Tue Jan 24 22:45:20 2006 @@ -5,9 +5,9 @@ except ImportError: WinDLL = None -cdll = _DLLS( CDLL ) +cdll = _DLLS(CDLL) if WinDLL: - windll = _DLLS( WinDLL ) + windll = _DLLS(WinDLL) """ Modified: pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py Tue Jan 24 22:45:20 2006 @@ -66,7 +66,7 @@ class Test_structure: def setup_class(cls): - compile_c_module( [ py.path.local( "_rctypes_test.c" ) ], "_rctypes_test" ) + compile_c_module([py.path.local("_rctypes_test.c")], "_rctypes_test") def test_simple_as_extension_module(self): import _rctypes_test as t0 @@ -76,14 +76,14 @@ def test_simple(self): if sys.platform == "win32": - dll = cdll.LoadLibrary( "_rctypes_test.pyd" ) + dll = cdll.LoadLibrary("_rctypes_test.pyd") else: - dll = cdll.LoadLibrary( "_rctypes_test.so" ) + dll = cdll.LoadLibrary("_rctypes_test.so") in_point = tagpoint() in_point.x = 42 in_point.y = 17 out_point = tagpoint() - assert in_point.x + in_point.y == dll._testfunc_byval( in_point, byref( out_point ) ) + assert in_point.x + in_point.y == dll._testfunc_byval(in_point, byref(out_point)) assert out_point.x == 42 assert out_point.y == 17 From rxe at codespeak.net Wed Jan 25 09:41:23 2006 From: rxe at codespeak.net (rxe at codespeak.net) Date: Wed, 25 Jan 2006 09:41:23 +0100 (CET) Subject: [pypy-svn] r22624 - pypy/dist/pypy/translator/llvm/module Message-ID: <20060125084123.108BE27B5C@code1.codespeak.net> Author: rxe Date: Wed Jan 25 09:41:22 2006 New Revision: 22624 Modified: pypy/dist/pypy/translator/llvm/module/excsupport.py Log: cosmetic Modified: pypy/dist/pypy/translator/llvm/module/excsupport.py ============================================================================== --- pypy/dist/pypy/translator/llvm/module/excsupport.py (original) +++ pypy/dist/pypy/translator/llvm/module/excsupport.py Wed Jan 25 09:41:22 2006 @@ -31,7 +31,7 @@ import sys if sys.maxint != 2**31-1: #XXX need to move the ringbuffer code to another level anyway - ringbuffer_decl = ringbuffer_code = '' + ringbuffer_decl = ringbuffer_code = '' invokeunwind_code = ''' ccc %(returntype)s%%__entrypoint__%(entrypointname)s { From rxe at codespeak.net Wed Jan 25 09:52:47 2006 From: rxe at codespeak.net (rxe at codespeak.net) Date: Wed, 25 Jan 2006 09:52:47 +0100 (CET) Subject: [pypy-svn] r22625 - pypy/dist/pypy/translator/goal Message-ID: <20060125085247.6584B27B69@code1.codespeak.net> Author: rxe Date: Wed Jan 25 09:52:45 2006 New Revision: 22625 Modified: pypy/dist/pypy/translator/goal/bench-cronjob.py Log: This hack isnt needed anymore. Modified: pypy/dist/pypy/translator/goal/bench-cronjob.py ============================================================================== --- pypy/dist/pypy/translator/goal/bench-cronjob.py (original) +++ pypy/dist/pypy/translator/goal/bench-cronjob.py Wed Jan 25 09:52:45 2006 @@ -91,12 +91,6 @@ def compile(backend): - os.chdir(homedir + '/projects/pypy-dist') - os.system('rm `find . -name *.pyc`') - - os.chdir(homedir + '/projects/pypy-dist/pypy/_cache') - os.system('rm *') - os.chdir(homedir + '/projects/pypy-dist/pypy/translator/goal') os.system('/usr/local/bin/python translate_pypy.py --backend=%(backend)s --text --batch targetpypystandalone.py 2>&1' % locals()) From rxe at codespeak.net Wed Jan 25 09:55:03 2006 From: rxe at codespeak.net (rxe at codespeak.net) Date: Wed, 25 Jan 2006 09:55:03 +0100 (CET) Subject: [pypy-svn] r22626 - pypy/dist/pypy/rpython/memory/test Message-ID: <20060125085503.1F1E927DB8@code1.codespeak.net> Author: rxe Date: Wed Jan 25 09:55:01 2006 New Revision: 22626 Modified: pypy/dist/pypy/rpython/memory/test/test_convertlltype.py Log: Trivial - tests another branch of code. Modified: pypy/dist/pypy/rpython/memory/test/test_convertlltype.py ============================================================================== --- pypy/dist/pypy/rpython/memory/test/test_convertlltype.py (original) +++ pypy/dist/pypy/rpython/memory/test/test_convertlltype.py Wed Jan 25 09:55:01 2006 @@ -3,7 +3,6 @@ def test_convert_primitives(): cvter = LLTypeConverter(lladdress.NULL) - addr = lladdress.raw_malloc(10) c1 = cvter.convert(1) c = cvter.convert("c") assert c1 == 1 @@ -18,7 +17,9 @@ a = cvter.convert(lls) assert a[0] == 1 assert a[1] == 2 - + b = cvter.convert(lls) + assert a == b + def test_convert_array_of_structs(): cvter = LLTypeConverter(lladdress.raw_malloc(1000)) S = lltype.Struct("test", ("v1", lltype.Signed), ("v2", lltype.Signed)) From auc at codespeak.net Wed Jan 25 10:16:45 2006 From: auc at codespeak.net (auc at codespeak.net) Date: Wed, 25 Jan 2006 10:16:45 +0100 (CET) Subject: [pypy-svn] r22627 - pypy/dist/pypy/lib/logic Message-ID: <20060125091645.2D18127B5B@code1.codespeak.net> Author: auc Date: Wed Jan 25 10:16:42 2006 New Revision: 22627 Added: pypy/dist/pypy/lib/logic/test_computationspace.py Modified: pypy/dist/pypy/lib/logic/computationspace.py pypy/dist/pypy/lib/logic/unification.py Log: (ale, auc) * first test on computation space * CS creation Modified: pypy/dist/pypy/lib/logic/computationspace.py ============================================================================== --- pypy/dist/pypy/lib/logic/computationspace.py (original) +++ pypy/dist/pypy/lib/logic/computationspace.py Wed Jan 25 10:16:42 2006 @@ -184,11 +184,12 @@ self.program = program self.parent = parent self.store = Store() - self.root = var('root') + self.root = self.store.var('root') self.store.bind(self.root, program(self.store)) - - - + def branch(self): + return ComputationSpace(self.program, parent=self) + def ask(self): + pass Added: pypy/dist/pypy/lib/logic/test_computationspace.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/lib/logic/test_computationspace.py Wed Jan 25 10:16:42 2006 @@ -0,0 +1,32 @@ +import unification as u +import variable as v +import constraint as c +import computationspace as cs +from py.test import raises + + +class TestComputationSpace: + + def setUp(self): + pass + + def test_bind_cs_root(self): + + def dummy_problem(store): + s = store # i'm lazy + x, y, z, w = (s.var('x'), s.var('y'), + s.var('z'), s.var('w')) + s.set_domain(x, c.FiniteDomain([2, 6])) + s.set_domain(y, c.FiniteDomain([2, 3])) + s.set_domain(z, c.FiniteDomain([4, 5])) + s.set_domain(w, c.FiniteDomain([1, 4, 5])) + s.add_constraint(c.Expression([x, y, z], 'x == y + z')) + s.add_constraint(c.Expression([z, w], 'z < w')) + # we don't know yet how to + # set up a distribution strategy + return (x, y, z) + + spc = cs.ComputationSpace(dummy_problem) + assert 'root' in spc.store.names + assert ['x', 'y', 'z'] == [var.name for var + in spc.root.val] Modified: pypy/dist/pypy/lib/logic/unification.py ============================================================================== --- pypy/dist/pypy/lib/logic/unification.py (original) +++ pypy/dist/pypy/lib/logic/unification.py Wed Jan 25 10:16:42 2006 @@ -178,8 +178,17 @@ self.in_transaction = False self.lock = threading.RLock() + #-- Variables ---------------------------- + + def var(self, name): + """creates a variable of name name and put + it into the store""" + v = Var(name, self) + self.add_unbound(v) + return v + def add_unbound(self, var): - # register globally + """add unbound variable to the store""" if var in self.vars: raise AlreadyInStore(var.name) print "adding %s to the store" % var @@ -188,9 +197,8 @@ # put into new singleton equiv. set var.val = EqSet([var]) - #-- Bind var to domain -------------------- - def set_domain(self, var, dom): + """bind variable to domain""" assert(isinstance(var, Var) and (var in self.vars)) if var.is_bound(): raise AlreadyBound From pedronis at codespeak.net Wed Jan 25 10:29:32 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Wed, 25 Jan 2006 10:29:32 +0100 (CET) Subject: [pypy-svn] r22628 - pypy/extradoc/sprintinfo/mallorca Message-ID: <20060125092932.8391127B5B@code1.codespeak.net> Author: pedronis Date: Wed Jan 25 10:29:31 2006 New Revision: 22628 Modified: pypy/extradoc/sprintinfo/mallorca/planning.txt Log: round of planning for today. Modified: pypy/extradoc/sprintinfo/mallorca/planning.txt ============================================================================== --- pypy/extradoc/sprintinfo/mallorca/planning.txt (original) +++ pypy/extradoc/sprintinfo/mallorca/planning.txt Wed Jan 25 10:29:31 2006 @@ -28,22 +28,38 @@ Tasks ====== - - JIT: Armin, Carl. Graph generating ops - - - stackless application level interface: Christian, Eric - Design thinking. Coding could start today. - - - l3interp/offsetof/sizeof work: Michael, Arre - Progress. Next sizeof... - - (- integration of the garbage collection toolkit with the - offsetof work done: Carl Friedrich, Richard - Offsetof refactoring finished. Continuing with sizeof ...) - + - JIT: Arre, Samuele + Graph generating ops + Designed interface (see draft-jit-ideas), + started refactoring llabstractinterp to use the operations + (issues: llabsinterp was inspecting the graphs a lot ) + + - stackless application level interface: Christian, + started from the demo coroutine impl, trying to expose + at application level, both greenlet and stackless-like interfaces. + (issues: testing needs translation, the primitives are not + there on top of CPython, maybe use stackless or greenlet + to provide them or make the bytecode itself on top of CPython + stackless). Next step: fastest possible translation for testing. + + ( - l3interp/offsetof/sizeof work: + Progress. Progress needs either hacks and assuming Boehm + or GC integration. Try to progress assuming Boehm ) + + - integration of the garbage collection toolkit with the + offsetof, sizeof work done: Carl Friedrich, Michael (Richard) + - improving the way we interface with external functions. (also think about how exceptions fit in there): add static ctypes functionality into RPython itself - Gerald, Stephan, Samuele + Gerald, Stephan, Armin + glued atoi from ctypes (cheating a bit) + + Next: rtyper implementation of ctypes-like structures, + probably use Address as much as possible + + + later: ====== From gromit at codespeak.net Wed Jan 25 10:46:48 2006 From: gromit at codespeak.net (gromit at codespeak.net) Date: Wed, 25 Jan 2006 10:46:48 +0100 (CET) Subject: [pypy-svn] r22630 - pypy/dist/pypy/rpython/rctypes/test Message-ID: <20060125094648.EAF7D27B5B@code1.codespeak.net> Author: gromit Date: Wed Jan 25 10:46:47 2006 New Revision: 22630 Added: pypy/dist/pypy/rpython/rctypes/test/_rctypes_test.c Log: ADD: Forgot the actual C-level test Added: pypy/dist/pypy/rpython/rctypes/test/_rctypes_test.c ============================================================================== --- (empty file) +++ pypy/dist/pypy/rpython/rctypes/test/_rctypes_test.c Wed Jan 25 10:46:47 2006 @@ -0,0 +1,36 @@ +/* simple test, currently only for structures */ +#include +#ifdef MS_WIN32 +#include +#endif +#if defined(MS_WIN32) || defined(__CYGWIN__) +#define EXPORT(x) __declspec(dllexport) x +#else +#define EXPORT(x) x +#endif + +PyMethodDef module_methods[] = { + { NULL, NULL, 0, NULL}, +}; + + +typedef struct tagpoint { + int x; + int y; +} point; + +EXPORT(int) _testfunc_byval(point in, point *pout) +{ + if (pout) { + pout->x = in.x; + pout->y = in.y; + } + return in.x + in.y; +} + +DL_EXPORT(void) +init_rctypes_test(void) +{ + Py_InitModule("_rctypes_test", module_methods); +} + From mwh at codespeak.net Wed Jan 25 11:15:08 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Wed, 25 Jan 2006 11:15:08 +0100 (CET) Subject: [pypy-svn] r22631 - in pypy/dist/pypy/rpython/lltypesystem: . test Message-ID: <20060125101508.46FAD27B52@code1.codespeak.net> Author: mwh Date: Wed Jan 25 11:15:05 2006 New Revision: 22631 Modified: pypy/dist/pypy/rpython/lltypesystem/llmemory.py pypy/dist/pypy/rpython/lltypesystem/test/test_llmemory.py Log: small fixes and tests to llmemory. Modified: pypy/dist/pypy/rpython/lltypesystem/llmemory.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/llmemory.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/llmemory.py Wed Jan 25 11:15:05 2006 @@ -17,6 +17,8 @@ return lltype.Signed def __add__(self, other): + if not isinstance(other, AddressOffset): + return NotImplemented return CompositeOffset(self, other) @@ -93,9 +95,9 @@ assert not TYPE._is_varsize() return ItemOffset(TYPE) else: - if isinstance(TYPE, Array): + if isinstance(TYPE, lltype.Array): return itemoffsetof(TYPE, n) - elif isinstance(TYPE, Struct): + elif isinstance(TYPE, lltype.Struct): return FieldOffset(TYPE, TYPE._arrayfld) + \ itemoffsetof(TYPE._flds[TYPE._arrayfld], n) else: Modified: pypy/dist/pypy/rpython/lltypesystem/test/test_llmemory.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/test/test_llmemory.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/test/test_llmemory.py Wed Jan 25 11:15:05 2006 @@ -1,4 +1,6 @@ from pypy.rpython.lltypesystem.llmemory import * +from pypy.rpython.lltypesystem import lltype +import py def test_simple(): class C: @@ -33,3 +35,20 @@ b.set(14) assert x[3] == 14 +def test_dont_mix_offsets_and_ints(): + o = AddressOffset() + py.test.raises(TypeError, "1 + o") + py.test.raises(TypeError, "o + 1") + +def test_sizeof(): + # this is mostly an "assert not raises" sort of test + array = lltype.Array(lltype.Signed) + struct = lltype.Struct("S", ('x', lltype.Signed)) + varstruct = lltype.Struct("S", ('x', lltype.Signed), ('y', array)) + sizeof(struct) + sizeof(lltype.Signed) + py.test.raises(AssertionError, "sizeof(array)") + py.test.raises(AssertionError, "sizeof(varstruct)") + sizeof(array, 1) + sizeof(varstruct, 2) + From auc at codespeak.net Wed Jan 25 11:19:43 2006 From: auc at codespeak.net (auc at codespeak.net) Date: Wed, 25 Jan 2006 11:19:43 +0100 (CET) Subject: [pypy-svn] r22632 - pypy/dist/pypy/lib/logic Message-ID: <20060125101943.E05A627B69@code1.codespeak.net> Author: auc Date: Wed Jan 25 11:19:37 2006 New Revision: 22632 Modified: pypy/dist/pypy/lib/logic/computationspace.py pypy/dist/pypy/lib/logic/test_computationspace.py pypy/dist/pypy/lib/logic/unification.py Log: (ale, auc) * add process method to narrow all domains of variables in a space * tests Modified: pypy/dist/pypy/lib/logic/computationspace.py ============================================================================== --- pypy/dist/pypy/lib/logic/computationspace.py (original) +++ pypy/dist/pypy/lib/logic/computationspace.py Wed Jan 25 11:19:37 2006 @@ -141,22 +141,24 @@ ## Choose ## ------ -## Y=choose(N) waits until te current space becomes stable, blocks the -## current thread, and then creates a choice point with N alternatives in -## the current space. The blocked choose call waits for an alternative to -## be chosen by a commit operation on the space. The choose call only -## defines how many alternatives tere are; it does not specify what toi -## do for an alternative.Eventually, choose continues its execution with -## Y=I when alternative I (1= Author: rxe Date: Wed Jan 25 11:23:25 2006 New Revision: 22633 Modified: pypy/dist/pypy/translator/llvm/extfuncnode.py Log: To run tests on 32 bit machines again. Modified: pypy/dist/pypy/translator/llvm/extfuncnode.py ============================================================================== --- pypy/dist/pypy/translator/llvm/extfuncnode.py (original) +++ pypy/dist/pypy/translator/llvm/extfuncnode.py Wed Jan 25 11:23:25 2006 @@ -4,6 +4,8 @@ log = log.extfuncnode +from sys import maxint + class ExtFuncSig(object): def __init__(self, rettype, args): self.rettype = rettype @@ -14,11 +16,13 @@ "%LL_os_isatty" : ExtFuncSig("int", None), "%LL_stack_too_big" : ExtFuncSig("int", None), "%LL_os_lseek" : ExtFuncSig("int", None), - "%LL_os_write" : ExtFuncSig(None, ["int", None]), "%LL_thread_acquirelock" : ExtFuncSig("int", [None, "int"]), "%LL_thread_start" : ExtFuncSig(None, ["sbyte*", "sbyte*"]), } +if maxint != 2**31-1: + ext_func_sigs["%LL_os_write"] = ExtFuncSig(None, ["int", None]) + class ExternalFuncNode(ConstantLLVMNode): def __init__(self, db, value): @@ -40,7 +44,6 @@ def _get_wrapper(self): wrapper = ext_func_sigs.get(self.ref, None) - from sys import maxint if wrapper is None and maxint != 2**31-1: #log("ref=%s" % self.ref) rettype, args = self.getdecl_parts() From rxe at codespeak.net Wed Jan 25 11:24:33 2006 From: rxe at codespeak.net (rxe at codespeak.net) Date: Wed, 25 Jan 2006 11:24:33 +0100 (CET) Subject: [pypy-svn] r22634 - pypy/dist/pypy/translator/llvm/module Message-ID: <20060125102433.7969927B70@code1.codespeak.net> Author: rxe Date: Wed Jan 25 11:24:31 2006 New Revision: 22634 Modified: pypy/dist/pypy/translator/llvm/module/genexterns.c Log: More prototype forwarding hacks - we dont always have a RPyListOfString declared. Modified: pypy/dist/pypy/translator/llvm/module/genexterns.c ============================================================================== --- pypy/dist/pypy/translator/llvm/module/genexterns.c (original) +++ pypy/dist/pypy/translator/llvm/module/genexterns.c Wed Jan 25 11:24:31 2006 @@ -8,13 +8,16 @@ #define RPyRaiseSimpleException(exctype, errormsg) raise##exctype(errormsg) -// generated by rpython - argggh have to feed in prototypes +// XXX generated by rpython - argggh have to feed in prototypes RPyFREXP_RESULT *ll_frexp_result(double, int); RPyMODF_RESULT *ll_modf_result(double, double); RPySTAT_RESULT *ll_stat_result(int, int, int, int, int, int, int, int, int, int); void RPYTHON_RAISE_OSERROR(int error); -RPyListOfString *_RPyListOfString_New(long); -void _RPyListOfString_SetItem(RPyListOfString *, int, RPyString *); +#ifdef RPyListOfString + RPyListOfString *_RPyListOfString_New(long); + void _RPyListOfString_SetItem(RPyListOfString *, int, RPyString *); +#endif +// XXX end of proto hacks // overflows/zeros/values raising operations #include "raisingop.h" From rxe at codespeak.net Wed Jan 25 11:27:44 2006 From: rxe at codespeak.net (rxe at codespeak.net) Date: Wed, 25 Jan 2006 11:27:44 +0100 (CET) Subject: [pypy-svn] r22635 - pypy/dist/pypy/translator/llvm/test Message-ID: <20060125102744.3C32927B69@code1.codespeak.net> Author: rxe Date: Wed Jan 25 11:27:42 2006 New Revision: 22635 Modified: pypy/dist/pypy/translator/llvm/test/test_lladdresses.py Log: Skip a temporary failing test. Add another flavoured malloc test. Modified: pypy/dist/pypy/translator/llvm/test/test_lladdresses.py ============================================================================== --- pypy/dist/pypy/translator/llvm/test/test_lladdresses.py (original) +++ pypy/dist/pypy/translator/llvm/test/test_lladdresses.py Wed Jan 25 11:27:42 2006 @@ -4,7 +4,6 @@ from pypy.rpython.objectmodel import free_non_gc_object import py - def test_null(): def f(): return NULL - NULL @@ -89,6 +88,7 @@ assert res == 3 def test_pointer_comparison(): + py.test.skip("XXX tmp") def f(): result = 0 for addr1 in [raw_malloc(1), NULL]: @@ -104,7 +104,7 @@ res = fc() assert res == int('011100' * 2, 2) -def test_flavored_malloci_raw(): +def test_flavored_malloc1_raw(): class A(object): _alloc_flavor_ = "raw" def __init__(self, val): @@ -117,6 +117,40 @@ fn = compile(f, [int]) assert fn(1) == 2 +def test_flavored_malloc2_raw(): + from pypy.rpython.memory.support import AddressLinkedList + + def f(): + addr = raw_malloc(100) + ll = AddressLinkedList() + ll.append(addr) + ll.append(addr + 1) + ll.append(addr + 2) + a = ll.pop() + assert a == addr + a = ll.pop() + assert a - addr == 1 + a = ll.pop() + assert a - addr == 2 + assert ll.pop() == NULL + assert ll.pop() == NULL + ll.append(addr) + ll.free() + free_non_gc_object(ll) + ll = AddressLinkedList() + ll.append(addr) + ll.append(addr + 1) + ll.append(addr + 2) + a = ll.pop() + res = a - addr + ll.free() + free_non_gc_object(ll) + raw_free(addr) + return res + + fn = compile(f, []) + assert fn() == 0 + def test_flavored_varmalloc_raw(): py.test.skip("flavored_malloc not working?") A = lltype.Array(lltype.Signed) @@ -143,4 +177,3 @@ return result fn = compile(f, [int]) assert fn(1) == 2 - From mwh at codespeak.net Wed Jan 25 11:45:52 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Wed, 25 Jan 2006 11:45:52 +0100 (CET) Subject: [pypy-svn] r22636 - pypy/dist/pypy/rpython/memory Message-ID: <20060125104552.72A5127B64@code1.codespeak.net> Author: mwh Date: Wed Jan 25 11:45:50 2006 New Revision: 22636 Modified: pypy/dist/pypy/rpython/memory/lltypelayout.py Log: (cfbolz, mwh) adapt to changing times in llmemory. Modified: pypy/dist/pypy/rpython/memory/lltypelayout.py ============================================================================== --- pypy/dist/pypy/rpython/memory/lltypelayout.py (original) +++ pypy/dist/pypy/rpython/memory/lltypelayout.py Wed Jan 25 11:45:50 2006 @@ -131,7 +131,7 @@ def varsize_offsets_to_gcpointers_in_var_part(TYPE): if isinstance(TYPE, lltype.Array): if isinstance(TYPE.OF, lltype.Ptr): - return [llmemory.OffsetOf(TYPE.OF)] + return [0] elif isinstance(TYPE.OF, lltype.Struct): return offsets_to_gc_pointers(TYPE.OF) return [] From mwh at codespeak.net Wed Jan 25 11:47:46 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Wed, 25 Jan 2006 11:47:46 +0100 (CET) Subject: [pypy-svn] r22637 - in pypy/dist/pypy: annotation rpython rpython/lltypesystem rpython/memory/test Message-ID: <20060125104746.A1EEF27B44@code1.codespeak.net> Author: mwh Date: Wed Jan 25 11:47:41 2006 New Revision: 22637 Modified: pypy/dist/pypy/annotation/binaryop.py pypy/dist/pypy/annotation/builtin.py pypy/dist/pypy/annotation/model.py pypy/dist/pypy/rpython/lltypesystem/llmemory.py pypy/dist/pypy/rpython/memory/test/test_address.py pypy/dist/pypy/rpython/raddress.py pypy/dist/pypy/rpython/rbuiltin.py pypy/dist/pypy/rpython/rint.py Log: (cfbolz, mwh) Kill off SomeOffset. It's all SomeIntegers now. Modified: pypy/dist/pypy/annotation/binaryop.py ============================================================================== --- pypy/dist/pypy/annotation/binaryop.py (original) +++ pypy/dist/pypy/annotation/binaryop.py Wed Jan 25 11:47:41 2006 @@ -11,7 +11,7 @@ from pypy.annotation.model import SomeTuple, SomeImpossibleValue, s_ImpossibleValue from pypy.annotation.model import SomeInstance, SomeBuiltin, SomeIterator from pypy.annotation.model import SomePBC, SomeSlice, SomeFloat, s_None -from pypy.annotation.model import SomeExternalObject, SomeOffset +from pypy.annotation.model import SomeExternalObject from pypy.annotation.model import SomeAddress, SomeTypedAddressAccess from pypy.annotation.model import unionof, UnionError, set, missing_operation, TLS from pypy.annotation.model import add_knowntypedata, merge_knowntypedata @@ -732,15 +732,3 @@ class __extend__(pairtype(SomeObject, SomeAddress)): def union((s_obj, s_addr)): raise UnionError, "union of address and anything else makes no sense" - -class __extend__(pairtype(SomeOffset, SomeOffset)): - def add((s_off1, s_off2)): - return SomeOffset() - -class __extend__(pairtype(SomeOffset, SomeInteger)): - def mul((s_off1, s_off2)): - return SomeOffset() - -class __extend__(pairtype(SomeAddress, SomeOffset)): - def add((s_addr, s_off)): - return SomeAddress() Modified: pypy/dist/pypy/annotation/builtin.py ============================================================================== --- pypy/dist/pypy/annotation/builtin.py (original) +++ pypy/dist/pypy/annotation/builtin.py Wed Jan 25 11:47:41 2006 @@ -8,7 +8,7 @@ from pypy.annotation.model import SomeUnicodeCodePoint, SomeAddress from pypy.annotation.model import SomeFloat, unionof from pypy.annotation.model import SomePBC, SomeInstance, SomeDict -from pypy.annotation.model import SomeExternalObject, SomeOffset +from pypy.annotation.model import SomeExternalObject from pypy.annotation.model import annotation_to_lltype, lltype_to_annotation from pypy.annotation.model import add_knowntypedata from pypy.annotation.model import s_ImpossibleValue @@ -507,7 +507,7 @@ from pypy.rpython.lltypesystem import llmemory def offsetof(TYPE, fldname): - return SomeOffset() + return SomeInteger() BUILTIN_ANALYZERS[llmemory.offsetof] = offsetof Modified: pypy/dist/pypy/annotation/model.py ============================================================================== --- pypy/dist/pypy/annotation/model.py (original) +++ pypy/dist/pypy/annotation/model.py Wed Jan 25 11:47:41 2006 @@ -452,11 +452,6 @@ def can_be_none(self): return False -class SomeOffset(SomeObject): - immutable = True - def can_be_none(self): - return False - # The following class is used to annotate the intermediate value that # appears in expressions of the form: @@ -514,7 +509,6 @@ annotation_to_ll_map = [ (s_None, lltype.Void), # also matches SomeImpossibleValue() (SomeBool(), lltype.Bool), - (SomeOffset(), lltype.Signed), (SomeInteger(), lltype.Signed), (SomeInteger(size=2), lltype.SignedLongLong), (SomeInteger(nonneg=True, unsigned=True), lltype.Unsigned), Modified: pypy/dist/pypy/rpython/lltypesystem/llmemory.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/llmemory.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/llmemory.py Wed Jan 25 11:47:41 2006 @@ -11,7 +11,7 @@ def annotation(self): from pypy.annotation import model - return model.SomeOffset() + return model.SomeInteger() def lltype(self): return lltype.Signed Modified: pypy/dist/pypy/rpython/memory/test/test_address.py ============================================================================== --- pypy/dist/pypy/rpython/memory/test/test_address.py (original) +++ pypy/dist/pypy/rpython/memory/test/test_address.py Wed Jan 25 11:47:41 2006 @@ -122,7 +122,7 @@ f() a = RPythonAnnotator() s = a.build_types(f, []) - assert isinstance(s, annmodel.SomeOffset) + assert isinstance(s, annmodel.SomeInteger) coff = offsetof(S, 'y') def f(): @@ -130,7 +130,7 @@ f() a = RPythonAnnotator() s = a.build_types(f, []) - assert isinstance(s, annmodel.SomeOffset) + assert isinstance(s, annmodel.SomeInteger) def test_offset_addition(self): from pypy.rpython.lltypesystem import lltype @@ -142,7 +142,7 @@ f() a = RPythonAnnotator() s = a.build_types(f, []) - assert isinstance(s, annmodel.SomeOffset) + assert isinstance(s, annmodel.SomeInteger) coff = offsetof(T, 's2') + offsetof(S, 'y') def f(): @@ -150,7 +150,7 @@ f() a = RPythonAnnotator() s = a.build_types(f, []) - assert isinstance(s, annmodel.SomeOffset) + assert isinstance(s, annmodel.SomeInteger) class TestAddressRTyping(object): Modified: pypy/dist/pypy/rpython/raddress.py ============================================================================== --- pypy/dist/pypy/rpython/raddress.py (original) +++ pypy/dist/pypy/rpython/raddress.py Wed Jan 25 11:47:41 2006 @@ -111,30 +111,3 @@ v_addr1, v_addr2 = hop.inputargs(Address, Address) return hop.genop('adr_ge', [v_addr1, v_addr2], resulttype=lltype.Bool) - -class __extend__(annmodel.SomeOffset): - def rtyper_makerepr(self, rtyper): - return offset_repr - - def rtyper_makekey(self): - return self.__class__, - -class OffsetRepr(Repr): - lowleveltype = lltype.Signed - -offset_repr = OffsetRepr() - -class __extend__(pairtype(OffsetRepr, OffsetRepr)): - def rtype_add((r_offset1, r_offset2), hop): - v_offset1, v_offset2 = hop.inputargs(offset_repr, offset_repr) - return hop.genop('int_add', [v_offset1, v_offset2], resulttype=lltype.Signed) - -class __extend__(pairtype(OffsetRepr, IntegerRepr)): - def rtype_mul((r_offset, r_int), hop): - v_offset, v_int = hop.inputargs(r_offset, r_int) - return hop.genop('int_mul', [v_offset, v_int], resulttype=lltype.Signed) - -class __extend__(pairtype(AddressRepr, OffsetRepr)): - def rtype_add((r_offset1, r_offset2), hop): - v_offset1, v_offset2 = hop.inputargs(Address, offset_repr) - return hop.genop('adr_add', [v_offset1, v_offset2], resulttype=Address) Modified: pypy/dist/pypy/rpython/rbuiltin.py ============================================================================== --- pypy/dist/pypy/rpython/rbuiltin.py (original) +++ pypy/dist/pypy/rpython/rbuiltin.py Wed Jan 25 11:47:41 2006 @@ -392,10 +392,10 @@ # XXX this next little bit is a monstrous hack. the Real Thing awaits # some kind of proper GC integration from pypy.rpython.l3interp import l3interp -from pypy.rpython.raddress import offset_repr +#from pypy.rpython.raddress import offset_repr def rtype_l3malloc(hop): - v_list = hop.inputargs(offset_repr) + v_list = hop.inputargs(lltype.Signed) return hop.genop("call_boehm_gc_alloc", v_list, resulttype=llmemory.Address) Modified: pypy/dist/pypy/rpython/rint.py ============================================================================== --- pypy/dist/pypy/rpython/rint.py (original) +++ pypy/dist/pypy/rpython/rint.py Wed Jan 25 11:47:41 2006 @@ -10,6 +10,7 @@ from pypy.rpython.rarithmetic import intmask, r_int, r_uint, r_ulonglong, r_longlong from pypy.rpython.error import TyperError from pypy.rpython.rmodel import log +from pypy.rpython import objectmodel class __extend__(annmodel.SomeInteger): @@ -204,6 +205,8 @@ class __extend__(IntegerRepr): def convert_const(self, value): + if isinstance(value, objectmodel.Symbolic): + return value if not isinstance(value, (int, r_uint, r_int, r_longlong, r_ulonglong)): # can be bool raise TyperError("not an integer: %r" % (value,)) if self.lowleveltype == Signed: From ac at codespeak.net Wed Jan 25 13:12:00 2006 From: ac at codespeak.net (ac at codespeak.net) Date: Wed, 25 Jan 2006 13:12:00 +0100 (CET) Subject: [pypy-svn] r22638 - in pypy/dist/pypy: doc/discussion jit rpython Message-ID: <20060125121200.75EE427B44@code1.codespeak.net> Author: ac Date: Wed Jan 25 13:11:59 2006 New Revision: 22638 Modified: pypy/dist/pypy/doc/discussion/draft-jit-ideas.txt pypy/dist/pypy/jit/llabstractinterp.py pypy/dist/pypy/jit/llcontainer.py pypy/dist/pypy/jit/llvalue.py pypy/dist/pypy/jit/vlist.py pypy/dist/pypy/rpython/rgenop.py pypy/dist/pypy/rpython/rlist.py Log: (pedronis, arre) First steps in refactoring llabsinterp to use the API in rgenop. Modified: pypy/dist/pypy/doc/discussion/draft-jit-ideas.txt ============================================================================== --- pypy/dist/pypy/doc/discussion/draft-jit-ideas.txt (original) +++ pypy/dist/pypy/doc/discussion/draft-jit-ideas.txt Wed Jan 25 13:11:59 2006 @@ -100,6 +100,8 @@ * genconst(block, llvalue) -> (result var) +* addconst(block, const) -> (result var) + * closeblock1(block) -> link * closeblock2(block, exitswitch-var) -> (false_link, true_link) Modified: pypy/dist/pypy/jit/llabstractinterp.py ============================================================================== --- pypy/dist/pypy/jit/llabstractinterp.py (original) +++ pypy/dist/pypy/jit/llabstractinterp.py Wed Jan 25 13:11:59 2006 @@ -9,6 +9,7 @@ from pypy.jit.llvalue import FlattenMemo, MatchMemo, FreezeMemo, UnfreezeMemo from pypy.jit.llcontainer import LLAbstractContainer, virtualcontainervalue from pypy.jit.llcontainer import hasllcontent +from pypy.rpython import rgenop # ____________________________________________________________ @@ -93,15 +94,15 @@ result.frozen = True # for debugging return result - def unfreeze(self, memo): + def unfreeze(self, memo, block): assert self.frozen, "unfreeze(): state not frozen" if self.back is not None: - new_back = self.back.unfreeze(memo) + new_back = self.back.unfreeze(memo, block) else: new_back = None new_args_a = [] for v, a in zip(self.getlivevars(), self.args_a): - a = a.unfreeze(memo) + a = a.unfreeze(memo, block) # try to preserve the name if isinstance(a.runtimevar, Variable): a.runtimevar.rename(v) @@ -536,18 +537,17 @@ def __init__(self, interp, initialstate, key): self.interp = interp + self.newblock = rgenop.newblock() memo = UnfreezeMemo(key) - self.runningstate = initialstate.unfreeze(memo) + self.runningstate = initialstate.unfreeze(memo, self.newblock) assert list(memo.propagateconsts) == [] # all items consumed self.newinputargs = self.runningstate.getruntimevars() assert len(self.newinputargs) == len(key) # {Variables-of-origblock: a_value} self.bindings = self.runningstate.getbindings() - self.residual_operations = [] def buildblock(self, newexitswitch, newlinkstates): - b = Block([v for v in self.newinputargs if isinstance(v, Variable)]) - b.operations = self.residual_operations + b = self.newblock b.exitswitch = newexitswitch exits = [] for ls in newlinkstates: @@ -559,6 +559,15 @@ b.closeblock(*exits) return b + def genop(self, opname, args, RESULT_TYPE): + return rgenop.genop(self.newblock, opname, args, RESULT_TYPE) + + def genconst(self, llvalue): + return rgenop.genconst(self.newblock, llvalue) + + def addconst(self, const): + return rgenop.addconst(self.newblock, const) + def binding(self, v): if isinstance(v, Constant): return LLAbstractValue(v) @@ -598,16 +607,12 @@ self.record_origin(a_result, args_a) return a_result - def residual(self, opname, args_a, a_result): - v_result = a_result.forcevarorconst(self) - if isinstance(v_result, Constant): - v_result = newvar(v_result.concretetype) - op = SpaceOperation(opname, - [a.forcevarorconst(self) for a in args_a], - v_result) - print 'keep:', op - self.residual_operations.append(op) - + def residual(self, op, args_a): + retvar = rgenop.genop(self.newblock, op.opname, + [a.forcevarorconst(self) for a in args_a], + op.result.concretetype) + return LLAbstractValue(retvar) + def residualize(self, op, args_a, constant_op=None): if constant_op: RESULT = op.result.concretetype @@ -616,10 +621,9 @@ a_result = self.constantfold(constant_op, args_a) if a_result is not None: return a_result - a_result = LLAbstractValue(dupvar(op.result)) + a_result = self.residual(op, args_a) if constant_op: self.record_origin(a_result, args_a) - self.residual(op.opname, args_a, a_result) return a_result def record_origin(self, a_result, args_a): @@ -762,8 +766,6 @@ if not any_concrete: return None - a_result = LLAbstractValue(dupvar(op.result)) - a_real_result = a_result graphstate = self.interp.schedule_graph(args_a, origgraph) #print 'SCHEDULE_GRAPH', args_a, '==>', graphstate.copygraph.name if graphstate.state != "during": @@ -782,12 +784,11 @@ ARGS.append(a.getconcretetype()) new_args_a.append(a) args_a = new_args_a - TYPE = lltype.FuncType(ARGS, a_result.getconcretetype()) + TYPE = lltype.FuncType(ARGS, op.result.concretetype) fptr = lltype.functionptr( TYPE, graphstate.copygraph.name, graph=graphstate.copygraph) a_func = LLAbstractValue(const(fptr)) - self.residual("direct_call", [a_func] + args_a, a_result) - return a_real_result + return self.residual(op, [a_func] + args_a) def op_getfield(self, op, a_ptr, a_attrname): if hasllcontent(a_ptr): @@ -878,9 +879,7 @@ for a in memo.result: v = a.runtimevar if isinstance(v, Variable) and not v.concretetype._is_atomic(): - op = SpaceOperation('keepalive', [v], newvar(lltype.Void)) - print 'virtual:', op - self.residual_operations.append(op) + rgenop.genop(self.newblock, 'keepalive', [v], lltype.Void) return ll_no_return_value # High-level operation dispatcher Modified: pypy/dist/pypy/jit/llcontainer.py ============================================================================== --- pypy/dist/pypy/jit/llcontainer.py (original) +++ pypy/dist/pypy/jit/llcontainer.py Wed Jan 25 13:11:59 2006 @@ -22,7 +22,7 @@ def freeze(self, memo): return self - def unfreeze(self, memo): + def unfreeze(self, memo, block): return self @@ -50,26 +50,22 @@ self.fields[name] = a_value def build_runtime_container(self, builder): - v_result = newvar(lltype.Ptr(self.T)) + RESULT_TYPE = lltype.Ptr(self.T) if self.a_parent is not None: v_parent = self.a_parent.forcevarorconst(builder) - op = SpaceOperation('getsubstruct', [v_parent, - const(self.parentindex, - lltype.Void)], - v_result) - print 'force:', op - builder.residual_operations.append(op) + parentindex = builder.addconst(const(self.parentindex, + lltype.Void)) + v_result = builder.genop('getsubstruct', + [v_parent,parentindex], RESULT_TYPE) else: + t = builder.addconst(const(self.T, lltype.Void)) if self.T._is_varsize(): - op = SpaceOperation('malloc_varsize', [ - const(self.T, lltype.Void), - const(self.length, lltype.Signed)], - v_result) + + v_result = builder.genop('malloc_varsize', + [t, builder.genconst(self.length)], + RESULT_TYPE) else: - op = SpaceOperation('malloc', [const(self.T, lltype.Void)], - v_result) - print 'force:', op - builder.residual_operations.append(op) + v_result = builder.genop('malloc', [t], RESULT_TYPE) self.buildcontent(builder, v_result) return v_result @@ -81,19 +77,15 @@ T = self.fieldtype(name) if isinstance(T, lltype.ContainerType): # initialize the substructure/subarray - v_subptr = newvar(lltype.Ptr(T)) - op = SpaceOperation('getsubstruct', - [v_target, const(name, lltype.Void)], - v_subptr) - print 'force:', op - builder.residual_operations.append(op) + c_name = builder.addconst(const(name, lltype.Void)) + v_subptr = builder.genop('getsubstruct', + [v_target, c_name], + lltype.Ptr(T)) assert isinstance(a_value.content, LLVirtualContainer) a_value.content.buildcontent(builder, v_subptr) else: v_value = a_value.forcevarorconst(builder) - op = self.setop(v_target, name, v_value) - print 'force:', op - builder.residual_operations.append(op) + self.setop(builder, v_target, name, v_value) def flatten(self, memo): assert self not in memo.seen; memo.seen[self] = True # debugging only @@ -138,13 +130,13 @@ result.fields[name] = a_frozen return result - def unfreeze(self, memo): + def unfreeze(self, memo, block): result = self.__class__(self.T, self.length) if self.a_parent is not None: - result.a_parent = self.a_parent.unfreeze(memo) + result.a_parent = self.a_parent.unfreeze(memo, block) result.parentindex = self.parentindex for name in self.names: - a = self.fields[name].unfreeze(memo) + a = self.fields[name].unfreeze(memo, block) result.fields[name] = a return result @@ -168,11 +160,10 @@ def fieldtype(self, name): return getattr(self.T, name) - def setop(self, v_target, name, v_value): - return SpaceOperation('setfield', [v_target, - const(name, lltype.Void), - v_value], - newvar(lltype.Void)) + def setop(self, builder, v_target, name, v_value): + c_name = builder.addconst(const(name, lltype.Void)) + builder.genop('setfield', [v_target, c_name, v_value], + lltype.Void) class LLVirtualArray(LLVirtualContainer): @@ -196,10 +187,10 @@ return self.T.OF def setop(self, v_target, name, v_value): - return SpaceOperation('setarrayitem', [v_target, - const(name, lltype.Signed), - v_value], - newvar(lltype.Void)) + builder.genop('setarrayitem', [v_target, + builder.genconst(name), + v_value], + lltype.Void) def virtualcontainervalue(T, length=None): Modified: pypy/dist/pypy/jit/llvalue.py ============================================================================== --- pypy/dist/pypy/jit/llvalue.py (original) +++ pypy/dist/pypy/jit/llvalue.py Wed Jan 25 13:11:59 2006 @@ -1,6 +1,6 @@ from pypy.objspace.flow.model import Variable, Constant from pypy.rpython.lltypesystem import lltype - +from pypy.rpython import rgenop class LLAbstractValue(object): """An abstract value, propagated through the blocks of the low-level @@ -112,7 +112,7 @@ def flatten(self, memo): pass - def unfreeze(self, memo): + def unfreeze(self, memo, block): return ll_dummy_value def match(self, a_value, memo): @@ -129,7 +129,7 @@ """ pass - def unfreeze(self, memo): + def unfreeze(self, memo, block): """Create an un-frozen copy of a frozen value recursively, creating fresh Variables. """ @@ -160,7 +160,7 @@ """ memo.result.append(self) - def unfreeze(self, memo): + def unfreeze(self, memo, block): """Create an un-frozen copy of a frozen value recursively, creating fresh Variables. """ @@ -171,7 +171,7 @@ v = propagateconst # allowed to propagate as a Constant assert v.concretetype == self.concretetype else: - v = newvar(self.concretetype) + v = rgenop.geninputarg(block, self.concretetype) result = LLAbstractValue(v) result.origin.append(self) return result @@ -196,7 +196,7 @@ memo.seen[self] = True self.fz_content.flatten(memo) - def unfreeze(self, memo): + def unfreeze(self, memo, block): """Create an un-frozen copy of a frozen value recursively, creating fresh Variables. """ @@ -206,7 +206,7 @@ else: a = LLAbstractValue() memo.seen[self] = a - a.content = self.fz_content.unfreeze(memo) + a.content = self.fz_content.unfreeze(memo, block) return a def match(self, a_value, memo): Modified: pypy/dist/pypy/jit/vlist.py ============================================================================== --- pypy/dist/pypy/jit/vlist.py (original) +++ pypy/dist/pypy/jit/vlist.py Wed Jan 25 13:11:59 2006 @@ -34,18 +34,13 @@ items_a = [a.freeze(memo) for a in self.items_a] return LLVirtualList(self.T, items_a) - def unfreeze(self, memo): - items_a = [a.unfreeze(memo) for a in self.items_a] + def unfreeze(self, memo, block): + items_a = [a.unfreeze(memo, block) for a in self.items_a] return LLVirtualList(self.T, items_a) def build_runtime_container(self, builder): items_v = [a.forcevarorconst(builder) for a in self.items_a] - llop = LowLevelOpList(None) - v_result = self.T.list_builder(llop, items_v) - print 'list_builder:' - for op in llop: - print '\t', op - builder.residual_operations.append(op) + v_result = self.T.list_builder(builder, items_v) return v_result # ____________________________________________________________ Modified: pypy/dist/pypy/rpython/rgenop.py ============================================================================== --- pypy/dist/pypy/rpython/rgenop.py (original) +++ pypy/dist/pypy/rpython/rgenop.py Wed Jan 25 13:11:59 2006 @@ -18,6 +18,9 @@ return v def genop(block, opname, vars, RESULT_TYPE): + for v in vars: + assert isinstance(v, (flowmodel.Constant, flowmodel.Variable)) + v = flowmodel.Variable() v.concretetype = RESULT_TYPE op = flowmodel.SpaceOperation(opname, vars, v) @@ -29,6 +32,10 @@ v.concretetype = lltype.typeOf(llvalue) return v +def addconst(block, const): + assert isinstance(const, flowmodel.Constant) + return const + def closeblock1(block): link = flowmodel.Link([], None) block.closeblock(link) Modified: pypy/dist/pypy/rpython/rlist.py ============================================================================== --- pypy/dist/pypy/rpython/rlist.py (original) +++ pypy/dist/pypy/rpython/rlist.py Wed Jan 25 13:11:59 2006 @@ -175,21 +175,25 @@ fnptr = list_repr.rtyper.annotate_helper_fn(ll_setitem_nonneg, argtypes) self.c_setitem_nonneg = inputconst(typeOf(fnptr), fnptr) self.c_dum_nocheck = inputconst(Void, dum_nocheck) + self.c_LIST = inputconst(Void, self.LIST) - def build(self, llop, items_v): + def build(self, builder, items_v): """Make the operations that would build a list containing the provided items.""" - # This is called by the JIT with llop.rtyper == None, so - # llop.gendirectcall() doesn't work. Instead, we need to call - # directly the c_newitem and c_setitem_nonneg precomputed above. - c_list = inputconst(Void, self.LIST) - c_len = inputconst(Signed, len(items_v)) - v_result = llop.genop('direct_call', [self.c_newlist, c_list, c_len], - resulttype = self.LISTPTR) + c_list = builder.addconst(self.c_LIST) + c_newlist = builder.addconst(self.c_newlist) + c_len = builder.genconst(len(items_v)) + v_result = builder.genop('direct_call', + [c_newlist, c_list, c_len], + self.LISTPTR) + c_setitem_nonneg = builder.addconst(self.c_setitem_nonneg) + c_dum_nocheck = builder.addconst(self.c_dum_nocheck) for i, v in enumerate(items_v): - llop.genop('direct_call', [self.c_setitem_nonneg, - self.c_dum_nocheck, - v_result, inputconst(Signed, i), v]) + c_i = builder.genconst(i) + builder.genop('direct_call', [c_setitem_nonneg, + c_dum_nocheck, + v_result, c_i, v], + Void) return v_result class ListRepr(BaseListRepr): From auc at codespeak.net Wed Jan 25 13:50:29 2006 From: auc at codespeak.net (auc at codespeak.net) Date: Wed, 25 Jan 2006 13:50:29 +0100 (CET) Subject: [pypy-svn] r22640 - pypy/dist/pypy/lib/logic Message-ID: <20060125125029.48AFE27B44@code1.codespeak.net> Author: auc Date: Wed Jan 25 13:50:25 2006 New Revision: 22640 Added: pypy/dist/pypy/lib/logic/distributor.py Modified: pypy/dist/pypy/lib/logic/computationspace.py pypy/dist/pypy/lib/logic/test_computationspace.py pypy/dist/pypy/lib/logic/test_unification.py pypy/dist/pypy/lib/logic/test_variable.py pypy/dist/pypy/lib/logic/unification.py pypy/dist/pypy/lib/logic/variable.py Log: (ale, auc) * add distributor from logilab constraint solver * adapt it * method to get vars by name * tests Modified: pypy/dist/pypy/lib/logic/computationspace.py ============================================================================== --- pypy/dist/pypy/lib/logic/computationspace.py (original) +++ pypy/dist/pypy/lib/logic/computationspace.py Wed Jan 25 13:50:25 2006 @@ -141,15 +141,15 @@ ## Choose ## ------ -## Y=choose(N) waits until the current space becomes stable, blocks the -## current thread, and then creates a choice point with N alternatives -## in the current space. The blocked choose call waits for an -## alternative to be chosen by a commit operation on the space. The -## choose call only defines how many alternatives there are; it does -## not specify what to do for an alternative. Eventually, choose -## continues its execution with Y=I when alternative I (1= 1] + domlist.sort() + return domlist[0][1] + + def findLargestDomain(self, domains): + """returns the variable having the largest domain. + (or one of such variables if there is a tie) + """ + domlist = [(dom.size(), variable) for variable, dom in domains.items() + if dom.size() > 1] + domlist.sort() + return domlist[-1][1] + + def nb_subdomains(self, domains): + """return number of sub domains to explore""" + return self.nb_subspaces + + def distribute(self, domains, verbose=0): + """do the minimal job and let concrete class distribute variables + """ + self.verbose = verbose + replicas = [] + for i in range(self.nb_subdomains(domains)): + replicas.append(make_new_domains(domains)) + modified_domains = self._distribute(*replicas) + for domain in modified_domains: + domain.reset_flags() + return replicas + + def _distribute(self, *args): + """ method to implement in concrete class + + take self.nb_subspaces copy of the original domains as argument + distribute the domains and return each modified domain + """ + raise NotImplementedError("Use a concrete implementation of " + "the Distributor interface") + +class NaiveDistributor(AbstractDistributor): + """distributes domains by splitting the smallest domain in 2 new domains + The first new domain has a size of one, + and the second has all the other values""" + + def __init__(self): + AbstractDistributor.__init__(self) + + def _distribute(self, dom1, dom2): + """See AbstractDistributor""" + variable = self.findSmallestDomain(dom1) + values = dom1[variable].get_values() + if self.verbose: + print 'Distributing domain for variable', variable, \ + 'at value', values[0] + dom1[variable].remove_values(values[1:]) + dom2[variable].remove_value(values[0]) + return (dom1[variable], dom2[variable]) + + +class RandomizingDistributor(AbstractDistributor): + """distributes domains as the NaiveDistrutor, except that the unique + value of the first domain is picked at random.""" + + def __init__(self): + AbstractDistributor.__init__(self) + + def _distribute(self, dom1, dom2): + """See AbstractDistributor""" + variable = self.findSmallestDomain(dom1) + values = dom1[variable].get_values() + distval = random.choice(values) + values.remove(distval) + if self.verbose: + print 'Distributing domain for variable', variable, \ + 'at value', distval + dom1[variable].remove_values(values) + dom2[variable].remove_value(distval) + return (dom1[variable], dom2[variable]) + + +class SplitDistributor(AbstractDistributor): + """distributes domains by splitting the smallest domain in + nb_subspaces equal parts or as equal as possible. + If nb_subspaces is 0, then the smallest domain is split in + domains of size 1""" + + def __init__(self, nb_subspaces=3): + AbstractDistributor.__init__(self, nb_subspaces) + self.__to_split = None + def nb_subdomains(self, domains): + """See AbstractDistributor""" + self.__to_split = self.findSmallestDomain(domains) + if self.nb_subspaces: + return min(self.nb_subspaces, domains[self.__to_split].size()) + else: + return domains[self.__to_split].size() + + def _distribute(self, *args): + """See AbstractDistributor""" + variable = self.__to_split + nb_subspaces = len(args) + values = args[0][variable].get_values() + nb_elts = max(1, len(values)*1./nb_subspaces) + slices = [(int(math.floor(index * nb_elts)), + int(math.floor((index + 1) * nb_elts))) + for index in range(nb_subspaces)] + if self.verbose: + print 'Distributing domain for variable', variable + modified = [] + for (dom, (end, start)) in zip(args, slices) : + dom[variable].remove_values(values[:end]) + dom[variable].remove_values(values[start:]) + modified.append(dom[variable]) + return modified + +class DichotomyDistributor(SplitDistributor): + """distributes domains by splitting the smallest domain in + two equal parts or as equal as possible""" + def __init__(self): + SplitDistributor.__init__(self, 2) + + +class EnumeratorDistributor(SplitDistributor): + """distributes domains by splitting the smallest domain + in domains of size 1.""" + def __init__(self): + SplitDistributor.__init__(self, 0) + +DefaultDistributor = DichotomyDistributor Modified: pypy/dist/pypy/lib/logic/test_computationspace.py ============================================================================== --- pypy/dist/pypy/lib/logic/test_computationspace.py (original) +++ pypy/dist/pypy/lib/logic/test_computationspace.py Wed Jan 25 13:50:25 2006 @@ -2,24 +2,27 @@ import variable as v import constraint as c import computationspace as cs +import distributor as di from py.test import raises -def satisfiable_problem(store): - s = store # i'm lazy +def satisfiable_problem(computation_space): + cs = computation_space + s = cs.store x, y, z, w = (s.var('x'), s.var('y'), s.var('z'), s.var('w')) s.set_domain(x, c.FiniteDomain([2, 6])) s.set_domain(y, c.FiniteDomain([2, 3])) s.set_domain(z, c.FiniteDomain([4, 5])) - s.set_domain(w, c.FiniteDomain([1, 4, 5])) + s.set_domain(w, c.FiniteDomain([1, 4, 5, 6, 7])) s.add_constraint(c.Expression([x, y, z], 'x == y + z')) s.add_constraint(c.Expression([z, w], 'z < w')) - # we don't know yet how to # set up a distribution strategy + cs.set_distributor(di.DichotomyDistributor()) return (x, w, y) -def unsatisfiable_problem(store): - s = store # i'm lazy +def unsatisfiable_problem(computation_space): + cs = computation_space + s = cs.store x, y, z, w = (s.var('x'), s.var('y'), s.var('z'), s.var('w')) s.set_domain(x, c.FiniteDomain([2, 6])) @@ -28,8 +31,8 @@ s.set_domain(w, c.FiniteDomain([1])) s.add_constraint(c.Expression([x, y, z], 'x == y + z')) s.add_constraint(c.Expression([z, w], 'z < w')) - # we don't know yet how to # set up a distribution strategy + cs.set_distributor(di.DichotomyDistributor()) return (x, w, y) @@ -56,5 +59,23 @@ assert spc.ask() == cs.Unprocessed spc.process() assert spc.ask() == cs.Failed - - + + def test_distribute(self): + spc = cs.ComputationSpace(satisfiable_problem) + spc.process() + domains = dict([(var, var.dom) for var in spc.store.vars + if var.dom]) + new_domains = spc.distributor.distribute(domains) + x, y, z, w = (spc.store.get_var_by_name('x'), + spc.store.get_var_by_name('y'), + spc.store.get_var_by_name('z'), + spc.store.get_var_by_name('w')) + assert new_domains == [{x: c.FiniteDomain([6]), + y: c.FiniteDomain([2]), + z: c.FiniteDomain([4]), + w: c.FiniteDomain([5])}, + {x: c.FiniteDomain([6]), + y: c.FiniteDomain([2]), + z: c.FiniteDomain([4]), + w: c.FiniteDomain([6, 7])}] + Modified: pypy/dist/pypy/lib/logic/test_unification.py ============================================================================== --- pypy/dist/pypy/lib/logic/test_unification.py (original) +++ pypy/dist/pypy/lib/logic/test_unification.py Wed Jan 25 13:50:25 2006 @@ -21,7 +21,7 @@ def test_already_in_store(self): x = u.var('x') - raises(v.AlreadyExists, u.var, 'x') + raises(v.AlreadyInStore, u.var, 'x') def test_already_bound(self): x = u.var('x') Modified: pypy/dist/pypy/lib/logic/test_variable.py ============================================================================== --- pypy/dist/pypy/lib/logic/test_variable.py (original) +++ pypy/dist/pypy/lib/logic/test_variable.py Wed Jan 25 13:50:25 2006 @@ -24,10 +24,14 @@ def setup_method(self, meth): u._store = u.Store() - def test_no_same_name(self): x = u.var('x') - raises(v.AlreadyExists, u.var, 'x') + raises(u.AlreadyInStore, u.var, 'x') + + def test_get_by_name(self): + x = u.var('x') + assert x == u._store.get_var_by_name('x') + raises(u.NotInStore, u._store.get_var_by_name, 'y') def test_one_thread_reading_one_var(self): cons = Consumer() Modified: pypy/dist/pypy/lib/logic/unification.py ============================================================================== --- pypy/dist/pypy/lib/logic/unification.py (original) +++ pypy/dist/pypy/lib/logic/unification.py Wed Jan 25 13:50:25 2006 @@ -120,7 +120,8 @@ import threading -from variable import EqSet, Var, VariableException, NotAVariable +from variable import EqSet, Var, \ + VariableException, NotAVariable, AlreadyInStore from constraint import FiniteDomain, ConsistencyFailure #----------- Store Exceptions ---------------------------- @@ -132,9 +133,9 @@ def __str__(self): return "%s is already bound" % self.name -class AlreadyInStore(VariableException): +class NotInStore(VariableException): def __str__(self): - return "%s already in store" % self.name + return "%s not in the store" % self.name class OutOfDomain(VariableException): def __str__(self): @@ -169,9 +170,9 @@ (also called determined variables).""" def __init__(self): - # mapping of names to vars (all of them) self.vars = set() - self.names = set() + # mapping of names to vars (all of them) + self.names = {} # set of all constraints self.constraints = set() # consistency-preserving stuff @@ -193,7 +194,7 @@ raise AlreadyInStore(var.name) print "adding %s to the store" % var self.vars.add(var) - self.names.add(var.name) + self.names[var.name] = var # put into new singleton equiv. set var.val = EqSet([var]) @@ -203,6 +204,12 @@ if var.is_bound(): raise AlreadyBound var.dom = FiniteDomain(dom) + + def get_var_by_name(self, name): + try: + return self.names[name] + except KeyError: + raise NotInStore(name) #-- Constraints ------------------------- Modified: pypy/dist/pypy/lib/logic/variable.py ============================================================================== --- pypy/dist/pypy/lib/logic/variable.py (original) +++ pypy/dist/pypy/lib/logic/variable.py Wed Jan 25 13:50:25 2006 @@ -7,13 +7,13 @@ def __init__(self, name): self.name = name -class NotAVariable(VariableException): +class AlreadyInStore(VariableException): def __str__(self): - return "%s is not a variable" % self.name + return "%s already in store" % self.name -class AlreadyExists(VariableException): +class NotAVariable(VariableException): def __str__(self): - return "%s already exists" % self.name + return "%s is not a variable" % self.name #----------- Variables ---------------------------------- class EqSet(set): @@ -32,7 +32,7 @@ def __init__(self, name, store): if name in store.names: - raise AlreadyExists(name) + raise AlreadyInStore(name) self.name = name self.store = store # top-level 'commited' binding @@ -98,16 +98,10 @@ def add_constraint(self, constraint): self.constraints.add(constraint) - #---- Concurrent public ops -------------------------- + is_bound = _is_bound - def is_bound(self): - try: - self.mutex.acquire() - res = self._is_bound() - finally: - self.mutex.release() - return res + #---- Concurrent public ops -------------------------- # should be used by threads that want to block on # unbound variables def get(self): From tismer at codespeak.net Wed Jan 25 14:07:38 2006 From: tismer at codespeak.net (tismer at codespeak.net) Date: Wed, 25 Jan 2006 14:07:38 +0100 (CET) Subject: [pypy-svn] r22641 - pypy/dist/pypy/translator/goal Message-ID: <20060125130738.E884627B69@code1.codespeak.net> Author: tismer Date: Wed Jan 25 14:07:34 2006 New Revision: 22641 Modified: pypy/dist/pypy/translator/goal/translate_pypy.py Log: typo in option passing from target Modified: pypy/dist/pypy/translator/goal/translate_pypy.py ============================================================================== --- pypy/dist/pypy/translator/goal/translate_pypy.py (original) +++ pypy/dist/pypy/translator/goal/translate_pypy.py Wed Jan 25 14:07:34 2006 @@ -224,7 +224,7 @@ # target specific defaults taking over if 'opt_defaults' in targetspec_dic: - opt_parser.set_defaults(targetspec_dic['op_defaults']) + opt_parser.set_defaults(targetspec_dic['opt_defaults']) if options.help: opt_parser.print_help() From stephan at codespeak.net Wed Jan 25 14:11:07 2006 From: stephan at codespeak.net (stephan at codespeak.net) Date: Wed, 25 Jan 2006 14:11:07 +0100 (CET) Subject: [pypy-svn] r22642 - in pypy/dist/pypy: annotation rpython/rctypes rpython/rctypes/test Message-ID: <20060125131107.D602327B70@code1.codespeak.net> Author: stephan Date: Wed Jan 25 14:10:57 2006 New Revision: 22642 Modified: pypy/dist/pypy/annotation/bookkeeper.py pypy/dist/pypy/annotation/model.py pypy/dist/pypy/rpython/rctypes/implementation.py pypy/dist/pypy/rpython/rctypes/interface.py pypy/dist/pypy/rpython/rctypes/test/_rctypes_test.c pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py Log: ctype Structures can be annotated Modified: pypy/dist/pypy/annotation/bookkeeper.py ============================================================================== --- pypy/dist/pypy/annotation/bookkeeper.py (original) +++ pypy/dist/pypy/annotation/bookkeeper.py Wed Jan 25 14:10:57 2006 @@ -354,8 +354,10 @@ elif ishashable(x) and x in BUILTIN_ANALYZERS: _module = getattr(x,"__module__","unknown") result = SomeBuiltin(BUILTIN_ANALYZERS[x], methodname="%s.%s" % (_module, x.__name__)) - elif hasattr(x, "compute_result_annotation"): + elif hasattr(tp, "compute_result_annotation"): result = SomeBuiltin(x.compute_result_annotation, methodname=x.__name__) + elif hasattr(tp, "compute_annotation"): + result = tp.compute_annotation() elif tp in EXTERNAL_TYPE_ANALYZERS: result = SomeExternalObject(tp) elif isinstance(x, lltype._ptr): @@ -501,6 +503,8 @@ return s_None elif t in EXTERNAL_TYPE_ANALYZERS: return SomeExternalObject(t) + elif hasattr(t, "compute_annotation"): + return t.compute_annotation() elif t.__module__ != '__builtin__' and t not in self.pbctypes: classdef = self.getuniqueclassdef(t) return SomeInstance(classdef) Modified: pypy/dist/pypy/annotation/model.py ============================================================================== --- pypy/dist/pypy/annotation/model.py (original) +++ pypy/dist/pypy/annotation/model.py Wed Jan 25 14:10:57 2006 @@ -426,6 +426,15 @@ def can_be_none(self): return True +class SomeCTypesObject(SomeObject): + """Stands for an object of the ctypes module.""" + + def __init__(self, knowntype): + self.knowntype = knowntype + + def can_be_none(self): + return False + class SomeImpossibleValue(SomeObject): """The empty set. Instances are placeholders for objects that Modified: pypy/dist/pypy/rpython/rctypes/implementation.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/implementation.py (original) +++ pypy/dist/pypy/rpython/rctypes/implementation.py Wed Jan 25 14:10:57 2006 @@ -7,7 +7,7 @@ from ctypes import _FUNCFLAG_CDECL if sys.platform == "win32": from ctypes import _FUNCFLAG_STDCALL -from pypy.annotation.model import SomeInteger +from pypy.annotation.model import SomeInteger, SomeCTypesObject from pypy.rpython.lltypesystem.lltype import Signed @@ -22,13 +22,17 @@ POINTER(c_char).wrap_arg = staticmethod( lambda ll_type, arg_name: "RPyString_AsString(%s)" % arg_name) + class FunctionPointerTranslation(object): def compute_result_annotation(self, *args_s): """ Answer the annotation of the external function's result """ - return self.restype.annotator_type + try: + return self.restype.annotator_type + except AttributeError: + return SomeCTypesObject(self.restype) def __hash__(self): return id(self) @@ -50,6 +54,13 @@ return answer +class RStructure(Structure): + + def compute_annotation(cls): + return SomeCTypesObject(cls) + compute_annotation = classmethod(compute_annotation) + + class RCDLL(CDLL): """ This is the restricted version of ctypes' CDLL class. Modified: pypy/dist/pypy/rpython/rctypes/interface.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/interface.py (original) +++ pypy/dist/pypy/rpython/rctypes/interface.py Wed Jan 25 14:10:57 2006 @@ -1,5 +1,5 @@ from ctypes import _DLLS -from implementation import RCDLL as CDLL, c_int, c_char_p, c_char, POINTER, Structure, byref +from implementation import RCDLL as CDLL, c_int, c_char_p, c_char, POINTER, RStructure as Structure, byref try: from implementation import RWinDLL as WinDLL except ImportError: Modified: pypy/dist/pypy/rpython/rctypes/test/_rctypes_test.c ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/_rctypes_test.c (original) +++ pypy/dist/pypy/rpython/rctypes/test/_rctypes_test.c Wed Jan 25 14:10:57 2006 @@ -28,6 +28,16 @@ return in.x + in.y; } +EXPORT(int) _testfunc_struct(point in) +{ + return in.x + in.y; +} + +EXPORT(point) _testfunc_struct_id(point in) +{ + return in; +} + DL_EXPORT(void) init_rctypes_test(void) { Modified: pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py Wed Jan 25 14:10:57 2006 @@ -35,6 +35,34 @@ mod.tagpoint = tagpoint mod.byref = byref + # compile and load our local test C file + compile_c_module([py.path.local("_rctypes_test.c")], "_rctypes_test") + + if sys.platform == "win32": + _rctypes_test = cdll.LoadLibrary("_rctypes_test.pyd") + else: + _rctypes_test = cdll.LoadLibrary("_rctypes_test.so") + + # _test_struct + testfunc_struct = _rctypes_test._testfunc_struct + testfunc_struct.restype = c_int + testfunc_struct.argtypes = [tagpoint] + + def py_testfunc_struct(inpoint): + return testfunc_struct(inpoint) + + mod.py_testfunc_struct = py_testfunc_struct + + # _test_struct_id + testfunc_struct_id = _rctypes_test._testfunc_struct_id + testfunc_struct_id.restype = tagpoint + testfunc_struct_id.argtypes = [tagpoint] + + def py_testfunc_struct_id(inpoint): + return testfunc_struct_id(inpoint) + + mod.py_testfunc_struct_id = py_testfunc_struct_id + class Test_rctypes: @@ -65,9 +93,6 @@ class Test_structure: - def setup_class(cls): - compile_c_module([py.path.local("_rctypes_test.c")], "_rctypes_test") - def test_simple_as_extension_module(self): import _rctypes_test as t0 import _rctypes_test as t1 @@ -87,4 +112,24 @@ assert out_point.x == 42 assert out_point.y == 17 + def test_structure(self): + in_point = tagpoint() + in_point.x = 10 + in_point.y = 20 + res = py_testfunc_struct(in_point) + assert res == 30 + + def test_annotate_struct(self): + a = RPythonAnnotator() + s = a.build_types(py_testfunc_struct, [int]) + assert s.knowntype == int + + + def test_annotate_struct(self): + a = RPythonAnnotator() + #try: + s = a.build_types(py_testfunc_struct_id, [tagpoint]) + #finally: + # a.translator.view() + assert s.knowntype == tagpoint From tismer at codespeak.net Wed Jan 25 14:15:32 2006 From: tismer at codespeak.net (tismer at codespeak.net) Date: Wed, 25 Jan 2006 14:15:32 +0100 (CET) Subject: [pypy-svn] r22643 - pypy/dist/pypy/translator/goal Message-ID: <20060125131532.E653A27B5B@code1.codespeak.net> Author: tismer Date: Wed Jan 25 14:15:29 2006 New Revision: 22643 Modified: pypy/dist/pypy/translator/goal/translate_pypy.py Log: optparse.set_defaults needs to be called using ** syntax Modified: pypy/dist/pypy/translator/goal/translate_pypy.py ============================================================================== --- pypy/dist/pypy/translator/goal/translate_pypy.py (original) +++ pypy/dist/pypy/translator/goal/translate_pypy.py Wed Jan 25 14:15:29 2006 @@ -224,7 +224,7 @@ # target specific defaults taking over if 'opt_defaults' in targetspec_dic: - opt_parser.set_defaults(targetspec_dic['opt_defaults']) + opt_parser.set_defaults(**targetspec_dic['opt_defaults']) if options.help: opt_parser.print_help() From tismer at codespeak.net Wed Jan 25 14:16:07 2006 From: tismer at codespeak.net (tismer at codespeak.net) Date: Wed, 25 Jan 2006 14:16:07 +0100 (CET) Subject: [pypy-svn] r22644 - pypy/dist/pypy/translator/goal Message-ID: <20060125131607.22A8C27B84@code1.codespeak.net> Author: tismer Date: Wed Jan 25 14:16:02 2006 New Revision: 22644 Modified: pypy/dist/pypy/translator/goal/targetpypy_stacklesstest.py Log: refining and adjusting the build Modified: pypy/dist/pypy/translator/goal/targetpypy_stacklesstest.py ============================================================================== --- pypy/dist/pypy/translator/goal/targetpypy_stacklesstest.py (original) +++ pypy/dist/pypy/translator/goal/targetpypy_stacklesstest.py Wed Jan 25 14:16:02 2006 @@ -58,7 +58,13 @@ stackless_options = { 'text': True, - 'lowmem': True, + 'stackless': True, + } + +# this gets folded into the global options + +opt_defaults = { + "lowmem": True, } def opt_parser(): From stephan at codespeak.net Wed Jan 25 15:02:01 2006 From: stephan at codespeak.net (stephan at codespeak.net) Date: Wed, 25 Jan 2006 15:02:01 +0100 (CET) Subject: [pypy-svn] r22649 - in pypy/dist/pypy: annotation rpython/rctypes rpython/rctypes/test Message-ID: <20060125140201.9740927B64@code1.codespeak.net> Author: stephan Date: Wed Jan 25 15:01:56 2006 New Revision: 22649 Modified: pypy/dist/pypy/annotation/bookkeeper.py pypy/dist/pypy/annotation/model.py pypy/dist/pypy/annotation/unaryop.py pypy/dist/pypy/rpython/rctypes/implementation.py pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py Log: Structure creation, setattr, getattr gets annotated Modified: pypy/dist/pypy/annotation/bookkeeper.py ============================================================================== --- pypy/dist/pypy/annotation/bookkeeper.py (original) +++ pypy/dist/pypy/annotation/bookkeeper.py Wed Jan 25 15:01:56 2006 @@ -354,7 +354,7 @@ elif ishashable(x) and x in BUILTIN_ANALYZERS: _module = getattr(x,"__module__","unknown") result = SomeBuiltin(BUILTIN_ANALYZERS[x], methodname="%s.%s" % (_module, x.__name__)) - elif hasattr(tp, "compute_result_annotation"): + elif hasattr(x, "compute_result_annotation"): result = SomeBuiltin(x.compute_result_annotation, methodname=x.__name__) elif hasattr(tp, "compute_annotation"): result = tp.compute_annotation() Modified: pypy/dist/pypy/annotation/model.py ============================================================================== --- pypy/dist/pypy/annotation/model.py (original) +++ pypy/dist/pypy/annotation/model.py Wed Jan 25 15:01:56 2006 @@ -435,7 +435,6 @@ def can_be_none(self): return False - class SomeImpossibleValue(SomeObject): """The empty set. Instances are placeholders for objects that will never show up at run-time, e.g. elements of an empty list.""" Modified: pypy/dist/pypy/annotation/unaryop.py ============================================================================== --- pypy/dist/pypy/annotation/unaryop.py (original) +++ pypy/dist/pypy/annotation/unaryop.py Wed Jan 25 15:01:56 2006 @@ -7,6 +7,7 @@ SomeDict, SomeUnicodeCodePoint, SomeTuple, SomeImpossibleValue, \ SomeInstance, SomeBuiltin, SomeFloat, SomeIterator, SomePBC, \ SomeExternalObject, SomeTypedAddressAccess, SomeAddress, \ + SomeCTypesObject,\ unionof, set, missing_operation, add_knowntypedata from pypy.annotation.bookkeeper import getbookkeeper from pypy.annotation import builtin @@ -620,6 +621,21 @@ v = smeth(*llargs) return ll_to_annotation(v) +class __extend__(SomeCTypesObject): + def setattr(cto, s_attr, s_value): + pass + + def getattr(cto, s_attr): + if s_attr.is_constant() and isinstance(s_attr.const, str): + attr = s_attr.const + atype = cto.knowntype._fielddef_[attr] + try: + return atype.annotator_type + except AttributeError: + return SomeCTypesObject(atype) + else: + return SomeObject() + #_________________________________________ # memory addresses Modified: pypy/dist/pypy/rpython/rctypes/implementation.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/implementation.py (original) +++ pypy/dist/pypy/rpython/rctypes/implementation.py Wed Jan 25 15:01:56 2006 @@ -54,12 +54,32 @@ return answer +class RStructureMeta(type(Structure)): + def __new__(mta,name,bases,clsdict): + _fields = clsdict.get('_fields_',None) + _adict = {} + if _fields is not None: + for attr, atype in _fields: + _adict[attr] = atype + clsdict['_fielddef_'] = _adict + + return super(RStructureMeta,mta).__new__(mta, name, bases, clsdict) + class RStructure(Structure): + __metaclass__ = RStructureMeta + def compute_annotation(cls): return SomeCTypesObject(cls) compute_annotation = classmethod(compute_annotation) + def compute_result_annotation(cls, *args_s): + """ + Answer the result annotation of calling 'cls'. + """ + return SomeCTypesObject(cls) + compute_result_annotation = classmethod(compute_result_annotation) + class RCDLL(CDLL): """ Modified: pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py Wed Jan 25 15:01:56 2006 @@ -63,6 +63,14 @@ mod.py_testfunc_struct_id = py_testfunc_struct_id + def py_create_point(): + p = tagpoint() + p.x = 10 + p.y = 20 + return p.x + p.y + + mod.py_create_point = py_create_point + class Test_rctypes: @@ -127,9 +135,11 @@ def test_annotate_struct(self): a = RPythonAnnotator() - #try: s = a.build_types(py_testfunc_struct_id, [tagpoint]) - #finally: - # a.translator.view() assert s.knowntype == tagpoint + def test_create_point(self): + a = RPythonAnnotator() + s = a.build_types(py_create_point,[]) + assert s.knowntype == int + From stephan at codespeak.net Wed Jan 25 15:55:05 2006 From: stephan at codespeak.net (stephan at codespeak.net) Date: Wed, 25 Jan 2006 15:55:05 +0100 (CET) Subject: [pypy-svn] r22650 - in pypy/dist/pypy: annotation rpython/rctypes rpython/rctypes/test Message-ID: <20060125145505.0B22327B64@code1.codespeak.net> Author: stephan Date: Wed Jan 25 15:55:01 2006 New Revision: 22650 Modified: pypy/dist/pypy/annotation/unaryop.py pypy/dist/pypy/rpython/rctypes/implementation.py pypy/dist/pypy/rpython/rctypes/interface.py pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py Log: ctypes byref function is annotated Modified: pypy/dist/pypy/annotation/unaryop.py ============================================================================== --- pypy/dist/pypy/annotation/unaryop.py (original) +++ pypy/dist/pypy/annotation/unaryop.py Wed Jan 25 15:55:01 2006 @@ -628,7 +628,7 @@ def getattr(cto, s_attr): if s_attr.is_constant() and isinstance(s_attr.const, str): attr = s_attr.const - atype = cto.knowntype._fielddef_[attr] + atype = cto.knowntype._fields_def_[attr] try: return atype.annotator_type except AttributeError: Modified: pypy/dist/pypy/rpython/rctypes/implementation.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/implementation.py (original) +++ pypy/dist/pypy/rpython/rctypes/implementation.py Wed Jan 25 15:55:01 2006 @@ -10,7 +10,6 @@ from pypy.annotation.model import SomeInteger, SomeCTypesObject from pypy.rpython.lltypesystem.lltype import Signed - c_int.annotator_type = SomeInteger() c_int.ll_type = Signed c_char_p.annotator_type = None @@ -61,7 +60,7 @@ if _fields is not None: for attr, atype in _fields: _adict[attr] = atype - clsdict['_fielddef_'] = _adict + clsdict['_fields_def_'] = _adict return super(RStructureMeta,mta).__new__(mta, name, bases, clsdict) @@ -80,6 +79,23 @@ return SomeCTypesObject(cls) compute_result_annotation = classmethod(compute_result_annotation) +class RByrefObj(object): + + def __init__(self): + self.__name__ = 'RByrefObj' + + def compute_result_annotation(cls, s_arg): + """ + Answer the result annotation of calling 'byref'. + """ + return SomeCTypesObject(POINTER(s_arg.knowntype)) + + compute_result_annotation = classmethod(compute_result_annotation) + + def __call__(self,obj): + return byref(obj) + +RByref = RByrefObj() class RCDLL(CDLL): """ Modified: pypy/dist/pypy/rpython/rctypes/interface.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/interface.py (original) +++ pypy/dist/pypy/rpython/rctypes/interface.py Wed Jan 25 15:55:01 2006 @@ -1,5 +1,6 @@ from ctypes import _DLLS -from implementation import RCDLL as CDLL, c_int, c_char_p, c_char, POINTER, RStructure as Structure, byref +from implementation import RCDLL as CDLL, c_int, c_char_p, c_char, POINTER, \ + RStructure as Structure, RByref as byref try: from implementation import RWinDLL as WinDLL except ImportError: Modified: pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py Wed Jan 25 15:55:01 2006 @@ -29,6 +29,7 @@ return atoi(a) mod.o_atoi = o_atoi mod.cdll = cdll + class tagpoint(Structure): _fields_ = [("x", c_int), ("y", c_int)] @@ -43,6 +44,19 @@ else: _rctypes_test = cdll.LoadLibrary("_rctypes_test.so") + # _testfunc_byval + testfunc_byval = _rctypes_test._testfunc_byval + testfunc_byval.restype = c_int + testfunc_byval.argtypes = [tagpoint,POINTER(tagpoint)] + + def py_testfunc_byval(inpoint): + opoint = tagpoint() + res = testfunc_byval(inpoint,byref(opoint)) + + return res, opoint + + mod.py_testfunc_byval = py_testfunc_byval + # _test_struct testfunc_struct = _rctypes_test._testfunc_struct testfunc_struct.restype = c_int @@ -143,3 +157,11 @@ s = a.build_types(py_create_point,[]) assert s.knowntype == int + def test_annotate_byval(self): + a = RPythonAnnotator() + s = a.build_types(py_testfunc_byval,[tagpoint]) + assert s.knowntype == tuple + assert len(s.items) == 2 + assert s.items[0].knowntype == int + assert s.items[1].knowntype == tagpoint + From tismer at codespeak.net Wed Jan 25 16:07:27 2006 From: tismer at codespeak.net (tismer at codespeak.net) Date: Wed, 25 Jan 2006 16:07:27 +0100 (CET) Subject: [pypy-svn] r22651 - in pypy/dist/pypy/module/stackless: . test Message-ID: <20060125150727.77BA727B64@code1.codespeak.net> Author: tismer Date: Wed Jan 25 16:07:21 2006 New Revision: 22651 Added: pypy/dist/pypy/module/stackless/ (props changed) pypy/dist/pypy/module/stackless/__init__.py (contents, props changed) pypy/dist/pypy/module/stackless/interp_coroutine.py (contents, props changed) pypy/dist/pypy/module/stackless/interp_stackless.py (contents, props changed) pypy/dist/pypy/module/stackless/test/ (props changed) pypy/dist/pypy/module/stackless/test/__init__.py (contents, props changed) pypy/dist/pypy/module/stackless/test/test_coroutine.py (contents, props changed) pypy/dist/pypy/module/stackless/test/test_interp_coroutine.py (contents, props changed) pypy/dist/pypy/module/stackless/test/test_stackless.py (contents, props changed) Log: starting to expose coroutines at application level, providing the building blocks for greenlets and tasklets and whatsoever Added: pypy/dist/pypy/module/stackless/__init__.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/module/stackless/__init__.py Wed Jan 25 16:07:21 2006 @@ -0,0 +1,15 @@ +# Package initialisation +from pypy.interpreter.mixedmodule import MixedModule + +class Module(MixedModule): + """ + This module implements Stackless for applications. + """ + + appleveldefs = { + } + + interpleveldefs = { + 'tasklet' : 'interp_stackless.tasklet', + 'Coroutine' : 'interp_coroutine.AppCoroutine', + } Added: pypy/dist/pypy/module/stackless/interp_coroutine.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/module/stackless/interp_coroutine.py Wed Jan 25 16:07:21 2006 @@ -0,0 +1,89 @@ +from pypy.interpreter.baseobjspace import Wrappable +from pypy.interpreter.argument import Arguments +from pypy.interpreter.typedef import GetSetProperty, TypeDef +from pypy.interpreter.typedef import interp_attrproperty, interp_attrproperty_w +from pypy.interpreter.gateway import interp2app, ObjSpace, W_Root +from pypy.interpreter.error import OperationError +from pypy.rpython.rarithmetic import intmask + +from pypy.rpython.rstack import yield_current_frame_to_caller + +import sys + +class CoState(object): + def __init__(self): + self.last = self.current = self.main = Coroutine() + +costate = None + +class CoroutineDamage(SystemError): + pass + +class Coroutine(Wrappable): + + def __init__(self): + self.frame = None + if costate is None: + self.parent = self + else: + self.parent = costate.main + + def bind(self, thunk): + if self.frame is not None: + raise CoroutineDamage + self.frame = self._bind(thunk) + + def _bind(self, thunk): + self.parent = costate.current + costate.last.frame = yield_current_frame_to_caller() + thunk.call() + if self.parent.frame is None: + self.parent = costate.main + return self._update_state(self.parent) + + def switch(self): + if self.frame is None: + raise CoroutineDamage + costate.last.frame = self._update_state(self).switch() + # note that last gets updated before assignment! + + def _update_state(new): + costate.last, costate.current = costate.current, new + frame, new.frame = new.frame, None + return frame + _update_state = staticmethod(_update_state) + +costate = CoState() + + +class _AppThunk(object): + + def __init__(self, space, w_callable, args): + self.space = space + self.w_func = w_callable + self.args = args + + def call(self): + self.space.call_args(self.w_func, self.args) + +class AppCoroutine(Coroutine): + + def descr_method__new__(space, w_subtype): + co = space.allocate_instance(AppCoroutine, w_subtype) + AppCoroutine.__init__(co) + co.space = space + return space.wrap(co) + + def w_bind(self, w_func, __args__): + thunk = _AppThunk(self.space, w_func, __args__) + self.bind(thunk) + + def w_switch(self): + self.switch() + +AppCoroutine.typedef = TypeDef("Coroutine", + __new__ = interp2app(AppCoroutine.descr_method__new__.im_func), + bind = interp2app(AppCoroutine.w_bind, + unwrap_spec=['self', W_Root, Arguments]), + switch = interp2app(AppCoroutine.w_switch), +) Added: pypy/dist/pypy/module/stackless/interp_stackless.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/module/stackless/interp_stackless.py Wed Jan 25 16:07:21 2006 @@ -0,0 +1,31 @@ +from pypy.interpreter.baseobjspace import Wrappable +from pypy.interpreter.typedef import GetSetProperty, TypeDef +from pypy.interpreter.typedef import interp_attrproperty, interp_attrproperty_w +from pypy.interpreter.gateway import interp2app, ObjSpace, W_Root +from pypy.interpreter.error import OperationError +from pypy.rpython.rarithmetic import intmask +import os + + +class tasklet(Wrappable): + + def __init__(self, space): + self.space = space + self.flags = 0 + self.state = None + + def descr_method__new__(space, w_subtype): + t = space.allocate_instance(tasklet, w_subtype) + tasklet.__init__(t, space) + return space.wrap(t) + + def w_demo(self): + output("42") + +tasklet.typedef = TypeDef("tasklet", + __new__ = interp2app(tasklet.descr_method__new__.im_func), + demo = interp2app(tasklet.w_demo), +) + +def output(stuff): + os.write(2, stuff + '\n') Added: pypy/dist/pypy/module/stackless/test/__init__.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/module/stackless/test/__init__.py Wed Jan 25 16:07:21 2006 @@ -0,0 +1,18 @@ +# Package initialisation +from pypy.interpreter.mixedmodule import MixedModule + +class Module(MixedModule): + """ + This module implements marshal at interpreter level. + """ + + appleveldefs = { + } + + interpleveldefs = { + 'dump' : 'interp_marshal.dump', + 'dumps' : 'interp_marshal.dumps', + 'load' : 'interp_marshal.load', + 'loads' : 'interp_marshal.loads', + 'version' : 'space.wrap(interp_marshal.Py_MARSHAL_VERSION)', + } Added: pypy/dist/pypy/module/stackless/test/test_coroutine.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/module/stackless/test/test_coroutine.py Wed Jan 25 16:07:21 2006 @@ -0,0 +1,13 @@ +from pypy.conftest import gettestobjspace + +class AppTest_Coroutine: + + def setup_class(cls): + space = gettestobjspace(usemodules=('stackless',)) + cls.space = space + + def test_one(self): + import stackless + print stackless.__file__ + co = stackless.Coroutine() + print co Added: pypy/dist/pypy/module/stackless/test/test_interp_coroutine.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/module/stackless/test/test_interp_coroutine.py Wed Jan 25 16:07:21 2006 @@ -0,0 +1,169 @@ +""" +testing coroutines at interprepter level +""" + +import os +from pypy.module.stackless.interp_coroutine import costate, Coroutine + +def wrap_stackless_function(fn): + from pypy.translator.translator import TranslationContext + from pypy.translator.c.genc import CStandaloneBuilder + from pypy.annotation.model import SomeList, SomeString + from pypy.annotation.listdef import ListDef + from pypy.translator.backendopt.all import backend_optimizations + + def entry_point(argv): + os.write(1, str(fn())) + return 0 + + s_list_of_strings = SomeList(ListDef(None, SomeString())) + s_list_of_strings.listdef.resize() + t = TranslationContext() + t.buildannotator().build_types(entry_point, [s_list_of_strings]) + #t.view() + t.buildrtyper().specialize() + backend_optimizations(t) + cbuilder = CStandaloneBuilder(t, entry_point) + cbuilder.stackless = True + cbuilder.generate_source() + cbuilder.compile() + return cbuilder.cmdexec('') + + +def output(stuff): + os.write(2, stuff + '\n') + +def test_coroutine(): + + def g(lst, coros): + coro_f, coro_g, coro_h = coros + lst.append(2) + output('g appended 2') + coro_h.switch() + lst.append(5) + output('g appended 5') + + def h(lst, coros): + coro_f, coro_g, coro_h = coros + lst.append(3) + output('h appended 3') + coro_f.switch() + lst.append(7) + output('h appended 7') + + class T: + def __init__(self, func, arg1, arg2): + self.func = func + self.arg1 = arg1 + self.arg2 = arg2 + def call(self): + self.func(self.arg1, self.arg2) + + def f(): + lst = [1] + coro_f = costate.main + coro_g = Coroutine() + coro_h = Coroutine() + coros = [coro_f, coro_g, coro_h] + thunk_g = T(g, lst, coros) + output('binding g after f set 1') + coro_g.bind(thunk_g) + thunk_h = T(h, lst, coros) + output('binding h after f set 1') + coro_h.bind(thunk_h) + output('switching to g') + coro_g.switch() + lst.append(4) + output('f appended 4') + coro_g.switch() + lst.append(6) + output('f appended 6') + coro_h.switch() + lst.append(8) + output('f appended 8') + n = 0 + for i in lst: + n = n*10 + i + return n + + data = wrap_stackless_function(f) + assert int(data.strip()) == 12345678 + +def test_coroutine2(): + + class TBase: + def call(self): + pass + + class T(TBase): + def __init__(self, func, arg1, arg2): + self.func = func + self.arg1 = arg1 + self.arg2 = arg2 + def call(self): + self.res = self.func(self.arg1, self.arg2) + + class T1(TBase): + def __init__(self, func, arg1): + self.func = func + self.arg1 = arg1 + def call(self): + self.res = self.func(self.arg1) + + def g(lst, coros): + coro_f1, coro_g, coro_h = coros + lst.append(2) + output('g appended 2') + coro_h.switch() + lst.append(5) + output('g appended 5') + output('exiting g') + + def h(lst, coros): + coro_f1, coro_g, coro_h = coros + lst.append(3) + output('h appended 3') + coro_f1.switch() + lst.append(7) + output('h appended 7') + output('exiting h') + + def f1(coro_f1): + lst = [1] + coro_g = Coroutine() + coro_h = Coroutine() + coros = [coro_f1, coro_g, coro_h] + thunk_g = T(g, lst, coros) + output('binding g after f1 set 1') + coro_g.bind(thunk_g) + thunk_h = T(h, lst, coros) + output('binding h after f1 set 1') + coro_h.bind(thunk_h) + output('switching to g') + coro_g.switch() + lst.append(4) + output('f1 appended 4') + coro_g.switch() + lst.append(6) + output('f1 appended 6') + coro_h.switch() + lst.append(8) + output('f1 appended 8') + n = 0 + for i in lst: + n = n*10 + i + output('exiting f1') + return n + + def f(): + coro_f = costate.main + coro_f1 = Coroutine() + thunk_f1 = T1(f1, coro_f1) + output('binding f1 after f set 1') + coro_f1.bind(thunk_f1) + coro_f1.switch() + output('return to main :-(') + return thunk_f1.res + + data = wrap_stackless_function(f) + assert int(data.strip()) == 12345678 Added: pypy/dist/pypy/module/stackless/test/test_stackless.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/module/stackless/test/test_stackless.py Wed Jan 25 16:07:21 2006 @@ -0,0 +1,21 @@ +from pypy.conftest import gettestobjspace + +class AppTest_Stackless: + + def setup_class(cls): + space = gettestobjspace(usemodules=('stackless',)) + cls.space = space + + def test_one(self): + import stackless + print stackless.__file__ + t = stackless.tasklet() + t.demo() + class A(stackless.tasklet): + def __init__(self, name): + self.name = name + def __new__(subtype, *args): + return stackless.tasklet.__new__(subtype) + x = A("heinz") + x.demo() + print x.name From tismer at codespeak.net Wed Jan 25 16:11:50 2006 From: tismer at codespeak.net (tismer at codespeak.net) Date: Wed, 25 Jan 2006 16:11:50 +0100 (CET) Subject: [pypy-svn] r22652 - pypy/dist/pypy/module/stackless/test Message-ID: <20060125151150.2A16027B64@code1.codespeak.net> Author: tismer Date: Wed Jan 25 16:11:48 2006 New Revision: 22652 Modified: pypy/dist/pypy/module/stackless/test/__init__.py Log: wrong __init__.py Modified: pypy/dist/pypy/module/stackless/test/__init__.py ============================================================================== --- pypy/dist/pypy/module/stackless/test/__init__.py (original) +++ pypy/dist/pypy/module/stackless/test/__init__.py Wed Jan 25 16:11:48 2006 @@ -1,18 +1 @@ -# Package initialisation -from pypy.interpreter.mixedmodule import MixedModule - -class Module(MixedModule): - """ - This module implements marshal at interpreter level. - """ - - appleveldefs = { - } - - interpleveldefs = { - 'dump' : 'interp_marshal.dump', - 'dumps' : 'interp_marshal.dumps', - 'load' : 'interp_marshal.load', - 'loads' : 'interp_marshal.loads', - 'version' : 'space.wrap(interp_marshal.Py_MARSHAL_VERSION)', - } +# \ No newline at end of file From ac at codespeak.net Wed Jan 25 16:14:36 2006 From: ac at codespeak.net (ac at codespeak.net) Date: Wed, 25 Jan 2006 16:14:36 +0100 (CET) Subject: [pypy-svn] r22653 - in pypy/dist/pypy/jit: . test Message-ID: <20060125151436.925ED27B84@code1.codespeak.net> Author: ac Date: Wed Jan 25 16:14:36 2006 New Revision: 22653 Modified: pypy/dist/pypy/jit/llabstractinterp.py pypy/dist/pypy/jit/llcontainer.py pypy/dist/pypy/jit/llvalue.py pypy/dist/pypy/jit/test/test_jit_tl.py pypy/dist/pypy/jit/test/test_jit_tlr.py pypy/dist/pypy/jit/test/test_llabstractinterp.py pypy/dist/pypy/jit/test/test_vlist.py pypy/dist/pypy/jit/vlist.py Log: (pedronis, arre) More refactoring to use rgenop. Modified: pypy/dist/pypy/jit/llabstractinterp.py ============================================================================== --- pypy/dist/pypy/jit/llabstractinterp.py (original) +++ pypy/dist/pypy/jit/llabstractinterp.py Wed Jan 25 16:14:36 2006 @@ -3,8 +3,7 @@ from pypy.objspace.flow.model import Block, Link, FunctionGraph from pypy.objspace.flow.model import checkgraph, c_last_exception from pypy.rpython.lltypesystem import lltype -from pypy.translator.simplify import eliminate_empty_blocks, join_blocks -from pypy.jit.llvalue import LLAbstractValue, const, newvar, dupvar +from pypy.jit.llvalue import LLAbstractValue, const from pypy.jit.llvalue import ll_no_return_value from pypy.jit.llvalue import FlattenMemo, MatchMemo, FreezeMemo, UnfreezeMemo from pypy.jit.llcontainer import LLAbstractContainer, virtualcontainervalue @@ -176,11 +175,11 @@ a = LLAbstractValue(const(arghints[i])) a.concrete = self.policy.concrete_args else: - a = LLAbstractValue(dupvar(v)) + a = LLAbstractValue(input=v.concretetype) args_a.append(a) graphstate = self.schedule_graph(args_a, origgraph) graphstate.complete() - return graphstate.copygraph + return rgenop.buildgraph(graphstate.startblock) def schedule_graph(self, args_a, origgraph): inputstate = LLBlockState(None, args_a, origgraph.startblock) @@ -260,6 +259,9 @@ self.frozenstate = frozenstate self.link = None + def setreturn(self): + rgenop.closereturnlink(self.link, self.args_v[0]) + def settarget(self, block, blockargs): args = [] for v1, v2 in zip(self.args_v, blockargs): @@ -267,8 +269,7 @@ assert v1 == v2 else: args.append(v1) - self.link.args = args - self.link.settarget(block) + rgenop.closelink(self.link, args, block) class GraphState(object): @@ -277,17 +278,7 @@ def __init__(self, interp, origgraph, inputstate, frozenstate, n): self.interp = interp self.origgraph = origgraph - name = '%s_%d' % (origgraph.name, n) - self.copygraph = FunctionGraph(name, Block([])) # grumble - interp.graphs.append(self.copygraph) - for orig_v, copy_v in [(origgraph.getreturnvar(), - self.copygraph.getreturnvar()), - (origgraph.exceptblock.inputargs[0], - self.copygraph.exceptblock.inputargs[0]), - (origgraph.exceptblock.inputargs[1], - self.copygraph.exceptblock.inputargs[1])]: - if hasattr(orig_v, 'concretetype'): - copy_v.concretetype = orig_v.concretetype + self.name = '%s_%d' % (origgraph.name, n) # The 'args_v' attribute is needed by try_to_complete(), # which looks for it on either a GraphState or a LinkState self.args_v = inputstate.getruntimevars() @@ -296,8 +287,7 @@ self.state = "before" def settarget(self, block, blockargs): - block.isstartblock = True - self.copygraph.startblock = block + self.startblock = block def complete(self): assert self.state != "during" @@ -313,7 +303,6 @@ continue def try_to_complete(self): - graph = self.copygraph interp = self.interp pending = [self] # pending list of a GraphState and many LinkStates seen = {} @@ -405,21 +394,9 @@ # resolve the LinkState to go to the return # or except block if len(ls.args_v) == 1: - target = graph.returnblock + ls.setreturn() elif len(ls.args_v) == 2: - target = graph.exceptblock - else: - raise Exception("uh?") - ls.settarget(target, target.inputargs) - else: - # the LinkState is already going to a return or except - # block; make sure that it is really the one from - # 'graph' -- by patching 'graph' if necessary. - if len(ls.link.target.inputargs) == 1: - #self.a_return = state.args_a[0] - graph.returnblock = ls.link.target - elif len(link.target.inputargs) == 2: - graph.exceptblock = ls.link.target + XXX_handle_exception_here else: raise Exception("uh?") @@ -427,17 +404,6 @@ raise RestartCompleting # the graph should be complete now; sanity-check - try: - checkgraph(graph) - except Exception, e: - print 'INVALID GRAPH:' - import traceback - traceback.print_exc() - print 'graph.show()...' - graph.show() - raise - eliminate_empty_blocks(graph) - join_blocks(graph) self.state = "after" def flowin(self, state, key): @@ -612,6 +578,10 @@ [a.forcevarorconst(self) for a in args_a], op.result.concretetype) return LLAbstractValue(retvar) + + def residual_direct_call(self, name, block, args_a,): + + return LLAbstractValue(retvar) def residualize(self, op, args_a, constant_op=None): if constant_op: @@ -769,13 +739,13 @@ graphstate = self.interp.schedule_graph(args_a, origgraph) #print 'SCHEDULE_GRAPH', args_a, '==>', graphstate.copygraph.name if graphstate.state != "during": - print 'ENTERING', graphstate.copygraph.name, args_a + print 'ENTERING', graphstate.name, args_a graphstate.complete() #if graphstate.a_return is not None: # a = graphstate.a_return.unfreeze(UnfreezeMemo()) # if a.maybe_get_constant() is not None: # a_real_result = a # propagate a constant result - print 'LEAVING', graphstate.copygraph.name#, graphstate.a_return + print 'LEAVING', graphstate.name#, graphstate.a_return ARGS = [] new_args_a = [] @@ -785,9 +755,10 @@ new_args_a.append(a) args_a = new_args_a TYPE = lltype.FuncType(ARGS, op.result.concretetype) - fptr = lltype.functionptr( - TYPE, graphstate.copygraph.name, graph=graphstate.copygraph) - a_func = LLAbstractValue(const(fptr)) + a_func = LLAbstractValue(rgenop.gengraphconst(self.newblock, + graphstate.name, + graphstate.startblock, + TYPE)) return self.residual(op, [a_func] + args_a) def op_getfield(self, op, a_ptr, a_attrname): Modified: pypy/dist/pypy/jit/llcontainer.py ============================================================================== --- pypy/dist/pypy/jit/llcontainer.py (original) +++ pypy/dist/pypy/jit/llcontainer.py Wed Jan 25 16:14:36 2006 @@ -1,6 +1,6 @@ from pypy.objspace.flow.model import Variable, Constant, SpaceOperation from pypy.rpython.lltypesystem import lltype -from pypy.jit.llvalue import LLAbstractValue, newvar, const +from pypy.jit.llvalue import LLAbstractValue, const class LLAbstractContainer(object): Modified: pypy/dist/pypy/jit/llvalue.py ============================================================================== --- pypy/dist/pypy/jit/llvalue.py (original) +++ pypy/dist/pypy/jit/llvalue.py Wed Jan 25 16:14:36 2006 @@ -15,13 +15,16 @@ concrete = False # concrete constants propagate eagerly - def __init__(self, runtimevar=None, content=None): + def __init__(self, runtimevar=None, content=None, input=None): self.runtimevar = runtimevar # None or a Variable or a Constant self.content = content # None or an LLAbstractContainer self.origin = [] # list of frozen values: the sources that did or # could allow 'self' to be computed as a constant + self.input = input def __repr__(self): + if self.input: + return '' if self.runtimevar is None: if self.content is None: return '' @@ -40,6 +43,8 @@ def freeze(self, memo): # turn a run-time value into a frozen value + if self.input: + return LLFrozenRuntimeValue(self) if self.runtimevar is not None: if self.concrete: return LLFrozenConcreteValue(self) @@ -60,6 +65,8 @@ return frozen_dummy_value # dummy def getconcretetype(self): + if self.input: + return self.input if self.runtimevar is not None: return self.runtimevar.concretetype elif self.content is not None: @@ -85,7 +92,9 @@ """Recursively flatten the LLAbstractValue into a list of run-time LLAbstractValues. """ - if self.runtimevar is not None: + if self.input: + memo.result.append(self) + elif self.runtimevar is not None: if not self.concrete: # skip concrete values, they don't need # to be present in the residual graph at all memo.result.append(self) @@ -96,7 +105,8 @@ else: pass # dummy -# ____________________________________________________________ +# _____________________________________________________________ + class LLFrozenValue(object): """An abstract value frozen in a saved state. @@ -152,7 +162,7 @@ def __init__(self, a_value): # get the necessary information from the a_value - self.concretetype = a_value.runtimevar.concretetype + self.concretetype = a_value.getconcretetype() self.origin = a_value.origin def flatten(self, memo): @@ -247,16 +257,6 @@ c.concretetype = T or lltype.typeOf(value) return c -def newvar(T): - v = Variable() - v.concretetype = T - return v - -def dupvar(v): - v1 = Variable(v) - v1.concretetype = v.concretetype - return v1 - ll_no_return_value = LLAbstractValue(const(None, lltype.Void)) ll_dummy_value = LLAbstractValue() frozen_dummy_value = LLFrozenDummyValue() Modified: pypy/dist/pypy/jit/test/test_jit_tl.py ============================================================================== --- pypy/dist/pypy/jit/test/test_jit_tl.py (original) +++ pypy/dist/pypy/jit/test/test_jit_tl.py Wed Jan 25 16:14:36 2006 @@ -36,7 +36,7 @@ #interp.graphs[0].show() # return a summary of the instructions left in graph2 - return summary(interp) + return summary(graph2) def run_jit(code): Modified: pypy/dist/pypy/jit/test/test_jit_tlr.py ============================================================================== --- pypy/dist/pypy/jit/test/test_jit_tlr.py (original) +++ pypy/dist/pypy/jit/test/test_jit_tlr.py Wed Jan 25 16:14:36 2006 @@ -97,6 +97,6 @@ res = llinterp.eval_graph(graph2, [17]) assert res == 289 - insns = summary(interp) + insns = summary(graph2) assert insns == {'int_add': 2, 'int_is_true': 1} Modified: pypy/dist/pypy/jit/test/test_llabstractinterp.py ============================================================================== --- pypy/dist/pypy/jit/test/test_llabstractinterp.py (original) +++ pypy/dist/pypy/jit/test/test_llabstractinterp.py Wed Jan 25 16:14:36 2006 @@ -6,7 +6,7 @@ from pypy.rpython import rstr from pypy.annotation import model as annmodel from pypy.jit.llabstractinterp import LLAbstractInterp, Policy - +from pypy.objspace.flow import model as flowmodel def annotation(a, x): T = lltype.typeOf(x) @@ -54,15 +54,31 @@ result1 = llinterp.eval_graph(graph1, argvalues) result2 = llinterp.eval_graph(graph2, argvalues2) assert result1 == result2 - return graph2, summary(interp) + return graph2, summary(graph2) -def summary(interp): +def summary(graph): # return a summary of the instructions left in all the residual graphs insns = {} - for copygraph in interp.itercopygraphs(): - for block in copygraph.iterblocks(): + graphs = [graph] + found = set((graph,)) + while graphs: + graph = graphs.pop() + for block in graph.iterblocks(): for op in block.operations: insns[op.opname] = insns.get(op.opname, 0) + 1 + for arg in op.args: + if isinstance(arg, flowmodel.Constant): + if (isinstance(arg.concretetype, lltype.Ptr) and + isinstance(arg.concretetype.TO, lltype.FuncType)): + try: + graph = arg.value._obj.graph + graph.rgenop + except AttributeError: + pass + else: + if graph not in found: + graphs.append(graph) + found.add(graph) return insns P_INLINE = Policy(inlining=True) Modified: pypy/dist/pypy/jit/test/test_vlist.py ============================================================================== --- pypy/dist/pypy/jit/test/test_vlist.py (original) +++ pypy/dist/pypy/jit/test/test_vlist.py Wed Jan 25 16:14:36 2006 @@ -33,8 +33,7 @@ assert result1 == result2 - return graph2, summary(interp) - + return graph2, summary(graph2) def test_fixed_newlistfill_force(): def fn(n): Modified: pypy/dist/pypy/jit/vlist.py ============================================================================== --- pypy/dist/pypy/jit/vlist.py (original) +++ pypy/dist/pypy/jit/vlist.py Wed Jan 25 16:14:36 2006 @@ -1,6 +1,6 @@ from pypy.rpython.lltypesystem import lltype from pypy.rpython.rtyper import LowLevelOpList -from pypy.jit.llvalue import LLAbstractValue, newvar, const, ll_dummy_value +from pypy.jit.llvalue import LLAbstractValue, const, ll_dummy_value from pypy.jit.llcontainer import LLAbstractContainer From gromit at codespeak.net Wed Jan 25 16:40:40 2006 From: gromit at codespeak.net (gromit at codespeak.net) Date: Wed, 25 Jan 2006 16:40:40 +0100 (CET) Subject: [pypy-svn] r22654 - in pypy/dist/pypy/rpython/rctypes: . test Message-ID: <20060125154040.9619227B70@code1.codespeak.net> Author: gromit Date: Wed Jan 25 16:40:37 2006 New Revision: 22654 Modified: pypy/dist/pypy/rpython/rctypes/implementation.py pypy/dist/pypy/rpython/rctypes/interface.py pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py Log: ADD: (stephan,gromit) annotation of rcytpes' pointers Modified: pypy/dist/pypy/rpython/rctypes/implementation.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/implementation.py (original) +++ pypy/dist/pypy/rpython/rctypes/implementation.py Wed Jan 25 16:40:37 2006 @@ -97,6 +97,19 @@ RByref = RByrefObj() + +def RPOINTER(cls): + answer = POINTER(cls) + def compute_result_annotation(cls, s_arg): + """ + Answer the result annotation of calling 'cls'. + """ + assert answer is cls + return SomeCTypesObject(cls) + answer.compute_result_annotation = classmethod(compute_result_annotation) + return answer + + class RCDLL(CDLL): """ This is the restricted version of ctypes' CDLL class. Modified: pypy/dist/pypy/rpython/rctypes/interface.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/interface.py (original) +++ pypy/dist/pypy/rpython/rctypes/interface.py Wed Jan 25 16:40:37 2006 @@ -1,6 +1,6 @@ from ctypes import _DLLS -from implementation import RCDLL as CDLL, c_int, c_char_p, c_char, POINTER, \ - RStructure as Structure, RByref as byref +from implementation import RCDLL as CDLL, c_int, c_char_p, c_char, \ + RStructure as Structure, RByref as byref, RPOINTER as POINTER try: from implementation import RWinDLL as WinDLL except ImportError: Modified: pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py Wed Jan 25 16:40:37 2006 @@ -85,6 +85,16 @@ mod.py_create_point = py_create_point + oppoint_type = POINTER(tagpoint) + def py_testfunc_POINTER(inpoint): + point = tagpoint() + oppoint = oppoint_type(point) + res = testfunc_byval(inpoint,oppoint) + return res, oppoint + + mod.py_testfunc_POINTER = py_testfunc_POINTER + mod.POINTER = POINTER + class Test_rctypes: @@ -165,3 +175,11 @@ assert s.items[0].knowntype == int assert s.items[1].knowntype == tagpoint + def test_annotate_POINTER(self): + a = RPythonAnnotator() + s = a.build_types(py_testfunc_POINTER,[tagpoint]) + assert s.knowntype == tuple + assert len(s.items) == 2 + assert s.items[0].knowntype == int + assert s.items[1].knowntype == POINTER(tagpoint) + From ac at codespeak.net Wed Jan 25 16:45:54 2006 From: ac at codespeak.net (ac at codespeak.net) Date: Wed, 25 Jan 2006 16:45:54 +0100 (CET) Subject: [pypy-svn] r22655 - pypy/dist/pypy/jit Message-ID: <20060125154554.1822827B84@code1.codespeak.net> Author: ac Date: Wed Jan 25 16:45:53 2006 New Revision: 22655 Modified: pypy/dist/pypy/jit/llabstractinterp.py Log: (samuele, arre) Morerefactoring. Modified: pypy/dist/pypy/jit/llabstractinterp.py ============================================================================== --- pypy/dist/pypy/jit/llabstractinterp.py (original) +++ pypy/dist/pypy/jit/llabstractinterp.py Wed Jan 25 16:45:53 2006 @@ -1,6 +1,5 @@ import operator from pypy.objspace.flow.model import Variable, Constant, SpaceOperation -from pypy.objspace.flow.model import Block, Link, FunctionGraph from pypy.objspace.flow.model import checkgraph, c_last_exception from pypy.rpython.lltypesystem import lltype from pypy.jit.llvalue import LLAbstractValue, const @@ -514,15 +513,17 @@ def buildblock(self, newexitswitch, newlinkstates): b = self.newblock - b.exitswitch = newexitswitch - exits = [] + if newexitswitch is None: + link = rgenop.closeblock1(b) + newlinkstates[0].link = link + return b + + assert len(newlinkstates) == 2 + + false_link, true_link = rgenop.closeblock2(b, newexitswitch) + cases = {False: false_link, True: true_link} for ls in newlinkstates: - link = Link([], None) - link.exitcase = ls.exitcase - link.llexitcase = ls.llexitcase - ls.link = link - exits.append(link) - b.closeblock(*exits) + ls.link = cases[ls.exitcase] return b def genop(self, opname, args, RESULT_TYPE): From ac at codespeak.net Wed Jan 25 16:55:16 2006 From: ac at codespeak.net (ac at codespeak.net) Date: Wed, 25 Jan 2006 16:55:16 +0100 (CET) Subject: [pypy-svn] r22656 - pypy/dist/pypy/rpython Message-ID: <20060125155516.C8B5A27B64@code1.codespeak.net> Author: ac Date: Wed Jan 25 16:55:16 2006 New Revision: 22656 Modified: pypy/dist/pypy/rpython/rgenop.py pypy/dist/pypy/rpython/rlist.py Log: Oops, forgot these files in the last chekin. Modified: pypy/dist/pypy/rpython/rgenop.py ============================================================================== --- pypy/dist/pypy/rpython/rgenop.py (original) +++ pypy/dist/pypy/rpython/rgenop.py Wed Jan 25 16:55:16 2006 @@ -6,6 +6,7 @@ from pypy.rpython.lltypesystem import lltype from pypy.objspace.flow import model as flowmodel +from pypy.translator.simplify import eliminate_empty_blocks, join_blocks def newblock(): @@ -27,6 +28,11 @@ block.operations.append(op) return v +def gengraphconst(block, name, target, FUNCTYPE): + fptr = lltype.functionptr(FUNCTYPE, name, + graph=buildgraph(target)) + return genconst(block, fptr) + def genconst(block, llvalue): v = flowmodel.Constant(llvalue) v.concretetype = lltype.typeOf(llvalue) @@ -91,10 +97,17 @@ from pypy.rpython.typesystem import LowLevelTypeSystem self.type_system = LowLevelTypeSystem.instance -def runblock(block, args): - from pypy.rpython.llinterp import LLInterpreter +def buildgraph(block): graph = flowmodel.FunctionGraph('?', block) _patchgraph(graph) flowmodel.checkgraph(graph) + eliminate_empty_blocks(graph) + join_blocks(graph) + graph.rgenop = True + return graph + +def runblock(block, args): + from pypy.rpython.llinterp import LLInterpreter + graph = buildgraph(block) llinterp = LLInterpreter(PseudoRTyper()) return llinterp.eval_graph(graph, args) Modified: pypy/dist/pypy/rpython/rlist.py ============================================================================== --- pypy/dist/pypy/rpython/rlist.py (original) +++ pypy/dist/pypy/rpython/rlist.py Wed Jan 25 16:55:16 2006 @@ -168,12 +168,12 @@ argtypes = [Signed] fnptr = list_repr.rtyper.annotate_helper_fn(LIST.ll_newlist, argtypes) - self.c_newlist = inputconst(typeOf(fnptr), fnptr) + self.newlist_ptr = fnptr bk = list_repr.rtyper.annotator.bookkeeper argtypes = [bk.immutablevalue(dum_nocheck), LISTPTR, Signed, ITEM] fnptr = list_repr.rtyper.annotate_helper_fn(ll_setitem_nonneg, argtypes) - self.c_setitem_nonneg = inputconst(typeOf(fnptr), fnptr) + self.setitem_nonneg_ptr = fnptr self.c_dum_nocheck = inputconst(Void, dum_nocheck) self.c_LIST = inputconst(Void, self.LIST) @@ -181,12 +181,12 @@ """Make the operations that would build a list containing the provided items.""" c_list = builder.addconst(self.c_LIST) - c_newlist = builder.addconst(self.c_newlist) + c_newlist = builder.genconst(self.newlist_ptr) c_len = builder.genconst(len(items_v)) v_result = builder.genop('direct_call', [c_newlist, c_list, c_len], self.LISTPTR) - c_setitem_nonneg = builder.addconst(self.c_setitem_nonneg) + c_setitem_nonneg = builder.genconst(self.setitem_nonneg_ptr) c_dum_nocheck = builder.addconst(self.c_dum_nocheck) for i, v in enumerate(items_v): c_i = builder.genconst(i) From ac at codespeak.net Wed Jan 25 17:10:15 2006 From: ac at codespeak.net (ac at codespeak.net) Date: Wed, 25 Jan 2006 17:10:15 +0100 (CET) Subject: [pypy-svn] r22657 - in pypy/dist/pypy: jit rpython Message-ID: <20060125161015.13D6727B64@code1.codespeak.net> Author: ac Date: Wed Jan 25 17:10:14 2006 New Revision: 22657 Modified: pypy/dist/pypy/jit/llabstractinterp.py pypy/dist/pypy/jit/llcontainer.py pypy/dist/pypy/rpython/rgenop.py pypy/dist/pypy/rpython/rlist.py Log: (samuele, arre) remove the addconst hack. Modified: pypy/dist/pypy/jit/llabstractinterp.py ============================================================================== --- pypy/dist/pypy/jit/llabstractinterp.py (original) +++ pypy/dist/pypy/jit/llabstractinterp.py Wed Jan 25 17:10:14 2006 @@ -532,9 +532,6 @@ def genconst(self, llvalue): return rgenop.genconst(self.newblock, llvalue) - def addconst(self, const): - return rgenop.addconst(self.newblock, const) - def binding(self, v): if isinstance(v, Constant): return LLAbstractValue(v) Modified: pypy/dist/pypy/jit/llcontainer.py ============================================================================== --- pypy/dist/pypy/jit/llcontainer.py (original) +++ pypy/dist/pypy/jit/llcontainer.py Wed Jan 25 17:10:14 2006 @@ -52,17 +52,16 @@ def build_runtime_container(self, builder): RESULT_TYPE = lltype.Ptr(self.T) if self.a_parent is not None: + parentindex = const(self.parentindex, lltype.Void) v_parent = self.a_parent.forcevarorconst(builder) - parentindex = builder.addconst(const(self.parentindex, - lltype.Void)) v_result = builder.genop('getsubstruct', - [v_parent,parentindex], RESULT_TYPE) + [v_parent, parentindex], RESULT_TYPE) else: - t = builder.addconst(const(self.T, lltype.Void)) + t = const(self.T, lltype.Void) if self.T._is_varsize(): - v_result = builder.genop('malloc_varsize', - [t, builder.genconst(self.length)], + [t, + builder.genconst(self.length)], RESULT_TYPE) else: v_result = builder.genop('malloc', [t], RESULT_TYPE) @@ -77,9 +76,8 @@ T = self.fieldtype(name) if isinstance(T, lltype.ContainerType): # initialize the substructure/subarray - c_name = builder.addconst(const(name, lltype.Void)) v_subptr = builder.genop('getsubstruct', - [v_target, c_name], + [v_target, const(name, lltype.Void)], lltype.Ptr(T)) assert isinstance(a_value.content, LLVirtualContainer) a_value.content.buildcontent(builder, v_subptr) @@ -161,8 +159,7 @@ return getattr(self.T, name) def setop(self, builder, v_target, name, v_value): - c_name = builder.addconst(const(name, lltype.Void)) - builder.genop('setfield', [v_target, c_name, v_value], + builder.genop('setfield', [v_target, const(name, lltype.Void), v_value], lltype.Void) Modified: pypy/dist/pypy/rpython/rgenop.py ============================================================================== --- pypy/dist/pypy/rpython/rgenop.py (original) +++ pypy/dist/pypy/rpython/rgenop.py Wed Jan 25 17:10:14 2006 @@ -38,10 +38,6 @@ v.concretetype = lltype.typeOf(llvalue) return v -def addconst(block, const): - assert isinstance(const, flowmodel.Constant) - return const - def closeblock1(block): link = flowmodel.Link([], None) block.closeblock(link) Modified: pypy/dist/pypy/rpython/rlist.py ============================================================================== --- pypy/dist/pypy/rpython/rlist.py (original) +++ pypy/dist/pypy/rpython/rlist.py Wed Jan 25 17:10:14 2006 @@ -180,18 +180,16 @@ def build(self, builder, items_v): """Make the operations that would build a list containing the provided items.""" - c_list = builder.addconst(self.c_LIST) c_newlist = builder.genconst(self.newlist_ptr) c_len = builder.genconst(len(items_v)) v_result = builder.genop('direct_call', - [c_newlist, c_list, c_len], + [c_newlist, self.c_LIST, c_len], self.LISTPTR) c_setitem_nonneg = builder.genconst(self.setitem_nonneg_ptr) - c_dum_nocheck = builder.addconst(self.c_dum_nocheck) for i, v in enumerate(items_v): c_i = builder.genconst(i) builder.genop('direct_call', [c_setitem_nonneg, - c_dum_nocheck, + self.c_dum_nocheck, v_result, c_i, v], Void) return v_result From auc at codespeak.net Wed Jan 25 17:24:10 2006 From: auc at codespeak.net (auc at codespeak.net) Date: Wed, 25 Jan 2006 17:24:10 +0100 (CET) Subject: [pypy-svn] r22658 - pypy/dist/pypy/lib/logic Message-ID: <20060125162410.D8FF227B64@code1.codespeak.net> Author: auc Date: Wed Jan 25 17:24:07 2006 New Revision: 22658 Modified: pypy/dist/pypy/lib/logic/computationspace.py pypy/dist/pypy/lib/logic/distributor.py pypy/dist/pypy/lib/logic/test_computationspace.py pypy/dist/pypy/lib/logic/unification.py pypy/dist/pypy/lib/logic/variable.py Log: (ale, auc) * added scope discipline to domains in spaces * preliminary Choose support (with distribute & friends) * many fixes & adjustments * removed the GPL snippet * maybe discovered a bug in CPython Modified: pypy/dist/pypy/lib/logic/computationspace.py ============================================================================== --- pypy/dist/pypy/lib/logic/computationspace.py (original) +++ pypy/dist/pypy/lib/logic/computationspace.py Wed Jan 25 17:24:07 2006 @@ -198,12 +198,17 @@ class ComputationSpace(object): def __init__(self, program, parent=None): + if parent is None: + self.store = Store() + self.root = self.store.var('root') + self.store.bind(self.root, program(self)) + else: + self.store = parent.store + self.root = parent.root self.program = program self.parent = parent - self.store = Store() self.status = Unprocessed - self.root = self.store.var('root') - self.store.bind(self.root, program(self)) + self.children = set() def _stable(self): #XXX: really ? @@ -234,10 +239,14 @@ else: self.status = Succeeded - def make_child(self): - return ComputationSpace(self.program, parent=self) + def make_children(self): + for dommap in self.distributor.distribute(): + cs = ComputationSpace(lambda cs : True, + parent=self) + self.children.add(cs) + for var, dom in dommap.items(): + var.cs_set_dom(cs, dom) + + - def choose(self, alternative): - """distribute the domains to new spaces - create the spaces""" Modified: pypy/dist/pypy/lib/logic/distributor.py ============================================================================== --- pypy/dist/pypy/lib/logic/distributor.py (original) +++ pypy/dist/pypy/lib/logic/distributor.py Wed Jan 25 17:24:07 2006 @@ -1,71 +1,61 @@ -# (c) 2000-2001 LOGILAB S.A. (Paris, FRANCE). -# http://www.logilab.fr/ -- mailto:contact at logilab.fr -# -# This program is free software; you can redistribute it and/or modify it under -# the terms of the GNU General Public License as published by the Free Software -# Foundation; either version 2 of the License, or (at your option) any later -# version. -# -# This program is distributed in the hope that it will be useful, but WITHOUT -# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS -# FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. -# -# You should have received a copy of the GNU General Public License along with -# this program; if not, write to the Free Software Foundation, Inc., -# 59 Temple Place - Suite 330, Boston, MA 02111-1307 -# USA. - -""" -distributors - part of Logilab's constraint satisfaction solver. -""" - -__revision__ = '$Id: distributors.py,v 1.25 2005/01/14 15:16:21 alf Exp $' - import math, random -def make_new_domains(domains): - """return a shallow copy of dict of domains passed in argument""" - domain = {} - for key, value in domains.items(): - domain[key] = value.copy() - return domain +def make_new_domains(variables): + """updates variables with copied domains indexed by computation space""" + new_doms = {} + for var in variables: + new_doms[var] = var.dom.copy() + return new_doms class AbstractDistributor(object): """_distribute is left unimplemented.""" - def __init__(self, nb_subspaces=2): + def __init__(self, c_space, nb_subspaces=2): self.nb_subspaces = nb_subspaces + self.c_space = c_space self.verbose = 0 - - def findSmallestDomain(self, domains): + + def findSmallestDomain(self): """returns the variable having the smallest domain. (or one of such varibles if there is a tie) """ - domlist = [(dom.size(), variable ) for variable, dom in domains.items() - if dom.size() > 1] - domlist.sort() - return domlist[0][1] + vars_ = [var for var in self.c_space.store.get_variables_with_a_domain() + if var.dom.size() > 1] + + best = vars_[0] + for var in vars_: + if var.dom.size() < best.dom.size(): + best = var + + return best - def findLargestDomain(self, domains): + def findLargestDomain(self): """returns the variable having the largest domain. (or one of such variables if there is a tie) """ - domlist = [(dom.size(), variable) for variable, dom in domains.items() - if dom.size() > 1] - domlist.sort() - return domlist[-1][1] + vars_ = [var for var in self.c_space.store.get_variables_with_a_domain() + if var.dom.size() > 1] + + best = vars_[0] + for var in vars_: + if var.dom.size() > best.dom.size(): + best = var + + return best + def nb_subdomains(self, domains): """return number of sub domains to explore""" return self.nb_subspaces - def distribute(self, domains, verbose=0): + def distribute(self, verbose=0): """do the minimal job and let concrete class distribute variables """ self.verbose = verbose + variables = self.c_space.store.get_variables_with_a_domain() replicas = [] - for i in range(self.nb_subdomains(domains)): - replicas.append(make_new_domains(domains)) + for i in range(self.nb_subdomains(variables)): + replicas.append(make_new_domains(variables)) modified_domains = self._distribute(*replicas) for domain in modified_domains: domain.reset_flags() @@ -85,9 +75,6 @@ The first new domain has a size of one, and the second has all the other values""" - def __init__(self): - AbstractDistributor.__init__(self) - def _distribute(self, dom1, dom2): """See AbstractDistributor""" variable = self.findSmallestDomain(dom1) @@ -103,9 +90,6 @@ class RandomizingDistributor(AbstractDistributor): """distributes domains as the NaiveDistrutor, except that the unique value of the first domain is picked at random.""" - - def __init__(self): - AbstractDistributor.__init__(self) def _distribute(self, dom1, dom2): """See AbstractDistributor""" @@ -127,16 +111,17 @@ If nb_subspaces is 0, then the smallest domain is split in domains of size 1""" - def __init__(self, nb_subspaces=3): - AbstractDistributor.__init__(self, nb_subspaces) + def __init__(self, c_space, nb_subspaces=3): + AbstractDistributor.__init__(self, c_space, nb_subspaces) self.__to_split = None + def nb_subdomains(self, domains): """See AbstractDistributor""" - self.__to_split = self.findSmallestDomain(domains) + self.__to_split = self.findSmallestDomain() if self.nb_subspaces: - return min(self.nb_subspaces, domains[self.__to_split].size()) + return min(self.nb_subspaces, self.__to_split.dom.size()) #domains[self.__to_split].size()) else: - return domains[self.__to_split].size() + return self.__to_split.dom.size() # domains[self.__to_split].size() def _distribute(self, *args): """See AbstractDistributor""" @@ -159,14 +144,14 @@ class DichotomyDistributor(SplitDistributor): """distributes domains by splitting the smallest domain in two equal parts or as equal as possible""" - def __init__(self): - SplitDistributor.__init__(self, 2) + def __init__(self, c_space): + SplitDistributor.__init__(self, c_space, 2) class EnumeratorDistributor(SplitDistributor): """distributes domains by splitting the smallest domain in domains of size 1.""" - def __init__(self): - SplitDistributor.__init__(self, 0) + def __init__(self, c_space): + SplitDistributor.__init__(self, c_space, 0) DefaultDistributor = DichotomyDistributor Modified: pypy/dist/pypy/lib/logic/test_computationspace.py ============================================================================== --- pypy/dist/pypy/lib/logic/test_computationspace.py (original) +++ pypy/dist/pypy/lib/logic/test_computationspace.py Wed Jan 25 17:24:07 2006 @@ -5,6 +5,16 @@ import distributor as di from py.test import raises +#-- utility ------------------ + +class hdict(dict): + """a hashable dict""" + + def __hash__(self): + return id(self) + +#-- helpers ----------------- + def satisfiable_problem(computation_space): cs = computation_space s = cs.store @@ -17,7 +27,7 @@ s.add_constraint(c.Expression([x, y, z], 'x == y + z')) s.add_constraint(c.Expression([z, w], 'z < w')) # set up a distribution strategy - cs.set_distributor(di.DichotomyDistributor()) + cs.set_distributor(di.DichotomyDistributor(cs)) return (x, w, y) def unsatisfiable_problem(computation_space): @@ -32,9 +42,10 @@ s.add_constraint(c.Expression([x, y, z], 'x == y + z')) s.add_constraint(c.Expression([z, w], 'z < w')) # set up a distribution strategy - cs.set_distributor(di.DichotomyDistributor()) + cs.set_distributor(di.DichotomyDistributor(cs)) return (x, w, y) +#-- meat ------------------------ class TestComputationSpace: @@ -63,19 +74,50 @@ def test_distribute(self): spc = cs.ComputationSpace(satisfiable_problem) spc.process() - domains = dict([(var, var.dom) for var in spc.store.vars - if var.dom]) - new_domains = spc.distributor.distribute(domains) + new_domains = [d.items() for d in + spc.distributor.distribute()] + x, y, z, w = (spc.store.get_var_by_name('x'), + spc.store.get_var_by_name('y'), + spc.store.get_var_by_name('z'), + spc.store.get_var_by_name('w')) + expected_domains = [{x: c.FiniteDomain([6]), + y: c.FiniteDomain([2]), + z: c.FiniteDomain([4]), + w: c.FiniteDomain([5])}.items(), + {x: c.FiniteDomain([6]), + y: c.FiniteDomain([2]), + z: c.FiniteDomain([4]), + w: c.FiniteDomain([6, 7])}.items()] + for (d1, d2) in zip(new_domains, expected_domains): + for (e1, e2) in zip(d1, d2): + print e1, '=?', e2 + assert e1 == e2 + # the following assertion fails for mysterious reasons + # have we discovered a bug in CPython ? + # assert set(new_domains) == set(expected_domains) + + def test_make_children(self): + spc = cs.ComputationSpace(satisfiable_problem) x, y, z, w = (spc.store.get_var_by_name('x'), spc.store.get_var_by_name('y'), spc.store.get_var_by_name('z'), spc.store.get_var_by_name('w')) - assert new_domains == [{x: c.FiniteDomain([6]), - y: c.FiniteDomain([2]), - z: c.FiniteDomain([4]), - w: c.FiniteDomain([5])}, - {x: c.FiniteDomain([6]), - y: c.FiniteDomain([2]), - z: c.FiniteDomain([4]), - w: c.FiniteDomain([6, 7])}] - + spc.process() + spc.make_children() + assert len(spc.children) == 2 + new_domains = [] + all_vars = spc.store.get_variables_with_a_domain() + for child in spc.children: + new_domains.append([(var, var.cs_get_dom(child)) + for var in all_vars]) + + expected_domains = [{x: c.FiniteDomain([6]), + y: c.FiniteDomain([2]), + z: c.FiniteDomain([4]), + w: c.FiniteDomain([5])}.items(), + {x: c.FiniteDomain([6]), + y: c.FiniteDomain([2]), + z: c.FiniteDomain([4]), + w: c.FiniteDomain([6, 7])}.items()] + + Modified: pypy/dist/pypy/lib/logic/unification.py ============================================================================== --- pypy/dist/pypy/lib/logic/unification.py (original) +++ pypy/dist/pypy/lib/logic/unification.py Wed Jan 25 17:24:07 2006 @@ -216,6 +216,12 @@ def add_constraint(self, constraint): self.constraints.add(constraint) + def get_variables_with_a_domain(self): + varset = set() + for var in self.vars: + if var.dom: varset.add(var) + return varset + def satisfiable(self, constraint): """ * satisfiable (k) checks that the constraint k can be satisfied wrt its variable domains Modified: pypy/dist/pypy/lib/logic/variable.py ============================================================================== --- pypy/dist/pypy/lib/logic/variable.py (original) +++ pypy/dist/pypy/lib/logic/variable.py Wed Jan 25 17:24:07 2006 @@ -37,8 +37,10 @@ self.store = store # top-level 'commited' binding self._val = NoValue - # domain + # domain in a flat world self.dom = None + # domains in multiple spaces + self.doms = {} # constraints self.constraints = set() # when updated in a 'transaction', keep track @@ -100,6 +102,11 @@ is_bound = _is_bound + def cs_set_dom(self, cs, dom): + self.doms[cs] = dom + + def cs_get_dom(self, cs): + return self.doms[cs] #---- Concurrent public ops -------------------------- # should be used by threads that want to block on From auc at codespeak.net Wed Jan 25 17:30:53 2006 From: auc at codespeak.net (auc at codespeak.net) Date: Wed, 25 Jan 2006 17:30:53 +0100 (CET) Subject: [pypy-svn] r22659 - pypy/dist/pypy/lib/logic Message-ID: <20060125163053.83E5E27B64@code1.codespeak.net> Author: auc Date: Wed Jan 25 17:30:52 2006 New Revision: 22659 Modified: pypy/dist/pypy/lib/logic/test_computationspace.py Log: (ale, auc) forgot small assert Modified: pypy/dist/pypy/lib/logic/test_computationspace.py ============================================================================== --- pypy/dist/pypy/lib/logic/test_computationspace.py (original) +++ pypy/dist/pypy/lib/logic/test_computationspace.py Wed Jan 25 17:30:52 2006 @@ -120,4 +120,8 @@ z: c.FiniteDomain([4]), w: c.FiniteDomain([6, 7])}.items()] - + for (d1, d2) in zip(new_domains, expected_domains): + for (e1, e2) in zip(d1, d2): + print e1, '=?', e2 + assert e1 == e2 + From stephan at codespeak.net Wed Jan 25 18:19:04 2006 From: stephan at codespeak.net (stephan at codespeak.net) Date: Wed, 25 Jan 2006 18:19:04 +0100 (CET) Subject: [pypy-svn] r22660 - pypy/dist/pypy/rpython/rctypes Message-ID: <20060125171904.7D78627B64@code1.codespeak.net> Author: stephan Date: Wed Jan 25 18:19:02 2006 New Revision: 22660 Modified: pypy/dist/pypy/rpython/rctypes/implementation.py Log: some cleanupwork + integration of missing ctypes basetypes (no tests for that yet) Modified: pypy/dist/pypy/rpython/rctypes/implementation.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/implementation.py (original) +++ pypy/dist/pypy/rpython/rctypes/implementation.py Wed Jan 25 18:19:02 2006 @@ -7,20 +7,50 @@ from ctypes import _FUNCFLAG_CDECL if sys.platform == "win32": from ctypes import _FUNCFLAG_STDCALL -from pypy.annotation.model import SomeInteger, SomeCTypesObject -from pypy.rpython.lltypesystem.lltype import Signed +from pypy.annotation.model import SomeInteger, SomeCTypesObject, \ + SomeString, SomeFloat +from pypy.rpython.lltypesystem.lltype import Signed, SignedLongLong, \ + Unsigned, UnsignedLongLong, Char, Float + +# ctypes_annotation_list contains various attributes that +# are used by the pypy annotation. + +ctypes_annotation_list = [ + (c_char, Char, None), + (c_byte, Signed, None), + (c_ubyte, Unsigned, None), + (c_short, Signed, None), + (c_ushort, Unsigned, None), + (c_int, Signed, None), + (c_uint, Unsigned, None), + (c_long, Signed, None), + (c_ulong, Unsigned, None), + (c_longlong, SignedLongLong, None), + (c_ulonglong, UnsignedLongLong, None), + (c_float, Float, None), + (c_double, Float, None), + (c_char_p, None, + staticmethod(lambda ll_type, arg_name:"RPyString_AsString(%s)" % arg_name)), + (POINTER(c_char), None, + staticmethod(lambda ll_type, arg_name:"RPyString_AsString(%s)" % arg_name)), +] -c_int.annotator_type = SomeInteger() -c_int.ll_type = Signed -c_char_p.annotator_type = None -c_char_p.ll_type = None -c_char_p.wrap_arg = staticmethod( - lambda ll_type, arg_name: "RPyString_AsString(%s)" % arg_name) -POINTER(c_char).annotator_type = None -POINTER(c_char).ll_type = None -POINTER(c_char).wrap_arg = staticmethod( - lambda ll_type, arg_name: "RPyString_AsString(%s)" % arg_name) +def create_ctypes_annotations(): + """ + create_ctypes_annotation creates a map between + ctypes, annotation types and low level types. + For convenience, an existing map from low level types to + annotation types is used ('ll_to_annotation_map'). + """ + + from pypy.annotation.model import ll_to_annotation_map + for the_type, ll_type, wrap_arg in ctypes_annotation_list: + the_type.annotator_type = ll_to_annotation_map.get(ll_type) + the_type.ll_type = ll_type + if wrap_arg is not None: + the_type.wrap_arg = wrap_arg +create_ctypes_annotations() class FunctionPointerTranslation(object): From mwh at codespeak.net Wed Jan 25 18:25:53 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Wed, 25 Jan 2006 18:25:53 +0100 (CET) Subject: [pypy-svn] r22661 - in pypy/dist/pypy/rpython/memory: . test Message-ID: <20060125172553.F183E27B64@code1.codespeak.net> Author: mwh Date: Wed Jan 25 18:25:51 2006 New Revision: 22661 Added: pypy/dist/pypy/rpython/memory/gctransform.py (contents, props changed) pypy/dist/pypy/rpython/memory/test/test_gctransform.py (contents, props changed) Log: (cfbolz, mwh) A start at a gctransformer that transforms l2 graphs, inserting (at least potentially) the operations needed by the GC. Some over complicated tests. Added: pypy/dist/pypy/rpython/memory/gctransform.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/rpython/memory/gctransform.py Wed Jan 25 18:25:51 2006 @@ -0,0 +1,91 @@ +from pypy.rpython.lltypesystem import lltype +from pypy.objspace.flow.model import SpaceOperation, Variable +from pypy.translator.unsimplify import insert_empty_block +import sets + +""" +thought experiments + +'setfield' obj field value -> + a1 <- 'cast_ptr_to_adr' obj + a2 <- 'cast_ptr_to_adr' value + 'direct_call' write_barrier a1, offset(TYPE(obj), field), a2 + +operations that need hooks: + +setfield, setarrayitem, direct_call, indirect_call, malloc, getfield, +getarrayitem, getsubstruct? + +push_alive, pop_alive, + +""" + +def var_needsgc(var): + vartype = var.concretetype + return isinstance(vartype, lltype.Ptr) and vartype._needsgc() + +class GCTransformer: + def __init__(self, graphs): + self.graphs = graphs + + def transform(self): + for graph in self.graphs: + self.transform_graph(graph) + + def transform_graph(self, graph): + self.links_to_split = {} # link -> vars to pop_alive across the link + for block in graph.iterblocks(): + self.transform_block(block) + for link, vars in self.links_to_split.iteritems(): + newops = [] + for var in vars: + newops.extend(self.pop_alive(var)) + insert_empty_block(None, link, newops) + + def transform_block(self, block): + newops = [] + livevars = [var for var in block.inputargs if var_needsgc(var)] + for op in block.operations: + newops.extend(self.replacement_operations(op)) + if var_needsgc(op.result): + if op.opname not in ('direct_call', 'indirect_call'): + newops.extend(self.push_alive(op.result)) + livevars.append(op.result) + if len(block.exits) == 0: + # everything is fine already for returnblocks and exceptblocks + pass + elif len(block.exits) == 1: + for var in livevars: + if var not in block.exits[0].args: + newops.extend(self.pop_alive(var)) + else: + deadinallexits = sets.Set(livevars) + for link in block.exits: + deadinallexits.difference_update(sets.Set(link.args)) + for var in deadinallexits: + newops.extend(self.pop_alive(var)) + for link in block.exits: + deadvarsforlink = sets.Set(livevars) - deadinallexits - sets.Set(link.args) + if deadvarsforlink: + self.links_to_split[link] = deadvarsforlink + if newops: + block.operations = newops + + def replacement_operations(self, op): + m = getattr(self, 'replace_' + op.opname, None) + if m: + return m(op) + else: + return [op] + + def push_alive(self, var): + result = Variable() + result.concretetype = lltype.Void + return [SpaceOperation("push_alive", [var], result)] + + def pop_alive(self, var): + result = Variable() + result.concretetype = lltype.Void + return [SpaceOperation("pop_alive", [var], result)] + + Added: pypy/dist/pypy/rpython/memory/test/test_gctransform.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/rpython/memory/test/test_gctransform.py Wed Jan 25 18:25:51 2006 @@ -0,0 +1,91 @@ +from pypy.rpython.memory import gctransform +from pypy.translator.translator import TranslationContext, graphof +from pypy.rpython.lltypesystem import lltype + +def rtype_and_transform(func, inputtypes, transformcls): + t = TranslationContext() + t.buildannotator().build_types(func, inputtypes) + t.buildrtyper().specialize(t) + transformer = transformcls(t.graphs) + transformer.transform() + t.checkgraphs() + return t + +def test_simple(): + def f(): + return 1 + rtype_and_transform(f, [], gctransform.GCTransformer) + +def test_fairly_simple(): + class C: + pass + def f(): + c = C() + c.x = 1 + return c.x + t = rtype_and_transform(f, [], gctransform.GCTransformer) + +def test_return_gcpointer(): + class C: + pass + def f(): + c = C() + c.x = 1 + return c + t = rtype_and_transform(f, [], gctransform.GCTransformer) + +def test_call_function(): + class C: + pass + def f(): + c = C() + c.x = 1 + return c + def g(): + return f().x + t = rtype_and_transform(g, [], gctransform.GCTransformer) + ggraph = graphof(t, g) + for i, op in enumerate(ggraph.startblock.operations): + if op.opname == "direct_call": + break + else: + assert False, "direct_call not found!" + assert ggraph.startblock.operations[i + 1].opname != 'push_alive' + +def test_multiple_exits(): + S = lltype.GcStruct("S", ('x', lltype.Signed)) + T = lltype.GcStruct("T", ('y', lltype.Signed)) + def f(n): + c = lltype.malloc(S) + d = lltype.malloc(T) + e = lltype.malloc(T) + if n: + x = d + else: + x = e + return x + t = rtype_and_transform(f, [int], gctransform.GCTransformer) + fgraph = graphof(t, f) + from pypy.translator.backendopt.ssa import SSI_to_SSA + SSI_to_SSA(fgraph) # *cough* + #t.view() + pop_alive_count = 0 + for i, op in enumerate(fgraph.startblock.operations): + if op.opname == "pop_alive": + var, = op.args + assert var.concretetype == lltype.Ptr(S) + pop_alive_count += 1 + + assert pop_alive_count == 1, "pop_alive not found!" + for link in fgraph.startblock.exits: + assert len(link.args) == 2 + ops = link.target.operations + assert len(ops) == 1 + assert ops[0].opname == 'pop_alive' + assert len(ops[0].args) == len(link.target.exits) == \ + len(link.target.exits[0].args) == 1 + dyingname = ops[0].args[0].name + passedname = link.target.exits[0].args[0].name + assert dyingname != passedname + + From tismer at codespeak.net Wed Jan 25 18:48:34 2006 From: tismer at codespeak.net (tismer at codespeak.net) Date: Wed, 25 Jan 2006 18:48:34 +0100 (CET) Subject: [pypy-svn] r22662 - pypy/dist/pypy/translator/c/test Message-ID: <20060125174834.566FF27DC3@code1.codespeak.net> Author: tismer Date: Wed Jan 25 18:48:32 2006 New Revision: 22662 Removed: pypy/dist/pypy/translator/c/test/test_coroutine.py Log: dropped test_coroutine, because all of this is now living in module/stackless From gromit at codespeak.net Wed Jan 25 19:12:10 2006 From: gromit at codespeak.net (gromit at codespeak.net) Date: Wed, 25 Jan 2006 19:12:10 +0100 (CET) Subject: [pypy-svn] r22663 - in pypy/dist/pypy/rpython/rctypes: . test Message-ID: <20060125181210.C65ED27DBF@code1.codespeak.net> Author: gromit Date: Wed Jan 25 19:12:08 2006 New Revision: 22663 Modified: pypy/dist/pypy/rpython/rctypes/__init__.py pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py Log: CHG: refactored the rcytpes tests. Expose interface.py at the package interface. Modified: pypy/dist/pypy/rpython/rctypes/__init__.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/__init__.py (original) +++ pypy/dist/pypy/rpython/rctypes/__init__.py Wed Jan 25 19:12:08 2006 @@ -0,0 +1 @@ +from pypy.rpython.rctypes.interface import * Modified: pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py Wed Jan 25 19:12:08 2006 @@ -5,95 +5,78 @@ from pypy.translator.tool.cbuild import compile_c_module import sys - -def setup_module(mod): - try: - import ctypes - except ImportError: - py.test.skip("this test needs ctypes installed") - else: - from pypy.rpython.rctypes.interface import cdll, c_char_p, c_int, c_char, POINTER, Structure, byref - if sys.platform == 'win32': - mylib = cdll.LoadLibrary('msvcrt.dll') - elif sys.platform == 'linux2': - mylib = cdll.LoadLibrary('libc.so.6') - else: - py.test.skip("don't know how to load the c lib for %s" % - sys.platform) - - atoi = mylib.atoi - atoi.restype = c_int - atoi.argtypes = [c_char_p] - atoi.argtypes = [POINTER(c_char)] - def o_atoi(a): - return atoi(a) - mod.o_atoi = o_atoi - mod.cdll = cdll - - class tagpoint(Structure): - _fields_ = [("x", c_int), - ("y", c_int)] - mod.tagpoint = tagpoint - mod.byref = byref - - # compile and load our local test C file - compile_c_module([py.path.local("_rctypes_test.c")], "_rctypes_test") - - if sys.platform == "win32": - _rctypes_test = cdll.LoadLibrary("_rctypes_test.pyd") - else: - _rctypes_test = cdll.LoadLibrary("_rctypes_test.so") - - # _testfunc_byval - testfunc_byval = _rctypes_test._testfunc_byval - testfunc_byval.restype = c_int - testfunc_byval.argtypes = [tagpoint,POINTER(tagpoint)] - - def py_testfunc_byval(inpoint): - opoint = tagpoint() - res = testfunc_byval(inpoint,byref(opoint)) - - return res, opoint - - mod.py_testfunc_byval = py_testfunc_byval - - # _test_struct - testfunc_struct = _rctypes_test._testfunc_struct - testfunc_struct.restype = c_int - testfunc_struct.argtypes = [tagpoint] - - def py_testfunc_struct(inpoint): - return testfunc_struct(inpoint) - - mod.py_testfunc_struct = py_testfunc_struct - - # _test_struct_id - testfunc_struct_id = _rctypes_test._testfunc_struct_id - testfunc_struct_id.restype = tagpoint - testfunc_struct_id.argtypes = [tagpoint] - - def py_testfunc_struct_id(inpoint): - return testfunc_struct_id(inpoint) - - mod.py_testfunc_struct_id = py_testfunc_struct_id - - def py_create_point(): - p = tagpoint() - p.x = 10 - p.y = 20 - return p.x + p.y - - mod.py_create_point = py_create_point - - oppoint_type = POINTER(tagpoint) - def py_testfunc_POINTER(inpoint): - point = tagpoint() - oppoint = oppoint_type(point) - res = testfunc_byval(inpoint,oppoint) - return res, oppoint - - mod.py_testfunc_POINTER = py_testfunc_POINTER - mod.POINTER = POINTER +try: + import ctypes +except ImportError: + py.test.skip("this test needs ctypes installed") + + +from pypy.rpython.rctypes import cdll, c_char_p, c_int, c_char, POINTER, Structure, byref +if sys.platform == 'win32': + mylib = cdll.LoadLibrary('msvcrt.dll') +elif sys.platform == 'linux2': + mylib = cdll.LoadLibrary('libc.so.6') +else: + py.test.skip("don't know how to load the c lib for %s" % + sys.platform) +atoi = mylib.atoi +atoi.restype = c_int +atoi.argtypes = [c_char_p] +atoi.argtypes = [POINTER(c_char)] + +def o_atoi(a): + return atoi(a) + +class tagpoint(Structure): + _fields_ = [("x", c_int), + ("y", c_int)] +# compile and load our local test C file +compile_c_module([py.path.local("_rctypes_test.c")], "_rctypes_test") + +if sys.platform == "win32": + _rctypes_test = cdll.LoadLibrary("_rctypes_test.pyd") +else: + _rctypes_test = cdll.LoadLibrary("_rctypes_test.so") + +# _testfunc_byval +testfunc_byval = _rctypes_test._testfunc_byval +testfunc_byval.restype = c_int +testfunc_byval.argtypes = [tagpoint,POINTER(tagpoint)] + +def py_testfunc_byval(inpoint): + opoint = tagpoint() + res = testfunc_byval(inpoint,byref(opoint)) + + return res, opoint + +# _test_struct +testfunc_struct = _rctypes_test._testfunc_struct +testfunc_struct.restype = c_int +testfunc_struct.argtypes = [tagpoint] + +def py_testfunc_struct(inpoint): + return testfunc_struct(inpoint) + +# _test_struct_id +testfunc_struct_id = _rctypes_test._testfunc_struct_id +testfunc_struct_id.restype = tagpoint +testfunc_struct_id.argtypes = [tagpoint] + +def py_testfunc_struct_id(inpoint): + return testfunc_struct_id(inpoint) + +def py_create_point(): + p = tagpoint() + p.x = 10 + p.y = 20 + return p.x + p.y + +oppoint_type = POINTER(tagpoint) +def py_testfunc_POINTER(inpoint): + point = tagpoint() + oppoint = oppoint_type(point) + res = testfunc_byval(inpoint,oppoint) + return res, oppoint class Test_rctypes: From ac at codespeak.net Wed Jan 25 19:31:29 2006 From: ac at codespeak.net (ac at codespeak.net) Date: Wed, 25 Jan 2006 19:31:29 +0100 (CET) Subject: [pypy-svn] r22664 - pypy/branch/jit-refactoring Message-ID: <20060125183129.9F74827DBF@code1.codespeak.net> Author: ac Date: Wed Jan 25 19:31:29 2006 New Revision: 22664 Added: pypy/branch/jit-refactoring/ - copied from r22663, pypy/dist/ Log: Create a branch to save work on refactoring the jit without b reaking everything for others. From cfbolz at codespeak.net Wed Jan 25 19:42:58 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Wed, 25 Jan 2006 19:42:58 +0100 (CET) Subject: [pypy-svn] r22665 - pypy/dist/pypy/doc Message-ID: <20060125184258.8BCB627DBF@code1.codespeak.net> Author: cfbolz Date: Wed Jan 25 19:42:57 2006 New Revision: 22665 Modified: pypy/dist/pypy/doc/news.txt Log: news item for the sprint Modified: pypy/dist/pypy/doc/news.txt ============================================================================== --- pypy/dist/pypy/doc/news.txt (original) +++ pypy/dist/pypy/doc/news.txt Wed Jan 25 19:42:57 2006 @@ -9,6 +9,21 @@ .. _Python: http://www.python.org/doc/current/ref/ref.html .. _`more...`: http://codespeak.net/pypy/dist/pypy/doc/architecture.html#mission-statement +Current PyPy Sprint in Palma De Mallorca 23rd - 29th January 2006 +=================================================================== + +The current sprint is taking place right now in Palma de Mallorca. +Topics include progressing with the JIT work started in G?teborg +and Paris, GC and optimisation work, stackless, and +improving our way to write glue code for C libraries. + +Read more in `the announcement`_, there is a `sprint report`_ +for the first three days. + + +.. _`the announcement`: http://codespeak.net/pypy/extradoc/sprintinfo/mallorca/sprint-announcement.html +.. _`sprint report`: http://codespeak.net/pipermail/pypy-dev/2006q1/002746.html + Preliminary EU reports released =============================== @@ -20,20 +35,6 @@ .. _`reports for the EU`: http://codespeak.net/pypy/dist/pypy/doc/index-report.html -Next PyPy Sprint in Palma De Mallorca 23rd - 29th January 2006 -=============================================================== - -The next sprint is organized for end of January in Palma de Mallorca. -Topics will include progressing with the JIT work started in G?teborg -and Paris, GC and optimisation work, logic programming, and possibly -improving our way to write glue code for C libraries. As usual we will -give newcomer-friendly introductions, and other topics are possible. - -Read more in `the announcement`_, see who is planning to attend -on the `people page`_. - -.. _`the announcement`: http://codespeak.net/pypy/extradoc/sprintinfo/mallorca/sprint-announcement.html -.. _`People page`: http://codespeak.net/svn/pypy/extradoc/sprintinfo/mallorca/people.txt PyPy Sprint in G?teborg 7th - 11th December 2005 ================================================= From ac at codespeak.net Wed Jan 25 19:45:52 2006 From: ac at codespeak.net (ac at codespeak.net) Date: Wed, 25 Jan 2006 19:45:52 +0100 (CET) Subject: [pypy-svn] r22667 - pypy/branch/jit-refactoring Message-ID: <20060125184552.C978427DCA@code1.codespeak.net> Author: ac Date: Wed Jan 25 19:45:52 2006 New Revision: 22667 Modified: pypy/branch/jit-refactoring/ (props changed) Log: Initialized merge tracking via "svnmerge" with revisions "1-22663" from svn+ssh://codespeak.net/svn/pypy/dist From ac at codespeak.net Wed Jan 25 19:46:28 2006 From: ac at codespeak.net (ac at codespeak.net) Date: Wed, 25 Jan 2006 19:46:28 +0100 (CET) Subject: [pypy-svn] r22668 - in pypy/branch/jit-refactoring/pypy: jit rpython Message-ID: <20060125184628.06AD427DCA@code1.codespeak.net> Author: ac Date: Wed Jan 25 19:46:27 2006 New Revision: 22668 Modified: pypy/branch/jit-refactoring/pypy/jit/llabstractinterp.py pypy/branch/jit-refactoring/pypy/jit/llcontainer.py pypy/branch/jit-refactoring/pypy/jit/llvalue.py pypy/branch/jit-refactoring/pypy/jit/vlist.py pypy/branch/jit-refactoring/pypy/rpython/rgenop.py Log: Intermediary work. Modified: pypy/branch/jit-refactoring/pypy/jit/llabstractinterp.py ============================================================================== --- pypy/branch/jit-refactoring/pypy/jit/llabstractinterp.py (original) +++ pypy/branch/jit-refactoring/pypy/jit/llabstractinterp.py Wed Jan 25 19:46:27 2006 @@ -2,7 +2,7 @@ from pypy.objspace.flow.model import Variable, Constant, SpaceOperation from pypy.objspace.flow.model import checkgraph, c_last_exception from pypy.rpython.lltypesystem import lltype -from pypy.jit.llvalue import LLAbstractValue, const +from pypy.jit.llvalue import LLAbstractValue, const, aconst, AConstant, AVariable from pypy.jit.llvalue import ll_no_return_value from pypy.jit.llvalue import FlattenMemo, MatchMemo, FreezeMemo, UnfreezeMemo from pypy.jit.llcontainer import LLAbstractContainer, virtualcontainervalue @@ -41,7 +41,7 @@ assert len(incoming_link_args) == len(selfvalues) for linkarg, fr in zip(incoming_link_args, selfvalues): if fr.fixed: - assert isinstance(linkarg, Constant), ( + assert isinstance(linkarg, AConstant), ( "unexpected Variable %r reaching the fixed input arg %r" % (linkarg, fr)) yield linkarg @@ -52,6 +52,10 @@ assert not self.frozen, "getruntimevars(): not for frozen states" return [a.runtimevar for a in self.flatten()] + def getgenvars(self, builder): + assert not self.frozen, "getgenvars(): not for frozen states" + return [a.getgenvar(builder) for a in self.flatten()] + def flatten(self, memo=None): if memo is None: memo = FlattenMemo() @@ -102,8 +106,8 @@ for v, a in zip(self.getlivevars(), self.args_a): a = a.unfreeze(memo, block) # try to preserve the name - if isinstance(a.runtimevar, Variable): - a.runtimevar.rename(v) + #if isinstance(a.runtimevar, Variable): + # a.runtimevar.rename(v) new_args_a.append(a) return self.__class__(new_back, new_args_a, *self.localkey()) @@ -171,10 +175,10 @@ args_a = [] for i, v in enumerate(origgraph.getargs()): if i in arghints: - a = LLAbstractValue(const(arghints[i])) + a = LLAbstractValue(aconst(arghints[i])) a.concrete = self.policy.concrete_args else: - a = LLAbstractValue(input=v.concretetype) + a = LLAbstractValue(AVariable(), input=v.concretetype) args_a.append(a) graphstate = self.schedule_graph(args_a, origgraph) graphstate.complete() @@ -192,10 +196,10 @@ #print "SCHEDULE_GRAPH", graphstate return graphstate - def schedule(self, inputstate): + def schedule(self, inputstate, builder): #print "SCHEDULE", args_a, origblock frozenstate = self.schedule_getstate(inputstate) - args_v = inputstate.getruntimevars() + args_v = inputstate.getgenvars(builder) return LinkState(args_v, frozenstate) def schedule_getstate(self, inputstate): @@ -259,7 +263,7 @@ self.link = None def setreturn(self): - rgenop.closereturnlink(self.link, self.args_v[0]) + rgenop.closereturnlink(self.link, self.args_v[0]) def settarget(self, block, blockargs): args = [] @@ -280,7 +284,7 @@ self.name = '%s_%d' % (origgraph.name, n) # The 'args_v' attribute is needed by try_to_complete(), # which looks for it on either a GraphState or a LinkState - self.args_v = inputstate.getruntimevars() + self.args_v = inputstate.getgenvars(None) self.frozenstate = frozenstate #self.a_return = None self.state = "before" @@ -370,7 +374,7 @@ merged_key = [] recompute = False for c1, c2 in zip(blockargs, next.args_v): - if isinstance(c1, Constant) and c1 != c2: + if isinstance(c1, AConstant) and c1 != c2: # incompatibility merged_key.append(None) # force a Variable recompute = True @@ -418,7 +422,7 @@ for a1, a2, k in zip(state.flatten(), builder.runningstate.flatten(), key): - if isinstance(k, Constant): + if isinstance(k, AConstant): arglist.append('%s => %s' % (a1, k)) else: arglist.append('%s => %s' % (a1, a2)) @@ -488,7 +492,7 @@ args_a = [builder.binding(v) for v in origlink.args] nextinputstate = LLBlockState(builder.runningstate.back, args_a, origlink.target) - newlinkstate = self.interp.schedule(nextinputstate) + newlinkstate = self.interp.schedule(nextinputstate, builder) if newexitswitch is not None: newlinkstate.exitcase = origlink.exitcase newlinkstate.llexitcase = origlink.llexitcase @@ -534,7 +538,7 @@ def binding(self, v): if isinstance(v, Constant): - return LLAbstractValue(v) + return LLAbstractValue(aconst(v.value, v.concretetype)) else: return self.bindings[v] @@ -564,7 +568,7 @@ # can constant-fold print 'fold:', constant_op.__name__, concretevalues concreteresult = constant_op(*concretevalues) - a_result = LLAbstractValue(const(concreteresult)) + a_result = LLAbstractValue(aconst(concreteresult)) if any_concrete and self.interp.policy.concrete_propagate: a_result.concrete = True else: @@ -572,8 +576,8 @@ return a_result def residual(self, op, args_a): - retvar = rgenop.genop(self.newblock, op.opname, - [a.forcevarorconst(self) for a in args_a], + retvar = self.genop(op.opname, + [a.forcegenvarorconst(self) for a in args_a], op.result.concretetype) return LLAbstractValue(retvar) @@ -723,7 +727,7 @@ parentstate = LLSuspendedBlockState(self.runningstate.back, alive_a, origblock, origposition) nextstate = LLBlockState(parentstate, args_a, origgraph.startblock) - raise InsertNextLink(self.interp.schedule(nextstate)) + raise InsertNextLink(self.interp.schedule(nextstate, self)) def handle_call_residual(self, op, origgraph, *args_a): # residual call: for now we need to force all arguments @@ -781,7 +785,7 @@ def op_getarraysize(self, op, a_ptr): if hasllcontent(a_ptr): - return LLAbstractValue(const(a_ptr.content.length)) + return LLAbstractValue(aconst(a_ptr.content.length)) return self.residualize(op, [a_ptr], len) def op_getarrayitem(self, op, a_ptr, a_index): @@ -846,9 +850,10 @@ memo = FlattenMemo() a_ptr.flatten(memo) for a in memo.result: - v = a.runtimevar - if isinstance(v, Variable) and not v.concretetype._is_atomic(): - rgenop.genop(self.newblock, 'keepalive', [v], lltype.Void) + if (a.is_variable and + not a.getconcretetype()._is_atomic()): + self.genop('keepalive', [a.forcegenvarorconst()], + lltype.Void) return ll_no_return_value # High-level operation dispatcher @@ -863,7 +868,7 @@ args_a = [] for a in argtuple: if not isinstance(a, LLAbstractValue): - a = LLAbstractValue(const(a)) + a = LLAbstractValue(aconst(a)) args_a.append(a) # end of rather XXX'edly hackish parsing Modified: pypy/branch/jit-refactoring/pypy/jit/llcontainer.py ============================================================================== --- pypy/branch/jit-refactoring/pypy/jit/llcontainer.py (original) +++ pypy/branch/jit-refactoring/pypy/jit/llcontainer.py Wed Jan 25 19:46:27 2006 @@ -216,7 +216,7 @@ a.content.parentindex = parentindex else: # primitive initialized to zero - a = LLAbstractValue(const(T._defl())) + a = LLAbstractValue(aconst(T._defl())) return a def hasllcontent(a_ptr): Modified: pypy/branch/jit-refactoring/pypy/jit/llvalue.py ============================================================================== --- pypy/branch/jit-refactoring/pypy/jit/llvalue.py (original) +++ pypy/branch/jit-refactoring/pypy/jit/llvalue.py Wed Jan 25 19:46:27 2006 @@ -1,6 +1,18 @@ from pypy.objspace.flow.model import Variable, Constant from pypy.rpython.lltypesystem import lltype from pypy.rpython import rgenop +from pypy.tool.uid import Hashable + +class AVariable(object): + def __init__(self, genvar=None): + self.genvar = genvar + + def getgenvar(self, builder): + return self.genvar + +class AConstant(Hashable): + def getgenvar(self, v, builder): + class LLAbstractValue(object): """An abstract value, propagated through the blocks of the low-level @@ -15,13 +27,14 @@ concrete = False # concrete constants propagate eagerly - def __init__(self, runtimevar=None, content=None, input=None): + def __init__(self, runtimevar=None, genvar=None, content=None, input=None): self.runtimevar = runtimevar # None or a Variable or a Constant + self.genvar = genvar self.content = content # None or an LLAbstractContainer self.origin = [] # list of frozen values: the sources that did or # could allow 'self' to be computed as a constant self.input = input - + def __repr__(self): if self.input: return '' @@ -78,12 +91,26 @@ if self.runtimevar is None: if self.content is None: raise ValueError("ll_dummy_value.forcevarorconst()") - self.runtimevar = self.content.build_runtime_container(builder) + XXX + self.content.build_runtime_container(builder) self.content = None return self.runtimevar + def forcegenvarorconst(self, builder): + if self.runtimevar is None: + if self.content is None: + raise ValueError("ll_dummy_value.forcevarorconst()") + self.genvar = self.content.build_runtime_container(builder) + self.content = None + return self.getgenvar(builder) + + def getgenvar(self, builder): + if self.genvar is None: + assert isinstance(self.runtimevar, AConstant) + self.genvar = builder.genconst(self.runtimevar.value) + return self.genvar + def maybe_get_constant(self): - if isinstance(self.runtimevar, Constant): + if isinstance(self.runtimevar, AConstant): return self.runtimevar else: return None @@ -177,12 +204,13 @@ # no need to worry about sharing here: LLFrozenRuntimeValues are # never shared propagateconst = memo.propagateconsts.next() - if isinstance(propagateconst, Constant): + if isinstance(propagateconst, AConstant): v = propagateconst # allowed to propagate as a Constant assert v.concretetype == self.concretetype + result = LLAbstractValue(v) else: v = rgenop.geninputarg(block, self.concretetype) - result = LLAbstractValue(v) + result = LLAbstractValue(v, genvar=v) result.origin.append(self) return result @@ -257,6 +285,11 @@ c.concretetype = T or lltype.typeOf(value) return c -ll_no_return_value = LLAbstractValue(const(None, lltype.Void)) +def aconst(value, T=None): + c = AConstant(value) + c.concretetype = T or lltype.typeOf(value) + return c + +ll_no_return_value = LLAbstractValue(aconst(None, lltype.Void)) ll_dummy_value = LLAbstractValue() frozen_dummy_value = LLFrozenDummyValue() Modified: pypy/branch/jit-refactoring/pypy/jit/vlist.py ============================================================================== --- pypy/branch/jit-refactoring/pypy/jit/vlist.py (original) +++ pypy/branch/jit-refactoring/pypy/jit/vlist.py Wed Jan 25 19:46:27 2006 @@ -1,6 +1,6 @@ from pypy.rpython.lltypesystem import lltype from pypy.rpython.rtyper import LowLevelOpList -from pypy.jit.llvalue import LLAbstractValue, const, ll_dummy_value +from pypy.jit.llvalue import LLAbstractValue, aconst, ll_dummy_value from pypy.jit.llcontainer import LLAbstractContainer @@ -47,10 +47,10 @@ # High-level operations def oop_len(self, op): - return LLAbstractValue(const(len(self.items_a))) + return LLAbstractValue(aconst(len(self.items_a))) def oop_nonzero(self, op): - return LLAbstractValue(const(bool(self.items_a))) + return LLAbstractValue(aconst(bool(self.items_a))) def oop_getitem(self, op, a_index): c_index = a_index.maybe_get_constant() Modified: pypy/branch/jit-refactoring/pypy/rpython/rgenop.py ============================================================================== --- pypy/branch/jit-refactoring/pypy/rpython/rgenop.py (original) +++ pypy/branch/jit-refactoring/pypy/rpython/rgenop.py Wed Jan 25 19:46:27 2006 @@ -44,6 +44,7 @@ return link def closeblock2(block, exitswitch): + assert isinstance(exitswitch, flowmodel.Variable) block.exitswitch = exitswitch false_link = flowmodel.Link([], None) false_link.exitcase = False @@ -56,6 +57,8 @@ def closelink(link, vars, targetblock): if isinstance(link, flowmodel.Link): + for v in vars: + assert isinstance(v, (flowmodel.Variable, flowmodel.Constant)) assert ([v.concretetype for v in vars] == [v.concretetype for v in targetblock.inputargs]) link.args[:] = vars From cfbolz at codespeak.net Thu Jan 26 10:35:19 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Thu, 26 Jan 2006 10:35:19 +0100 (CET) Subject: [pypy-svn] r22672 - in pypy/dist/pypy/rpython/memory: . test Message-ID: <20060126093519.07DA327DB5@code1.codespeak.net> Author: cfbolz Date: Thu Jan 26 10:35:17 2006 New Revision: 22672 Modified: pypy/dist/pypy/rpython/memory/gctransform.py pypy/dist/pypy/rpython/memory/test/test_gctransform.py Log: attach the variables that need to be pop_alived when an exception is raised to the (in)direct_call. Modified: pypy/dist/pypy/rpython/memory/gctransform.py ============================================================================== --- pypy/dist/pypy/rpython/memory/gctransform.py (original) +++ pypy/dist/pypy/rpython/memory/gctransform.py Thu Jan 26 10:35:17 2006 @@ -1,6 +1,7 @@ from pypy.rpython.lltypesystem import lltype from pypy.objspace.flow.model import SpaceOperation, Variable from pypy.translator.unsimplify import insert_empty_block +from pypy.rpython import rmodel import sets """ @@ -47,6 +48,8 @@ livevars = [var for var in block.inputargs if var_needsgc(var)] for op in block.operations: newops.extend(self.replacement_operations(op)) + if op.opname in ('direct_call', 'indirect_call') and livevars: + op.args.append(rmodel.inputconst(lltype.Void, livevars[:])) if var_needsgc(op.result): if op.opname not in ('direct_call', 'indirect_call'): newops.extend(self.push_alive(op.result)) @@ -88,4 +91,3 @@ result.concretetype = lltype.Void return [SpaceOperation("pop_alive", [var], result)] - Modified: pypy/dist/pypy/rpython/memory/test/test_gctransform.py ============================================================================== --- pypy/dist/pypy/rpython/memory/test/test_gctransform.py (original) +++ pypy/dist/pypy/rpython/memory/test/test_gctransform.py Thu Jan 26 10:35:17 2006 @@ -88,4 +88,20 @@ passedname = link.target.exits[0].args[0].name assert dyingname != passedname - +def test_cleanup_vars_on_call(): + S = lltype.GcStruct("S", ('x', lltype.Signed)) + def f(): + return lltype.malloc(S) + def g(): + s1 = f() + s2 = f() + s3 = f() + return s1 + t = rtype_and_transform(g, [], gctransform.GCTransformer) + ggraph = graphof(t, g) + direct_calls = [op for op in ggraph.startblock.operations if op.opname == "direct_call"] + assert len(direct_calls) == 3 + assert len(direct_calls[0].args) == 1 + assert direct_calls[1].args[1].value[0] == direct_calls[0].result + assert direct_calls[2].args[1].value == [direct_calls[0].result, direct_calls[1].result] + From cfbolz at codespeak.net Thu Jan 26 10:52:29 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Thu, 26 Jan 2006 10:52:29 +0100 (CET) Subject: [pypy-svn] r22673 - in pypy/dist/pypy/rpython/memory: . test Message-ID: <20060126095229.3D0CE27B4C@code1.codespeak.net> Author: cfbolz Date: Thu Jan 26 10:52:27 2006 New Revision: 22673 Modified: pypy/dist/pypy/rpython/memory/gctransform.py pypy/dist/pypy/rpython/memory/test/test_gctransform.py Log: actually attach the pop_alive operations themselves -- the gctransformers might want to do strange stuff there after all Modified: pypy/dist/pypy/rpython/memory/gctransform.py ============================================================================== --- pypy/dist/pypy/rpython/memory/gctransform.py (original) +++ pypy/dist/pypy/rpython/memory/gctransform.py Thu Jan 26 10:52:27 2006 @@ -49,7 +49,10 @@ for op in block.operations: newops.extend(self.replacement_operations(op)) if op.opname in ('direct_call', 'indirect_call') and livevars: - op.args.append(rmodel.inputconst(lltype.Void, livevars[:])) + cleanup_on_exception = [] + for var in livevars: + cleanup_on_exception.extend(self.pop_alive(var)) + op.args.append(rmodel.inputconst(lltype.Void, cleanup_on_exception)) if var_needsgc(op.result): if op.opname not in ('direct_call', 'indirect_call'): newops.extend(self.push_alive(op.result)) Modified: pypy/dist/pypy/rpython/memory/test/test_gctransform.py ============================================================================== --- pypy/dist/pypy/rpython/memory/test/test_gctransform.py (original) +++ pypy/dist/pypy/rpython/memory/test/test_gctransform.py Thu Jan 26 10:52:27 2006 @@ -102,6 +102,6 @@ direct_calls = [op for op in ggraph.startblock.operations if op.opname == "direct_call"] assert len(direct_calls) == 3 assert len(direct_calls[0].args) == 1 - assert direct_calls[1].args[1].value[0] == direct_calls[0].result - assert direct_calls[2].args[1].value == [direct_calls[0].result, direct_calls[1].result] + assert direct_calls[1].args[1].value[0].args[0] == direct_calls[0].result + assert [op.args[0] for op in direct_calls[2].args[1].value] == [direct_calls[0].result, direct_calls[1].result] From bea at codespeak.net Thu Jan 26 11:38:11 2006 From: bea at codespeak.net (bea at codespeak.net) Date: Thu, 26 Jan 2006 11:38:11 +0100 (CET) Subject: [pypy-svn] r22674 - pypy/extradoc/planning/malaga_exhibition_feb2006 Message-ID: <20060126103811.80C4927B68@code1.codespeak.net> Author: bea Date: Thu Jan 26 11:37:51 2006 New Revision: 22674 Added: pypy/extradoc/planning/malaga_exhibition_feb2006/ pypy/extradoc/planning/malaga_exhibition_feb2006/pypy_IICISL memoria descriptiva stand.doc (contents, props changed) pypy/extradoc/planning/malaga_exhibition_feb2006/pypy_programa CISL 2006 - WEB.xls (contents, props changed) pypy/extradoc/planning/malaga_exhibition_feb2006/pypy_programme CISL 2006 - web.xls (contents, props changed) Log: created a folder for the malaga exhibition feb 2006 planning Added: pypy/extradoc/planning/malaga_exhibition_feb2006/pypy_IICISL memoria descriptiva stand.doc ============================================================================== Binary file. No diff available. Added: pypy/extradoc/planning/malaga_exhibition_feb2006/pypy_programa CISL 2006 - WEB.xls ============================================================================== Binary file. No diff available. Added: pypy/extradoc/planning/malaga_exhibition_feb2006/pypy_programme CISL 2006 - web.xls ============================================================================== Binary file. No diff available. From auc at codespeak.net Thu Jan 26 11:43:07 2006 From: auc at codespeak.net (auc at codespeak.net) Date: Thu, 26 Jan 2006 11:43:07 +0100 (CET) Subject: [pypy-svn] r22675 - pypy/dist/pypy/lib/logic Message-ID: <20060126104307.DEAB827B64@code1.codespeak.net> Author: auc Date: Thu Jan 26 11:43:03 2006 New Revision: 22675 Modified: pypy/dist/pypy/lib/logic/computationspace.py pypy/dist/pypy/lib/logic/distributor.py pypy/dist/pypy/lib/logic/test_computationspace.py Log: (ale, auc) * ask & initialisation refactoring * status & distributable more close to correctness Modified: pypy/dist/pypy/lib/logic/computationspace.py ============================================================================== --- pypy/dist/pypy/lib/logic/computationspace.py (original) +++ pypy/dist/pypy/lib/logic/computationspace.py Thu Jan 26 11:43:03 2006 @@ -177,23 +177,32 @@ ## space = ComputationSpace(fun=my_problem) +from unification import Store, var +from constraint import ConsistencyFailure +from threading import Thread, Condition + class Unprocessed: pass class Working: pass -class Failed: +class Failed(Exception): pass class Merged: + """Its constraint store has been added to a parent. + Any further operation operation on the space is + an error. + """ pass class Succeeded: + """It contains no choice points but a solution to + the logic program. + """ pass -from unification import Store, var -from constraint import ConsistencyFailure class ComputationSpace(object): @@ -207,37 +216,57 @@ self.root = parent.root self.program = program self.parent = parent + # status self.status = Unprocessed - self.children = set() + self.status_condition = Condition() + # run ... + self._process() + + def _process(self): + try: + self.store.satisfy_all() + except ConsistencyFailure: + self.status = Failed + else: + self.status = Succeeded def _stable(self): #XXX: really ? - return self.status in (Failed, Succeeded, Merged) + return self.status in (Failed, Succeeded, Merged) \ + or self._distributable() - def _distributable(self): + def _suspended(self): pass - #return not self._stable + # additional basic constraints done in an ancestor can + # make it runnable ; it is a temporary condition due + # to concurrency ; it means that some ancestor space + # has not yet transferred all required information to + # the space + + + def _distributable(self): + if self.status not in (Failed, Succeeded, Merged): + return self.distributor.findSmallestDomain() > 1 + # in The Book : "the space has one thread that is + # suspended on a choice point with two or more alternatives. + # A space canhave at most one choice point; attempting to + # create another gives an error." def set_distributor(self, dist): self.distributor = dist def ask(self): # XXX: block on status being not stable for threads - if self._stable(): - return self.status - #return self.store.nb_candidates() - else: - #TBD + try: + self.status_condition.acquire() + while not self._stable(): + self.status_condition.wait() + if self._distributable(): + return self.distributor.nb_subdomains() return self.status + finally: + self.status_condition.release() - def process(self): - self.status = Working - try: - self.store.satisfy_all() - except ConsistencyFailure: - self.status = Failed - else: - self.status = Succeeded def make_children(self): for dommap in self.distributor.distribute(): @@ -248,5 +277,36 @@ var.cs_set_dom(cs, dom) + def commit(self, some_number): + """if self is distributable, causes the Choose call in the + space ot complete and return some_number as a result. This + may cause the spzce to resume execution. + some_number must satisfy 1= Author: tismer Date: Thu Jan 26 12:00:38 2006 New Revision: 22676 Modified: pypy/dist/pypy/module/stackless/interp_coroutine.py pypy/dist/pypy/module/stackless/test/test_interp_coroutine.py Log: basic __del__ support and exception handling. Yet to be exposed/extended for app-level Modified: pypy/dist/pypy/module/stackless/interp_coroutine.py ============================================================================== --- pypy/dist/pypy/module/stackless/interp_coroutine.py (original) +++ pypy/dist/pypy/module/stackless/interp_coroutine.py Thu Jan 26 12:00:38 2006 @@ -13,12 +13,22 @@ class CoState(object): def __init__(self): self.last = self.current = self.main = Coroutine() + self.things_to_do = False + self.temp_exc = None + self.del_first = None + self.del_last = None costate = None class CoroutineDamage(SystemError): pass +class CoroutineExit(SystemExit): + # XXX SystemExit's __init__ creates problems in bookkeeper. + # XXX discuss this with the gurus :-) + def __init__(self): + pass + class Coroutine(Wrappable): def __init__(self): @@ -36,7 +46,15 @@ def _bind(self, thunk): self.parent = costate.current costate.last.frame = yield_current_frame_to_caller() - thunk.call() + try: + thunk.call() + except CoroutineExit: + # ignore a shutdown exception + pass + except Exception, e: + # redirect all unhandled exceptions to the parent + costate.things_to_do = True + costate.temp_exc = e if self.parent.frame is None: self.parent = costate.main return self._update_state(self.parent) @@ -46,6 +64,8 @@ raise CoroutineDamage costate.last.frame = self._update_state(self).switch() # note that last gets updated before assignment! + if costate.things_to_do: + do_things_to_do(self) def _update_state(new): costate.last, costate.current = costate.current, new @@ -53,6 +73,58 @@ return frame _update_state = staticmethod(_update_state) + def kill(self): + self._userdel() + if costate.current is self: + raise CoroutineExit + costate.things_to_do = True + costate.temp_exc = CoroutineExit() + self.parent = costate.current + self.switch() + + def __del__(self): + # provide the necessary clean-up if this coro is left + # with a frame. + # note that AppCoroutine has to take care about this + # as well, including a check for user-supplied __del__. + # Additionally note that in the context of __del__, we are + # not in the position to issue a switch. + # we defer it completely. + if self.frame is not None: + postpone_deletion(self) + + def _userdel(self): + # override this for exposed coros + pass + +## later: or self.space.lookup(self, '__del__') is not None + +def postpone_deletion(obj): + costate.things_to_do = True + if costate.del_first is None: + costate.del_first = costate.del_last = obj + costate.del_last.parent = obj + costate.del_last = obj + obj.parent = costate.del_first + +def do_things_to_do(obj): + if costate.temp_exc is not None: + # somebody left an unhandled exception and switched to us. + # this both provides default exception handling and the + # way to inject an exception, like CoroutineExit. + e, costate.temp_exc = costate.temp_exc, None + costate.things_to_do = costate.del_first is not None + raise e + if costate.del_first is not None: + obj = costate.del_first + costate.del_first = obj.parent + obj.parent = costate.current + if obj is costate.del_last: + costate.del_first = costate.del_last = None + obj.kill() + else: + costate.things_to_do = False + costate = CoState() Modified: pypy/dist/pypy/module/stackless/test/test_interp_coroutine.py ============================================================================== --- pypy/dist/pypy/module/stackless/test/test_interp_coroutine.py (original) +++ pypy/dist/pypy/module/stackless/test/test_interp_coroutine.py Thu Jan 26 12:00:38 2006 @@ -167,3 +167,43 @@ data = wrap_stackless_function(f) assert int(data.strip()) == 12345678 + +def test_kill_raise_coro(): + class T: + def __init__(self, func, arg): + self.func = func + self.arg = arg + def call(self): + self.func(self.arg, self) + + def g(nrec, t, count=0): + t.count = count + if nrec < 0: + raise ValueError + if nrec: + g(nrec-1, t, count+1) + costate.main.switch() + + def f(): + coro_g = Coroutine() + thunk_g = T(g, 42) + coro_g.bind(thunk_g) + coro_g.switch() + res = thunk_g.count + res *= 10 + res |= coro_g.frame is not None + # testing kill + coro_g.kill() + res *= 10 + res |= coro_g.frame is None + coro_g = Coroutine() + thunk_g = T(g, -42) + coro_g.bind(thunk_g) + try: + coro_g.switch() + except ValueError: + res += 500 + return res + + data = wrap_stackless_function(f) + assert int(data.strip()) == 4711 From stephan at codespeak.net Thu Jan 26 12:55:19 2006 From: stephan at codespeak.net (stephan at codespeak.net) Date: Thu, 26 Jan 2006 12:55:19 +0100 (CET) Subject: [pypy-svn] r22677 - in pypy/dist/pypy/rpython/rctypes: . test Message-ID: <20060126115519.467E127B70@code1.codespeak.net> Author: stephan Date: Thu Jan 26 12:55:16 2006 New Revision: 22677 Modified: pypy/dist/pypy/rpython/rctypes/implementation.py pypy/dist/pypy/rpython/rctypes/interface.py pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py Log: provides annotation information for basic ctypes (+ tests) Modified: pypy/dist/pypy/rpython/rctypes/implementation.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/implementation.py (original) +++ pypy/dist/pypy/rpython/rctypes/implementation.py Thu Jan 26 12:55:16 2006 @@ -3,6 +3,7 @@ """ import sys +import ctypes from ctypes import * from ctypes import _FUNCFLAG_CDECL if sys.platform == "win32": @@ -49,6 +50,12 @@ the_type.ll_type = ll_type if wrap_arg is not None: the_type.wrap_arg = wrap_arg + else: + # !!!! attention !!!! + # the basic c_types need some annotation information + # at the moment that are exactly the types that have + # no 'wrap_arg'. This might change in the future + the_type.compute_result_annotation = classmethod(lambda cls, s_arg:SomeCTypesObject(cls)) create_ctypes_annotations() @@ -82,7 +89,12 @@ answer.append(ctype_type.wrap_arg(ll_type, arg_name)) return answer +class CtypesBasicTypeInstantiationTranslation( FunctionPointerTranslation ): + compute_result_annotation = classmethod( + FunctionPointerTranslation.compute_result_annotation) + + class RStructureMeta(type(Structure)): def __new__(mta,name,bases,clsdict): _fields = clsdict.get('_fields_',None) Modified: pypy/dist/pypy/rpython/rctypes/interface.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/interface.py (original) +++ pypy/dist/pypy/rpython/rctypes/interface.py Thu Jan 26 12:55:16 2006 @@ -1,5 +1,8 @@ from ctypes import _DLLS -from implementation import RCDLL as CDLL, c_int, c_char_p, c_char, \ +from implementation import RCDLL as CDLL, c_int, c_char_p, \ + c_char, c_byte, c_ubyte, \ + c_short, c_ushort, c_uint,\ + c_long, c_ulong, c_longlong, c_ulonglong, c_float, c_double, \ RStructure as Structure, RByref as byref, RPOINTER as POINTER try: from implementation import RWinDLL as WinDLL Modified: pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py Thu Jan 26 12:55:16 2006 @@ -11,7 +11,10 @@ py.test.skip("this test needs ctypes installed") -from pypy.rpython.rctypes import cdll, c_char_p, c_int, c_char, POINTER, Structure, byref +from pypy.rpython.rctypes import cdll, c_char_p, c_int, c_char, \ + c_char, c_byte, c_ubyte, c_short, c_ushort, c_uint,\ + c_long, c_ulong, c_longlong, c_ulonglong, c_float, c_double, \ + POINTER, Structure, byref if sys.platform == 'win32': mylib = cdll.LoadLibrary('msvcrt.dll') elif sys.platform == 'linux2': @@ -78,6 +81,26 @@ res = testfunc_byval(inpoint,oppoint) return res, oppoint +def py_test_simple_cint(): + return c_int(10) + +def py_test_simple_ctypes(): + return ( + c_char('a'), + c_byte(1), + c_ubyte(1), + c_short(1), + c_ushort(1), + c_int(1), + c_uint(1), + c_long(1), + c_ulong(1), + c_longlong(1), + c_ulonglong(1), + c_float(1.0), + c_double(1.0) + ) + class Test_rctypes: @@ -166,3 +189,27 @@ assert s.items[0].knowntype == int assert s.items[1].knowntype == POINTER(tagpoint) + def test_annotate_simple_cint(self): + a = RPythonAnnotator() + s = a.build_types(py_test_simple_cint,[]) + assert s.knowntype == c_int + + def test_annotate_simple_types(self): + a = RPythonAnnotator() + s = a.build_types(py_test_simple_ctypes,[]) + assert s.knowntype == tuple + assert len(s.items) == 13 + assert s.items[0].knowntype == c_char + assert s.items[1].knowntype == c_byte + assert s.items[2].knowntype == c_ubyte + assert s.items[3].knowntype == c_short + assert s.items[4].knowntype == c_ushort + assert s.items[5].knowntype == c_int + assert s.items[6].knowntype == c_uint + assert s.items[7].knowntype == c_long + assert s.items[8].knowntype == c_ulong + assert s.items[9].knowntype == c_longlong + assert s.items[10].knowntype == c_ulonglong + assert s.items[11].knowntype == c_float + assert s.items[12].knowntype == c_double + From pedronis at codespeak.net Thu Jan 26 12:59:26 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Thu, 26 Jan 2006 12:59:26 +0100 (CET) Subject: [pypy-svn] r22678 - pypy/branch/jit-refactoring/pypy/jit Message-ID: <20060126115926.3352D27DB6@code1.codespeak.net> Author: pedronis Date: Thu Jan 26 12:59:23 2006 New Revision: 22678 Modified: pypy/branch/jit-refactoring/pypy/jit/llabstractinterp.py pypy/branch/jit-refactoring/pypy/jit/llcontainer.py pypy/branch/jit-refactoring/pypy/jit/llvalue.py pypy/branch/jit-refactoring/pypy/jit/vlist.py Log: use AConstant and AVariable for LLAbstractValue.runtimevar, these have a getgenvar(builder) to retrieve a stored corresponding rgenop API var/const opaque object, or to lazily comnpute such a const object in the context of the given builder. Modified: pypy/branch/jit-refactoring/pypy/jit/llabstractinterp.py ============================================================================== --- pypy/branch/jit-refactoring/pypy/jit/llabstractinterp.py (original) +++ pypy/branch/jit-refactoring/pypy/jit/llabstractinterp.py Thu Jan 26 12:59:23 2006 @@ -1,8 +1,8 @@ import operator -from pypy.objspace.flow.model import Variable, Constant, SpaceOperation +from pypy.objspace.flow.model import Variable, Constant from pypy.objspace.flow.model import checkgraph, c_last_exception from pypy.rpython.lltypesystem import lltype -from pypy.jit.llvalue import LLAbstractValue, const, aconst, AConstant, AVariable +from pypy.jit.llvalue import LLAbstractValue, AConstant, AVariable, const from pypy.jit.llvalue import ll_no_return_value from pypy.jit.llvalue import FlattenMemo, MatchMemo, FreezeMemo, UnfreezeMemo from pypy.jit.llcontainer import LLAbstractContainer, virtualcontainervalue @@ -51,10 +51,6 @@ def getruntimevars(self): assert not self.frozen, "getruntimevars(): not for frozen states" return [a.runtimevar for a in self.flatten()] - - def getgenvars(self, builder): - assert not self.frozen, "getgenvars(): not for frozen states" - return [a.getgenvar(builder) for a in self.flatten()] def flatten(self, memo=None): if memo is None: @@ -175,10 +171,10 @@ args_a = [] for i, v in enumerate(origgraph.getargs()): if i in arghints: - a = LLAbstractValue(aconst(arghints[i])) + a = LLAbstractValue(AConstant(arghints[i])) a.concrete = self.policy.concrete_args else: - a = LLAbstractValue(AVariable(), input=v.concretetype) + a = LLAbstractValue(AVariable(v.concretetype)) args_a.append(a) graphstate = self.schedule_graph(args_a, origgraph) graphstate.complete() @@ -199,8 +195,8 @@ def schedule(self, inputstate, builder): #print "SCHEDULE", args_a, origblock frozenstate = self.schedule_getstate(inputstate) - args_v = inputstate.getgenvars(builder) - return LinkState(args_v, frozenstate) + args_v = inputstate.getruntimevars() + return LinkState(builder, args_v, frozenstate) def schedule_getstate(self, inputstate): # NOTA BENE: copyblocks can get shared between different copygraphs! @@ -257,19 +253,21 @@ exitcase = None llexitcase = None - def __init__(self, args_v, frozenstate): + def __init__(self, builder, args_v, frozenstate): self.args_v = args_v + self.args_genv = [v.getgenvar(builder) for v in args_v] self.frozenstate = frozenstate self.link = None def setreturn(self): - rgenop.closereturnlink(self.link, self.args_v[0]) + rgenop.closereturnlink(self.link, self.args_genv[0]) def settarget(self, block, blockargs): args = [] - for v1, v2 in zip(self.args_v, blockargs): - if isinstance(v2, Constant): - assert v1 == v2 + for v1, v2 in zip(self.args_genv, blockargs): + assert isinstance(v2, (AVariable, AConstant)) + if isinstance(v2, AConstant): + assert v1.value == v2.value # sanity check violating encapsulation else: args.append(v1) rgenop.closelink(self.link, args, block) @@ -284,7 +282,7 @@ self.name = '%s_%d' % (origgraph.name, n) # The 'args_v' attribute is needed by try_to_complete(), # which looks for it on either a GraphState or a LinkState - self.args_v = inputstate.getgenvars(None) + self.args_v = inputstate.getruntimevars() self.frozenstate = frozenstate #self.a_return = None self.state = "before" @@ -439,7 +437,7 @@ # except block of the copygraph. args_v = [builder.binding(v).forcevarorconst(builder) for v in origblock.inputargs] - ls = LinkState(args_v, frozenstate=None) + ls = LinkState(builder, args_v, frozenstate=None) raise InsertNextLink(ls) else: # finishing a handle_call_inlining(): link back to @@ -481,7 +479,7 @@ else: a = builder.bindings[origblock.exitswitch] v = a.forcevarorconst(builder) - if isinstance(v, Variable): + if isinstance(v, AVariable): newexitswitch = v links = origblock.exits else: @@ -524,7 +522,7 @@ assert len(newlinkstates) == 2 - false_link, true_link = rgenop.closeblock2(b, newexitswitch) + false_link, true_link = rgenop.closeblock2(b, newexitswitch.getgenvar(self)) cases = {False: false_link, True: true_link} for ls in newlinkstates: ls.link = cases[ls.exitcase] @@ -537,8 +535,9 @@ return rgenop.genconst(self.newblock, llvalue) def binding(self, v): + assert isinstance(v, (Constant, Variable)) if isinstance(v, Constant): - return LLAbstractValue(aconst(v.value, v.concretetype)) + return LLAbstractValue(AConstant(v.value, v.concretetype)) else: return self.bindings[v] @@ -568,7 +567,7 @@ # can constant-fold print 'fold:', constant_op.__name__, concretevalues concreteresult = constant_op(*concretevalues) - a_result = LLAbstractValue(aconst(concreteresult)) + a_result = LLAbstractValue(AConstant(concreteresult)) if any_concrete and self.interp.policy.concrete_propagate: a_result.concrete = True else: @@ -576,14 +575,11 @@ return a_result def residual(self, op, args_a): + T= op.result.concretetype retvar = self.genop(op.opname, [a.forcegenvarorconst(self) for a in args_a], - op.result.concretetype) - return LLAbstractValue(retvar) - - def residual_direct_call(self, name, block, args_a,): - - return LLAbstractValue(retvar) + T) + return LLAbstractValue(AVariable(T, genvar=retvar)) def residualize(self, op, args_a, constant_op=None): if constant_op: @@ -757,10 +753,11 @@ new_args_a.append(a) args_a = new_args_a TYPE = lltype.FuncType(ARGS, op.result.concretetype) - a_func = LLAbstractValue(rgenop.gengraphconst(self.newblock, - graphstate.name, - graphstate.startblock, - TYPE)) + genconst = rgenop.gengraphconst(self.newblock, + graphstate.name, + graphstate.startblock, + TYPE) + a_func = LLAbstractValue(AConstant(genconst.value, TYPE, genvar=genconst)) # XXX const.value! return self.residual(op, [a_func] + args_a) def op_getfield(self, op, a_ptr, a_attrname): @@ -785,7 +782,7 @@ def op_getarraysize(self, op, a_ptr): if hasllcontent(a_ptr): - return LLAbstractValue(aconst(a_ptr.content.length)) + return LLAbstractValue(AConstant(a_ptr.content.length)) return self.residualize(op, [a_ptr], len) def op_getarrayitem(self, op, a_ptr, a_index): @@ -868,7 +865,7 @@ args_a = [] for a in argtuple: if not isinstance(a, LLAbstractValue): - a = LLAbstractValue(aconst(a)) + a = LLAbstractValue(AConstant(a)) args_a.append(a) # end of rather XXX'edly hackish parsing Modified: pypy/branch/jit-refactoring/pypy/jit/llcontainer.py ============================================================================== --- pypy/branch/jit-refactoring/pypy/jit/llcontainer.py (original) +++ pypy/branch/jit-refactoring/pypy/jit/llcontainer.py Thu Jan 26 12:59:23 2006 @@ -1,6 +1,5 @@ -from pypy.objspace.flow.model import Variable, Constant, SpaceOperation from pypy.rpython.lltypesystem import lltype -from pypy.jit.llvalue import LLAbstractValue, const +from pypy.jit.llvalue import LLAbstractValue, AConstant, const class LLAbstractContainer(object): @@ -53,7 +52,7 @@ RESULT_TYPE = lltype.Ptr(self.T) if self.a_parent is not None: parentindex = const(self.parentindex, lltype.Void) - v_parent = self.a_parent.forcevarorconst(builder) + v_parent = self.a_parent.forcegenvarorconst(builder) v_result = builder.genop('getsubstruct', [v_parent, parentindex], RESULT_TYPE) else: @@ -82,7 +81,7 @@ assert isinstance(a_value.content, LLVirtualContainer) a_value.content.buildcontent(builder, v_subptr) else: - v_value = a_value.forcevarorconst(builder) + v_value = a_value.forcegenvarorconst(builder) self.setop(builder, v_target, name, v_value) def flatten(self, memo): @@ -216,7 +215,7 @@ a.content.parentindex = parentindex else: # primitive initialized to zero - a = LLAbstractValue(aconst(T._defl())) + a = LLAbstractValue(AConstant(T._defl())) return a def hasllcontent(a_ptr): Modified: pypy/branch/jit-refactoring/pypy/jit/llvalue.py ============================================================================== --- pypy/branch/jit-refactoring/pypy/jit/llvalue.py (original) +++ pypy/branch/jit-refactoring/pypy/jit/llvalue.py Thu Jan 26 12:59:23 2006 @@ -1,18 +1,26 @@ -from pypy.objspace.flow.model import Variable, Constant +from pypy.objspace.flow.model import Constant from pypy.rpython.lltypesystem import lltype from pypy.rpython import rgenop from pypy.tool.uid import Hashable class AVariable(object): - def __init__(self, genvar=None): + def __init__(self, T, genvar=None): + self.concretetype = T self.genvar = genvar def getgenvar(self, builder): return self.genvar class AConstant(Hashable): - def getgenvar(self, v, builder): + def __init__(self, value, T=None, genvar=None): + Hashable.__init__(self, value) + self.concretetype = T or lltype.typeOf(value) + self.genvar = genvar + def getgenvar(self, builder): + if self.genvar is None: + self.genvar = builder.genconst(self.value) + return self.genvar class LLAbstractValue(object): """An abstract value, propagated through the blocks of the low-level @@ -27,17 +35,13 @@ concrete = False # concrete constants propagate eagerly - def __init__(self, runtimevar=None, genvar=None, content=None, input=None): - self.runtimevar = runtimevar # None or a Variable or a Constant - self.genvar = genvar + def __init__(self, runtimevar=None, content=None): + self.runtimevar = runtimevar # None or a AVariable or a AConstant self.content = content # None or an LLAbstractContainer self.origin = [] # list of frozen values: the sources that did or # could allow 'self' to be computed as a constant - self.input = input def __repr__(self): - if self.input: - return '' if self.runtimevar is None: if self.content is None: return '' @@ -46,7 +50,7 @@ else: # runtime value -- display more precisely if it's a # Variable or a Constant - if isinstance(self.runtimevar, Variable): + if isinstance(self.runtimevar, AVariable): kind = 'runtime' elif self.concrete: kind = 'concrete' @@ -56,8 +60,6 @@ def freeze(self, memo): # turn a run-time value into a frozen value - if self.input: - return LLFrozenRuntimeValue(self) if self.runtimevar is not None: if self.concrete: return LLFrozenConcreteValue(self) @@ -78,8 +80,6 @@ return frozen_dummy_value # dummy def getconcretetype(self): - if self.input: - return self.input if self.runtimevar is not None: return self.runtimevar.concretetype elif self.content is not None: @@ -91,23 +91,14 @@ if self.runtimevar is None: if self.content is None: raise ValueError("ll_dummy_value.forcevarorconst()") - XXX + self.content.build_runtime_container(builder) + genvar = self.content.build_runtime_container(builder) + assert self.content.T == genvar.concretetype.TO # sanity check + self.runtimevar = AVariable(lltype.Ptr(self.content.T), genvar=genvar) self.content = None return self.runtimevar def forcegenvarorconst(self, builder): - if self.runtimevar is None: - if self.content is None: - raise ValueError("ll_dummy_value.forcevarorconst()") - self.genvar = self.content.build_runtime_container(builder) - self.content = None - return self.getgenvar(builder) - - def getgenvar(self, builder): - if self.genvar is None: - assert isinstance(self.runtimevar, AConstant) - self.genvar = builder.genconst(self.runtimevar.value) - return self.genvar + return self.forcevarorconst(builder).getgenvar(builder) def maybe_get_constant(self): if isinstance(self.runtimevar, AConstant): @@ -119,9 +110,7 @@ """Recursively flatten the LLAbstractValue into a list of run-time LLAbstractValues. """ - if self.input: - memo.result.append(self) - elif self.runtimevar is not None: + if self.runtimevar is not None: if not self.concrete: # skip concrete values, they don't need # to be present in the residual graph at all memo.result.append(self) @@ -205,12 +194,12 @@ # never shared propagateconst = memo.propagateconsts.next() if isinstance(propagateconst, AConstant): - v = propagateconst # allowed to propagate as a Constant - assert v.concretetype == self.concretetype - result = LLAbstractValue(v) + c = propagateconst # allowed to propagate as a Constant + assert c.concretetype == self.concretetype + result = LLAbstractValue(c) else: - v = rgenop.geninputarg(block, self.concretetype) - result = LLAbstractValue(v, genvar=v) + gen_v = rgenop.geninputarg(block, self.concretetype) + result = LLAbstractValue(AVariable(self.concretetype, genvar=gen_v)) result.origin.append(self) return result @@ -285,11 +274,6 @@ c.concretetype = T or lltype.typeOf(value) return c -def aconst(value, T=None): - c = AConstant(value) - c.concretetype = T or lltype.typeOf(value) - return c - -ll_no_return_value = LLAbstractValue(aconst(None, lltype.Void)) +ll_no_return_value = LLAbstractValue(AConstant(None, lltype.Void)) ll_dummy_value = LLAbstractValue() frozen_dummy_value = LLFrozenDummyValue() Modified: pypy/branch/jit-refactoring/pypy/jit/vlist.py ============================================================================== --- pypy/branch/jit-refactoring/pypy/jit/vlist.py (original) +++ pypy/branch/jit-refactoring/pypy/jit/vlist.py Thu Jan 26 12:59:23 2006 @@ -1,6 +1,6 @@ from pypy.rpython.lltypesystem import lltype from pypy.rpython.rtyper import LowLevelOpList -from pypy.jit.llvalue import LLAbstractValue, aconst, ll_dummy_value +from pypy.jit.llvalue import LLAbstractValue, AConstant, ll_dummy_value from pypy.jit.llcontainer import LLAbstractContainer @@ -39,7 +39,7 @@ return LLVirtualList(self.T, items_a) def build_runtime_container(self, builder): - items_v = [a.forcevarorconst(builder) for a in self.items_a] + items_v = [a.forcegenvarorconst(builder) for a in self.items_a] v_result = self.T.list_builder(builder, items_v) return v_result @@ -47,10 +47,10 @@ # High-level operations def oop_len(self, op): - return LLAbstractValue(aconst(len(self.items_a))) + return LLAbstractValue(AConstant(len(self.items_a))) def oop_nonzero(self, op): - return LLAbstractValue(aconst(bool(self.items_a))) + return LLAbstractValue(AConstant(bool(self.items_a))) def oop_getitem(self, op, a_index): c_index = a_index.maybe_get_constant() From gromit at codespeak.net Thu Jan 26 13:10:18 2006 From: gromit at codespeak.net (gromit at codespeak.net) Date: Thu, 26 Jan 2006 13:10:18 +0100 (CET) Subject: [pypy-svn] r22679 - pypy/dist/pypy/rpython/rctypes/test Message-ID: <20060126121018.8FE9A27B70@code1.codespeak.net> Author: gromit Date: Thu Jan 26 13:10:17 2006 New Revision: 22679 Modified: pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py Log: ADD: Another test, that annotates a non constant ctypes' basic type. Modified: pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py Thu Jan 26 13:10:17 2006 @@ -101,6 +101,9 @@ c_double(1.0) ) +def py_test_simple_ctypes_non_const(): + a = 10 + return c_float( a + 10 ) class Test_rctypes: @@ -213,3 +216,8 @@ assert s.items[11].knowntype == c_float assert s.items[12].knowntype == c_double + def test_annotate_simple_types_non_const(self): + a = RPythonAnnotator() + s = a.build_types(py_test_simple_ctypes_non_const,[]) + assert s.knowntype == c_float + From rxe at codespeak.net Thu Jan 26 13:16:21 2006 From: rxe at codespeak.net (rxe at codespeak.net) Date: Thu, 26 Jan 2006 13:16:21 +0100 (CET) Subject: [pypy-svn] r22680 - in pypy/dist/pypy/rpython/memory: . test Message-ID: <20060126121621.6057F27B70@code1.codespeak.net> Author: rxe Date: Thu Jan 26 13:16:19 2006 New Revision: 22680 Added: pypy/dist/pypy/rpython/memory/test/test_lltypelayout.py (contents, props changed) Modified: pypy/dist/pypy/rpython/memory/lltypelayout.py Log: My money is on this is a bug. Added test. Please review! Modified: pypy/dist/pypy/rpython/memory/lltypelayout.py ============================================================================== --- pypy/dist/pypy/rpython/memory/lltypelayout.py (original) +++ pypy/dist/pypy/rpython/memory/lltypelayout.py Thu Jan 26 13:16:19 2006 @@ -139,5 +139,6 @@ return offsets_to_gc_pointers(TYPE.OF) return [] elif isinstance(TYPE, lltype.Struct): - return offsets_to_gc_pointers(getattr(TYPE, TYPE._arrayfld)) + return varsize_offsets_to_gcpointers_in_var_part(getattr(TYPE, + TYPE._arrayfld)) Added: pypy/dist/pypy/rpython/memory/test/test_lltypelayout.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/rpython/memory/test/test_lltypelayout.py Thu Jan 26 13:16:19 2006 @@ -0,0 +1,35 @@ +from pypy.rpython.memory.lltypelayout import offsets_to_gc_pointers, \ + varsize_offset_to_length, varsize_offsets_to_gcpointers_in_var_part +from pypy.rpython.lltypesystem import lltype + +def getname(T): + try: + return "field:" + T._name + except: + return "field:" + T.__name__ + +S = lltype.Struct('S', ('s', lltype.Signed), ('char', lltype.Char)) +GC_S = lltype.GcStruct('GC_S', ('S', S)) + +A = lltype.Array(S) +GC_A = lltype.GcArray(S) + +S2 = lltype.Struct('SPTRS', + *[(getname(TYPE), lltype.Ptr(TYPE)) for TYPE in (GC_S, GC_A)]) +GC_S2 = lltype.GcStruct('GC_S2', ('S2', S2)) + +A2 = lltype.Array(S2) +GC_A2 = lltype.GcArray(S2) + +l = [(getname(TYPE), lltype.Ptr(TYPE)) for TYPE in (GC_S, GC_A)] +l.append(('vararray', A2)) + +GC_S3 = lltype.GcStruct('GC_S3', *l) + +def test_struct(): + for T, c in [(GC_S, 0), (GC_S2, 2), (GC_A, 0), (GC_A2, 0), (GC_S3, 2)]: + assert len(offsets_to_gc_pointers(T)) == c + + for T1, T2 in [(GC_A, GC_S), (GC_A2, GC_S2), (GC_S3, GC_S2)]: + assert (len(varsize_offsets_to_gcpointers_in_var_part(T1)) == + len(offsets_to_gc_pointers(T2))) From pedronis at codespeak.net Thu Jan 26 13:55:07 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Thu, 26 Jan 2006 13:55:07 +0100 (CET) Subject: [pypy-svn] r22681 - pypy/branch/jit-refactoring/pypy/rpython/test Message-ID: <20060126125507.3865927B70@code1.codespeak.net> Author: pedronis Date: Thu Jan 26 13:55:05 2006 New Revision: 22681 Modified: pypy/branch/jit-refactoring/pypy/rpython/test/test_rlist.py Log: fix test that was broken by changes. Modified: pypy/branch/jit-refactoring/pypy/rpython/test/test_rlist.py ============================================================================== --- pypy/branch/jit-refactoring/pypy/rpython/test/test_rlist.py (original) +++ pypy/branch/jit-refactoring/pypy/rpython/test/test_rlist.py Thu Jan 26 13:55:05 2006 @@ -902,12 +902,33 @@ lst.append(42) return lst + from pypy.rpython import rgenop + + class DummyBlockBuilder: + + def __init__(self): + self.newblock = rgenop.newblock() + + def genop(self, opname, args, RESULT_TYPE): + return rgenop.genop(self.newblock, opname, args, RESULT_TYPE) + + def genconst(self, llvalue): + return rgenop.genconst(self.newblock, llvalue) + + # inspection + def __getitem__(self, index): + return self.newblock.operations[index] + + def __len__(self): + return len(self.newblock.operations) + + for fn in [fixed_size_case, variable_size_case]: t = TranslationContext() t.buildannotator().build_types(fn, []) t.buildrtyper().specialize() LIST = t.graphs[0].getreturnvar().concretetype.TO - llop = LowLevelOpList(None) + llop = DummyBlockBuilder() v0 = Constant(42) v0.concretetype = Signed v1 = Variable() From pedronis at codespeak.net Thu Jan 26 14:02:50 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Thu, 26 Jan 2006 14:02:50 +0100 (CET) Subject: [pypy-svn] r22682 - pypy/branch/jit-refactoring/pypy/rpython/test Message-ID: <20060126130250.2BD0027B70@code1.codespeak.net> Author: pedronis Date: Thu Jan 26 14:02:48 2006 New Revision: 22682 Modified: pypy/branch/jit-refactoring/pypy/rpython/test/test_rlist.py Log: remove unncessary import Modified: pypy/branch/jit-refactoring/pypy/rpython/test/test_rlist.py ============================================================================== --- pypy/branch/jit-refactoring/pypy/rpython/test/test_rlist.py (original) +++ pypy/branch/jit-refactoring/pypy/rpython/test/test_rlist.py Thu Jan 26 14:02:48 2006 @@ -1,6 +1,5 @@ import sys from pypy.rpython.lltypesystem.lltype import * -from pypy.rpython.rtyper import LowLevelOpList from pypy.rpython.rlist import * from pypy.rpython.rslice import ll_newslice from pypy.rpython.rint import signed_repr From pedronis at codespeak.net Thu Jan 26 14:07:03 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Thu, 26 Jan 2006 14:07:03 +0100 (CET) Subject: [pypy-svn] r22683 - pypy/branch/jit-refactoring/pypy/jit Message-ID: <20060126130703.835D627B70@code1.codespeak.net> Author: pedronis Date: Thu Jan 26 14:07:02 2006 New Revision: 22683 Modified: pypy/branch/jit-refactoring/pypy/jit/llabstractinterp.py Log: refactor to avoid accessing rgenop opaque constant .value! Modified: pypy/branch/jit-refactoring/pypy/jit/llabstractinterp.py ============================================================================== --- pypy/branch/jit-refactoring/pypy/jit/llabstractinterp.py (original) +++ pypy/branch/jit-refactoring/pypy/jit/llabstractinterp.py Thu Jan 26 14:07:02 2006 @@ -580,6 +580,18 @@ [a.forcegenvarorconst(self) for a in args_a], T) return LLAbstractValue(AVariable(T, genvar=retvar)) + + def residual_direct_call(self, FUNCTYPE, name, target, args_a): + T = FUNCTYPE.RESULT + gen_fn_const = rgenop.gengraphconst(self.newblock, + name, + target, + FUNCTYPE) + retvar = self.genop('direct_call', [gen_fn_const] + + [a.forcegenvarorconst(self) for a in args_a], + T) + return LLAbstractValue(AVariable(T, genvar=retvar)) + def residualize(self, op, args_a, constant_op=None): if constant_op: @@ -751,14 +763,9 @@ if not a.concrete: ARGS.append(a.getconcretetype()) new_args_a.append(a) - args_a = new_args_a TYPE = lltype.FuncType(ARGS, op.result.concretetype) - genconst = rgenop.gengraphconst(self.newblock, - graphstate.name, - graphstate.startblock, - TYPE) - a_func = LLAbstractValue(AConstant(genconst.value, TYPE, genvar=genconst)) # XXX const.value! - return self.residual(op, [a_func] + args_a) + return self.residual_direct_call(TYPE, graphstate.name, + graphstate.startblock, new_args_a) def op_getfield(self, op, a_ptr, a_attrname): if hasllcontent(a_ptr): From pedronis at codespeak.net Thu Jan 26 14:12:21 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Thu, 26 Jan 2006 14:12:21 +0100 (CET) Subject: [pypy-svn] r22684 - in pypy/branch/jit-refactoring/pypy: jit rpython Message-ID: <20060126131221.5E0FE27B4E@code1.codespeak.net> Author: pedronis Date: Thu Jan 26 14:12:19 2006 New Revision: 22684 Modified: pypy/branch/jit-refactoring/pypy/jit/llabstractinterp.py pypy/branch/jit-refactoring/pypy/rpython/rgenop.py Log: more consistent naming: gengraphconst -> gencallableconst. rgenop API doesn't expose the notion of a graph. Modified: pypy/branch/jit-refactoring/pypy/jit/llabstractinterp.py ============================================================================== --- pypy/branch/jit-refactoring/pypy/jit/llabstractinterp.py (original) +++ pypy/branch/jit-refactoring/pypy/jit/llabstractinterp.py Thu Jan 26 14:12:19 2006 @@ -583,10 +583,10 @@ def residual_direct_call(self, FUNCTYPE, name, target, args_a): T = FUNCTYPE.RESULT - gen_fn_const = rgenop.gengraphconst(self.newblock, - name, - target, - FUNCTYPE) + gen_fn_const = rgenop.gencallableconst(self.newblock, + name, + target, + FUNCTYPE) retvar = self.genop('direct_call', [gen_fn_const] + [a.forcegenvarorconst(self) for a in args_a], T) Modified: pypy/branch/jit-refactoring/pypy/rpython/rgenop.py ============================================================================== --- pypy/branch/jit-refactoring/pypy/rpython/rgenop.py (original) +++ pypy/branch/jit-refactoring/pypy/rpython/rgenop.py Thu Jan 26 14:12:19 2006 @@ -28,7 +28,7 @@ block.operations.append(op) return v -def gengraphconst(block, name, target, FUNCTYPE): +def gencallableconst(block, name, target, FUNCTYPE): fptr = lltype.functionptr(FUNCTYPE, name, graph=buildgraph(target)) return genconst(block, fptr) From pedronis at codespeak.net Thu Jan 26 14:17:37 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Thu, 26 Jan 2006 14:17:37 +0100 (CET) Subject: [pypy-svn] r22685 - pypy/branch/jit-refactoring/pypy/doc/discussion Message-ID: <20060126131737.8756427B4E@code1.codespeak.net> Author: pedronis Date: Thu Jan 26 14:17:36 2006 New Revision: 22685 Modified: pypy/branch/jit-refactoring/pypy/doc/discussion/draft-jit-ideas.txt Log: addconst seems not necessary, for now for Void constant we simply constuct Constant directly. Document gencallableconst. Modified: pypy/branch/jit-refactoring/pypy/doc/discussion/draft-jit-ideas.txt ============================================================================== --- pypy/branch/jit-refactoring/pypy/doc/discussion/draft-jit-ideas.txt (original) +++ pypy/branch/jit-refactoring/pypy/doc/discussion/draft-jit-ideas.txt Thu Jan 26 14:17:36 2006 @@ -100,7 +100,7 @@ * genconst(block, llvalue) -> (result var) -* addconst(block, const) -> (result var) +* gencallableconst(block, name, target-block, FUNCTYPE) -> (result var) * closeblock1(block) -> link From tismer at codespeak.net Thu Jan 26 15:20:01 2006 From: tismer at codespeak.net (tismer at codespeak.net) Date: Thu, 26 Jan 2006 15:20:01 +0100 (CET) Subject: [pypy-svn] r22686 - pypy/dist/pypy/module/stackless Message-ID: <20060126142001.0DCEE27DCC@code1.codespeak.net> Author: tismer Date: Thu Jan 26 15:19:59 2006 New Revision: 22686 Added: pypy/dist/pypy/module/stackless/stackless_flags.py (contents, props changed) Modified: pypy/dist/pypy/module/stackless/interp_coroutine.py Log: added a compatible flags structure to support Stackless in some future Modified: pypy/dist/pypy/module/stackless/interp_coroutine.py ============================================================================== --- pypy/dist/pypy/module/stackless/interp_coroutine.py (original) +++ pypy/dist/pypy/module/stackless/interp_coroutine.py Thu Jan 26 15:19:59 2006 @@ -7,6 +7,7 @@ from pypy.rpython.rarithmetic import intmask from pypy.rpython.rstack import yield_current_frame_to_caller +from pypy.module.stackless.stackless_flags import StacklessFlags import sys @@ -138,7 +139,11 @@ def call(self): self.space.call_args(self.w_func, self.args) -class AppCoroutine(Coroutine): +class AppCoroutine(Coroutine, StacklessFlags): + + def __init__(self): + Coroutine.__init__(self) + self.flags = 0 def descr_method__new__(space, w_subtype): co = space.allocate_instance(AppCoroutine, w_subtype) Added: pypy/dist/pypy/module/stackless/stackless_flags.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/module/stackless/stackless_flags.py Thu Jan 26 15:19:59 2006 @@ -0,0 +1,201 @@ +""" +basic definitions for tasklet flags. +For simplicity and compatibility, +they are defined the same for coroutines, +even if they are not used. + +taken from tasklet_structs.h +---------------------------- + +/*************************************************************************** + + Tasklet Flag Definition + ----------------------- + + blocked: The tasklet is either waiting in a channel for + writing (1) or reading (-1) or not blocked (0). + Maintained by the channel logic. Do not change. + + atomic: If true, schedulers will never switch. Driven by + the code object or dynamically, see below. + + ignore_nesting: Allows auto-scheduling, even if nesting_level + is not zero. + + autoschedule: The tasklet likes to be auto-scheduled. User driven. + + block_trap: Debugging aid. Whenever the tasklet would be + blocked by a channel, an exception is raised. + + is_zombie: This tasklet is almost dead, its deallocation has + started. The tasklet *must* die at some time, or the + process can never end. + + pending_irq: If set, an interrupt was issued during an atomic + operation, and should be handled when possible. + + + Policy for atomic/autoschedule and switching: + --------------------------------------------- + A tasklet switch can always be done explicitly by calling schedule(). + Atomic and schedule are concerned with automatic features. + + atomic autoschedule + + 1 any Neither a scheduler nor a watchdog will + try to switch this tasklet. + + 0 0 The tasklet can be stopped on desire, or it + can be killed by an exception. + + 0 1 Like above, plus auto-scheduling is enabled. + + Default settings: + ----------------- + All flags are zero by default. + + ***************************************************************************/ + +typedef struct _tasklet_flags { + int blocked: 2; + unsigned int atomic: 1; + unsigned int ignore_nesting: 1; + unsigned int autoschedule: 1; + unsigned int block_trap: 1; + unsigned int is_zombie: 1; + unsigned int pending_irq: 1; +} PyTaskletFlagStruc; +""" + +from pypy.rpython.rarithmetic import LONG_BIT, intmask + +class BitSetDef(object): + __slots__ = "_names __dict__ _attrname".split() + + def __init__(self, _attrname): + self._names = [] + self._attrname = _attrname + + def __setattr__(self, key, value): + if key not in self.__slots__: + assert key not in self.__dict__ + self._names.append(key) + object.__setattr__(self, key, value) + + def __iter__(self): + return self._enum_objects() + + def _enum_objects(self): + for name in self._names: + yield name, getattr(self, name) + +# negative values are user-writable +flags = BitSetDef("flags") +flags.blocked = 2, """writing (1) or reading (-1) or not blocked (0)""" +flags.atomic = -1, """If true, schedulers will never switch""" +flags.ignore_nesting = -1, """allow auto-scheduling in nested interpreters""" +flags.autoschedule = -1, """enable auto-scheduling""" +flags.block_trap = -1, """raise an exception instead of blocking""" +flags.is_zombie = 1, """__del__ is in progress""" +flags.pending_irq = 1, """an interrupt occured while being atomic""" + +def make_get_bits(name, bits, shift): + """ return a bool for single bits, signed int otherwise """ + signmask = 1 << (bits - 1 + shift) + lshift = bits + shift + rshift = bits + if bits == 1: + return "bool(%s & 0x%x)" % (name, signmask) + else: + return "intmask(%s << (LONG_BIT-%d)) >> (LONG_BIT-%d)" % (name, lshift, rshift) + +def make_set_bits(name, bits, shift): + datamask = int('1' * bits, 2) + clearmask = datamask << shift + return "%s & ~0x%x | (value & 0x%x) << %d" % (name, clearmask, datamask, shift) + +def gen_code(): + from cStringIO import StringIO + f = StringIO() + print >> f, "class StacklessFlags(object):" + print >> f, " _mixin_ = True" + shift = 0 + field = "self.%s" % flags._attrname + for name, (bits, doc) in flags: + write, bits = bits < 0, abs(bits) + print >> f + print >> f, ' def get_%s(self):' % name + print >> f, ' """%s"""' % doc + print >> f, ' return %s' % make_get_bits(field, bits, shift) + print >> f, ' def set_%s(self, value):' % name + print >> f, ' """%s"""' % doc + print >> f, ' %s = %s' % (field, make_set_bits(field, bits, shift)) + print >> f, ' set_%s._public = %s' % (name, write) + shift += bits + return f.getvalue() + +# BEGIN generated code +class StacklessFlags(object): + _mixin_ = True + + def get_blocked(self): + """writing (1) or reading (-1) or not blocked (0)""" + return intmask(self.flags << (LONG_BIT-2)) >> (LONG_BIT-2) + def set_blocked(self, value): + """writing (1) or reading (-1) or not blocked (0)""" + self.flags = self.flags & ~0x3 | (value & 0x3) << 0 + set_blocked._public = False + + def get_atomic(self): + """If true, schedulers will never switch""" + return bool(self.flags & 0x4) + def set_atomic(self, value): + """If true, schedulers will never switch""" + self.flags = self.flags & ~0x4 | (value & 0x1) << 2 + set_atomic._public = True + + def get_ignore_nesting(self): + """allow auto-scheduling in nested interpreters""" + return bool(self.flags & 0x8) + def set_ignore_nesting(self, value): + """allow auto-scheduling in nested interpreters""" + self.flags = self.flags & ~0x8 | (value & 0x1) << 3 + set_ignore_nesting._public = True + + def get_autoschedule(self): + """enable auto-scheduling""" + return bool(self.flags & 0x10) + def set_autoschedule(self, value): + """enable auto-scheduling""" + self.flags = self.flags & ~0x10 | (value & 0x1) << 4 + set_autoschedule._public = True + + def get_block_trap(self): + """raise an exception instead of blocking""" + return bool(self.flags & 0x20) + def set_block_trap(self, value): + """raise an exception instead of blocking""" + self.flags = self.flags & ~0x20 | (value & 0x1) << 5 + set_block_trap._public = True + + def get_is_zombie(self): + """__del__ is in progress""" + return bool(self.flags & 0x40) + def set_is_zombie(self, value): + """__del__ is in progress""" + self.flags = self.flags & ~0x40 | (value & 0x1) << 6 + set_is_zombie._public = False + + def get_pending_irq(self): + """an interrupt occured while being atomic""" + return bool(self.flags & 0x80) + def set_pending_irq(self, value): + """an interrupt occured while being atomic""" + self.flags = self.flags & ~0x80 | (value & 0x1) << 7 + set_pending_irq._public = False + +# END generated code + +if __name__ == '__main__': + # paste this into the file + print gen_code() \ No newline at end of file From auc at codespeak.net Thu Jan 26 15:48:39 2006 From: auc at codespeak.net (auc at codespeak.net) Date: Thu, 26 Jan 2006 15:48:39 +0100 (CET) Subject: [pypy-svn] r22687 - pypy/dist/pypy/lib/logic/store_without_cs Message-ID: <20060126144839.CBED527DC8@code1.codespeak.net> Author: auc Date: Thu Jan 26 15:48:35 2006 New Revision: 22687 Added: pypy/dist/pypy/lib/logic/store_without_cs/ pypy/dist/pypy/lib/logic/store_without_cs/constraint.py pypy/dist/pypy/lib/logic/store_without_cs/test_unification.py pypy/dist/pypy/lib/logic/store_without_cs/test_variable.py pypy/dist/pypy/lib/logic/store_without_cs/unification.py pypy/dist/pypy/lib/logic/store_without_cs/variable.py Log: (ale, auc) * create a 'branch' with current version of store/variables/constraints which are not (almost not) aware of computation spaces * this will allow us to try a merge of computation spaces / store Added: pypy/dist/pypy/lib/logic/store_without_cs/constraint.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/lib/logic/store_without_cs/constraint.py Thu Jan 26 15:48:35 2006 @@ -0,0 +1,389 @@ +# a) new requirement : be able to postpone asking fo the +# values of the domain + +#-- Exceptions --------------------------------------- + +class ConsistencyFailure(Exception): + """The repository is not in a consistent state""" + pass + +class DomainlessVariables(Exception): + """A constraint can't be defined on variables + without a domain""" + pass + +#-- Domains -------------------------------------------- + +class AbstractDomain(object): + """Implements the functionnality related to the changed flag. + Can be used as a starting point for concrete domains""" + + #__implements__ = DomainInterface + def __init__(self): + self.__changed = 0 + + def reset_flags(self): + self.__changed = 0 + + def has_changed(self): + return self.__changed + + def _value_removed(self): + """The implementation of remove_value should call this method""" + self.__changed = 1 + if self.size() == 0: + raise ConsistencyFailure() + + +class FiniteDomain(AbstractDomain): + """ + Variable Domain with a finite set of possible values + """ + + _copy_count = 0 + _write_count = 0 + + def __init__(self, values): + """values is a list of values in the domain + This class uses a dictionnary to make sure that there are + no duplicate values""" + AbstractDomain.__init__(self) + if isinstance(values,FiniteDomain): + # do a copy on write + self._cow = True + values._cow = True + FiniteDomain._copy_count += 1 + self._values = values._values + else: + # don't check this there (a) + #assert len(values) > 0 + self.set_values(values) + + ##self.getValues = self._values.keys + + def set_values(self, values): + self._cow = False + FiniteDomain._write_count += 1 + self._values = set(values) + + def remove_value(self, value): + """Remove value of domain and check for consistency""" +## print "removing", value, "from", self._values.keys() + if self._cow: + self.set_values(self._values) + del self._values[value] + self._value_removed() + + def remove_values(self, values): + """Remove values of domain and check for consistency""" + if self._cow: + self.set_values(self._values) + if values: +## print "removing", values, "from", self._values.keys() + for val in values : + self._values.remove(val) + self._value_removed() + __delitem__ = remove_value + + def size(self): + """computes the size of a finite domain""" + return len(self._values) + __len__ = size + + def get_values(self): + """return all the values in the domain + in an indexable sequence""" + return list(self._values) + + def __iter__(self): + return iter(self._values) + + def copy(self): + """clone the domain""" + return FiniteDomain(self) + + def __repr__(self): + return '' % str(self.get_values()) + + def __eq__(self, other): + if other is None: return False + return self._values == other._values + + def intersection(self, other): + if other is None: return self.get_values() + return self._values & other._values + +#-- Constraints ------------------------------------------ + +class AbstractConstraint(object): + + def __init__(self, variables): + """variables is a list of variables which appear in the formula""" + self._names_to_vars = {} + for var in variables: + if var.dom is None: + raise DomainlessVariables + self._names_to_vars[var.name] = var + var.add_constraint(self) + self._variables = variables + + def affectedVariables(self): + """ Return a list of all variables affected by this constraint """ + return self._variables + + def isVariableRelevant(self, variable): + return variable in self._variables + + def estimateCost(self, domains): + """Return an estimate of the cost of the narrowing of the constraint""" + return reduce(operator.mul, + [domains[var].size() for var in self._variables]) + + +class BasicConstraint(object): + """A BasicConstraint, which is never queued by the Repository + A BasicConstraint affects only one variable, and will be entailed + on the first call to narrow()""" + + def __init__(self, variable, reference, operator): + """variables is a list of variables on which + the constraint is applied""" + self._variable = variable + self._reference = reference + self._operator = operator + + def __repr__(self): + return '<%s %s %s>'% (self.__class__, self._variable, self._reference) + + def isVariableRelevant(self, variable): + return variable == self._variable + + def estimateCost(self, domains): + return 0 # get in the first place in the queue + + def affectedVariables(self): + return [self._variable] + + def getVariable(self): + return self._variable + + def narrow(self, domains): + domain = domains[self._variable] + operator = self._operator + ref = self._reference + try: + for val in domain.get_values() : + if not operator(val, ref) : + domain.remove_value(val) + except ConsistencyFailure: + raise ConsistencyFailure('inconsistency while applying %s' % \ + repr(self)) + return 1 + + +def make_lambda_head(vars): + var_ids = ','.join([var.name for var in vars]) + return 'lambda ' + var_ids + ':' + +def expand_expr_template(expr, vars): + for var in vars: + expr.replace(var.name, var.name + '.val') + return expr + +class Expression(AbstractConstraint): + """A constraint represented as a python expression.""" + _FILTER_CACHE = {} + + def __init__(self, variables, formula, type='fd.Expression'): + """variables is a list of variables which appear in the formula + formula is a python expression that will be evaluated as a boolean""" + AbstractConstraint.__init__(self, variables) + self.formula = formula + self.type = type + try: + self.filterFunc = Expression._FILTER_CACHE[formula] + except KeyError: + self.filterFunc = eval(make_lambda_head(variables) \ + + expand_expr_template(formula, variables), {}, {}) + Expression._FILTER_CACHE[formula] = self.filterFunc + + def _init_result_cache(self): + """key = (variable,value), value = [has_success,has_failure]""" + result_cache = {} + for var_name in self._variables: + result_cache[var_name.name] = {} + return result_cache + + + def _assign_values(self): + variables = [] + kwargs = {} + for variable in self._variables: + domain = variable.dom + values = domain.get_values() + variables.append((domain.size(), [variable, values, 0, len(values)])) + kwargs[variable.name] = values[0] + # sort variables to instanciate those with fewer possible values first + variables.sort() + + go_on = 1 + while go_on: + yield kwargs + # try to instanciate the next variable + for size, curr in variables: + if (curr[2] + 1) < curr[-1]: + curr[2] += 1 + kwargs[curr[0].name] = curr[1][curr[2]] + break + else: + curr[2] = 0 + kwargs[curr[0].name] = curr[1][0] + else: + # it's over + go_on = 0 + + + def narrow(self): + # removed domain arg. (auc, ale) + """generic narrowing algorithm for n-ary expressions""" + maybe_entailed = 1 + ffunc = self.filterFunc + result_cache = self._init_result_cache() + for kwargs in self._assign_values(): + if maybe_entailed: + for var, val in kwargs.iteritems(): + if val not in result_cache[var]: + break + else: + continue + if ffunc(**kwargs): + for var, val in kwargs.items(): + result_cache[var][val] = 1 + else: + maybe_entailed = 0 + + try: + for var, keep in result_cache.iteritems(): + domain = self._names_to_vars[var].dom + domain.remove_values([val for val in domain if val not in keep]) + + except ConsistencyFailure: + raise ConsistencyFailure('Inconsistency while applying %s' % \ + repr(self)) + except KeyError: + # There are no more value in result_cache + pass + + return maybe_entailed + + def __repr__(self): + return '<%s "%s">' % (self.type, self.formula) + +class BinaryExpression(Expression): + """A binary constraint represented as a python expression + + This implementation uses a narrowing algorithm optimized for + binary constraints.""" + + def __init__(self, variables, formula, type = 'fd.BinaryExpression'): + assert len(variables) == 2 + Expression.__init__(self, variables, formula, type) + + def narrow(self, domains): + """specialized narrowing algorithm for binary expressions + Runs much faster than the generic version""" + maybe_entailed = 1 + var1 = self._variables[0] + dom1 = domains[var1] + values1 = dom1.get_values() + var2 = self._variables[1] + dom2 = domains[var2] + values2 = dom2.get_values() + ffunc = self.filterFunc + if dom2.size() < dom1.size(): + var1, var2 = var2, var1 + dom1, dom2 = dom2, dom1 + values1, values2 = values2, values1 + + kwargs = {} + keep1 = {} + keep2 = {} + maybe_entailed = 1 + try: + # iterate for all values + for val1 in values1: + kwargs[var1] = val1 + for val2 in values2: + kwargs[var2] = val2 + if val1 in keep1 and val2 in keep2 and maybe_entailed == 0: + continue + if ffunc(**kwargs): + keep1[val1] = 1 + keep2[val2] = 1 + else: + maybe_entailed = 0 + + dom1.remove_values([val for val in values1 if val not in keep1]) + dom2.remove_values([val for val in values2 if val not in keep2]) + + except ConsistencyFailure: + raise ConsistencyFailure('Inconsistency while applying %s' % \ + repr(self)) + except Exception: + print self, kwargs + raise + return maybe_entailed + + +def make_expression(variables, formula, constraint_type=None): + """create a new constraint of type Expression or BinaryExpression + The chosen class depends on the number of variables in the constraint""" + # encode unicode + vars = [] + for var in variables: + if type(var) == type(u''): + vars.append(var.encode()) + else: + vars.append(var) + if len(vars) == 2: + if constraint_type is not None: + return BinaryExpression(vars, formula, constraint_type) + else: + return BinaryExpression(vars, formula) + + else: + if constraint_type is not None: + return Expression(vars, formula, constraint_type) + else: + return Expression(vars, formula) + + +class Equals(BasicConstraint): + """A basic constraint variable == constant value""" + def __init__(self, variable, reference): + BasicConstraint.__init__(self, variable, reference, operator.eq) + +class NotEquals(BasicConstraint): + """A basic constraint variable != constant value""" + def __init__(self, variable, reference): + BasicConstraint.__init__(self, variable, reference, operator.ne) + +class LesserThan(BasicConstraint): + """A basic constraint variable < constant value""" + def __init__(self, variable, reference): + BasicConstraint.__init__(self, variable, reference, operator.lt) + +class LesserOrEqual(BasicConstraint): + """A basic constraint variable <= constant value""" + def __init__(self, variable, reference): + BasicConstraint.__init__(self, variable, reference, operator.le) + +class GreaterThan(BasicConstraint): + """A basic constraint variable > constant value""" + def __init__(self, variable, reference): + BasicConstraint.__init__(self, variable, reference, operator.gt) + +class GreaterOrEqual(BasicConstraint): + """A basic constraint variable >= constant value""" + def __init__(self, variable, reference): + BasicConstraint.__init__(self, variable, reference, operator.ge) Added: pypy/dist/pypy/lib/logic/store_without_cs/test_unification.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/lib/logic/store_without_cs/test_unification.py Thu Jan 26 15:48:35 2006 @@ -0,0 +1,376 @@ +import unification as u +import variable as v +import constraint as c +from py.test import raises, skip +from threading import Thread + +class FunThread(Thread): + + def __init__(self, fun, *args): + Thread.__init__(self) + self.fun = fun + self.args = args + + def run(self): + self.fun(self, *self.args) + +class TestUnification: + + def setup_method(self, meth): + u._store = u.Store() + + def test_already_in_store(self): + x = u.var('x') + raises(v.AlreadyInStore, u.var, 'x') + + def test_already_bound(self): + x = u.var('x') + u.bind(x, 42) + raises(u.AlreadyBound, u.bind, x, 42) + + def test_bind_var_var(self): + x = u.var('x') + y = u.var('y') + z = u.var('z') + u.bind(x, z) + assert x.val == u.EqSet([x, z]) + assert y.val == u.EqSet([y]) + assert z.val == u.EqSet([x, z]) + + def test_bind_var_val(self): + x, y, z = u.var('x'), u.var('y'), u.var('z') + u.bind(x, z) + u.bind(y, 42) + u.bind(z, 3.14) + assert x.val == 3.14 + assert y.val == 42 + assert z.val == 3.14 + + def test_unify_same(self): + x,y,z,w = (u.var('x'), u.var('y'), + u.var('z'), u.var('w')) + u.bind(x, [42, z]) + u.bind(y, [z, 42]) + u.bind(w, [z, 43]) + raises(u.UnificationFailure, u.unify, x, w) + u.unify(x, y) + assert z.val == 42 + + def test_double_unification(self): + x, y, z = (u.var('x'), u.var('y'), + u.var('z')) + u.bind(x, 42) + u.bind(y, z) + u.unify(x, y) + assert z.val == 42 + u.unify(x, y) + assert (z.val == x.val) and (x.val == y.val) + + + def test_unify_values(self): + x, y = u.var('x'), u.var('y') + u.bind(x, [1, 2, 3]) + u.bind(y, [1, 2, 3]) + u.unify(x, y) + assert x.val == [1, 2, 3] + assert y.val == [1, 2, 3] + + def test_unify_lists_success(self): + x,y,z,w = (u.var('x'), u.var('y'), + u.var('z'), u.var('w')) + u.bind(x, [42, z]) + u.bind(y, [w, 44]) + u.unify(x, y) + assert x.val == [42, z] + assert y.val == [w, 44] + assert z.val == 44 + assert w.val == 42 + + def test_unify_dicts_success(self): + x,y,z,w = (u.var('x'), u.var('y'), + u.var('z'), u.var('w')) + u.bind(x, {1:42, 2:z}) + u.bind(y, {1:w, 2:44}) + u.unify(x, y) + assert x.val == {1:42, 2:z} + assert y.val == {1:w, 2:44} + assert z.val == 44 + assert w.val == 42 + + def test_unify_failure(self): + x,y,z = u.var('x'), u.var('y'), u.var('z') + u.bind(x, [42, z]) + u.bind(y, [z, 44]) + raises(u.UnificationFailure, u.unify, x, y) + # check store consistency + assert x.val == [42, z] + assert y.val == [z, 44] + assert z.val == u.EqSet([z]) + + def test_unify_failure2(self): + x,y,z,w = (u.var('x'), u.var('y'), + u.var('z'), u.var('w')) + u.bind(x, [42, z]) + u.bind(y, [w, 44]) + u.bind(z, w) + assert u._store.in_transaction == False + raises(u.UnificationFailure, u.unify, x, y) + assert u._store.in_transaction == False + # check store consistency + assert x.val == [42, z] + assert y.val == [w, 44] + assert z.val == u.EqSet([z,w]) + assert w.val == u.EqSet([z,w]) + + def test_unify_circular(self): + x, y, z, w, a, b = (u.var('x'), u.var('y'), + u.var('z'), u.var('w'), + u.var('a'), u.var('b')) + u.bind(x, [y]) + u.bind(y, [x]) + raises(u.UnificationFailure, u.unify, x, y) + u.bind(z, [1, w]) + u.bind(w, [z, 2]) + raises(u.UnificationFailure, u.unify, z, w) + u.bind(a, {1:42, 2:b}) + u.bind(b, {1:a, 2:42}) + raises(u.UnificationFailure, u.unify, a, b) + # check store consistency + assert x.val == [y] + assert y.val == [x] + assert z.val == [1, w] + assert w.val == [z, 2] + assert a.val == {1:42, 2:b} + assert b.val == {1:a, 2:42} + + + def test_threads_creating_vars(self): + def create_var(thread, *args): + x = u.var('x') + + def create_var2(thread, *args): + raises(v.AlreadyExists, u.var, 'x') + + t1, t2 = (FunThread(create_var), + FunThread(create_var2)) + t1.start() + t2.start() + + + def test_threads_binding_vars(self): + + def do_stuff(thread, var, val): + thread.raised = False + try: + u.bind(var, val) + except u.AlreadyBound: + thread.raised = True + + x = u.var('x') + vars_ = [] + for nvar in range(1000): + v = u.var('x-'+str(nvar)) + u.bind(x, v) + vars_.append(v) + + for var in u._store.vars: + assert var.val == x.val + + t1, t2 = (FunThread(do_stuff, x, 42), + FunThread(do_stuff, x, 43)) + t1.start() + t2.start() + t1.join() + t2.join() + #check that every var is really bound to 42 or 43 + for var in u._store.vars: + assert var.val == x.val + assert (t2.raised and not t1.raised) or \ + (t1.raised and not t2.raised) + + + def test_set_var_domain(self): + x = u.var('x') + u.set_domain(x, [1, 3, 5]) + assert x.dom == c.FiniteDomain([1, 3, 5]) + + def test_bind_with_domain(self): + x = u.var('x') + u.set_domain(x, [1, 2, 3]) + raises(u.OutOfDomain, u.bind, x, 42) + u.bind(x, 3) + assert x.val == 3 + + def test_bind_with_incompatible_domains(self): + x, y = u.var('x'), u.var('y') + u.set_domain(x, [1, 2]) + u.set_domain(y, [3, 4]) + raises(u.IncompatibleDomains, u.bind, x, y) + u.set_domain(y, [2, 4]) + u.bind(x, y) + # check x and y are in the same equiv. set + assert x.val == y.val + + + def test_unify_with_domains(self): + x,y,z = u.var('x'), u.var('y'), u.var('z') + u.bind(x, [42, z]) + u.bind(y, [z, 42]) + u.set_domain(z, [1, 2, 3]) + raises(u.UnificationFailure, u.unify, x, y) + u.set_domain(z, [41, 42, 43]) + u.unify(x, y) + assert z.val == 42 + assert z.dom == c.FiniteDomain([41, 42, 43]) + + def test_add_constraint(self): + x,y,z = u.var('x'), u.var('y'), u.var('z') + raises(c.DomainlessVariables, + c.Expression, [x, y, z], 'x == y + z') + x.dom = c.FiniteDomain([1, 2]) + y.dom = c.FiniteDomain([2, 3]) + z.dom = c.FiniteDomain([3, 4]) + k = c.Expression([x, y, z], 'x == y + z') + u.add_constraint(k) + assert k in u._store.constraints + + def test_narrowing_domains_failure(self): + x,y,z = u.var('x'), u.var('y'), u.var('z') + x.dom = c.FiniteDomain([1, 2]) + y.dom = c.FiniteDomain([2, 3]) + z.dom = c.FiniteDomain([3, 4]) + k = c.Expression([x, y, z], 'x == y + z') + raises(c.ConsistencyFailure, k.narrow) + + def test_narrowing_domains_success(self): + x,y,z = u.var('x'), u.var('y'), u.var('z') + x.dom = c.FiniteDomain([1, 2, 5]) + y.dom = c.FiniteDomain([2, 3]) + z.dom = c.FiniteDomain([3, 4]) + k = c.Expression([x, y, z], 'x == y + z') + k.narrow() + assert x.dom == c.FiniteDomain([5]) + assert y.dom == c.FiniteDomain([2]) + assert z.dom == c.FiniteDomain([3]) + + def test_store_satisfiable_success(self): + x,y,z = u.var('x'), u.var('y'), u.var('z') + x.dom = c.FiniteDomain([1, 2, 5]) + y.dom = c.FiniteDomain([2, 3]) + z.dom = c.FiniteDomain([3, 4]) + k = c.Expression([x, y, z], 'x == y + z') + u.add_constraint(k) + assert u.satisfiable(k) == True + assert x.dom == c.FiniteDomain([1, 2, 5]) + assert y.dom == c.FiniteDomain([2, 3]) + assert z.dom == c.FiniteDomain([3, 4]) + + def test_store_satisfiable_failure(self): + x,y,z = u.var('x'), u.var('y'), u.var('z') + x.dom = c.FiniteDomain([1, 2]) + y.dom = c.FiniteDomain([2, 3]) + z.dom = c.FiniteDomain([3, 4]) + k = c.Expression([x, y, z], 'x == y + z') + u.add_constraint(k) + assert u.satisfiable(k) == False + assert x.dom == c.FiniteDomain([1, 2]) + assert y.dom == c.FiniteDomain([2, 3]) + assert z.dom == c.FiniteDomain([3, 4]) + + def test_satisfiable_many_const_success(self): + x,y,z,w = (u.var('x'), u.var('y'), + u.var('z'), u.var('w')) + x.dom = c.FiniteDomain([1, 2, 5]) + y.dom = c.FiniteDomain([2, 3]) + z.dom = c.FiniteDomain([3, 4]) + w.dom = c.FiniteDomain([1, 4, 5]) + k1 = c.Expression([x, y, z], 'x == y + z') + k2 = c.Expression([z, w], 'z < w') + u.add_constraint(k1) + u.add_constraint(k2) + assert u.satisfiable(k1) == True + assert x.dom == c.FiniteDomain([1, 2, 5]) + assert y.dom == c.FiniteDomain([2, 3]) + assert z.dom == c.FiniteDomain([3, 4]) + assert w.dom == c.FiniteDomain([1, 4, 5]) + assert u.satisfiable(k2) == True + assert x.dom == c.FiniteDomain([1, 2, 5]) + assert y.dom == c.FiniteDomain([2, 3]) + assert z.dom == c.FiniteDomain([3, 4]) + assert w.dom == c.FiniteDomain([1, 4, 5]) + narrowed_doms = u.get_satisfying_domains(k1) + assert narrowed_doms == {x:c.FiniteDomain([5]), + y:c.FiniteDomain([2]), + z:c.FiniteDomain([3]), + w:c.FiniteDomain([4, 5])} + narrowed_doms = u.get_satisfying_domains(k2) + assert narrowed_doms == {x:c.FiniteDomain([5]), + y:c.FiniteDomain([2]), + z:c.FiniteDomain([3]), + w:c.FiniteDomain([4, 5])} + + + def test_satisfiable_many_const_failure(self): + x,y,z,w = (u.var('x'), u.var('y'), + u.var('z'), u.var('w')) + x.dom = c.FiniteDomain([1, 2, 5]) + y.dom = c.FiniteDomain([2, 3]) + z.dom = c.FiniteDomain([3, 4]) + w.dom = c.FiniteDomain([1]) + k1 = c.Expression([x, y, z], 'x == y + z') + k2 = c.Expression([z, w], 'z < w') + u.add_constraint(k1) + u.add_constraint(k2) + assert u.satisfiable(k1) == False + assert x.dom == c.FiniteDomain([1, 2, 5]) + assert y.dom == c.FiniteDomain([2, 3]) + assert z.dom == c.FiniteDomain([3, 4]) + assert w.dom == c.FiniteDomain([1]) + assert u.satisfiable(k2) == False + assert x.dom == c.FiniteDomain([1, 2, 5]) + assert y.dom == c.FiniteDomain([2, 3]) + assert z.dom == c.FiniteDomain([3, 4]) + assert w.dom == c.FiniteDomain([1]) + narrowed_doms = u.get_satisfying_domains(k1) + assert narrowed_doms == {} + narrowed_doms = u.get_satisfying_domains(k2) + assert narrowed_doms == {} + + def test_satisfy_many_const_failure(self): + x,y,z,w = (u.var('x'), u.var('y'), + u.var('z'), u.var('w')) + x.dom = c.FiniteDomain([1, 2, 5]) + y.dom = c.FiniteDomain([2, 3]) + z.dom = c.FiniteDomain([3, 4]) + w.dom = c.FiniteDomain([1]) + k1 = c.Expression([x, y, z], 'x == y + z') + k2 = c.Expression([z, w], 'z < w') + u.add_constraint(k1) + u.add_constraint(k2) + raises(u.ConsistencyFailure, u.satisfy, k1) + assert x.dom == c.FiniteDomain([1, 2, 5]) + assert y.dom == c.FiniteDomain([2, 3]) + assert z.dom == c.FiniteDomain([3, 4]) + assert w.dom == c.FiniteDomain([1]) + raises(u.ConsistencyFailure, u.satisfy, k2) + assert x.dom == c.FiniteDomain([1, 2, 5]) + assert y.dom == c.FiniteDomain([2, 3]) + assert z.dom == c.FiniteDomain([3, 4]) + assert w.dom == c.FiniteDomain([1]) + + def test_satisfy_many_const_success(self): + x,y,z,w = (u.var('x'), u.var('y'), + u.var('z'), u.var('w')) + x.dom = c.FiniteDomain([1, 2, 5]) + y.dom = c.FiniteDomain([2, 3]) + z.dom = c.FiniteDomain([3, 4]) + w.dom = c.FiniteDomain([1, 4, 5]) + k1 = c.Expression([x, y, z], 'x == y + z') + k2 = c.Expression([z, w], 'z < w') + u.add_constraint(k1) + u.add_constraint(k2) + u.satisfy(k2) + assert x.dom == c.FiniteDomain([5]) + assert y.dom == c.FiniteDomain([2]) + assert z.dom == c.FiniteDomain([3]) + assert w.dom == c.FiniteDomain([4, 5]) Added: pypy/dist/pypy/lib/logic/store_without_cs/test_variable.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/lib/logic/store_without_cs/test_variable.py Thu Jan 26 15:48:35 2006 @@ -0,0 +1,67 @@ +from py.test import raises +import unification as u +import variable as v +from threading import Thread + +class Consumer(Thread): + + def give_var(self, var): + self.var = var + + def run(self): + val = self.var.get() + +class NConsumer(Thread): + + def give_vars(self, vars_): + self.vars = vars_ + + def run(self): + val = [var.get() for var in self.vars] + +class TestVariable: + + def setup_method(self, meth): + u._store = u.Store() + + def test_no_same_name(self): + x = u.var('x') + raises(u.AlreadyInStore, u.var, 'x') + + def test_get_by_name(self): + x = u.var('x') + assert x == u._store.get_var_by_name('x') + raises(u.NotInStore, u._store.get_var_by_name, 'y') + + def test_one_thread_reading_one_var(self): + cons = Consumer() + x = u.var('x') + cons.give_var(x) + cons.start() + u.bind(x, 42) + cons.join() + assert cons.var.val == 42 + + def test_many_threads_reading_one_var(self): + conss = [Consumer() for i in range(10)] + x = u.var('x') + for cons in conss: + cons.give_var(x) + cons.start() + u.bind(x, 42) + for cons in conss: + cons.join() + assert cons.var.val == 42 + + def test_many_thread_reading_many_var(self): + conss = [NConsumer() for i in range(10)] + vars_ = [u.var(str(i)) for i in range(10)] + for cons in conss: + cons.give_vars(vars_) + cons.start() + for var in vars_: + u.bind(var, var.name) + for cons in conss: + cons.join() + for i in range(10): + assert vars_[i].val == str(i) Added: pypy/dist/pypy/lib/logic/store_without_cs/unification.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/lib/logic/store_without_cs/unification.py Thu Jan 26 15:48:35 2006 @@ -0,0 +1,573 @@ +## Oz-like unification of dataflow variables in Python 2.4 +## within a single assignment store +## crude ... + +## Unification over dataflow variables in Oz is a powerful mechanism +## which is the basis of relational, logic and constraint programming. +## Please read the stuff in variable.py to understand the nature of +## dataflow variables. +## +## Unification in Oz (almost verbatim from CTM*) +## ============================================= +## +## Unification as put in Oz is "a very powerful operation". It is +## provided through the '=' operator. +## +## Binding a variable to a value is a special case of an operation +## called unification. The unification = makes the +## partial values and equal, if possible, by adding +## zero or more bindings to the store. For example, f(X Y)=f(1 2) does +## two bindings: X=1 and Y=2. If the two terms cannot be made equal, +## then an exception is raised. Unification exists because of partial +## values; if there would be only complete values, then it would have +## no meaning. +## +## Unification adds information to the single-assignment store (a set +## of dataflow variables, where each variable is either unbound or +## bound to some other store entity). +## +## Example : if the store has X=foo(A) and Y=foo(25), then doing X=Y +## will bind A to 25. +## +## *failure* is the exception raised when unification cannot +## happen. For instance (unification of two records) : +## +## person(name:X1 age:23) +## person(name:"Auc" age:32) +## +## will raise it. The fail statement raises this exception. +## +## It is symetric : X=Y means the same as Y=X +## It can create cyclic structures, as in : X=person(grandfather:X) +## It can bind cyclic structures : +## +## X=f(a:X b:_) +## Y=f(a:_ b:Y) +## X=Y +## +## creates a two cycles structure writable as : X=f(a:X b:X) +## +## Unification algorithm : +## +## operation unify(x, y) that unify two partial values x and y in the +## store st. +## +## The Store consists of a set of k variables : x1, ... xk that are +## partitioned as follows: +## +## * set of unbound variables that are equal (also called equivalence +## sets of variables). The variables in each set are equal to each +## other but not to any other variables. +## +## * variables bound to a number, record or procedure (also called +## determined variables). +## +## Example : st = {x1=foo(a:x2), x2=25, x3=x4=x5, x6, x7=x8} that has +## 8 variables. It has three equivalence sets : +## {x3,x4,x5},{x6},{x7,x8}. It has two determined variables x1 and x2. +## +## The primitive bind operation : unification is defined in terms of a +## primitive bind operation on the store st. The operation binds all +## variables in an equivalence set: +## +## * bind(ES,) binds all variables in the equivalence set ES to the +## number or record . For example, the operation +## bind({x7,x8},foo(a:x2)) binds x7 and x8, which no longer are in +## an equivalence set. +## +## * bind(ES1,ES2) merges the equivalence set ES1 with the equivalence +## set ES2. +## +## Algorithm unify(x,y) +## +## 1. if x is in the equivalence set ESx and y is in the equivalence +## set ESy, then do bind(ESx,ESy). It is a noop if ESx == ESy. +## +## 2. if x is in the equivalence set ESx and y is determined**, then do +## bind(ESx,y) +## +## 3. if y is in the equivalence set ESy and y is determined, then do +## bind(ESy,x) +## +## 4. if x is bound to l(l1:x1,...,ln:xn) and y is bound to +## l'(l'1:y1,...,l'm:ym) with l!=l' or {l1,...,ln}!={l'1,...,l'm}, +## then raise a failure exception +## +## 5. if x is bound to l(l1:x1,...,ln:xn) and y is bound to +## l(l1:y1,...,ln:yn), then for i from 1 to n do unify(xi,yi). +## +## +## With cycles : the above algorithm does not handle unification of +## partial values with cycles. For example, on x=f(a:x) and y=f(a:y) a +## call to unify(x,y) results in the recursive call unify(x,y), which +## is identical to the original call, looping forever. A simple fix +## is to make sure that unify(x,y) is called at most once for each +## possible pair of two variables (x,y); We do this through +## memoization of called pairs. + +## * CTM : Concepts, Techniques and Models of Computer Programming +## ** determined == bound + +#TODO : +# * understand this : +# http://www.mozart-oz.org/papers/abstracts/ProgrammingConstraintServices.html +# * support '_' as shown above +# * turn Var into some dataflow-ish thing (as far as Python allows) +# * ensure that the store supports concurrent access +# (using the implicit blocking provided by dataflow vars) +# * add entailment checks +# * add constraints support + +import threading + +from variable import EqSet, Var, \ + VariableException, NotAVariable, AlreadyInStore +from constraint import FiniteDomain, ConsistencyFailure + +#----------- Store Exceptions ---------------------------- +class UnboundVariable(VariableException): + def __str__(self): + return "%s has no value yet" % self.name + +class AlreadyBound(VariableException): + def __str__(self): + return "%s is already bound" % self.name + +class NotInStore(VariableException): + def __str__(self): + return "%s not in the store" % self.name + +class OutOfDomain(VariableException): + def __str__(self): + return "value not in domain of %s" % self.name + +class UnificationFailure(Exception): + def __init__(self, var1, var2, cause=None): + self.var1, self.var2 = (var1, var2) + self.cause = cause + def __str__(self): + diag = "%s %s can't be unified" + if self.cause: + diag += " because %s" % self.cause + return diag % (self.var1, self.var2) + +class IncompatibleDomains(Exception): + def __init__(self, var1, var2): + self.var1, self.var2 = (var1, var2) + def __str__(self): + return "%s %s have incompatible domains" % \ + (self.var1, self.var2) + +#----------- Store ------------------------------------ +class Store(object): + """The Store consists of a set of k variables + x1,...,xk that are partitioned as follows: + * set of unbound variables that are equal + (also called equivalence sets of variables). + The variables in each set are equal to each + other but not to any other variables. + * variables bound to a number, record or procedure + (also called determined variables).""" + + def __init__(self): + self.vars = set() + # mapping of names to vars (all of them) + self.names = {} + # set of all constraints + self.constraints = set() + # consistency-preserving stuff + self.in_transaction = False + self.lock = threading.RLock() + + #-- Variables ---------------------------- + + def var(self, name): + """creates a variable of name name and put + it into the store""" + v = Var(name, self) + self.add_unbound(v) + return v + + def add_unbound(self, var): + """add unbound variable to the store""" + if var in self.vars: + raise AlreadyInStore(var.name) + print "adding %s to the store" % var + self.vars.add(var) + self.names[var.name] = var + # put into new singleton equiv. set + var.val = EqSet([var]) + + def set_domain(self, var, dom): + """bind variable to domain""" + assert(isinstance(var, Var) and (var in self.vars)) + if var.is_bound(): + raise AlreadyBound + var.dom = FiniteDomain(dom) + + def get_var_by_name(self, name): + try: + return self.names[name] + except KeyError: + raise NotInStore(name) + + #-- Constraints ------------------------- + + def add_constraint(self, constraint): + self.constraints.add(constraint) + + def get_variables_with_a_domain(self): + varset = set() + for var in self.vars: + if var.dom: varset.add(var) + return varset + + def satisfiable(self, constraint): + """ * satisfiable (k) checks that the constraint k + can be satisfied wrt its variable domains + and other constraints on these variables + * does NOT mutate the store + """ + # Satisfiability of one constraints entails + # satisfiability of the transitive closure + # of all constraints associated with the vars + # of our given constraint. + # We make a copy of the domains + # then traverse the constraints & attached vars + # to collect all (in)directly affected vars + # then compute narrow() on all (in)directly + # affected constraints. + assert constraint in self.constraints + varset = set() + constset = set() + compute_dependant_vars(constraint, varset, constset) + old_domains = collect_domains(varset) + + for const in constset: + try: + const.narrow() + except ConsistencyFailure: + restore_domains(old_domains) + return False + restore_domains(old_domains) + return True + + + def get_satisfying_domains(self, constraint): + assert constraint in self.constraints + varset = set() + constset = set() + compute_dependant_vars(constraint, varset, constset) + old_domains = collect_domains(varset) + + for const in constset: + try: + const.narrow() + except ConsistencyFailure: + restore_domains(old_domains) + return {} + narrowed_domains = collect_domains(varset) + restore_domains(old_domains) + return narrowed_domains + + def satisfy(self, constraint): + assert constraint in self.constraints + varset = set() + constset = set() + compute_dependant_vars(constraint, varset, constset) + old_domains = collect_domains(varset) + + for const in constset: + try: + const.narrow() + except ConsistencyFailure: + restore_domains(old_domains) + raise + + def satisfy_all(self): + old_domains = collect_domains(self.vars) + for const in self.constraints: + try: + const.narrow() + except ConsistencyFailure: + restore_domains(old_domains) + raise + + + #-- BIND ------------------------------------------- + + def bind(self, var, val): + """1. (unbound)Variable/(unbound)Variable or + 2. (unbound)Variable/(bound)Variable or + 3. (unbound)Variable/Value binding + """ + try: + self.lock.acquire() + assert(isinstance(var, Var) and (var in self.vars)) + if var == val: + return + if _both_are_vars(var, val): + if _both_are_bound(var, val): + raise AlreadyBound(var.name) + if var._is_bound(): # 2b. var is bound, not var + self.bind(val, var) + elif val._is_bound(): # 2a.var is bound, not val + self._bind(var.val, val.val) + else: # 1. both are unbound + self._merge(var, val) + else: # 3. val is really a value + print "%s, is that you ?" % var + if var._is_bound(): + raise AlreadyBound(var.name) + self._bind(var.val, val) + finally: + self.lock.release() + + + def _bind(self, eqs, val): + # print "variable - value binding : %s %s" % (eqs, val) + # bind all vars in the eqset to val + for var in eqs: + if var.dom != None: + if val not in var.dom.get_values(): + # undo the half-done binding + for v in eqs: + v.val = eqs + raise OutOfDomain(var) + var.val = val + + def _merge(self, v1, v2): + for v in v1.val: + if not _compatible_domains(v, v2.val): + raise IncompatibleDomains(v1, v2) + self._really_merge(v1.val, v2.val) + + def _really_merge(self, eqs1, eqs2): + # print "unbound variables binding : %s %s" % (eqs1, eqs2) + if eqs1 == eqs2: return + # merge two equisets into one + eqs1 |= eqs2 + # let's reassign everybody to the merged eq + for var in eqs1: + var.val = eqs1 + + #-- UNIFY ------------------------------------------ + + def unify(self, x, y): + self.in_transaction = True + try: + try: + self._really_unify(x, y) + for var in self.vars: + if var.changed: + var._commit() + except Exception, cause: + for var in self.vars: + if var.changed: + var._abort() + if isinstance(cause, UnificationFailure): + raise + raise UnificationFailure(x, y, cause) + finally: + self.in_transaction = False + + def _really_unify(self, x, y): + # print "unify %s with %s" % (x,y) + if not _unifiable(x, y): raise UnificationFailure(x, y) + if not x in self.vars: + if not y in self.vars: + # duh ! x & y not vars + if x != y: raise UnificationFailure(x, y) + else: return + # same call, reverse args. order + self._unify_var_val(y, x) + elif not y in self.vars: + # x is Var, y a value + self._unify_var_val(x, y) + elif _both_are_bound(x, y): + self._unify_bound(x,y) + elif x._is_bound(): + self.bind(x,y) + else: + self.bind(y,x) + + def _unify_var_val(self, x, y): + if x.val != y: + try: + self.bind(x, y) + except AlreadyBound: + raise UnificationFailure(x, y) + + def _unify_bound(self, x, y): + # print "unify bound %s %s" % (x, y) + vx, vy = (x.val, y.val) + if type(vx) in [list, set] and isinstance(vy, type(vx)): + self._unify_iterable(x, y) + elif type(vx) is dict and isinstance(vy, type(vx)): + self._unify_mapping(x, y) + else: + if vx != vy: + raise UnificationFailure(x, y) + + def _unify_iterable(self, x, y): + print "unify sequences %s %s" % (x, y) + vx, vy = (x.val, y.val) + idx, top = (0, len(vx)) + while (idx < top): + self._really_unify(vx[idx], vy[idx]) + idx += 1 + + def _unify_mapping(self, x, y): + # print "unify mappings %s %s" % (x, y) + vx, vy = (x.val, y.val) + for xk in vx.keys(): + self._really_unify(vx[xk], vy[xk]) + + +def _compatible_domains(var, eqs): + """check that the domain of var is compatible + with the domains of the vars in the eqs + """ + if var.dom == None: return True + empty = set() + for v in eqs: + if v.dom == None: continue + if v.dom.intersection(var.dom) == empty: + return False + return True + + +def compute_dependant_vars(constraint, varset, + constset): + if constraint in constset: return + constset.add(constraint) + for var in constraint.affectedVariables(): + varset.add(var) + dep_consts = var.constraints + for const in dep_consts: + if const in constset: + continue + compute_dependant_vars(const, varset, constset) + + +#-- collect / restore utilities for domains + +def collect_domains(varset): + """makes a copy of domains of a set of vars + into a var -> dom mapping + """ + dom = {} + for var in varset: + if var.dom: + dom[var] = FiniteDomain(var.dom) + return dom + +def restore_domains(domains): + """sets the domain of the vars in the domains mapping + to their (previous) value + """ + for var, dom in domains.items(): + var.dom = dom + + +#-- Unifiability checks--------------------------------------- +#-- +#-- quite costly & could be merged back in unify + +def _iterable(thing): + return type(thing) in [list, set] + +def _mapping(thing): + return type(thing) is dict + +# memoizer for _unifiable +_unifiable_memo = set() + +def _unifiable(term1, term2): + global _unifiable_memo + _unifiable_memo = set() + return _really_unifiable(term1, term2) + +def _really_unifiable(term1, term2): + """Checks wether two terms can be unified""" + if ((id(term1), id(term2))) in _unifiable_memo: return False + _unifiable_memo.add((id(term1), id(term2))) + # print "unifiable ? %s %s" % (term1, term2) + if _iterable(term1): + if _iterable(term2): + return _iterable_unifiable(term1, term2) + return False + if _mapping(term1) and _mapping(term2): + return _mapping_unifiable(term1, term2) + if not(isinstance(term1, Var) or isinstance(term2, Var)): + return term1 == term2 # same 'atomic' object + return True + +def _iterable_unifiable(c1, c2): + """Checks wether two iterables can be unified""" + # print "unifiable sequences ? %s %s" % (c1, c2) + if len(c1) != len(c2): return False + idx, top = (0, len(c1)) + while(idx < top): + if not _really_unifiable(c1[idx], c2[idx]): + return False + idx += 1 + return True + +def _mapping_unifiable(m1, m2): + """Checks wether two mappings can be unified""" + # print "unifiable mappings ? %s %s" % (m1, m2) + if len(m1) != len(m2): return False + if m1.keys() != m2.keys(): return False + v1, v2 = (m1.items(), m2.items()) + v1.sort() + v2.sort() + return _iterable_unifiable([e[1] for e in v1], + [e[1] for e in v2]) + +#-- Some utilities ----------------------------------------------- + +def _both_are_vars(v1, v2): + return isinstance(v1, Var) and isinstance(v2, Var) + +def _both_are_bound(v1, v2): + return v1._is_bound() and v2._is_bound() + +#-- +#-- the global store +_store = Store() + +#-- global accessor functions +def var(name): + v = Var(name, _store) + _store.add_unbound(v) + return v + +def set_domain(var, dom): + return _store.set_domain(var, dom) + +def add_constraint(constraint): + return _store.add_constraint(constraint) + +def satisfiable(constraint): + return _store.satisfiable(constraint) + +def get_satisfying_domains(constraint): + return _store.get_satisfying_domains(constraint) + +def satisfy(constraint): + return _store.satisfy(constraint) + +def bind(var, val): + return _store.bind(var, val) + +def unify(var1, var2): + return _store.unify(var1, var2) + +def bound(): + return map(_store.vars, + lambda v: v.is_bound()) + +def unbound(): + return map(_store.vars, + lambda v: not v.is_bound()) Added: pypy/dist/pypy/lib/logic/store_without_cs/variable.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/lib/logic/store_without_cs/variable.py Thu Jan 26 15:48:35 2006 @@ -0,0 +1,125 @@ +# First cut at representing Oz dataflow variable + +import threading + +#----------- Exceptions --------------------------------- +class VariableException(Exception): + def __init__(self, name): + self.name = name + +class AlreadyInStore(VariableException): + def __str__(self): + return "%s already in store" % self.name + +class NotAVariable(VariableException): + def __str__(self): + return "%s is not a variable" % self.name + +#----------- Variables ---------------------------------- +class EqSet(set): + """An equivalence set for variables""" + +## def __str__(self): +## if len(self) == 0: +## return '' +## for var in self: +## '='.join(var.name) + +class NoValue: + pass + +class Var(object): + + def __init__(self, name, store): + if name in store.names: + raise AlreadyInStore(name) + self.name = name + self.store = store + # top-level 'commited' binding + self._val = NoValue + # domains in a flat world + self.dom = None + # domains in multiple spaces + self.doms = {} + # constraints + self.constraints = set() + # when updated in a 'transaction', keep track + # of our initial value (for abort cases) + self.previous = None + self.changed = False + # a condition variable for concurrent access + self.mutex = threading.Lock() + self.value_condition = threading.Condition(self.mutex) + + # for consumption by the global store + + def _is_bound(self): + return not isinstance(self._val, EqSet) \ + and self._val != NoValue + + # 'transaction' support + + def _commit(self): + self.changed = False + + def _abort(self): + self.val = self.previous + self.changed = False + + # value accessors + def _set_val(self, val): + self.value_condition.acquire() + if self.store.in_transaction: + if not self.changed: + self.previous = self._val + self.changed = True + self._val = val + self.value_condition.notifyAll() + self.value_condition.release() + + def _get_val(self): + return self._val + val = property(_get_val, _set_val) + + def __str__(self): + if self.is_bound(): + return "%s = %s" % (self.name, self.val) + return "%s" % self.name + + def __repr__(self): + return self.__str__() + + def __eq__(self, thing): + return isinstance(thing, Var) \ + and self.name == thing.name + + def __hash__(self): + return self.name.__hash__() + + + def add_constraint(self, constraint): + self.constraints.add(constraint) + + is_bound = _is_bound + + def cs_set_dom(self, cs, dom): + self.doms[cs] = dom + + def cs_get_dom(self, cs): + return self.doms[cs] + + #---- Concurrent public ops -------------------------- + # should be used by threads that want to block on + # unbound variables + + def get(self): + """Make threads wait on the variable + being bound + """ + try: + self.value_condition.acquire() + while not self._is_bound(): + self.value_condition.wait() + return self.val + finally: + self.value_condition.release() From tismer at codespeak.net Thu Jan 26 16:54:58 2006 From: tismer at codespeak.net (tismer at codespeak.net) Date: Thu, 26 Jan 2006 16:54:58 +0100 (CET) Subject: [pypy-svn] r22688 - pypy/dist/pypy/module/stackless Message-ID: <20060126155458.BEBE327DC8@code1.codespeak.net> Author: tismer Date: Thu Jan 26 16:54:56 2006 New Revision: 22688 Modified: pypy/dist/pypy/module/stackless/interp_coroutine.py Log: supporting __del__ and flags in AppCoroutine Modified: pypy/dist/pypy/module/stackless/interp_coroutine.py ============================================================================== --- pypy/dist/pypy/module/stackless/interp_coroutine.py (original) +++ pypy/dist/pypy/module/stackless/interp_coroutine.py Thu Jan 26 16:54:56 2006 @@ -98,7 +98,6 @@ # override this for exposed coros pass -## later: or self.space.lookup(self, '__del__') is not None def postpone_deletion(obj): costate.things_to_do = True @@ -139,7 +138,8 @@ def call(self): self.space.call_args(self.w_func, self.args) -class AppCoroutine(Coroutine, StacklessFlags): + +class AppCoroutine(Coroutine): # XXX, StacklessFlags): def __init__(self): Coroutine.__init__(self) @@ -158,9 +158,32 @@ def w_switch(self): self.switch() + def __del__(self): + if self.frame is not None or self.space.lookup(self, '__del__') is not None: + postpone_deletion(self) + + def _userdel(self): + if self.get_is_zombie(): + return + self.set_is_zombie(True) + self.space.userdel(self) + + +# _mixin_ did not work +for methname in StacklessFlags.__dict__: + meth = getattr(StacklessFlags, methname) + if hasattr(meth, 'im_func'): + setattr(AppCoroutine, meth.__name__, meth.im_func) +del meth, methname + +def w_get_is_zombie(space, self): + return space.wrap(self.get_is_zombie()) +AppCoroutine.w_get_is_zombie = w_get_is_zombie + AppCoroutine.typedef = TypeDef("Coroutine", __new__ = interp2app(AppCoroutine.descr_method__new__.im_func), bind = interp2app(AppCoroutine.w_bind, unwrap_spec=['self', W_Root, Arguments]), switch = interp2app(AppCoroutine.w_switch), + is_zombie = GetSetProperty(AppCoroutine.w_get_is_zombie, doc=AppCoroutine.get_is_zombie.__doc__), ) From ericvrp at codespeak.net Thu Jan 26 20:54:45 2006 From: ericvrp at codespeak.net (ericvrp at codespeak.net) Date: Thu, 26 Jan 2006 20:54:45 +0100 (CET) Subject: [pypy-svn] r22692 - pypy/extradoc/sprintinfo/tokyo Message-ID: <20060126195445.5DA1527DCB@code1.codespeak.net> Author: ericvrp Date: Thu Jan 26 20:54:43 2006 New Revision: 22692 Modified: pypy/extradoc/sprintinfo/tokyo/people.txt Log: tokyo yeah Modified: pypy/extradoc/sprintinfo/tokyo/people.txt ============================================================================== --- pypy/extradoc/sprintinfo/tokyo/people.txt (original) +++ pypy/extradoc/sprintinfo/tokyo/people.txt Thu Jan 26 20:54:43 2006 @@ -10,6 +10,7 @@ ==================== ============== ===================== Beatrice D?ring Christian Tismer +Eric van Riet Paap ==================== ============== ===================== From lac at codespeak.net Thu Jan 26 21:13:09 2006 From: lac at codespeak.net (lac at codespeak.net) Date: Thu, 26 Jan 2006 21:13:09 +0100 (CET) Subject: [pypy-svn] r22693 - pypy/extradoc/sprintinfo/tokyo Message-ID: <20060126201309.61A0C27DCB@code1.codespeak.net> Author: lac Date: Thu Jan 26 21:13:08 2006 New Revision: 22693 Modified: pypy/extradoc/sprintinfo/tokyo/people.txt Log: Tokyo! :-) Modified: pypy/extradoc/sprintinfo/tokyo/people.txt ============================================================================== --- pypy/extradoc/sprintinfo/tokyo/people.txt (original) +++ pypy/extradoc/sprintinfo/tokyo/people.txt Thu Jan 26 21:13:08 2006 @@ -11,8 +11,8 @@ Beatrice D?ring Christian Tismer Eric van Riet Paap - - +Jacob Hall?n +Laura Creighton ==================== ============== ===================== People on the following list were present at previous sprints: @@ -24,7 +24,6 @@ Adrien Di Mascio Laura Creighton Lene Wagner -Anders Lehmann Niklaus Haldimann Johan Hahn Amaury Forgeot d'Arc From ale at codespeak.net Fri Jan 27 09:28:01 2006 From: ale at codespeak.net (ale at codespeak.net) Date: Fri, 27 Jan 2006 09:28:01 +0100 (CET) Subject: [pypy-svn] r22714 - pypy/extradoc/sprintinfo/tokyo Message-ID: <20060127082801.C998E27B70@code1.codespeak.net> Author: ale Date: Fri Jan 27 09:28:00 2006 New Revision: 22714 Modified: pypy/extradoc/sprintinfo/tokyo/ (props changed) pypy/extradoc/sprintinfo/tokyo/people.txt (contents, props changed) Log: I want to come too + fixeol Modified: pypy/extradoc/sprintinfo/tokyo/people.txt ============================================================================== --- pypy/extradoc/sprintinfo/tokyo/people.txt (original) +++ pypy/extradoc/sprintinfo/tokyo/people.txt Fri Jan 27 09:28:00 2006 @@ -1,33 +1,34 @@ -People coming to the Tokyo sprint 2006 -================================================== - -People who have a ``?`` in their arrive/depart or accomodation -column are known to be coming but there are no details -available yet from them. - -==================== ============== ===================== - Name Arrive/Depart Accomodation -==================== ============== ===================== -Beatrice D?ring -Christian Tismer -Eric van Riet Paap -Jacob Hall?n -Laura Creighton -==================== ============== ===================== - -People on the following list were present at previous sprints: - -==================== ============== ===================== - Name Arrive/Depart Accomodation -==================== ============== ===================== -Ludovic Aubry -Adrien Di Mascio -Laura Creighton -Lene Wagner -Niklaus Haldimann -Johan Hahn -Amaury Forgeot d'Arc -Valentino Volonghi -Boris Feigin -Bert Freudenberg -==================== ============== ===================== +People coming to the Tokyo sprint 2006 +================================================== + +People who have a ``?`` in their arrive/depart or accomodation +column are known to be coming but there are no details +available yet from them. + +==================== ============== ===================== + Name Arrive/Depart Accomodation +==================== ============== ===================== +Beatrice D?ring +Christian Tismer +Eric van Riet Paap +Jacob Hall?n +Laura Creighton +Anders Lehmann +==================== ============== ===================== + +People on the following list were present at previous sprints: + +==================== ============== ===================== + Name Arrive/Depart Accomodation +==================== ============== ===================== +Ludovic Aubry +Adrien Di Mascio +Laura Creighton +Lene Wagner +Niklaus Haldimann +Johan Hahn +Amaury Forgeot d'Arc +Valentino Volonghi +Boris Feigin +Bert Freudenberg +==================== ============== ===================== From ac at codespeak.net Fri Jan 27 10:26:12 2006 From: ac at codespeak.net (ac at codespeak.net) Date: Fri, 27 Jan 2006 10:26:12 +0100 (CET) Subject: [pypy-svn] r22715 - in pypy/dist: . pypy/doc/discussion pypy/jit pypy/rpython pypy/rpython/test Message-ID: <20060127092612.592C027DBE@code1.codespeak.net> Author: ac Date: Fri Jan 27 10:26:11 2006 New Revision: 22715 Modified: pypy/dist/ (props changed) pypy/dist/pypy/doc/discussion/draft-jit-ideas.txt pypy/dist/pypy/jit/llabstractinterp.py pypy/dist/pypy/jit/llcontainer.py pypy/dist/pypy/jit/llvalue.py pypy/dist/pypy/jit/vlist.py pypy/dist/pypy/rpython/rgenop.py pypy/dist/pypy/rpython/test/test_rlist.py Log: Merged revisions 22667-22668,22678,22681-22685 via svnmerge from svn+ssh://codespeak.net/svn/pypy/branch/jit-refactoring ........ r22667 | ac | 2006-01-25 19:45:52 +0100 (Wed, 25 Jan 2006) | 3 lines Initialized merge tracking via "svnmerge" with revisions "1-22663" from svn+ssh://codespeak.net/svn/pypy/dist ........ r22668 | ac | 2006-01-25 19:46:27 +0100 (Wed, 25 Jan 2006) | 1 line Intermediary work. ........ r22678 | pedronis | 2006-01-26 12:59:23 +0100 (Thu, 26 Jan 2006) | 5 lines use AConstant and AVariable for LLAbstractValue.runtimevar, these have a getgenvar(builder) to retrieve a stored corresponding rgenop API var/const opaque object, or to lazily comnpute such a const object in the context of the given builder. ........ r22681 | pedronis | 2006-01-26 13:55:05 +0100 (Thu, 26 Jan 2006) | 3 lines fix test that was broken by changes. ........ r22682 | pedronis | 2006-01-26 14:02:48 +0100 (Thu, 26 Jan 2006) | 3 lines remove unncessary import ........ r22683 | pedronis | 2006-01-26 14:07:02 +0100 (Thu, 26 Jan 2006) | 3 lines refactor to avoid accessing rgenop opaque constant .value! ........ r22684 | pedronis | 2006-01-26 14:12:19 +0100 (Thu, 26 Jan 2006) | 3 lines more consistent naming: gengraphconst -> gencallableconst. rgenop API doesn't expose the notion of a graph. ........ r22685 | pedronis | 2006-01-26 14:17:36 +0100 (Thu, 26 Jan 2006) | 5 lines addconst seems not necessary, for now for Void constant we simply constuct Constant directly. Document gencallableconst. ........ Modified: pypy/dist/pypy/doc/discussion/draft-jit-ideas.txt ============================================================================== --- pypy/dist/pypy/doc/discussion/draft-jit-ideas.txt (original) +++ pypy/dist/pypy/doc/discussion/draft-jit-ideas.txt Fri Jan 27 10:26:11 2006 @@ -100,7 +100,7 @@ * genconst(block, llvalue) -> (result var) -* addconst(block, const) -> (result var) +* gencallableconst(block, name, target-block, FUNCTYPE) -> (result var) * closeblock1(block) -> link Modified: pypy/dist/pypy/jit/llabstractinterp.py ============================================================================== --- pypy/dist/pypy/jit/llabstractinterp.py (original) +++ pypy/dist/pypy/jit/llabstractinterp.py Fri Jan 27 10:26:11 2006 @@ -1,8 +1,8 @@ import operator -from pypy.objspace.flow.model import Variable, Constant, SpaceOperation +from pypy.objspace.flow.model import Variable, Constant from pypy.objspace.flow.model import checkgraph, c_last_exception from pypy.rpython.lltypesystem import lltype -from pypy.jit.llvalue import LLAbstractValue, const +from pypy.jit.llvalue import LLAbstractValue, AConstant, AVariable, const from pypy.jit.llvalue import ll_no_return_value from pypy.jit.llvalue import FlattenMemo, MatchMemo, FreezeMemo, UnfreezeMemo from pypy.jit.llcontainer import LLAbstractContainer, virtualcontainervalue @@ -41,7 +41,7 @@ assert len(incoming_link_args) == len(selfvalues) for linkarg, fr in zip(incoming_link_args, selfvalues): if fr.fixed: - assert isinstance(linkarg, Constant), ( + assert isinstance(linkarg, AConstant), ( "unexpected Variable %r reaching the fixed input arg %r" % (linkarg, fr)) yield linkarg @@ -51,7 +51,7 @@ def getruntimevars(self): assert not self.frozen, "getruntimevars(): not for frozen states" return [a.runtimevar for a in self.flatten()] - + def flatten(self, memo=None): if memo is None: memo = FlattenMemo() @@ -102,8 +102,8 @@ for v, a in zip(self.getlivevars(), self.args_a): a = a.unfreeze(memo, block) # try to preserve the name - if isinstance(a.runtimevar, Variable): - a.runtimevar.rename(v) + #if isinstance(a.runtimevar, Variable): + # a.runtimevar.rename(v) new_args_a.append(a) return self.__class__(new_back, new_args_a, *self.localkey()) @@ -171,10 +171,10 @@ args_a = [] for i, v in enumerate(origgraph.getargs()): if i in arghints: - a = LLAbstractValue(const(arghints[i])) + a = LLAbstractValue(AConstant(arghints[i])) a.concrete = self.policy.concrete_args else: - a = LLAbstractValue(input=v.concretetype) + a = LLAbstractValue(AVariable(v.concretetype)) args_a.append(a) graphstate = self.schedule_graph(args_a, origgraph) graphstate.complete() @@ -192,11 +192,11 @@ #print "SCHEDULE_GRAPH", graphstate return graphstate - def schedule(self, inputstate): + def schedule(self, inputstate, builder): #print "SCHEDULE", args_a, origblock frozenstate = self.schedule_getstate(inputstate) args_v = inputstate.getruntimevars() - return LinkState(args_v, frozenstate) + return LinkState(builder, args_v, frozenstate) def schedule_getstate(self, inputstate): # NOTA BENE: copyblocks can get shared between different copygraphs! @@ -253,19 +253,21 @@ exitcase = None llexitcase = None - def __init__(self, args_v, frozenstate): + def __init__(self, builder, args_v, frozenstate): self.args_v = args_v + self.args_genv = [v.getgenvar(builder) for v in args_v] self.frozenstate = frozenstate self.link = None def setreturn(self): - rgenop.closereturnlink(self.link, self.args_v[0]) + rgenop.closereturnlink(self.link, self.args_genv[0]) def settarget(self, block, blockargs): args = [] - for v1, v2 in zip(self.args_v, blockargs): - if isinstance(v2, Constant): - assert v1 == v2 + for v1, v2 in zip(self.args_genv, blockargs): + assert isinstance(v2, (AVariable, AConstant)) + if isinstance(v2, AConstant): + assert v1.value == v2.value # sanity check violating encapsulation else: args.append(v1) rgenop.closelink(self.link, args, block) @@ -370,7 +372,7 @@ merged_key = [] recompute = False for c1, c2 in zip(blockargs, next.args_v): - if isinstance(c1, Constant) and c1 != c2: + if isinstance(c1, AConstant) and c1 != c2: # incompatibility merged_key.append(None) # force a Variable recompute = True @@ -418,7 +420,7 @@ for a1, a2, k in zip(state.flatten(), builder.runningstate.flatten(), key): - if isinstance(k, Constant): + if isinstance(k, AConstant): arglist.append('%s => %s' % (a1, k)) else: arglist.append('%s => %s' % (a1, a2)) @@ -435,7 +437,7 @@ # except block of the copygraph. args_v = [builder.binding(v).forcevarorconst(builder) for v in origblock.inputargs] - ls = LinkState(args_v, frozenstate=None) + ls = LinkState(builder, args_v, frozenstate=None) raise InsertNextLink(ls) else: # finishing a handle_call_inlining(): link back to @@ -477,7 +479,7 @@ else: a = builder.bindings[origblock.exitswitch] v = a.forcevarorconst(builder) - if isinstance(v, Variable): + if isinstance(v, AVariable): newexitswitch = v links = origblock.exits else: @@ -488,7 +490,7 @@ args_a = [builder.binding(v) for v in origlink.args] nextinputstate = LLBlockState(builder.runningstate.back, args_a, origlink.target) - newlinkstate = self.interp.schedule(nextinputstate) + newlinkstate = self.interp.schedule(nextinputstate, builder) if newexitswitch is not None: newlinkstate.exitcase = origlink.exitcase newlinkstate.llexitcase = origlink.llexitcase @@ -520,7 +522,7 @@ assert len(newlinkstates) == 2 - false_link, true_link = rgenop.closeblock2(b, newexitswitch) + false_link, true_link = rgenop.closeblock2(b, newexitswitch.getgenvar(self)) cases = {False: false_link, True: true_link} for ls in newlinkstates: ls.link = cases[ls.exitcase] @@ -533,8 +535,9 @@ return rgenop.genconst(self.newblock, llvalue) def binding(self, v): + assert isinstance(v, (Constant, Variable)) if isinstance(v, Constant): - return LLAbstractValue(v) + return LLAbstractValue(AConstant(v.value, v.concretetype)) else: return self.bindings[v] @@ -564,7 +567,7 @@ # can constant-fold print 'fold:', constant_op.__name__, concretevalues concreteresult = constant_op(*concretevalues) - a_result = LLAbstractValue(const(concreteresult)) + a_result = LLAbstractValue(AConstant(concreteresult)) if any_concrete and self.interp.policy.concrete_propagate: a_result.concrete = True else: @@ -572,14 +575,23 @@ return a_result def residual(self, op, args_a): - retvar = rgenop.genop(self.newblock, op.opname, - [a.forcevarorconst(self) for a in args_a], - op.result.concretetype) - return LLAbstractValue(retvar) - - def residual_direct_call(self, name, block, args_a,): + T= op.result.concretetype + retvar = self.genop(op.opname, + [a.forcegenvarorconst(self) for a in args_a], + T) + return LLAbstractValue(AVariable(T, genvar=retvar)) + + def residual_direct_call(self, FUNCTYPE, name, target, args_a): + T = FUNCTYPE.RESULT + gen_fn_const = rgenop.gencallableconst(self.newblock, + name, + target, + FUNCTYPE) + retvar = self.genop('direct_call', [gen_fn_const] + + [a.forcegenvarorconst(self) for a in args_a], + T) + return LLAbstractValue(AVariable(T, genvar=retvar)) - return LLAbstractValue(retvar) def residualize(self, op, args_a, constant_op=None): if constant_op: @@ -723,7 +735,7 @@ parentstate = LLSuspendedBlockState(self.runningstate.back, alive_a, origblock, origposition) nextstate = LLBlockState(parentstate, args_a, origgraph.startblock) - raise InsertNextLink(self.interp.schedule(nextstate)) + raise InsertNextLink(self.interp.schedule(nextstate, self)) def handle_call_residual(self, op, origgraph, *args_a): # residual call: for now we need to force all arguments @@ -751,13 +763,9 @@ if not a.concrete: ARGS.append(a.getconcretetype()) new_args_a.append(a) - args_a = new_args_a TYPE = lltype.FuncType(ARGS, op.result.concretetype) - a_func = LLAbstractValue(rgenop.gengraphconst(self.newblock, - graphstate.name, - graphstate.startblock, - TYPE)) - return self.residual(op, [a_func] + args_a) + return self.residual_direct_call(TYPE, graphstate.name, + graphstate.startblock, new_args_a) def op_getfield(self, op, a_ptr, a_attrname): if hasllcontent(a_ptr): @@ -781,7 +789,7 @@ def op_getarraysize(self, op, a_ptr): if hasllcontent(a_ptr): - return LLAbstractValue(const(a_ptr.content.length)) + return LLAbstractValue(AConstant(a_ptr.content.length)) return self.residualize(op, [a_ptr], len) def op_getarrayitem(self, op, a_ptr, a_index): @@ -846,9 +854,10 @@ memo = FlattenMemo() a_ptr.flatten(memo) for a in memo.result: - v = a.runtimevar - if isinstance(v, Variable) and not v.concretetype._is_atomic(): - rgenop.genop(self.newblock, 'keepalive', [v], lltype.Void) + if (a.is_variable and + not a.getconcretetype()._is_atomic()): + self.genop('keepalive', [a.forcegenvarorconst()], + lltype.Void) return ll_no_return_value # High-level operation dispatcher @@ -863,7 +872,7 @@ args_a = [] for a in argtuple: if not isinstance(a, LLAbstractValue): - a = LLAbstractValue(const(a)) + a = LLAbstractValue(AConstant(a)) args_a.append(a) # end of rather XXX'edly hackish parsing Modified: pypy/dist/pypy/jit/llcontainer.py ============================================================================== --- pypy/dist/pypy/jit/llcontainer.py (original) +++ pypy/dist/pypy/jit/llcontainer.py Fri Jan 27 10:26:11 2006 @@ -1,6 +1,5 @@ -from pypy.objspace.flow.model import Variable, Constant, SpaceOperation from pypy.rpython.lltypesystem import lltype -from pypy.jit.llvalue import LLAbstractValue, const +from pypy.jit.llvalue import LLAbstractValue, AConstant, const class LLAbstractContainer(object): @@ -53,7 +52,7 @@ RESULT_TYPE = lltype.Ptr(self.T) if self.a_parent is not None: parentindex = const(self.parentindex, lltype.Void) - v_parent = self.a_parent.forcevarorconst(builder) + v_parent = self.a_parent.forcegenvarorconst(builder) v_result = builder.genop('getsubstruct', [v_parent, parentindex], RESULT_TYPE) else: @@ -82,7 +81,7 @@ assert isinstance(a_value.content, LLVirtualContainer) a_value.content.buildcontent(builder, v_subptr) else: - v_value = a_value.forcevarorconst(builder) + v_value = a_value.forcegenvarorconst(builder) self.setop(builder, v_target, name, v_value) def flatten(self, memo): @@ -216,7 +215,7 @@ a.content.parentindex = parentindex else: # primitive initialized to zero - a = LLAbstractValue(const(T._defl())) + a = LLAbstractValue(AConstant(T._defl())) return a def hasllcontent(a_ptr): Modified: pypy/dist/pypy/jit/llvalue.py ============================================================================== --- pypy/dist/pypy/jit/llvalue.py (original) +++ pypy/dist/pypy/jit/llvalue.py Fri Jan 27 10:26:11 2006 @@ -1,6 +1,26 @@ -from pypy.objspace.flow.model import Variable, Constant +from pypy.objspace.flow.model import Constant from pypy.rpython.lltypesystem import lltype from pypy.rpython import rgenop +from pypy.tool.uid import Hashable + +class AVariable(object): + def __init__(self, T, genvar=None): + self.concretetype = T + self.genvar = genvar + + def getgenvar(self, builder): + return self.genvar + +class AConstant(Hashable): + def __init__(self, value, T=None, genvar=None): + Hashable.__init__(self, value) + self.concretetype = T or lltype.typeOf(value) + self.genvar = genvar + + def getgenvar(self, builder): + if self.genvar is None: + self.genvar = builder.genconst(self.value) + return self.genvar class LLAbstractValue(object): """An abstract value, propagated through the blocks of the low-level @@ -15,16 +35,13 @@ concrete = False # concrete constants propagate eagerly - def __init__(self, runtimevar=None, content=None, input=None): - self.runtimevar = runtimevar # None or a Variable or a Constant + def __init__(self, runtimevar=None, content=None): + self.runtimevar = runtimevar # None or a AVariable or a AConstant self.content = content # None or an LLAbstractContainer self.origin = [] # list of frozen values: the sources that did or # could allow 'self' to be computed as a constant - self.input = input - + def __repr__(self): - if self.input: - return '' if self.runtimevar is None: if self.content is None: return '' @@ -33,7 +50,7 @@ else: # runtime value -- display more precisely if it's a # Variable or a Constant - if isinstance(self.runtimevar, Variable): + if isinstance(self.runtimevar, AVariable): kind = 'runtime' elif self.concrete: kind = 'concrete' @@ -43,8 +60,6 @@ def freeze(self, memo): # turn a run-time value into a frozen value - if self.input: - return LLFrozenRuntimeValue(self) if self.runtimevar is not None: if self.concrete: return LLFrozenConcreteValue(self) @@ -65,8 +80,6 @@ return frozen_dummy_value # dummy def getconcretetype(self): - if self.input: - return self.input if self.runtimevar is not None: return self.runtimevar.concretetype elif self.content is not None: @@ -78,12 +91,17 @@ if self.runtimevar is None: if self.content is None: raise ValueError("ll_dummy_value.forcevarorconst()") - self.runtimevar = self.content.build_runtime_container(builder) + genvar = self.content.build_runtime_container(builder) + assert self.content.T == genvar.concretetype.TO # sanity check + self.runtimevar = AVariable(lltype.Ptr(self.content.T), genvar=genvar) self.content = None return self.runtimevar + def forcegenvarorconst(self, builder): + return self.forcevarorconst(builder).getgenvar(builder) + def maybe_get_constant(self): - if isinstance(self.runtimevar, Constant): + if isinstance(self.runtimevar, AConstant): return self.runtimevar else: return None @@ -92,9 +110,7 @@ """Recursively flatten the LLAbstractValue into a list of run-time LLAbstractValues. """ - if self.input: - memo.result.append(self) - elif self.runtimevar is not None: + if self.runtimevar is not None: if not self.concrete: # skip concrete values, they don't need # to be present in the residual graph at all memo.result.append(self) @@ -177,12 +193,13 @@ # no need to worry about sharing here: LLFrozenRuntimeValues are # never shared propagateconst = memo.propagateconsts.next() - if isinstance(propagateconst, Constant): - v = propagateconst # allowed to propagate as a Constant - assert v.concretetype == self.concretetype + if isinstance(propagateconst, AConstant): + c = propagateconst # allowed to propagate as a Constant + assert c.concretetype == self.concretetype + result = LLAbstractValue(c) else: - v = rgenop.geninputarg(block, self.concretetype) - result = LLAbstractValue(v) + gen_v = rgenop.geninputarg(block, self.concretetype) + result = LLAbstractValue(AVariable(self.concretetype, genvar=gen_v)) result.origin.append(self) return result @@ -257,6 +274,6 @@ c.concretetype = T or lltype.typeOf(value) return c -ll_no_return_value = LLAbstractValue(const(None, lltype.Void)) +ll_no_return_value = LLAbstractValue(AConstant(None, lltype.Void)) ll_dummy_value = LLAbstractValue() frozen_dummy_value = LLFrozenDummyValue() Modified: pypy/dist/pypy/jit/vlist.py ============================================================================== --- pypy/dist/pypy/jit/vlist.py (original) +++ pypy/dist/pypy/jit/vlist.py Fri Jan 27 10:26:11 2006 @@ -1,6 +1,6 @@ from pypy.rpython.lltypesystem import lltype from pypy.rpython.rtyper import LowLevelOpList -from pypy.jit.llvalue import LLAbstractValue, const, ll_dummy_value +from pypy.jit.llvalue import LLAbstractValue, AConstant, ll_dummy_value from pypy.jit.llcontainer import LLAbstractContainer @@ -39,7 +39,7 @@ return LLVirtualList(self.T, items_a) def build_runtime_container(self, builder): - items_v = [a.forcevarorconst(builder) for a in self.items_a] + items_v = [a.forcegenvarorconst(builder) for a in self.items_a] v_result = self.T.list_builder(builder, items_v) return v_result @@ -47,10 +47,10 @@ # High-level operations def oop_len(self, op): - return LLAbstractValue(const(len(self.items_a))) + return LLAbstractValue(AConstant(len(self.items_a))) def oop_nonzero(self, op): - return LLAbstractValue(const(bool(self.items_a))) + return LLAbstractValue(AConstant(bool(self.items_a))) def oop_getitem(self, op, a_index): c_index = a_index.maybe_get_constant() Modified: pypy/dist/pypy/rpython/rgenop.py ============================================================================== --- pypy/dist/pypy/rpython/rgenop.py (original) +++ pypy/dist/pypy/rpython/rgenop.py Fri Jan 27 10:26:11 2006 @@ -28,7 +28,7 @@ block.operations.append(op) return v -def gengraphconst(block, name, target, FUNCTYPE): +def gencallableconst(block, name, target, FUNCTYPE): fptr = lltype.functionptr(FUNCTYPE, name, graph=buildgraph(target)) return genconst(block, fptr) @@ -44,6 +44,7 @@ return link def closeblock2(block, exitswitch): + assert isinstance(exitswitch, flowmodel.Variable) block.exitswitch = exitswitch false_link = flowmodel.Link([], None) false_link.exitcase = False @@ -56,6 +57,8 @@ def closelink(link, vars, targetblock): if isinstance(link, flowmodel.Link): + for v in vars: + assert isinstance(v, (flowmodel.Variable, flowmodel.Constant)) assert ([v.concretetype for v in vars] == [v.concretetype for v in targetblock.inputargs]) link.args[:] = vars Modified: pypy/dist/pypy/rpython/test/test_rlist.py ============================================================================== --- pypy/dist/pypy/rpython/test/test_rlist.py (original) +++ pypy/dist/pypy/rpython/test/test_rlist.py Fri Jan 27 10:26:11 2006 @@ -1,6 +1,5 @@ import sys from pypy.rpython.lltypesystem.lltype import * -from pypy.rpython.rtyper import LowLevelOpList from pypy.rpython.rlist import * from pypy.rpython.rslice import ll_newslice from pypy.rpython.rint import signed_repr @@ -902,12 +901,33 @@ lst.append(42) return lst + from pypy.rpython import rgenop + + class DummyBlockBuilder: + + def __init__(self): + self.newblock = rgenop.newblock() + + def genop(self, opname, args, RESULT_TYPE): + return rgenop.genop(self.newblock, opname, args, RESULT_TYPE) + + def genconst(self, llvalue): + return rgenop.genconst(self.newblock, llvalue) + + # inspection + def __getitem__(self, index): + return self.newblock.operations[index] + + def __len__(self): + return len(self.newblock.operations) + + for fn in [fixed_size_case, variable_size_case]: t = TranslationContext() t.buildannotator().build_types(fn, []) t.buildrtyper().specialize() LIST = t.graphs[0].getreturnvar().concretetype.TO - llop = LowLevelOpList(None) + llop = DummyBlockBuilder() v0 = Constant(42) v0.concretetype = Signed v1 = Variable() From ac at codespeak.net Fri Jan 27 10:33:37 2006 From: ac at codespeak.net (ac at codespeak.net) Date: Fri, 27 Jan 2006 10:33:37 +0100 (CET) Subject: [pypy-svn] r22716 - in pypy/dist/pypy/interpreter: . test Message-ID: <20060127093337.A92B227DBE@code1.codespeak.net> Author: ac Date: Fri Jan 27 10:33:37 2006 New Revision: 22716 Modified: pypy/dist/pypy/interpreter/argument.py pypy/dist/pypy/interpreter/test/test_function.py Log: Add a test for a bug in argument parsing reported by Seo Sanghyeon, and fix the bug. Modified: pypy/dist/pypy/interpreter/argument.py ============================================================================== --- pypy/dist/pypy/interpreter/argument.py (original) +++ pypy/dist/pypy/interpreter/argument.py Fri Jan 27 10:33:37 2006 @@ -244,8 +244,6 @@ ### Construction ### - blind_arguments = 0 - def __init__(self, space, args_w=None, kwds_w=None, w_stararg=None, w_starstararg=None): self.space = space @@ -403,11 +401,11 @@ input_argcount = len(args_w) + blindargs # check that no keyword argument conflicts with these - # note that for this purpose we ignore the first blind_arguments, + # note that for this purpose we ignore the first blindargs, # which were put into place by prepend(). This way, keywords do # not conflict with the hidden extra argument bound by methods. - if kwds_w and input_argcount > self.blind_arguments: - for name in argnames[self.blind_arguments:input_argcount]: + if kwds_w and input_argcount > blindargs: + for name in argnames[blindargs:input_argcount]: if name in kwds_w: raise ArgErrMultipleValues(name) Modified: pypy/dist/pypy/interpreter/test/test_function.py ============================================================================== --- pypy/dist/pypy/interpreter/test/test_function.py (original) +++ pypy/dist/pypy/interpreter/test/test_function.py Fri Jan 27 10:33:37 2006 @@ -217,7 +217,14 @@ def f(self): pass assert repr(B.f) == "" - assert repr(B().f).startswith(" Author: tismer Date: Fri Jan 27 10:52:15 2006 New Revision: 22717 Modified: pypy/dist/pypy/module/stackless/interp_coroutine.py Log: small refinements, exceptions, more to come... Modified: pypy/dist/pypy/module/stackless/interp_coroutine.py ============================================================================== --- pypy/dist/pypy/module/stackless/interp_coroutine.py (original) +++ pypy/dist/pypy/module/stackless/interp_coroutine.py Fri Jan 27 10:52:15 2006 @@ -56,12 +56,15 @@ # redirect all unhandled exceptions to the parent costate.things_to_do = True costate.temp_exc = e - if self.parent.frame is None: - self.parent = costate.main + while self.parent.frame is None: + # greenlet behavior is fine + self.parent = self.parent.parent return self._update_state(self.parent) def switch(self): if self.frame is None: + # considered a programming error. + # greenlets and tasklets have different ideas about this. raise CoroutineDamage costate.last.frame = self._update_state(self).switch() # note that last gets updated before assignment! @@ -75,7 +78,6 @@ _update_state = staticmethod(_update_state) def kill(self): - self._userdel() if costate.current is self: raise CoroutineExit costate.things_to_do = True @@ -83,6 +85,10 @@ self.parent = costate.current self.switch() + def _kill_finally(self): + self._userdel() + self.kill() + def __del__(self): # provide the necessary clean-up if this coro is left # with a frame. @@ -121,7 +127,7 @@ obj.parent = costate.current if obj is costate.del_last: costate.del_first = costate.del_last = None - obj.kill() + obj._kill_finally() else: costate.things_to_do = False @@ -152,12 +158,21 @@ return space.wrap(co) def w_bind(self, w_func, __args__): + if self.frame is not None: + raise OperationError(space.w_ValueError, space.wrap( + "cannot bind a bound Coroutine")) thunk = _AppThunk(self.space, w_func, __args__) self.bind(thunk) def w_switch(self): + if self.frame is None: + raise OperationError(space.w_ValueError, space.wrap( + "cannot switch to an unbound Coroutine")) self.switch() + def w_kill(self): + self.kill() + def __del__(self): if self.frame is not None or self.space.lookup(self, '__del__') is not None: postpone_deletion(self) @@ -185,5 +200,6 @@ bind = interp2app(AppCoroutine.w_bind, unwrap_spec=['self', W_Root, Arguments]), switch = interp2app(AppCoroutine.w_switch), + kill = interp2app(AppCoroutine.w_kill), is_zombie = GetSetProperty(AppCoroutine.w_get_is_zombie, doc=AppCoroutine.get_is_zombie.__doc__), ) From auc at codespeak.net Fri Jan 27 10:56:24 2006 From: auc at codespeak.net (auc at codespeak.net) Date: Fri, 27 Jan 2006 10:56:24 +0100 (CET) Subject: [pypy-svn] r22718 - pypy/dist/pypy/lib/logic Message-ID: <20060127095624.6974127DBE@code1.codespeak.net> Author: auc Date: Fri Jan 27 10:56:20 2006 New Revision: 22718 Added: pypy/dist/pypy/lib/logic/problems.py Removed: pypy/dist/pypy/lib/logic/test_unification.py pypy/dist/pypy/lib/logic/unification.py Modified: pypy/dist/pypy/lib/logic/computationspace.py pypy/dist/pypy/lib/logic/constraint.py pypy/dist/pypy/lib/logic/distributor.py pypy/dist/pypy/lib/logic/test_computationspace.py pypy/dist/pypy/lib/logic/test_variable.py pypy/dist/pypy/lib/logic/variable.py Log: (ale, auc) merge store and computation space - the diff contains a lot of garbage Modified: pypy/dist/pypy/lib/logic/computationspace.py ============================================================================== --- pypy/dist/pypy/lib/logic/computationspace.py (original) +++ pypy/dist/pypy/lib/logic/computationspace.py Fri Jan 27 10:56:20 2006 @@ -1,4 +1,4 @@ -## The Constraint-based computation model +# The Constraint-based computation model ## ====================================== ## A computation space collects together basic constraints (in fact @@ -177,14 +177,21 @@ ## space = ComputationSpace(fun=my_problem) -from unification import Store, var -from constraint import ConsistencyFailure -from threading import Thread, Condition +from threading import Thread, Condition, RLock, local -class Unprocessed: +from variable import EqSet, Var, \ + VariableException, NotAVariable, AlreadyInStore +from constraint import FiniteDomain, ConsistencyFailure + +EmptyDom = FiniteDomain([]) + +class Succeeded: + """It contains no choice points but a solution to + the logic program. + """ pass -class Working: +class Distributable: pass class Failed(Exception): @@ -197,38 +204,372 @@ """ pass -class Succeeded: - """It contains no choice points but a solution to - the logic program. - """ - pass +class Alternatives(object): + def __init__(self, nb_alternatives): + self._nbalt = nb_alternatives + def __eq__(self, other): + if other is None: return False + return self._nbalt == other._nbalt + + +#----------- Store Exceptions ---------------------------- +class UnboundVariable(VariableException): + def __str__(self): + return "%s has no value yet" % self.name + +class AlreadyBound(VariableException): + def __str__(self): + return "%s is already bound" % self.name + +class NotInStore(VariableException): + def __str__(self): + return "%s not in the store" % self.name + +class OutOfDomain(VariableException): + def __str__(self): + return "value not in domain of %s" % self.name + +class UnificationFailure(Exception): + def __init__(self, var1, var2, cause=None): + self.var1, self.var2 = (var1, var2) + self.cause = cause + def __str__(self): + diag = "%s %s can't be unified" + if self.cause: + diag += " because %s" % self.cause + return diag % (self.var1, self.var2) + +class IncompatibleDomains(Exception): + def __init__(self, var1, var2): + self.var1, self.var2 = (var1, var2) + def __str__(self): + return "%s %s have incompatible domains" % \ + (self.var1, self.var2) + +class Foo(object): + pass + +#---- ComputationSpace ------------------------------- class ComputationSpace(object): + """The Store consists of a set of k variables + x1,...,xk that are partitioned as follows: + * set of unbound variables that are equal + (also called equivalence sets of variables). + The variables in each set are equal to each + other but not to any other variables. + * variables bound to a number, record or procedure + (also called determined variables).""" + + def __init__(self, problem, parent=None): + self.parent = parent - def __init__(self, program, parent=None): + # current/working computation space, per thread + # self.TLS = local() + self.TLS = Foo() + self.TLS.current_cs = self + + # consistency-preserving stuff + self.in_transaction = False + self.bind_lock = RLock() + self.status = None + self.status_condition = Condition() + if parent is None: - self.store = Store() - self.root = self.store.var('root') - self.store.bind(self.root, program(self)) + self.vars = set() + # mapping of names to vars (all of them) + self.names = {} + # mapping of vars to constraints + self.var_const_map = {} + # set of all constraints + self.constraints = set() + self.root = self.var('__root__') + self.bind(self.root, problem(self)) else: - self.store = parent.store + self.vars = parent.vars + self.names = parent.names + self.var_const_map = parent.var_const_map + self.constraints = parent.constraints self.root = parent.root - self.program = program - self.parent = parent - # status - self.status = Unprocessed - self.status_condition = Condition() + # run ... self._process() + #-- Variables ---------------------------- + + def var(self, name): + """creates a variable of name name and put + it into the store""" + v = Var(name, self) + self.add_unbound(v) + return v + + def add_unbound(self, var): + """add unbound variable to the store""" + if var in self.vars: + raise AlreadyInStore(var.name) + print "adding %s to the store" % var + self.vars.add(var) + self.names[var.name] = var + # put into new singleton equiv. set + var.val = EqSet([var]) + + def set_domain(self, var, dom): + """bind variable to domain""" + assert(isinstance(var, Var) and (var in self.vars)) + if var.is_bound(): + raise AlreadyBound + var.dom = FiniteDomain(dom) + + def get_var_by_name(self, name): + try: + return self.names[name] + except KeyError: + raise NotInStore(name) + + #-- Constraints ------------------------- + + def add_constraint(self, constraint): + self.constraints.add(constraint) + for var in constraint.affectedVariables(): + self.var_const_map.setdefault(var, []) + self.var_const_map[var].append(constraint) + + def get_variables_with_a_domain(self): + varset = set() + for var in self.vars: + if var.dom != EmptyDom: varset.add(var) + return varset + + def satisfiable(self, constraint): + """ * satisfiable (k) checks that the constraint k + can be satisfied wrt its variable domains + and other constraints on these variables + * does NOT mutate the store + """ + # Satisfiability of one constraints entails + # satisfiability of the transitive closure + # of all constraints associated with the vars + # of our given constraint. + # We make a copy of the domains + # then traverse the constraints & attached vars + # to collect all (in)directly affected vars + # then compute narrow() on all (in)directly + # affected constraints. + assert constraint in self.constraints + varset = set() + constset = set() + self._compute_dependant_vars(constraint, varset, constset) + old_domains = collect_domains(varset) + + for const in constset: + try: + const.narrow() + except ConsistencyFailure: + restore_domains(old_domains) + return False + restore_domains(old_domains) + return True + + + def get_satisfying_domains(self, constraint): + assert constraint in self.constraints + varset = set() + constset = set() + self._compute_dependant_vars(constraint, varset, + constset) + old_domains = collect_domains(varset) + + for const in constset: + try: + const.narrow() + except ConsistencyFailure: + restore_domains(old_domains) + return {} + narrowed_domains = collect_domains(varset) + restore_domains(old_domains) + return narrowed_domains + + def satisfy(self, constraint): + assert constraint in self.constraints + varset = set() + constset = set() + self._compute_dependant_vars(constraint, varset, constset) + old_domains = collect_domains(varset) + + for const in constset: + try: + const.narrow() + except ConsistencyFailure: + restore_domains(old_domains) + raise + + def satisfy_all(self): + old_domains = collect_domains(self.vars) + for const in self.constraints: + try: + const.narrow() + except ConsistencyFailure: + restore_domains(old_domains) + raise + + def _compute_dependant_vars(self, constraint, varset, + constset): + if constraint in constset: return + constset.add(constraint) + for var in constraint.affectedVariables(): + varset.add(var) + dep_consts = self.var_const_map[var] + for const in dep_consts: + if const in constset: + continue + self._compute_dependant_vars(const, varset, + constset) + + #-- BIND ------------------------------------------- + + def bind(self, var, val): + """1. (unbound)Variable/(unbound)Variable or + 2. (unbound)Variable/(bound)Variable or + 3. (unbound)Variable/Value binding + """ + try: + self.bind_lock.acquire() + assert(isinstance(var, Var) and (var in self.vars)) + if var == val: + return + if _both_are_vars(var, val): + if _both_are_bound(var, val): + raise AlreadyBound(var.name) + if var._is_bound(): # 2b. var is bound, not var + self.bind(val, var) + elif val._is_bound(): # 2a.var is bound, not val + self._bind(var.val, val.val) + else: # 1. both are unbound + self._merge(var, val) + else: # 3. val is really a value + print "%s, is that you ?" % var + if var._is_bound(): + raise AlreadyBound(var.name) + self._bind(var.val, val) + finally: + self.bind_lock.release() + + + def _bind(self, eqs, val): + # print "variable - value binding : %s %s" % (eqs, val) + # bind all vars in the eqset to val + for var in eqs: + if var.dom != EmptyDom: + if val not in var.dom.get_values(): + print val, var.dom, var.dom.get_values() + # undo the half-done binding + for v in eqs: + v.val = eqs + raise OutOfDomain(var) + var.val = val + + def _merge(self, v1, v2): + for v in v1.val: + if not _compatible_domains(v, v2.val): + raise IncompatibleDomains(v1, v2) + self._really_merge(v1.val, v2.val) + + def _really_merge(self, eqs1, eqs2): + # print "unbound variables binding : %s %s" % (eqs1, eqs2) + if eqs1 == eqs2: return + # merge two equisets into one + eqs1 |= eqs2 + # let's reassign everybody to the merged eq + for var in eqs1: + var.val = eqs1 + + #-- UNIFY ------------------------------------------ + + def unify(self, x, y): + self.in_transaction = True + try: + try: + self._really_unify(x, y) + for var in self.vars: + if var.changed: + var._commit() + except Exception, cause: + for var in self.vars: + if var.changed: + var._abort() + if isinstance(cause, UnificationFailure): + raise + raise UnificationFailure(x, y, cause) + finally: + self.in_transaction = False + + def _really_unify(self, x, y): + # print "unify %s with %s" % (x,y) + if not _unifiable(x, y): raise UnificationFailure(x, y) + if not x in self.vars: + if not y in self.vars: + # duh ! x & y not vars + if x != y: raise UnificationFailure(x, y) + else: return + # same call, reverse args. order + self._unify_var_val(y, x) + elif not y in self.vars: + # x is Var, y a value + self._unify_var_val(x, y) + elif _both_are_bound(x, y): + self._unify_bound(x,y) + elif x._is_bound(): + self.bind(x,y) + else: + self.bind(y,x) + + def _unify_var_val(self, x, y): + if x.val != y: + try: + self.bind(x, y) + except AlreadyBound: + raise UnificationFailure(x, y) + + def _unify_bound(self, x, y): + # print "unify bound %s %s" % (x, y) + vx, vy = (x.val, y.val) + if type(vx) in [list, set] and isinstance(vy, type(vx)): + self._unify_iterable(x, y) + elif type(vx) is dict and isinstance(vy, type(vx)): + self._unify_mapping(x, y) + else: + if vx != vy: + raise UnificationFailure(x, y) + + def _unify_iterable(self, x, y): + print "unify sequences %s %s" % (x, y) + vx, vy = (x.val, y.val) + idx, top = (0, len(vx)) + while (idx < top): + self._really_unify(vx[idx], vy[idx]) + idx += 1 + + def _unify_mapping(self, x, y): + # print "unify mappings %s %s" % (x, y) + vx, vy = (x.val, y.val) + for xk in vx.keys(): + self._really_unify(vx[xk], vy[xk]) + +#-- Computation Space ----------------------------------------- + + def _process(self): try: - self.store.satisfy_all() + self.satisfy_all() except ConsistencyFailure: self.status = Failed else: - self.status = Succeeded + if self._distributable(): + self.status = Distributable + else: + self.status = Succeeded def _stable(self): #XXX: really ? @@ -246,12 +587,18 @@ def _distributable(self): if self.status not in (Failed, Succeeded, Merged): - return self.distributor.findSmallestDomain() > 1 + return self._distributable_domains() # in The Book : "the space has one thread that is # suspended on a choice point with two or more alternatives. # A space canhave at most one choice point; attempting to # create another gives an error." + def _distributable_domains(self): + for var in self.vars: + if var.dom.size() > 1 : + return True + return False + def set_distributor(self, dist): self.distributor = dist @@ -262,7 +609,7 @@ while not self._stable(): self.status_condition.wait() if self._distributable(): - return self.distributor.nb_subdomains() + return Alternatives(self.distributor.nb_subdomains()) return self.status finally: self.status_condition.release() @@ -309,4 +656,137 @@ self.do_something_with(result) + + + +#------------------------------------------------------- + + +def _compatible_domains(var, eqs): + """check that the domain of var is compatible + with the domains of the vars in the eqs + """ + if var.dom == EmptyDom: return True + empty = set() + for v in eqs: + if v.dom == EmptyDom: continue + if v.dom.intersection(var.dom) == empty: + return False + return True + +#-- collect / restore utilities for domains + +def collect_domains(varset): + """makes a copy of domains of a set of vars + into a var -> dom mapping + """ + dom = {} + for var in varset: + if var.dom != EmptyDom: + dom[var] = FiniteDomain(var.dom) + return dom + +def restore_domains(domains): + """sets the domain of the vars in the domains mapping + to their (previous) value + """ + for var, dom in domains.items(): + var.dom = dom + + +#-- Unifiability checks--------------------------------------- +#-- +#-- quite costly & could be merged back in unify + +def _iterable(thing): + return type(thing) in [list, set] + +def _mapping(thing): + return type(thing) is dict + +# memoizer for _unifiable +_unifiable_memo = set() + +def _unifiable(term1, term2): + global _unifiable_memo + _unifiable_memo = set() + return _really_unifiable(term1, term2) +def _really_unifiable(term1, term2): + """Checks wether two terms can be unified""" + if ((id(term1), id(term2))) in _unifiable_memo: return False + _unifiable_memo.add((id(term1), id(term2))) + # print "unifiable ? %s %s" % (term1, term2) + if _iterable(term1): + if _iterable(term2): + return _iterable_unifiable(term1, term2) + return False + if _mapping(term1) and _mapping(term2): + return _mapping_unifiable(term1, term2) + if not(isinstance(term1, Var) or isinstance(term2, Var)): + return term1 == term2 # same 'atomic' object + return True + +def _iterable_unifiable(c1, c2): + """Checks wether two iterables can be unified""" + # print "unifiable sequences ? %s %s" % (c1, c2) + if len(c1) != len(c2): return False + idx, top = (0, len(c1)) + while(idx < top): + if not _really_unifiable(c1[idx], c2[idx]): + return False + idx += 1 + return True + +def _mapping_unifiable(m1, m2): + """Checks wether two mappings can be unified""" + # print "unifiable mappings ? %s %s" % (m1, m2) + if len(m1) != len(m2): return False + if m1.keys() != m2.keys(): return False + v1, v2 = (m1.items(), m2.items()) + v1.sort() + v2.sort() + return _iterable_unifiable([e[1] for e in v1], + [e[1] for e in v2]) + +#-- Some utilities ----------------------------------------------- + +def _both_are_vars(v1, v2): + return isinstance(v1, Var) and isinstance(v2, Var) + +def _both_are_bound(v1, v2): + return v1._is_bound() and v2._is_bound() + + + +#-- +#-- the global store +from problems import dummy_problem +_store = ComputationSpace(dummy_problem) + +#-- global accessor functions +def var(name): + v = Var(name, _store) + _store.add_unbound(v) + return v + +def set_domain(var, dom): + return _store.set_domain(var, dom) + +def add_constraint(constraint): + return _store.add_constraint(constraint) + +def satisfiable(constraint): + return _store.satisfiable(constraint) + +def get_satisfying_domains(constraint): + return _store.get_satisfying_domains(constraint) + +def satisfy(constraint): + return _store.satisfy(constraint) + +def bind(var, val): + return _store.bind(var, val) + +def unify(var1, var2): + return _store.unify(var1, var2) Modified: pypy/dist/pypy/lib/logic/constraint.py ============================================================================== --- pypy/dist/pypy/lib/logic/constraint.py (original) +++ pypy/dist/pypy/lib/logic/constraint.py Fri Jan 27 10:56:20 2006 @@ -109,22 +109,27 @@ if other is None: return False return self._values == other._values + def __ne__(self, other): + return not self == other + def intersection(self, other): if other is None: return self.get_values() return self._values & other._values + #-- Constraints ------------------------------------------ +EmptyDom = FiniteDomain([]) + class AbstractConstraint(object): def __init__(self, variables): """variables is a list of variables which appear in the formula""" self._names_to_vars = {} for var in variables: - if var.dom is None: + if var.dom == EmptyDom: raise DomainlessVariables self._names_to_vars[var.name] = var - var.add_constraint(self) self._variables = variables def affectedVariables(self): @@ -194,12 +199,12 @@ """A constraint represented as a python expression.""" _FILTER_CACHE = {} - def __init__(self, variables, formula, type='fd.Expression'): + def __init__(self, variables, formula, typ='fd.Expression'): """variables is a list of variables which appear in the formula formula is a python expression that will be evaluated as a boolean""" - AbstractConstraint.__init__(self, variables) self.formula = formula - self.type = type + self.type = typ + AbstractConstraint.__init__(self, variables) try: self.filterFunc = Expression._FILTER_CACHE[formula] except KeyError: Modified: pypy/dist/pypy/lib/logic/distributor.py ============================================================================== --- pypy/dist/pypy/lib/logic/distributor.py (original) +++ pypy/dist/pypy/lib/logic/distributor.py Fri Jan 27 10:56:20 2006 @@ -19,7 +19,7 @@ """returns the variable having the smallest domain. (or one of such varibles if there is a tie) """ - vars_ = [var for var in self.c_space.store.get_variables_with_a_domain() + vars_ = [var for var in self.c_space.get_variables_with_a_domain() if var.dom.size() > 1] best = vars_[0] @@ -33,7 +33,7 @@ """returns the variable having the largest domain. (or one of such variables if there is a tie) """ - vars_ = [var for var in self.c_space.store.get_variables_with_a_domain() + vars_ = [var for var in self.c_space.get_variables_with_a_domain() if var.dom.size() > 1] best = vars_[0] @@ -52,7 +52,7 @@ """do the minimal job and let concrete class distribute variables """ self.verbose = verbose - variables = self.c_space.store.get_variables_with_a_domain() + variables = self.c_space.get_variables_with_a_domain() replicas = [] for i in range(self.nb_subdomains()): replicas.append(make_new_domains(variables)) Added: pypy/dist/pypy/lib/logic/problems.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/lib/logic/problems.py Fri Jan 27 10:56:20 2006 @@ -0,0 +1,52 @@ +import computationspace as cs +import constraint as c +import distributor as di + + +def satisfiable_problem(computation_space): + cs = computation_space + x, y, z, w = (cs.var('x'), cs.var('y'), + cs.var('z'), cs.var('w')) + x.cs_set_dom(cs, c.FiniteDomain([2, 6])) + y.cs_set_dom(cs, c.FiniteDomain([2, 3])) + z.cs_set_dom(cs, c.FiniteDomain([4, 5])) + w.cs_set_dom(cs, c.FiniteDomain([1, 4, 5, 6, 7])) + cs.add_constraint(c.Expression([x, y, z], 'x == y + z')) + cs.add_constraint(c.Expression([z, w], 'z < w')) + # set up a distribution strategy + cs.set_distributor(di.DichotomyDistributor(cs)) + return (x, w, y) + +def one_solution_problem(computation_space): + cs = computation_space + x, y, z, w = (cs.var('x'), cs.var('y'), + cs.var('z'), cs.var('w')) + x.cs_set_dom(cs, c.FiniteDomain([2, 6])) + y.cs_set_dom(cs, c.FiniteDomain([2, 3])) + z.cs_set_dom(cs, c.FiniteDomain([4, 5])) + w.cs_set_dom(cs, c.FiniteDomain([1, 4, 5])) + cs.add_constraint(c.Expression([x, y, z], 'x == y + z')) + cs.add_constraint(c.Expression([z, w], 'z < w')) + # set up a distribution strategy + cs.set_distributor(di.DichotomyDistributor(cs)) + return (x, w, y) + + +def unsatisfiable_problem(computation_space): + cs = computation_space + x, y, z, w = (cs.var('x'), cs.var('y'), + cs.var('z'), cs.var('w')) + x.cs_set_dom(cs, c.FiniteDomain([2, 6])) + y.cs_set_dom(cs, c.FiniteDomain([2, 3])) + z.cs_set_dom(cs, c.FiniteDomain([4, 5])) + w.cs_set_dom(cs, c.FiniteDomain([1])) + cs.add_constraint(c.Expression([x, y, z], 'x == y + z')) + cs.add_constraint(c.Expression([z, w], 'z < w')) + # set up a distribution strategy + cs.set_distributor(di.DichotomyDistributor(cs)) + return (x, w, y) + +def dummy_problem(computation_space): + ret = computation_space.var('__dummy__') + ret.dom = c.FiniteDomain([1, 2]) + return ret Modified: pypy/dist/pypy/lib/logic/test_computationspace.py ============================================================================== --- pypy/dist/pypy/lib/logic/test_computationspace.py (original) +++ pypy/dist/pypy/lib/logic/test_computationspace.py Fri Jan 27 10:56:20 2006 @@ -1,51 +1,412 @@ -import unification as u +from threading import Thread + import variable as v import constraint as c import computationspace as cs import distributor as di +from problems import * from py.test import raises -#-- utility ------------------ +#-- helpers ----------------- -class hdict(dict): - """a hashable dict""" - def __hash__(self): - return id(self) +#-- meat ------------------------ -#-- helpers ----------------- +class FunThread(Thread): -def satisfiable_problem(computation_space): - cs = computation_space - s = cs.store - x, y, z, w = (s.var('x'), s.var('y'), - s.var('z'), s.var('w')) - s.set_domain(x, c.FiniteDomain([2, 6])) - s.set_domain(y, c.FiniteDomain([2, 3])) - s.set_domain(z, c.FiniteDomain([4, 5])) - s.set_domain(w, c.FiniteDomain([1, 4, 5, 6, 7])) - s.add_constraint(c.Expression([x, y, z], 'x == y + z')) - s.add_constraint(c.Expression([z, w], 'z < w')) - # set up a distribution strategy - cs.set_distributor(di.DichotomyDistributor(cs)) - return (x, w, y) - -def unsatisfiable_problem(computation_space): - cs = computation_space - s = cs.store - x, y, z, w = (s.var('x'), s.var('y'), - s.var('z'), s.var('w')) - s.set_domain(x, c.FiniteDomain([2, 6])) - s.set_domain(y, c.FiniteDomain([2, 3])) - s.set_domain(z, c.FiniteDomain([4, 5])) - s.set_domain(w, c.FiniteDomain([1])) - s.add_constraint(c.Expression([x, y, z], 'x == y + z')) - s.add_constraint(c.Expression([z, w], 'z < w')) - # set up a distribution strategy - cs.set_distributor(di.DichotomyDistributor(cs)) - return (x, w, y) + def __init__(self, fun, *args): + Thread.__init__(self) + self.fun = fun + self.args = args + + def run(self): + self.fun(self, *self.args) + +class TestStoreUnification: + + def setup_method(self, meth): + cs._store = cs.ComputationSpace(dummy_problem) + + def test_already_in_store(self): + x = cs.var('x') + raises(v.AlreadyInStore, cs.var, 'x') + + def test_already_bound(self): + x = cs.var('x') + cs.bind(x, 42) + raises(cs.AlreadyBound, cs.bind, x, 42) + + def test_bind_var_var(self): + x = cs.var('x') + y = cs.var('y') + z = cs.var('z') + cs.bind(x, z) + assert x.val == cs.EqSet([x, z]) + assert y.val == cs.EqSet([y]) + assert z.val == cs.EqSet([x, z]) + + def test_bind_var_val(self): + x, y, z = cs.var('x'), cs.var('y'), cs.var('z') + cs.bind(x, z) + cs.bind(y, 42) + cs.bind(z, 3.14) + assert x.val == 3.14 + assert y.val == 42 + assert z.val == 3.14 + + def test_unify_same(self): + x,y,z,w = (cs.var('x'), cs.var('y'), + cs.var('z'), cs.var('w')) + cs.bind(x, [42, z]) + cs.bind(y, [z, 42]) + cs.bind(w, [z, 43]) + raises(cs.UnificationFailure, cs.unify, x, w) + cs.unify(x, y) + assert z.val == 42 + + def test_double_unification(self): + x, y, z = (cs.var('x'), cs.var('y'), + cs.var('z')) + cs.bind(x, 42) + cs.bind(y, z) + cs.unify(x, y) + assert z.val == 42 + cs.unify(x, y) + assert (z.val == x.val) and (x.val == y.val) + + + def test_unify_values(self): + x, y = cs.var('x'), cs.var('y') + cs.bind(x, [1, 2, 3]) + cs.bind(y, [1, 2, 3]) + cs.unify(x, y) + assert x.val == [1, 2, 3] + assert y.val == [1, 2, 3] + + def test_unify_lists_success(self): + x,y,z,w = (cs.var('x'), cs.var('y'), + cs.var('z'), cs.var('w')) + cs.bind(x, [42, z]) + cs.bind(y, [w, 44]) + cs.unify(x, y) + assert x.val == [42, z] + assert y.val == [w, 44] + assert z.val == 44 + assert w.val == 42 + + def test_unify_dicts_success(self): + x,y,z,w = (cs.var('x'), cs.var('y'), + cs.var('z'), cs.var('w')) + cs.bind(x, {1:42, 2:z}) + cs.bind(y, {1:w, 2:44}) + cs.unify(x, y) + assert x.val == {1:42, 2:z} + assert y.val == {1:w, 2:44} + assert z.val == 44 + assert w.val == 42 + + def test_unify_failure(self): + x,y,z = cs.var('x'), cs.var('y'), cs.var('z') + cs.bind(x, [42, z]) + cs.bind(y, [z, 44]) + raises(cs.UnificationFailure, cs.unify, x, y) + # check store consistency + assert x.val == [42, z] + assert y.val == [z, 44] + assert z.val == cs.EqSet([z]) + + def test_unify_failure2(self): + x,y,z,w = (cs.var('x'), cs.var('y'), + cs.var('z'), cs.var('w')) + cs.bind(x, [42, z]) + cs.bind(y, [w, 44]) + cs.bind(z, w) + assert cs._store.in_transaction == False + raises(cs.UnificationFailure, cs.unify, x, y) + assert cs._store.in_transaction == False + # check store consistency + assert x.val == [42, z] + assert y.val == [w, 44] + assert z.val == cs.EqSet([z,w]) + assert w.val == cs.EqSet([z,w]) + + def test_unify_circular(self): + x, y, z, w, a, b = (cs.var('x'), cs.var('y'), + cs.var('z'), cs.var('w'), + cs.var('a'), cs.var('b')) + cs.bind(x, [y]) + cs.bind(y, [x]) + raises(cs.UnificationFailure, cs.unify, x, y) + cs.bind(z, [1, w]) + cs.bind(w, [z, 2]) + raises(cs.UnificationFailure, cs.unify, z, w) + cs.bind(a, {1:42, 2:b}) + cs.bind(b, {1:a, 2:42}) + raises(cs.UnificationFailure, cs.unify, a, b) + # check store consistency + assert x.val == [y] + assert y.val == [x] + assert z.val == [1, w] + assert w.val == [z, 2] + assert a.val == {1:42, 2:b} + assert b.val == {1:a, 2:42} + + + def test_threads_creating_vars(self): + def create_var(thread, *args): + x = cs.var('x') + + def create_var2(thread, *args): + raises(v.AlreadyExists, cs.var, 'x') + + t1, t2 = (FunThread(create_var), + FunThread(create_var2)) + t1.start() + t2.start() + + + def test_threads_binding_vars(self): + + def do_stuff(thread, var, val): + thread.raised = False + try: + # pb. with TLS (thread-local-stuff) in + # cs class + cs.bind(var, val) + except Exception, e: + print e + thread.raised = True + assert isinstance(e, cs.AlreadyBound) + + x = cs.var('x') + vars_ = [] + for nvar in range(1000): + v = cs.var('x-'+str(nvar)) + cs.bind(x, v) + vars_.append(v) + + for var in vars_: + assert var in cs._store.vars + assert var.val == x.val + + t1, t2 = (FunThread(do_stuff, x, 42), + FunThread(do_stuff, x, 43)) + t1.start() + t2.start() + t1.join() + t2.join() + #check that every var is really bound to 42 or 43 + for var in vars_: + assert var in cs._store.vars + assert var.val == x.val + assert (t2.raised and not t1.raised) or \ + (t1.raised and not t2.raised) + + + def test_set_var_domain(self): + x = cs.var('x') + cs.set_domain(x, [1, 3, 5]) + assert x.dom == c.FiniteDomain([1, 3, 5]) + + def test_bind_with_domain(self): + x = cs.var('x') + cs.set_domain(x, [1, 2, 3]) + raises(cs.OutOfDomain, cs.bind, x, 42) + cs.bind(x, 3) + assert x.val == 3 + + def test_bind_with_incompatible_domains(self): + x, y = cs.var('x'), cs.var('y') + cs.set_domain(x, [1, 2]) + cs.set_domain(y, [3, 4]) + raises(cs.IncompatibleDomains, cs.bind, x, y) + cs.set_domain(y, [2, 4]) + cs.bind(x, y) + # check x and y are in the same equiv. set + assert x.val == y.val + + + def test_unify_with_domains(self): + x,y,z = cs.var('x'), cs.var('y'), cs.var('z') + cs.bind(x, [42, z]) + cs.bind(y, [z, 42]) + cs.set_domain(z, [1, 2, 3]) + raises(cs.UnificationFailure, cs.unify, x, y) + cs.set_domain(z, [41, 42, 43]) + cs.unify(x, y) + assert z.val == 42 + assert z.dom == c.FiniteDomain([41, 42, 43]) + + def test_add_constraint(self): + x,y,z = cs.var('x'), cs.var('y'), cs.var('z') + raises(c.DomainlessVariables, + c.Expression, [x, y, z], 'x == y + z') + x.dom = c.FiniteDomain([1, 2]) + y.dom = c.FiniteDomain([2, 3]) + z.dom = c.FiniteDomain([3, 4]) + k = c.Expression([x, y, z], 'x == y + z') + cs.add_constraint(k) + assert k in cs._store.constraints + + def test_narrowing_domains_failure(self): + x,y,z = cs.var('x'), cs.var('y'), cs.var('z') + x.dom = c.FiniteDomain([1, 2]) + y.dom = c.FiniteDomain([2, 3]) + z.dom = c.FiniteDomain([3, 4]) + k = c.Expression([x, y, z], 'x == y + z') + raises(c.ConsistencyFailure, k.narrow) + + def test_narrowing_domains_success(self): + x,y,z = cs.var('x'), cs.var('y'), cs.var('z') + x.dom = c.FiniteDomain([1, 2, 5]) + y.dom = c.FiniteDomain([2, 3]) + z.dom = c.FiniteDomain([3, 4]) + k = c.Expression([x, y, z], 'x == y + z') + k.narrow() + assert x.dom == c.FiniteDomain([5]) + assert y.dom == c.FiniteDomain([2]) + assert z.dom == c.FiniteDomain([3]) + + def test_store_satisfiable_success(self): + x,y,z = cs.var('x'), cs.var('y'), cs.var('z') + x.dom = c.FiniteDomain([1, 2, 5]) + y.dom = c.FiniteDomain([2, 3]) + z.dom = c.FiniteDomain([3, 4]) + k = c.Expression([x, y, z], 'x == y + z') + cs.add_constraint(k) + assert cs.satisfiable(k) == True + assert x.dom == c.FiniteDomain([1, 2, 5]) + assert y.dom == c.FiniteDomain([2, 3]) + assert z.dom == c.FiniteDomain([3, 4]) + + def test_store_satisfiable_failure(self): + x,y,z = cs.var('x'), cs.var('y'), cs.var('z') + x.dom = c.FiniteDomain([1, 2]) + y.dom = c.FiniteDomain([2, 3]) + z.dom = c.FiniteDomain([3, 4]) + k = c.Expression([x, y, z], 'x == y + z') + cs.add_constraint(k) + assert cs.satisfiable(k) == False + assert x.dom == c.FiniteDomain([1, 2]) + assert y.dom == c.FiniteDomain([2, 3]) + assert z.dom == c.FiniteDomain([3, 4]) + + def test_compute_dependant_vars(self): + x,y,z,w = (cs.var('x'), cs.var('y'), + cs.var('z'), cs.var('w')) + x.dom = c.FiniteDomain([1, 2, 5]) + y.dom = c.FiniteDomain([2, 3]) + z.dom = c.FiniteDomain([3, 4]) + w.dom = c.FiniteDomain([1, 4, 5]) + k1 = c.Expression([x, y, z], 'x == y + z') + k2 = c.Expression([z, w], 'z < w') + cs.add_constraint(k1) + cs.add_constraint(k2) + varset = set() + constset = set() + cs._store._compute_dependant_vars(k1, varset, constset) + assert varset == set([x, y, z, w]) + assert constset == set([k1, k2]) + + def test_satisfiable_many_const_success(self): + x,y,z,w = (cs.var('x'), cs.var('y'), + cs.var('z'), cs.var('w')) + x.dom = c.FiniteDomain([1, 2, 5]) + y.dom = c.FiniteDomain([2, 3]) + z.dom = c.FiniteDomain([3, 4]) + w.dom = c.FiniteDomain([1, 4, 5]) + k1 = c.Expression([x, y, z], 'x == y + z') + k2 = c.Expression([z, w], 'z < w') + cs.add_constraint(k1) + cs.add_constraint(k2) + assert cs.satisfiable(k1) == True + assert x.dom == c.FiniteDomain([1, 2, 5]) + assert y.dom == c.FiniteDomain([2, 3]) + assert z.dom == c.FiniteDomain([3, 4]) + assert w.dom == c.FiniteDomain([1, 4, 5]) + assert cs.satisfiable(k2) == True + assert x.dom == c.FiniteDomain([1, 2, 5]) + assert y.dom == c.FiniteDomain([2, 3]) + assert z.dom == c.FiniteDomain([3, 4]) + assert w.dom == c.FiniteDomain([1, 4, 5]) + narrowed_doms = cs.get_satisfying_domains(k1) + assert narrowed_doms == {x:c.FiniteDomain([5]), + y:c.FiniteDomain([2]), + z:c.FiniteDomain([3]), + w:c.FiniteDomain([4, 5])} + narrowed_doms = cs.get_satisfying_domains(k2) + assert narrowed_doms == {x:c.FiniteDomain([5]), + y:c.FiniteDomain([2]), + z:c.FiniteDomain([3]), + w:c.FiniteDomain([4, 5])} + + + def test_satisfiable_many_const_failure(self): + x,y,z,w = (cs.var('x'), cs.var('y'), + cs.var('z'), cs.var('w')) + x.dom = c.FiniteDomain([1, 2, 5]) + y.dom = c.FiniteDomain([2, 3]) + z.dom = c.FiniteDomain([3, 4]) + w.dom = c.FiniteDomain([1]) + k1 = c.Expression([x, y, z], 'x == y + z') + k2 = c.Expression([z, w], 'z < w') + cs.add_constraint(k1) + cs.add_constraint(k2) + assert cs.satisfiable(k1) == False + assert x.dom == c.FiniteDomain([1, 2, 5]) + assert y.dom == c.FiniteDomain([2, 3]) + assert z.dom == c.FiniteDomain([3, 4]) + assert w.dom == c.FiniteDomain([1]) + assert cs.satisfiable(k2) == False + assert x.dom == c.FiniteDomain([1, 2, 5]) + assert y.dom == c.FiniteDomain([2, 3]) + assert z.dom == c.FiniteDomain([3, 4]) + assert w.dom == c.FiniteDomain([1]) + narrowed_doms = cs.get_satisfying_domains(k1) + assert narrowed_doms == {} + narrowed_doms = cs.get_satisfying_domains(k2) + assert narrowed_doms == {} + + def test_satisfy_many_const_failure(self): + x,y,z,w = (cs.var('x'), cs.var('y'), + cs.var('z'), cs.var('w')) + x.dom = c.FiniteDomain([1, 2, 5]) + y.dom = c.FiniteDomain([2, 3]) + z.dom = c.FiniteDomain([3, 4]) + w.dom = c.FiniteDomain([1]) + k1 = c.Expression([x, y, z], 'x == y + z') + k2 = c.Expression([z, w], 'z < w') + cs.add_constraint(k1) + cs.add_constraint(k2) + raises(cs.ConsistencyFailure, cs.satisfy, k1) + assert x.dom == c.FiniteDomain([1, 2, 5]) + assert y.dom == c.FiniteDomain([2, 3]) + assert z.dom == c.FiniteDomain([3, 4]) + assert w.dom == c.FiniteDomain([1]) + raises(cs.ConsistencyFailure, cs.satisfy, k2) + assert x.dom == c.FiniteDomain([1, 2, 5]) + assert y.dom == c.FiniteDomain([2, 3]) + assert z.dom == c.FiniteDomain([3, 4]) + assert w.dom == c.FiniteDomain([1]) + + def test_satisfy_many_const_success(self): + x,y,z,w = (cs.var('x'), cs.var('y'), + cs.var('z'), cs.var('w')) + x.dom = c.FiniteDomain([1, 2, 5]) + y.dom = c.FiniteDomain([2, 3]) + z.dom = c.FiniteDomain([3, 4]) + w.dom = c.FiniteDomain([1, 4, 5]) + k1 = c.Expression([x, y, z], 'x == y + z') + k2 = c.Expression([z, w], 'z < w') + cs.add_constraint(k1) + cs.add_constraint(k2) + cs.satisfy(k2) + print x.dom + assert x.dom == c.FiniteDomain([5]) + assert y.dom == c.FiniteDomain([2]) + assert z.dom == c.FiniteDomain([3]) + assert w.dom == c.FiniteDomain([4, 5]) -#-- meat ------------------------ class TestComputationSpace: @@ -54,26 +415,30 @@ def test_bind_cs_root(self): spc = cs.ComputationSpace(satisfiable_problem) - assert 'root' in spc.store.names + assert '__root__' in spc.names assert set(['x', 'y', 'w']) == \ set([var.name for var in spc.root.val]) - def test_process_and_ask_success(self): - spc = cs.ComputationSpace(satisfiable_problem) + def test_ask_success(self): + spc = cs.ComputationSpace(one_solution_problem) assert spc.ask() == cs.Succeeded - - def test_process_and_ask_failure(self): + + def test_ask_failure(self): spc = cs.ComputationSpace(unsatisfiable_problem) assert spc.ask() == cs.Failed + def test_ask_alternatives(self): + spc = cs.ComputationSpace(satisfiable_problem) + assert spc.ask() == cs.Alternatives(2) + def test_distribute(self): spc = cs.ComputationSpace(satisfiable_problem) new_domains = [tuple(d.items()) for d in spc.distributor.distribute()] - x, y, z, w = (spc.store.get_var_by_name('x'), - spc.store.get_var_by_name('y'), - spc.store.get_var_by_name('z'), - spc.store.get_var_by_name('w')) + x, y, z, w = (spc.get_var_by_name('x'), + spc.get_var_by_name('y'), + spc.get_var_by_name('z'), + spc.get_var_by_name('w')) expected_domains = [tuple({x: c.FiniteDomain([6]), y: c.FiniteDomain([2]), z: c.FiniteDomain([4]), Modified: pypy/dist/pypy/lib/logic/test_variable.py ============================================================================== --- pypy/dist/pypy/lib/logic/test_variable.py (original) +++ pypy/dist/pypy/lib/logic/test_variable.py Fri Jan 27 10:56:20 2006 @@ -1,7 +1,10 @@ +from threading import Thread + from py.test import raises -import unification as u + +import computationspace as u import variable as v -from threading import Thread +from problems import * class Consumer(Thread): @@ -22,7 +25,7 @@ class TestVariable: def setup_method(self, meth): - u._store = u.Store() + u._store = u.ComputationSpace(dummy_problem) def test_no_same_name(self): x = u.var('x') Modified: pypy/dist/pypy/lib/logic/variable.py ============================================================================== --- pypy/dist/pypy/lib/logic/variable.py (original) +++ pypy/dist/pypy/lib/logic/variable.py Fri Jan 27 10:56:20 2006 @@ -1,7 +1,8 @@ -# First cut at representing Oz dataflow variable - import threading +from constraint import FiniteDomain + + #----------- Exceptions --------------------------------- class VariableException(Exception): def __init__(self, name): @@ -30,19 +31,15 @@ class Var(object): - def __init__(self, name, store): - if name in store.names: + def __init__(self, name, cs): + if name in cs.names: raise AlreadyInStore(name) self.name = name - self.store = store + self.cs = cs # top-level 'commited' binding self._val = NoValue - # domain in a flat world - self.dom = None # domains in multiple spaces - self.doms = {} - # constraints - self.constraints = set() + self._doms = {cs : FiniteDomain([])} # when updated in a 'transaction', keep track # of our initial value (for abort cases) self.previous = None @@ -51,7 +48,7 @@ self.mutex = threading.Lock() self.value_condition = threading.Condition(self.mutex) - # for consumption by the global store + # for consumption by the global cs def _is_bound(self): return not isinstance(self._val, EqSet) \ @@ -69,7 +66,7 @@ # value accessors def _set_val(self, val): self.value_condition.acquire() - if self.store.in_transaction: + if self.cs.in_transaction: if not self.changed: self.previous = self._val self.changed = True @@ -103,15 +100,27 @@ is_bound = _is_bound def cs_set_dom(self, cs, dom): - self.doms[cs] = dom + self._doms[cs] = dom def cs_get_dom(self, cs): - return self.doms[cs] + return self._doms[cs] #---- Concurrent public ops -------------------------- # should be used by threads that want to block on # unbound variables + + def set_dom(self, dom): + self.cs_set_dom(self.cs.TLS.current_cs, dom) + + def get_dom(self): + return self.cs_get_dom(self.cs.TLS.current_cs) + + dom = property(get_dom, set_dom) + def get(self): + """Make threads wait on the variable + being bound + """ try: self.value_condition.acquire() while not self._is_bound(): From pedronis at codespeak.net Fri Jan 27 11:14:31 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Fri, 27 Jan 2006 11:14:31 +0100 (CET) Subject: [pypy-svn] r22719 - pypy/extradoc/sprintinfo/mallorca Message-ID: <20060127101431.3996A27DBE@code1.codespeak.net> Author: pedronis Date: Fri Jan 27 11:14:29 2006 New Revision: 22719 Modified: pypy/extradoc/sprintinfo/mallorca/planning.txt Log: day planning. Modified: pypy/extradoc/sprintinfo/mallorca/planning.txt ============================================================================== --- pypy/extradoc/sprintinfo/mallorca/planning.txt (original) +++ pypy/extradoc/sprintinfo/mallorca/planning.txt Fri Jan 27 11:14:29 2006 @@ -31,8 +31,8 @@ - JIT: Arre, Samuele Graph generating ops Designed interface (see draft-jit-ideas), - started refactoring llabstractinterp to use the operations - (issues: llabsinterp was inspecting the graphs a lot ) + Refactored llabstractinterp to use the operations + Next discussion: arre, samuele, armin +... - stackless application level interface: Christian, started from the demo coroutine impl, trying to expose @@ -41,25 +41,32 @@ there on top of CPython, maybe use stackless or greenlet to provide them or make the bytecode itself on top of CPython stackless). Next step: fastest possible translation for testing. + DONE: Exposed coroutines at application-level, support for flags on + the coroutines to use for other concurrency primitives to + construct on top of them. + + - express exception raising operations in the graphs as + calls to helpers (backends could use their special + operations) + Eric, Richard ( - l3interp/offsetof/sizeof work: Progress. Progress needs either hacks and assuming Boehm or GC integration. Try to progress assuming Boehm ) - integration of the garbage collection toolkit with the - offsetof, sizeof work done: Carl Friedrich, Michael (Richard) - + DONE: offsetof, sizeof work done + Started: insert GC neeeded behaviour as graph transformations. + Carl, Michael + - improving the way we interface with external functions. (also think about how exceptions fit in there): add static ctypes functionality into RPython itself - Gerald, Stephan, Armin - glued atoi from ctypes (cheating a bit) - - Next: rtyper implementation of ctypes-like structures, - probably use Address as much as possible - - - + DONE: glued atoi from ctypes (cheating a bit) + DONE: basic annotation for ctypes structures and basics types. + Working on annotating ctypes arrays + Next would be rtyping... + Gerald, Stephan later: ====== From ericvrp at codespeak.net Fri Jan 27 11:18:43 2006 From: ericvrp at codespeak.net (ericvrp at codespeak.net) Date: Fri, 27 Jan 2006 11:18:43 +0100 (CET) Subject: [pypy-svn] r22720 - pypy/dist/pypy/translator/goal Message-ID: <20060127101843.EC6C127DBE@code1.codespeak.net> Author: ericvrp Date: Fri Jan 27 11:18:43 2006 New Revision: 22720 Modified: pypy/dist/pypy/translator/goal/bench-cronjob.py pypy/dist/pypy/translator/goal/bench-unix.py Log: updated nightly benchmark scripts Modified: pypy/dist/pypy/translator/goal/bench-cronjob.py ============================================================================== --- pypy/dist/pypy/translator/goal/bench-cronjob.py (original) +++ pypy/dist/pypy/translator/goal/bench-cronjob.py Fri Jan 27 11:18:43 2006 @@ -23,39 +23,24 @@ print cmd os.system(cmd) - #this assumes entry_point.bc is created with llvm 1.6 and pypy-llvm- is already created with the llvm c backend - b = "pypy/translator/goal/pypy-llvm-%s" % revision - cmd = "mv %s %s-llvm1_6_c" % (b, b) + cmd = "rm pypy/translator/goal/pypy-llvm-" + revision print cmd os.system(cmd) - # - b = "%spypy-llvm-%s-llvm1_6_x86" % (tmpdir, revision) + ll2bc(tmpdir) + bc2c_exe(tmpdir, revision) + bc2x86_exe(tmpdir, revision, 'x86' , '') + bc2x86_exe(tmpdir, revision, 'x86dag', '-enable-x86-dag-isel') - cmd = "llc %sentry_point.bc -f -o %s.s" % (tmpdir, b) - print cmd - os.system(cmd) - - cmd = "as %s.s -o %s.o" % (b, b) - print cmd - os.system(cmd) - - cmd = "gcc %s.o -static -lgc -lm -lpthread -pipe -o %s" % (b, b) - print cmd - os.system(cmd) - - cmd = "cp %s pypy/translator/goal" % b - print cmd - os.system(cmd) - - - # - b = "%spypy-llvm-%s-llvm1_7_c" % (tmpdir, revision) +def ll2bc(tmpdir): cmd = '~/bin/llvm-as < %sentry_point.ll | ~/bin/opt -verify -lowersetjmp -funcresolve -raiseallocs -simplifycfg -mem2reg -globalopt -globaldce -ipconstprop -deadargelim -instcombine -simplifycfg -basiccg -prune-eh -inline -simplify-libcalls -basiccg -argpromotion -raise -tailduplicate -simplifycfg -scalarrepl -instcombine -break-crit-edges -condprop -tailcallelim -simplifycfg -reassociate -loopsimplify -licm -instcombine -indvars -loop-unroll -instcombine -load-vn -gcse -sccp -instcombine -break-crit-edges -condprop -dse -mergereturn -adce -simplifycfg -deadtypeelim -constmerge -verify -globalopt -constmerge -ipsccp -deadargelim -inline -instcombine -scalarrepl -globalsmodref-aa -licm -load-vn -gcse -instcombine -simplifycfg -globaldce -f -o %sentry_point_llvm1_7.bc' % (tmpdir, tmpdir) print cmd os.system(cmd) + +def bc2c_exe(tmpdir, revision): + b = "%spypy-llvm-%s-llvm1_7_c" % (tmpdir, revision) cmd = "~/bin/llc %sentry_point_llvm1_7.bc -march=c -f -o %s.c" % (tmpdir, b) print cmd os.system(cmd) @@ -69,11 +54,9 @@ os.system(cmd) - # - b = "%spypy-llvm-%s-llvm1_7_x86" % (tmpdir, revision) - os.system('~/bin/llc -version 2>&1') - - cmd = "~/bin/llc %sentry_point_llvm1_7.bc -f -o %s.s" % (tmpdir, b) +def bc2x86_exe(tmpdir, revision, name_extra, llc_extra_options): + b = "%spypy-llvm-%s-llvm1_7_%s" % (tmpdir, revision, name_extra) + cmd = "~/bin/llc %sentry_point_llvm1_7.bc %s -f -o %s.s" % (tmpdir, llc_extra_options, b) print cmd os.system(cmd) @@ -125,8 +108,8 @@ if backends == []: backends = 'llvm c'.split() print time.ctime() - #if 'llvm' in backends: - # update_llvm() + if 'llvm' in backends: + update_llvm() update_pypy() for backend in backends: try: Modified: pypy/dist/pypy/translator/goal/bench-unix.py ============================================================================== --- pypy/dist/pypy/translator/goal/bench-unix.py (original) +++ pypy/dist/pypy/translator/goal/bench-unix.py Fri Jan 27 11:18:43 2006 @@ -43,7 +43,7 @@ return exes HEADLINE = 'date executable richards pystone' -FMT = '%-26s %-30s %6dms (%6.2fx) %6d (%6.2fx)' +FMT = '%-26s %-30s %6dms (%6.1fx) %6d (%6.1fx)' def main(): print HEADLINE From cfbolz at codespeak.net Fri Jan 27 12:46:47 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Fri, 27 Jan 2006 12:46:47 +0100 (CET) Subject: [pypy-svn] r22721 - in pypy/dist/pypy/rpython/memory: . test Message-ID: <20060127114647.BF11627DC0@code1.codespeak.net> Author: cfbolz Date: Fri Jan 27 12:46:45 2006 New Revision: 22721 Modified: pypy/dist/pypy/rpython/memory/gctransform.py pypy/dist/pypy/rpython/memory/test/test_gctransform.py Log: (mwh, cfbolz): handle pyobjptrs correctly, rename special operations to contain a "gc_" prefix. Modified: pypy/dist/pypy/rpython/memory/gctransform.py ============================================================================== --- pypy/dist/pypy/rpython/memory/gctransform.py (original) +++ pypy/dist/pypy/rpython/memory/gctransform.py Fri Jan 27 12:46:45 2006 @@ -85,12 +85,34 @@ return [op] def push_alive(self, var): + if var.concretetype.TO is lltype.PyObject: + return self.push_alive_pyobj(var) + else: + return self.push_alive_nopyobj(var) + + def push_alive_nopyobj(self, var): + result = Variable() + result.concretetype = lltype.Void + return [SpaceOperation("gc_push_alive", [var], result)] + + def push_alive_pyobj(self, var): result = Variable() result.concretetype = lltype.Void - return [SpaceOperation("push_alive", [var], result)] + return [SpaceOperation("gc_push_alive_pyobj", [var], result)] def pop_alive(self, var): + if var.concretetype.TO is lltype.PyObject: + return self.pop_alive_pyobj(var) + else: + return self.pop_alive_nopyobj(var) + + def pop_alive_nopyobj(self, var): + result = Variable() + result.concretetype = lltype.Void + return [SpaceOperation("gc_pop_alive", [var], result)] + + def pop_alive_pyobj(self, var): result = Variable() result.concretetype = lltype.Void - return [SpaceOperation("pop_alive", [var], result)] + return [SpaceOperation("gc_pop_alive_pyobj", [var], result)] Modified: pypy/dist/pypy/rpython/memory/test/test_gctransform.py ============================================================================== --- pypy/dist/pypy/rpython/memory/test/test_gctransform.py (original) +++ pypy/dist/pypy/rpython/memory/test/test_gctransform.py Fri Jan 27 12:46:45 2006 @@ -50,7 +50,7 @@ break else: assert False, "direct_call not found!" - assert ggraph.startblock.operations[i + 1].opname != 'push_alive' + assert ggraph.startblock.operations[i + 1].opname != 'gc_push_alive' def test_multiple_exits(): S = lltype.GcStruct("S", ('x', lltype.Signed)) @@ -71,17 +71,17 @@ #t.view() pop_alive_count = 0 for i, op in enumerate(fgraph.startblock.operations): - if op.opname == "pop_alive": + if op.opname == "gc_pop_alive": var, = op.args assert var.concretetype == lltype.Ptr(S) pop_alive_count += 1 - assert pop_alive_count == 1, "pop_alive not found!" + assert pop_alive_count == 1, "gc_pop_alive not found!" for link in fgraph.startblock.exits: assert len(link.args) == 2 ops = link.target.operations assert len(ops) == 1 - assert ops[0].opname == 'pop_alive' + assert ops[0].opname == 'gc_pop_alive' assert len(ops[0].args) == len(link.target.exits) == \ len(link.target.exits[0].args) == 1 dyingname = ops[0].args[0].name @@ -105,3 +105,16 @@ assert direct_calls[1].args[1].value[0].args[0] == direct_calls[0].result assert [op.args[0] for op in direct_calls[2].args[1].value] == [direct_calls[0].result, direct_calls[1].result] +def test_pyobj(): + def f(x): + if x: + a = 1 + else: + a = "1" + return int(a) + t = rtype_and_transform(f, [int], gctransform.GCTransformer) + fgraph = graphof(t, f) + gcops = [op for op in fgraph.startblock.exits[0].target.operations + if op.opname.startswith("gc_")] + for op in gcops: + assert op.opname.endswith("_pyobj") From stephan at codespeak.net Fri Jan 27 12:57:40 2006 From: stephan at codespeak.net (stephan at codespeak.net) Date: Fri, 27 Jan 2006 12:57:40 +0100 (CET) Subject: [pypy-svn] r22722 - in pypy/dist/pypy: annotation rpython/rctypes rpython/rctypes/test Message-ID: <20060127115740.8FC0127DC0@code1.codespeak.net> Author: stephan Date: Fri Jan 27 12:57:36 2006 New Revision: 22722 Modified: pypy/dist/pypy/annotation/binaryop.py pypy/dist/pypy/rpython/rctypes/implementation.py pypy/dist/pypy/rpython/rctypes/interface.py pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py Log: added support for ctypes arrays Modified: pypy/dist/pypy/annotation/binaryop.py ============================================================================== --- pypy/dist/pypy/annotation/binaryop.py (original) +++ pypy/dist/pypy/annotation/binaryop.py Fri Jan 27 12:57:36 2006 @@ -13,6 +13,7 @@ from pypy.annotation.model import SomePBC, SomeSlice, SomeFloat, s_None from pypy.annotation.model import SomeExternalObject from pypy.annotation.model import SomeAddress, SomeTypedAddressAccess +from pypy.annotation.model import SomeCTypesObject from pypy.annotation.model import unionof, UnionError, set, missing_operation, TLS from pypy.annotation.model import add_knowntypedata, merge_knowntypedata from pypy.annotation.bookkeeper import getbookkeeper @@ -732,3 +733,18 @@ class __extend__(pairtype(SomeObject, SomeAddress)): def union((s_obj, s_addr)): raise UnionError, "union of address and anything else makes no sense" + +class __extend__(pairtype(SomeCTypesObject, SomeInteger)): + def setitem((s_cto, s_index), s_value): + pass + + def getitem((s_cto, s_index)): + if s_index.is_constant() and isinstance(s_index.const, int): + index = s_index.const + try: + return s_cto.knowntype._type_.annotator_type + except AttributeError: + return SomeCTypesObject(s_cto.knowntype._type_) + else: + return SomeObject() + Modified: pypy/dist/pypy/rpython/rctypes/implementation.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/implementation.py (original) +++ pypy/dist/pypy/rpython/rctypes/implementation.py Fri Jan 27 12:57:36 2006 @@ -179,3 +179,15 @@ """ _flags_ = _FUNCFLAG_STDCALL +def RARRAY(typ,length): + answer = ARRAY(typ,length) + def compute_result_annotation(cls, *arg_s): + """ + Answer the result annotation of calling 'cls'. + """ + assert answer is cls + return SomeCTypesObject(cls) + answer.compute_result_annotation = classmethod(compute_result_annotation) + return answer + + Modified: pypy/dist/pypy/rpython/rctypes/interface.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/interface.py (original) +++ pypy/dist/pypy/rpython/rctypes/interface.py Fri Jan 27 12:57:36 2006 @@ -3,7 +3,8 @@ c_char, c_byte, c_ubyte, \ c_short, c_ushort, c_uint,\ c_long, c_ulong, c_longlong, c_ulonglong, c_float, c_double, \ - RStructure as Structure, RByref as byref, RPOINTER as POINTER + RStructure as Structure, RByref as byref, RPOINTER as POINTER, \ + RARRAY as ARRAY try: from implementation import RWinDLL as WinDLL except ImportError: Modified: pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py Fri Jan 27 12:57:36 2006 @@ -14,7 +14,8 @@ from pypy.rpython.rctypes import cdll, c_char_p, c_int, c_char, \ c_char, c_byte, c_ubyte, c_short, c_ushort, c_uint,\ c_long, c_ulong, c_longlong, c_ulonglong, c_float, c_double, \ - POINTER, Structure, byref + POINTER, Structure, byref, ARRAY + if sys.platform == 'win32': mylib = cdll.LoadLibrary('msvcrt.dll') elif sys.platform == 'linux2': @@ -105,6 +106,18 @@ a = 10 return c_float( a + 10 ) +c_int_10 = ARRAY(c_int,10) + +def py_test_annotate_array(): + return c_int_10() + +def py_test_annotate_array_content(): + my_array = c_int_10() + my_array[0] = c_int(1) + my_array[1] = 2 + + return my_array[0] + class Test_rctypes: def test_simple(self): @@ -221,3 +234,15 @@ s = a.build_types(py_test_simple_ctypes_non_const,[]) assert s.knowntype == c_float + def test_annotate_array(self): + a = RPythonAnnotator() + s = a.build_types(py_test_annotate_array,[]) + assert s.knowntype == c_int_10 + + def test_annotate_array_access(self): + t = TranslationContext() + a = t.buildannotator() + s = a.build_types(py_test_annotate_array_content,[]) + assert s.knowntype == int + #d#t.view() + From tismer at codespeak.net Fri Jan 27 13:34:03 2006 From: tismer at codespeak.net (tismer at codespeak.net) Date: Fri, 27 Jan 2006 13:34:03 +0100 (CET) Subject: [pypy-svn] r22723 - pypy/dist/pypy/module/stackless Message-ID: <20060127123403.9E63E27DC0@code1.codespeak.net> Author: tismer Date: Fri Jan 27 13:34:01 2006 New Revision: 22723 Modified: pypy/dist/pypy/module/stackless/interp_coroutine.py Log: more exposure of coroutine state to interpreter-level Modified: pypy/dist/pypy/module/stackless/interp_coroutine.py ============================================================================== --- pypy/dist/pypy/module/stackless/interp_coroutine.py (original) +++ pypy/dist/pypy/module/stackless/interp_coroutine.py Fri Jan 27 13:34:01 2006 @@ -104,6 +104,31 @@ # override this for exposed coros pass + def is_alive(self): + return self.frame is not None or self is costate.current + + def is_zombie(self): + return self.frame is not None and check_for_zombie(self) + + def get_current(): + return costate.current + get_current = staticmethod(get_current) + + def get_main(): + return costate.main + get_main = staticmethod(get_main) + + +def check_for_zombie(self): + if costate.del_first is not None: + co = costate.del_first + while True: + if co is self: + return True + co = co.parent + if co is costate.del_first: + break + return False def postpone_deletion(obj): costate.things_to_do = True @@ -136,9 +161,14 @@ class _AppThunk(object): - def __init__(self, space, w_callable, args): + def __init__(self, space, w_obj, args): self.space = space - self.w_func = w_callable + if space.lookup(w_obj, '__call__') is None: + raise OperationError( + space.w_TypeError, + space.mod(space.wrap('object %r is not callable'), + space.newtuple([w_obj]))) + self.w_func = w_obj self.args = args def call(self): @@ -158,13 +188,15 @@ return space.wrap(co) def w_bind(self, w_func, __args__): + space = self.space if self.frame is not None: raise OperationError(space.w_ValueError, space.wrap( "cannot bind a bound Coroutine")) - thunk = _AppThunk(self.space, w_func, __args__) + thunk = _AppThunk(space, w_func, __args__) self.bind(thunk) def w_switch(self): + space = self.space if self.frame is None: raise OperationError(space.w_ValueError, space.wrap( "cannot switch to an unbound Coroutine")) From rxe at codespeak.net Fri Jan 27 13:45:01 2006 From: rxe at codespeak.net (rxe at codespeak.net) Date: Fri, 27 Jan 2006 13:45:01 +0100 (CET) Subject: [pypy-svn] r22725 - pypy/dist/pypy/translator/llvm Message-ID: <20060127124501.7DB3E27DC8@code1.codespeak.net> Author: rxe Date: Fri Jan 27 13:44:59 2006 New Revision: 22725 Modified: pypy/dist/pypy/translator/llvm/arraynode.py pypy/dist/pypy/translator/llvm/gc.py pypy/dist/pypy/translator/llvm/structnode.py Log: Die varsize constructor madness! Modified: pypy/dist/pypy/translator/llvm/arraynode.py ============================================================================== --- pypy/dist/pypy/translator/llvm/arraynode.py (original) +++ pypy/dist/pypy/translator/llvm/arraynode.py Fri Jan 27 13:44:59 2006 @@ -46,20 +46,6 @@ self.db.get_machine_word(), self.db.repr_type(self.arraytype)) - def writedecl(self, codewriter): - return #XXX tmp? - - codewriter.declare(self.constructor_decl) - - def writeimpl(self, codewriter): - return #XXX tmp? - - current, indices_to_array = self.var_malloc_info() - log.writeimpl(self.ref) - gp = self.db.gcpolicy - gp.write_constructor(codewriter, self.ref, self.constructor_decl, - current, atomic=self.array._is_atomic()) - def var_malloc_info(self): return self.array, () Modified: pypy/dist/pypy/translator/llvm/gc.py ============================================================================== --- pypy/dist/pypy/translator/llvm/gc.py (original) +++ pypy/dist/pypy/translator/llvm/gc.py Fri Jan 27 13:44:59 2006 @@ -15,43 +15,7 @@ return [] def pyrex_code(self): - return '' - - #XXX tmp? -## def write_constructor(self, codewriter, ref, constructor_decl, ARRAY, -## indices_to_array=(), atomic=False): - -## # the following indices access the last element in the array -## elemtype = self.db.repr_type(ARRAY.OF) -## word = lentype = self.db.get_machine_word() -## uword = self.db.get_machine_uword() - -## codewriter.openfunc(constructor_decl) - -## # Need room for NUL terminator -## if ARRAY is STR.chars: -## codewriter.binaryop("add", "%actuallen", lentype, "%len", 1) -## else: -## codewriter.cast("%actuallen", lentype, "%len", lentype) - -## elemindices = list(indices_to_array) -## elemindices += [("uint", 1), (lentype, "%actuallen")] -## codewriter.getelementptr("%size", ref + "*", "null", elemindices) -## codewriter.cast("%usize", elemtype + "*", "%size", uword) - -## self._malloc(codewriter, "%ptr", "%usize", atomic=atomic) -## codewriter.cast("%result", "sbyte*", "%ptr", ref + "*") - -## indices_to_arraylength = tuple(indices_to_array) + (("uint", 0),) - -## # the following accesses the length field of the array -## codewriter.getelementptr("%arraylength", ref + "*", -## "%result", -## indices_to_arraylength) -## codewriter.store(lentype, "%len", "%arraylength") - -## codewriter.ret(ref + "*", "%result") -## codewriter.closefunc() + return '' def malloc(self, codewriter, targetvar, type_, size=1, atomic=False): raise NotImplementedError, 'GcPolicy should not be used directly' Modified: pypy/dist/pypy/translator/llvm/structnode.py ============================================================================== --- pypy/dist/pypy/translator/llvm/structnode.py (original) +++ pypy/dist/pypy/translator/llvm/structnode.py Fri Jan 27 13:44:59 2006 @@ -53,25 +53,6 @@ # ______________________________________________________________________ # main entry points from genllvm - def writedecl(self, codewriter): - # declaration for constructor - return #XXX - - codewriter.declare(self.constructor_decl) - - def writeimpl(self, codewriter): - return #XXX - - log.writeimpl(self.ref) - current, indices_to_array = self.var_malloc_info() - gp = self.db.gcpolicy - gp.write_constructor(codewriter, - self.ref, - self.constructor_decl, - current, - indices_to_array, - self.struct._is_atomic()) - def var_malloc_info(self): # build up a list of indices to get to the last # var-sized struct (or rather the according array) From rxe at codespeak.net Fri Jan 27 13:52:26 2006 From: rxe at codespeak.net (rxe at codespeak.net) Date: Fri, 27 Jan 2006 13:52:26 +0100 (CET) Subject: [pypy-svn] r22726 - pypy/dist/pypy/translator/llvm/demo Message-ID: <20060127125226.4A9C327DC8@code1.codespeak.net> Author: rxe Date: Fri Jan 27 13:52:24 2006 New Revision: 22726 Modified: pypy/dist/pypy/translator/llvm/demo/run.py Log: Added back 'c' option for comparison fun! Modified: pypy/dist/pypy/translator/llvm/demo/run.py ============================================================================== --- pypy/dist/pypy/translator/llvm/demo/run.py (original) +++ pypy/dist/pypy/translator/llvm/demo/run.py Fri Jan 27 13:52:24 2006 @@ -5,12 +5,28 @@ from pypy.annotation.model import SomeList, SomeString from pypy.annotation.listdef import ListDef +from pypy.translator.translator import TranslationContext +from pypy.translator.c.genc import CStandaloneBuilder + from pypy.translator.llvm.genllvm import genllvm_compile def p(): print 'Running on top of CPython:' entry_point([]) +def c(name): + s_list_of_strings = SomeList(ListDef(None, SomeString())) + s_list_of_strings.listdef.resize() + t = TranslationContext() + t.buildannotator().build_types(entry_point, [s_list_of_strings]) + t.buildrtyper().specialize() + from pypy.translator.backendopt.all import backend_optimizations + backend_optimizations(t) + cbuilder = CStandaloneBuilder(t, entry_point) + cbuilder.generate_source() + cbuilder.compile() + os.system("XXX") + def l(name): s_list_of_strings = SomeList(ListDef(None, SomeString())) s_list_of_strings.listdef.resize() @@ -32,7 +48,11 @@ elif sys.argv[1] == "l": l(name) run_all = False + elif sys.argv[1] == "c": + c(name) + run_all = False if run_all: l(name) + c(name) p() From gromit at codespeak.net Fri Jan 27 13:57:06 2006 From: gromit at codespeak.net (gromit at codespeak.net) Date: Fri, 27 Jan 2006 13:57:06 +0100 (CET) Subject: [pypy-svn] r22727 - pypy/dist/pypy/rpython/rctypes/test Message-ID: <20060127125706.2A9C227DCB@code1.codespeak.net> Author: gromit Date: Fri Jan 27 13:57:04 2006 New Revision: 22727 Modified: pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py Log: ADD: A test that treats pointers like arrays. Modified: pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py Fri Jan 27 13:57:04 2006 @@ -107,6 +107,7 @@ return c_float( a + 10 ) c_int_10 = ARRAY(c_int,10) +c_int_p_test = POINTER(c_int) def py_test_annotate_array(): return c_int_10() @@ -118,6 +119,17 @@ return my_array[0] +def py_test_annotate_pointer_content(): + # Never run this function! + # See test_annotate_pointer_access_as_array_or_whatever + # for the weird reasons why this gets annotated + my_pointer = c_int_p_test(10) + my_pointer[0] = c_int(1) + my_pointer[1] = 2 + + return my_pointer[0] + + class Test_rctypes: def test_simple(self): @@ -234,6 +246,9 @@ s = a.build_types(py_test_simple_ctypes_non_const,[]) assert s.knowntype == c_float + +class Test_array: + def test_annotate_array(self): a = RPythonAnnotator() s = a.build_types(py_test_annotate_array,[]) @@ -246,3 +261,19 @@ assert s.knowntype == int #d#t.view() + def test_annotate_pointer_access_as_array(self): + """ + Make sure that pointers work the same way as arrays, for + ctypes compatibility. + + :Note: This works because pointer and array classes both + have a _type_ attribute, that contains the type of the + object pointed to or in the case of an array the element type. + """ + t = TranslationContext() + a = t.buildannotator() + s = a.build_types(py_test_annotate_pointer_content,[]) + assert s.knowntype == int + #t#t.view() + + From tismer at codespeak.net Fri Jan 27 14:00:30 2006 From: tismer at codespeak.net (tismer at codespeak.net) Date: Fri, 27 Jan 2006 14:00:30 +0100 (CET) Subject: [pypy-svn] r22728 - pypy/dist/pypy/module/stackless Message-ID: <20060127130030.02EFF27DC8@code1.codespeak.net> Author: tismer Date: Fri Jan 27 14:00:29 2006 New Revision: 22728 Modified: pypy/dist/pypy/module/stackless/interp_coroutine.py Log: added considerations about co-existing concepts like coroutines, greenlets and tasklets Modified: pypy/dist/pypy/module/stackless/interp_coroutine.py ============================================================================== --- pypy/dist/pypy/module/stackless/interp_coroutine.py (original) +++ pypy/dist/pypy/module/stackless/interp_coroutine.py Fri Jan 27 14:00:29 2006 @@ -235,3 +235,32 @@ kill = interp2app(AppCoroutine.w_kill), is_zombie = GetSetProperty(AppCoroutine.w_get_is_zombie, doc=AppCoroutine.get_is_zombie.__doc__), ) + +""" +Considerations about "current" +------------------------------ +Both greenlets and tasklets have some perception +of a "current" object, which represents the +currently running tasklet/greenlet. + +There is an issue how to make these structures +co-exist without interference. +One possible approach is to use the coroutines +as the basic implementation and always use +one level of indirection for the higher structures. +This allows for easy coexistence. + +An alternative is to arrange things in a way that +does not interfere. Then the different classes +need to keep track of their own "current". +After a stackless task switch, stackless gets +a new current. After a greenlet's switch, greenlet +gets a new current. + +This concept has a collision, because there is also +some notation of a "main". The solution is to let +every exposed class create its own main in advance. +This is nice, because the different concepts +can completely co-exist. + +""" \ No newline at end of file From rxe at codespeak.net Fri Jan 27 14:05:47 2006 From: rxe at codespeak.net (rxe at codespeak.net) Date: Fri, 27 Jan 2006 14:05:47 +0100 (CET) Subject: [pypy-svn] r22729 - in pypy/dist/pypy/translator/llvm: . test Message-ID: <20060127130547.40F8327DC8@code1.codespeak.net> Author: rxe Date: Fri Jan 27 14:05:45 2006 New Revision: 22729 Modified: pypy/dist/pypy/translator/llvm/database.py pypy/dist/pypy/translator/llvm/test/test_lladdresses.py Log: Comment out assert until the gc hacking is settled down a bit. Modified: pypy/dist/pypy/translator/llvm/database.py ============================================================================== --- pypy/dist/pypy/translator/llvm/database.py (original) +++ pypy/dist/pypy/translator/llvm/database.py Fri Jan 27 14:05:45 2006 @@ -363,7 +363,8 @@ elif type_ is lltype.Float: repr = self.float_to_str(value) elif type_ is llmemory.Address: - assert value == NULL + # XXXXX things are happening in the gc world... + # assert value == NULL repr = 'null' else: repr = str(value) Modified: pypy/dist/pypy/translator/llvm/test/test_lladdresses.py ============================================================================== --- pypy/dist/pypy/translator/llvm/test/test_lladdresses.py (original) +++ pypy/dist/pypy/translator/llvm/test/test_lladdresses.py Fri Jan 27 14:05:45 2006 @@ -88,7 +88,6 @@ assert res == 3 def test_pointer_comparison(): - py.test.skip("XXX tmp") def f(): result = 0 for addr1 in [raw_malloc(1), NULL]: From ericvrp at codespeak.net Fri Jan 27 14:07:25 2006 From: ericvrp at codespeak.net (ericvrp at codespeak.net) Date: Fri, 27 Jan 2006 14:07:25 +0100 (CET) Subject: [pypy-svn] r22730 - in pypy/dist/pypy: rpython/raisingops translator/backendopt translator/c/test Message-ID: <20060127130725.55C4D27DCB@code1.codespeak.net> Author: ericvrp Date: Fri Jan 27 14:07:21 2006 New Revision: 22730 Added: pypy/dist/pypy/rpython/raisingops/ pypy/dist/pypy/rpython/raisingops/__init__.py pypy/dist/pypy/rpython/raisingops/raisingops.py pypy/dist/pypy/translator/backendopt/raisingop2direct_call.py Modified: pypy/dist/pypy/translator/backendopt/all.py pypy/dist/pypy/translator/backendopt/support.py pypy/dist/pypy/translator/c/test/test_backendoptimized.py Log: WIP for exception raising operations to direct_call transformation Added: pypy/dist/pypy/rpython/raisingops/__init__.py ============================================================================== Added: pypy/dist/pypy/rpython/raisingops/raisingops.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/rpython/raisingops/raisingops.py Fri Jan 27 14:07:21 2006 @@ -0,0 +1,26 @@ +from pypy.rpython.rarithmetic import r_longlong + +def _pypyop_divmod_adj(x, y, p_rem=None): + xdivy = r_longlong(x / y) + xmody = r_longlong(x - xdivy * y) + + # If the signs of x and y differ, and the remainder is non-0, + # C89 doesn't define whether xdivy is now the floor or the + # ceiling of the infinitely precise quotient. We want the floor, + # and we have it iff the remainder's sign matches y's. + if xmody and ((y ^ xmody) < 0): + + xmody += y + xdivy -= 1 + assert xmody and ((y ^ xmody) >= 0) + + #XXX was a pointer + # if p_rem + # p_rem = xmody; + return xdivy + +def int_floordiv_zer(x,y): + if y: + return _pypyop_divmod_adj(x, y) + else: + raise ZeroDivisionError("integer division") Modified: pypy/dist/pypy/translator/backendopt/all.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/all.py (original) +++ pypy/dist/pypy/translator/backendopt/all.py Fri Jan 27 14:07:21 2006 @@ -1,3 +1,4 @@ +from pypy.translator.backendopt.raisingop2direct_call import raisingop2direct_call from pypy.translator.backendopt.removenoops import remove_same_as from pypy.translator.backendopt.inline import auto_inlining from pypy.translator.backendopt.malloc import remove_simple_mallocs @@ -5,13 +6,20 @@ from pypy.translator.backendopt.propagate import propagate_all from pypy.translator.backendopt.merge_if_blocks import merge_if_blocks from pypy.translator import simplify +from pypy.translator.backendopt.escape import malloc_to_stack -def backend_optimizations(translator, inline_threshold=1, +def backend_optimizations(translator, raisingop2direct_call_all=False, + inline_threshold=1, mallocs=True, ssa_form=True, merge_if_blocks_to_switch=True, - propagate=False): + propagate=False, + heap2stack=False): + + if raisingop2direct_call_all: + raisingop2direct_call(translator) + # remove obvious no-ops for graph in translator.graphs: remove_same_as(graph) @@ -37,6 +45,9 @@ if propagate: propagate_all(translator) + if heap2stack: + malloc_to_stack(translator) + if merge_if_blocks_to_switch: for graph in translator.graphs: merge_if_blocks(graph) Added: pypy/dist/pypy/translator/backendopt/raisingop2direct_call.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/backendopt/raisingop2direct_call.py Fri Jan 27 14:07:21 2006 @@ -0,0 +1,19 @@ +from pypy.objspace.flow.model import Block, Constant, flatten +from pypy.translator.backendopt.support import log, all_operations +import pypy.rpython.raisingops +log = log.raisingop2directcall + +def raisingop2direct_call(translator): + """search for operations that could raise an exception and change that + operation into a direct_call to a function from the raisingops directory. + This function also needs to be annotated. + """ + for op in all_operations(translator): + s = op.opname + if not s.startswith('int_') and not s.startswith('uint_') and not s.startswith('float_'): + continue + if not s.endswith('_zer') and not s.endswith('_ovf') and not s.endswith('_val'): + continue + log(s) + op.args.insert(0, s) + op.opname = 'direct_call' Modified: pypy/dist/pypy/translator/backendopt/support.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/support.py (original) +++ pypy/dist/pypy/translator/backendopt/support.py Fri Jan 27 14:07:21 2006 @@ -4,3 +4,14 @@ from pypy.tool.ansi_print import ansi_log log = py.log.Producer("backendopt") py.log.setconsumer("backendopt", ansi_log) + +def graph_operations(graph): + for block in graph.iterblocks(): + for op in block.operations: + yield op + +def all_operations(translator): + for graph in translator.graphs: + for block in graph.iterblocks(): + for op in block.operations: + yield op Modified: pypy/dist/pypy/translator/c/test/test_backendoptimized.py ============================================================================== --- pypy/dist/pypy/translator/c/test/test_backendoptimized.py (original) +++ pypy/dist/pypy/translator/c/test/test_backendoptimized.py Fri Jan 27 14:07:21 2006 @@ -1,4 +1,4 @@ -import autopath +import py from pypy.translator.c.test.test_typed import TestTypedTestCase as _TestTypedTestCase from pypy.translator.backendopt.all import backend_optimizations from pypy.rpython import objectmodel @@ -174,3 +174,23 @@ y = ord(x) assert fn(y) == f(y) +class TestTypedOptimizedRaisingOps: + + class CodeGenerator(_TestTypedTestCase): + def process(self, t): + _TestTypedTestCase.process(self, t) + self.t = t + backend_optimizations(t, raisingop2direct_call_all=True) + + def test_int_floordiv_zer(self): + py.test.skip("WIP") + def f(x=int): + try: + y = 123 / x + except: + y = 456 + return y + codegenerator = self.CodeGenerator() + fn = codegenerator.getcompiled(f) + for x in (0,1,2,3,9,27,48, -9): + assert fn(x) == f(x) From rxe at codespeak.net Fri Jan 27 14:07:35 2006 From: rxe at codespeak.net (rxe at codespeak.net) Date: Fri, 27 Jan 2006 14:07:35 +0100 (CET) Subject: [pypy-svn] r22731 - in pypy/dist/pypy/translator/llvm/externs: . test Message-ID: <20060127130735.DDF9827DD1@code1.codespeak.net> Author: rxe Date: Fri Jan 27 14:07:33 2006 New Revision: 22731 Added: pypy/dist/pypy/translator/llvm/externs/ pypy/dist/pypy/translator/llvm/externs/__init__.py (contents, props changed) pypy/dist/pypy/translator/llvm/externs/ringbuffer.py (contents, props changed) pypy/dist/pypy/translator/llvm/externs/test/ pypy/dist/pypy/translator/llvm/externs/test/test_ringbuffer.py (contents, props changed) Log: (ericvrp, rxe) experimental ring malloc code ported to rpython from llvm code. Added: pypy/dist/pypy/translator/llvm/externs/__init__.py ============================================================================== Added: pypy/dist/pypy/translator/llvm/externs/ringbuffer.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/llvm/externs/ringbuffer.py Fri Jan 27 14:07:33 2006 @@ -0,0 +1,22 @@ +from pypy.rpython.memory.lladdress import raw_malloc + +class RingBufferData: + size = 8192 + entry_maxsize = 16 + + def __init__(self): + self.index = 0 + def init(self): + self.data = raw_malloc(self.size + self.entry_maxsize) + +ringbuffer = RingBufferData() + +def initialise(): + ringbuffer.init() + +def malloc_exception(nbytes): + assert nbytes <= ringbuffer.entry_maxsize + addr = ringbuffer.data + ringbuffer.index + ringbuffer.index = (ringbuffer.index + nbytes) & (ringbuffer.size - 1) + return addr + Added: pypy/dist/pypy/translator/llvm/externs/test/test_ringbuffer.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/llvm/externs/test/test_ringbuffer.py Fri Jan 27 14:07:33 2006 @@ -0,0 +1,14 @@ +import py +from pypy.translator.llvm.test.runtest import compile_function as compile +from pypy.translator.llvm.externs import ringbuffer + +def test_malloc_something(): + def f(value): + ringbuffer.initialise() + buf = ringbuffer.malloc_exception(value) + buf.signed[0] = 10 + return buf.signed[0] + + fc = compile(f, [int]) + assert fc(16) == f(16) + assert fc(10) == f(10) From auc at codespeak.net Fri Jan 27 14:10:04 2006 From: auc at codespeak.net (auc at codespeak.net) Date: Fri, 27 Jan 2006 14:10:04 +0100 (CET) Subject: [pypy-svn] r22732 - pypy/dist/pypy/lib/logic Message-ID: <20060127131004.78A2627DBE@code1.codespeak.net> Author: auc Date: Fri Jan 27 14:10:00 2006 New Revision: 22732 Modified: pypy/dist/pypy/lib/logic/computationspace.py pypy/dist/pypy/lib/logic/constraint.py pypy/dist/pypy/lib/logic/distributor.py pypy/dist/pypy/lib/logic/problems.py pypy/dist/pypy/lib/logic/test_computationspace.py pypy/dist/pypy/lib/logic/test_variable.py pypy/dist/pypy/lib/logic/variable.py Log: (ale, auc) * add clone * API change on variables/domains Modified: pypy/dist/pypy/lib/logic/computationspace.py ============================================================================== --- pypy/dist/pypy/lib/logic/computationspace.py (original) +++ pypy/dist/pypy/lib/logic/computationspace.py Fri Jan 27 14:10:00 2006 @@ -213,6 +213,10 @@ if other is None: return False return self._nbalt == other._nbalt +def NoProblem(): + """the empty problem, used by clone()""" + pass + #----------- Store Exceptions ---------------------------- class UnboundVariable(VariableException): @@ -247,9 +251,6 @@ def __str__(self): return "%s %s have incompatible domains" % \ (self.var1, self.var2) - -class Foo(object): - pass #---- ComputationSpace ------------------------------- class ComputationSpace(object): @@ -263,13 +264,6 @@ (also called determined variables).""" def __init__(self, problem, parent=None): - self.parent = parent - - # current/working computation space, per thread - # self.TLS = local() - self.TLS = Foo() - self.TLS.current_cs = self - # consistency-preserving stuff self.in_transaction = False self.bind_lock = RLock() @@ -285,16 +279,18 @@ # set of all constraints self.constraints = set() self.root = self.var('__root__') + # set up the problem self.bind(self.root, problem(self)) + # check satisfiability of the space + self._process() else: self.vars = parent.vars self.names = parent.names self.var_const_map = parent.var_const_map self.constraints = parent.constraints self.root = parent.root - - # run ... - self._process() + +#-- Store ------------------------------------------------ #-- Variables ---------------------------- @@ -320,7 +316,7 @@ assert(isinstance(var, Var) and (var in self.vars)) if var.is_bound(): raise AlreadyBound - var.dom = FiniteDomain(dom) + var.cs_set_dom(self, FiniteDomain(dom)) def get_var_by_name(self, name): try: @@ -339,7 +335,7 @@ def get_variables_with_a_domain(self): varset = set() for var in self.vars: - if var.dom != EmptyDom: varset.add(var) + if var.cs_get_dom(self) != EmptyDom: varset.add(var) return varset def satisfiable(self, constraint): @@ -361,15 +357,15 @@ varset = set() constset = set() self._compute_dependant_vars(constraint, varset, constset) - old_domains = collect_domains(varset) + old_domains = self.collect_domains(varset) for const in constset: try: const.narrow() except ConsistencyFailure: - restore_domains(old_domains) + self.restore_domains(old_domains) return False - restore_domains(old_domains) + self.restore_domains(old_domains) return True @@ -379,16 +375,16 @@ constset = set() self._compute_dependant_vars(constraint, varset, constset) - old_domains = collect_domains(varset) - + old_domains = self.collect_domains(varset) + for const in constset: try: const.narrow() except ConsistencyFailure: - restore_domains(old_domains) + self.restore_domains(old_domains) return {} - narrowed_domains = collect_domains(varset) - restore_domains(old_domains) + narrowed_domains = self.collect_domains(varset) + self.restore_domains(old_domains) return narrowed_domains def satisfy(self, constraint): @@ -396,22 +392,22 @@ varset = set() constset = set() self._compute_dependant_vars(constraint, varset, constset) - old_domains = collect_domains(varset) + old_domains = self.collect_domains(varset) for const in constset: try: const.narrow() except ConsistencyFailure: - restore_domains(old_domains) + self.restore_domains(old_domains) raise def satisfy_all(self): - old_domains = collect_domains(self.vars) + old_domains = self.collect_domains(self.vars) for const in self.constraints: try: const.narrow() except ConsistencyFailure: - restore_domains(old_domains) + self.restore_domains(old_domains) raise def _compute_dependant_vars(self, constraint, varset, @@ -426,6 +422,38 @@ continue self._compute_dependant_vars(const, varset, constset) + + def _compatible_domains(self, var, eqs): + """check that the domain of var is compatible + with the domains of the vars in the eqs + """ + if var.cs_get_dom(self) == EmptyDom: return True + empty = set() + for v in eqs: + if v.cs_get_dom(self) == EmptyDom: continue + if v.cs_get_dom(self).intersection(var.cs_get_dom(self)) == empty: + return False + return True + + #-- collect / restore utilities for domains + + def collect_domains(self, varset): + """makes a copy of domains of a set of vars + into a var -> dom mapping + """ + dom = {} + for var in varset: + if var.cs_get_dom(self) != EmptyDom: + dom[var] = var.cs_get_dom(self).copy() + return dom + + def restore_domains(self, domains): + """sets the domain of the vars in the domains mapping + to their (previous) value + """ + for var, dom in domains.items(): + var.cs_set_dom(self, dom) + #-- BIND ------------------------------------------- @@ -461,9 +489,8 @@ # print "variable - value binding : %s %s" % (eqs, val) # bind all vars in the eqset to val for var in eqs: - if var.dom != EmptyDom: - if val not in var.dom.get_values(): - print val, var.dom, var.dom.get_values() + if var.cs_get_dom(self) != EmptyDom: + if val not in var.cs_get_dom(self).get_values(): # undo the half-done binding for v in eqs: v.val = eqs @@ -472,7 +499,7 @@ def _merge(self, v1, v2): for v in v1.val: - if not _compatible_domains(v, v2.val): + if not self._compatible_domains(v, v2.val): raise IncompatibleDomains(v1, v2) self._really_merge(v1.val, v2.val) @@ -595,7 +622,7 @@ def _distributable_domains(self): for var in self.vars: - if var.dom.size() > 1 : + if var.cs_get_dom(self).size() > 1 : return True return False @@ -614,6 +641,14 @@ finally: self.status_condition.release() + def clone(self): + spc = ComputationSpace(NoProblem, parent=self) + for var in spc.vars: + var.cs_set_dom(spc, var.cs_get_dom(self).copy()) + return spc + + def inject(self, restricting_problem): + pass def make_children(self): for dommap in self.distributor.distribute(): @@ -654,44 +689,8 @@ # (a) might be more Oz-ish, maybe allowing some very fancy # stuff with distribution or whatever self.do_something_with(result) - - - - - -#------------------------------------------------------- - - -def _compatible_domains(var, eqs): - """check that the domain of var is compatible - with the domains of the vars in the eqs - """ - if var.dom == EmptyDom: return True - empty = set() - for v in eqs: - if v.dom == EmptyDom: continue - if v.dom.intersection(var.dom) == empty: - return False - return True -#-- collect / restore utilities for domains -def collect_domains(varset): - """makes a copy of domains of a set of vars - into a var -> dom mapping - """ - dom = {} - for var in varset: - if var.dom != EmptyDom: - dom[var] = FiniteDomain(var.dom) - return dom - -def restore_domains(domains): - """sets the domain of the vars in the domains mapping - to their (previous) value - """ - for var, dom in domains.items(): - var.dom = dom #-- Unifiability checks--------------------------------------- @@ -762,31 +761,31 @@ #-- #-- the global store from problems import dummy_problem -_store = ComputationSpace(dummy_problem) +_cs = ComputationSpace(dummy_problem) #-- global accessor functions def var(name): - v = Var(name, _store) - _store.add_unbound(v) + v = Var(name, _cs) + _cs.add_unbound(v) return v def set_domain(var, dom): - return _store.set_domain(var, dom) + return _cs.set_domain(var, dom) def add_constraint(constraint): - return _store.add_constraint(constraint) + return _cs.add_constraint(constraint) def satisfiable(constraint): - return _store.satisfiable(constraint) + return _cs.satisfiable(constraint) def get_satisfying_domains(constraint): - return _store.get_satisfying_domains(constraint) + return _cs.get_satisfying_domains(constraint) def satisfy(constraint): - return _store.satisfy(constraint) + return _cs.satisfy(constraint) def bind(var, val): - return _store.bind(var, val) + return _cs.bind(var, val) def unify(var1, var2): - return _store.unify(var1, var2) + return _cs.unify(var1, var2) Modified: pypy/dist/pypy/lib/logic/constraint.py ============================================================================== --- pypy/dist/pypy/lib/logic/constraint.py (original) +++ pypy/dist/pypy/lib/logic/constraint.py Fri Jan 27 14:10:00 2006 @@ -123,11 +123,12 @@ class AbstractConstraint(object): - def __init__(self, variables): + def __init__(self, c_space, variables): """variables is a list of variables which appear in the formula""" + self.cs = c_space self._names_to_vars = {} for var in variables: - if var.dom == EmptyDom: + if var.cs_get_dom(self.cs) == EmptyDom: raise DomainlessVariables self._names_to_vars[var.name] = var self._variables = variables @@ -199,12 +200,12 @@ """A constraint represented as a python expression.""" _FILTER_CACHE = {} - def __init__(self, variables, formula, typ='fd.Expression'): + def __init__(self, c_space, variables, formula, typ='fd.Expression'): """variables is a list of variables which appear in the formula formula is a python expression that will be evaluated as a boolean""" self.formula = formula self.type = typ - AbstractConstraint.__init__(self, variables) + AbstractConstraint.__init__(self, c_space, variables) try: self.filterFunc = Expression._FILTER_CACHE[formula] except KeyError: @@ -224,7 +225,7 @@ variables = [] kwargs = {} for variable in self._variables: - domain = variable.dom + domain = variable.cs_get_dom(self.cs) values = domain.get_values() variables.append((domain.size(), [variable, values, 0, len(values)])) kwargs[variable.name] = values[0] @@ -269,7 +270,7 @@ try: for var, keep in result_cache.iteritems(): - domain = self._names_to_vars[var].dom + domain = self._names_to_vars[var].cs_get_dom(self.cs) domain.remove_values([val for val in domain if val not in keep]) except ConsistencyFailure: Modified: pypy/dist/pypy/lib/logic/distributor.py ============================================================================== --- pypy/dist/pypy/lib/logic/distributor.py (original) +++ pypy/dist/pypy/lib/logic/distributor.py Fri Jan 27 14:10:00 2006 @@ -1,10 +1,10 @@ import math, random -def make_new_domains(variables): +def make_new_domains(cs, variables): """updates variables with copied domains indexed by computation space""" new_doms = {} for var in variables: - new_doms[var] = var.dom.copy() + new_doms[var] = var.cs_get_dom(cs).copy() return new_doms class AbstractDistributor(object): @@ -12,19 +12,19 @@ def __init__(self, c_space, nb_subspaces=2): self.nb_subspaces = nb_subspaces - self.c_space = c_space + self.cs = c_space self.verbose = 0 def findSmallestDomain(self): """returns the variable having the smallest domain. (or one of such varibles if there is a tie) """ - vars_ = [var for var in self.c_space.get_variables_with_a_domain() - if var.dom.size() > 1] + vars_ = [var for var in self.cs.get_variables_with_a_domain() + if var.cs_get_dom(self.cs).size() > 1] best = vars_[0] for var in vars_: - if var.dom.size() < best.dom.size(): + if var.cs_get_dom(self.cs).size() < best.cs_get_dom(self.cs).size(): best = var return best @@ -33,12 +33,12 @@ """returns the variable having the largest domain. (or one of such variables if there is a tie) """ - vars_ = [var for var in self.c_space.get_variables_with_a_domain() - if var.dom.size() > 1] + vars_ = [var for var in self.cs.get_variables_with_a_domain() + if var.cs_get_dom(self.cs).size() > 1] best = vars_[0] for var in vars_: - if var.dom.size() > best.dom.size(): + if var.cs_get_dom(self.cs).size() > best.cs_get_dom(self.cs).size(): best = var return best @@ -52,10 +52,10 @@ """do the minimal job and let concrete class distribute variables """ self.verbose = verbose - variables = self.c_space.get_variables_with_a_domain() + variables = self.cs.get_variables_with_a_domain() replicas = [] for i in range(self.nb_subdomains()): - replicas.append(make_new_domains(variables)) + replicas.append(make_new_domains(self.cs, variables)) modified_domains = self._distribute(*replicas) for domain in modified_domains: domain.reset_flags() @@ -119,9 +119,9 @@ """See AbstractDistributor""" self.__to_split = self.findSmallestDomain() if self.nb_subspaces: - return min(self.nb_subspaces, self.__to_split.dom.size()) #domains[self.__to_split].size()) + return min(self.nb_subspaces, self.__to_split.cs_get_dom(self.cs).size()) #domains[self.__to_split].size()) else: - return self.__to_split.dom.size() # domains[self.__to_split].size() + return self.__to_split.cs_get_dom(self.cs).size() # domains[self.__to_split].size() def _distribute(self, *args): """See AbstractDistributor""" Modified: pypy/dist/pypy/lib/logic/problems.py ============================================================================== --- pypy/dist/pypy/lib/logic/problems.py (original) +++ pypy/dist/pypy/lib/logic/problems.py Fri Jan 27 14:10:00 2006 @@ -11,8 +11,8 @@ y.cs_set_dom(cs, c.FiniteDomain([2, 3])) z.cs_set_dom(cs, c.FiniteDomain([4, 5])) w.cs_set_dom(cs, c.FiniteDomain([1, 4, 5, 6, 7])) - cs.add_constraint(c.Expression([x, y, z], 'x == y + z')) - cs.add_constraint(c.Expression([z, w], 'z < w')) + cs.add_constraint(c.Expression(cs, [x, y, z], 'x == y + z')) + cs.add_constraint(c.Expression(cs, [z, w], 'z < w')) # set up a distribution strategy cs.set_distributor(di.DichotomyDistributor(cs)) return (x, w, y) @@ -25,8 +25,8 @@ y.cs_set_dom(cs, c.FiniteDomain([2, 3])) z.cs_set_dom(cs, c.FiniteDomain([4, 5])) w.cs_set_dom(cs, c.FiniteDomain([1, 4, 5])) - cs.add_constraint(c.Expression([x, y, z], 'x == y + z')) - cs.add_constraint(c.Expression([z, w], 'z < w')) + cs.add_constraint(c.Expression(cs, [x, y, z], 'x == y + z')) + cs.add_constraint(c.Expression(cs, [z, w], 'z < w')) # set up a distribution strategy cs.set_distributor(di.DichotomyDistributor(cs)) return (x, w, y) @@ -40,8 +40,8 @@ y.cs_set_dom(cs, c.FiniteDomain([2, 3])) z.cs_set_dom(cs, c.FiniteDomain([4, 5])) w.cs_set_dom(cs, c.FiniteDomain([1])) - cs.add_constraint(c.Expression([x, y, z], 'x == y + z')) - cs.add_constraint(c.Expression([z, w], 'z < w')) + cs.add_constraint(c.Expression(cs, [x, y, z], 'x == y + z')) + cs.add_constraint(c.Expression(cs, [z, w], 'z < w')) # set up a distribution strategy cs.set_distributor(di.DichotomyDistributor(cs)) return (x, w, y) Modified: pypy/dist/pypy/lib/logic/test_computationspace.py ============================================================================== --- pypy/dist/pypy/lib/logic/test_computationspace.py (original) +++ pypy/dist/pypy/lib/logic/test_computationspace.py Fri Jan 27 14:10:00 2006 @@ -7,8 +7,6 @@ from problems import * from py.test import raises -#-- helpers ----------------- - #-- meat ------------------------ @@ -25,7 +23,7 @@ class TestStoreUnification: def setup_method(self, meth): - cs._store = cs.ComputationSpace(dummy_problem) + cs._cs = cs.ComputationSpace(dummy_problem) def test_already_in_store(self): x = cs.var('x') @@ -121,9 +119,9 @@ cs.bind(x, [42, z]) cs.bind(y, [w, 44]) cs.bind(z, w) - assert cs._store.in_transaction == False + assert cs._cs.in_transaction == False raises(cs.UnificationFailure, cs.unify, x, y) - assert cs._store.in_transaction == False + assert cs._cs.in_transaction == False # check store consistency assert x.val == [42, z] assert y.val == [w, 44] @@ -186,7 +184,7 @@ vars_.append(v) for var in vars_: - assert var in cs._store.vars + assert var in cs._cs.vars assert var.val == x.val t1, t2 = (FunThread(do_stuff, x, 42), @@ -197,7 +195,7 @@ t2.join() #check that every var is really bound to 42 or 43 for var in vars_: - assert var in cs._store.vars + assert var in cs._cs.vars assert var.val == x.val assert (t2.raised and not t1.raised) or \ (t1.raised and not t2.raised) @@ -206,7 +204,7 @@ def test_set_var_domain(self): x = cs.var('x') cs.set_domain(x, [1, 3, 5]) - assert x.dom == c.FiniteDomain([1, 3, 5]) + assert x.cs_get_dom(cs._cs) == c.FiniteDomain([1, 3, 5]) def test_bind_with_domain(self): x = cs.var('x') @@ -235,100 +233,100 @@ cs.set_domain(z, [41, 42, 43]) cs.unify(x, y) assert z.val == 42 - assert z.dom == c.FiniteDomain([41, 42, 43]) + assert z.cs_get_dom(cs._cs) == c.FiniteDomain([41, 42, 43]) def test_add_constraint(self): x,y,z = cs.var('x'), cs.var('y'), cs.var('z') raises(c.DomainlessVariables, - c.Expression, [x, y, z], 'x == y + z') - x.dom = c.FiniteDomain([1, 2]) - y.dom = c.FiniteDomain([2, 3]) - z.dom = c.FiniteDomain([3, 4]) - k = c.Expression([x, y, z], 'x == y + z') + c.Expression, cs._cs, [x, y, z], 'x == y + z') + x.cs_set_dom(cs._cs, c.FiniteDomain([1, 2])) + y.cs_set_dom(cs._cs, c.FiniteDomain([2, 3])) + z.cs_set_dom(cs._cs, c.FiniteDomain([3, 4])) + k = c.Expression(cs._cs, [x, y, z], 'x == y + z') cs.add_constraint(k) - assert k in cs._store.constraints + assert k in cs._cs.constraints def test_narrowing_domains_failure(self): x,y,z = cs.var('x'), cs.var('y'), cs.var('z') - x.dom = c.FiniteDomain([1, 2]) - y.dom = c.FiniteDomain([2, 3]) - z.dom = c.FiniteDomain([3, 4]) - k = c.Expression([x, y, z], 'x == y + z') + x.cs_set_dom(cs._cs, c.FiniteDomain([1, 2])) + y.cs_set_dom(cs._cs, c.FiniteDomain([2, 3])) + z.cs_set_dom(cs._cs, c.FiniteDomain([3, 4])) + k = c.Expression(cs._cs, [x, y, z], 'x == y + z') raises(c.ConsistencyFailure, k.narrow) def test_narrowing_domains_success(self): x,y,z = cs.var('x'), cs.var('y'), cs.var('z') - x.dom = c.FiniteDomain([1, 2, 5]) - y.dom = c.FiniteDomain([2, 3]) - z.dom = c.FiniteDomain([3, 4]) - k = c.Expression([x, y, z], 'x == y + z') + x.cs_set_dom(cs._cs, c.FiniteDomain([1, 2, 5])) + y.cs_set_dom(cs._cs, c.FiniteDomain([2, 3])) + z.cs_set_dom(cs._cs, c.FiniteDomain([3, 4])) + k = c.Expression(cs._cs, [x, y, z], 'x == y + z') k.narrow() - assert x.dom == c.FiniteDomain([5]) - assert y.dom == c.FiniteDomain([2]) - assert z.dom == c.FiniteDomain([3]) - - def test_store_satisfiable_success(self): - x,y,z = cs.var('x'), cs.var('y'), cs.var('z') - x.dom = c.FiniteDomain([1, 2, 5]) - y.dom = c.FiniteDomain([2, 3]) - z.dom = c.FiniteDomain([3, 4]) - k = c.Expression([x, y, z], 'x == y + z') - cs.add_constraint(k) - assert cs.satisfiable(k) == True - assert x.dom == c.FiniteDomain([1, 2, 5]) - assert y.dom == c.FiniteDomain([2, 3]) - assert z.dom == c.FiniteDomain([3, 4]) - - def test_store_satisfiable_failure(self): - x,y,z = cs.var('x'), cs.var('y'), cs.var('z') - x.dom = c.FiniteDomain([1, 2]) - y.dom = c.FiniteDomain([2, 3]) - z.dom = c.FiniteDomain([3, 4]) - k = c.Expression([x, y, z], 'x == y + z') - cs.add_constraint(k) - assert cs.satisfiable(k) == False - assert x.dom == c.FiniteDomain([1, 2]) - assert y.dom == c.FiniteDomain([2, 3]) - assert z.dom == c.FiniteDomain([3, 4]) + assert x.cs_get_dom(cs._cs) == c.FiniteDomain([5]) + assert y.cs_get_dom(cs._cs) == c.FiniteDomain([2]) + assert z.cs_get_dom(cs._cs) == c.FiniteDomain([3]) def test_compute_dependant_vars(self): x,y,z,w = (cs.var('x'), cs.var('y'), cs.var('z'), cs.var('w')) - x.dom = c.FiniteDomain([1, 2, 5]) - y.dom = c.FiniteDomain([2, 3]) - z.dom = c.FiniteDomain([3, 4]) - w.dom = c.FiniteDomain([1, 4, 5]) - k1 = c.Expression([x, y, z], 'x == y + z') - k2 = c.Expression([z, w], 'z < w') + x.cs_set_dom(cs._cs, c.FiniteDomain([1, 2, 5])) + y.cs_set_dom(cs._cs, c.FiniteDomain([2, 3])) + z.cs_set_dom(cs._cs, c.FiniteDomain([3, 4])) + w.cs_set_dom(cs._cs, c.FiniteDomain([1, 4, 5])) + k1 = c.Expression(cs._cs, [x, y, z], 'x == y + z') + k2 = c.Expression(cs._cs, [z, w], 'z < w') cs.add_constraint(k1) cs.add_constraint(k2) varset = set() constset = set() - cs._store._compute_dependant_vars(k1, varset, constset) + cs._cs._compute_dependant_vars(k1, varset, constset) assert varset == set([x, y, z, w]) assert constset == set([k1, k2]) + def test_store_satisfiable_success(self): + x,y,z = cs.var('x'), cs.var('y'), cs.var('z') + x.cs_set_dom(cs._cs, c.FiniteDomain([1, 2, 5])) + y.cs_set_dom(cs._cs, c.FiniteDomain([2, 3])) + z.cs_set_dom(cs._cs, c.FiniteDomain([3, 4])) + k = c.Expression(cs._cs, [x, y, z], 'x == y + z') + cs.add_constraint(k) + assert cs.satisfiable(k) == True + assert x.cs_get_dom(cs._cs) == c.FiniteDomain([1, 2, 5]) + assert y.cs_get_dom(cs._cs) == c.FiniteDomain([2, 3]) + assert z.cs_get_dom(cs._cs) == c.FiniteDomain([3, 4]) + + def test_store_satisfiable_failure(self): + x,y,z = cs.var('x'), cs.var('y'), cs.var('z') + x.cs_set_dom(cs._cs, c.FiniteDomain([1, 2])) + y.cs_set_dom(cs._cs, c.FiniteDomain([2, 3])) + z.cs_set_dom(cs._cs, c.FiniteDomain([3, 4])) + k = c.Expression(cs._cs, [x, y, z], 'x == y + z') + cs.add_constraint(k) + assert cs.satisfiable(k) == False + assert x.cs_get_dom(cs._cs) == c.FiniteDomain([1, 2]) + assert y.cs_get_dom(cs._cs) == c.FiniteDomain([2, 3]) + assert z.cs_get_dom(cs._cs) == c.FiniteDomain([3, 4]) + def test_satisfiable_many_const_success(self): x,y,z,w = (cs.var('x'), cs.var('y'), cs.var('z'), cs.var('w')) - x.dom = c.FiniteDomain([1, 2, 5]) - y.dom = c.FiniteDomain([2, 3]) - z.dom = c.FiniteDomain([3, 4]) - w.dom = c.FiniteDomain([1, 4, 5]) - k1 = c.Expression([x, y, z], 'x == y + z') - k2 = c.Expression([z, w], 'z < w') + x.cs_set_dom(cs._cs, c.FiniteDomain([1, 2, 5])) + y.cs_set_dom(cs._cs, c.FiniteDomain([2, 3])) + z.cs_set_dom(cs._cs, c.FiniteDomain([3, 4])) + w.cs_set_dom(cs._cs, c.FiniteDomain([1, 4, 5])) + k1 = c.Expression(cs._cs, [x, y, z], 'x == y + z') + k2 = c.Expression(cs._cs, [z, w], 'z < w') cs.add_constraint(k1) cs.add_constraint(k2) assert cs.satisfiable(k1) == True - assert x.dom == c.FiniteDomain([1, 2, 5]) - assert y.dom == c.FiniteDomain([2, 3]) - assert z.dom == c.FiniteDomain([3, 4]) - assert w.dom == c.FiniteDomain([1, 4, 5]) + assert x.cs_get_dom(cs._cs) == c.FiniteDomain([1, 2, 5]) + assert y.cs_get_dom(cs._cs) == c.FiniteDomain([2, 3]) + assert z.cs_get_dom(cs._cs) == c.FiniteDomain([3, 4]) + assert w.cs_get_dom(cs._cs) == c.FiniteDomain([1, 4, 5]) assert cs.satisfiable(k2) == True - assert x.dom == c.FiniteDomain([1, 2, 5]) - assert y.dom == c.FiniteDomain([2, 3]) - assert z.dom == c.FiniteDomain([3, 4]) - assert w.dom == c.FiniteDomain([1, 4, 5]) + assert x.cs_get_dom(cs._cs) == c.FiniteDomain([1, 2, 5]) + assert y.cs_get_dom(cs._cs) == c.FiniteDomain([2, 3]) + assert z.cs_get_dom(cs._cs) == c.FiniteDomain([3, 4]) + assert w.cs_get_dom(cs._cs) == c.FiniteDomain([1, 4, 5]) narrowed_doms = cs.get_satisfying_domains(k1) assert narrowed_doms == {x:c.FiniteDomain([5]), y:c.FiniteDomain([2]), @@ -344,24 +342,24 @@ def test_satisfiable_many_const_failure(self): x,y,z,w = (cs.var('x'), cs.var('y'), cs.var('z'), cs.var('w')) - x.dom = c.FiniteDomain([1, 2, 5]) - y.dom = c.FiniteDomain([2, 3]) - z.dom = c.FiniteDomain([3, 4]) - w.dom = c.FiniteDomain([1]) - k1 = c.Expression([x, y, z], 'x == y + z') - k2 = c.Expression([z, w], 'z < w') + x.cs_set_dom(cs._cs, c.FiniteDomain([1, 2, 5])) + y.cs_set_dom(cs._cs, c.FiniteDomain([2, 3])) + z.cs_set_dom(cs._cs, c.FiniteDomain([3, 4])) + w.cs_set_dom(cs._cs, c.FiniteDomain([1])) + k1 = c.Expression(cs._cs, [x, y, z], 'x == y + z') + k2 = c.Expression(cs._cs, [z, w], 'z < w') cs.add_constraint(k1) cs.add_constraint(k2) assert cs.satisfiable(k1) == False - assert x.dom == c.FiniteDomain([1, 2, 5]) - assert y.dom == c.FiniteDomain([2, 3]) - assert z.dom == c.FiniteDomain([3, 4]) - assert w.dom == c.FiniteDomain([1]) + assert x.cs_get_dom(cs._cs) == c.FiniteDomain([1, 2, 5]) + assert y.cs_get_dom(cs._cs) == c.FiniteDomain([2, 3]) + assert z.cs_get_dom(cs._cs) == c.FiniteDomain([3, 4]) + assert w.cs_get_dom(cs._cs) == c.FiniteDomain([1]) assert cs.satisfiable(k2) == False - assert x.dom == c.FiniteDomain([1, 2, 5]) - assert y.dom == c.FiniteDomain([2, 3]) - assert z.dom == c.FiniteDomain([3, 4]) - assert w.dom == c.FiniteDomain([1]) + assert x.cs_get_dom(cs._cs) == c.FiniteDomain([1, 2, 5]) + assert y.cs_get_dom(cs._cs) == c.FiniteDomain([2, 3]) + assert z.cs_get_dom(cs._cs) == c.FiniteDomain([3, 4]) + assert w.cs_get_dom(cs._cs) == c.FiniteDomain([1]) narrowed_doms = cs.get_satisfying_domains(k1) assert narrowed_doms == {} narrowed_doms = cs.get_satisfying_domains(k2) @@ -370,42 +368,42 @@ def test_satisfy_many_const_failure(self): x,y,z,w = (cs.var('x'), cs.var('y'), cs.var('z'), cs.var('w')) - x.dom = c.FiniteDomain([1, 2, 5]) - y.dom = c.FiniteDomain([2, 3]) - z.dom = c.FiniteDomain([3, 4]) - w.dom = c.FiniteDomain([1]) - k1 = c.Expression([x, y, z], 'x == y + z') - k2 = c.Expression([z, w], 'z < w') + x.cs_set_dom(cs._cs, c.FiniteDomain([1, 2, 5])) + y.cs_set_dom(cs._cs, c.FiniteDomain([2, 3])) + z.cs_set_dom(cs._cs, c.FiniteDomain([3, 4])) + w.cs_set_dom(cs._cs, c.FiniteDomain([1])) + k1 = c.Expression(cs._cs, [x, y, z], 'x == y + z') + k2 = c.Expression(cs._cs, [z, w], 'z < w') cs.add_constraint(k1) cs.add_constraint(k2) raises(cs.ConsistencyFailure, cs.satisfy, k1) - assert x.dom == c.FiniteDomain([1, 2, 5]) - assert y.dom == c.FiniteDomain([2, 3]) - assert z.dom == c.FiniteDomain([3, 4]) - assert w.dom == c.FiniteDomain([1]) + assert x.cs_get_dom(cs._cs) == c.FiniteDomain([1, 2, 5]) + assert y.cs_get_dom(cs._cs) == c.FiniteDomain([2, 3]) + assert z.cs_get_dom(cs._cs) == c.FiniteDomain([3, 4]) + assert w.cs_get_dom(cs._cs) == c.FiniteDomain([1]) raises(cs.ConsistencyFailure, cs.satisfy, k2) - assert x.dom == c.FiniteDomain([1, 2, 5]) - assert y.dom == c.FiniteDomain([2, 3]) - assert z.dom == c.FiniteDomain([3, 4]) - assert w.dom == c.FiniteDomain([1]) + assert x.cs_get_dom(cs._cs) == c.FiniteDomain([1, 2, 5]) + assert y.cs_get_dom(cs._cs) == c.FiniteDomain([2, 3]) + assert z.cs_get_dom(cs._cs) == c.FiniteDomain([3, 4]) + assert w.cs_get_dom(cs._cs) == c.FiniteDomain([1]) def test_satisfy_many_const_success(self): x,y,z,w = (cs.var('x'), cs.var('y'), cs.var('z'), cs.var('w')) - x.dom = c.FiniteDomain([1, 2, 5]) - y.dom = c.FiniteDomain([2, 3]) - z.dom = c.FiniteDomain([3, 4]) - w.dom = c.FiniteDomain([1, 4, 5]) - k1 = c.Expression([x, y, z], 'x == y + z') - k2 = c.Expression([z, w], 'z < w') + x.cs_set_dom(cs._cs, c.FiniteDomain([1, 2, 5])) + y.cs_set_dom(cs._cs, c.FiniteDomain([2, 3])) + z.cs_set_dom(cs._cs, c.FiniteDomain([3, 4])) + w.cs_set_dom(cs._cs, c.FiniteDomain([1, 4, 5])) + k1 = c.Expression(cs._cs, [x, y, z], 'x == y + z') + k2 = c.Expression(cs._cs, [z, w], 'z < w') cs.add_constraint(k1) cs.add_constraint(k2) cs.satisfy(k2) - print x.dom - assert x.dom == c.FiniteDomain([5]) - assert y.dom == c.FiniteDomain([2]) - assert z.dom == c.FiniteDomain([3]) - assert w.dom == c.FiniteDomain([4, 5]) + print x.cs_get_dom(cs._cs) + assert x.cs_get_dom(cs._cs) == c.FiniteDomain([5]) + assert y.cs_get_dom(cs._cs) == c.FiniteDomain([2]) + assert z.cs_get_dom(cs._cs) == c.FiniteDomain([3]) + assert w.cs_get_dom(cs._cs) == c.FiniteDomain([4, 5]) class TestComputationSpace: @@ -422,6 +420,11 @@ def test_ask_success(self): spc = cs.ComputationSpace(one_solution_problem) assert spc.ask() == cs.Succeeded + + def test_double_ask(self): + spc = cs.ComputationSpace(one_solution_problem) + assert spc.ask() == cs.Succeeded + assert spc.ask() == cs.Succeeded def test_ask_failure(self): spc = cs.ComputationSpace(unsatisfiable_problem) @@ -460,4 +463,19 @@ #print hash(expected_domains[0]), hash(expected_domains[1]) #assert set(new_domains) == set(expected_domains) - + def test_clone(self): + spc = cs.ComputationSpace(satisfiable_problem) + w = spc.get_var_by_name('w') + assert spc.ask() == cs.Alternatives(2) + new_spc = spc.clone() + new_spc.add_constraint(c.Expression(new_spc, [w], 'w == 5')) + new_spc._process() + assert spc.ask() == cs.Alternatives(2) + assert new_spc.ask() == cs.Succeeded + assert w.cs_get_dom(spc) == c.FiniteDomain([5, 6, 7]) + assert w.cs_get_dom(new_spc) == c.FiniteDomain([5]) + +## with-space(spc): +## assert w.dom == ... [5, 6, 7] +## with-space(new_spc): +## assert w.dom == ... [5] Modified: pypy/dist/pypy/lib/logic/test_variable.py ============================================================================== --- pypy/dist/pypy/lib/logic/test_variable.py (original) +++ pypy/dist/pypy/lib/logic/test_variable.py Fri Jan 27 14:10:00 2006 @@ -25,7 +25,7 @@ class TestVariable: def setup_method(self, meth): - u._store = u.ComputationSpace(dummy_problem) + u._cs = u.ComputationSpace(dummy_problem) def test_no_same_name(self): x = u.var('x') @@ -33,8 +33,8 @@ def test_get_by_name(self): x = u.var('x') - assert x == u._store.get_var_by_name('x') - raises(u.NotInStore, u._store.get_var_by_name, 'y') + assert x == u._cs.get_var_by_name('x') + raises(u.NotInStore, u._cs.get_var_by_name, 'y') def test_one_thread_reading_one_var(self): cons = Consumer() Modified: pypy/dist/pypy/lib/logic/variable.py ============================================================================== --- pypy/dist/pypy/lib/logic/variable.py (original) +++ pypy/dist/pypy/lib/logic/variable.py Fri Jan 27 14:10:00 2006 @@ -109,13 +109,13 @@ # should be used by threads that want to block on # unbound variables - def set_dom(self, dom): - self.cs_set_dom(self.cs.TLS.current_cs, dom) +## def set_dom(self, dom): +## self.cs_set_dom(self.cs, dom) - def get_dom(self): - return self.cs_get_dom(self.cs.TLS.current_cs) +## def get_dom(self): +## return self.cs_get_dom(self.cs) - dom = property(get_dom, set_dom) +## dom = property(get_dom, set_dom) def get(self): """Make threads wait on the variable From cfbolz at codespeak.net Fri Jan 27 15:00:02 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Fri, 27 Jan 2006 15:00:02 +0100 (CET) Subject: [pypy-svn] r22739 - pypy/extradoc/talk/mallorca-talk-2006 Message-ID: <20060127140002.2A96927DCB@code1.codespeak.net> Author: cfbolz Date: Fri Jan 27 14:59:58 2006 New Revision: 22739 Added: pypy/extradoc/talk/mallorca-talk-2006/ (props changed) pypy/extradoc/talk/mallorca-talk-2006/interpreter-overview.png - copied unchanged from r22720, pypy/eu-tracking/talk/review1/image/interpreter-overview.png pypy/extradoc/talk/mallorca-talk-2006/run_example1.py (contents, props changed) pypy/extradoc/talk/mallorca-talk-2006/talk.txt - copied, changed from r22720, pypy/extradoc/talk/22c3/hpk-tech.txt pypy/extradoc/talk/mallorca-talk-2006/translation-overview.png - copied unchanged from r22720, pypy/extradoc/talk/22c3/translation-overview.png pypy/extradoc/talk/mallorca-talk-2006/wp5_example.py (contents, props changed) Log: add the slides for the mallorca talk Added: pypy/extradoc/talk/mallorca-talk-2006/run_example1.py ============================================================================== --- (empty file) +++ pypy/extradoc/talk/mallorca-talk-2006/run_example1.py Fri Jan 27 14:59:58 2006 @@ -0,0 +1,21 @@ +from pypy.translator.interactive import Translation +from pypy.annotation.model import SomeInteger + +try: + import rlcompleter2 + rlcompleter2.setup() +except ImportError: + pass + +import wp5_example + +t = Translation(wp5_example.is_prime) +t.annotate([SomeInteger(nonneg=True)]) +t.view() +t.rtype() +t.view() +t.backendopt(backend="c") +t.view() +f = t.compile() +print "value returned by the compiled function with argument 5" +print f(5) Added: pypy/extradoc/talk/mallorca-talk-2006/wp5_example.py ============================================================================== --- (empty file) +++ pypy/extradoc/talk/mallorca-talk-2006/wp5_example.py Fri Jan 27 14:59:58 2006 @@ -0,0 +1,51 @@ +def find_divisors(n): + divisors = [] + for i in range(1, n + 1): + if n % i == 0: + divisors.append(i) + return divisors + +def is_prime(n): + divisors = find_divisors(n) + return len(divisors) == 2 + + + + + + + + + + + + + + + + + + + +class Expr(object): + pass + +class Value(Expr): + def __init__(self, value): + self.value = value + + def eval(self): + return self.value + +class Plus(Expr): + def __init__(self, expr1, expr2): + self.expr1 = expr1 + self.expr2 = expr2 + + def eval(self): + return self.expr1.eval() + self.expr2.eval() + +expr = Plus(Value(5), Plus(Value(30), Value(7))) + +def example(): + return expr.eval() From cfbolz at codespeak.net Fri Jan 27 15:08:21 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Fri, 27 Jan 2006 15:08:21 +0100 (CET) Subject: [pypy-svn] r22740 - in pypy/extradoc/talk/mallorca-talk-2006: . ui Message-ID: <20060127140821.CAB9A27DCF@code1.codespeak.net> Author: cfbolz Date: Fri Jan 27 15:08:19 2006 New Revision: 22740 Added: pypy/extradoc/talk/mallorca-talk-2006/ui/ - copied from r22720, pypy/eu-tracking/talk/review1/ui/ Modified: pypy/extradoc/talk/mallorca-talk-2006/talk.txt Log: add css, fix footer Modified: pypy/extradoc/talk/mallorca-talk-2006/talk.txt ============================================================================== --- pypy/extradoc/talk/mallorca-talk-2006/talk.txt (original) +++ pypy/extradoc/talk/mallorca-talk-2006/talk.txt Fri Jan 27 15:08:19 2006 @@ -257,8 +257,6 @@ -http://codespeak.net/pypy - .. |bullet| unicode:: U+02022 -.. footer:: Holger Krekel, Carl Friedrich Bolz (merlinux) |bullet| 22C3 |bullet| 28th December 2005 +.. footer:: Samuele Pedroni, Carl Friedrich Bolz, Armin Rigo |bullet| UIB |bullet| 27th January 2006 From gromit at codespeak.net Fri Jan 27 15:15:48 2006 From: gromit at codespeak.net (gromit at codespeak.net) Date: Fri, 27 Jan 2006 15:15:48 +0100 (CET) Subject: [pypy-svn] r22741 - in pypy/dist/pypy: annotation rpython/rctypes/test Message-ID: <20060127141548.06E8727DC2@code1.codespeak.net> Author: gromit Date: Fri Jan 27 15:15:45 2006 New Revision: 22741 Modified: pypy/dist/pypy/annotation/binaryop.py pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py Log: ADD: (stephan, gromit) Annotation for slice access on ctypes's arras ADD: checking for out bounds ctypes array access in getitem FIX: a bug in getitem annotation ADD: some tests for the error checking code Modified: pypy/dist/pypy/annotation/binaryop.py ============================================================================== --- pypy/dist/pypy/annotation/binaryop.py (original) +++ pypy/dist/pypy/annotation/binaryop.py Fri Jan 27 15:15:45 2006 @@ -18,6 +18,7 @@ from pypy.annotation.model import add_knowntypedata, merge_knowntypedata from pypy.annotation.bookkeeper import getbookkeeper from pypy.objspace.flow.model import Variable +from pypy.annotation.listdef import ListDef # convenience only! def immutablevalue(x): @@ -739,12 +740,33 @@ pass def getitem((s_cto, s_index)): - if s_index.is_constant() and isinstance(s_index.const, int): - index = s_index.const + if s_index.is_constant(): + idx = s_index.const + knowntype = s_cto.knowntype try: - return s_cto.knowntype._type_.annotator_type + length = knowntype._length_ except AttributeError: - return SomeCTypesObject(s_cto.knowntype._type_) - else: - return SomeObject() + pass + else: + if idx < 0: + idx += length + if idx >= length: + raise IndexError( "invalid index" ) + elif idx < 0: + raise IndexError( "invalid index" ) + try: + return s_cto.knowntype._type_.annotator_type + except AttributeError: + return SomeCTypesObject(s_cto.knowntype._type_) + +class __extend__(pairtype(SomeCTypesObject, SomeSlice)): + def setitem((s_cto, s_slice), s_iterable): + pass + + def getitem((s_cto, s_slice)): + try: + listdef = ListDef(None, s_cto.knowntype._type_.annotator_type) + except AttributeError: + listdef = ListDef(None, SomeCTypesObject(s_cto.knowntype._type_)) + return SomeList(listdef) Modified: pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py Fri Jan 27 15:15:45 2006 @@ -129,6 +129,29 @@ return my_pointer[0] +def py_test_annotate_array_slice_content(): + my_array = c_int_10() + #f#my_array[0:7] = c_int(1) * 7 + my_array[0:5] = range(5) + + return my_array[0:5] + +def py_test_annotate_array_content_variable_index(): + my_array = c_int_10() + my_array[2] = 2 + sum = 0 + for idx in range(10): + sum += my_array[idx] + + return sum + +def py_test_annotate_array_content_index_error_on_positive_index(): + my_array = c_int_10() + return my_array[10] + +def py_test_annotate_array_content_index_error_on_negative_index(): + my_array = c_int_10() + return my_array[-10] class Test_rctypes: @@ -276,4 +299,31 @@ assert s.knowntype == int #t#t.view() + def test_annotate_array_slice_access(self): + t = TranslationContext() + a = t.buildannotator() + s = a.build_types(py_test_annotate_array_slice_content,[]) + #t#t.view() + #d#print "v90:", s, type(s) + assert s.knowntype == list + s.listdef.listitem.s_value.knowntype == int + + def test_annotate_array_access_variable(self): + t = TranslationContext() + a = t.buildannotator() + s = a.build_types(py_test_annotate_array_content_variable_index,[]) + assert s.knowntype == int + #t#t.view() + + def test_annotate_array_access_index_error_on_positive_index(self): + t = TranslationContext() + a = t.buildannotator() + + py.test.raises(IndexError, "s = a.build_types(py_test_annotate_array_content_index_error_on_positive_index,[])") + + def test_annotate_array_access_index_error_on_negative_index(self): + t = TranslationContext() + a = t.buildannotator() + + py.test.raises(IndexError, "s = a.build_types(py_test_annotate_array_content_index_error_on_negative_index,[])") From gromit at codespeak.net Fri Jan 27 17:29:49 2006 From: gromit at codespeak.net (gromit at codespeak.net) Date: Fri, 27 Jan 2006 17:29:49 +0100 (CET) Subject: [pypy-svn] r22744 - pypy/dist/pypy/rpython/rctypes/test Message-ID: <20060127162949.0B0A927DC8@code1.codespeak.net> Author: gromit Date: Fri Jan 27 17:29:47 2006 New Revision: 22744 Modified: pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py Log: FIX: Fixed a one of error in the index error test code Modified: pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py Fri Jan 27 17:29:47 2006 @@ -151,7 +151,8 @@ def py_test_annotate_array_content_index_error_on_negative_index(): my_array = c_int_10() - return my_array[-10] + return my_array[-11] + class Test_rctypes: From auc at codespeak.net Fri Jan 27 17:37:12 2006 From: auc at codespeak.net (auc at codespeak.net) Date: Fri, 27 Jan 2006 17:37:12 +0100 (CET) Subject: [pypy-svn] r22745 - in pypy/dist/pypy/lib/logic: . basic_store store_without_cs Message-ID: <20060127163712.0606327DCD@code1.codespeak.net> Author: auc Date: Fri Jan 27 17:37:08 2006 New Revision: 22745 Added: pypy/dist/pypy/lib/logic/basic_store/ - copied from r22714, pypy/dist/pypy/lib/logic/store_without_cs/ pypy/dist/pypy/lib/logic/state.py Removed: pypy/dist/pypy/lib/logic/store_without_cs/ Modified: pypy/dist/pypy/lib/logic/computationspace.py pypy/dist/pypy/lib/logic/distributor.py pypy/dist/pypy/lib/logic/test_computationspace.py pypy/dist/pypy/lib/logic/variable.py Log: (ale, auc) * basic choose / commit implementation (cooperation between space and distributor) * lot of fixes * rename store_without_cs basic_store Modified: pypy/dist/pypy/lib/logic/computationspace.py ============================================================================== --- pypy/dist/pypy/lib/logic/computationspace.py (original) +++ pypy/dist/pypy/lib/logic/computationspace.py Fri Jan 27 17:37:08 2006 @@ -179,30 +179,16 @@ from threading import Thread, Condition, RLock, local +from state import Succeeded, Distributable, Failed, Merged + from variable import EqSet, Var, \ VariableException, NotAVariable, AlreadyInStore from constraint import FiniteDomain, ConsistencyFailure +from distributor import DefaultDistributor -EmptyDom = FiniteDomain([]) - -class Succeeded: - """It contains no choice points but a solution to - the logic program. - """ - pass - -class Distributable: - pass -class Failed(Exception): - pass +EmptyDom = FiniteDomain([]) -class Merged: - """Its constraint store has been added to a parent. - Any further operation operation on the space is - an error. - """ - pass class Alternatives(object): @@ -262,6 +248,9 @@ other but not to any other variables. * variables bound to a number, record or procedure (also called determined variables).""" + + _nb_choices = 0 + def __init__(self, problem, parent=None): # consistency-preserving stuff @@ -269,6 +258,7 @@ self.bind_lock = RLock() self.status = None self.status_condition = Condition() + self.distributor = DefaultDistributor(self) if parent is None: self.vars = set() @@ -283,6 +273,8 @@ self.bind(self.root, problem(self)) # check satisfiability of the space self._process() + if self.status == Distributable: + self.distributor.start() else: self.vars = parent.vars self.names = parent.names @@ -290,6 +282,13 @@ self.constraints = parent.constraints self.root = parent.root + # create a unique choice point + self.CHOICE = self._make_choice_var() + + def __del__(self): + self.status = Failed + self.bind(self.CHOICE, 0) + #-- Store ------------------------------------------------ #-- Variables ---------------------------- @@ -615,6 +614,7 @@ def _distributable(self): if self.status not in (Failed, Succeeded, Merged): return self._distributable_domains() + return False # in The Book : "the space has one thread that is # suspended on a choice point with two or more alternatives. # A space canhave at most one choice point; attempting to @@ -645,11 +645,44 @@ spc = ComputationSpace(NoProblem, parent=self) for var in spc.vars: var.cs_set_dom(spc, var.cs_get_dom(self).copy()) + # check satisfiability of the space + spc._process() + if spc.status == Distributable: + spc.distributor.start() return spc def inject(self, restricting_problem): + """add additional stuff into a space""" pass + def commit(self, choice): + """if self is distributable, causes the Choose call in the + space to complete and return some_number as a result. This + may cause the spzce to resume execution. + some_number must satisfy 1= Author: mwh Date: Fri Jan 27 17:38:50 2006 New Revision: 22746 Modified: pypy/dist/pypy/rpython/memory/gctransform.py pypy/dist/pypy/rpython/memory/test/test_gctransform.py Log: (cfbolz,mwh) Fix logic for what to do with variables passed across links in the case that more than one variable is passed. Add a general consistency check that would have caught this and apply it to every block of every graph that gets transformed. Modified: pypy/dist/pypy/rpython/memory/gctransform.py ============================================================================== --- pypy/dist/pypy/rpython/memory/gctransform.py (original) +++ pypy/dist/pypy/rpython/memory/gctransform.py Fri Jan 27 17:38:50 2006 @@ -37,11 +37,18 @@ self.links_to_split = {} # link -> vars to pop_alive across the link for block in graph.iterblocks(): self.transform_block(block) - for link, vars in self.links_to_split.iteritems(): + for link, livecounts in self.links_to_split.iteritems(): newops = [] - for var in vars: - newops.extend(self.pop_alive(var)) - insert_empty_block(None, link, newops) + for var, livecount in livecounts.iteritems(): + for i in range(livecount): + newops.extend(self.pop_alive(var)) + for i in range(-livecount): + newops.extend(self.push_alive(var)) + if newops: + if len(link.prevblock.exits) == 1: + link.prevblock.operations.extend(newops) + else: + insert_empty_block(None, link, newops) def transform_block(self, block): newops = [] @@ -60,10 +67,6 @@ if len(block.exits) == 0: # everything is fine already for returnblocks and exceptblocks pass - elif len(block.exits) == 1: - for var in livevars: - if var not in block.exits[0].args: - newops.extend(self.pop_alive(var)) else: deadinallexits = sets.Set(livevars) for link in block.exits: @@ -71,9 +74,11 @@ for var in deadinallexits: newops.extend(self.pop_alive(var)) for link in block.exits: - deadvarsforlink = sets.Set(livevars) - deadinallexits - sets.Set(link.args) - if deadvarsforlink: - self.links_to_split[link] = deadvarsforlink + livecounts = dict.fromkeys(sets.Set(livevars) - deadinallexits, 1) + for v in link.args: + if v in livecounts: + livecounts[v] -= 1 + self.links_to_split[link] = livecounts if newops: block.operations = newops Modified: pypy/dist/pypy/rpython/memory/test/test_gctransform.py ============================================================================== --- pypy/dist/pypy/rpython/memory/test/test_gctransform.py (original) +++ pypy/dist/pypy/rpython/memory/test/test_gctransform.py Fri Jan 27 17:38:50 2006 @@ -1,6 +1,25 @@ from pypy.rpython.memory import gctransform +from pypy.rpython.memory.gctransform import var_needsgc from pypy.translator.translator import TranslationContext, graphof from pypy.rpython.lltypesystem import lltype +from pypy.objspace.flow.model import Variable + +def checkblock(block): + vars_in = len([v for v in block.inputargs if var_needsgc(v)]) + push_alives = len([op for op in block.operations + if op.opname.startswith('gc_push_alive')]) + pop_alives = len([op for op in block.operations + if op.opname.startswith('gc_pop_alive')]) + calls = len([op for op in block.operations + if 'direct_call' in op.opname and var_needsgc(op.result)]) + if pop_alives == len(block.operations): + # it's a block we inserted + return + for link in block.exits: + vars_out = len([v for v in link.args + if isinstance(v, Variable) and var_needsgc(v)]) + assert vars_in + push_alives + calls == pop_alives + vars_out + def rtype_and_transform(func, inputtypes, transformcls): t = TranslationContext() @@ -9,6 +28,9 @@ transformer = transformcls(t.graphs) transformer.transform() t.checkgraphs() + for graph in t.graphs: + for block in graph.iterblocks(): + checkblock(block) return t def test_simple(): @@ -105,6 +127,21 @@ assert direct_calls[1].args[1].value[0].args[0] == direct_calls[0].result assert [op.args[0] for op in direct_calls[2].args[1].value] == [direct_calls[0].result, direct_calls[1].result] +def test_multiply_passed_var(): + S = lltype.GcStruct("S", ('x', lltype.Signed)) + def f(x): + if x: + a = lltype.malloc(S) + a.x = 1 + b = a + else: + a = lltype.malloc(S) + a.x = 1 + b = lltype.malloc(S) + b.x = 2 + return a.x + b.x + t = rtype_and_transform(f, [int], gctransform.GCTransformer) + def test_pyobj(): def f(x): if x: From mwh at codespeak.net Fri Jan 27 18:02:09 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Fri, 27 Jan 2006 18:02:09 +0100 (CET) Subject: [pypy-svn] r22747 - in pypy/dist/pypy/translator/c: . test Message-ID: <20060127170209.E75CD27B4C@code1.codespeak.net> Author: mwh Date: Fri Jan 27 18:02:06 2006 New Revision: 22747 Added: pypy/dist/pypy/translator/c/newfuncgen.py - copied unchanged from r22720, pypy/dist/pypy/translator/c/funcgen.py pypy/dist/pypy/translator/c/test/test_newgc.py (contents, props changed) Modified: pypy/dist/pypy/translator/c/genc.py pypy/dist/pypy/translator/c/node.py Log: (cfbolz,mwh) hacks towards being able to use a custom function code generator, the idea being to make this code generator work with the new style of handling gc operations. Modified: pypy/dist/pypy/translator/c/genc.py ============================================================================== --- pypy/dist/pypy/translator/c/genc.py (original) +++ pypy/dist/pypy/translator/c/genc.py Fri Jan 27 18:02:06 2006 @@ -19,12 +19,14 @@ _compiled = False symboltable = None stackless = False + use_new_funcgen = False - def __init__(self, translator, entrypoint, gcpolicy=None, libraries=None, thread_enabled=False): + def __init__(self, translator, entrypoint, gcpolicy=None, libraries=None, thread_enabled=False, use_new_funcgen=False): self.translator = translator self.entrypoint = entrypoint self.gcpolicy = gcpolicy self.thread_enabled = thread_enabled + self.use_new_funcgen = use_new_funcgen if libraries is None: libraries = [] @@ -38,6 +40,7 @@ if self.stackless: from pypy.translator.c.stackless import StacklessData db.stacklessdata = StacklessData(db) + db.use_new_funcgen = self.use_new_funcgen # we need a concrete gcpolicy to do this self.libraries += db.gcpolicy.gc_libraries() Modified: pypy/dist/pypy/translator/c/node.py ============================================================================== --- pypy/dist/pypy/translator/c/node.py (original) +++ pypy/dist/pypy/translator/c/node.py Fri Jan 27 18:02:06 2006 @@ -5,6 +5,7 @@ parentlink, Ptr, PyObject, Void, OpaqueType, Float, \ RuntimeTypeInfo, getRuntimeTypeInfo, Char from pypy.translator.c.funcgen import FunctionCodeGenerator +from pypy.translator.c.newfuncgen import FunctionCodeGenerator as NewFunctionCodeGenerator from pypy.translator.c.external import CExternalFunctionCodeGenerator from pypy.translator.c.support import USESLOTS # set to False if necessary while refactoring from pypy.translator.c.support import cdecl, somelettersfrom, c_string_constant @@ -548,8 +549,11 @@ elif hasattr(fnobj, 'graph'): cpython_exc = getattr(fnobj, 'exception_policy', None) == "CPython" if hasattr(db, 'stacklessdata'): + assert not db.use_new_funcgen, "can't use stackless with the new funcgen yet" from pypy.translator.c.stackless import SlpFunctionCodeGenerator gencls = SlpFunctionCodeGenerator + elif db.use_new_funcgen: + gencls = NewFunctionCodeGenerator else: gencls = FunctionCodeGenerator return gencls(fnobj.graph, db, cpython_exc, functionname) Added: pypy/dist/pypy/translator/c/test/test_newgc.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/c/test/test_newgc.py Fri Jan 27 18:02:06 2006 @@ -0,0 +1,29 @@ +import autopath +import sys +import py +from py.test import raises + +from pypy.translator.tool.cbuild import skip_missing_compiler +from pypy.translator.translator import TranslationContext +from pypy.translator.c import genc + +from pypy.rpython.memory.gctransform import GCTransformer + +def compile_func(fn, inputtypes): + t = TranslationContext() + t.buildannotator().build_types(fn, inputtypes) + t.buildrtyper().specialize() +# GCTransformer(t.graphs).transform() + + builder = genc.CExtModuleBuilder(t, fn, use_new_funcgen=True) + builder.generate_source() + skip_missing_compiler(builder.compile) + builder.import_module() + return builder.get_entry_point() + + +def test_something(): + def f(): + return 1 + fn = compile_func(f, []) + assert fn() == 1 From mwh at codespeak.net Fri Jan 27 18:06:07 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Fri, 27 Jan 2006 18:06:07 +0100 (CET) Subject: [pypy-svn] r22748 - in pypy/dist/pypy/translator/c: . test Message-ID: <20060127170607.7D4F327DCD@code1.codespeak.net> Author: mwh Date: Fri Jan 27 18:06:04 2006 New Revision: 22748 Added: pypy/dist/pypy/translator/c/newgc.py - copied unchanged from r22720, pypy/dist/pypy/translator/c/gc.py Modified: pypy/dist/pypy/translator/c/test/test_newgc.py Log: (cfbolz, mwh) same kind of trick as last time, but with the gc policy Modified: pypy/dist/pypy/translator/c/test/test_newgc.py ============================================================================== --- pypy/dist/pypy/translator/c/test/test_newgc.py (original) +++ pypy/dist/pypy/translator/c/test/test_newgc.py Fri Jan 27 18:06:04 2006 @@ -5,7 +5,7 @@ from pypy.translator.tool.cbuild import skip_missing_compiler from pypy.translator.translator import TranslationContext -from pypy.translator.c import genc +from pypy.translator.c import genc, newgc from pypy.rpython.memory.gctransform import GCTransformer @@ -15,7 +15,8 @@ t.buildrtyper().specialize() # GCTransformer(t.graphs).transform() - builder = genc.CExtModuleBuilder(t, fn, use_new_funcgen=True) + builder = genc.CExtModuleBuilder(t, fn, use_new_funcgen=True, + gcpolicy=newgc.RefcountingGcPolicy) builder.generate_source() skip_missing_compiler(builder.compile) builder.import_module() From ale at codespeak.net Fri Jan 27 18:11:05 2006 From: ale at codespeak.net (ale at codespeak.net) Date: Fri, 27 Jan 2006 18:11:05 +0100 (CET) Subject: [pypy-svn] r22749 - pypy/extradoc/sprintinfo/mallorca Message-ID: <20060127171105.032AC27DCF@code1.codespeak.net> Author: ale Date: Fri Jan 27 18:11:03 2006 New Revision: 22749 Added: pypy/extradoc/sprintinfo/mallorca/Paris_mini_sprint_report.txt Log: A report from Paris Added: pypy/extradoc/sprintinfo/mallorca/Paris_mini_sprint_report.txt ============================================================================== --- (empty file) +++ pypy/extradoc/sprintinfo/mallorca/Paris_mini_sprint_report.txt Fri Jan 27 18:11:03 2006 @@ -0,0 +1,31 @@ +Mini Sprint report Paris January 23-27 +================================================================ + +Logic programming +================================ + +Aurelien and Anders worked the whole week on trying to understand how Oz/Mozart +works. After a lot of trials they achieved in implementing: + + Aurelien had already made basic single assignment store. + + The store was completed (with tests) + + They made the basic infrastructure for computation spaces based on the store. + + From Logilab they stole and adapted code for the FiniteDomain, Constraints and + DichotomyDistributor. + + On the computation space they implemented the semantics of Choose, Commit, Ask + and Clone. + + Some experiments on making these constructs concurrent were made (not finished). + +Aspects and contracts +======================== + +Annotation and cleanup/refactoring of the grammar/parser. + +Research of different implementations of aspects and contracts. One reference is here_ + +.. _here: http://aosd-europe.net/documents/aspLang.pdf \ No newline at end of file From mwh at codespeak.net Fri Jan 27 18:34:00 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Fri, 27 Jan 2006 18:34:00 +0100 (CET) Subject: [pypy-svn] r22750 - pypy/dist/pypy/translator/c Message-ID: <20060127173400.D070627DCD@code1.codespeak.net> Author: mwh Date: Fri Jan 27 18:33:59 2006 New Revision: 22750 Modified: pypy/dist/pypy/translator/c/newfuncgen.py Log: dispatch to the gc policy for "gc_" operations Modified: pypy/dist/pypy/translator/c/newfuncgen.py ============================================================================== --- pypy/dist/pypy/translator/c/newfuncgen.py (original) +++ pypy/dist/pypy/translator/c/newfuncgen.py Fri Jan 27 18:33:59 2006 @@ -226,7 +226,10 @@ for op in block.operations: err = 'err%d_%d' % (myblocknum, len(to_release)) macro = 'OP_%s' % op.opname.upper() - meth = getattr(self, macro, None) + if op.opname.startswith('gc_'): + meth = getattr(self.gcpolicy, macro, None) + else: + meth = getattr(self, macro, None) if meth: line = meth(op, err) else: From mwh at codespeak.net Fri Jan 27 18:34:52 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Fri, 27 Jan 2006 18:34:52 +0100 (CET) Subject: [pypy-svn] r22751 - pypy/dist/pypy/translator/c Message-ID: <20060127173452.9433027DCF@code1.codespeak.net> Author: mwh Date: Fri Jan 27 18:34:50 2006 New Revision: 22751 Modified: pypy/dist/pypy/translator/c/newfuncgen.py Log: use iterblocks() not traverse Modified: pypy/dist/pypy/translator/c/newfuncgen.py ============================================================================== --- pypy/dist/pypy/translator/c/newfuncgen.py (original) +++ pypy/dist/pypy/translator/c/newfuncgen.py Fri Jan 27 18:34:50 2006 @@ -3,7 +3,7 @@ from pypy.translator.c.support import cdecl, ErrorValue from pypy.translator.c.support import llvalue_from_constant, gen_assignments from pypy.objspace.flow.model import Variable, Constant, Block -from pypy.objspace.flow.model import traverse, c_last_exception +from pypy.objspace.flow.model import c_last_exception from pypy.rpython.lltypesystem.lltype import \ Ptr, PyObject, Void, Bool, Signed, Unsigned, SignedLongLong, UnsignedLongLong,Char, UniChar, pyobjectptr, Struct, Array @@ -206,11 +206,9 @@ yield 'goto block%d;' % blocknum[link.target] # collect all blocks - def visit(block): - if isinstance(block, Block): - allblocks.append(block) - blocknum[block] = len(blocknum) - traverse(visit, graph) + for block in graph.iterblocks(): + allblocks.append(block) + blocknum[block] = len(blocknum) assert graph.startblock is allblocks[0] From ericvrp at codespeak.net Fri Jan 27 18:52:13 2006 From: ericvrp at codespeak.net (ericvrp at codespeak.net) Date: Fri, 27 Jan 2006 18:52:13 +0100 (CET) Subject: [pypy-svn] r22752 - in pypy/dist/pypy/translator: backendopt c/test Message-ID: <20060127175213.E2C5527DC8@code1.codespeak.net> Author: ericvrp Date: Fri Jan 27 18:52:11 2006 New Revision: 22752 Modified: pypy/dist/pypy/translator/backendopt/raisingop2direct_call.py pypy/dist/pypy/translator/backendopt/support.py pypy/dist/pypy/translator/c/test/test_backendoptimized.py Log: Added first working raising operation in rpython Modified: pypy/dist/pypy/translator/backendopt/raisingop2direct_call.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/raisingop2direct_call.py (original) +++ pypy/dist/pypy/translator/backendopt/raisingop2direct_call.py Fri Jan 27 18:52:11 2006 @@ -1,19 +1,29 @@ -from pypy.objspace.flow.model import Block, Constant, flatten -from pypy.translator.backendopt.support import log, all_operations -import pypy.rpython.raisingops +from pypy.translator.backendopt.support import log, all_operations, annotate +import pypy.rpython.raisingops.raisingops log = log.raisingop2directcall def raisingop2direct_call(translator): """search for operations that could raise an exception and change that operation into a direct_call to a function from the raisingops directory. - This function also needs to be annotated. + This function also needs to be annotated and specialized. """ + seen = {} for op in all_operations(translator): s = op.opname if not s.startswith('int_') and not s.startswith('uint_') and not s.startswith('float_'): continue if not s.endswith('_zer') and not s.endswith('_ovf') and not s.endswith('_val'): continue - log(s) - op.args.insert(0, s) + func = getattr(pypy.rpython.raisingops.raisingops, s, None) + assert func, "exception raising operation %s was not found" % s + if s not in seen: + seen[s] = 0 + log.info(s) + seen[s] += 1 + op.args.insert(0, annotate(translator, func, op.result, op.args)) op.opname = 'direct_call' + for k, v in seen.iteritems(): + log("%4dx %s" % (v, k)) + if seen != {}: + translator.rtyper.specialize_more_blocks() + #translator.view() Modified: pypy/dist/pypy/translator/backendopt/support.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/support.py (original) +++ pypy/dist/pypy/translator/backendopt/support.py Fri Jan 27 18:52:11 2006 @@ -1,7 +1,8 @@ -# logging - import py +from pypy.rpython.lltypesystem.lltype import functionptr, FuncType, typeOf +from pypy.rpython.rmodel import inputconst from pypy.tool.ansi_print import ansi_log + log = py.log.Producer("backendopt") py.log.setconsumer("backendopt", ansi_log) @@ -15,3 +16,10 @@ for block in graph.iterblocks(): for op in block.operations: yield op + +def annotate(translator, func, result, args): + args = [arg.concretetype for arg in args] + graph = translator.rtyper.annotate_helper(func, args) + fptr = functionptr(FuncType(args, result.concretetype), func.func_name, graph=graph) + c = inputconst(typeOf(fptr), fptr) + return c Modified: pypy/dist/pypy/translator/c/test/test_backendoptimized.py ============================================================================== --- pypy/dist/pypy/translator/c/test/test_backendoptimized.py (original) +++ pypy/dist/pypy/translator/c/test/test_backendoptimized.py Fri Jan 27 18:52:11 2006 @@ -183,7 +183,6 @@ backend_optimizations(t, raisingop2direct_call_all=True) def test_int_floordiv_zer(self): - py.test.skip("WIP") def f(x=int): try: y = 123 / x From mwh at codespeak.net Fri Jan 27 19:08:36 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Fri, 27 Jan 2006 19:08:36 +0100 (CET) Subject: [pypy-svn] r22753 - in pypy/dist/pypy: objspace/flow rpython/memory rpython/memory/test Message-ID: <20060127180836.B2E1F27DCD@code1.codespeak.net> Author: mwh Date: Fri Jan 27 19:08:33 2006 New Revision: 22753 Modified: pypy/dist/pypy/objspace/flow/model.py pypy/dist/pypy/rpython/memory/gctransform.py pypy/dist/pypy/rpython/memory/test/test_gctransform.py Log: (cfbolz, mwh) Attach the operations-to-be-executed-for-clean-up to the space operation as ".cleanup" not as a Void argument. Modified: pypy/dist/pypy/objspace/flow/model.py ============================================================================== --- pypy/dist/pypy/objspace/flow/model.py (original) +++ pypy/dist/pypy/objspace/flow/model.py Fri Jan 27 19:08:33 2006 @@ -317,7 +317,7 @@ class SpaceOperation(object): - __slots__ = "opname args result offset".split() + __slots__ = "opname args result offset cleanup".split() def __init__(self, opname, args, result, offset=-1): self.opname = intern(opname) # operation name Modified: pypy/dist/pypy/rpython/memory/gctransform.py ============================================================================== --- pypy/dist/pypy/rpython/memory/gctransform.py (original) +++ pypy/dist/pypy/rpython/memory/gctransform.py Fri Jan 27 19:08:33 2006 @@ -21,6 +21,8 @@ """ +EXCEPTION_RAISING_OPS = ['direct_call', 'indirect_call'] + def var_needsgc(var): vartype = var.concretetype return isinstance(vartype, lltype.Ptr) and vartype._needsgc() @@ -55,11 +57,11 @@ livevars = [var for var in block.inputargs if var_needsgc(var)] for op in block.operations: newops.extend(self.replacement_operations(op)) - if op.opname in ('direct_call', 'indirect_call') and livevars: + if op.opname in EXCEPTION_RAISING_OPS and livevars: cleanup_on_exception = [] for var in livevars: cleanup_on_exception.extend(self.pop_alive(var)) - op.args.append(rmodel.inputconst(lltype.Void, cleanup_on_exception)) + op.cleanup = cleanup_on_exception if var_needsgc(op.result): if op.opname not in ('direct_call', 'indirect_call'): newops.extend(self.push_alive(op.result)) Modified: pypy/dist/pypy/rpython/memory/test/test_gctransform.py ============================================================================== --- pypy/dist/pypy/rpython/memory/test/test_gctransform.py (original) +++ pypy/dist/pypy/rpython/memory/test/test_gctransform.py Fri Jan 27 19:08:33 2006 @@ -5,6 +5,10 @@ from pypy.objspace.flow.model import Variable def checkblock(block): + if not block.operations == (): + # a return/exception block -- don't want to think about them + # (even though the test passes for somewhat accidental reasons) + return vars_in = len([v for v in block.inputargs if var_needsgc(v)]) push_alives = len([op for op in block.operations if op.opname.startswith('gc_push_alive')]) @@ -123,9 +127,9 @@ ggraph = graphof(t, g) direct_calls = [op for op in ggraph.startblock.operations if op.opname == "direct_call"] assert len(direct_calls) == 3 - assert len(direct_calls[0].args) == 1 - assert direct_calls[1].args[1].value[0].args[0] == direct_calls[0].result - assert [op.args[0] for op in direct_calls[2].args[1].value] == [direct_calls[0].result, direct_calls[1].result] + assert direct_calls[1].cleanup[0].args[0] == direct_calls[0].result + assert [op.args[0] for op in direct_calls[2].cleanup] == \ + [direct_calls[0].result, direct_calls[1].result] def test_multiply_passed_var(): S = lltype.GcStruct("S", ('x', lltype.Signed)) From stephan at codespeak.net Fri Jan 27 19:14:40 2006 From: stephan at codespeak.net (stephan at codespeak.net) Date: Fri, 27 Jan 2006 19:14:40 +0100 (CET) Subject: [pypy-svn] r22754 - in pypy/dist/pypy/rpython/rctypes: . test Message-ID: <20060127181440.C5EAD27DCF@code1.codespeak.net> Author: stephan Date: Fri Jan 27 19:14:38 2006 New Revision: 22754 Modified: pypy/dist/pypy/rpython/rctypes/implementation.py pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py Log: added one disabled structure specialize test Modified: pypy/dist/pypy/rpython/rctypes/implementation.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/implementation.py (original) +++ pypy/dist/pypy/rpython/rctypes/implementation.py Fri Jan 27 19:14:38 2006 @@ -88,12 +88,12 @@ else: answer.append(ctype_type.wrap_arg(ll_type, arg_name)) return answer - -class CtypesBasicTypeInstantiationTranslation( FunctionPointerTranslation ): - - compute_result_annotation = classmethod( - FunctionPointerTranslation.compute_result_annotation) - +# +#class CtypesBasicTypeInstantiationTranslation( FunctionPointerTranslation ): +# +# compute_result_annotation = classmethod( +# FunctionPointerTranslation.compute_result_annotation) +# class RStructureMeta(type(Structure)): def __new__(mta,name,bases,clsdict): @@ -114,6 +114,9 @@ return SomeCTypesObject(cls) compute_annotation = classmethod(compute_annotation) + def specialize(self, hop): + raise NotImplementedError + def compute_result_annotation(cls, *args_s): """ Answer the result annotation of calling 'cls'. Modified: pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py Fri Jan 27 19:14:38 2006 @@ -23,6 +23,7 @@ else: py.test.skip("don't know how to load the c lib for %s" % sys.platform) + atoi = mylib.atoi atoi.restype = c_int atoi.argtypes = [c_char_p] @@ -153,6 +154,13 @@ my_array = c_int_10() return my_array[-11] +def py_test_specialize_struct(): + p = tagpoint() + p.x = 1 + p.y = 2 + + return p.x + class Test_rctypes: @@ -270,6 +278,14 @@ s = a.build_types(py_test_simple_ctypes_non_const,[]) assert s.knowntype == c_float + def x_test_specialize_struct(self): + t = TranslationContext() + a = t.buildannotator() + s = a.build_types(py_test_specialize_struct, []) + # result should be an integer + assert s.knowntype == int + t.buildrtyper().specialize() + #d#t.view() class Test_array: From mwh at codespeak.net Fri Jan 27 19:37:11 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Fri, 27 Jan 2006 19:37:11 +0100 (CET) Subject: [pypy-svn] r22755 - in pypy/dist/pypy/translator/c: . test Message-ID: <20060127183711.D7FDC27DCD@code1.codespeak.net> Author: mwh Date: Fri Jan 27 19:37:09 2006 New Revision: 22755 Modified: pypy/dist/pypy/translator/c/newfuncgen.py pypy/dist/pypy/translator/c/newgc.py pypy/dist/pypy/translator/c/test/test_newgc.py Log: (cfbolz,mwh) First hacks towards using the new gc related operation. Modified: pypy/dist/pypy/translator/c/newfuncgen.py ============================================================================== --- pypy/dist/pypy/translator/c/newfuncgen.py (original) +++ pypy/dist/pypy/translator/c/newfuncgen.py Fri Jan 27 19:37:09 2006 @@ -171,9 +171,7 @@ linklocalvars = linklocalvars or {} for v in to_release: linklocalvars[v] = self.expr(v) - is_alive_and_dies = linklocalvars.copy() assignments = [] - multiple_times_alive = [] for a1, a2 in zip(link.args, link.target.inputargs): a2type, a2typename = self.lltypes[id(a2)] if a2type is Void: @@ -184,25 +182,8 @@ src = self.expr(a1) dest = LOCALVAR % a2.name assignments.append((a2typename, dest, src)) - if a1 in is_alive_and_dies: - del is_alive_and_dies[a1] - else: - #assert self.lltypemap(a1) == self.lltypemap(a2) - multiple_times_alive.append(a2) - # warning, the order below is delicate to get right: - # 1. forget the old variables that are not passed over - for v in is_alive_and_dies: - line = self.pop_alive(v, linklocalvars[v]) - if line: - yield line - # 2. perform the assignments with collision-avoidance for line in gen_assignments(assignments): yield line - # 3. keep alive the new variables if needed - for a2 in multiple_times_alive: - line = self.push_alive(a2) - if line: - yield line yield 'goto block%d;' % blocknum[link.target] # collect all blocks @@ -226,11 +207,13 @@ macro = 'OP_%s' % op.opname.upper() if op.opname.startswith('gc_'): meth = getattr(self.gcpolicy, macro, None) + if meth: + line = meth(self, op, err) else: meth = getattr(self, macro, None) - if meth: - line = meth(op, err) - else: + if meth: + line = meth(op, err) + if meth is None: lst = [self.expr(v) for v in op.args] lst.append(self.expr(op.result)) lst.append(err) @@ -244,13 +227,6 @@ reachable_err = len(to_release) to_release.append(op.result) - T = self.lltypemap(op.result) - if T is not Void: - res = LOCALVAR % op.result.name - line = push_alive_op_result(op.opname, res, T) - if line: - yield line - fallthrough = False if len(block.exits) == 0: if len(block.inputargs) == 2: # exc_cls, exc_value Modified: pypy/dist/pypy/translator/c/newgc.py ============================================================================== --- pypy/dist/pypy/translator/c/newgc.py (original) +++ pypy/dist/pypy/translator/c/newgc.py Fri Jan 27 19:37:09 2006 @@ -283,6 +283,27 @@ eresult, err) + def OP_GC_PUSH_ALIVE(self, funcgen, op, err): + expr = funcgen.expr(op.args[0]) + defnode = self.db.gettypedefnode(op.args[0].concretetype.TO) + assert defnode.gcheader is not None + return 'pypy_IncRf_%s(%s);' % (defnode.barename, expr) + + def OP_GC_POP_ALIVE(self, funcgen, op, err): + expr = funcgen.expr(op.args[0]) + defnode = self.db.gettypedefnode(op.args[0].concretetype.TO) + assert defnode.gcheader is not None + return 'pypy_DecRf_%s(%s);' % (defnode.barename, expr) + + def OP_GC_PUSH_ALIVE_PYOBJ(self, funcgen, op, err): + expr = funcgen.expr(op.args[0]) + return 'Py_XINCREF(%s);' % expr + + def OP_GC_POP_ALIVE_PYOBJ(self, funcgen, op, err): + expr = funcgen.expr(op.args[0]) + return 'Py_XDECREF(%s);' % expr + + class RefcountingRuntimeTypeInfo_OpaqueNode(ContainerNode): nodekind = 'refcnt rtti' globalcontainer = True Modified: pypy/dist/pypy/translator/c/test/test_newgc.py ============================================================================== --- pypy/dist/pypy/translator/c/test/test_newgc.py (original) +++ pypy/dist/pypy/translator/c/test/test_newgc.py Fri Jan 27 19:37:09 2006 @@ -6,6 +6,7 @@ from pypy.translator.tool.cbuild import skip_missing_compiler from pypy.translator.translator import TranslationContext from pypy.translator.c import genc, newgc +from pypy.rpython.lltypesystem import lltype from pypy.rpython.memory.gctransform import GCTransformer @@ -13,7 +14,7 @@ t = TranslationContext() t.buildannotator().build_types(fn, inputtypes) t.buildrtyper().specialize() -# GCTransformer(t.graphs).transform() + GCTransformer(t.graphs).transform() builder = genc.CExtModuleBuilder(t, fn, use_new_funcgen=True, gcpolicy=newgc.RefcountingGcPolicy) @@ -28,3 +29,12 @@ return 1 fn = compile_func(f, []) assert fn() == 1 + +def test_something_more(): + S = lltype.GcStruct("S", ('x', lltype.Signed)) + def f(x): + s = lltype.malloc(S) + s.x = x + return s.x + fn = compile_func(f, [int]) + assert fn(1) == 1 From ericvrp at codespeak.net Fri Jan 27 19:45:48 2006 From: ericvrp at codespeak.net (ericvrp at codespeak.net) Date: Fri, 27 Jan 2006 19:45:48 +0100 (CET) Subject: [pypy-svn] r22757 - in pypy/dist/pypy/translator/js: . test Message-ID: <20060127184548.62E7227DCD@code1.codespeak.net> Author: ericvrp Date: Fri Jan 27 19:45:44 2006 New Revision: 22757 Modified: pypy/dist/pypy/translator/js/database.py pypy/dist/pypy/translator/js/optimize.py pypy/dist/pypy/translator/js/opwriter.py pypy/dist/pypy/translator/js/support.py pypy/dist/pypy/translator/js/test/runtest.py pypy/dist/pypy/translator/js/test/test_exc_operation.py pypy/dist/pypy/translator/js/test/test_jseval.py pypy/dist/pypy/translator/js/test/test_stackless.py Log: * Added longlong support * Added first of exception raising operations through transformation that can be shared between genc/genllvm and genjs in due time. * Added native javascript code for string concat. Modified: pypy/dist/pypy/translator/js/database.py ============================================================================== --- pypy/dist/pypy/translator/js/database.py (original) +++ pypy/dist/pypy/translator/js/database.py Fri Jan 27 19:45:44 2006 @@ -20,6 +20,8 @@ lltype.Float: "double", lltype.Signed: "int", lltype.Unsigned: "uint", + lltype.SignedLongLong: "int", + lltype.UnsignedLongLong: "uint", lltype.UniChar: "uint", lltype.Void: "void"} Modified: pypy/dist/pypy/translator/js/optimize.py ============================================================================== --- pypy/dist/pypy/translator/js/optimize.py (original) +++ pypy/dist/pypy/translator/js/optimize.py Fri Jan 27 19:45:44 2006 @@ -8,6 +8,7 @@ 'll_str__IntegerR_SignedConst_Signed', 'll_str__FloatR_FloatConst_Float', 'll_int__rpy_stringPtr_Signed', + 'll_join_strs__Signed_arrayPtr', #'ll_issubclass__object_vtablePtr_object_vtablePtr', @@ -52,6 +53,9 @@ elif funcname == 'll_int__rpy_stringPtr_Signed' and params[1] == '10': return True, '%s = parseInt(%s)' % (targetvar, params[0]) + elif funcname == 'll_join_strs__Signed_arrayPtr' and params[0] == '2': + return True, '%s = {hash:0, chars:%s + %s}' % (targetvar, params[0], params[1]) + #externals... elif funcname == 'll_js_jseval__rpy_stringPtr': return True, '%s = eval(%s.chars)' % (targetvar, params[0]) Modified: pypy/dist/pypy/translator/js/opwriter.py ============================================================================== --- pypy/dist/pypy/translator/js/opwriter.py (original) +++ pypy/dist/pypy/translator/js/opwriter.py Fri Jan 27 19:45:44 2006 @@ -79,6 +79,11 @@ #log(str(op)) #self.codewriter.comment(str(op)) + if op.opname.startswith('llong_'): + op.opname = 'int_' + op.opname[6:] + elif op.opname.startswith('ullong_'): + op.opname = 'uint_' + op.opname[7:] + invoke = op.opname.startswith('invoke:') if invoke: self.invoke(op) @@ -97,7 +102,7 @@ if not meth: raise Exception, "operation %s not found" % op.opname return - meth(op) + meth(op) def _generic_pow(self, op, onestr): targetvar = self.db.repr_arg(op.result) Modified: pypy/dist/pypy/translator/js/support.py ============================================================================== --- pypy/dist/pypy/translator/js/support.py (original) +++ pypy/dist/pypy/translator/js/support.py Fri Jan 27 19:45:44 2006 @@ -10,7 +10,7 @@ reserved_names_string = ''' if then else function for while witch continue - break super var + break super var do bool char int float Array String Struct Number ''' Modified: pypy/dist/pypy/translator/js/test/runtest.py ============================================================================== --- pypy/dist/pypy/translator/js/test/runtest.py (original) +++ pypy/dist/pypy/translator/js/test/runtest.py Fri Jan 27 19:45:44 2006 @@ -1,13 +1,12 @@ import py, os from pypy.translator.translator import TranslationContext -from pypy.translator.backendopt.all import backend_optimizations +from pypy.translator.backendopt.all import backend_optimizations from pypy.translator.js.js import JS from pypy.translator.js.test.browsertest import jstest from pypy.translator.js import conftest from pypy.translator.js.log import log log = log.runtest use_browsertest = conftest.option.jsbrowser -#from pypy.translator.js.test.test_jseval import add_callback_function def _CLI_is_on_path(): try: @@ -26,9 +25,7 @@ t.buildrtyper().specialize() - #add_callback_function(t.translator) - - backend_optimizations(t, inline_threshold=0, mallocs=False) + backend_optimizations(t, raisingop2direct_call_all=True, inline_threshold=0, mallocs=False) #backend_optimizations(t) if view: t.view() Modified: pypy/dist/pypy/translator/js/test/test_exc_operation.py ============================================================================== --- pypy/dist/pypy/translator/js/test/test_exc_operation.py (original) +++ pypy/dist/pypy/translator/js/test/test_exc_operation.py Fri Jan 27 19:45:44 2006 @@ -4,7 +4,7 @@ from pypy.rpython.rarithmetic import r_uint, ovfcheck, ovfcheck_lshift from pypy.translator.test import snippet -def DONTtest_zerodiv_int(): #issue no exception raising operations yet +def test_zerodiv_int(): #issue no exception raising operations yet def zerodiv_int(n): try: r=100/n Modified: pypy/dist/pypy/translator/js/test/test_jseval.py ============================================================================== --- pypy/dist/pypy/translator/js/test/test_jseval.py (original) +++ pypy/dist/pypy/translator/js/test/test_jseval.py Fri Jan 27 19:45:44 2006 @@ -6,12 +6,10 @@ from pypy.rpython.rjs import jseval from pypy.translator.js import conftest -class jsnative(object): - def __init__(self, cmd): - self._cmd = cmd - - def __call__(self): - return jseval(self._cmd) +def jsnative(cmd): + def do(): + return jseval(cmd) + return do getDate = jsnative("new Date().getDate()") getTime = jsnative("Math.floor(new Date().getTime())") @@ -29,8 +27,8 @@ from time import localtime def jsnative1(): - return jseval("new Date().getDate()") - #return getDate() + getTime() + return getDate() jsnative1_fn = compile_function(jsnative1, []) assert jsnative1_fn() == localtime()[2] @@ -40,30 +38,20 @@ def callback_function(): n_times_called[0] += 1 - jseval("setTimeout('callback_function()', 10)") + jseval("document.title=" + str(n_times_called[0])) + jseval("setTimeout('callback_function()', 100)") -#def add_callback_function(translator): -# a = translator.annotator -# bk = a.bookkeeper -# s_cb = bk.immutablevalue(callback_function) -# bk.emulate_pbc_call('callback_function', s_cb, []) -# a.complete() -# translator.rtyper.specialize_more_blocks() - def test_register_callback(): if not conftest.option.jsbrowser: py.test.skip("works only in a browser (use py.test --browser)") def register_callback(): - callback_function() - #callbacks.append(callback_function) - #jseval("setTimeout('callback_function()', 10)") - start_time = current_time = int(jseval("Math.floor(new Date().getTime())")) + callback_function() #..start timer + start_time = current_time = int(getTime()) while current_time - start_time < 1000: - current_time = int(jseval("Math.floor(new Date().getTime())")) - return n_times_called + current_time = int(getTime()) + return n_times_called[0] register_callback_fn = compile_function(register_callback, []) result = register_callback_fn() - print 'result=%d' % result - assert result > 1 + assert result == 1 Modified: pypy/dist/pypy/translator/js/test/test_stackless.py ============================================================================== --- pypy/dist/pypy/translator/js/test/test_stackless.py (original) +++ pypy/dist/pypy/translator/js/test/test_stackless.py Fri Jan 27 19:45:44 2006 @@ -194,16 +194,18 @@ def getTime(): return int(jseval("Math.floor(new Date().getTime())")) - def g(): - for i in range(100): + def g(n): + for i in range(10): pass if getTime() - start_time[0] < 10*1000: - g() - return 123 + x = g(n-1) + if x != n-1: + jseval("log('x != n-1')") + return n def lp(): start_time[0] = getTime() - return g() + return g(100000) data = wrap_stackless_function(lp) From pedronis at codespeak.net Fri Jan 27 19:51:31 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Fri, 27 Jan 2006 19:51:31 +0100 (CET) Subject: [pypy-svn] r22758 - in pypy/dist/pypy: annotation jit jit/test Message-ID: <20060127185131.7686527DCD@code1.codespeak.net> Author: pedronis Date: Fri Jan 27 19:51:27 2006 New Revision: 22758 Added: pypy/dist/pypy/jit/hintannotator.py (contents, props changed) pypy/dist/pypy/jit/hintbookkeeper.py (contents, props changed) pypy/dist/pypy/jit/hintmodel.py (contents, props changed) pypy/dist/pypy/jit/test/test_hint_annotation.py (contents, props changed) Modified: pypy/dist/pypy/annotation/annrpython.py Log: (arigo, arre, pedronis) start of off-line partial evalutation, based on the annotator framework. see PyPy The Movie, coming soon ... Modified: pypy/dist/pypy/annotation/annrpython.py ============================================================================== --- pypy/dist/pypy/annotation/annrpython.py (original) +++ pypy/dist/pypy/annotation/annrpython.py Fri Jan 27 19:51:27 2006 @@ -624,23 +624,6 @@ assert isinstance(op.result, Variable) self.setbinding(op.result, resultcell) # bind resultcell to op.result - def _registeroperations(loc): - # All unary operations - for opname in annmodel.UNARY_OPERATIONS: - exec """ -def consider_op_%s(self, arg, *args): - return arg.%s(*args) -""" % (opname, opname) in globals(), loc - # All binary operations - for opname in annmodel.BINARY_OPERATIONS: - exec """ -def consider_op_%s(self, arg1, arg2, *args): - return pair(arg1,arg2).%s(*args) -""" % (opname, opname) in globals(), loc - - _registeroperations(locals()) - del _registeroperations - # XXX "contains" clash with SomeObject method def consider_op_contains(self, seq, elem): self.bookkeeper.count("contains", seq) @@ -664,6 +647,24 @@ return annmodel.SomeSlice(start, stop, step) +def _registeroperations(ns, model): + # All unary operations + for opname in model.UNARY_OPERATIONS: + exec """ +def consider_op_%s(self, arg, *args): + return arg.%s(*args) +""" % (opname, opname) in globals(), ns + # All binary operations + for opname in model.BINARY_OPERATIONS: + exec """ +def consider_op_%s(self, arg1, arg2, *args): + return pair(arg1,arg2).%s(*args) +""" % (opname, opname) in globals(), ns + +# register simple operations handling +_registeroperations(RPythonAnnotator.__dict__, annmodel) + + class CannotSimplify(Exception): pass Added: pypy/dist/pypy/jit/hintannotator.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/jit/hintannotator.py Fri Jan 27 19:51:27 2006 @@ -0,0 +1,12 @@ +from pypy.annotation.annrpython import RPythonAnnotator, _registeroperations +from pypy.jit import hintmodel +from pypy.jit.hintbookkeeper import HintBookkeeper + +class HintAnnotator(RPythonAnnotator): + + def __init__(self): + RPythonAnnotator.__init__(self) + self.bookkeeper = HintBookkeeper() # XXX + + +_registeroperations(HintAnnotator.__dict__, hintmodel) Added: pypy/dist/pypy/jit/hintbookkeeper.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/jit/hintbookkeeper.py Fri Jan 27 19:51:27 2006 @@ -0,0 +1,42 @@ +from pypy.tool.tls import tlsobject + +TLS = tlsobject() + +class HintBookkeeper(object): + + def __init__(self): + self.pending_specializations = [] + self.origins = {} + + def enter(self, position_key): + """Start of an operation. + The operation is uniquely identified by the given key.""" + self.position_key = position_key + TLS.bookkeeper = self + + def leave(self): + """End of an operation.""" + del TLS.bookkeeper + del self.position_key + + def myorigin(self): + try: + origin = self.origins[self.position_key] + except KeyError: + from pypy.jit import hintmodel + origin = hintmodel.OriginTreeNode() + self.origins[self.position_key] = origin + return origin + + def compute_at_fixpoint(self): + pass + +# get current bookkeeper + +def getbookkeeper(): + """Get the current Bookkeeper. + Only works during the analysis of an operation.""" + try: + return TLS.bookkeeper + except AttributeError: + return None Added: pypy/dist/pypy/jit/hintmodel.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/jit/hintmodel.py Fri Jan 27 19:51:27 2006 @@ -0,0 +1,42 @@ +from pypy.annotation import model as annmodel +from pypy.annotation.pairtype import pair, pairtype +from pypy.jit.hintbookkeeper import getbookkeeper +from pypy.rpython.lltypesystem import lltype + +UNARY_OPERATIONS = "".split() + +BINARY_OPERATIONS = "int_add".split() + +class OriginTreeNode(object): + + def __init__(self, origins=None): + if origins is None: + origins = {} + self.origins = origins + + def merge(self, nodes): + self.origins.update(nodes) + +class SomeLLAbstractValue(annmodel.SomeObject): + + def __init__(self, T): + self.concretetype = T + +class SomeLLAbstractVariable(SomeLLAbstractValue): + pass + +class SomeLLAbstractConstant(SomeLLAbstractValue): + + def __init__(self, T, origins): + SomeLLAbstractValue.__init__(self, T) + self.origins = origins + + +class __extend__(pairtype(SomeLLAbstractConstant, SomeLLAbstractConstant)): + + def int_add((hs_c1, hs_c2)): + origin = getbookkeeper().myorigin() + origin.merge(hs_c1.origins) + origin.merge(hs_c2.origins) + return SomeLLAbstractConstant(lltype.Signed, {origin: True}) + Added: pypy/dist/pypy/jit/test/test_hint_annotation.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/jit/test/test_hint_annotation.py Fri Jan 27 19:51:27 2006 @@ -0,0 +1,27 @@ +from pypy.translator.translator import TranslationContext, graphof +from pypy.jit.hintannotator import HintAnnotator +from pypy.jit.hintmodel import SomeLLAbstractConstant, OriginTreeNode +from pypy.rpython.lltypesystem import lltype + +def hannotate(func, argtypes): + # build the normal ll graphs for ll_function + t = TranslationContext() + a = t.buildannotator() + a.build_types(func, argtypes) + rtyper = t.buildrtyper() + rtyper.specialize() + graph1 = graphof(t, func) + # build hint annotator types + hannotator = HintAnnotator() + hs = hannotator.build_graph_types(graph1, [SomeLLAbstractConstant(v.concretetype, + {OriginTreeNode(): True}) + for v in graph1.getargs()]) + return hs + +def test_simple(): + def ll_function(x, y): + return x + y + hs = hannotate(ll_function, [int, int]) + assert isinstance(hs, SomeLLAbstractConstant) + assert len(hs.origins) == 1 + assert hs.concretetype == lltype.Signed From tismer at codespeak.net Fri Jan 27 20:01:49 2006 From: tismer at codespeak.net (tismer at codespeak.net) Date: Fri, 27 Jan 2006 20:01:49 +0100 (CET) Subject: [pypy-svn] r22759 - pypy/dist/pypy/module/stackless Message-ID: <20060127190149.42AFD27DCB@code1.codespeak.net> Author: tismer Date: Fri Jan 27 20:01:47 2006 New Revision: 22759 Modified: pypy/dist/pypy/module/stackless/__init__.py pypy/dist/pypy/module/stackless/interp_coroutine.py Log: more progress: managed to implement get_main and get_current as static methods (not really supported, yet) Modified: pypy/dist/pypy/module/stackless/__init__.py ============================================================================== --- pypy/dist/pypy/module/stackless/__init__.py (original) +++ pypy/dist/pypy/module/stackless/__init__.py Fri Jan 27 20:01:47 2006 @@ -13,3 +13,9 @@ 'tasklet' : 'interp_stackless.tasklet', 'Coroutine' : 'interp_coroutine.AppCoroutine', } + + def setup_after_space_initialization(self): + # post-installing classmethods/staticmethods which + # are not yet directly supported + from pypy.module.stackless.interp_coroutine import post_install + post_install(self) Modified: pypy/dist/pypy/module/stackless/interp_coroutine.py ============================================================================== --- pypy/dist/pypy/module/stackless/interp_coroutine.py (original) +++ pypy/dist/pypy/module/stackless/interp_coroutine.py Fri Jan 27 20:01:47 2006 @@ -5,6 +5,7 @@ from pypy.interpreter.gateway import interp2app, ObjSpace, W_Root from pypy.interpreter.error import OperationError from pypy.rpython.rarithmetic import intmask +from pypy.interpreter.function import StaticMethod from pypy.rpython.rstack import yield_current_frame_to_caller from pypy.module.stackless.stackless_flags import StacklessFlags @@ -201,6 +202,7 @@ raise OperationError(space.w_ValueError, space.wrap( "cannot switch to an unbound Coroutine")) self.switch() + appcostate.current = self def w_kill(self): self.kill() @@ -227,6 +229,24 @@ return space.wrap(self.get_is_zombie()) AppCoroutine.w_get_is_zombie = w_get_is_zombie +def w_coro_get_current(space): + return space.wrap(appcostate.current) + +def w_coro_get_main(space): + return space.wrap(appcostate.main) + +def installStaticMethod(space, w_klass, func, name=None): + if name is None: + name = func.__name__ + smeth = StaticMethod(space.wrap(interp2app(func, name))) + space.setattr(w_klass, space.wrap(name), space.wrap(smeth)) + +def post_install(module): + space = module.space + w_klass = space.getattr(space.wrap(module), space.wrap('Coroutine')) + installStaticMethod(space, w_klass, w_coro_get_current, 'get_current') + installStaticMethod(space, w_klass, w_coro_get_main, 'get_main') + AppCoroutine.typedef = TypeDef("Coroutine", __new__ = interp2app(AppCoroutine.descr_method__new__.im_func), bind = interp2app(AppCoroutine.w_bind, @@ -236,6 +256,12 @@ is_zombie = GetSetProperty(AppCoroutine.w_get_is_zombie, doc=AppCoroutine.get_is_zombie.__doc__), ) +class AppCoState(object): + def __init__(self): + self.current = self.main = AppCoroutine() + +appcostate = AppCoState() + """ Considerations about "current" ------------------------------ From ludal at codespeak.net Sat Jan 28 02:10:50 2006 From: ludal at codespeak.net (ludal at codespeak.net) Date: Sat, 28 Jan 2006 02:10:50 +0100 (CET) Subject: [pypy-svn] r22761 - in pypy/branch/ast-experiments/pypy: interpreter/pyparser module/recparser translator/tool Message-ID: <20060128011050.7D70127DD4@code1.codespeak.net> Author: ludal Date: Sat Jan 28 02:10:44 2006 New Revision: 22761 Modified: pypy/branch/ast-experiments/pypy/interpreter/pyparser/astbuilder.py pypy/branch/ast-experiments/pypy/interpreter/pyparser/ebnfgrammar.py pypy/branch/ast-experiments/pypy/interpreter/pyparser/ebnflexer.py pypy/branch/ast-experiments/pypy/interpreter/pyparser/ebnfparse.py pypy/branch/ast-experiments/pypy/interpreter/pyparser/grammar.py pypy/branch/ast-experiments/pypy/interpreter/pyparser/pysymbol.py pypy/branch/ast-experiments/pypy/interpreter/pyparser/pythonlexer.py pypy/branch/ast-experiments/pypy/interpreter/pyparser/pythonparse.py pypy/branch/ast-experiments/pypy/interpreter/pyparser/pytoken.py pypy/branch/ast-experiments/pypy/interpreter/pyparser/tuplebuilder.py pypy/branch/ast-experiments/pypy/module/recparser/__init__.py pypy/branch/ast-experiments/pypy/translator/tool/make_dot.py Log: big refactoring, of the parser -- part I isolates management of symbols and grammar rules into a Parser class Modified: pypy/branch/ast-experiments/pypy/interpreter/pyparser/astbuilder.py ============================================================================== --- pypy/branch/ast-experiments/pypy/interpreter/pyparser/astbuilder.py (original) +++ pypy/branch/ast-experiments/pypy/interpreter/pyparser/astbuilder.py Sat Jan 28 02:10:44 2006 @@ -5,13 +5,17 @@ from grammar import BaseGrammarBuilder, AbstractContext from pypy.interpreter.astcompiler import ast, consts -from pypy.interpreter.pyparser.pysymbol import _cpython_symbols as sym -import pypy.interpreter.pyparser.pytoken as tok +from pypy.interpreter.pyparser.pythonparse import PYTHON_PARSER from pypy.interpreter.pyparser.error import SyntaxError from pypy.interpreter.pyparser.parsestring import parsestr DEBUG_MODE = 0 +# XXX : use builder.parser instead +sym = PYTHON_PARSER.symbols +rsym = PYTHON_PARSER.symbol_repr +tok = PYTHON_PARSER + ### Parsing utilites ################################################# def parse_except_clause(tokens): """parses 'except' [test [',' test]] ':' suite @@ -1501,8 +1505,8 @@ self.count = count self.lineno = lineno # src.getline() self.col = 0 # src.getcol() - - + +# XXX : replace sym and rsym by a ref to parser class RuleObject(BaseRuleObject): """A simple object used to wrap a rule or token""" def __init__(self, name, count, lineno): @@ -1510,26 +1514,25 @@ self.rulename = name def __str__(self): - return "" % (sym.sym_name[self.rulename], self.count) + return "" % ( rsym[self.rulename], self.count) def __repr__(self): - return "" % (sym.sym_name[self.rulename], self.count) + return "" % ( rsym[self.rulename], self.count) class TempRuleObject(BaseRuleObject): """used to keep track of how many items get_atom() should pop""" - def __init__(self, name, count, lineno): BaseRuleObject.__init__(self, count, lineno) self.temp_rulename = name - + def __str__(self): return "" % (self.temp_rulename, self.count) def __repr__(self): return "" % (self.temp_rulename, self.count) - + class TokenObject(ast.Node): """A simple object used to wrap a rule or token""" def __init__(self, name, value, lineno): @@ -1539,7 +1542,7 @@ # self.line = 0 # src.getline() self.col = 0 # src.getcol() self.lineno = lineno - + def get_name(self): return tok.tok_rpunct.get(self.name, tok.tok_name.get(self.name, str(self.name))) @@ -1549,10 +1552,10 @@ if value is None: value = '' return value - + def __str__(self): return "" % (self.get_name(), self.value) - + def __repr__(self): return "" % (self.get_name(), self.value) @@ -1622,8 +1625,10 @@ class AstBuilder(BaseGrammarBuilder): """A builder that directly produce the AST""" - def __init__(self, rules=None, debug=0, space=None): - BaseGrammarBuilder.__init__(self, rules, debug) + def __init__(self, parser=None, debug=0, space=None): + if parser is None: + parser = PYTHON_PARSER + BaseGrammarBuilder.__init__(self, parser, debug) self.rule_stack = [] self.space = space self.source_encoding = None @@ -1632,8 +1637,6 @@ return AstBuilderContext(self.rule_stack) def restore(self, ctx): -## if DEBUG_MODE: -## print "Restoring context (%s)" % (len(ctx.rule_stack)) assert isinstance(ctx, AstBuilderContext) assert len(self.rule_stack) >= ctx.d del self.rule_stack[ctx.d:] @@ -1644,15 +1647,10 @@ def push(self, obj): self.rule_stack.append(obj) - if not isinstance(obj, RuleObject) and not isinstance(obj, TokenObject): -## if DEBUG_MODE: -## print "Pushed:", str(obj), len(self.rule_stack) - pass - elif isinstance(obj, TempRuleObject): -## if DEBUG_MODE: -## print "Pushed:", str(obj), len(self.rule_stack) - pass - # print "\t", self.rule_stack +## if not isinstance(obj, RuleObject) and not isinstance(obj, TokenObject): +## pass +## elif isinstance(obj, TempRuleObject): +## pass def push_tok(self, name, value, src ): self.push( TokenObject( name, value, src._token_lnum ) ) @@ -1664,48 +1662,29 @@ # Do nothing, keep rule on top of the stack ## rule_stack = self.rule_stack[:] if rule.is_root(): -## if DEBUG_MODE: -## print "ALT:", sym.sym_name[rule.codename], self.rule_stack builder_func = ASTRULES.get(rule.codename, None) if builder_func: builder_func(self, 1) else: -## if DEBUG_MODE: -## print "No reducing implementation for %s, just push it on stack" % ( -## sym.sym_name[rule.codename]) self.push_rule(rule.codename, 1, source) else: self.push_rule(rule.codename, 1, source) -## if DEBUG_MODE > 1: -## show_stack(rule_stack, self.rule_stack) -## x = raw_input("Continue ?") return True def sequence(self, rule, source, elts_number): """ """ ## rule_stack = self.rule_stack[:] if rule.is_root(): -## if DEBUG_MODE: -## print "SEQ:", sym.sym_name[rule.codename] builder_func = ASTRULES.get(rule.codename, None) if builder_func: - # print "REDUCING SEQUENCE %s" % sym.sym_name[rule.codename] builder_func(self, elts_number) else: -## if DEBUG_MODE: -## print "No reducing implementation for %s, just push it on stack" % ( -## sym.sym_name[rule.codename]) self.push_rule(rule.codename, elts_number, source) else: self.push_rule(rule.codename, elts_number, source) -## if DEBUG_MODE > 1: -## show_stack(rule_stack, self.rule_stack) -## raw_input("Continue ?") return True def token(self, name, value, source): -## if DEBUG_MODE: -## print "TOK:", tok.tok_name[name], name, value self.push_tok(name, value, source) return True @@ -1723,12 +1702,12 @@ l = space.builtin.get('long') return space.call_function(l, space.wrap(value), space.wrap(base)) if value.endswith('j') or value.endswith('J'): - c = space.builtin.get('complex') + c = space.builtin.get('complex') return space.call_function(c, space.wrap(value)) try: i = space.builtin.get('int') return space.call_function(i, space.wrap(value), space.wrap(base)) - except: + except: f = space.builtin.get('float') return space.call_function(f, space.wrap(value)) @@ -1765,4 +1744,4 @@ else: obj2 = "-" print "% 3d | %30s | %30s" % (i, obj1, obj2) - + Modified: pypy/branch/ast-experiments/pypy/interpreter/pyparser/ebnfgrammar.py ============================================================================== --- pypy/branch/ast-experiments/pypy/interpreter/pyparser/ebnfgrammar.py (original) +++ pypy/branch/ast-experiments/pypy/interpreter/pyparser/ebnfgrammar.py Sat Jan 28 02:10:44 2006 @@ -2,45 +2,49 @@ # and the symbol mappings from grammar import BaseGrammarBuilder, Alternative, Sequence, Token, \ - KleeneStar, GrammarElement, build_first_sets, EmptyToken + KleeneStar, GrammarElement +from pypy.interpreter.pyparser.parser import Parser -sym_map = {} -sym_rmap = {} -_count = 0 - -def g_add_symbol( name ): - global _count - if name in sym_rmap: - return sym_rmap[name] - val = _count - _count += 1 - sym_map[val] = name - sym_rmap[name] = val - globals()[name] = val - return val - - -tok_map = {} -tok_rmap = {} - -def g_add_token(sym, name): - global _count - if name in tok_rmap: - return tok_rmap[name] - val = _count - _count += 1 - tok_map[val] = name - tok_rmap[name] = val - sym_map[val] = sym - sym_rmap[sym] = val - globals()[sym] = val - return val +## sym_map = {} +## sym_rmap = {} +## _count = 0 + +## def g_add_symbol( name ): +## global _count +## if name in sym_rmap: +## return sym_rmap[name] +## val = _count +## _count += 1 +## sym_map[val] = name +## sym_rmap[name] = val +## globals()[name] = val +## return val + + +## tok_map = {} +## tok_rmap = {} + +## def g_add_token(sym, name): +## global _count +## if name in tok_rmap: +## return tok_rmap[name] +## val = _count +## _count += 1 +## tok_map[val] = name +## tok_rmap[name] = val +## sym_map[val] = sym +## sym_rmap[sym] = val +## globals()[sym] = val +## return val -g_add_token('EOF', 'EOF') +## g_add_token('EOF', 'EOF') +class GrammarParser(Parser): + pass +GRAMMAR_GRAMMAR = GrammarParser() def grammar_grammar(): @@ -60,54 +64,57 @@ group: '(' alternative ')' star? """ global sym_map - S = g_add_symbol - T = g_add_token + p = GRAMMAR_GRAMMAR + p.add_token('EOF','EOF') + # star: '*' | '+' - star = Alternative( S("star"), [Token(T('TOK_STAR', '*')), Token(T('TOK_ADD', '+'))] ) - star_opt = KleeneStar ( S("star_opt"), 0, 1, rule=star ) + star = p.Alternative( "star", [p.Token('TOK_STAR', '*'), p.Token('TOK_ADD', '+')] ) + star_opt = p.KleeneStar ( "star_opt", 0, 1, rule=star ) # rule: SYMBOL ':' alternative - symbol = Sequence( S("symbol"), [Token(T('TOK_SYMBOL', 'SYMBOL')), star_opt] ) - symboldef = Token( T('TOK_SYMDEF', 'SYMDEF') ) - alternative = Sequence( S("alternative"), []) - rule = Sequence( S("rule"), [symboldef, alternative] ) + symbol = p.Sequence( "symbol", [p.Token('TOK_SYMBOL'), star_opt] ) + symboldef = p.Token( 'TOK_SYMDEF' ) + alternative = p.Sequence( "alternative", []) + rule = p.Sequence( "rule", [symboldef, alternative] ) # grammar: rule+ - grammar = KleeneStar( S("grammar"), _min=1, rule=rule ) + grammar = p.KleeneStar( "grammar", _min=1, rule=rule ) # alternative: sequence ( '|' sequence )* - sequence = KleeneStar( S("sequence"), 1 ) - seq_cont_list = Sequence( S("seq_cont_list"), [Token(T('TOK_BAR', '|')), sequence] ) - sequence_cont = KleeneStar( S("sequence_cont"),0, rule=seq_cont_list ) - + sequence = p.KleeneStar( "sequence", 1 ) + seq_cont_list = p.Sequence( "seq_cont_list", [p.Token('TOK_BAR', '|'), sequence] ) + sequence_cont = p.KleeneStar( "sequence_cont",0, rule=seq_cont_list ) + alternative.args = [ sequence, sequence_cont ] # option: '[' alternative ']' - option = Sequence( S("option"), [Token(T('TOK_LBRACKET', '[')), alternative, Token(T('TOK_RBRACKET', ']'))] ) + option = p.Sequence( "option", [p.Token('TOK_LBRACKET', '['), alternative, p.Token('TOK_RBRACKET', ']')] ) # group: '(' alternative ')' - group = Sequence( S("group"), [Token(T('TOK_LPAR', '(')), alternative, Token(T('TOK_RPAR', ')')), star_opt] ) + group = p.Sequence( "group", [p.Token('TOK_LPAR', '('), alternative, p.Token('TOK_RPAR', ')'), star_opt] ) # sequence: (SYMBOL | STRING | option | group )+ - string = Token(T('TOK_STRING', 'STRING')) - alt = Alternative( S("sequence_alt"), [symbol, string, option, group] ) + string = p.Token('TOK_STRING') + alt = p.Alternative( "sequence_alt", [symbol, string, option, group] ) sequence.args = [ alt ] + p.root_rules['grammar'] = grammar + p.build_first_sets() + return p - rules = [ star, star_opt, symbol, alternative, rule, grammar, sequence, - seq_cont_list, sequence_cont, option, group, alt ] - build_first_sets( rules ) - return grammar - - -GRAMMAR_GRAMMAR = grammar_grammar() -for _sym, _value in sym_rmap.items(): - globals()[_sym] = _value +grammar_grammar() +for _sym, _value in GRAMMAR_GRAMMAR.symbols.items(): + assert not hasattr( GRAMMAR_GRAMMAR, _sym ) + setattr(GRAMMAR_GRAMMAR, _sym, _value ) + +for _sym, _value in GRAMMAR_GRAMMAR.tokens.items(): + assert not hasattr( GRAMMAR_GRAMMAR, _sym ) + setattr(GRAMMAR_GRAMMAR, _sym, _value ) # cleanup -del _sym -del _value +## del _sym +## del _value del grammar_grammar -del g_add_symbol +## del g_add_symbol # del g_add_token Modified: pypy/branch/ast-experiments/pypy/interpreter/pyparser/ebnflexer.py ============================================================================== --- pypy/branch/ast-experiments/pypy/interpreter/pyparser/ebnflexer.py (original) +++ pypy/branch/ast-experiments/pypy/interpreter/pyparser/ebnflexer.py Sat Jan 28 02:10:44 2006 @@ -4,7 +4,7 @@ """ from grammar import TokenSource, Token, AbstractContext -from ebnfgrammar import * +from ebnfgrammar import GRAMMAR_GRAMMAR as G def match_symbol( input, start, stop ): @@ -31,8 +31,9 @@ SYMBOL: a rule symbol usually appeary right of a SYMDEF tokens: '[', ']', '(' ,')', '*', '+', '|' """ - def __init__(self, inpstring): + def __init__(self, parser, inpstring): # TokenSource.__init__(self) + self.parser = parser self.input = inpstring self.pos = 0 self.begin = 0 @@ -58,7 +59,6 @@ assert isinstance( ctx, GrammarSourceContext ) self.pos = ctx.pos self._peeked = ctx.peek - def current_linesource(self): pos = idx = self.begin @@ -74,7 +74,6 @@ def current_lineno(self): return self.current_line - def skip_empty_lines(self, input, start, end ): idx = start # assume beginning of a line @@ -130,13 +129,13 @@ peeked = self._peeked self._peeked = None return peeked - + pos = self.pos inp = self.input end = len(self.input) pos = self.skip_empty_lines(inp,pos,end) if pos==end: - return Token(EOF, None) + return self.parser.Token( 'EOF', None) # at this point nextchar is not a white space nor \n nextchr = inp[pos] @@ -148,22 +147,22 @@ self.pos = npos _endpos = npos - 1 assert _endpos>=0 - return Token(TOK_STRING,inp[pos+1:_endpos]) + return self.parser.Token( 'TOK_STRING', inp[pos+1:_endpos]) else: npos = match_symbol( inp, pos, end) if npos!=pos: self.pos = npos if npos!=end and inp[npos]==":": self.pos += 1 - return Token(TOK_SYMDEF,inp[pos:npos]) + return self.parser.Token( 'TOK_SYMDEF', inp[pos:npos]) else: - return Token(TOK_SYMBOL,inp[pos:npos]) - + return self.parser.Token( 'TOK_SYMBOL', inp[pos:npos]) + # we still have pos!=end here chr = inp[pos] if chr in "[]()*+|": self.pos = pos+1 - return Token(tok_rmap[chr], chr) + return Token( self.parser, self.parser.tok_values[chr], chr) self.RaiseError( "Unknown token" ) def peek(self): Modified: pypy/branch/ast-experiments/pypy/interpreter/pyparser/ebnfparse.py ============================================================================== --- pypy/branch/ast-experiments/pypy/interpreter/pyparser/ebnfparse.py (original) +++ pypy/branch/ast-experiments/pypy/interpreter/pyparser/ebnfparse.py Sat Jan 28 02:10:44 2006 @@ -1,13 +1,12 @@ #!/usr/bin/env python from grammar import BaseGrammarBuilder, Alternative, Sequence, Token from grammar import GrammarProxy, KleeneStar, GrammarElement, build_first_sets -from grammar import EmptyToken, AbstractBuilder, AbstractContext +from grammar import AbstractBuilder, AbstractContext from ebnflexer import GrammarSource import ebnfgrammar -from ebnfgrammar import GRAMMAR_GRAMMAR, sym_map +from ebnfgrammar import GRAMMAR_GRAMMAR from syntaxtree import AbstractSyntaxVisitor -import pytoken -import pysymbol +from parser import Parser ORDA = ord("A") @@ -56,8 +55,8 @@ class NameToken(Token): """A token that is not a keyword""" - def __init__(self, keywords=None ): - Token.__init__(self, pytoken.NAME) + def __init__(self, parser, keywords=None ): + Token.__init__(self, parser, parser.tokens['NAME'] ) self.keywords = keywords def match(self, source, builder, level=0): @@ -95,191 +94,73 @@ -def ebnf_handle_grammar(self, node): - for rule in node.nodes: - rule.visit(self) - # the rules are registered already - # we do a pass through the variables to detect - # terminal symbols from non terminals - for r in self.items: - for i in range(len(r.args)): - a = r.args[i] - if a.codename in self.rules: - assert isinstance(a,Token) - r.args[i] = self.rules[a.codename] - if a.codename in self.terminals: - del self.terminals[a.codename] - # XXX .keywords also contains punctuations - self.terminals['NAME'].keywords = self.keywords - -def ebnf_handle_rule(self, node): - symdef = node.nodes[0].value - self.current_rule = symdef - self.current_subrule = 0 - alt = node.nodes[1] - rule = alt.visit(self) - if not isinstance(rule, Token): - rule.codename = self.symbols.add_symbol( symdef ) - self.rules[rule.codename] = rule - -def ebnf_handle_alternative(self, node): - items = [node.nodes[0].visit(self)] - items += node.nodes[1].visit(self) - if len(items) == 1 and not items[0].is_root(): - return items[0] - alt = Alternative(self.new_symbol(), items) - return self.new_item(alt) - -def ebnf_handle_sequence( self, node ): - """ """ - items = [] - for n in node.nodes: - items.append( n.visit(self) ) - if len(items)==1: - return items[0] - elif len(items)>1: - return self.new_item( Sequence( self.new_symbol(), items) ) - raise RuntimeError("Found empty sequence") - -def ebnf_handle_sequence_cont( self, node ): - """Returns a list of sequences (possibly empty)""" - return [n.visit(self) for n in node.nodes] - -def ebnf_handle_seq_cont_list(self, node): - return node.nodes[1].visit(self) - - -def ebnf_handle_symbol(self, node): - star_opt = node.nodes[1] - sym = node.nodes[0].value - terminal = self.terminals.get( sym, None ) - if not terminal: - tokencode = pytoken.tok_values.get( sym, None ) - if tokencode is None: - tokencode = self.symbols.add_symbol( sym ) - terminal = Token( tokencode ) - else: - terminal = Token( tokencode ) - self.terminals[sym] = terminal - - return self.repeat( star_opt, terminal ) - -def ebnf_handle_option( self, node ): - rule = node.nodes[1].visit(self) - return self.new_item( KleeneStar( self.new_symbol(), 0, 1, rule ) ) - -def ebnf_handle_group( self, node ): - rule = node.nodes[1].visit(self) - return self.repeat( node.nodes[3], rule ) - -def ebnf_handle_TOK_STRING( self, node ): - value = node.value - tokencode = pytoken.tok_punct.get( value, None ) - if tokencode is None: - if not is_py_name( value ): - raise RuntimeError("Unknown STRING value ('%s')" % value ) - # assume a keyword - tok = Token( pytoken.NAME, value ) - if value not in self.keywords: - self.keywords.append( value ) - else: - # punctuation - tok = Token( tokencode ) - return tok - -def ebnf_handle_sequence_alt( self, node ): - res = node.nodes[0].visit(self) - assert isinstance( res, GrammarElement ) - return res - -# This will setup a mapping between -# ebnf_handle_xxx functions and ebnfgrammar.xxx -ebnf_handles = {} -for name, value in globals().items(): - if name.startswith("ebnf_handle_"): - name = name[12:] - key = getattr(ebnfgrammar, name ) - ebnf_handles[key] = value - -def handle_unknown( self, node ): - raise RuntimeError("Unknown Visitor for %r" % node.name) - - - class EBNFBuilder(AbstractBuilder): """Build a grammar tree""" - def __init__(self, rules=None, debug=0, symbols=None ): - if symbols is None: - symbols = pysymbol.SymbolMapper() - AbstractBuilder.__init__(self, rules, debug, symbols) + def __init__(self, gram_parser, dest_parser ): + AbstractBuilder.__init__(self, dest_parser ) + self.gram = gram_parser self.rule_stack = [] - self.root_rules = {} self.seqcounts = [] # number of items in the current sequence self.altcounts = [] # number of sequence in the current alternative self.curaltcount = 0 self.curseqcount = 0 self.current_subrule = 0 self.current_rule = -1 - self.all_rules = [] + self.current_rule_name = "" self.tokens = {} self.keywords = [] - self.tokens[pytoken.NAME] = NameToken(keywords=self.keywords) + NAME = dest_parser.add_token('NAME') + self.tokens[NAME] = NameToken(dest_parser, keywords=self.keywords) def new_symbol(self): """Allocate and return a new (anonymous) grammar symbol whose name is based on the current grammar rule being parsed""" - current_rule_name = self.symbols.sym_name.get(self.current_rule,"x") - rule_name = ":" + current_rule_name + "_%d" % self.current_subrule + rule_name = ":" + self.current_rule_name + "_%d" % self.current_subrule self.current_subrule += 1 - symval = self.symbols.add_anon_symbol( rule_name ) - return symval + return rule_name def new_rule(self, rule): """A simple helper method that registers a new rule as 'known'""" - self.all_rules.append(rule) + self.parser.all_rules.append(rule) return rule def resolve_rules(self): """Remove GrammarProxy objects""" to_be_deleted = {} - for rule in self.all_rules: + for rule in self.parser.all_rules: for i, arg in enumerate(rule.args): if isinstance(arg, GrammarProxy): - real_rule = self.root_rules[arg.codename] + real_rule = self.parser.root_rules[arg.codename] if isinstance(real_rule, GrammarProxy): # If we still have a GrammarProxy associated to this codename # this means we have encountered a terminal symbol to_be_deleted[ arg.codename ] = True - rule.args[i] = self.get_token( arg.codename ) + rule.args[i] = Token( self.parser, arg.codename ) #print arg, "-> Token(",arg.rule_name,")" else: #print arg, "->", real_rule rule.args[i] = real_rule for codename in to_be_deleted.keys(): - del self.root_rules[codename] + del self.parser.root_rules[codename] - def get_token(self, codename ): - """Returns a new or existing token""" - if codename in self.tokens: - return self.tokens[codename] - token = self.tokens[codename] = Token(codename) - return token +## def get_token(self, codename ): +## """Returns a new or existing Token""" +## if codename in self.tokens: +## return self.tokens[codename] +## token = self.tokens[codename] = self.parser.Token(codename) +## return token def get_symbolcode(self, name ): - codename = self.symbols.sym_values.get( name, -1 ) - if codename == -1: - codename = self.symbols.add_symbol( name ) - return codename + return self.parser.add_symbol( name ) def get_rule( self, name ): - tokencode = pytoken.tok_values.get( name, -1 ) - if tokencode>=0: - return self.get_token( tokencode ) + if name in self.parser.tokens: + return self.parser.Token( name ) codename = self.get_symbolcode( name ) - if codename in self.root_rules: - return self.root_rules[codename] - proxy = GrammarProxy( name, codename ) - self.root_rules[codename] = proxy + if codename in self.parser.root_rules: + return self.parser.root_rules[codename] + proxy = GrammarProxy( self.parser, name, codename ) + self.parser.root_rules[codename] = proxy return proxy def context(self): @@ -291,7 +172,6 @@ assert False, "Not supported" def alternative(self, rule, source): -# print " alternative", rule.display(level=0,symbols=ebnfgrammar.sym_map) return True def pop_rules( self, count ): @@ -302,89 +182,89 @@ return rules def sequence(self, rule, source, elts_number): -# print " sequence", rule.display(level=0,symbols=ebnfgrammar.sym_map) _rule = rule.codename - if _rule == ebnfgrammar.sequence: + if _rule == self.gram.sequence: # print " -sequence", self.curaltcount, self.curseqcount if self.curseqcount==1: self.curseqcount = 0 self.curaltcount += 1 return True rules = self.pop_rules(self.curseqcount) - new_rule = self.new_rule(Sequence( self.new_symbol(), rules )) + new_rule = self.parser.Sequence( self.new_symbol(), rules ) self.rule_stack.append( new_rule ) self.curseqcount = 0 self.curaltcount += 1 - elif _rule == ebnfgrammar.alternative: + elif _rule == self.gram.alternative: # print " -alternative", self.curaltcount, self.curseqcount if self.curaltcount == 1: self.curaltcount = 0 return True rules = self.pop_rules(self.curaltcount) - new_rule = self.new_rule(Alternative( self.new_symbol(), rules )) + new_rule = self.parser.Alternative( self.new_symbol(), rules ) self.rule_stack.append( new_rule ) self.curaltcount = 0 - elif _rule == ebnfgrammar.group: + elif _rule == self.gram.group: # print " -group", self.curaltcount, self.curseqcount self.curseqcount += 1 - elif _rule == ebnfgrammar.option: + elif _rule == self.gram.option: # print " -option", self.curaltcount, self.curseqcount # pops the last alternative rules = self.pop_rules( 1 ) - new_rule = self.new_rule(KleeneStar( self.new_symbol(), _min=0, _max=1, rule=rules[0] )) + new_rule = self.parser.KleeneStar( self.new_symbol(), _min=0, _max=1, rule=rules[0] ) self.rule_stack.append( new_rule ) self.curseqcount += 1 - elif _rule == ebnfgrammar.rule: + elif _rule == self.gram.rule: # print " -rule", self.curaltcount, self.curseqcount assert len(self.rule_stack)==1 old_rule = self.rule_stack[0] del self.rule_stack[0] if isinstance(old_rule,Token): # Wrap a token into an alternative - old_rule = self.new_rule(Alternative( self.current_rule, [old_rule] )) + old_rule = self.parser.Alternative( self.current_rule_name, [old_rule] ) else: # Make sure we use the codename from the named rule old_rule.codename = self.current_rule - self.root_rules[self.current_rule] = old_rule + self.parser.root_rules[self.current_rule] = old_rule self.current_subrule = 0 return True def token(self, name, value, source): # print "token", name, value - if name == ebnfgrammar.TOK_STRING: + if name == self.gram.TOK_STRING: self.handle_TOK_STRING( name, value ) self.curseqcount += 1 - elif name == ebnfgrammar.TOK_SYMDEF: + elif name == self.gram.TOK_SYMDEF: self.current_rule = self.get_symbolcode( value ) - elif name == ebnfgrammar.TOK_SYMBOL: + self.current_rule_name = value + elif name == self.gram.TOK_SYMBOL: rule = self.get_rule( value ) self.rule_stack.append( rule ) self.curseqcount += 1 - elif name == ebnfgrammar.TOK_STAR: + elif name == self.gram.TOK_STAR: top = self.rule_stack[-1] - rule = self.new_rule(KleeneStar( self.new_symbol(), _min=0, rule=top)) + rule = self.parser.KleeneStar( self.new_symbol(), _min=0, rule=top) self.rule_stack[-1] = rule - elif name == ebnfgrammar.TOK_ADD: + elif name == self.gram.TOK_ADD: top = self.rule_stack[-1] - rule = self.new_rule(KleeneStar( self.new_symbol(), _min=1, rule=top)) + rule = self.parser.KleeneStar( self.new_symbol(), _min=1, rule=top) self.rule_stack[-1] = rule - elif name == ebnfgrammar.TOK_BAR: + elif name == self.gram.TOK_BAR: assert self.curseqcount == 0 - elif name == ebnfgrammar.TOK_LPAR: + elif name == self.gram.TOK_LPAR: self.altcounts.append( self.curaltcount ) self.seqcounts.append( self.curseqcount ) self.curseqcount = 0 self.curaltcount = 0 - elif name == ebnfgrammar.TOK_RPAR: + elif name == self.gram.TOK_RPAR: assert self.curaltcount == 0 self.curaltcount = self.altcounts.pop() self.curseqcount = self.seqcounts.pop() - elif name == ebnfgrammar.TOK_LBRACKET: + elif name == self.gram.TOK_LBRACKET: self.altcounts.append( self.curaltcount ) self.seqcounts.append( self.curseqcount ) self.curseqcount = 0 self.curaltcount = 0 - elif name == ebnfgrammar.TOK_RBRACKET: + elif name == self.gram.TOK_RBRACKET: assert self.curaltcount == 0 assert self.curseqcount == 0 self.curaltcount = self.altcounts.pop() @@ -392,95 +272,31 @@ return True def handle_TOK_STRING( self, name, value ): - try: - tokencode = pytoken.tok_punct[value] - except KeyError: + if value in self.parser.tok_values: + # punctuation + tokencode = self.parser.tok_values[value] + tok = Token( self.parser, tokencode, None ) + else: if not is_py_name(value): raise RuntimeError("Unknown STRING value ('%s')" % value) # assume a keyword - tok = Token(pytoken.NAME, value) + tok = Token( self.parser, self.parser.NAME, value) if value not in self.keywords: self.keywords.append(value) - else: - # punctuation - tok = Token(tokencode, None) self.rule_stack.append(tok) -class EBNFVisitor(AbstractSyntaxVisitor): - - def __init__(self): - self.rules = {} - self.terminals = {} - self.current_rule = None - self.current_subrule = 0 - self.keywords = [] - self.items = [] - self.terminals['NAME'] = NameToken() - self.symbols = pysymbol.SymbolMapper( pysymbol._cpython_symbols.sym_name ) - - def new_symbol(self): - current_rule_name = self.symbols.sym_name.get(self.current_rule,"x") - rule_name = ":" + self.current_rule + "_" + str(self.current_subrule) - self.current_subrule += 1 - symval = self.symbols.add_anon_symbol( rule_name ) - return symval - - def new_item(self, itm): - self.items.append(itm) - return itm - - def visit_syntaxnode( self, node ): - visit_func = ebnf_handles.get( node.name, handle_unknown ) - return visit_func( self, node ) - - def visit_tokennode( self, node ): - return self.visit_syntaxnode( node ) - - def visit_tempsyntaxnode( self, node ): - return self.visit_syntaxnode( node ) - - - def repeat( self, star_opt, myrule ): - assert isinstance( myrule, GrammarElement ) - if star_opt.nodes: - rule_name = self.new_symbol() - tok = star_opt.nodes[0].nodes[0] - if tok.value == '+': - item = KleeneStar(rule_name, _min=1, rule=myrule) - return self.new_item(item) - elif tok.value == '*': - item = KleeneStar(rule_name, _min=0, rule=myrule) - return self.new_item(item) - else: - raise RuntimeError("Got symbol star_opt with value='%s'" - % tok.value) - return myrule - - - -def parse_grammar(stream): - """parses the grammar file - - stream : file-like object representing the grammar to parse - """ - source = GrammarSource(stream.read()) - builder = BaseGrammarBuilder() - result = GRAMMAR_GRAMMAR.match(source, builder) - node = builder.stack[-1] - vis = EBNFVisitor() - node.visit(vis) - return vis - -def parse_grammar_text(txt): +def parse_grammar_text( parser, txt): """parses a grammar input stream : file-like object representing the grammar to parse """ - source = GrammarSource(txt) - builder = EBNFBuilder(pysymbol._cpython_symbols) - result = GRAMMAR_GRAMMAR.match(source, builder) - return builder + source = GrammarSource( GRAMMAR_GRAMMAR, txt) + builder = EBNFBuilder(GRAMMAR_GRAMMAR, dest_parser=parser ) + result = GRAMMAR_GRAMMAR.root_rules['grammar'].match(source, builder) + builder.resolve_rules() + parser.build_first_sets() + return parser def target_parse_grammar_text(txt): vis = parse_grammar_text(txt) Modified: pypy/branch/ast-experiments/pypy/interpreter/pyparser/grammar.py ============================================================================== --- pypy/branch/ast-experiments/pypy/interpreter/pyparser/grammar.py (original) +++ pypy/branch/ast-experiments/pypy/interpreter/pyparser/grammar.py Sat Jan 28 02:10:44 2006 @@ -10,13 +10,13 @@ try: from pypy.interpreter.baseobjspace import Wrappable from pypy.interpreter.pyparser.pytoken import NULLTOKEN - from pypy.interpreter.pyparser.pysymbol import SymbolMapper + from pypy.interpreter.pyparser.parser import Parser except ImportError: # allows standalone testing Wrappable = object NULLTOKEN = -1 # None - from pysymbol import SymbolMapper - + from parser import Parser + from syntaxtree import SyntaxNode, TempSyntaxNode, TokenNode @@ -75,7 +75,8 @@ def build_first_sets(rules): - """builds the real first tokens set for each rule in + """XXX : dead + builds the real first tokens set for each rule in Because a rule can be recursive (directly or indirectly), the *simplest* algorithm to build each first set is to recompute them @@ -106,17 +107,12 @@ class AbstractBuilder(object): """Abstract base class for builder objects""" - def __init__(self, symbols, rules=None, debug=0 ): - # a dictionary of grammar rules for debug/reference - if rules is not None: - self.rules = rules - else: - self.rules = {} + def __init__(self, parser, debug=0 ): # This attribute is here for convenience self.debug = debug - # mapping from codename to symbols - assert isinstance( symbols, SymbolMapper ) - self.symbols = symbols + # the parser that represent the grammar used + assert isinstance( parser, Parser ) + self.parser = parser def context(self): """Return an opaque context object""" @@ -148,24 +144,20 @@ class BaseGrammarBuilder(AbstractBuilder): """Base/default class for a builder""" - def __init__(self, rules=None, debug=0, symbols={} ): - if rules is None: - rules = SymbolMapper() - AbstractBuilder.__init__(self, rules, debug, symbols ) + def __init__(self, parser, debug=0 ): + AbstractBuilder.__init__(self, parser, debug ) # stacks contain different objects depending on the builder class # to be RPython they should not be defined in the base class self.stack = [] def context(self): """Returns the state of the builder to be restored later""" - #print "Save Stack:", self.stack return BaseGrammarBuilderContext(len(self.stack)) def restore(self, ctx): assert isinstance(ctx, BaseGrammarBuilderContext) del self.stack[ctx.stackpos:] - #print "Restore Stack:", self.stack - + def alternative(self, rule, source): # Do nothing, keep rule on top of the stack if rule.is_root(): @@ -216,9 +208,11 @@ symbols = {} # dirty trick to provide a symbols mapping while printing (and not putting it in every object) - def __init__(self, codename): + def __init__(self, parser, codename): # the rule name #assert type(codename)==int + assert isinstance(parser, Parser) + self.parser = parser self.codename = codename # integer mapping to either a token value or rule symbol value self.args = [] self.first_set = [] @@ -233,7 +227,6 @@ if self.codename >=0: return True return False - def match(self, source, builder, level=0): """Try to match a grammar rule @@ -256,17 +249,17 @@ pos1 = source.get_pos() in_first_set = self.match_first_set(token) if not in_first_set: # and not EmptyToken in self.first_set: - if EmptyToken in self.first_set: + if self.parser.EmptyToken in self.first_set: ret = builder.sequence(self, source, 0 ) if self._trace: - self._debug_display(token, level, 'eee', builder.symbols) + self._debug_display(token, level, 'eee' ) return ret if self._trace: - self._debug_display(token, level, 'rrr', builder.symbols) + self._debug_display(token, level, 'rrr' ) return 0 elif self._trace: - self._debug_display(token, level, '>>>', builder.symbols) - + self._debug_display(token, level, '>>>') + res = self._match(source, builder, level) if self._trace: pos2 = source.get_pos() @@ -274,21 +267,20 @@ prefix = '+++' else: prefix = '---' - self._debug_display(token, level, prefix, builder.symbols) + self._debug_display(token, level, prefix) print ' '*level, prefix, " TEXT ='%s'" % ( source.get_source_text(pos1,pos2)) if res: print "*" * 50 return res - def _debug_display(self, token, level, prefix, symbols): + def _debug_display(self, token, level, prefix): """prints context debug informations""" prefix = '%s%s' % (' ' * level, prefix) print prefix, " RULE =", self print prefix, " TOKEN =", token print prefix, " FIRST SET =", self.first_set - - + def _match(self, source, builder, level=0): """Try to match a grammar rule @@ -302,7 +294,7 @@ returns None if no match or an object build by builder """ return 0 - + def parse(self, source): """Returns a simplified grammar if the rule matched at the source current context or None""" @@ -311,43 +303,27 @@ pass def __str__(self): - # XXX: remove me after debug - symbols = {} - import pytoken - import pysymbol - symbols.update( pysymbol._cpython_symbols.sym_name ) - symbols.update( pytoken.tok_name ) - - return self.display(0, symbols ) -# return self.display(0, GrammarElement.symbols ) + return self.display(0) def __repr__(self): - # XXX: remove me after debug - symbols = {} - import pytoken - import pysymbol - symbols.update( pysymbol._cpython_symbols.sym_name ) - symbols.update( pytoken.tok_name ) - - return self.display(0, symbols ) -# return self.display(0, GrammarElement.symbols ) + return self.display(0) - def display(self, level=0, symbols={}): + def display(self, level=0): """Helper function used to represent the grammar. mostly used for debugging the grammar itself""" return "GrammarElement" - def debug_return(self, ret, symbols, arg="" ): + def debug_return(self, ret, arg="" ): # FIXME: use a wrapper of match() methods instead of debug_return() # to prevent additional indirection even better a derived # Debugging builder class if ret and DEBUG > 0: print "matched %s (%s): %s" % (self.__class__.__name__, - arg, self.display(0, symbols=symbols) ) + arg, self.display(0) ) return ret - + def calc_first_set(self): """returns the list of possible next tokens *must* be implemented in subclasses @@ -380,27 +356,27 @@ class GrammarProxy(GrammarElement): - def __init__(self, rule_name, codename=-1 ): - GrammarElement.__init__(self, codename ) + def __init__(self, parser, rule_name, codename=-1 ): + GrammarElement.__init__(self, parser, codename ) self.rule_name = rule_name self.object = None - def display(self, level=0, symbols={}): + def display(self, level=0): """Helper function used to represent the grammar. mostly used for debugging the grammar itself""" - name = symbols.get(self.codename, self.rule_name) + name = self.parser.symbol_repr(self.codename) repr = "Proxy("+name if self.object: - repr+=","+self.object.display(level=1,symbols=symbols) + repr+=","+self.object.display(1) repr += ")" return repr - + class Alternative(GrammarElement): """Represents an alternative in a grammar rule (as in S -> A | B | C)""" - def __init__(self, name, args): - GrammarElement.__init__(self, name ) + def __init__(self, parser, name, args): + GrammarElement.__init__(self, parser, name ) self.args = args self._reordered = False for i in self.args: @@ -418,7 +394,7 @@ # to see if this solve our problems with infinite recursion for rule in self.args: if USE_LOOKAHEAD: - if not rule.match_first_set(tok) and EmptyToken not in rule.first_set: + if not rule.match_first_set(tok) and self.parser.EmptyToken not in rule.first_set: if self._trace: print "Skipping impossible rule: %s" % (rule,) continue @@ -428,15 +404,15 @@ return ret return 0 - def display(self, level=0, symbols={}): - name = get_symbol( self.codename, symbols ) + def display(self, level=0): + name = self.parser.symbol_repr( self.codename ) if level == 0: name = name + " -> " elif self.is_root(): return name else: name = "" - items = [ a.display(1,symbols) for a in self.args ] + items = [ a.display(1) for a in self.args ] return name+"(" + "|".join( items ) + ")" def calc_first_set(self): @@ -460,7 +436,7 @@ # is only needed for warning / debugging purposes tokens_set = [] for rule in self.args: - if EmptyToken in rule.first_set: + if self.parser.EmptyToken in rule.first_set: empty_set.append(rule) else: not_empty_set.append(rule) @@ -469,7 +445,7 @@ # It will check if a token is part of several first sets of # a same alternative for token in rule.first_set: - if token is not EmptyToken and token in tokens_set: + if token is not self.parser.EmptyToken and token in tokens_set: print "Warning, token %s in\n\t%s's first set is " \ " part of a previous rule's first set in " \ " alternative\n\t%s" % (token, rule, self) @@ -497,8 +473,8 @@ class Sequence(GrammarElement): """Reprensents a Sequence in a grammar rule (as in S -> A B C)""" - def __init__(self, name, args): - GrammarElement.__init__(self, name ) + def __init__(self, parser, name, args): + GrammarElement.__init__(self, parser, name ) self.args = args for i in self.args: assert isinstance( i, GrammarElement ) @@ -520,15 +496,15 @@ ret = builder.sequence(self, source, len(self.args)) return ret - def display(self, level=0, symbols={}): - name = get_symbol( self.codename, symbols ) + def display(self, level=0): + name = self.parser.symbol_repr( self.codename ) if level == 0: name = name + " -> " elif self.is_root(): return name else: name = "" - items = [a.display(1,symbols) for a in self.args] + items = [a.display(1) for a in self.args] return name + "(" + " ".join( items ) + ")" def calc_first_set(self): @@ -543,18 +519,18 @@ for rule in self.args: if not rule.first_set: break - if EmptyToken in self.first_set: - self.first_set.remove( EmptyToken ) + if self.parser.EmptyToken in self.first_set: + self.first_set.remove( self.parser.EmptyToken ) - # del self.first_set[EmptyToken] + # del self.first_set[self.parser.EmptyToken] # while we're in this loop, keep agregating possible tokens for t in rule.first_set: if t not in self.first_set: self.first_set.append(t) # self.first_set[t] = 1 - if EmptyToken not in rule.first_set: + if self.parser.EmptyToken not in rule.first_set: break - + def validate( self, syntax_node ): """validate a syntax tree/subtree from this grammar node""" if self.codename != syntax_node.name: @@ -570,13 +546,10 @@ - - - class KleeneStar(GrammarElement): """Represents a KleeneStar in a grammar rule as in (S -> A+) or (S -> A*)""" - def __init__(self, name, _min = 0, _max = -1, rule=None): - GrammarElement.__init__( self, name ) + def __init__(self, parser, name, _min = 0, _max = -1, rule=None): + GrammarElement.__init__( self, parser, name ) self.args = [rule] self.min = _min if _max == 0: @@ -584,8 +557,8 @@ self.max = _max self.star = "x" if self.min == 0: - self.first_set.append( EmptyToken ) - # self.first_set[EmptyToken] = 1 + self.first_set.append( self.parser.EmptyToken ) + # self.first_set[self.parser.EmptyToken] = 1 def _match(self, source, builder, level=0): """matches a number of times self.args[0]. the number must be @@ -616,14 +589,19 @@ ret = builder.sequence(self, source, rules) return ret - def display(self, level=0, symbols={}): - name = get_symbol( self.codename, symbols ) + def display(self, level=0): + name = self.parser.symbol_repr( self.codename ) if level==0: name = name + " -> " elif self.is_root(): return name else: name = "" + star = self.get_star() + s = self.args[0].display(1) + return name + "%s%s" % (s, star) + + def get_star(self): star = "{%d,%d}" % (self.min,self.max) if self.min==0 and self.max==1: star = "?" @@ -631,23 +609,21 @@ star = "*" elif self.min==1 and self.max==-1: star = "+" - s = self.args[0].display(1, symbols) - return name + "%s%s" % (s, star) - + return star def calc_first_set(self): """returns the list of possible next tokens if S -> A*: - LAH(S) = Union( LAH(A), EmptyToken ) + LAH(S) = Union( LAH(A), self.parser.EmptyToken ) if S -> A+: LAH(S) = LAH(A) """ rule = self.args[0] self.first_set = rule.first_set[:] # self.first_set = dict(rule.first_set) - if self.min == 0 and EmptyToken not in self.first_set: - self.first_set.append(EmptyToken) - # self.first_set[EmptyToken] = 1 + if self.min == 0 and self.parser.EmptyToken not in self.first_set: + self.first_set.append(self.parser.EmptyToken) + # self.first_set[self.parser.EmptyToken] = 1 def validate( self, syntax_node ): """validate a syntax tree/subtree from this grammar node""" @@ -666,8 +642,8 @@ class Token(GrammarElement): """Represents a Token in a grammar rule (a lexer token)""" - def __init__(self, codename, value=None): - GrammarElement.__init__(self, codename) + def __init__(self, parser, codename, value=None): + GrammarElement.__init__(self, parser, codename) self.value = value self.first_set = [self] # self.first_set = {self: 1} @@ -697,13 +673,13 @@ source.restore( ctx ) return 0 - def display(self, level=0, symbols={}): - name = get_symbol( self.codename, symbols ) + def display(self, level=0): + name = self.parser.symbol_repr( self.codename ) if self.value is None: return "<%s>" % name else: return "<%s>=='%s'" % (name, self.value) - + def match_token(self, other): """convenience '==' implementation, this is *not* a *real* equality test @@ -715,12 +691,12 @@ """ if not isinstance(other, Token): raise RuntimeError("Unexpected token type") - if other is EmptyToken: + if other is self.parser.EmptyToken: return False res = other.codename == self.codename and self.value in [None, other.value] #print "matching", self, other, res return res - + def __eq__(self, other): return self.codename == other.codename and self.value == other.value @@ -740,8 +716,5 @@ return False -EmptyToken = Token(NULLTOKEN, None) - - Modified: pypy/branch/ast-experiments/pypy/interpreter/pyparser/pysymbol.py ============================================================================== --- pypy/branch/ast-experiments/pypy/interpreter/pyparser/pysymbol.py (original) +++ pypy/branch/ast-experiments/pypy/interpreter/pyparser/pysymbol.py Sat Jan 28 02:10:44 2006 @@ -10,6 +10,7 @@ # important here class SymbolMapper(object): + """XXX dead""" def __init__(self, sym_name=None ): _anoncount = self._anoncount = -10 _count = self._count = 0 Modified: pypy/branch/ast-experiments/pypy/interpreter/pyparser/pythonlexer.py ============================================================================== --- pypy/branch/ast-experiments/pypy/interpreter/pyparser/pythonlexer.py (original) +++ pypy/branch/ast-experiments/pypy/interpreter/pyparser/pythonlexer.py Sat Jan 28 02:10:44 2006 @@ -7,8 +7,8 @@ from pypy.interpreter.pyparser.grammar import TokenSource, Token, AbstractContext from pypy.interpreter.pyparser.error import SyntaxError + import pytoken -from pytoken import NEWLINE # Don't import string for that ... NAMECHARS = 'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ_' @@ -369,8 +369,8 @@ return (self._current_line, self._lineno) # return 'line %s : %s' % ('XXX', self._current_line) -NONE_LIST = [pytoken.ENDMARKER, pytoken.INDENT, pytoken.DEDENT] -NAMED_LIST = [pytoken.OP] +#NONE_LIST = [pytoken.ENDMARKER, pytoken.INDENT, pytoken.DEDENT] +#NAMED_LIST = [pytoken.OP] Source = PythonSource Modified: pypy/branch/ast-experiments/pypy/interpreter/pyparser/pythonparse.py ============================================================================== --- pypy/branch/ast-experiments/pypy/interpreter/pyparser/pythonparse.py (original) +++ pypy/branch/ast-experiments/pypy/interpreter/pyparser/pythonparse.py Sat Jan 28 02:10:44 2006 @@ -5,26 +5,31 @@ helper functions are provided that use the grammar to parse using file_input, single_input and eval_input targets """ +import autopath +import sys +import os from pypy.interpreter.error import OperationError, debug_print from pypy.interpreter import gateway from pypy.interpreter.pyparser.error import SyntaxError from pypy.tool.option import Options -from pythonlexer import Source, match_encoding_declaration -import pysymbol -import ebnfparse -import sys -import os -import grammar +from pypy.interpreter.pyparser.pythonlexer import Source, match_encoding_declaration +import pypy.interpreter.pyparser.pysymbol as pysymbol +import pypy.interpreter.pyparser.pytoken as pytoken +import pypy.interpreter.pyparser.ebnfparse as ebnfparse +import pypy.interpreter.pyparser.grammar as grammar +from pypy.interpreter.pyparser.parser import Parser +try: + from pypy.interpreter.pyparser import symbol +except ImportError: + # for standalone testing + import symbol from codeop import PyCF_DONT_IMPLY_DEDENT -class PythonParser(object): +class PythonParser(Parser): """Wrapper class for python grammar""" - def __init__(self, rules, items ): - self.items = items - self.rules = rules - # Build first sets for each rule (including anonymous ones) - grammar.build_first_sets(self.items) + def __init__(self): + Parser.__init__(self) def parse_source(self, textsrc, goal, builder, flags=0): """Parse a python source according to goal""" @@ -46,9 +51,9 @@ def parse_lines(self, lines, goal, builder, flags=0): goalnumber = pysymbol._cpython_symbols.sym_values[goal] - target = self.rules[goalnumber] + target = self.root_rules[goalnumber] src = Source(lines, flags) - + result = target.match(src, builder) if not result: line, lineno = src.debug() @@ -56,7 +61,7 @@ raise SyntaxError("invalid syntax", lineno, -1, line) # return None return builder - + _recode_to_utf8 = gateway.applevel(r''' def _recode_to_utf8(text, encoding): return unicode(text, encoding).encode("utf-8") @@ -96,7 +101,7 @@ if eol2 < 0: return _check_line_for_encoding(s[eol + 1:]) return _check_line_for_encoding(s[eol + 1:eol2]) - + def _check_line_for_encoding(line): """returns the declared encoding or None""" i = 0 @@ -119,28 +124,15 @@ # unfortunately the command line options are not parsed yet PYTHON_GRAMMAR, PYPY_VERSION = get_grammar_file( Options.version ) -def python_grammar(fname): - """returns a PythonParser build from the specified grammar file""" - level = grammar.DEBUG - grammar.DEBUG = 0 - gram = ebnfparse.parse_grammar( file(fname) ) - grammar.DEBUG = level - parser = PythonParser( gram.rules, gram.items ) - return parser -def python_grammar_dyn(fname): +def load_python_grammar(fname): """Loads the grammar using the 'dynamic' rpython parser""" _grammar_file = file(fname) - ebnfbuilder = ebnfparse.parse_grammar_text( file(fname).read() ) - ebnfbuilder.resolve_rules() - parser = PythonParser( ebnfbuilder.root_rules, ebnfbuilder.all_rules ) + parser = PYTHON_PARSER + # populate symbols + ebnfparse.parse_grammar_text( parser, file(fname).read() ) return parser - -debug_print( "Loading grammar %s" % PYTHON_GRAMMAR ) -PYTHON_PARSER = python_grammar( PYTHON_GRAMMAR ) -PYTHON_PARSER_DYN = python_grammar_dyn( PYTHON_GRAMMAR ) - def reload_grammar(version): """helper function to test with pypy different grammars""" global PYTHON_GRAMMAR, PYTHON_PARSER, PYPY_VERSION @@ -151,7 +143,7 @@ def parse_file_input(pyf, gram, builder ): """Parse a python file""" return gram.parse_source( pyf.read(), "file_input", builder ) - + def parse_single_input(textsrc, gram, builder ): """Parse a python single statement""" return gram.parse_source( textsrc, "single_input", builder ) @@ -160,11 +152,82 @@ """Parse a python expression""" return gram.parse_source( textsrc, "eval_input", builder ) - def grammar_rules( space ): - return space.wrap( PYTHON_PARSER.rules ) + return space.wrap( PYTHON_PARSER.root_rules ) + +def dot_node( gen, rule_name, rule, symbols, edges, count ): + from pypy.interpreter.pyparser.grammar import KleeneStar, Sequence, Alternative, Token + subrule_name = symbols.get( rule.codename, rule.codename ) + label = None + if not subrule_name.startswith(":"+rule_name): + node_name = rule_name + "_ext_" + str(count[0]) + count[0]+=1 + label = subrule_name + gen.emit_node( node_name, shape="parallelogram", label=subrule_name ) + edges.append( (node_name, subrule_name) ) + return node_name + subrule_name = subrule_name.replace(":","_") + if isinstance(rule, KleeneStar): + node = dot_node( gen, rule_name, rule.args[0], symbols, edges, count ) + gen.emit_edge( node, node, label=rule.get_star(), style='solid' ) + return node + elif isinstance(rule, Sequence): + gen.enter_subgraph( subrule_name ) + first_node = None + for n in rule.args: + node_name = dot_node( gen, rule_name, n, symbols, edges, count ) + if first_node: + gen.emit_edge( first_node, node_name, style='solid' ) + first_node = node_name + gen.leave_subgraph() + return subrule_name + elif isinstance(rule, Alternative): + gen.enter_subgraph( subrule_name ) + for n in rule.args: + node_name = dot_node( gen, rule_name, n, symbols, edges, count ) + gen.leave_subgraph() + return subrule_name + elif isinstance(rule, Token): + node_name = rule_name + "_ext_" + str(count[0]) + count[0]+=1 + gen.emit_node( node_name, shape='box', label=rule.display( 0, symbols ) ) + return node_name + raise RuntimeError("Unknown node type") + +def gen_grammar_dot( name, root_rules, rules, symbols ): + """Quick hack to output a dot graph of the grammar""" + from pypy.translator.tool.make_dot import DotGen + gen = DotGen(name) + edges = [] + count = [0] + for r in root_rules: + rule_name = symbols.get( r.codename, r.codename ) + gen.emit_node( rule_name, shape='hexagon', label=r.display(0,symbols) ) + for rule in r.args: + node = dot_node( gen, rule_name, rule, symbols, edges, count ) + gen.emit_edge( rule_name, node, style='solid' ) + for left, right in edges: + gen.emit_edge( left, right, style='solid' ) + gen.generate(target='ps') + + +def parse_grammar(space, w_src): + """Loads the grammar using the 'dynamic' rpython parser""" + src = space.str_w( w_src ) + ebnfbuilder = ebnfparse.parse_grammar_text( src ) + ebnfbuilder.resolve_rules() + grammar.build_first_sets(ebnfbuilder.all_rules) + return space.wrap( ebnfbuilder.root_rules ) + +debug_print( "Loading grammar %s" % PYTHON_GRAMMAR ) +PYTHON_PARSER = PythonParser() +PYTHON_PARSER.load_symbols( symbol.sym_name ) +pytoken.setup_tokens( PYTHON_PARSER ) +load_python_grammar( PYTHON_GRAMMAR ) -def make_rule( space, w_rule ): - rule = space.str_w( w_rule ) - +if __name__=="__main__": + symbols = {} + symbols.update( pytoken.tok_name ) + symbols.update( pysymbol._cpython_symbols.sym_name ) + gen_grammar_dot("grammar", PYTHON_PARSER.rules.values(), PYTHON_PARSER.items, symbols ) Modified: pypy/branch/ast-experiments/pypy/interpreter/pyparser/pytoken.py ============================================================================== --- pypy/branch/ast-experiments/pypy/interpreter/pyparser/pytoken.py (original) +++ pypy/branch/ast-experiments/pypy/interpreter/pyparser/pytoken.py Sat Jan 28 02:10:44 2006 @@ -11,125 +11,71 @@ tok_name = {-1 : 'NULLTOKEN'} tok_values = {'NULLTOKEN' : -1} -def add_token(name): - global N_TOKENS - value = N_TOKENS - N_TOKENS += 1 - _g = globals() - _g[name] = value - tok_name[value] = name - tok_values[name] = value - +tok_rpunct = {} +def setup_tokens( parser ): + global tok_rpunct # For compatibility, this produces the same constant values as Python 2.4. -add_token( 'ENDMARKER' ) -add_token( 'NAME' ) -add_token( 'NUMBER' ) -add_token( 'STRING' ) -add_token( 'NEWLINE' ) -add_token( 'INDENT' ) -add_token( 'DEDENT' ) -add_token( 'LPAR' ) -add_token( 'RPAR' ) -add_token( 'LSQB' ) -add_token( 'RSQB' ) -add_token( 'COLON' ) -add_token( 'COMMA' ) -add_token( 'SEMI' ) -add_token( 'PLUS' ) -add_token( 'MINUS' ) -add_token( 'STAR' ) -add_token( 'SLASH' ) -add_token( 'VBAR' ) -add_token( 'AMPER' ) -add_token( 'LESS' ) -add_token( 'GREATER' ) -add_token( 'EQUAL' ) -add_token( 'DOT' ) -add_token( 'PERCENT' ) -add_token( 'BACKQUOTE' ) -add_token( 'LBRACE' ) -add_token( 'RBRACE' ) -add_token( 'EQEQUAL' ) -add_token( 'NOTEQUAL' ) -add_token( 'LESSEQUAL' ) -add_token( 'GREATEREQUAL' ) -add_token( 'TILDE' ) -add_token( 'CIRCUMFLEX' ) -add_token( 'LEFTSHIFT' ) -add_token( 'RIGHTSHIFT' ) -add_token( 'DOUBLESTAR' ) -add_token( 'PLUSEQUAL' ) -add_token( 'MINEQUAL' ) -add_token( 'STAREQUAL' ) -add_token( 'SLASHEQUAL' ) -add_token( 'PERCENTEQUAL' ) -add_token( 'AMPEREQUAL' ) -add_token( 'VBAREQUAL' ) -add_token( 'CIRCUMFLEXEQUAL' ) -add_token( 'LEFTSHIFTEQUAL' ) -add_token( 'RIGHTSHIFTEQUAL' ) -add_token( 'DOUBLESTAREQUAL' ) -add_token( 'DOUBLESLASH' ) -add_token( 'DOUBLESLASHEQUAL' ) -add_token( 'AT' ) -add_token( 'OP' ) -add_token( 'ERRORTOKEN' ) + parser.add_token( 'ENDMARKER' ) + parser.add_token( 'NAME' ) + parser.add_token( 'NUMBER' ) + parser.add_token( 'STRING' ) + parser.add_token( 'NEWLINE' ) + parser.add_token( 'INDENT' ) + parser.add_token( 'DEDENT' ) + parser.add_token( 'LPAR', "(" ) + parser.add_token( 'RPAR', ")" ) + parser.add_token( 'LSQB', "[" ) + parser.add_token( 'RSQB', "]" ) + parser.add_token( 'COLON', ":" ) + parser.add_token( 'COMMA', "," ) + parser.add_token( 'SEMI', ";" ) + parser.add_token( 'PLUS', "+" ) + parser.add_token( 'MINUS', "-" ) + parser.add_token( 'STAR', "*" ) + parser.add_token( 'SLASH', "/" ) + parser.add_token( 'VBAR', "|" ) + parser.add_token( 'AMPER', "&" ) + parser.add_token( 'LESS', "<" ) + parser.add_token( 'GREATER', ">" ) + parser.add_token( 'EQUAL', "=" ) + parser.add_token( 'DOT', "." ) + parser.add_token( 'PERCENT', "%" ) + parser.add_token( 'BACKQUOTE', "`" ) + parser.add_token( 'LBRACE', "{" ) + parser.add_token( 'RBRACE', "}" ) + parser.add_token( 'EQEQUAL', "==" ) + ne = parser.add_token( 'NOTEQUAL', "!=" ) + parser.tok_values["<>"] = ne + parser.add_token( 'LESSEQUAL', "<=" ) + parser.add_token( 'GREATEREQUAL', ">=" ) + parser.add_token( 'TILDE', "~" ) + parser.add_token( 'CIRCUMFLEX', "^" ) + parser.add_token( 'LEFTSHIFT', "<<" ) + parser.add_token( 'RIGHTSHIFT', ">>" ) + parser.add_token( 'DOUBLESTAR', "**" ) + parser.add_token( 'PLUSEQUAL', "+=" ) + parser.add_token( 'MINEQUAL', "-=" ) + parser.add_token( 'STAREQUAL', "*=" ) + parser.add_token( 'SLASHEQUAL', "/=" ) + parser.add_token( 'PERCENTEQUAL', "%=" ) + parser.add_token( 'AMPEREQUAL', "&=" ) + parser.add_token( 'VBAREQUAL', "|=" ) + parser.add_token( 'CIRCUMFLEXEQUAL', "^=" ) + parser.add_token( 'LEFTSHIFTEQUAL', "<<=" ) + parser.add_token( 'RIGHTSHIFTEQUAL', ">>=" ) + parser.add_token( 'DOUBLESTAREQUAL', "**=" ) + parser.add_token( 'DOUBLESLASH', "//" ) + parser.add_token( 'DOUBLESLASHEQUAL',"//=" ) + parser.add_token( 'AT', "@" ) + parser.add_token( 'OP' ) + parser.add_token( 'ERRORTOKEN' ) # extra PyPy-specific tokens -add_token( "COMMENT" ) -add_token( "NL" ) - -# a reverse mapping from internal tokens def to more pythonic tokens -tok_punct = { - "&" : AMPER, - "&=" : AMPEREQUAL, - "`" : BACKQUOTE, - "^" : CIRCUMFLEX, - "^=" : CIRCUMFLEXEQUAL, - ":" : COLON, - "," : COMMA, - "." : DOT, - "//" : DOUBLESLASH, - "//=" : DOUBLESLASHEQUAL, - "**" : DOUBLESTAR, - "**=" : DOUBLESTAREQUAL, - "==" : EQEQUAL, - "=" : EQUAL, - ">" : GREATER, - ">=" : GREATEREQUAL, - "{" : LBRACE, - "}" : RBRACE, - "<<" : LEFTSHIFT, - "<<=" : LEFTSHIFTEQUAL, - "<" : LESS, - "<=" : LESSEQUAL, - "(" : LPAR, - "[" : LSQB, - "-=" : MINEQUAL, - "-" : MINUS, - "!=" : NOTEQUAL, - "<>" : NOTEQUAL, - "%" : PERCENT, - "%=" : PERCENTEQUAL, - "+" : PLUS, - "+=" : PLUSEQUAL, - ")" : RBRACE, - ">>" : RIGHTSHIFT, - ">>=" : RIGHTSHIFTEQUAL, - ")" : RPAR, - "]" : RSQB, - ";" : SEMI, - "/" : SLASH, - "/=" : SLASHEQUAL, - "*" : STAR, - "*=" : STAREQUAL, - "~" : TILDE, - "|" : VBAR, - "|=" : VBAREQUAL, - "@": AT, - } -tok_rpunct = {} -for string, value in tok_punct.items(): - tok_rpunct[value] = string + parser.add_token( "COMMENT" ) + parser.add_token( "NL" ) + tok_rpunct = parser.tok_values.copy() + for _name, _value in parser.tokens.items(): + globals()[_name] = _value + setattr(parser, _name, _value) Modified: pypy/branch/ast-experiments/pypy/interpreter/pyparser/tuplebuilder.py ============================================================================== --- pypy/branch/ast-experiments/pypy/interpreter/pyparser/tuplebuilder.py (original) +++ pypy/branch/ast-experiments/pypy/interpreter/pyparser/tuplebuilder.py Sat Jan 28 02:10:44 2006 @@ -54,7 +54,7 @@ class TupleBuilderContext(AbstractContext): def __init__(self, stackpos ): self.stackpos = stackpos - + class TupleBuilder(AbstractBuilder): """A builder that directly produce the AST""" @@ -81,7 +81,7 @@ nodes = expand_nodes( [self.stack[-1]] ) self.stack[-1] = NonTerminal( rule.codename, nodes ) return True - + def sequence(self, rule, source, elts_number): """ """ num = rule.codename Modified: pypy/branch/ast-experiments/pypy/module/recparser/__init__.py ============================================================================== --- pypy/branch/ast-experiments/pypy/module/recparser/__init__.py (original) +++ pypy/branch/ast-experiments/pypy/module/recparser/__init__.py Sat Jan 28 02:10:44 2006 @@ -48,6 +48,7 @@ 'decode_string_literal': 'pyparser.decode_string_literal', 'install_compiler_hook' : 'pypy.interpreter.pycompiler.install_compiler_hook', 'rules' : 'pypy.interpreter.pyparser.pythonparse.grammar_rules', + 'parse_grammar' : 'pypy.interpreter.pyparser.pythonparse.parse_grammar', } # Automatically exports each AST class Modified: pypy/branch/ast-experiments/pypy/translator/tool/make_dot.py ============================================================================== --- pypy/branch/ast-experiments/pypy/translator/tool/make_dot.py (original) +++ pypy/branch/ast-experiments/pypy/translator/tool/make_dot.py Sat Jan 28 02:10:44 2006 @@ -49,9 +49,9 @@ def leave_subgraph(self): self.emit("}") - def emit_edge(self, name1, name2, label="", - style="dashed", - color="black", + def emit_edge(self, name1, name2, label="", + style="dashed", + color="black", dir="forward", weight="5", ): From cfbolz at codespeak.net Sat Jan 28 10:14:04 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Sat, 28 Jan 2006 10:14:04 +0100 (CET) Subject: [pypy-svn] r22762 - in pypy/dist/pypy/translator/c: . test Message-ID: <20060128091404.DBA1827DD4@code1.codespeak.net> Author: cfbolz Date: Sat Jan 28 10:14:02 2006 New Revision: 22762 Modified: pypy/dist/pypy/translator/c/newfuncgen.py pypy/dist/pypy/translator/c/test/test_newgc.py Log: added all the gctransform tests to the tests for the new function generator. moved the operation generation into an method on the function generator because we will need it somewhere else Modified: pypy/dist/pypy/translator/c/newfuncgen.py ============================================================================== --- pypy/dist/pypy/translator/c/newfuncgen.py (original) +++ pypy/dist/pypy/translator/c/newfuncgen.py Sat Jan 28 10:14:02 2006 @@ -204,20 +204,7 @@ reachable_err = -1 # the number of the first reachable err label for op in block.operations: err = 'err%d_%d' % (myblocknum, len(to_release)) - macro = 'OP_%s' % op.opname.upper() - if op.opname.startswith('gc_'): - meth = getattr(self.gcpolicy, macro, None) - if meth: - line = meth(self, op, err) - else: - meth = getattr(self, macro, None) - if meth: - line = meth(op, err) - if meth is None: - lst = [self.expr(v) for v in op.args] - lst.append(self.expr(op.result)) - lst.append(err) - line = '%s(%s);' % (macro, ', '.join(lst)) + line = self.gen_op(op, err) if '\n' in line: for subline in line.split('\n'): yield subline @@ -359,6 +346,23 @@ else: yield self.pop_alive(to_release[i-1]) + def gen_op(self, op, err): + macro = 'OP_%s' % op.opname.upper() + if op.opname.startswith('gc_'): + meth = getattr(self.gcpolicy, macro, None) + if meth: + line = meth(self, op, err) + else: + meth = getattr(self, macro, None) + if meth: + line = meth(op, err) + if meth is None: + lst = [self.expr(v) for v in op.args] + lst.append(self.expr(op.result)) + lst.append(err) + line = '%s(%s);' % (macro, ', '.join(lst)) + return line + # ____________________________________________________________ # the C preprocessor cannot handle operations taking a variable number Modified: pypy/dist/pypy/translator/c/test/test_newgc.py ============================================================================== --- pypy/dist/pypy/translator/c/test/test_newgc.py (original) +++ pypy/dist/pypy/translator/c/test/test_newgc.py Sat Jan 28 10:14:02 2006 @@ -38,3 +38,75 @@ return s.x fn = compile_func(f, [int]) assert fn(1) == 1 + +def test_call_function(): + class C: + pass + def f(): + c = C() + c.x = 1 + return c + def g(): + return f().x + fn = compile_func(g, []) + assert fn() == 1 + +def test_multiple_exits(): + S = lltype.GcStruct("S", ('x', lltype.Signed)) + T = lltype.GcStruct("T", ('y', lltype.Signed)) + def f(n): + c = lltype.malloc(S) + d = lltype.malloc(T) + d.y = 1 + e = lltype.malloc(T) + e.y = 2 + if n: + x = d + else: + x = e + return x.y + fn = compile_func(f, [int]) + assert fn(1) == 1 + assert fn(0) == 2 + + +def test_cleanup_vars_on_call(): + S = lltype.GcStruct("S", ('x', lltype.Signed)) + def f(): + return lltype.malloc(S) + def g(): + s1 = f() + s1.x = 42 + s2 = f() + s3 = f() + return s1.x + fn = compile_func(g, []) + assert fn() == 42 + +def test_multiply_passed_var(): + S = lltype.GcStruct("S", ('x', lltype.Signed)) + def f(x): + if x: + a = lltype.malloc(S) + a.x = 1 + b = a + else: + a = lltype.malloc(S) + a.x = 1 + b = lltype.malloc(S) + b.x = 2 + return a.x + b.x + fn = compile_func(f, [int]) + fn(1) == 2 + fn(0) == 3 + +def test_pyobj(): + def f(x): + if x: + a = 1 + else: + a = "1" + return int(a) + fn = compile_func(f, [int]) + assert fn(1) == 1 +# assert fn(0) == 0 #XXX this should work but it's not my fault From tismer at codespeak.net Sat Jan 28 10:48:33 2006 From: tismer at codespeak.net (tismer at codespeak.net) Date: Sat, 28 Jan 2006 10:48:33 +0100 (CET) Subject: [pypy-svn] r22763 - pypy/dist/pypy/translator/c Message-ID: <20060128094833.5F95527DD4@code1.codespeak.net> Author: tismer Date: Sat Jan 28 10:48:31 2006 New Revision: 22763 Modified: pypy/dist/pypy/translator/c/gc.py Log: fixed debug building with Boehm on Windows - assertion on non-existing flag Modified: pypy/dist/pypy/translator/c/gc.py ============================================================================== --- pypy/dist/pypy/translator/c/gc.py (original) +++ pypy/dist/pypy/translator/c/gc.py Sat Jan 28 10:48:31 2006 @@ -419,7 +419,7 @@ def gc_startup_code(self): if sys.platform == 'win32': - yield 'assert(GC_all_interior_pointers == 0);' + pass # yield 'assert(GC_all_interior_pointers == 0);' else: yield 'GC_all_interior_pointers = 0;' yield 'GC_init();' From tismer at codespeak.net Sat Jan 28 10:50:38 2006 From: tismer at codespeak.net (tismer at codespeak.net) Date: Sat, 28 Jan 2006 10:50:38 +0100 (CET) Subject: [pypy-svn] r22764 - pypy/dist/pypy/module/stackless Message-ID: <20060128095038.14E9427DD4@code1.codespeak.net> Author: tismer Date: Sat Jan 28 10:50:36 2006 New Revision: 22764 Modified: pypy/dist/pypy/module/stackless/interp_coroutine.py Log: progressing on co-existance of multiple concurrency models Modified: pypy/dist/pypy/module/stackless/interp_coroutine.py ============================================================================== --- pypy/dist/pypy/module/stackless/interp_coroutine.py (original) +++ pypy/dist/pypy/module/stackless/interp_coroutine.py Sat Jan 28 10:50:36 2006 @@ -205,10 +205,13 @@ appcostate.current = self def w_kill(self): + if appcostate.current is self: + costate.current = self self.kill() def __del__(self): - if self.frame is not None or self.space.lookup(self, '__del__') is not None: + if postpone_deletion is not None: + # we might be very late (happens with interpreted pypy) postpone_deletion(self) def _userdel(self): @@ -217,6 +220,13 @@ self.set_is_zombie(True) self.space.userdel(self) + def get_current(space): + return space.wrap(appcostate.current) + get_current = staticmethod(get_current) + + def get_main(space): + return space.wrap(appcostate.main) + get_main = staticmethod(get_main) # _mixin_ did not work for methname in StacklessFlags.__dict__: @@ -229,24 +239,24 @@ return space.wrap(self.get_is_zombie()) AppCoroutine.w_get_is_zombie = w_get_is_zombie -def w_coro_get_current(space): - return space.wrap(appcostate.current) +def makeStaticMethod(module, classname, funcname): + space = module.space + space.appexec(map(space.wrap, (module, classname, funcname)), """ + (module, klassname, funcname): + klass = getattr(module, klassname) + func = getattr(klass, funcname) + setattr(klass, funcname, staticmethod(func.im_func)) + """) -def w_coro_get_main(space): - return space.wrap(appcostate.main) +def post_install(module): + appcostate.main.space = module.space + makeStaticMethod(module, 'Coroutine', 'get_current') + makeStaticMethod(module, 'Coroutine', 'get_main') -def installStaticMethod(space, w_klass, func, name=None): - if name is None: - name = func.__name__ - smeth = StaticMethod(space.wrap(interp2app(func, name))) - space.setattr(w_klass, space.wrap(name), space.wrap(smeth)) +# space.appexec("""() : + +# maybe use __spacebind__ for postprocessing -def post_install(module): - space = module.space - w_klass = space.getattr(space.wrap(module), space.wrap('Coroutine')) - installStaticMethod(space, w_klass, w_coro_get_current, 'get_current') - installStaticMethod(space, w_klass, w_coro_get_main, 'get_main') - AppCoroutine.typedef = TypeDef("Coroutine", __new__ = interp2app(AppCoroutine.descr_method__new__.im_func), bind = interp2app(AppCoroutine.w_bind, @@ -254,6 +264,8 @@ switch = interp2app(AppCoroutine.w_switch), kill = interp2app(AppCoroutine.w_kill), is_zombie = GetSetProperty(AppCoroutine.w_get_is_zombie, doc=AppCoroutine.get_is_zombie.__doc__), + get_current = interp2app(AppCoroutine.get_current), + get_main = interp2app(AppCoroutine.get_main), ) class AppCoState(object): From pedronis at codespeak.net Sat Jan 28 11:35:28 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Sat, 28 Jan 2006 11:35:28 +0100 (CET) Subject: [pypy-svn] r22765 - pypy/extradoc/sprintinfo/mallorca Message-ID: <20060128103528.14BC327DD4@code1.codespeak.net> Author: pedronis Date: Sat Jan 28 11:35:27 2006 New Revision: 22765 Modified: pypy/extradoc/sprintinfo/mallorca/planning.txt Log: planning. Modified: pypy/extradoc/sprintinfo/mallorca/planning.txt ============================================================================== --- pypy/extradoc/sprintinfo/mallorca/planning.txt (original) +++ pypy/extradoc/sprintinfo/mallorca/planning.txt Sat Jan 28 11:35:27 2006 @@ -28,35 +28,27 @@ Tasks ====== - - JIT: Arre, Samuele + - JIT: Graph generating ops Designed interface (see draft-jit-ideas), Refactored llabstractinterp to use the operations - Next discussion: arre, samuele, armin +... + Started trying to reuse the annotator framework + for off-line partial evaluation: arre, armin, samuele - stackless application level interface: Christian, started from the demo coroutine impl, trying to expose at application level, both greenlet and stackless-like interfaces. - (issues: testing needs translation, the primitives are not - there on top of CPython, maybe use stackless or greenlet - to provide them or make the bytecode itself on top of CPython - stackless). Next step: fastest possible translation for testing. - DONE: Exposed coroutines at application-level, support for flags on - the coroutines to use for other concurrency primitives to - construct on top of them. - - - express exception raising operations in the graphs as - calls to helpers (backends could use their special - operations) - Eric, Richard - - ( - l3interp/offsetof/sizeof work: - Progress. Progress needs either hacks and assuming Boehm - or GC integration. Try to progress assuming Boehm ) + DONE: Exposed coroutines at application-level, support for flags on + the coroutines to use for other concurrency primitives to + construct on top of them. + IN-PROGRESS: extending interface such that coroutines, tasklet, + greenlets can co-exist peacefully. Now implementing greenlets... - integration of the garbage collection toolkit with the DONE: offsetof, sizeof work done Started: insert GC neeeded behaviour as graph transformations. + Writing a new function generator in genc for the transformed + graphs... Added cleanup attribute to operations. Carl, Michael - improving the way we interface with external functions. @@ -65,9 +57,20 @@ DONE: glued atoi from ctypes (cheating a bit) DONE: basic annotation for ctypes structures and basics types. Working on annotating ctypes arrays + Annotating derefencing. Trying to track ownerships... Next would be rtyping... Gerald, Stephan + - express exception raising operations in the graphs as + calls to helpers (backends could use their special + operations) SOME PROGRESS + Eric, Richard + + ( - l3interp/offsetof/sizeof work: + Progress. Progress needs either hacks and assuming Boehm + or GC integration. Try to progress assuming Boehm ) + + later: ====== From pedronis at codespeak.net Sat Jan 28 11:49:27 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Sat, 28 Jan 2006 11:49:27 +0100 (CET) Subject: [pypy-svn] r22766 - in pypy/dist/pypy/jit: . test Message-ID: <20060128104927.3B15A27DD4@code1.codespeak.net> Author: pedronis Date: Sat Jan 28 11:49:25 2006 New Revision: 22766 Modified: pypy/dist/pypy/jit/hintmodel.py pypy/dist/pypy/jit/test/test_hint_annotation.py Log: (arre, armin, pedronis) union for SomeLLAbstractConstants with simple test Modified: pypy/dist/pypy/jit/hintmodel.py ============================================================================== --- pypy/dist/pypy/jit/hintmodel.py (original) +++ pypy/dist/pypy/jit/hintmodel.py Sat Jan 28 11:49:25 2006 @@ -3,9 +3,9 @@ from pypy.jit.hintbookkeeper import getbookkeeper from pypy.rpython.lltypesystem import lltype -UNARY_OPERATIONS = "".split() +UNARY_OPERATIONS = "same_as".split() -BINARY_OPERATIONS = "int_add".split() +BINARY_OPERATIONS = "int_add int_sub".split() class OriginTreeNode(object): @@ -31,6 +31,10 @@ SomeLLAbstractValue.__init__(self, T) self.origins = origins +class __extend__(SomeLLAbstractValue): + + def same_as(hs_v1): + return hs_v1 class __extend__(pairtype(SomeLLAbstractConstant, SomeLLAbstractConstant)): @@ -40,3 +44,9 @@ origin.merge(hs_c2.origins) return SomeLLAbstractConstant(lltype.Signed, {origin: True}) + int_sub = int_add + + def union((hs_c1, hs_c2)): + assert hs_c1.concretetype == hs_c2.concretetype + origins = annmodel.setunion(hs_c1.origins, hs_c2.origins) + return SomeLLAbstractConstant(hs_c1.concretetype, origins) Modified: pypy/dist/pypy/jit/test/test_hint_annotation.py ============================================================================== --- pypy/dist/pypy/jit/test/test_hint_annotation.py (original) +++ pypy/dist/pypy/jit/test/test_hint_annotation.py Sat Jan 28 11:49:25 2006 @@ -25,3 +25,15 @@ assert isinstance(hs, SomeLLAbstractConstant) assert len(hs.origins) == 1 assert hs.concretetype == lltype.Signed + +def test_join(): + def ll_function(cond, x,y): + if cond: + z = x+y + else: + z = x-y + return z + hs = hannotate(ll_function, [bool, int, int]) + assert isinstance(hs, SomeLLAbstractConstant) + assert len(hs.origins) == 2 + assert hs.concretetype == lltype.Signed From cfbolz at codespeak.net Sat Jan 28 12:05:21 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Sat, 28 Jan 2006 12:05:21 +0100 (CET) Subject: [pypy-svn] r22767 - pypy/dist/pypy/translator/c Message-ID: <20060128110521.5F14F27DD4@code1.codespeak.net> Author: cfbolz Date: Sat Jan 28 12:05:19 2006 New Revision: 22767 Modified: pypy/dist/pypy/translator/c/newfuncgen.py Log: (mwh, cfbolz): make gen_link a method on the function generator Modified: pypy/dist/pypy/translator/c/newfuncgen.py ============================================================================== --- pypy/dist/pypy/translator/c/newfuncgen.py (original) +++ pypy/dist/pypy/translator/c/newfuncgen.py Sat Jan 28 12:05:19 2006 @@ -165,26 +165,6 @@ if line: yield line - def gen_link(link, linklocalvars=None): - "Generate the code to jump across the given Link." - is_alive = {} - linklocalvars = linklocalvars or {} - for v in to_release: - linklocalvars[v] = self.expr(v) - assignments = [] - for a1, a2 in zip(link.args, link.target.inputargs): - a2type, a2typename = self.lltypes[id(a2)] - if a2type is Void: - continue - if a1 in linklocalvars: - src = linklocalvars[a1] - else: - src = self.expr(a1) - dest = LOCALVAR % a2.name - assignments.append((a2typename, dest, src)) - for line in gen_assignments(assignments): - yield line - yield 'goto block%d;' % blocknum[link.target] # collect all blocks for block in graph.iterblocks(): @@ -231,7 +211,7 @@ elif block.exitswitch is None: # single-exit block assert len(block.exits) == 1 - for op in gen_link(block.exits[0]): + for op in self.gen_link(block.exits[0], to_release, blocknum): yield op yield '' elif block.exitswitch == c_last_exception: @@ -239,7 +219,7 @@ # we handle the non-exceptional case first link = block.exits[0] assert link.exitcase is None - for op in gen_link(link): + for op in self.gen_link(link, to_release, blocknum): yield op # we must catch the exception raised by the last operation, # which goes to the last err%d_%d label written above. @@ -277,7 +257,7 @@ d[link.last_exc_value] = 'exc_value' else: yield '\t' + self.pop_alive_expr('exc_value', T2) - for op in gen_link(link, d): + for op in self.gen_link(link, to_release, blocknum, d): yield '\t' + op yield '}' fallthrough = True @@ -299,14 +279,14 @@ else: expr = '%s == Py_False' % expr yield 'if (%s) {' % expr - for op in gen_link(link): + for op in self.gen_link(link, to_release, blocknum): yield '\t' + op yield '}' link = block.exits[-1] assert link.exitcase in (False, True) #yield 'assert(%s == %s);' % (self.expr(block.exitswitch), # self.genc.nameofvalue(link.exitcase, ct)) - for op in gen_link(block.exits[-1]): + for op in self.gen_link(block.exits[-1], to_release, blocknum): yield op yield '' elif TYPE in (Signed, Unsigned, SignedLongLong, @@ -319,7 +299,7 @@ defaultlink = link continue yield 'case %s:' % self.db.get(link.llexitcase) - for op in gen_link(link): + for op in self.gen_link(link, to_release, blocknum): yield '\t' + op yield 'break;' @@ -327,7 +307,7 @@ if defaultlink is None: raise TypeError('switches must have a default case.') yield 'default:' - for op in gen_link(defaultlink): + for op in self.gen_link(defaultlink, to_release, blocknum): yield '\t' + op yield '}' @@ -346,6 +326,27 @@ else: yield self.pop_alive(to_release[i-1]) + def gen_link(self, link, to_release, blocknum, linklocalvars=None): + "Generate the code to jump across the given Link." + is_alive = {} + linklocalvars = linklocalvars or {} + for v in to_release: + linklocalvars[v] = self.expr(v) + assignments = [] + for a1, a2 in zip(link.args, link.target.inputargs): + a2type, a2typename = self.lltypes[id(a2)] + if a2type is Void: + continue + if a1 in linklocalvars: + src = linklocalvars[a1] + else: + src = self.expr(a1) + dest = LOCALVAR % a2.name + assignments.append((a2typename, dest, src)) + for line in gen_assignments(assignments): + yield line + yield 'goto block%d;' % blocknum[link.target] + def gen_op(self, op, err): macro = 'OP_%s' % op.opname.upper() if op.opname.startswith('gc_'): From pedronis at codespeak.net Sat Jan 28 12:24:55 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Sat, 28 Jan 2006 12:24:55 +0100 (CET) Subject: [pypy-svn] r22768 - in pypy/dist/pypy: annotation jit jit/test Message-ID: <20060128112455.809CC27DD4@code1.codespeak.net> Author: pedronis Date: Sat Jan 28 12:24:51 2006 New Revision: 22768 Modified: pypy/dist/pypy/annotation/annrpython.py pypy/dist/pypy/annotation/bookkeeper.py pypy/dist/pypy/jit/hintbookkeeper.py pypy/dist/pypy/jit/hintmodel.py pypy/dist/pypy/jit/test/test_hint_annotation.py Log: (arre, arigo, pedronis) implemented hint support (constant -> concrete). Modified: pypy/dist/pypy/annotation/annrpython.py ============================================================================== --- pypy/dist/pypy/annotation/annrpython.py (original) +++ pypy/dist/pypy/annotation/annrpython.py Sat Jan 28 12:24:51 2006 @@ -197,7 +197,7 @@ elif isinstance(arg, Constant): #if arg.value is undefined_value: # undefined local variables # return annmodel.SomeImpossibleValue() - return self.bookkeeper.immutablevalue(arg.value) + return self.bookkeeper.immutableconstant(arg) else: raise TypeError, 'Variable or Constant expected, got %r' % (arg,) Modified: pypy/dist/pypy/annotation/bookkeeper.py ============================================================================== --- pypy/dist/pypy/annotation/bookkeeper.py (original) +++ pypy/dist/pypy/annotation/bookkeeper.py Sat Jan 28 12:24:51 2006 @@ -292,6 +292,9 @@ dictdef.generalize_value(s_value) return SomeDict(dictdef) + def immutableconstant(self, const): + return self.immutablevalue(const.value) + def immutablevalue(self, x): """The most precise SomeValue instance that contains the immutable value x.""" Modified: pypy/dist/pypy/jit/hintbookkeeper.py ============================================================================== --- pypy/dist/pypy/jit/hintbookkeeper.py (original) +++ pypy/dist/pypy/jit/hintbookkeeper.py Sat Jan 28 12:24:51 2006 @@ -31,6 +31,12 @@ def compute_at_fixpoint(self): pass + def immutableconstant(self, const): + from pypy.jit import hintmodel + res = hintmodel.SomeLLAbstractConstant(const.concretetype, {}) + res.const = const.value + return res + # get current bookkeeper def getbookkeeper(): Modified: pypy/dist/pypy/jit/hintmodel.py ============================================================================== --- pypy/dist/pypy/jit/hintmodel.py (original) +++ pypy/dist/pypy/jit/hintmodel.py Sat Jan 28 12:24:51 2006 @@ -3,12 +3,14 @@ from pypy.jit.hintbookkeeper import getbookkeeper from pypy.rpython.lltypesystem import lltype -UNARY_OPERATIONS = "same_as".split() +UNARY_OPERATIONS = "same_as hint".split() BINARY_OPERATIONS = "int_add int_sub".split() class OriginTreeNode(object): + fixed = False + def __init__(self, origins=None): if origins is None: origins = {} @@ -17,6 +19,16 @@ def merge(self, nodes): self.origins.update(nodes) + def visit(self, seen=None): + if seen is None: + seen = {} + yield self + for o in self.origins: + if o not in seen: + seen[o] = True + for o1 in o.visit(seen): + yield o1 + class SomeLLAbstractValue(annmodel.SomeObject): def __init__(self, T): @@ -31,11 +43,26 @@ SomeLLAbstractValue.__init__(self, T) self.origins = origins +class SomeLLConcreteValue(SomeLLAbstractValue): + pass + +# ____________________________________________________________ +# operations + class __extend__(SomeLLAbstractValue): def same_as(hs_v1): return hs_v1 +class __extend__(SomeLLAbstractConstant): + + def hint(hs_c1, hs_flags): + assert hs_flags.const['concrete'] + for o in hs_c1.origins: + for o1 in o.visit(): + o1.fixed = True + return SomeLLConcreteValue(hs_c1.concretetype) + class __extend__(pairtype(SomeLLAbstractConstant, SomeLLAbstractConstant)): def int_add((hs_c1, hs_c2)): Modified: pypy/dist/pypy/jit/test/test_hint_annotation.py ============================================================================== --- pypy/dist/pypy/jit/test/test_hint_annotation.py (original) +++ pypy/dist/pypy/jit/test/test_hint_annotation.py Sat Jan 28 12:24:51 2006 @@ -1,7 +1,8 @@ from pypy.translator.translator import TranslationContext, graphof from pypy.jit.hintannotator import HintAnnotator -from pypy.jit.hintmodel import SomeLLAbstractConstant, OriginTreeNode +from pypy.jit.hintmodel import SomeLLAbstractConstant, SomeLLConcreteValue, OriginTreeNode from pypy.rpython.lltypesystem import lltype +from pypy.rpython.objectmodel import hint def hannotate(func, argtypes): # build the normal ll graphs for ll_function @@ -37,3 +38,36 @@ assert isinstance(hs, SomeLLAbstractConstant) assert len(hs.origins) == 2 assert hs.concretetype == lltype.Signed + + +def test_simple_hint_result(): + def ll_function(cond, x,y): + if cond: + z = x+y + else: + z = x-y + z = hint(z, concrete=True) + return z + hs = hannotate(ll_function, [bool, int, int]) + assert isinstance(hs, SomeLLConcreteValue) + assert hs.concretetype == lltype.Signed + +def test_simple_hint_origins(): + def ll_function(cond, x,y): + if cond: + z = x+y + else: + z = x-y + z1 = hint(z, concrete=True) + return z # origin of z1 + hs = hannotate(ll_function, [bool, int, int]) + assert isinstance(hs, SomeLLAbstractConstant) + assert len(hs.origins) == 2 + for o in hs.origins: + assert o.fixed + assert len(o.origins) == 2 + for o in o.origins: + assert o.fixed + assert not o.origins + assert hs.concretetype == lltype.Signed + From pedronis at codespeak.net Sat Jan 28 12:39:09 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Sat, 28 Jan 2006 12:39:09 +0100 (CET) Subject: [pypy-svn] r22769 - in pypy/dist/pypy/jit: . test Message-ID: <20060128113909.A989127DCB@code1.codespeak.net> Author: pedronis Date: Sat Jan 28 12:39:07 2006 New Revision: 22769 Modified: pypy/dist/pypy/jit/hintmodel.py pypy/dist/pypy/jit/test/test_hint_annotation.py Log: (arre, arigo, pedronis) started support for variables (as SomeLLAbstractValue). Modified: pypy/dist/pypy/jit/hintmodel.py ============================================================================== --- pypy/dist/pypy/jit/hintmodel.py (original) +++ pypy/dist/pypy/jit/hintmodel.py Sat Jan 28 12:39:07 2006 @@ -34,9 +34,6 @@ def __init__(self, T): self.concretetype = T -class SomeLLAbstractVariable(SomeLLAbstractValue): - pass - class SomeLLAbstractConstant(SomeLLAbstractValue): def __init__(self, T, origins): @@ -57,12 +54,21 @@ class __extend__(SomeLLAbstractConstant): def hint(hs_c1, hs_flags): + if hs_flags.const.get('variable', False): + return SomeLLAbstractValue(hs_c1.concretetype) assert hs_flags.const['concrete'] for o in hs_c1.origins: for o1 in o.visit(): o1.fixed = True return SomeLLConcreteValue(hs_c1.concretetype) + +class __extend__(pairtype(SomeLLAbstractValue, SomeLLAbstractValue)): + + def int_add((hs_v1, hs_v2)): + return SomeLLAbstractValue(lltype.Signed) + + class __extend__(pairtype(SomeLLAbstractConstant, SomeLLAbstractConstant)): def int_add((hs_c1, hs_c2)): Modified: pypy/dist/pypy/jit/test/test_hint_annotation.py ============================================================================== --- pypy/dist/pypy/jit/test/test_hint_annotation.py (original) +++ pypy/dist/pypy/jit/test/test_hint_annotation.py Sat Jan 28 12:39:07 2006 @@ -1,6 +1,6 @@ from pypy.translator.translator import TranslationContext, graphof from pypy.jit.hintannotator import HintAnnotator -from pypy.jit.hintmodel import SomeLLAbstractConstant, SomeLLConcreteValue, OriginTreeNode +from pypy.jit.hintmodel import * from pypy.rpython.lltypesystem import lltype from pypy.rpython.objectmodel import hint @@ -71,3 +71,12 @@ assert not o.origins assert hs.concretetype == lltype.Signed +def test_simple_variable(): + def ll_function(x,y): + x = hint(x, variable=True) # special hint only for testing purposes!!! + return x + y + hs = hannotate(ll_function, [int, int]) + assert type(hs) is SomeLLAbstractValue + assert hs.concretetype == lltype.Signed + + From mwh at codespeak.net Sat Jan 28 12:39:56 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Sat, 28 Jan 2006 12:39:56 +0100 (CET) Subject: [pypy-svn] r22770 - in pypy/dist/pypy/rpython/memory: . test Message-ID: <20060128113956.BC59127DCB@code1.codespeak.net> Author: mwh Date: Sat Jan 28 12:39:54 2006 New Revision: 22770 Modified: pypy/dist/pypy/rpython/memory/gctransform.py pypy/dist/pypy/rpython/memory/test/test_gctransform.py Log: (cfbolz, mwh) Fix logic around handling gc pointer arguments to functions. Modified: pypy/dist/pypy/rpython/memory/gctransform.py ============================================================================== --- pypy/dist/pypy/rpython/memory/gctransform.py (original) +++ pypy/dist/pypy/rpython/memory/gctransform.py Sat Jan 28 12:39:54 2006 @@ -37,6 +37,13 @@ def transform_graph(self, graph): self.links_to_split = {} # link -> vars to pop_alive across the link + + newops = [] + for var in graph.startblock.inputargs: + if var_needsgc(var): + newops.extend(self.push_alive(var)) + graph.startblock.operations[0:0] = newops + for block in graph.iterblocks(): self.transform_block(block) for link, livecounts in self.links_to_split.iteritems(): Modified: pypy/dist/pypy/rpython/memory/test/test_gctransform.py ============================================================================== --- pypy/dist/pypy/rpython/memory/test/test_gctransform.py (original) +++ pypy/dist/pypy/rpython/memory/test/test_gctransform.py Sat Jan 28 12:39:54 2006 @@ -5,11 +5,14 @@ from pypy.objspace.flow.model import Variable def checkblock(block): - if not block.operations == (): + if block.operations == (): # a return/exception block -- don't want to think about them # (even though the test passes for somewhat accidental reasons) return - vars_in = len([v for v in block.inputargs if var_needsgc(v)]) + if block.isstartblock: + refs_in = 0 + else: + refs_in = len([v for v in block.inputargs if var_needsgc(v)]) push_alives = len([op for op in block.operations if op.opname.startswith('gc_push_alive')]) pop_alives = len([op for op in block.operations @@ -20,9 +23,9 @@ # it's a block we inserted return for link in block.exits: - vars_out = len([v for v in link.args + refs_out = len([v for v in link.args if isinstance(v, Variable) and var_needsgc(v)]) - assert vars_in + push_alives + calls == pop_alives + vars_out + assert refs_in + push_alives + calls == pop_alives + refs_out def rtype_and_transform(func, inputtypes, transformcls): @@ -34,6 +37,7 @@ t.checkgraphs() for graph in t.graphs: for block in graph.iterblocks(): + print graph, block, block.isstartblock checkblock(block) return t @@ -159,3 +163,15 @@ if op.opname.startswith("gc_")] for op in gcops: assert op.opname.endswith("_pyobj") + +def test_pass_gc_pointer(): + S = lltype.GcStruct("S", ('x', lltype.Signed)) + def f(s): + s.x = 1 + def g(): + s = lltype.malloc(S) + f(s) + return s.x + t = rtype_and_transform(g, [], gctransform.GCTransformer) + ggraph = graphof(t, g) + From pedronis at codespeak.net Sat Jan 28 12:39:59 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Sat, 28 Jan 2006 12:39:59 +0100 (CET) Subject: [pypy-svn] r22771 - pypy/dist/pypy/jit Message-ID: <20060128113959.20FE727DD4@code1.codespeak.net> Author: pedronis Date: Sat Jan 28 12:39:56 2006 New Revision: 22771 Modified: pypy/dist/pypy/jit/hintmodel.py Log: (arre, arigo, pedronis) missing comment. Modified: pypy/dist/pypy/jit/hintmodel.py ============================================================================== --- pypy/dist/pypy/jit/hintmodel.py (original) +++ pypy/dist/pypy/jit/hintmodel.py Sat Jan 28 12:39:56 2006 @@ -54,7 +54,7 @@ class __extend__(SomeLLAbstractConstant): def hint(hs_c1, hs_flags): - if hs_flags.const.get('variable', False): + if hs_flags.const.get('variable', False): # only for testing purposes!!! return SomeLLAbstractValue(hs_c1.concretetype) assert hs_flags.const['concrete'] for o in hs_c1.origins: From cfbolz at codespeak.net Sat Jan 28 12:53:02 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Sat, 28 Jan 2006 12:53:02 +0100 (CET) Subject: [pypy-svn] r22772 - pypy/branch/genc-gc-refactoring Message-ID: <20060128115302.22E1F27DD4@code1.codespeak.net> Author: cfbolz Date: Sat Jan 28 12:53:00 2006 New Revision: 22772 Added: pypy/branch/genc-gc-refactoring/ - copied from r22771, pypy/dist/pypy/translator/c/ Log: (mwh, cfbolz) branch of translator/c for the gc refactoring of genc From pedronis at codespeak.net Sat Jan 28 12:53:46 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Sat, 28 Jan 2006 12:53:46 +0100 (CET) Subject: [pypy-svn] r22773 - in pypy/dist/pypy/annotation: . test Message-ID: <20060128115346.97CC327DD4@code1.codespeak.net> Author: pedronis Date: Sat Jan 28 12:53:44 2006 New Revision: 22773 Modified: pypy/dist/pypy/annotation/pairtype.py pypy/dist/pypy/annotation/test/test_pairtype.py Log: (arre, arigo, pedronis) support extending many classes in one __extend__ pseudo-class definition. Modified: pypy/dist/pypy/annotation/pairtype.py ============================================================================== --- pypy/dist/pypy/annotation/pairtype.py (original) +++ pypy/dist/pypy/annotation/pairtype.py Sat Jan 28 12:53:44 2006 @@ -7,12 +7,12 @@ the definition of 't' instead of creating a new subclass.""" def __new__(cls, name, bases, dict): if name == '__extend__': - cls = bases[0] # override data into the existing base - for key, value in dict.items(): - if key == '__module__': - continue + for cls in bases: + for key, value in dict.items(): + if key == '__module__': + continue # XXX do we need to provide something more for pickling? - setattr(cls, key, value) + setattr(cls, key, value) return None else: return super(extendabletype, cls).__new__(cls, name, bases, dict) Modified: pypy/dist/pypy/annotation/test/test_pairtype.py ============================================================================== --- pypy/dist/pypy/annotation/test/test_pairtype.py (original) +++ pypy/dist/pypy/annotation/test/test_pairtype.py Sat Jan 28 12:53:44 2006 @@ -1,5 +1,5 @@ -from pypy.annotation.pairtype import pairtype, pair +from pypy.annotation.pairtype import pairtype, pair, extendabletype def test_binop(): ### Binary operation example @@ -91,3 +91,20 @@ g = Lisp_Generator() pair(g, Block(Switch())).emit(['v1', 'v2']) assert g.progn == ["(do 'something)"] + +def test_multiple_extend(): + class A: + __metaclass__ = extendabletype + class B: + __metaclass__ = extendabletype + + class __extend__(A,B): + + def f(self): + pass + + assert hasattr(A, 'f') + assert hasattr(B, 'f') + + + From pedronis at codespeak.net Sat Jan 28 12:56:17 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Sat, 28 Jan 2006 12:56:17 +0100 (CET) Subject: [pypy-svn] r22774 - in pypy/dist/pypy/jit: . test Message-ID: <20060128115617.AD64727DD4@code1.codespeak.net> Author: pedronis Date: Sat Jan 28 12:56:15 2006 New Revision: 22774 Modified: pypy/dist/pypy/jit/hintmodel.py pypy/dist/pypy/jit/test/test_hint_annotation.py Log: (arre, arigo, pedronis) simple support for propagation of concrete (mixing with constants) Modified: pypy/dist/pypy/jit/hintmodel.py ============================================================================== --- pypy/dist/pypy/jit/hintmodel.py (original) +++ pypy/dist/pypy/jit/hintmodel.py Sat Jan 28 12:56:15 2006 @@ -83,3 +83,10 @@ assert hs_c1.concretetype == hs_c2.concretetype origins = annmodel.setunion(hs_c1.origins, hs_c2.origins) return SomeLLAbstractConstant(hs_c1.concretetype, origins) + +class __extend__(pairtype(SomeLLAbstractConstant, SomeLLConcreteValue), + pairtype(SomeLLConcreteValue, SomeLLAbstractConstant), + pairtype(SomeLLConcreteValue, SomeLLConcreteValue)): + + def int_add((hs_c1, hs_c2)): + return SomeLLConcreteValue(lltype.Signed) Modified: pypy/dist/pypy/jit/test/test_hint_annotation.py ============================================================================== --- pypy/dist/pypy/jit/test/test_hint_annotation.py (original) +++ pypy/dist/pypy/jit/test/test_hint_annotation.py Sat Jan 28 12:56:15 2006 @@ -76,7 +76,14 @@ x = hint(x, variable=True) # special hint only for testing purposes!!! return x + y hs = hannotate(ll_function, [int, int]) - assert type(hs) is SomeLLAbstractValue + assert type(hs) is SomeLLAbstractValue assert hs.concretetype == lltype.Signed - +def test_simple_concrete_propagation(): + def ll_function(x,y): + x = hint(x, concrete=True) + return x + y + hs = hannotate(ll_function, [int, int]) + assert type(hs) is SomeLLConcreteValue + assert hs.concretetype == lltype.Signed + From cfbolz at codespeak.net Sat Jan 28 12:59:41 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Sat, 28 Jan 2006 12:59:41 +0100 (CET) Subject: [pypy-svn] r22775 - pypy/branch/genc-gc-refactoring Message-ID: <20060128115941.A942227DD4@code1.codespeak.net> Author: cfbolz Date: Sat Jan 28 12:59:40 2006 New Revision: 22775 Removed: pypy/branch/genc-gc-refactoring/funcgen.py pypy/branch/genc-gc-refactoring/gc.py Log: (cfbolz, mwh) remove the old versions of gc.py and funcgen.py From cfbolz at codespeak.net Sat Jan 28 13:03:53 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Sat, 28 Jan 2006 13:03:53 +0100 (CET) Subject: [pypy-svn] r22776 - in pypy/branch/genc-gc-refactoring: . test Message-ID: <20060128120353.0A5B427DD3@code1.codespeak.net> Author: cfbolz Date: Sat Jan 28 13:03:48 2006 New Revision: 22776 Added: pypy/branch/genc-gc-refactoring/funcgen.py - copied unchanged from r22772, pypy/branch/genc-gc-refactoring/newfuncgen.py pypy/branch/genc-gc-refactoring/gc.py - copied unchanged from r22772, pypy/branch/genc-gc-refactoring/newgc.py Removed: pypy/branch/genc-gc-refactoring/newfuncgen.py pypy/branch/genc-gc-refactoring/newgc.py Modified: pypy/branch/genc-gc-refactoring/genc.py pypy/branch/genc-gc-refactoring/node.py pypy/branch/genc-gc-refactoring/test/test_newgc.py Log: (cfbolz, mwh) remove the new versions of funcgen and gc on the branch... Modified: pypy/branch/genc-gc-refactoring/genc.py ============================================================================== --- pypy/branch/genc-gc-refactoring/genc.py (original) +++ pypy/branch/genc-gc-refactoring/genc.py Sat Jan 28 13:03:48 2006 @@ -19,14 +19,12 @@ _compiled = False symboltable = None stackless = False - use_new_funcgen = False - def __init__(self, translator, entrypoint, gcpolicy=None, libraries=None, thread_enabled=False, use_new_funcgen=False): + def __init__(self, translator, entrypoint, gcpolicy=None, libraries=None, thread_enabled=False): self.translator = translator self.entrypoint = entrypoint self.gcpolicy = gcpolicy self.thread_enabled = thread_enabled - self.use_new_funcgen = use_new_funcgen if libraries is None: libraries = [] @@ -40,7 +38,6 @@ if self.stackless: from pypy.translator.c.stackless import StacklessData db.stacklessdata = StacklessData(db) - db.use_new_funcgen = self.use_new_funcgen # we need a concrete gcpolicy to do this self.libraries += db.gcpolicy.gc_libraries() Modified: pypy/branch/genc-gc-refactoring/node.py ============================================================================== --- pypy/branch/genc-gc-refactoring/node.py (original) +++ pypy/branch/genc-gc-refactoring/node.py Sat Jan 28 13:03:48 2006 @@ -5,7 +5,6 @@ parentlink, Ptr, PyObject, Void, OpaqueType, Float, \ RuntimeTypeInfo, getRuntimeTypeInfo, Char from pypy.translator.c.funcgen import FunctionCodeGenerator -from pypy.translator.c.newfuncgen import FunctionCodeGenerator as NewFunctionCodeGenerator from pypy.translator.c.external import CExternalFunctionCodeGenerator from pypy.translator.c.support import USESLOTS # set to False if necessary while refactoring from pypy.translator.c.support import cdecl, somelettersfrom, c_string_constant @@ -549,11 +548,8 @@ elif hasattr(fnobj, 'graph'): cpython_exc = getattr(fnobj, 'exception_policy', None) == "CPython" if hasattr(db, 'stacklessdata'): - assert not db.use_new_funcgen, "can't use stackless with the new funcgen yet" from pypy.translator.c.stackless import SlpFunctionCodeGenerator gencls = SlpFunctionCodeGenerator - elif db.use_new_funcgen: - gencls = NewFunctionCodeGenerator else: gencls = FunctionCodeGenerator return gencls(fnobj.graph, db, cpython_exc, functionname) Modified: pypy/branch/genc-gc-refactoring/test/test_newgc.py ============================================================================== --- pypy/branch/genc-gc-refactoring/test/test_newgc.py (original) +++ pypy/branch/genc-gc-refactoring/test/test_newgc.py Sat Jan 28 13:03:48 2006 @@ -5,7 +5,7 @@ from pypy.translator.tool.cbuild import skip_missing_compiler from pypy.translator.translator import TranslationContext -from pypy.translator.c import genc, newgc +from pypy.translator.c import genc, gc from pypy.rpython.lltypesystem import lltype from pypy.rpython.memory.gctransform import GCTransformer @@ -16,8 +16,7 @@ t.buildrtyper().specialize() GCTransformer(t.graphs).transform() - builder = genc.CExtModuleBuilder(t, fn, use_new_funcgen=True, - gcpolicy=newgc.RefcountingGcPolicy) + builder = genc.CExtModuleBuilder(t, fn, gcpolicy=gc.RefcountingGcPolicy) builder.generate_source() skip_missing_compiler(builder.compile) builder.import_module() From mwh at codespeak.net Sat Jan 28 13:07:29 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Sat, 28 Jan 2006 13:07:29 +0100 (CET) Subject: [pypy-svn] r22777 - in pypy/dist/pypy/translator/c: . test Message-ID: <20060128120729.6D3B527DD3@code1.codespeak.net> Author: mwh Date: Sat Jan 28 13:07:26 2006 New Revision: 22777 Removed: pypy/dist/pypy/translator/c/newfuncgen.py pypy/dist/pypy/translator/c/newgc.py pypy/dist/pypy/translator/c/test/test_newgc.py Modified: pypy/dist/pypy/translator/c/genc.py pypy/dist/pypy/translator/c/node.py Log: (cfbolz,mwh) ... and on the branch. We'll leave the trunk in peace. For now :) Modified: pypy/dist/pypy/translator/c/genc.py ============================================================================== --- pypy/dist/pypy/translator/c/genc.py (original) +++ pypy/dist/pypy/translator/c/genc.py Sat Jan 28 13:07:26 2006 @@ -19,14 +19,12 @@ _compiled = False symboltable = None stackless = False - use_new_funcgen = False - def __init__(self, translator, entrypoint, gcpolicy=None, libraries=None, thread_enabled=False, use_new_funcgen=False): + def __init__(self, translator, entrypoint, gcpolicy=None, libraries=None, thread_enabled=False): self.translator = translator self.entrypoint = entrypoint self.gcpolicy = gcpolicy self.thread_enabled = thread_enabled - self.use_new_funcgen = use_new_funcgen if libraries is None: libraries = [] @@ -40,7 +38,6 @@ if self.stackless: from pypy.translator.c.stackless import StacklessData db.stacklessdata = StacklessData(db) - db.use_new_funcgen = self.use_new_funcgen # we need a concrete gcpolicy to do this self.libraries += db.gcpolicy.gc_libraries() Modified: pypy/dist/pypy/translator/c/node.py ============================================================================== --- pypy/dist/pypy/translator/c/node.py (original) +++ pypy/dist/pypy/translator/c/node.py Sat Jan 28 13:07:26 2006 @@ -5,7 +5,6 @@ parentlink, Ptr, PyObject, Void, OpaqueType, Float, \ RuntimeTypeInfo, getRuntimeTypeInfo, Char from pypy.translator.c.funcgen import FunctionCodeGenerator -from pypy.translator.c.newfuncgen import FunctionCodeGenerator as NewFunctionCodeGenerator from pypy.translator.c.external import CExternalFunctionCodeGenerator from pypy.translator.c.support import USESLOTS # set to False if necessary while refactoring from pypy.translator.c.support import cdecl, somelettersfrom, c_string_constant @@ -549,11 +548,8 @@ elif hasattr(fnobj, 'graph'): cpython_exc = getattr(fnobj, 'exception_policy', None) == "CPython" if hasattr(db, 'stacklessdata'): - assert not db.use_new_funcgen, "can't use stackless with the new funcgen yet" from pypy.translator.c.stackless import SlpFunctionCodeGenerator gencls = SlpFunctionCodeGenerator - elif db.use_new_funcgen: - gencls = NewFunctionCodeGenerator else: gencls = FunctionCodeGenerator return gencls(fnobj.graph, db, cpython_exc, functionname) From gromit at codespeak.net Sat Jan 28 13:21:36 2006 From: gromit at codespeak.net (gromit at codespeak.net) Date: Sat, 28 Jan 2006 13:21:36 +0100 (CET) Subject: [pypy-svn] r22778 - in pypy/dist/pypy: annotation rpython/rctypes rpython/rctypes/test Message-ID: <20060128122136.CB97827DD3@code1.codespeak.net> Author: gromit Date: Sat Jan 28 13:21:32 2006 New Revision: 22778 Modified: pypy/dist/pypy/annotation/binaryop.py pypy/dist/pypy/annotation/model.py pypy/dist/pypy/annotation/unaryop.py pypy/dist/pypy/rpython/rctypes/implementation.py pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py Log: ADD: Memory ownership tracking for all cytpes objects. Modified: pypy/dist/pypy/annotation/binaryop.py ============================================================================== --- pypy/dist/pypy/annotation/binaryop.py (original) +++ pypy/dist/pypy/annotation/binaryop.py Sat Jan 28 13:21:32 2006 @@ -755,9 +755,13 @@ elif idx < 0: raise IndexError( "invalid index" ) try: + # This is the case for unboxed values, which + # are those having memorystate NOMEMORY return s_cto.knowntype._type_.annotator_type except AttributeError: - return SomeCTypesObject(s_cto.knowntype._type_) + return SomeCTypesObject( + s_cto.knowntype._type_, + memorystate=SomeCTypesObject.MEMORYALIAS) class __extend__(pairtype(SomeCTypesObject, SomeSlice)): def setitem((s_cto, s_slice), s_iterable): Modified: pypy/dist/pypy/annotation/model.py ============================================================================== --- pypy/dist/pypy/annotation/model.py (original) +++ pypy/dist/pypy/annotation/model.py Sat Jan 28 13:21:32 2006 @@ -429,8 +429,16 @@ class SomeCTypesObject(SomeObject): """Stands for an object of the ctypes module.""" - def __init__(self, knowntype): + NOMEMORY = "NOMEMORY" + OWNSMEMORY = "OWNSMEMORY" + MEMORYALIAS = "MEMORYALIAS" + MIXEDMEMORYOWNERSHIP = "MIXEDMEMORYOWNERSHIP" + + def __init__(self, knowntype, memorystate=None): + if memorystate is None: + memorystate = knowntype.default_memorystate self.knowntype = knowntype + self.memorystate = memorystate def can_be_none(self): return False Modified: pypy/dist/pypy/annotation/unaryop.py ============================================================================== --- pypy/dist/pypy/annotation/unaryop.py (original) +++ pypy/dist/pypy/annotation/unaryop.py Sat Jan 28 13:21:32 2006 @@ -628,7 +628,17 @@ def getattr(cto, s_attr): if s_attr.is_constant() and isinstance(s_attr.const, str): attr = s_attr.const - atype = cto.knowntype._fields_def_[attr] + try: + atype = cto.knowntype._fields_def_[attr] + except AttributeError: + # We are dereferencing a pointer by accessing its contents attribute + if s_attr.const == "contents": + return SomeCTypesObject( + cto.knowntype._type_, SomeCTypesObject.MEMORYALIAS) + else: + raise AttributeError( + "%r object has no attribute %r" % ( + cto.knowntype, s_attr.const)) try: return atype.annotator_type except AttributeError: Modified: pypy/dist/pypy/rpython/rctypes/implementation.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/implementation.py (original) +++ pypy/dist/pypy/rpython/rctypes/implementation.py Sat Jan 28 13:21:32 2006 @@ -50,12 +50,14 @@ the_type.ll_type = ll_type if wrap_arg is not None: the_type.wrap_arg = wrap_arg + the_type.default_memorystate = SomeCTypesObject.OWNSMEMORY else: # !!!! attention !!!! # the basic c_types need some annotation information # at the moment that are exactly the types that have # no 'wrap_arg'. This might change in the future the_type.compute_result_annotation = classmethod(lambda cls, s_arg:SomeCTypesObject(cls)) + the_type.default_memorystate = SomeCTypesObject.NOMEMORY create_ctypes_annotations() @@ -110,6 +112,8 @@ __metaclass__ = RStructureMeta + default_memorystate = SomeCTypesObject.OWNSMEMORY + def compute_annotation(cls): return SomeCTypesObject(cls) compute_annotation = classmethod(compute_annotation) @@ -121,11 +125,13 @@ """ Answer the result annotation of calling 'cls'. """ - return SomeCTypesObject(cls) + return SomeCTypesObject(cls,SomeCTypesObject.OWNSMEMORY) compute_result_annotation = classmethod(compute_result_annotation) class RByrefObj(object): + default_memorystate = SomeCTypesObject.MEMORYALIAS + def __init__(self): self.__name__ = 'RByrefObj' @@ -152,6 +158,8 @@ assert answer is cls return SomeCTypesObject(cls) answer.compute_result_annotation = classmethod(compute_result_annotation) + #o#answer._fields_def_ = {"contents": cls} + answer.default_memorystate = SomeCTypesObject.MEMORYALIAS return answer @@ -189,7 +197,7 @@ Answer the result annotation of calling 'cls'. """ assert answer is cls - return SomeCTypesObject(cls) + return SomeCTypesObject(cls, SomeCTypesObject.OWNSMEMORY) answer.compute_result_annotation = classmethod(compute_result_annotation) return answer Modified: pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py Sat Jan 28 13:21:32 2006 @@ -3,6 +3,7 @@ from pypy.translator.translator import TranslationContext from pypy.translator.c.test.test_genc import compile from pypy.translator.tool.cbuild import compile_c_module +from pypy.annotation.model import SomeCTypesObject, SomeObject import sys try: @@ -83,6 +84,18 @@ res = testfunc_byval(inpoint,oppoint) return res, oppoint +def py_testfunc_POINTER_dereference(inpoint): + point = tagpoint() + oppoint = oppoint_type(point) + res = testfunc_byval(inpoint,oppoint) + return res, oppoint.contents, oppoint[0] + +def py_test_mixed_memory_state( randomboolean ): + if randomboolean: + return tagpoint() + else: + return oppoint_type(tagpoint()).contents + def py_test_simple_cint(): return c_int(10) @@ -222,32 +235,62 @@ s = a.build_types(py_testfunc_struct, [int]) assert s.knowntype == int - def test_annotate_struct(self): - a = RPythonAnnotator() + t = TranslationContext() + a = t.buildannotator() s = a.build_types(py_testfunc_struct_id, [tagpoint]) assert s.knowntype == tagpoint + assert s.memorystate == SomeCTypesObject.OWNSMEMORY def test_create_point(self): - a = RPythonAnnotator() + t = TranslationContext() + a = t.buildannotator() s = a.build_types(py_create_point,[]) assert s.knowntype == int def test_annotate_byval(self): - a = RPythonAnnotator() + t = TranslationContext() + a = t.buildannotator() s = a.build_types(py_testfunc_byval,[tagpoint]) assert s.knowntype == tuple assert len(s.items) == 2 assert s.items[0].knowntype == int assert s.items[1].knowntype == tagpoint + assert s.items[1].memorystate == SomeCTypesObject.OWNSMEMORY def test_annotate_POINTER(self): - a = RPythonAnnotator() + t = TranslationContext() + a = t.buildannotator() s = a.build_types(py_testfunc_POINTER,[tagpoint]) assert s.knowntype == tuple assert len(s.items) == 2 assert s.items[0].knowntype == int assert s.items[1].knowntype == POINTER(tagpoint) + assert s.items[1].memorystate == SomeCTypesObject.MEMORYALIAS + #d#t.view() + + def test_annotate_POINTER_dereference(self): + t = TranslationContext() + a = t.buildannotator() + s = a.build_types(py_testfunc_POINTER_dereference, [tagpoint]) + assert s.knowntype == tuple + assert len(s.items) == 3 + assert s.items[0].knowntype == int + assert s.items[1].knowntype == tagpoint + assert s.items[1].memorystate == SomeCTypesObject.MEMORYALIAS + assert s.items[2].knowntype == tagpoint + assert s.items[2].memorystate == SomeCTypesObject.MEMORYALIAS + #d#t.view() + + def test_annotate_mixed_memorystate(self): + t = TranslationContext() + a = t.buildannotator() + s = a.build_types(py_test_mixed_memory_state, [int]) + #d#t.view() + assert s.knowntype == tagpoint + # This memory state will be supported in the future (#f#) + #f#assert s.memorystate == SomeCTypesObject.MIXEDMEMORYOWNERSHIP + assert isinstance(s, SomeObject) def test_annotate_simple_cint(self): a = RPythonAnnotator() From pedronis at codespeak.net Sat Jan 28 13:32:33 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Sat, 28 Jan 2006 13:32:33 +0100 (CET) Subject: [pypy-svn] r22779 - in pypy/dist/pypy/jit: . test Message-ID: <20060128123233.85D1A27DD3@code1.codespeak.net> Author: pedronis Date: Sat Jan 28 13:32:31 2006 New Revision: 22779 Modified: pypy/dist/pypy/jit/hintmodel.py pypy/dist/pypy/jit/test/test_hint_annotation.py Log: (arre, arigo, pedronis) use a distinguished hint annotation SomeLLAbstractVariable for unknown variable values. Test and support for unions of currently defined hint annotations. Modified: pypy/dist/pypy/jit/hintmodel.py ============================================================================== --- pypy/dist/pypy/jit/hintmodel.py (original) +++ pypy/dist/pypy/jit/hintmodel.py Sat Jan 28 13:32:31 2006 @@ -43,6 +43,9 @@ class SomeLLConcreteValue(SomeLLAbstractValue): pass +class SomeLLAbstractVariable(SomeLLAbstractValue): + pass + # ____________________________________________________________ # operations @@ -66,8 +69,17 @@ class __extend__(pairtype(SomeLLAbstractValue, SomeLLAbstractValue)): def int_add((hs_v1, hs_v2)): - return SomeLLAbstractValue(lltype.Signed) + return SomeLLAbstractVariable(lltype.Signed) + + def union((hs_v1, hs_v2)): + raise annmodel.UnionError("%s %s don't mix" % (hs_v1, hs_v2)) + +class __extend__(pairtype(SomeLLAbstractVariable, SomeLLAbstractConstant), + pairtype(SomeLLAbstractConstant, SomeLLAbstractVariable)): + def union((hs_v1, hs_v2)): + assert hs_v1.concretetype == hs_v2.concretetype + return SomeLLAbstractVariable(hs_v1.concretetype) class __extend__(pairtype(SomeLLAbstractConstant, SomeLLAbstractConstant)): Modified: pypy/dist/pypy/jit/test/test_hint_annotation.py ============================================================================== --- pypy/dist/pypy/jit/test/test_hint_annotation.py (original) +++ pypy/dist/pypy/jit/test/test_hint_annotation.py Sat Jan 28 13:32:31 2006 @@ -1,8 +1,10 @@ +import py from pypy.translator.translator import TranslationContext, graphof from pypy.jit.hintannotator import HintAnnotator from pypy.jit.hintmodel import * from pypy.rpython.lltypesystem import lltype from pypy.rpython.objectmodel import hint +from pypy.annotation import model as annmodel def hannotate(func, argtypes): # build the normal ll graphs for ll_function @@ -76,7 +78,7 @@ x = hint(x, variable=True) # special hint only for testing purposes!!! return x + y hs = hannotate(ll_function, [int, int]) - assert type(hs) is SomeLLAbstractValue + assert type(hs) is SomeLLAbstractVariable assert hs.concretetype == lltype.Signed def test_simple_concrete_propagation(): @@ -86,4 +88,31 @@ hs = hannotate(ll_function, [int, int]) assert type(hs) is SomeLLConcreteValue assert hs.concretetype == lltype.Signed - + +def test_union(): + unionof = annmodel.unionof + av1, av2 = SomeLLAbstractVariable(lltype.Signed), SomeLLAbstractVariable(lltype.Signed) + cv1, cv2 = SomeLLConcreteValue(lltype.Signed), SomeLLConcreteValue(lltype.Signed) + ac1, ac2 = SomeLLAbstractConstant(lltype.Signed, {}), SomeLLAbstractConstant(lltype.Signed, {}) + ac3 = SomeLLAbstractConstant(lltype.Signed, {}) + ac3.const = 3 + ac4 = SomeLLAbstractConstant(lltype.Signed, {}) + ac4.const = 4 + assert unionof(av1, av2) == av1 + assert unionof(cv1, cv2) == cv2 + assert unionof(ac1, ac2) == ac1 + assert unionof(ac3, ac3) == ac3 + assert unionof(ac3, ac2) == ac1 + assert unionof(ac4, ac3) == ac1 + # degenerating cases + py.test.raises(annmodel.UnionError, "unionof(cv1, av1)") + py.test.raises(annmodel.UnionError, "unionof(av1, cv1)") + py.test.raises(annmodel.UnionError, "unionof(ac1, cv1)") + py.test.raises(annmodel.UnionError, "unionof(cv1, ac1)") + # constant with values + assert unionof(av1, ac1) == av1 + assert unionof(ac1, av1) == av1 + assert unionof(ac3, av1) == av1 + assert unionof(av2, ac4) == av1 + + From cfbolz at codespeak.net Sat Jan 28 13:36:48 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Sat, 28 Jan 2006 13:36:48 +0100 (CET) Subject: [pypy-svn] r22780 - in pypy/dist/pypy/rpython/memory: . test Message-ID: <20060128123648.D151A27DD6@code1.codespeak.net> Author: cfbolz Date: Sat Jan 28 13:36:46 2006 New Revision: 22780 Modified: pypy/dist/pypy/rpython/memory/gctransform.py pypy/dist/pypy/rpython/memory/test/test_gctransform.py Log: (mwh, cfbolz) don't actually store the graphs on the transformer Modified: pypy/dist/pypy/rpython/memory/gctransform.py ============================================================================== --- pypy/dist/pypy/rpython/memory/gctransform.py (original) +++ pypy/dist/pypy/rpython/memory/gctransform.py Sat Jan 28 13:36:46 2006 @@ -28,14 +28,17 @@ return isinstance(vartype, lltype.Ptr) and vartype._needsgc() class GCTransformer: - def __init__(self, graphs): - self.graphs = graphs + def __init__(self): + self.seen_graphs = {} - def transform(self): - for graph in self.graphs: + def transform(self, graphs): + for graph in graphs: self.transform_graph(graph) def transform_graph(self, graph): + if graph in self.seen_graphs: + return + self.seen_graphs[graph] = True self.links_to_split = {} # link -> vars to pop_alive across the link newops = [] Modified: pypy/dist/pypy/rpython/memory/test/test_gctransform.py ============================================================================== --- pypy/dist/pypy/rpython/memory/test/test_gctransform.py (original) +++ pypy/dist/pypy/rpython/memory/test/test_gctransform.py Sat Jan 28 13:36:46 2006 @@ -32,8 +32,8 @@ t = TranslationContext() t.buildannotator().build_types(func, inputtypes) t.buildrtyper().specialize(t) - transformer = transformcls(t.graphs) - transformer.transform() + transformer = transformcls() + transformer.transform(t.graphs) t.checkgraphs() for graph in t.graphs: for block in graph.iterblocks(): From tismer at codespeak.net Sat Jan 28 13:48:51 2006 From: tismer at codespeak.net (tismer at codespeak.net) Date: Sat, 28 Jan 2006 13:48:51 +0100 (CET) Subject: [pypy-svn] r22781 - in pypy/dist/pypy/module/stackless: . test Message-ID: <20060128124851.2FE6B27DD3@code1.codespeak.net> Author: tismer Date: Sat Jan 28 13:48:48 2006 New Revision: 22781 Modified: pypy/dist/pypy/module/stackless/interp_coroutine.py pypy/dist/pypy/module/stackless/test/test_interp_coroutine.py Log: minimalistic support for coroutine return vales. This is intentionally still a bit less than greenlets Modified: pypy/dist/pypy/module/stackless/interp_coroutine.py ============================================================================== --- pypy/dist/pypy/module/stackless/interp_coroutine.py (original) +++ pypy/dist/pypy/module/stackless/interp_coroutine.py Sat Jan 28 13:48:48 2006 @@ -39,17 +39,19 @@ self.parent = self else: self.parent = costate.main + self.thunk = None def bind(self, thunk): if self.frame is not None: raise CoroutineDamage - self.frame = self._bind(thunk) + self.thunk = thunk + self.frame = self._bind() - def _bind(self, thunk): + def _bind(self): self.parent = costate.current costate.last.frame = yield_current_frame_to_caller() try: - thunk.call() + self.thunk.call() except CoroutineExit: # ignore a shutdown exception pass @@ -57,6 +59,7 @@ # redirect all unhandled exceptions to the parent costate.things_to_do = True costate.temp_exc = e + self.thunk = None while self.parent.frame is None: # greenlet behavior is fine self.parent = self.parent.parent @@ -173,7 +176,7 @@ self.args = args def call(self): - self.space.call_args(self.w_func, self.args) + appcostate.tempval = self.space.call_args(self.w_func, self.args) class AppCoroutine(Coroutine): # XXX, StacklessFlags): @@ -203,6 +206,8 @@ "cannot switch to an unbound Coroutine")) self.switch() appcostate.current = self + ret, appcostate.tempval = appcostate.tempval, space.w_None + return ret def w_kill(self): if appcostate.current is self: Modified: pypy/dist/pypy/module/stackless/test/test_interp_coroutine.py ============================================================================== --- pypy/dist/pypy/module/stackless/test/test_interp_coroutine.py (original) +++ pypy/dist/pypy/module/stackless/test/test_interp_coroutine.py Sat Jan 28 13:48:48 2006 @@ -168,7 +168,7 @@ data = wrap_stackless_function(f) assert int(data.strip()) == 12345678 -def test_kill_raise_coro(): +def test_kill_raise_del_coro(): class T: def __init__(self, func, arg): self.func = func From mwh at codespeak.net Sat Jan 28 14:08:13 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Sat, 28 Jan 2006 14:08:13 +0100 (CET) Subject: [pypy-svn] r22782 - in pypy/dist/pypy/rpython/memory: . test Message-ID: <20060128130813.8770F27DD3@code1.codespeak.net> Author: mwh Date: Sat Jan 28 14:08:11 2006 New Revision: 22782 Modified: pypy/dist/pypy/rpython/memory/gctransform.py pypy/dist/pypy/rpython/memory/test/test_gctransform.py Log: (cfbolz,mwh) Test that we handle variables without concretetypes correctly. When looking at the graph, we realised that we need to handle return values of type PyObject differently -- they never need to be push_alived as we are always given a new reference to a PyObject. So we fixed that and tested for it too. Modified: pypy/dist/pypy/rpython/memory/gctransform.py ============================================================================== --- pypy/dist/pypy/rpython/memory/gctransform.py (original) +++ pypy/dist/pypy/rpython/memory/gctransform.py Sat Jan 28 14:08:11 2006 @@ -24,8 +24,23 @@ EXCEPTION_RAISING_OPS = ['direct_call', 'indirect_call'] def var_needsgc(var): - vartype = var.concretetype - return isinstance(vartype, lltype.Ptr) and vartype._needsgc() + if hasattr(var, 'concretetype'): + vartype = var.concretetype + return isinstance(vartype, lltype.Ptr) and vartype._needsgc() + else: + # assume PyObjPtr + return True + +def var_ispyobj(var): + if hasattr(var, 'concretetype'): + if isinstance(var.concretetype, lltype.Ptr): + return var.concretetype.TO is lltype.PyObject + else: + return False + else: + # assume PyObjPtr + return True + class GCTransformer: def __init__(self): @@ -73,7 +88,7 @@ cleanup_on_exception.extend(self.pop_alive(var)) op.cleanup = cleanup_on_exception if var_needsgc(op.result): - if op.opname not in ('direct_call', 'indirect_call'): + if op.opname not in ('direct_call', 'indirect_call') and not var_ispyobj(op.result): newops.extend(self.push_alive(op.result)) livevars.append(op.result) if len(block.exits) == 0: @@ -102,7 +117,7 @@ return [op] def push_alive(self, var): - if var.concretetype.TO is lltype.PyObject: + if var_ispyobj(var): return self.push_alive_pyobj(var) else: return self.push_alive_nopyobj(var) @@ -118,7 +133,7 @@ return [SpaceOperation("gc_push_alive_pyobj", [var], result)] def pop_alive(self, var): - if var.concretetype.TO is lltype.PyObject: + if var_ispyobj(var): return self.pop_alive_pyobj(var) else: return self.pop_alive_nopyobj(var) Modified: pypy/dist/pypy/rpython/memory/test/test_gctransform.py ============================================================================== --- pypy/dist/pypy/rpython/memory/test/test_gctransform.py (original) +++ pypy/dist/pypy/rpython/memory/test/test_gctransform.py Sat Jan 28 14:08:11 2006 @@ -1,5 +1,5 @@ from pypy.rpython.memory import gctransform -from pypy.rpython.memory.gctransform import var_needsgc +from pypy.rpython.memory.gctransform import var_needsgc, var_ispyobj from pypy.translator.translator import TranslationContext, graphof from pypy.rpython.lltypesystem import lltype from pypy.objspace.flow.model import Variable @@ -14,7 +14,10 @@ else: refs_in = len([v for v in block.inputargs if var_needsgc(v)]) push_alives = len([op for op in block.operations - if op.opname.startswith('gc_push_alive')]) + if op.opname.startswith('gc_push_alive')]) + \ + len([op for op in block.operations + if var_ispyobj(op.result) and 'direct_call' not in op.opname]) + pop_alives = len([op for op in block.operations if op.opname.startswith('gc_pop_alive')]) calls = len([op for op in block.operations @@ -28,10 +31,11 @@ assert refs_in + push_alives + calls == pop_alives + refs_out -def rtype_and_transform(func, inputtypes, transformcls): +def rtype_and_transform(func, inputtypes, transformcls, specialize=True): t = TranslationContext() t.buildannotator().build_types(func, inputtypes) - t.buildrtyper().specialize(t) + if specialize: + t.buildrtyper().specialize(t) transformer = transformcls() transformer.transform(t.graphs) t.checkgraphs() @@ -173,5 +177,18 @@ f(s) return s.x t = rtype_and_transform(g, [], gctransform.GCTransformer) - ggraph = graphof(t, g) +def test_noconcretetype(): + def f(): + return [1][0] + t = rtype_and_transform(f, [], gctransform.GCTransformer, specialize=False) + fgraph = graphof(t, f) + push_count = 0 + pop_count = 0 + for op in fgraph.startblock.operations: + if op.opname == 'gc_push_alive_pyobj': + push_count += 1 + elif op.opname == 'gc_pop_alive_pyobj': + pop_count += 1 + assert push_count == 0 and pop_count == 1 + From pedronis at codespeak.net Sat Jan 28 14:28:39 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Sat, 28 Jan 2006 14:28:39 +0100 (CET) Subject: [pypy-svn] r22783 - pypy/dist/pypy/jit/test Message-ID: <20060128132839.1682427DD4@code1.codespeak.net> Author: pedronis Date: Sat Jan 28 14:28:37 2006 New Revision: 22783 Modified: pypy/dist/pypy/jit/test/test_hint_annotation.py Log: (arre, arigo, pedronis) test about various hint annotation meeting as operands to an operation. Modified: pypy/dist/pypy/jit/test/test_hint_annotation.py ============================================================================== --- pypy/dist/pypy/jit/test/test_hint_annotation.py (original) +++ pypy/dist/pypy/jit/test/test_hint_annotation.py Sat Jan 28 14:28:37 2006 @@ -1,6 +1,7 @@ import py from pypy.translator.translator import TranslationContext, graphof from pypy.jit.hintannotator import HintAnnotator +from pypy.jit.hintbookkeeper import HintBookkeeper from pypy.jit.hintmodel import * from pypy.rpython.lltypesystem import lltype from pypy.rpython.objectmodel import hint @@ -113,6 +114,28 @@ assert unionof(av1, ac1) == av1 assert unionof(ac1, av1) == av1 assert unionof(ac3, av1) == av1 - assert unionof(av2, ac4) == av1 + assert unionof(av2, ac4) == av1 + +def test_op_meet(): + def meet(hs1, hs2): + HintBookkeeper().enter(None) + return pair(hs1, hs2).int_add() + av1, av2 = SomeLLAbstractVariable(lltype.Signed), SomeLLAbstractVariable(lltype.Signed) + cv1, cv2 = SomeLLConcreteValue(lltype.Signed), SomeLLConcreteValue(lltype.Signed) + ac1, ac2 = SomeLLAbstractConstant(lltype.Signed, {}), SomeLLAbstractConstant(lltype.Signed, {}) + assert meet(av1, av2) == av1 + assert meet(cv1, cv2) == cv2 + assert isinstance(meet(ac1, ac2), SomeLLAbstractConstant) + assert meet(ac1, cv1) == cv1 + assert meet(cv1, ac1) == cv1 + assert meet(av1, cv1) == av1 + assert meet(cv1, av1) == av1 + assert meet(ac1, av1) == av1 + assert meet(av1, ac1) == av1 + + + + + From pedronis at codespeak.net Sat Jan 28 14:47:54 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Sat, 28 Jan 2006 14:47:54 +0100 (CET) Subject: [pypy-svn] r22785 - in pypy/dist/pypy/jit: . test Message-ID: <20060128134754.C547427DD4@code1.codespeak.net> Author: pedronis Date: Sat Jan 28 14:47:52 2006 New Revision: 22785 Modified: pypy/dist/pypy/jit/hintmodel.py pypy/dist/pypy/jit/test/test_hint_annotation.py Log: (arre, arigo, pedronis) some hint anntotator tests about looping. Modified: pypy/dist/pypy/jit/hintmodel.py ============================================================================== --- pypy/dist/pypy/jit/hintmodel.py (original) +++ pypy/dist/pypy/jit/hintmodel.py Sat Jan 28 14:47:52 2006 @@ -5,7 +5,7 @@ UNARY_OPERATIONS = "same_as hint".split() -BINARY_OPERATIONS = "int_add int_sub".split() +BINARY_OPERATIONS = "int_add int_sub int_gt int_eq".split() class OriginTreeNode(object): @@ -91,6 +91,12 @@ int_sub = int_add + def int_gt((hs_c1, hs_c2)): + origin = getbookkeeper().myorigin() + origin.merge(hs_c1.origins) + origin.merge(hs_c2.origins) + return SomeLLAbstractConstant(lltype.Bool, {origin: True}) + def union((hs_c1, hs_c2)): assert hs_c1.concretetype == hs_c2.concretetype origins = annmodel.setunion(hs_c1.origins, hs_c2.origins) @@ -102,3 +108,6 @@ def int_add((hs_c1, hs_c2)): return SomeLLConcreteValue(lltype.Signed) + + def int_eq((hs_c1, hs_c2)): + return SomeLLConcreteValue(lltype.Bool) Modified: pypy/dist/pypy/jit/test/test_hint_annotation.py ============================================================================== --- pypy/dist/pypy/jit/test/test_hint_annotation.py (original) +++ pypy/dist/pypy/jit/test/test_hint_annotation.py Sat Jan 28 14:47:52 2006 @@ -133,6 +133,29 @@ assert meet(ac1, av1) == av1 assert meet(av1, ac1) == av1 +def test_loop(): + def ll_function(x, y): + while x > 0: + y += x + x -= 1 + return y + hs = hannotate(ll_function, [int, int]) + assert isinstance(hs, SomeLLAbstractConstant) + assert hs.concretetype == lltype.Signed + assert len(hs.origins) == 2 + +def test_loop1(): + def ll_function(x, y): + while x > 0: + x1 = hint(x, concrete=True) + if x1 == 7: + y += x + x -= 1 + return y + hs = hannotate(ll_function, [int, int]) + assert isinstance(hs, SomeLLAbstractConstant) + assert hs.concretetype == lltype.Signed + assert len(hs.origins) == 2 From mwh at codespeak.net Sat Jan 28 15:04:53 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Sat, 28 Jan 2006 15:04:53 +0100 (CET) Subject: [pypy-svn] r22786 - in pypy/dist/pypy/rpython/memory: . test Message-ID: <20060128140453.4DF2C27DD4@code1.codespeak.net> Author: mwh Date: Sat Jan 28 15:04:51 2006 New Revision: 22786 Modified: pypy/dist/pypy/rpython/memory/gctransform.py pypy/dist/pypy/rpython/memory/test/test_gctransform.py Log: (cfbolz, mwh) ever more tortuous code to get the gc operations around exceptions to be correct. Modified: pypy/dist/pypy/rpython/memory/gctransform.py ============================================================================== --- pypy/dist/pypy/rpython/memory/gctransform.py (original) +++ pypy/dist/pypy/rpython/memory/gctransform.py Sat Jan 28 15:04:51 2006 @@ -1,5 +1,5 @@ from pypy.rpython.lltypesystem import lltype -from pypy.objspace.flow.model import SpaceOperation, Variable +from pypy.objspace.flow.model import SpaceOperation, Variable, c_last_exception from pypy.translator.unsimplify import insert_empty_block from pypy.rpython import rmodel import sets @@ -95,13 +95,26 @@ # everything is fine already for returnblocks and exceptblocks pass else: - deadinallexits = sets.Set(livevars) - for link in block.exits: - deadinallexits.difference_update(sets.Set(link.args)) + if block.exitswitch is c_last_exception: + # if we're in a try block, the last operation must + # remain the last operation, so don't add a pop_alive + # to the block, even if the variable dies in all + # linked blocks. + deadinallexits = sets.Set([]) + else: + deadinallexits = sets.Set(livevars) + for link in block.exits: + deadinallexits.difference_update(sets.Set(link.args)) for var in deadinallexits: newops.extend(self.pop_alive(var)) for link in block.exits: livecounts = dict.fromkeys(sets.Set(livevars) - deadinallexits, 1) + if (block.exitswitch is c_last_exception and link.exitcase is not None + and livevars[-1] is block.operations[-1].result): + # if the last operation in the block raised an + # exception, it can't have returned anything that + # might need pop_aliving. + del livecounts[livevars[-1]] for v in link.args: if v in livecounts: livecounts[v] -= 1 Modified: pypy/dist/pypy/rpython/memory/test/test_gctransform.py ============================================================================== --- pypy/dist/pypy/rpython/memory/test/test_gctransform.py (original) +++ pypy/dist/pypy/rpython/memory/test/test_gctransform.py Sat Jan 28 15:04:51 2006 @@ -1,4 +1,5 @@ from pypy.rpython.memory import gctransform +from pypy.objspace.flow.model import c_last_exception from pypy.rpython.memory.gctransform import var_needsgc, var_ispyobj from pypy.translator.translator import TranslationContext, graphof from pypy.rpython.lltypesystem import lltype @@ -26,9 +27,15 @@ # it's a block we inserted return for link in block.exits: + fudge = 0 + if (block.exitswitch is c_last_exception and link.exitcase is not None + and var_needsgc(block.operations[-1].result)): + fudge = 1 refs_out = len([v for v in link.args if isinstance(v, Variable) and var_needsgc(v)]) - assert refs_in + push_alives + calls == pop_alives + refs_out + if link.last_exception is not None and link.last_exception in link.args: + refs_out -= 1 + assert refs_in + push_alives + calls - fudge == pop_alives + refs_out def rtype_and_transform(func, inputtypes, transformcls, specialize=True): @@ -38,10 +45,10 @@ t.buildrtyper().specialize(t) transformer = transformcls() transformer.transform(t.graphs) +# t.view() t.checkgraphs() for graph in t.graphs: for block in graph.iterblocks(): - print graph, block, block.isstartblock checkblock(block) return t @@ -192,3 +199,35 @@ pop_count += 1 assert push_count == 0 and pop_count == 1 +def test_except_block(): + S = lltype.GcStruct("S", ('x', lltype.Signed)) + def f(a, n): + if n == 0: + raise ValueError + a.x = 1 + return a + def g(n): + a = lltype.malloc(S) + try: + return f(a, n).x + except ValueError: + return 0 + t = rtype_and_transform(g, [int], gctransform.GCTransformer) + +def test_except_block2(): + # the difference here is that f() returns Void, not a GcStruct + S = lltype.GcStruct("S", ('x', lltype.Signed)) + def f(a, n): + if n == 0: + raise ValueError + a.x = 1 + def g(n): + a = lltype.malloc(S) + try: + f(a, n) + return a.x + except ValueError: + return 0 + t = rtype_and_transform(g, [int], gctransform.GCTransformer) + + From rxe at codespeak.net Sat Jan 28 15:07:52 2006 From: rxe at codespeak.net (rxe at codespeak.net) Date: Sat, 28 Jan 2006 15:07:52 +0100 (CET) Subject: [pypy-svn] r22787 - pypy/dist/pypy/rpython/raisingops Message-ID: <20060128140752.504E827DD6@code1.codespeak.net> Author: rxe Date: Sat Jan 28 15:07:50 2006 New Revision: 22787 Modified: pypy/dist/pypy/rpython/raisingops/ (props changed) pypy/dist/pypy/rpython/raisingops/__init__.py (props changed) pypy/dist/pypy/rpython/raisingops/raisingops.py (props changed) Log: fixeol From cfbolz at codespeak.net Sat Jan 28 15:09:35 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Sat, 28 Jan 2006 15:09:35 +0100 (CET) Subject: [pypy-svn] r22788 - in pypy/branch/genc-gc-refactoring: . test Message-ID: <20060128140935.477B127DD6@code1.codespeak.net> Author: cfbolz Date: Sat Jan 28 15:09:32 2006 New Revision: 22788 Modified: pypy/branch/genc-gc-refactoring/database.py pypy/branch/genc-gc-refactoring/funcgen.py pypy/branch/genc-gc-refactoring/gc.py pypy/branch/genc-gc-refactoring/test/test_newgc.py Log: (mwh, cfbolz): attach the transformer to the gcpolicy, make funcgen call transformer on all graphs Modified: pypy/branch/genc-gc-refactoring/database.py ============================================================================== --- pypy/branch/genc-gc-refactoring/database.py (original) +++ pypy/branch/genc-gc-refactoring/database.py Sat Jan 28 15:09:32 2006 @@ -14,7 +14,7 @@ # ____________________________________________________________ -class LowLevelDatabase: +class LowLevelDatabase(object): def __init__(self, translator=None, standalone=False, gcpolicy=None, thread_enabled=False): self.translator = translator @@ -34,6 +34,7 @@ from pypy.translator.c import gc gcpolicy = gc.RefcountingGcPolicy self.gcpolicy = gcpolicy(self, thread_enabled) + self.gctransformer = gcpolicy.transformerclass() self.completed = False def gettypedefnode(self, T, varlength=1): Modified: pypy/branch/genc-gc-refactoring/funcgen.py ============================================================================== --- pypy/branch/genc-gc-refactoring/funcgen.py (original) +++ pypy/branch/genc-gc-refactoring/funcgen.py Sat Jan 28 15:09:32 2006 @@ -32,6 +32,9 @@ self.gcpolicy = db.gcpolicy self.cpython_exc = cpython_exc self.functionname = functionname + # apply the gc transformation + self.db.gctransformer.transform_graph(self.graph) + # # collect all variables and constants used in the body, # and get their types now @@ -165,7 +168,6 @@ if line: yield line - # collect all blocks for block in graph.iterblocks(): allblocks.append(block) Modified: pypy/branch/genc-gc-refactoring/gc.py ============================================================================== --- pypy/branch/genc-gc-refactoring/gc.py (original) +++ pypy/branch/genc-gc-refactoring/gc.py Sat Jan 28 15:09:32 2006 @@ -4,6 +4,7 @@ from pypy.rpython.lltypesystem.lltype import \ typeOf, Ptr, PyObject, ContainerType, GcArray, GcStruct, \ RuntimeTypeInfo, getRuntimeTypeInfo +from pypy.rpython.memory import gctransform PyObjPtr = Ptr(PyObject) @@ -87,6 +88,7 @@ destructor = None class RefcountingGcPolicy(BasicGcPolicy): + transformerclass = gctransform.GCTransformer def push_alive_nopyobj(self, expr, T): defnode = self.db.gettypedefnode(T.TO) Modified: pypy/branch/genc-gc-refactoring/test/test_newgc.py ============================================================================== --- pypy/branch/genc-gc-refactoring/test/test_newgc.py (original) +++ pypy/branch/genc-gc-refactoring/test/test_newgc.py Sat Jan 28 15:09:32 2006 @@ -14,15 +14,12 @@ t = TranslationContext() t.buildannotator().build_types(fn, inputtypes) t.buildrtyper().specialize() - GCTransformer(t.graphs).transform() - builder = genc.CExtModuleBuilder(t, fn, gcpolicy=gc.RefcountingGcPolicy) builder.generate_source() skip_missing_compiler(builder.compile) builder.import_module() return builder.get_entry_point() - def test_something(): def f(): return 1 From cfbolz at codespeak.net Sat Jan 28 15:11:16 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Sat, 28 Jan 2006 15:11:16 +0100 (CET) Subject: [pypy-svn] r22789 - pypy/branch/genc-gc-refactoring Message-ID: <20060128141116.9B21527DD4@code1.codespeak.net> Author: cfbolz Date: Sat Jan 28 15:11:15 2006 New Revision: 22789 Modified: pypy/branch/genc-gc-refactoring/funcgen.py Log: (mwh, cfbolz): remove this piece of increfing because it is now done correctly by the gctransformer. Modified: pypy/branch/genc-gc-refactoring/funcgen.py ============================================================================== --- pypy/branch/genc-gc-refactoring/funcgen.py (original) +++ pypy/branch/genc-gc-refactoring/funcgen.py Sat Jan 28 15:11:15 2006 @@ -162,12 +162,6 @@ blocknum = {} allblocks = [] - # match the subsequent pop_alive for each input argument - for a in self.graph.getargs(): - line = self.push_alive(a) - if line: - yield line - # collect all blocks for block in graph.iterblocks(): allblocks.append(block) From ludal at codespeak.net Sat Jan 28 15:13:55 2006 From: ludal at codespeak.net (ludal at codespeak.net) Date: Sat, 28 Jan 2006 15:13:55 +0100 (CET) Subject: [pypy-svn] r22790 - pypy/branch/ast-experiments/pypy/interpreter/pyparser Message-ID: <20060128141355.B910427DD4@code1.codespeak.net> Author: ludal Date: Sat Jan 28 15:13:53 2006 New Revision: 22790 Modified: pypy/branch/ast-experiments/pypy/interpreter/pyparser/ebnfgrammar.py pypy/branch/ast-experiments/pypy/interpreter/pyparser/ebnflexer.py pypy/branch/ast-experiments/pypy/interpreter/pyparser/ebnfparse.py pypy/branch/ast-experiments/pypy/interpreter/pyparser/grammar.py pypy/branch/ast-experiments/pypy/interpreter/pyparser/pythonlexer.py pypy/branch/ast-experiments/pypy/interpreter/pyparser/pythonparse.py Log: refactor, part II forgot the parser class make pythonlexer.py use it Modified: pypy/branch/ast-experiments/pypy/interpreter/pyparser/ebnfgrammar.py ============================================================================== --- pypy/branch/ast-experiments/pypy/interpreter/pyparser/ebnfgrammar.py (original) +++ pypy/branch/ast-experiments/pypy/interpreter/pyparser/ebnfgrammar.py Sat Jan 28 15:13:53 2006 @@ -2,9 +2,8 @@ # and the symbol mappings from grammar import BaseGrammarBuilder, Alternative, Sequence, Token, \ - KleeneStar, GrammarElement + KleeneStar, GrammarElement, Parser -from pypy.interpreter.pyparser.parser import Parser ## sym_map = {} ## sym_rmap = {} @@ -68,34 +67,34 @@ p.add_token('EOF','EOF') # star: '*' | '+' - star = p.Alternative( "star", [p.Token('TOK_STAR', '*'), p.Token('TOK_ADD', '+')] ) - star_opt = p.KleeneStar ( "star_opt", 0, 1, rule=star ) + star = p.Alternative_n( "star", [p.Token_n('TOK_STAR', '*'), p.Token_n('TOK_ADD', '+')] ) + star_opt = p.KleeneStar_n( "star_opt", 0, 1, rule=star ) # rule: SYMBOL ':' alternative - symbol = p.Sequence( "symbol", [p.Token('TOK_SYMBOL'), star_opt] ) - symboldef = p.Token( 'TOK_SYMDEF' ) - alternative = p.Sequence( "alternative", []) - rule = p.Sequence( "rule", [symboldef, alternative] ) + symbol = p.Sequence_n( "symbol", [p.Token_n('TOK_SYMBOL'), star_opt] ) + symboldef = p.Token_n( 'TOK_SYMDEF' ) + alternative = p.Sequence_n( "alternative", []) + rule = p.Sequence_n( "rule", [symboldef, alternative] ) # grammar: rule+ - grammar = p.KleeneStar( "grammar", _min=1, rule=rule ) + grammar = p.KleeneStar_n( "grammar", _min=1, rule=rule ) # alternative: sequence ( '|' sequence )* - sequence = p.KleeneStar( "sequence", 1 ) - seq_cont_list = p.Sequence( "seq_cont_list", [p.Token('TOK_BAR', '|'), sequence] ) - sequence_cont = p.KleeneStar( "sequence_cont",0, rule=seq_cont_list ) + sequence = p.KleeneStar_n( "sequence", 1 ) + seq_cont_list = p.Sequence_n( "seq_cont_list", [p.Token_n('TOK_BAR', '|'), sequence] ) + sequence_cont = p.KleeneStar_n( "sequence_cont",0, rule=seq_cont_list ) alternative.args = [ sequence, sequence_cont ] # option: '[' alternative ']' - option = p.Sequence( "option", [p.Token('TOK_LBRACKET', '['), alternative, p.Token('TOK_RBRACKET', ']')] ) + option = p.Sequence_n( "option", [p.Token_n('TOK_LBRACKET', '['), alternative, p.Token_n('TOK_RBRACKET', ']')] ) # group: '(' alternative ')' - group = p.Sequence( "group", [p.Token('TOK_LPAR', '('), alternative, p.Token('TOK_RPAR', ')'), star_opt] ) + group = p.Sequence_n( "group", [p.Token_n('TOK_LPAR', '('), alternative, p.Token_n('TOK_RPAR', ')'), star_opt] ) # sequence: (SYMBOL | STRING | option | group )+ - string = p.Token('TOK_STRING') - alt = p.Alternative( "sequence_alt", [symbol, string, option, group] ) + string = p.Token_n('TOK_STRING') + alt = p.Alternative_n( "sequence_alt", [symbol, string, option, group] ) sequence.args = [ alt ] p.root_rules['grammar'] = grammar Modified: pypy/branch/ast-experiments/pypy/interpreter/pyparser/ebnflexer.py ============================================================================== --- pypy/branch/ast-experiments/pypy/interpreter/pyparser/ebnflexer.py (original) +++ pypy/branch/ast-experiments/pypy/interpreter/pyparser/ebnflexer.py Sat Jan 28 15:13:53 2006 @@ -125,6 +125,7 @@ # means backtracking more than one token # will re-tokenize the stream (but this is the # grammar lexer so we don't care really!) + _p = self.parser if self._peeked is not None: peeked = self._peeked self._peeked = None @@ -135,7 +136,7 @@ end = len(self.input) pos = self.skip_empty_lines(inp,pos,end) if pos==end: - return self.parser.Token( 'EOF', None) + return _p.Token( _p.EOF, None) # at this point nextchar is not a white space nor \n nextchr = inp[pos] @@ -147,22 +148,22 @@ self.pos = npos _endpos = npos - 1 assert _endpos>=0 - return self.parser.Token( 'TOK_STRING', inp[pos+1:_endpos]) + return _p.Token( _p.TOK_STRING, inp[pos+1:_endpos]) else: npos = match_symbol( inp, pos, end) if npos!=pos: self.pos = npos if npos!=end and inp[npos]==":": self.pos += 1 - return self.parser.Token( 'TOK_SYMDEF', inp[pos:npos]) + return _p.Token( _p.TOK_SYMDEF, inp[pos:npos]) else: - return self.parser.Token( 'TOK_SYMBOL', inp[pos:npos]) + return _p.Token( _p.TOK_SYMBOL, inp[pos:npos]) # we still have pos!=end here chr = inp[pos] if chr in "[]()*+|": self.pos = pos+1 - return Token( self.parser, self.parser.tok_values[chr], chr) + return _p.Token( _p.tok_values[chr], chr) self.RaiseError( "Unknown token" ) def peek(self): Modified: pypy/branch/ast-experiments/pypy/interpreter/pyparser/ebnfparse.py ============================================================================== --- pypy/branch/ast-experiments/pypy/interpreter/pyparser/ebnfparse.py (original) +++ pypy/branch/ast-experiments/pypy/interpreter/pyparser/ebnfparse.py Sat Jan 28 15:13:53 2006 @@ -1,12 +1,11 @@ #!/usr/bin/env python from grammar import BaseGrammarBuilder, Alternative, Sequence, Token from grammar import GrammarProxy, KleeneStar, GrammarElement, build_first_sets -from grammar import AbstractBuilder, AbstractContext +from grammar import AbstractBuilder, AbstractContext, Parser from ebnflexer import GrammarSource import ebnfgrammar from ebnfgrammar import GRAMMAR_GRAMMAR from syntaxtree import AbstractSyntaxVisitor -from parser import Parser ORDA = ord("A") @@ -117,7 +116,8 @@ name is based on the current grammar rule being parsed""" rule_name = ":" + self.current_rule_name + "_%d" % self.current_subrule self.current_subrule += 1 - return rule_name + name_id = self.parser.add_anon_symbol( rule_name ) + return name_id def new_rule(self, rule): """A simple helper method that registers a new rule as 'known'""" @@ -155,7 +155,7 @@ def get_rule( self, name ): if name in self.parser.tokens: - return self.parser.Token( name ) + return self.parser.Token_n( name ) codename = self.get_symbolcode( name ) if codename in self.parser.root_rules: return self.parser.root_rules[codename] @@ -220,7 +220,7 @@ del self.rule_stack[0] if isinstance(old_rule,Token): # Wrap a token into an alternative - old_rule = self.parser.Alternative( self.current_rule_name, [old_rule] ) + old_rule = self.parser.Alternative( self.current_rule, [old_rule] ) else: # Make sure we use the codename from the named rule old_rule.codename = self.current_rule @@ -275,12 +275,12 @@ if value in self.parser.tok_values: # punctuation tokencode = self.parser.tok_values[value] - tok = Token( self.parser, tokencode, None ) + tok = self.parser.Token( tokencode, None ) else: if not is_py_name(value): raise RuntimeError("Unknown STRING value ('%s')" % value) # assume a keyword - tok = Token( self.parser, self.parser.NAME, value) + tok = self.parser.Token( self.parser.NAME, value) if value not in self.keywords: self.keywords.append(value) self.rule_stack.append(tok) Modified: pypy/branch/ast-experiments/pypy/interpreter/pyparser/grammar.py ============================================================================== --- pypy/branch/ast-experiments/pypy/interpreter/pyparser/grammar.py (original) +++ pypy/branch/ast-experiments/pypy/interpreter/pyparser/grammar.py Sat Jan 28 15:13:53 2006 @@ -10,12 +10,10 @@ try: from pypy.interpreter.baseobjspace import Wrappable from pypy.interpreter.pyparser.pytoken import NULLTOKEN - from pypy.interpreter.pyparser.parser import Parser except ImportError: # allows standalone testing Wrappable = object NULLTOKEN = -1 # None - from parser import Parser from syntaxtree import SyntaxNode, TempSyntaxNode, TokenNode @@ -303,10 +301,18 @@ pass def __str__(self): - return self.display(0) + try: + return self.display(0) + except Exception, e: + import traceback + traceback.print_exc() def __repr__(self): - return self.display(0) + try: + return self.display(0) + except Exception, e: + import traceback + traceback.print_exc() def display(self, level=0): """Helper function used to represent the grammar. @@ -717,4 +723,137 @@ - +class Parser(object): + def __init__(self): + pass + _anoncount = self._anoncount = -10 + _count = self._count = 0 + self.sym_name = {} # mapping symbol code -> symbol name + self.symbols = {} # mapping symbol name -> symbol code + self.tokens = { 'NULLTOKEN' : -1 } + self.EmptyToken = Token( self, -1, None ) + self.tok_name = {} + self.tok_values = {} + self._ann_sym_count = -10 + self._sym_count = 0 + self.all_rules = [] + self.root_rules = {} + + def symbol_repr( self, codename ): + if codename in self.tok_name: + return self.tok_name[codename] + elif codename in self.sym_name: + return self.sym_name[codename] + return "%d" % codename + + def add_symbol( self, sym ): + assert isinstance( sym, str ) + if not sym in self.symbols: + val = self._sym_count + self._sym_count += 1 + self.symbols[sym] = val + self.sym_name[val] = sym + return val + return self.symbols[ sym ] + + def add_anon_symbol( self, sym ): + assert isinstance( sym, str ) + if not sym in self.symbols: + val = self._ann_sym_count + self._ann_sym_count -= 1 + self.symbols[sym] = val + self.sym_name[val] = sym + return val + return self.symbols[ sym ] + + def add_token( self, tok, value = None ): + assert isinstance( tok, str ) + if not tok in self.tokens: + val = self._sym_count + self._sym_count += 1 + self.tokens[tok] = val + self.tok_name[val] = tok + if value is not None: + self.tok_values[value] = val + return val + return self.tokens[ tok ] + + def load_symbols( self, symbols ): + for _value, _name in symbols.items(): + if _value < self._ann_sym_count: + self._ann_sym_count = _value - 1 + if _value > self._sym_count: + self._sym_count = _value + 1 + self.symbols[_name] = _value + self.sym_name[_value] = _name + + def build_first_sets(self): + """builds the real first tokens set for each rule in + + Because a rule can be recursive (directly or indirectly), the + *simplest* algorithm to build each first set is to recompute them + until Computation(N) = Computation(N-1), N being the number of rounds. + As an example, on Python2.3's grammar, we need 19 cycles to compute + full first sets. + """ + rules = self.all_rules + changed = True + while changed: + # loop while one first set is changed + changed = False + for rule in rules: + # For each rule, recompute first set + size = len(rule.first_set) + rule.calc_first_set() + new_size = len(rule.first_set) + if new_size != size: + changed = True + for r in rules: + assert len(r.first_set) > 0, "Error: ot Empty firstset for %s" % r + r.reorder_rule() + + + def Alternative( self, name_id, args ): + assert isinstance( name_id, int ) + alt = Alternative( self, name_id, args ) + self.all_rules.append( alt ) + return alt + + def Alternative_n(self, name, args ): + assert isinstance(name, str) + name_id = self.add_symbol( name ) + return self.Alternative( name_id, args ) + + def Sequence( self, name_id, args ): + assert isinstance( name_id, int ) + alt = Sequence( self, name_id, args ) + self.all_rules.append( alt ) + return alt + + def Sequence_n(self, name, args ): + assert isinstance(name, str) + name_id = self.add_symbol( name ) + return self.Sequence( name_id, args ) + + def KleeneStar( self, name_id, _min = 0, _max = -1, rule = None ): + assert isinstance( name_id, int ) + alt = KleeneStar( self, name_id, _min, _max, rule ) + self.all_rules.append( alt ) + return alt + + def KleeneStar_n(self, name, _min = 0, _max = -1, rule = None ): + assert isinstance(name, str) + name_id = self.add_symbol( name ) + return self.KleeneStar( name_id, _min, _max, rule ) + + def Token_n(self, name, value = None ): + assert isinstance( name, str) + assert value is None or isinstance( value, str) + name_id = self.add_token( name, value ) + return self.Token( name_id, value ) + + def Token(self, name_id, value = None ): + assert isinstance( name_id, int ) + assert value is None or isinstance( value, str) + tok = Token( self, name_id, value ) + return tok Modified: pypy/branch/ast-experiments/pypy/interpreter/pyparser/pythonlexer.py ============================================================================== --- pypy/branch/ast-experiments/pypy/interpreter/pyparser/pythonlexer.py (original) +++ pypy/branch/ast-experiments/pypy/interpreter/pyparser/pythonlexer.py Sat Jan 28 15:13:53 2006 @@ -5,7 +5,7 @@ import sys from codeop import PyCF_DONT_IMPLY_DEDENT -from pypy.interpreter.pyparser.grammar import TokenSource, Token, AbstractContext +from pypy.interpreter.pyparser.grammar import TokenSource, Token, AbstractContext, Parser from pypy.interpreter.pyparser.error import SyntaxError import pytoken @@ -51,7 +51,7 @@ ################################################################################ from pypy.interpreter.pyparser import pytoken from pytokenize import tabsize, whiteSpaceDFA, triple_quoted, endDFAs, \ - single_quoted, pseudoDFA + single_quoted, pseudoDFA import automata @@ -62,7 +62,7 @@ SyntaxError.__init__(self, msg, lineno, offset, line) self.token_stack = token_stack -def generate_tokens(lines, flags): +def generate_tokens( parser, lines, flags): """ This is a rewrite of pypy.module.parser.pytokenize.generate_tokens since the original function is not RPYTHON (uses yield) @@ -91,6 +91,7 @@ #for line in lines: # print repr(line) #print '------------------- flags=%s ---->' % flags + assert isinstance( parser, Parser ) token_list = [] lnum = parenlev = continued = 0 namechars = NAMECHARS @@ -120,7 +121,7 @@ endmatch = endDFA.recognize(line) if endmatch >= 0: pos = end = endmatch - tok = Token(pytoken.STRING, contstr + line[:end]) + tok = parser.Token(parser.STRING, contstr + line[:end]) token_list.append((tok, line, lnum, pos)) last_comment = '' # token_list.append((STRING, contstr + line[:end], @@ -129,7 +130,7 @@ contline = None elif (needcont and not line.endswith('\\\n') and not line.endswith('\\\r\n')): - tok = Token(pytoken.ERRORTOKEN, contstr + line) + tok = parser.Token(parser.ERRORTOKEN, contstr + line) token_list.append((tok, line, lnum, pos)) last_comment = '' # token_list.append((ERRORTOKEN, contstr + line, @@ -155,10 +156,10 @@ if line[pos] in '#\r\n': # skip comments or blank lines if line[pos] == '#': - tok = Token(pytoken.COMMENT, line[pos:]) + tok = parser.Token(parser.COMMENT, line[pos:]) last_comment = line[pos:] else: - tok = Token(pytoken.NL, line[pos:]) + tok = parser.Token(parser.NL, line[pos:]) last_comment = '' # XXX Skip NL and COMMENT Tokens # token_list.append((tok, line, lnum, pos)) @@ -166,12 +167,12 @@ if column > indents[-1]: # count indents or dedents indents.append(column) - tok = Token(pytoken.INDENT, line[:pos]) + tok = parser.Token(parser.INDENT, line[:pos]) token_list.append((tok, line, lnum, pos)) last_comment = '' while column < indents[-1]: indents = indents[:-1] - tok = Token(pytoken.DEDENT, '') + tok = parser.Token(parser.DEDENT, '') token_list.append((tok, line, lnum, pos)) last_comment = '' else: # continued statement @@ -198,22 +199,22 @@ token, initial = line[start:end], line[start] if initial in numchars or \ (initial == '.' and token != '.'): # ordinary number - tok = Token(pytoken.NUMBER, token) + tok = parser.Token(parser.NUMBER, token) token_list.append((tok, line, lnum, pos)) last_comment = '' elif initial in '\r\n': if parenlev > 0: - tok = Token(pytoken.NL, token) + tok = parser.Token(parser.NL, token) last_comment = '' # XXX Skip NL else: - tok = Token(pytoken.NEWLINE, token) + tok = parser.Token(parser.NEWLINE, token) # XXX YUCK ! tok.value = last_comment token_list.append((tok, line, lnum, pos)) last_comment = '' elif initial == '#': - tok = Token(pytoken.COMMENT, token) + tok = parser.Token(parser.COMMENT, token) last_comment = token # XXX Skip # token_list.append((tok, line, lnum, pos)) # token_list.append((COMMENT, token, spos, epos, line)) @@ -223,7 +224,7 @@ if endmatch >= 0: # all on one line pos = endmatch token = line[start:pos] - tok = Token(pytoken.STRING, token) + tok = parser.Token(parser.STRING, token) token_list.append((tok, line, lnum, pos)) last_comment = '' else: @@ -240,11 +241,11 @@ contline = line break else: # ordinary string - tok = Token(pytoken.STRING, token) + tok = parser.Token(parser.STRING, token) token_list.append((tok, line, lnum, pos)) last_comment = '' elif initial in namechars: # ordinary name - tok = Token(pytoken.NAME, token) + tok = parser.Token(parser.NAME, token) token_list.append((tok, line, lnum, pos)) last_comment = '' elif initial == '\\': # continued stmt @@ -258,10 +259,11 @@ if parenlev < 0: raise TokenError("unmatched '%s'" % initial, line, (lnum-1, 0), token_list) - if token in pytoken.tok_punct: - tok = Token(pytoken.tok_punct[token]) + if token in parser.tok_values: + punct = parser.tok_values[token] + tok = parser.Token(punct) else: - tok = Token(pytoken.OP, token) + tok = parser.Token(parser.OP, token) token_list.append((tok, line, lnum, pos)) last_comment = '' else: @@ -271,7 +273,7 @@ if start Author: cfbolz Date: Sat Jan 28 15:15:09 2006 New Revision: 22791 Modified: pypy/branch/genc-gc-refactoring/funcgen.py Log: (mwh, cfbolz): attach blocknum to function generator Modified: pypy/branch/genc-gc-refactoring/funcgen.py ============================================================================== --- pypy/branch/genc-gc-refactoring/funcgen.py (original) +++ pypy/branch/genc-gc-refactoring/funcgen.py Sat Jan 28 15:15:09 2006 @@ -24,7 +24,8 @@ vars lltypes functionname - currentblock""".split() + currentblock + blocknum""".split() def __init__(self, graph, db, cpython_exc=False, functionname=None): self.graph = graph @@ -35,6 +36,7 @@ # apply the gc transformation self.db.gctransformer.transform_graph(self.graph) + self.blocknum = {} # # collect all variables and constants used in the body, # and get their types now @@ -44,6 +46,7 @@ mix = [self.graph.getreturnvar()] self.more_ll_values = [] for block in graph.iterblocks(): + self.blocknum[block] = len(self.blocknum) mix.extend(block.inputargs) for op in block.operations: mix.extend(op.args) @@ -159,21 +162,11 @@ def cfunction_body(self): graph = self.graph - blocknum = {} - allblocks = [] - - # collect all blocks - for block in graph.iterblocks(): - allblocks.append(block) - blocknum[block] = len(blocknum) - - assert graph.startblock is allblocks[0] - # generate the body of each block push_alive_op_result = self.gcpolicy.push_alive_op_result - for block in allblocks: + for block in graph.iterblocks(): self.currentblock = block - myblocknum = blocknum[block] + myblocknum = self.blocknum[block] yield '' yield 'block%d:' % myblocknum to_release = list(block.inputargs) @@ -207,7 +200,7 @@ elif block.exitswitch is None: # single-exit block assert len(block.exits) == 1 - for op in self.gen_link(block.exits[0], to_release, blocknum): + for op in self.gen_link(block.exits[0], to_release): yield op yield '' elif block.exitswitch == c_last_exception: @@ -215,7 +208,7 @@ # we handle the non-exceptional case first link = block.exits[0] assert link.exitcase is None - for op in self.gen_link(link, to_release, blocknum): + for op in self.gen_link(link, to_release): yield op # we must catch the exception raised by the last operation, # which goes to the last err%d_%d label written above. @@ -253,7 +246,7 @@ d[link.last_exc_value] = 'exc_value' else: yield '\t' + self.pop_alive_expr('exc_value', T2) - for op in self.gen_link(link, to_release, blocknum, d): + for op in self.gen_link(link, to_release, d): yield '\t' + op yield '}' fallthrough = True @@ -275,14 +268,14 @@ else: expr = '%s == Py_False' % expr yield 'if (%s) {' % expr - for op in self.gen_link(link, to_release, blocknum): + for op in self.gen_link(link, to_release): yield '\t' + op yield '}' link = block.exits[-1] assert link.exitcase in (False, True) #yield 'assert(%s == %s);' % (self.expr(block.exitswitch), # self.genc.nameofvalue(link.exitcase, ct)) - for op in self.gen_link(block.exits[-1], to_release, blocknum): + for op in self.gen_link(block.exits[-1], to_release): yield op yield '' elif TYPE in (Signed, Unsigned, SignedLongLong, @@ -295,7 +288,7 @@ defaultlink = link continue yield 'case %s:' % self.db.get(link.llexitcase) - for op in self.gen_link(link, to_release, blocknum): + for op in self.gen_link(link, to_release): yield '\t' + op yield 'break;' @@ -303,7 +296,7 @@ if defaultlink is None: raise TypeError('switches must have a default case.') yield 'default:' - for op in self.gen_link(defaultlink, to_release, blocknum): + for op in self.gen_link(defaultlink, to_release): yield '\t' + op yield '}' @@ -322,7 +315,7 @@ else: yield self.pop_alive(to_release[i-1]) - def gen_link(self, link, to_release, blocknum, linklocalvars=None): + def gen_link(self, link, to_release, linklocalvars=None): "Generate the code to jump across the given Link." is_alive = {} linklocalvars = linklocalvars or {} @@ -341,7 +334,7 @@ assignments.append((a2typename, dest, src)) for line in gen_assignments(assignments): yield line - yield 'goto block%d;' % blocknum[link.target] + yield 'goto block%d;' % self.blocknum[link.target] def gen_op(self, op, err): macro = 'OP_%s' % op.opname.upper() From pedronis at codespeak.net Sat Jan 28 15:19:29 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Sat, 28 Jan 2006 15:19:29 +0100 (CET) Subject: [pypy-svn] r22792 - in pypy/dist/pypy/jit: . test Message-ID: <20060128141929.0F19327DD4@code1.codespeak.net> Author: pedronis Date: Sat Jan 28 15:19:27 2006 New Revision: 22792 Modified: pypy/dist/pypy/jit/hintmodel.py pypy/dist/pypy/jit/test/test_hint_annotation.py Log: (arre, arigo, pedronis) getfield for immutable structures in the hint annotator. Modified: pypy/dist/pypy/jit/hintmodel.py ============================================================================== --- pypy/dist/pypy/jit/hintmodel.py (original) +++ pypy/dist/pypy/jit/hintmodel.py Sat Jan 28 15:19:27 2006 @@ -3,9 +3,9 @@ from pypy.jit.hintbookkeeper import getbookkeeper from pypy.rpython.lltypesystem import lltype -UNARY_OPERATIONS = "same_as hint".split() +UNARY_OPERATIONS = "same_as hint getfield".split() -BINARY_OPERATIONS = "int_add int_sub int_gt int_eq".split() +BINARY_OPERATIONS = "int_add int_sub int_mul int_gt int_eq".split() class OriginTreeNode(object): @@ -65,6 +65,15 @@ o1.fixed = True return SomeLLConcreteValue(hs_c1.concretetype) + def getfield(hs_c1, hs_fieldname): + S = hs_c1.concretetype.TO + FIELD_TYPE = getattr(S, hs_fieldname.const) + if S._hints.get('immutable', False): + origin = getbookkeeper().myorigin() + origin.merge(hs_c1.origins) + return SomeLLAbstractConstant(FIELD_TYPE, {origin: True}) + else: + return SomeLLAbstractVariable(FIELD_TYPE) class __extend__(pairtype(SomeLLAbstractValue, SomeLLAbstractValue)): @@ -89,7 +98,7 @@ origin.merge(hs_c2.origins) return SomeLLAbstractConstant(lltype.Signed, {origin: True}) - int_sub = int_add + int_mul = int_sub = int_add def int_gt((hs_c1, hs_c2)): origin = getbookkeeper().myorigin() Modified: pypy/dist/pypy/jit/test/test_hint_annotation.py ============================================================================== --- pypy/dist/pypy/jit/test/test_hint_annotation.py (original) +++ pypy/dist/pypy/jit/test/test_hint_annotation.py Sat Jan 28 15:19:27 2006 @@ -157,6 +157,17 @@ assert hs.concretetype == lltype.Signed assert len(hs.origins) == 2 +def test_simple_struct(): + S = lltype.GcStruct('helloworld', ('hello', lltype.Signed), + ('world', lltype.Signed), + hints={'immutable': True}) + def ll_function(s): + return s.hello * s.world + hs = hannotate(ll_function, [annmodel.SomePtr(lltype.Ptr(S))]) + assert isinstance(hs, SomeLLAbstractConstant) + assert hs.concretetype == lltype.Signed + assert len(hs.origins) == 1 + assert len(hs.origins.keys()[0].origins) == 2 From mwh at codespeak.net Sat Jan 28 15:38:47 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Sat, 28 Jan 2006 15:38:47 +0100 (CET) Subject: [pypy-svn] r22793 - in pypy/dist/pypy/rpython/memory: . test Message-ID: <20060128143847.5797C27DD4@code1.codespeak.net> Author: mwh Date: Sat Jan 28 15:38:45 2006 New Revision: 22793 Modified: pypy/dist/pypy/rpython/memory/gctransform.py pypy/dist/pypy/rpython/memory/test/test_gctransform.py Log: (cfbolz, mwh) don't go looking into the livevars list when it's empty Modified: pypy/dist/pypy/rpython/memory/gctransform.py ============================================================================== --- pypy/dist/pypy/rpython/memory/gctransform.py (original) +++ pypy/dist/pypy/rpython/memory/gctransform.py Sat Jan 28 15:38:45 2006 @@ -110,7 +110,7 @@ for link in block.exits: livecounts = dict.fromkeys(sets.Set(livevars) - deadinallexits, 1) if (block.exitswitch is c_last_exception and link.exitcase is not None - and livevars[-1] is block.operations[-1].result): + and livevars and livevars[-1] is block.operations[-1].result): # if the last operation in the block raised an # exception, it can't have returned anything that # might need pop_aliving. Modified: pypy/dist/pypy/rpython/memory/test/test_gctransform.py ============================================================================== --- pypy/dist/pypy/rpython/memory/test/test_gctransform.py (original) +++ pypy/dist/pypy/rpython/memory/test/test_gctransform.py Sat Jan 28 15:38:45 2006 @@ -230,4 +230,13 @@ return 0 t = rtype_and_transform(g, [int], gctransform.GCTransformer) - +def test_no_livevars_with_exception(): + def g(): + raise TypeError + def f(): + try: + g() + except TypeError: + return 0 + return 1 + t = rtype_and_transform(f, [], gctransform.GCTransformer) From rxe at codespeak.net Sat Jan 28 15:44:08 2006 From: rxe at codespeak.net (rxe at codespeak.net) Date: Sat, 28 Jan 2006 15:44:08 +0100 (CET) Subject: [pypy-svn] r22794 - in pypy/dist/pypy/translator/llvm: . backendopt externs externs/test module Message-ID: <20060128144408.A0E8B27DD4@code1.codespeak.net> Author: rxe Date: Sat Jan 28 15:43:57 2006 New Revision: 22794 Added: pypy/dist/pypy/translator/llvm/module/protos.h Removed: pypy/dist/pypy/translator/llvm/backendopt/removeexcmallocs.py Modified: pypy/dist/pypy/translator/llvm/exception.py pypy/dist/pypy/translator/llvm/externs/ (props changed) pypy/dist/pypy/translator/llvm/externs/ringbuffer.py pypy/dist/pypy/translator/llvm/externs/test/ (props changed) pypy/dist/pypy/translator/llvm/externs/test/test_ringbuffer.py pypy/dist/pypy/translator/llvm/externs2ll.py pypy/dist/pypy/translator/llvm/funcnode.py pypy/dist/pypy/translator/llvm/gc.py pypy/dist/pypy/translator/llvm/module/boehm.h pypy/dist/pypy/translator/llvm/module/excsupport.py pypy/dist/pypy/translator/llvm/module/genexterns.c pypy/dist/pypy/translator/llvm/opwriter.py Log: Not sure of the merits of this: replaced the removeexcmallocs transformation and ll code and replaced it with a rpython version. Far too much code need modified for this. Modified: pypy/dist/pypy/translator/llvm/exception.py ============================================================================== --- pypy/dist/pypy/translator/llvm/exception.py (original) +++ pypy/dist/pypy/translator/llvm/exception.py Sat Jan 28 15:43:57 2006 @@ -2,8 +2,8 @@ from pypy.translator.llvm.codewriter import DEFAULT_CCONV from pypy.translator.llvm.backendopt.exception import create_exception_handling -from pypy.translator.llvm.module.excsupport import \ - ringbuffer_decl, ringbuffer_code, invokeunwind_code, explicit_code +from pypy.translator.llvm.module.excsupport import invokeunwind_code, \ + explicit_code def repr_if_variable(db, arg): if isinstance(arg, Variable): @@ -89,7 +89,7 @@ returntype, entrypointname = entrynode.getdecl().split('%', 1) noresult = self._noresult(returntype) cconv = DEFAULT_CCONV - return invokeunwind_code % locals() + ringbuffer_code + return invokeunwind_code % locals() def invoke(self, codewriter, targetvar, tail_, cconv, returntype, functionref, args, label, except_label): @@ -182,13 +182,13 @@ self.invoke_count = 0 def llvm_declcode(self): - return ringbuffer_decl + return '' def llvm_implcode(self, entrynode): returntype, entrypointname = entrynode.getdecl().split('%', 1) noresult = self._noresult(returntype) cconv = DEFAULT_CCONV - return explicit_code % locals() + ringbuffer_code + return explicit_code % locals() def transform(self, translator, graph=None): if graph: Modified: pypy/dist/pypy/translator/llvm/externs/ringbuffer.py ============================================================================== --- pypy/dist/pypy/translator/llvm/externs/ringbuffer.py (original) +++ pypy/dist/pypy/translator/llvm/externs/ringbuffer.py Sat Jan 28 15:43:57 2006 @@ -1,22 +1,29 @@ -from pypy.rpython.memory.lladdress import raw_malloc +from pypy.rpython.memory import lladdress +from pypy.rpython.lltypesystem import lltype, llmemory -class RingBufferData: - size = 8192 - entry_maxsize = 16 - - def __init__(self): - self.index = 0 - def init(self): - self.data = raw_malloc(self.size + self.entry_maxsize) - -ringbuffer = RingBufferData() - -def initialise(): - ringbuffer.init() - -def malloc_exception(nbytes): - assert nbytes <= ringbuffer.entry_maxsize - addr = ringbuffer.data + ringbuffer.index - ringbuffer.index = (ringbuffer.index + nbytes) & (ringbuffer.size - 1) +# Cant store in class +size = 8192 +entry_maxsize = 16 +ringbufdata = lltype.malloc(lltype.GcArray(llmemory.Address), 1) +ringbufindex = lltype.malloc(lltype.GcArray(lltype.Signed), 1) + +def ringbuffer_initialise(): + ringbufdata[0] = lladdress.raw_malloc(size + entry_maxsize) + +def ringbuffer_malloc(nbytes): + assert nbytes <= entry_maxsize + addr = ringbufdata[0] + ringbufindex[0] + ringbufindex[0] = (ringbufindex[0] + nbytes) & (size - 1) return addr +# XXX would be nice to support something like this +# ringbufindex = lltype.malloc(lltype.GcArray(lltype.Char), size + entry_maxsize) + +# def ringbuffer_initialise(): +# pass + +# def ringbuffer_malloc(nbytes): +# assert nbytes <= entry_maxsize +# addr = lladdress.get_address_of_object(ringbufdata[ringbufindex[0]]) +# ringbufindex[0] = (ringbufindex[0] + nbytes) & (size - 1) +# return addr Modified: pypy/dist/pypy/translator/llvm/externs/test/test_ringbuffer.py ============================================================================== --- pypy/dist/pypy/translator/llvm/externs/test/test_ringbuffer.py (original) +++ pypy/dist/pypy/translator/llvm/externs/test/test_ringbuffer.py Sat Jan 28 15:43:57 2006 @@ -1,11 +1,22 @@ import py +from pypy.tool.udir import udir + from pypy.translator.llvm.test.runtest import compile_function as compile from pypy.translator.llvm.externs import ringbuffer +from pypy.translator.llvm import genllvm +def teardown_module(self): + for n in ["ccode.c", "ccode.ll"]: + f = udir.join(n) + if f.check(exists=1): + f.remove() + genllvm.GenLLVM.llexterns_header = None + genllvm.GenLLVM.llexterns_functions = None + def test_malloc_something(): def f(value): - ringbuffer.initialise() - buf = ringbuffer.malloc_exception(value) + ringbuffer.ringbuffer_initialise() + buf = ringbuffer.ringbuffer_malloc(value) buf.signed[0] = 10 return buf.signed[0] Modified: pypy/dist/pypy/translator/llvm/externs2ll.py ============================================================================== --- pypy/dist/pypy/translator/llvm/externs2ll.py (original) +++ pypy/dist/pypy/translator/llvm/externs2ll.py Sat Jan 28 15:43:57 2006 @@ -24,8 +24,8 @@ "%LLVM_RPython_StartupCode", ] -def get_genexterns_path(): - return os.path.join(get_llvm_cpath(), "genexterns.c") +def get_module_file(name): + return os.path.join(get_llvm_cpath(), name) def get_ll(ccode, function_names): function_names += support_functions @@ -101,6 +101,7 @@ raise "Can't compile external function code (llcode.c): ERROR:", llcode return decl, impl + def setup_externs(db): rtyper = db.translator.rtyper from pypy.translator.c.extfunc import predeclare_all @@ -123,6 +124,20 @@ else: assert False, "unhandled predeclare %s %s %s" % (c_name, type(obj), obj) + def annotatehelper(func, *argtypes): + graph = db.translator.rtyper.annotate_helper(func, argtypes) + fptr = rtyper.getcallable(graph) + c = inputconst(lltype.typeOf(fptr), fptr) + db.prepare_arg_value(c) + decls.append(("ll_" + func.func_name, graph)) + return graph.name + + if hasattr(db.gcpolicy, 'exc_useringbuf') and db.gcpolicy.exc_useringbuf: + from pypy.translator.llvm.externs import ringbuffer as rb + g = annotatehelper(rb.ringbuffer_initialise) + db.gcpolicy.ringbuf_malloc_name = \ + annotatehelper(rb.ringbuffer_malloc, lltype.Signed) + return decls def get_c_cpath(): @@ -179,6 +194,9 @@ assert False, "unhandled extern_decls %s %s %s" % (c_name, type(obj), obj) + # append protos + ccode.append(open(get_module_file('protos.h')).read()) + # include this early to get constants and macros for any further includes ccode.append('#include \n') @@ -186,7 +204,6 @@ ccode.append('%s\n' % db.gcpolicy.genextern_code()) # append our source file - ccode = "".join(ccode) - ccode += open(get_genexterns_path()).read() - - return get_ll(ccode, function_names) + ccode.append(open(get_module_file('genexterns.c')).read()) + + return get_ll("".join(ccode), function_names) Modified: pypy/dist/pypy/translator/llvm/funcnode.py ============================================================================== --- pypy/dist/pypy/translator/llvm/funcnode.py (original) +++ pypy/dist/pypy/translator/llvm/funcnode.py Sat Jan 28 15:43:57 2006 @@ -4,7 +4,6 @@ from pypy.translator.llvm.node import LLVMNode, ConstantLLVMNode from pypy.translator.llvm.opwriter import OpWriter from pypy.translator.llvm.log import log -from pypy.translator.llvm.backendopt.removeexcmallocs import remove_exception_mallocs from pypy.translator.unsimplify import remove_double_links log = log.funcnode @@ -72,9 +71,6 @@ def post_setup_transform(self): self.db.exceptionpolicy.transform(self.db.translator, self.graph) - import sys - if sys.maxint == 2**31-1: #XXX not yet 64bit compatible - remove_exception_mallocs(self.db.translator, self.graph, self.ref) remove_double_links(self.db.translator, self.graph) def writedecl(self, codewriter): Modified: pypy/dist/pypy/translator/llvm/gc.py ============================================================================== --- pypy/dist/pypy/translator/llvm/gc.py (original) +++ pypy/dist/pypy/translator/llvm/gc.py Sat Jan 28 15:43:57 2006 @@ -9,7 +9,7 @@ raise Exception, 'GcPolicy should not be used directly' def genextern_code(self): - return "" + return '' def gc_libraries(self): return [] @@ -53,12 +53,20 @@ #XXX memset class BoehmGcPolicy(GcPolicy): - def __init__(self, db): + + def __init__(self, db, exc_useringbuf=False): self.db = db self.n_malloced = 0 + self.exc_useringbuf = exc_useringbuf def genextern_code(self): - return '#include "boehm.h"' + r = '#include "boehm.h"\n' + + if self.exc_useringbuf: + r += '#define __GC_SETUP_CODE__ ll_ringbuffer_initialise();\n' + else: + r += '#define __GC_SETUP_CODE__\n' + return r def gc_libraries(self): return ['gc', 'pthread'] # XXX on windows? @@ -71,7 +79,13 @@ return GC_get_heap_size() ''' - def _malloc(self, codewriter, targetvar, size=1, atomic=False): + def get_count(self, inc=False): + if inc: + self.n_malloced += 1 + return '_%d' % self.n_malloced + + def _malloc(self, codewriter, targetvar, size=1, atomic=False, + exc_flag=False): """ assumes malloc of word size """ # XXX Boehm aligns on 8 byte boundaries if sys.platform == 'linux2' and sys.maxint == 2**63-1: @@ -79,9 +93,18 @@ else: boundary_size = 0 + word = self.db.get_machine_word() uword = self.db.get_machine_uword() - fnname = '%pypy_malloc' + (atomic and '_atomic' or '') - codewriter.call(targetvar, 'sbyte*', fnname, [uword], [size]) + + if self.exc_useringbuf and exc_flag: + fnname = '%pypy_' + self.ringbuf_malloc_name + else: + fnname = '%pypy_malloc' + (atomic and '_atomic' or '') + + # malloc_size is unsigned right now + sizei = '%malloc_sizei' + self.get_count() + codewriter.cast(sizei, uword, size, word) + codewriter.call(targetvar, 'sbyte*', fnname, [word], [sizei]) if atomic: codewriter.call(None, 'void', '%llvm.memset', @@ -89,18 +112,17 @@ [targetvar, 0, size, boundary_size], cconv='ccc') - def malloc(self, codewriter, targetvar, type_, size=1, atomic=False): + def malloc(self, codewriter, targetvar, type_, size=1, atomic=False, + exc_flag=False): uword = self.db.get_machine_uword() - self.n_malloced += 1 - cnt = '_%d' % self.n_malloced - malloc_ptr = '%malloc_ptr' + cnt - malloc_size = '%malloc_size' + cnt - malloc_sizeu = '%malloc_sizeu' + cnt + malloc_ptr = '%malloc_ptr' + self.get_count(True) + malloc_size = '%malloc_size' + self.get_count() + malloc_sizeu = '%malloc_sizeu' + self.get_count() codewriter.getelementptr(malloc_size, type_, 'null', [(uword, size)], getptr=False) codewriter.cast(malloc_sizeu, type_, malloc_size, uword) - self._malloc(codewriter, malloc_ptr, malloc_sizeu, atomic) + self._malloc(codewriter, malloc_ptr, malloc_sizeu, atomic, exc_flag) codewriter.cast(targetvar, 'sbyte*', malloc_ptr, type_) def var_malloc(self, codewriter, targetvar, @@ -108,13 +130,11 @@ word = lentype = self.db.get_machine_word() uword = self.db.get_machine_uword() - self.n_malloced += 1 - cnt = '_%d' % self.n_malloced - malloc_ptr = '%malloc_ptr' + cnt - malloc_size = '%malloc_size' + cnt - malloc_sizeu = '%malloc_sizeu' + cnt - actuallen = '%actuallen' + cnt - arraylength = '%arraylength' + cnt + malloc_ptr = '%malloc_ptr' + self.get_count(True) + malloc_size = '%malloc_size' + self.get_count() + malloc_sizeu = '%malloc_sizeu' + self.get_count() + actuallen = '%actuallen' + self.get_count() + arraylength = '%arraylength' + self.get_count() ARRAY, indices_to_array = node.var_malloc_info() Modified: pypy/dist/pypy/translator/llvm/module/boehm.h ============================================================================== --- pypy/dist/pypy/translator/llvm/module/boehm.h (original) +++ pypy/dist/pypy/translator/llvm/module/boehm.h Sat Jan 28 15:43:57 2006 @@ -15,11 +15,11 @@ #define USING_BOEHM_GC -char *pypy_malloc(unsigned long size) { +char *pypy_malloc(long size) { return GC_MALLOC(size); } -char *pypy_malloc_atomic(unsigned long size) { +char *pypy_malloc_atomic(long size) { return GC_MALLOC_ATOMIC(size); } Modified: pypy/dist/pypy/translator/llvm/module/excsupport.py ============================================================================== --- pypy/dist/pypy/translator/llvm/module/excsupport.py (original) +++ pypy/dist/pypy/translator/llvm/module/excsupport.py Sat Jan 28 15:43:57 2006 @@ -1,37 +1,3 @@ -RINGBUFFER_SIZE = 8192 -RINGBUFFER_ENTRY_MAXSIZE = 16 -RINGBUFFER_OVERSIZE = RINGBUFFER_SIZE + RINGBUFFER_ENTRY_MAXSIZE - -ringbuffer_decl = """ -; 8208=8192+16 in the next line because the last one (16 bytes maxsize) might -; start at 8190 for instance. [RINGBUFFER_SIZE + RINGBUFFER_ENTRY_MAXSIZE] - -%%exception_ringbuffer = internal global [%s x sbyte] zeroinitializer -%%exception_ringbuffer_index = internal global uint 0 -""" % (RINGBUFFER_SIZE + RINGBUFFER_ENTRY_MAXSIZE) - -ringbuffer_code = ''' -internal fastcc sbyte* %%malloc_exception(uint %%nbytes) { - %%cond = setle uint %%nbytes, %d - br bool %%cond, label %%then, label %%else - -then: - %%tmp.3 = load uint* %%exception_ringbuffer_index - %%tmp.4 = getelementptr [%d x sbyte]* %%exception_ringbuffer, int 0, uint %%tmp.3 - %%tmp.6 = add uint %%tmp.3, %%nbytes - %%tmp.7 = and uint %%tmp.6, %d - store uint %%tmp.7, uint* %%exception_ringbuffer_index - ret sbyte* %%tmp.4 - -else: - %%tmp.8 = call ccc sbyte* %%pypy_malloc(uint %%nbytes) - ret sbyte* %%tmp.8 -} -''' % (RINGBUFFER_ENTRY_MAXSIZE, RINGBUFFER_OVERSIZE, RINGBUFFER_SIZE - 1) - -import sys -if sys.maxint != 2**31-1: #XXX need to move the ringbuffer code to another level anyway - ringbuffer_decl = ringbuffer_code = '' invokeunwind_code = ''' ccc %(returntype)s%%__entrypoint__%(entrypointname)s { Modified: pypy/dist/pypy/translator/llvm/module/genexterns.c ============================================================================== --- pypy/dist/pypy/translator/llvm/module/genexterns.c (original) +++ pypy/dist/pypy/translator/llvm/module/genexterns.c Sat Jan 28 15:43:57 2006 @@ -1,24 +1,4 @@ -// we hand craft these in module/support.ll -char *RPyString_AsString(RPyString*); -long RPyString_Size(RPyString*); -RPyString *RPyString_FromString(char *); -int RPyExceptionOccurred(void); -char* LLVM_RPython_StartupCode(void); - -#define RPyRaiseSimpleException(exctype, errormsg) raise##exctype(errormsg) - -// XXX generated by rpython - argggh have to feed in prototypes -RPyFREXP_RESULT *ll_frexp_result(double, int); -RPyMODF_RESULT *ll_modf_result(double, double); -RPySTAT_RESULT *ll_stat_result(int, int, int, int, int, int, int, int, int, int); -void RPYTHON_RAISE_OSERROR(int error); -#ifdef RPyListOfString - RPyListOfString *_RPyListOfString_New(long); - void _RPyListOfString_SetItem(RPyListOfString *, int, RPyString *); -#endif -// XXX end of proto hacks - // overflows/zeros/values raising operations #include "raisingop.h" @@ -116,6 +96,9 @@ if (error != NULL) { return 0; } + + __GC_SETUP_CODE__ + return 1; } Added: pypy/dist/pypy/translator/llvm/module/protos.h ============================================================================== --- (empty file) +++ pypy/dist/pypy/translator/llvm/module/protos.h Sat Jan 28 15:43:57 2006 @@ -0,0 +1,22 @@ + +// we hand craft these in module/support.ll +char *RPyString_AsString(RPyString*); +long RPyString_Size(RPyString*); +RPyString *RPyString_FromString(char *); +int RPyExceptionOccurred(void); +char* LLVM_RPython_StartupCode(void); + +#define RPyRaiseSimpleException(exctype, errormsg) raise##exctype(errormsg) + +// XXX generated by rpython - argggh have to feed in prototypes +RPyFREXP_RESULT *ll_frexp_result(double, int); +RPyMODF_RESULT *ll_modf_result(double, double); +RPySTAT_RESULT *ll_stat_result(int, int, int, int, int, int, int, int, int, int); +void RPYTHON_RAISE_OSERROR(int error); +#ifdef RPyListOfString + RPyListOfString *_RPyListOfString_New(long); + void _RPyListOfString_SetItem(RPyListOfString *, int, RPyString *); +#endif +void ll_ringbuffer_initialise(void); + +// XXX end of proto hacks Modified: pypy/dist/pypy/translator/llvm/opwriter.py ============================================================================== --- pypy/dist/pypy/translator/llvm/opwriter.py (original) +++ pypy/dist/pypy/translator/llvm/opwriter.py Sat Jan 28 15:43:57 2006 @@ -265,8 +265,15 @@ def malloc(self, opr): arg_type = opr.op.args[0].value + + # XXX hack better to look at the actual structure + name = str(opr.op.args[0]) + exc = False + if 'Exception' in name or 'Error' in name: + exc = True + self.db.gcpolicy.malloc(self.codewriter, opr.retref, opr.rettype, - atomic=arg_type._is_atomic()) + atomic=arg_type._is_atomic(), exc_flag=exc) def malloc_varsize(self, opr): @@ -489,14 +496,6 @@ # # XXX exception specific - move to policy? - def malloc_exception(self, opr): - tmpvar1, tmpvar2, tmpvar3 = self._tmp(3) - cw = self.codewriter - cw.getelementptr(tmpvar1, opr.rettype, "null", [("int", 1)], getptr=False) - cw.cast(tmpvar2, opr.rettype, tmpvar1, 'uint') - cw.call(tmpvar3, 'sbyte*', '%malloc_exception', ['uint'], [tmpvar2]) - cw.cast(opr.retref, 'sbyte*', tmpvar3, opr.rettype) - def last_exception_type_ptr(self, opr): op = opr.op e = self.db.translator.rtyper.getexceptiondata() From cfbolz at codespeak.net Sat Jan 28 16:07:44 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Sat, 28 Jan 2006 16:07:44 +0100 (CET) Subject: [pypy-svn] r22795 - pypy/branch/genc-gc-refactoring Message-ID: <20060128150744.4312B27DD4@code1.codespeak.net> Author: cfbolz Date: Sat Jan 28 16:07:42 2006 New Revision: 22795 Modified: pypy/branch/genc-gc-refactoring/funcgen.py Log: (cfbolz, mwh): generate the correct cleanup code. remove all the incref/pop_alive methods on the function generator. Modified: pypy/branch/genc-gc-refactoring/funcgen.py ============================================================================== --- pypy/branch/genc-gc-refactoring/funcgen.py (original) +++ pypy/branch/genc-gc-refactoring/funcgen.py Sat Jan 28 16:07:42 2006 @@ -163,26 +163,19 @@ graph = self.graph # generate the body of each block - push_alive_op_result = self.gcpolicy.push_alive_op_result for block in graph.iterblocks(): self.currentblock = block myblocknum = self.blocknum[block] yield '' yield 'block%d:' % myblocknum - to_release = list(block.inputargs) - reachable_err = -1 # the number of the first reachable err label - for op in block.operations: - err = 'err%d_%d' % (myblocknum, len(to_release)) + for i, op in enumerate(block.operations): + err = 'err%d_%d' % (myblocknum, i) line = self.gen_op(op, err) if '\n' in line: - for subline in line.split('\n'): + for subline in line.splitlines(): yield subline else: yield line - if line.find(err) >= 0: - reachable_err = len(to_release) - to_release.append(op.result) - fallthrough = False if len(block.exits) == 0: if len(block.inputargs) == 2: # exc_cls, exc_value @@ -200,7 +193,7 @@ elif block.exitswitch is None: # single-exit block assert len(block.exits) == 1 - for op in self.gen_link(block.exits[0], to_release): + for op in self.gen_link(block.exits[0]): yield op yield '' elif block.exitswitch == c_last_exception: @@ -208,14 +201,12 @@ # we handle the non-exceptional case first link = block.exits[0] assert link.exitcase is None - for op in self.gen_link(link, to_release): + for op in self.gen_link(link): yield op # we must catch the exception raised by the last operation, # which goes to the last err%d_%d label written above. yield '' - to_release.pop() # skip default error handling for this label - yield 'err%d_%d:' % (myblocknum, len(to_release)) - reachable_err = len(to_release) # XXX assert they are == ? + yield 'err%d_%d:' % (myblocknum, len(block.operations) - 1) yield '' for link in block.exits[1:]: assert issubclass(link.exitcase, Exception) @@ -246,7 +237,7 @@ d[link.last_exc_value] = 'exc_value' else: yield '\t' + self.pop_alive_expr('exc_value', T2) - for op in self.gen_link(link, to_release, d): + for op in self.gen_link(link, d): yield '\t' + op yield '}' fallthrough = True @@ -268,14 +259,14 @@ else: expr = '%s == Py_False' % expr yield 'if (%s) {' % expr - for op in self.gen_link(link, to_release): + for op in self.gen_link(link): yield '\t' + op yield '}' link = block.exits[-1] assert link.exitcase in (False, True) #yield 'assert(%s == %s);' % (self.expr(block.exitswitch), # self.genc.nameofvalue(link.exitcase, ct)) - for op in self.gen_link(block.exits[-1], to_release): + for op in self.gen_link(block.exits[-1]): yield op yield '' elif TYPE in (Signed, Unsigned, SignedLongLong, @@ -288,7 +279,7 @@ defaultlink = link continue yield 'case %s:' % self.db.get(link.llexitcase) - for op in self.gen_link(link, to_release): + for op in self.gen_link(link): yield '\t' + op yield 'break;' @@ -296,7 +287,7 @@ if defaultlink is None: raise TypeError('switches must have a default case.') yield 'default:' - for op in self.gen_link(defaultlink, to_release): + for op in self.gen_link(defaultlink): yield '\t' + op yield '}' @@ -304,23 +295,25 @@ raise TypeError("exitswitch type not supported" " Got %r" % (TYPE,)) - for i in range(reachable_err, -1, -1): + for i, op in list(enumerate(block.operations))[::-1]: if not fallthrough: yield 'err%d_%d:' % (myblocknum, i) else: fallthrough = False # this label was already generated - if i == 0: - for line in self.return_with_error(): + for cleanupop in getattr(op, 'cleanup', ()): + line = self.gen_op(cleanupop, 'should_never_be_jumped_to') + if '\\n' in line: + for l in line.splitlines(): + yield l + else: yield line - else: - yield self.pop_alive(to_release[i-1]) + for line in self.return_with_error(): + yield line - def gen_link(self, link, to_release, linklocalvars=None): + def gen_link(self, link, linklocalvars=None): "Generate the code to jump across the given Link." is_alive = {} linklocalvars = linklocalvars or {} - for v in to_release: - linklocalvars[v] = self.expr(v) assignments = [] for a1, a2 in zip(link.args, link.target.inputargs): a2type, a2typename = self.lltypes[id(a2)] @@ -613,13 +606,6 @@ def pyobj_incref_expr(self, expr, T): return self.gcpolicy.pyobj_incref(expr, T) - def pyobj_decref_expr(self, expr, T): - return self.gcpolicy.pyobj_decref(expr, T) - - def push_alive(self, v): - T = self.lltypemap(v) - return self.gcpolicy.push_alive(LOCALVAR % v.name, T) - def pop_alive(self, v, expr=None): T = self.lltypemap(v) return self.gcpolicy.pop_alive(expr or (LOCALVAR % v.name), T) From pedronis at codespeak.net Sat Jan 28 16:29:57 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Sat, 28 Jan 2006 16:29:57 +0100 (CET) Subject: [pypy-svn] r22796 - in pypy/dist/pypy/jit: . test Message-ID: <20060128152957.2A1E127DD4@code1.codespeak.net> Author: pedronis Date: Sat Jan 28 16:29:53 2006 New Revision: 22796 Added: pypy/dist/pypy/jit/hintcontainer.py (contents, props changed) Modified: pypy/dist/pypy/jit/hintannotator.py pypy/dist/pypy/jit/hintbookkeeper.py pypy/dist/pypy/jit/hintmodel.py pypy/dist/pypy/jit/test/test_hint_annotation.py Log: (arre, arigo, pedronis) started support for hint annotating containers. started with structures. Modified: pypy/dist/pypy/jit/hintannotator.py ============================================================================== --- pypy/dist/pypy/jit/hintannotator.py (original) +++ pypy/dist/pypy/jit/hintannotator.py Sat Jan 28 16:29:53 2006 @@ -2,11 +2,19 @@ from pypy.jit import hintmodel from pypy.jit.hintbookkeeper import HintBookkeeper + class HintAnnotator(RPythonAnnotator): def __init__(self): RPythonAnnotator.__init__(self) self.bookkeeper = HintBookkeeper() # XXX + def consider_op_malloc(self, hs_TYPE): + TYPE = hs_TYPE.const + vstructdef = self.bookkeeper.getvirtualstructdef(TYPE) + return hintmodel.SomeLLAbstractContainer(vstructdef) + + + _registeroperations(HintAnnotator.__dict__, hintmodel) Modified: pypy/dist/pypy/jit/hintbookkeeper.py ============================================================================== --- pypy/dist/pypy/jit/hintbookkeeper.py (original) +++ pypy/dist/pypy/jit/hintbookkeeper.py Sat Jan 28 16:29:53 2006 @@ -7,6 +7,7 @@ def __init__(self): self.pending_specializations = [] self.origins = {} + self.virtual_containers = {} def enter(self, position_key): """Start of an operation. @@ -37,6 +38,17 @@ res.const = const.value return res + def getvirtualstructdef(self, TYPE): + from pypy.jit.hintcontainer import VirtualStructDef + try: + res = self.virtual_containers[self.position_key] + assert isinstance(res, VirtualStructDef) + assert res.T == TYPE + except KeyError: + res = VirtualStructDef(self, TYPE) + self.virtual_containers[self.position_key] = res + return res + # get current bookkeeper def getbookkeeper(): Added: pypy/dist/pypy/jit/hintcontainer.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/jit/hintcontainer.py Sat Jan 28 16:29:53 2006 @@ -0,0 +1,48 @@ +from pypy.annotation.listdef import ListItem +from pypy.jit import hintmodel + +class FieldValue(ListItem): + + def __init__(self, bookkeeper, name, hs_value): + ListItem.__init__(self, bookkeeper, hs_value) + self.name = name + + def patch(self): + for vstructdef in self.itemof: + vstructdef.fields[self.name] = self + + +class VirtualStructDef: + + def __init__(self, bookkeeper, TYPE): + self.T = TYPE + self.bookkeeper = bookkeeper + self.fields = {} + self.names = TYPE._names + for name in self.names: + hs = hintmodel.SomeLLAbstractConstant(self.fieldtype(name), {}) + fv = self.fields[name] = FieldValue(bookkeeper, name, hs) + fv.itemof[self] = True + + def fieldtype(self, name): + return getattr(self.T, name) + + def read_field(self, name): + fv = self.fields[name] + fv.read_locations[self.bookkeeper.position_key] = True + return fv.s_value + + def same_as(self, other): + return self.fields == other.fields + + def union(self, other): + assert self.T == other.T + for name in self.names: + self.fields[name].merge(other.fields[name]) + return self + + def generalize_field(self, name, hs_value): + self.fields[name].generalize(hs_value) + + def __repr__(self): + return "" % (self.T._name,) Modified: pypy/dist/pypy/jit/hintmodel.py ============================================================================== --- pypy/dist/pypy/jit/hintmodel.py (original) +++ pypy/dist/pypy/jit/hintmodel.py Sat Jan 28 16:29:53 2006 @@ -3,7 +3,7 @@ from pypy.jit.hintbookkeeper import getbookkeeper from pypy.rpython.lltypesystem import lltype -UNARY_OPERATIONS = "same_as hint getfield".split() +UNARY_OPERATIONS = "same_as hint getfield setfield".split() BINARY_OPERATIONS = "int_add int_sub int_mul int_gt int_eq".split() @@ -46,6 +46,12 @@ class SomeLLAbstractVariable(SomeLLAbstractValue): pass +class SomeLLAbstractContainer(SomeLLAbstractValue): + + def __init__(self, contentdef): + self.contentdef = contentdef + self.concretetype = lltype.Ptr(contentdef.T) + # ____________________________________________________________ # operations @@ -75,6 +81,15 @@ else: return SomeLLAbstractVariable(FIELD_TYPE) +class __extend__(SomeLLAbstractContainer): + + def setfield(hs_s1, hs_fieldname, hs_value): + hs_s1.contentdef.generalize_field(hs_fieldname.const, hs_value) + + def getfield(hs_s1, hs_fieldname): + return hs_s1.contentdef.read_field(hs_fieldname.const) + + class __extend__(pairtype(SomeLLAbstractValue, SomeLLAbstractValue)): def int_add((hs_v1, hs_v2)): Modified: pypy/dist/pypy/jit/test/test_hint_annotation.py ============================================================================== --- pypy/dist/pypy/jit/test/test_hint_annotation.py (original) +++ pypy/dist/pypy/jit/test/test_hint_annotation.py Sat Jan 28 16:29:53 2006 @@ -169,6 +169,22 @@ assert len(hs.origins) == 1 assert len(hs.origins.keys()[0].origins) == 2 +def test_simple_struct_malloc(): + S = lltype.GcStruct('helloworld', ('hello', lltype.Signed), + ('world', lltype.Signed)) + def ll_function(x): + s = lltype.malloc(S) + s.hello = x + return s.hello + s.world + + hs = hannotate(ll_function, [int]) + assert isinstance(hs, SomeLLAbstractConstant) + assert hs.concretetype == lltype.Signed + assert len(hs.origins) == 1 + assert len(hs.origins.keys()[0].origins) == 1 + + + From pedronis at codespeak.net Sat Jan 28 16:51:50 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Sat, 28 Jan 2006 16:51:50 +0100 (CET) Subject: [pypy-svn] r22797 - in pypy/dist/pypy/jit: . test Message-ID: <20060128155150.672DD27DCB@code1.codespeak.net> Author: pedronis Date: Sat Jan 28 16:51:47 2006 New Revision: 22797 Modified: pypy/dist/pypy/jit/hintannotator.py pypy/dist/pypy/jit/hintbookkeeper.py pypy/dist/pypy/jit/hintmodel.py pypy/dist/pypy/jit/test/test_hint_annotation.py Log: (arre, arigo, pedronis) union of abstract containers, structures for now. Modified: pypy/dist/pypy/jit/hintannotator.py ============================================================================== --- pypy/dist/pypy/jit/hintannotator.py (original) +++ pypy/dist/pypy/jit/hintannotator.py Sat Jan 28 16:51:47 2006 @@ -7,7 +7,7 @@ def __init__(self): RPythonAnnotator.__init__(self) - self.bookkeeper = HintBookkeeper() # XXX + self.bookkeeper = HintBookkeeper(self) # XXX def consider_op_malloc(self, hs_TYPE): TYPE = hs_TYPE.const Modified: pypy/dist/pypy/jit/hintbookkeeper.py ============================================================================== --- pypy/dist/pypy/jit/hintbookkeeper.py (original) +++ pypy/dist/pypy/jit/hintbookkeeper.py Sat Jan 28 16:51:47 2006 @@ -4,10 +4,11 @@ class HintBookkeeper(object): - def __init__(self): + def __init__(self, hannotator): self.pending_specializations = [] self.origins = {} self.virtual_containers = {} + self.annotator = hannotator def enter(self, position_key): """Start of an operation. Modified: pypy/dist/pypy/jit/hintmodel.py ============================================================================== --- pypy/dist/pypy/jit/hintmodel.py (original) +++ pypy/dist/pypy/jit/hintmodel.py Sat Jan 28 16:51:47 2006 @@ -135,3 +135,8 @@ def int_eq((hs_c1, hs_c2)): return SomeLLConcreteValue(lltype.Bool) + +class __extend__(pairtype(SomeLLAbstractContainer, SomeLLAbstractContainer)): + + def union((hs_cont1, hs_cont2)): + return SomeLLAbstractContainer(hs_cont1.contentdef.union(hs_cont2.contentdef)) Modified: pypy/dist/pypy/jit/test/test_hint_annotation.py ============================================================================== --- pypy/dist/pypy/jit/test/test_hint_annotation.py (original) +++ pypy/dist/pypy/jit/test/test_hint_annotation.py Sat Jan 28 16:51:47 2006 @@ -118,7 +118,7 @@ def test_op_meet(): def meet(hs1, hs2): - HintBookkeeper().enter(None) + HintBookkeeper(None).enter(None) return pair(hs1, hs2).int_add() av1, av2 = SomeLLAbstractVariable(lltype.Signed), SomeLLAbstractVariable(lltype.Signed) cv1, cv2 = SomeLLConcreteValue(lltype.Signed), SomeLLConcreteValue(lltype.Signed) @@ -183,6 +183,29 @@ assert len(hs.origins) == 1 assert len(hs.origins.keys()[0].origins) == 1 +def test_container_union(): + S = lltype.GcStruct('helloworld', ('hello', lltype.Signed), + ('world', lltype.Signed)) + def ll_function(cond, x, y): + if cond: + s = lltype.malloc(S) + s.hello = x + else: + s = lltype.malloc(S) + s.world = y + return s.hello + s.world + + hs = hannotate(ll_function, [bool, int, int]) + assert isinstance(hs, SomeLLAbstractConstant) + assert hs.concretetype == lltype.Signed + assert len(hs.origins) == 1 + assert len(hs.origins.keys()[0].origins) == 2 + + + + + + From arigo at codespeak.net Sat Jan 28 17:44:12 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sat, 28 Jan 2006 17:44:12 +0100 (CET) Subject: [pypy-svn] r22798 - pypy/dist/pypy/jit/test Message-ID: <20060128164412.81F7027DCD@code1.codespeak.net> Author: arigo Date: Sat Jan 28 17:44:10 2006 New Revision: 22798 Modified: pypy/dist/pypy/jit/test/test_llabstractinterp.py Log: (pedronis, arre, arigo) * 2.3 compatibility * added a test to check what llabstractinterp really does in corner cases Modified: pypy/dist/pypy/jit/test/test_llabstractinterp.py ============================================================================== --- pypy/dist/pypy/jit/test/test_llabstractinterp.py (original) +++ pypy/dist/pypy/jit/test/test_llabstractinterp.py Sat Jan 28 17:44:10 2006 @@ -60,7 +60,7 @@ # return a summary of the instructions left in all the residual graphs insns = {} graphs = [graph] - found = set((graph,)) + found = {graph: True} while graphs: graph = graphs.pop() for block in graph.iterblocks(): @@ -78,7 +78,7 @@ else: if graph not in found: graphs.append(graph) - found.add(graph) + found[graph] = True return insns P_INLINE = Policy(inlining=True) @@ -410,3 +410,28 @@ graph2, insns = abstrinterp(ll_interp, [bytecode], [0], policy=P_HINT_DRIVEN) assert insns == {'int_add': 4, 'int_lt': 1} + +def test_conditional_origin(): + from pypy.rpython.objectmodel import hint + def ll_function(x, y, variable): + result = 0 + i = 0 + while i < 10: + if i: + z = x + else: + z = y + z = hint(z, concrete=True) + if z == 42: + result += variable + i += 1 + return result + graph2, insns = abstrinterp(ll_function, [42, 71298, -12], [0, 1], + policy=P_HINT_DRIVEN) + # the result is not really specified. In theory, the hint() call could + # fix 'i', because 'z' depends on 'i'. Then we'd get: + # assert insns == {'int_add': 9} + # + # But llabstractinterp doesn't track this particular dependency for now, + # so what we get is: + assert insns == {'int_lt': 1, 'int_is_true': 1, 'int_add': 2} From cfbolz at codespeak.net Sat Jan 28 18:02:39 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Sat, 28 Jan 2006 18:02:39 +0100 (CET) Subject: [pypy-svn] r22801 - in pypy/dist/pypy/rpython/memory: . test Message-ID: <20060128170239.E1D1227DCB@code1.codespeak.net> Author: cfbolz Date: Sat Jan 28 18:02:37 2006 New Revision: 22801 Modified: pypy/dist/pypy/rpython/memory/gctransform.py pypy/dist/pypy/rpython/memory/test/test_gctransform.py Log: (mwh, cfbolz): we finally, maybe, possibly got the issues around exception handling correct. Modified: pypy/dist/pypy/rpython/memory/gctransform.py ============================================================================== --- pypy/dist/pypy/rpython/memory/gctransform.py (original) +++ pypy/dist/pypy/rpython/memory/gctransform.py Sat Jan 28 18:02:37 2006 @@ -109,12 +109,15 @@ newops.extend(self.pop_alive(var)) for link in block.exits: livecounts = dict.fromkeys(sets.Set(livevars) - deadinallexits, 1) - if (block.exitswitch is c_last_exception and link.exitcase is not None - and livevars and livevars[-1] is block.operations[-1].result): - # if the last operation in the block raised an - # exception, it can't have returned anything that - # might need pop_aliving. - del livecounts[livevars[-1]] + if (block.exitswitch is c_last_exception and + link.exitcase is not None): + if livevars and livevars[-1] is block.operations[-1].result: + # if the last operation in the block raised an + # exception, it can't have returned anything that + # might need pop_aliving. + del livecounts[livevars[-1]] + if link.last_exc_value not in link.args: + livecounts[link.last_exc_value] = 1 for v in link.args: if v in livecounts: livecounts[v] -= 1 Modified: pypy/dist/pypy/rpython/memory/test/test_gctransform.py ============================================================================== --- pypy/dist/pypy/rpython/memory/test/test_gctransform.py (original) +++ pypy/dist/pypy/rpython/memory/test/test_gctransform.py Sat Jan 28 18:02:37 2006 @@ -28,15 +28,16 @@ return for link in block.exits: fudge = 0 - if (block.exitswitch is c_last_exception and link.exitcase is not None - and var_needsgc(block.operations[-1].result)): - fudge = 1 + if (block.exitswitch is c_last_exception and link.exitcase is not None): + fudge -= 1 + if var_needsgc(block.operations[-1].result): + fudge += 1 refs_out = len([v for v in link.args if isinstance(v, Variable) and var_needsgc(v)]) - if link.last_exception is not None and link.last_exception in link.args: - refs_out -= 1 assert refs_in + push_alives + calls - fudge == pop_alives + refs_out - + + if block.exitswitch is c_last_exception and link.exitcase is not None: + assert link.last_exc_value in link.args def rtype_and_transform(func, inputtypes, transformcls, specialize=True): t = TranslationContext() From tismer at codespeak.net Sat Jan 28 18:10:02 2006 From: tismer at codespeak.net (tismer at codespeak.net) Date: Sat, 28 Jan 2006 18:10:02 +0100 (CET) Subject: [pypy-svn] r22802 - pypy/dist/pypy/module/stackless Message-ID: <20060128171002.ECDF227DD7@code1.codespeak.net> Author: tismer Date: Sat Jan 28 18:10:00 2006 New Revision: 22802 Modified: pypy/dist/pypy/module/stackless/interp_coroutine.py Log: there seems to be no need for something like a main coroutine Modified: pypy/dist/pypy/module/stackless/interp_coroutine.py ============================================================================== --- pypy/dist/pypy/module/stackless/interp_coroutine.py (original) +++ pypy/dist/pypy/module/stackless/interp_coroutine.py Sat Jan 28 18:10:00 2006 @@ -14,7 +14,7 @@ class CoState(object): def __init__(self): - self.last = self.current = self.main = Coroutine() + self.last = self.current = Coroutine() self.things_to_do = False self.temp_exc = None self.del_first = None @@ -38,7 +38,7 @@ if costate is None: self.parent = self else: - self.parent = costate.main + self.parent = costate.current self.thunk = None def bind(self, thunk): @@ -82,7 +82,9 @@ _update_state = staticmethod(_update_state) def kill(self): - if costate.current is self: +# if costate.current is self: + # raise CoroutineExit + if self.frame is None: raise CoroutineExit costate.things_to_do = True costate.temp_exc = CoroutineExit() @@ -118,10 +120,6 @@ return costate.current get_current = staticmethod(get_current) - def get_main(): - return costate.main - get_main = staticmethod(get_main) - def check_for_zombie(self): if costate.del_first is not None: @@ -197,6 +195,7 @@ raise OperationError(space.w_ValueError, space.wrap( "cannot bind a bound Coroutine")) thunk = _AppThunk(space, w_func, __args__) + costate.current = appcostate.current self.bind(thunk) def w_switch(self): @@ -204,6 +203,7 @@ if self.frame is None: raise OperationError(space.w_ValueError, space.wrap( "cannot switch to an unbound Coroutine")) + costate.current = appcostate.current self.switch() appcostate.current = self ret, appcostate.tempval = appcostate.tempval, space.w_None @@ -229,9 +229,6 @@ return space.wrap(appcostate.current) get_current = staticmethod(get_current) - def get_main(space): - return space.wrap(appcostate.main) - get_main = staticmethod(get_main) # _mixin_ did not work for methname in StacklessFlags.__dict__: @@ -254,9 +251,8 @@ """) def post_install(module): - appcostate.main.space = module.space + appcostate.post_install(module.space) makeStaticMethod(module, 'Coroutine', 'get_current') - makeStaticMethod(module, 'Coroutine', 'get_main') # space.appexec("""() : @@ -270,12 +266,16 @@ kill = interp2app(AppCoroutine.w_kill), is_zombie = GetSetProperty(AppCoroutine.w_get_is_zombie, doc=AppCoroutine.get_is_zombie.__doc__), get_current = interp2app(AppCoroutine.get_current), - get_main = interp2app(AppCoroutine.get_main), ) class AppCoState(object): def __init__(self): - self.current = self.main = AppCoroutine() + self.current = AppCoroutine() + + def post_install(self, space): + appcostate.current.space = space + appcostate.tempval = space.w_None + appcostate = AppCoState() @@ -300,10 +300,4 @@ a new current. After a greenlet's switch, greenlet gets a new current. -This concept has a collision, because there is also -some notation of a "main". The solution is to let -every exposed class create its own main in advance. -This is nice, because the different concepts -can completely co-exist. - """ \ No newline at end of file From pedronis at codespeak.net Sat Jan 28 18:46:51 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Sat, 28 Jan 2006 18:46:51 +0100 (CET) Subject: [pypy-svn] r22803 - pypy/extradoc/sprintinfo/mallorca Message-ID: <20060128174651.4E83527DD0@code1.codespeak.net> Author: pedronis Date: Sat Jan 28 18:46:49 2006 New Revision: 22803 Added: pypy/extradoc/sprintinfo/mallorca/post-mallorca-planning.txt (contents, props changed) Log: planning for between now and PyCon. Added: pypy/extradoc/sprintinfo/mallorca/post-mallorca-planning.txt ============================================================================== --- (empty file) +++ pypy/extradoc/sprintinfo/mallorca/post-mallorca-planning.txt Sat Jan 28 18:46:49 2006 @@ -0,0 +1,36 @@ +=========================================== +Rough planning between Mallorca and PyCon +=========================================== + +Preparation for PyCon +====================== + +- sprint announcement (soon, i.e. by Friday 3rd Feb) +- updated tutorial +- talks + +JIT +====== + +- more JIT work :) + +GC integration +===================== + +- continue Michael and Carl's genc-vs-gc refactoring +- use and improve GCs written with the framework + +Concurrency +================== + +- finish high-level interfaces for greenlets and tasklets (nearly done?) +- open: free threading +- open: performance with GIL threading +- open: tasklet pickling + +Compiling double use (PyPy+CPython) extensions +======================================================== + +- annotation and compilation of such double use extension modules +- rctypes + From pedronis at codespeak.net Sat Jan 28 19:10:21 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Sat, 28 Jan 2006 19:10:21 +0100 (CET) Subject: [pypy-svn] r22805 - pypy/dist/pypy/translator/tool Message-ID: <20060128181021.CA3F327DD0@code1.codespeak.net> Author: pedronis Date: Sat Jan 28 19:10:19 2006 New Revision: 22805 Modified: pypy/dist/pypy/translator/tool/graphpage.py Log: some support for showing hint annotated graphs Modified: pypy/dist/pypy/translator/tool/graphpage.py ============================================================================== --- pypy/dist/pypy/translator/tool/graphpage.py (original) +++ pypy/dist/pypy/translator/tool/graphpage.py Sat Jan 28 19:10:19 2006 @@ -165,6 +165,12 @@ #info = '(%s) %s' % (var.concretetype, info) info = str(var.concretetype) self.links[var.name] = info + + from pypy.jit import hintannotator + + if isinstance(self.annotator, hintannotator.HintAnnotator): + return + for graph in graphs: traverse(visit, graph) From pedronis at codespeak.net Sat Jan 28 19:17:12 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Sat, 28 Jan 2006 19:17:12 +0100 (CET) Subject: [pypy-svn] r22806 - in pypy/dist/pypy/jit: . test Message-ID: <20060128181712.E9AB827DD0@code1.codespeak.net> Author: pedronis Date: Sat Jan 28 19:17:10 2006 New Revision: 22806 Modified: pypy/dist/pypy/jit/hintmodel.py pypy/dist/pypy/jit/test/test_hint_annotation.py Log: fix. Modified: pypy/dist/pypy/jit/hintmodel.py ============================================================================== --- pypy/dist/pypy/jit/hintmodel.py (original) +++ pypy/dist/pypy/jit/hintmodel.py Sat Jan 28 19:17:10 2006 @@ -28,11 +28,15 @@ seen[o] = True for o1 in o.visit(seen): yield o1 + + def __repr__(self): + return "O" + (self.fixed and "f" or "") class SomeLLAbstractValue(annmodel.SomeObject): def __init__(self, T): self.concretetype = T + assert self.__class__ != SomeLLAbstractValue class SomeLLAbstractConstant(SomeLLAbstractValue): @@ -64,7 +68,7 @@ def hint(hs_c1, hs_flags): if hs_flags.const.get('variable', False): # only for testing purposes!!! - return SomeLLAbstractValue(hs_c1.concretetype) + return SomeLLAbstractVariable(hs_c1.concretetype) assert hs_flags.const['concrete'] for o in hs_c1.origins: for o1 in o.visit(): Modified: pypy/dist/pypy/jit/test/test_hint_annotation.py ============================================================================== --- pypy/dist/pypy/jit/test/test_hint_annotation.py (original) +++ pypy/dist/pypy/jit/test/test_hint_annotation.py Sat Jan 28 19:17:10 2006 @@ -20,6 +20,8 @@ hs = hannotator.build_graph_types(graph1, [SomeLLAbstractConstant(v.concretetype, {OriginTreeNode(): True}) for v in graph1.getargs()]) + #hannotator.translator.graphs.append(graph1) + #hannotator.translator.view() return hs def test_simple(): @@ -200,7 +202,9 @@ assert hs.concretetype == lltype.Signed assert len(hs.origins) == 1 assert len(hs.origins.keys()[0].origins) == 2 - + + + From pedronis at codespeak.net Sat Jan 28 19:49:10 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Sat, 28 Jan 2006 19:49:10 +0100 (CET) Subject: [pypy-svn] r22807 - in pypy/dist/pypy/jit: . test Message-ID: <20060128184910.F41FB27DD9@code1.codespeak.net> Author: pedronis Date: Sat Jan 28 19:49:08 2006 New Revision: 22807 Modified: pypy/dist/pypy/jit/hintmodel.py pypy/dist/pypy/jit/test/test_hint_annotation.py Log: (arre, arigo, pedronis) hint annotate simple direct_call. Modified: pypy/dist/pypy/jit/hintmodel.py ============================================================================== --- pypy/dist/pypy/jit/hintmodel.py (original) +++ pypy/dist/pypy/jit/hintmodel.py Sat Jan 28 19:49:08 2006 @@ -3,7 +3,7 @@ from pypy.jit.hintbookkeeper import getbookkeeper from pypy.rpython.lltypesystem import lltype -UNARY_OPERATIONS = "same_as hint getfield setfield".split() +UNARY_OPERATIONS = "same_as hint getfield setfield direct_call".split() BINARY_OPERATIONS = "int_add int_sub int_mul int_gt int_eq".split() @@ -38,12 +38,20 @@ self.concretetype = T assert self.__class__ != SomeLLAbstractValue + def reorigin(self, bookkeeper): + return self + class SomeLLAbstractConstant(SomeLLAbstractValue): def __init__(self, T, origins): SomeLLAbstractValue.__init__(self, T) self.origins = origins + def reorigin(self, bookkeeper): + origin = bookkeeper.myorigin() + origin.merge(self.origins) + return SomeLLAbstractConstant(self.concretetype, {origin: True}) + class SomeLLConcreteValue(SomeLLAbstractValue): pass @@ -85,6 +93,15 @@ else: return SomeLLAbstractVariable(FIELD_TYPE) + def direct_call(hs_f1, *args_hs): + bookkeeper = getbookkeeper() + graph = hs_f1.const._obj.graph + hs_res = bookkeeper.annotator.recursivecall(graph, bookkeeper.position_key, args_hs) + if isinstance(hs_res, SomeLLAbstractValue): + return hs_res.reorigin(bookkeeper) + else: + return hs_res # impossible value + class __extend__(SomeLLAbstractContainer): def setfield(hs_s1, hs_fieldname, hs_value): Modified: pypy/dist/pypy/jit/test/test_hint_annotation.py ============================================================================== --- pypy/dist/pypy/jit/test/test_hint_annotation.py (original) +++ pypy/dist/pypy/jit/test/test_hint_annotation.py Sat Jan 28 19:49:08 2006 @@ -203,10 +203,21 @@ assert len(hs.origins) == 1 assert len(hs.origins.keys()[0].origins) == 2 +def test_simple_call(): + def ll2(x, y): + return x + (y + 42) + def ll1(x, y, z): + return ll2(x, y - z) + hs = hannotate(ll1, [int, int, int]) + assert isinstance(hs, SomeLLAbstractConstant) + assert hs.concretetype == lltype.Signed + assert len(hs.origins) == 1 +def CUR_GOAL_test_hannotate_tl(): + from pypy.jit import tl - + hannotate(tl.interp, [str, int]) From arigo at codespeak.net Sat Jan 28 20:09:10 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sat, 28 Jan 2006 20:09:10 +0100 (CET) Subject: [pypy-svn] r22808 - in pypy/dist/pypy/jit: . test Message-ID: <20060128190910.3E2CA27DDA@code1.codespeak.net> Author: arigo Date: Sat Jan 28 20:09:07 2006 New Revision: 22808 Modified: pypy/dist/pypy/jit/hintcontainer.py pypy/dist/pypy/jit/hintmodel.py pypy/dist/pypy/jit/test/test_hint_annotation.py Log: (pedronis, arre around, arigo) Some more operations here and there, and substructures. Modified: pypy/dist/pypy/jit/hintcontainer.py ============================================================================== --- pypy/dist/pypy/jit/hintcontainer.py (original) +++ pypy/dist/pypy/jit/hintcontainer.py Sat Jan 28 20:09:07 2006 @@ -1,5 +1,6 @@ from pypy.annotation.listdef import ListItem from pypy.jit import hintmodel +from pypy.rpython.lltypesystem import lltype class FieldValue(ListItem): @@ -20,7 +21,13 @@ self.fields = {} self.names = TYPE._names for name in self.names: - hs = hintmodel.SomeLLAbstractConstant(self.fieldtype(name), {}) + FIELD_TYPE = self.fieldtype(name) + if isinstance(FIELD_TYPE, lltype.ContainerType): + assert isinstance(FIELD_TYPE, lltype.Struct) # for now + vstructdef = VirtualStructDef(bookkeeper, FIELD_TYPE) + hs = hintmodel.SomeLLAbstractContainer(vstructdef) + else: + hs = hintmodel.SomeLLAbstractConstant(FIELD_TYPE, {}) fv = self.fields[name] = FieldValue(bookkeeper, name, hs) fv.itemof[self] = True Modified: pypy/dist/pypy/jit/hintmodel.py ============================================================================== --- pypy/dist/pypy/jit/hintmodel.py (original) +++ pypy/dist/pypy/jit/hintmodel.py Sat Jan 28 20:09:07 2006 @@ -3,7 +3,8 @@ from pypy.jit.hintbookkeeper import getbookkeeper from pypy.rpython.lltypesystem import lltype -UNARY_OPERATIONS = "same_as hint getfield setfield direct_call".split() +UNARY_OPERATIONS = """same_as hint getfield setfield getsubstruct getarraysize + direct_call""".split() BINARY_OPERATIONS = "int_add int_sub int_mul int_gt int_eq".split() @@ -93,6 +94,18 @@ else: return SomeLLAbstractVariable(FIELD_TYPE) + def getsubstruct(hs_c1, hs_fieldname): + S = hs_c1.concretetype.TO + SUB_TYPE = getattr(S, hs_fieldname.const) + origin = getbookkeeper().myorigin() + origin.merge(hs_c1.origins) + return SomeLLAbstractConstant(lltype.Ptr(SUB_TYPE), {origin: True}) + + def getarraysize(hs_c1): + origin = getbookkeeper().myorigin() + origin.merge(hs_c1.origins) + return SomeLLAbstractConstant(lltype.Signed, {origin: True}) + def direct_call(hs_f1, *args_hs): bookkeeper = getbookkeeper() graph = hs_f1.const._obj.graph @@ -110,6 +123,8 @@ def getfield(hs_s1, hs_fieldname): return hs_s1.contentdef.read_field(hs_fieldname.const) + getsubstruct = getfield + class __extend__(pairtype(SomeLLAbstractValue, SomeLLAbstractValue)): Modified: pypy/dist/pypy/jit/test/test_hint_annotation.py ============================================================================== --- pypy/dist/pypy/jit/test/test_hint_annotation.py (original) +++ pypy/dist/pypy/jit/test/test_hint_annotation.py Sat Jan 28 20:09:07 2006 @@ -218,12 +218,3 @@ from pypy.jit import tl hannotate(tl.interp, [str, int]) - - - - - - - - - From arigo at codespeak.net Sat Jan 28 20:30:51 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sat, 28 Jan 2006 20:30:51 +0100 (CET) Subject: [pypy-svn] r22810 - pypy/dist/pypy/jit Message-ID: <20060128193051.5F55D27DD8@code1.codespeak.net> Author: arigo Date: Sat Jan 28 20:30:48 2006 New Revision: 22810 Modified: pypy/dist/pypy/jit/hintannotator.py pypy/dist/pypy/jit/hintbookkeeper.py pypy/dist/pypy/jit/hintcontainer.py Log: (pedronis, arre around, arigo) More random progress, more thinking needed. Modified: pypy/dist/pypy/jit/hintannotator.py ============================================================================== --- pypy/dist/pypy/jit/hintannotator.py (original) +++ pypy/dist/pypy/jit/hintannotator.py Sat Jan 28 20:30:48 2006 @@ -11,10 +11,13 @@ def consider_op_malloc(self, hs_TYPE): TYPE = hs_TYPE.const - vstructdef = self.bookkeeper.getvirtualstructdef(TYPE) + vstructdef = self.bookkeeper.getvirtualcontainerdef(TYPE) return hintmodel.SomeLLAbstractContainer(vstructdef) - - + + def consider_op_malloc_varsize(self, hs_TYPE, hs_length): + TYPE = hs_TYPE.const + vcontainerdef = self.bookkeeper.getvirtualcontainerdef(TYPE) + return hintmodel.SomeLLAbstractContainer(vcontainerdef) _registeroperations(HintAnnotator.__dict__, hintmodel) Modified: pypy/dist/pypy/jit/hintbookkeeper.py ============================================================================== --- pypy/dist/pypy/jit/hintbookkeeper.py (original) +++ pypy/dist/pypy/jit/hintbookkeeper.py Sat Jan 28 20:30:48 2006 @@ -39,14 +39,13 @@ res.const = const.value return res - def getvirtualstructdef(self, TYPE): - from pypy.jit.hintcontainer import VirtualStructDef + def getvirtualcontainerdef(self, TYPE): try: res = self.virtual_containers[self.position_key] - assert isinstance(res, VirtualStructDef) assert res.T == TYPE except KeyError: - res = VirtualStructDef(self, TYPE) + from pypy.jit.hintcontainer import virtualcontainerdef + res = virtualcontainerdef(self, TYPE) self.virtual_containers[self.position_key] = res return res Modified: pypy/dist/pypy/jit/hintcontainer.py ============================================================================== --- pypy/dist/pypy/jit/hintcontainer.py (original) +++ pypy/dist/pypy/jit/hintcontainer.py Sat Jan 28 20:30:48 2006 @@ -2,6 +2,27 @@ from pypy.jit import hintmodel from pypy.rpython.lltypesystem import lltype +def virtualcontainerdef(bookkeeper, T): + """Build and return a VirtualXxxDef() corresponding to a + freshly allocated virtual container. + """ + if isinstance(T, lltype.Struct): + cls = VirtualStructDef + elif isinstance(T, lltype.Array): + cls = VirtualArrayDef + else: + raise TypeError("unsupported container type %r" % (T,)) + return cls(bookkeeper, T) + +def make_item_annotation(bookkeeper, TYPE): + if isinstance(TYPE, lltype.ContainerType): + vdef = virtualcontainerdef(bookkeeper, TYPE) + return hintmodel.SomeLLAbstractContainer(vdef) + else: + return hintmodel.SomeLLAbstractConstant(TYPE, {}) + +# ____________________________________________________________ + class FieldValue(ListItem): def __init__(self, bookkeeper, name, hs_value): @@ -22,12 +43,7 @@ self.names = TYPE._names for name in self.names: FIELD_TYPE = self.fieldtype(name) - if isinstance(FIELD_TYPE, lltype.ContainerType): - assert isinstance(FIELD_TYPE, lltype.Struct) # for now - vstructdef = VirtualStructDef(bookkeeper, FIELD_TYPE) - hs = hintmodel.SomeLLAbstractContainer(vstructdef) - else: - hs = hintmodel.SomeLLAbstractConstant(FIELD_TYPE, {}) + hs = make_item_annotation(bookkeeper, FIELD_TYPE) fv = self.fields[name] = FieldValue(bookkeeper, name, hs) fv.itemof[self] = True @@ -53,3 +69,40 @@ def __repr__(self): return "" % (self.T._name,) + +# ____________________________________________________________ + + +class ArrayItem(ListItem): + + def patch(self): + for varraydef in self.itemof: + varraydef.arrayitem = self + + +class VirtualArrayDef: + + def __init__(self, bookkeeper, TYPE): + self.T = TYPE + self.bookkeeper = bookkeeper + hs = make_item_annotation(bookkeeper, TYPE.OF) + self.arrayitem = ArrayItem(bookkeeper, hs) + self.arrayitem.itemof[self] = True + + def read_item(self): + self.arrayitem.read_locations[self.bookkeeper.position_key] = True + return self.arrayitem.s_value + + def same_as(self, other): + return self.arrayitem is other.arrayitem + + def union(self, other): + assert self.T == other.T + self.arrayitem.merge(other.arrayitem) + return self + + def generalize_item(self, hs_value): + self.arrayitem.generalize(hs_value) + + def __repr__(self): + return "" % (self.T.OF,) From rxe at codespeak.net Sat Jan 28 22:08:41 2006 From: rxe at codespeak.net (rxe at codespeak.net) Date: Sat, 28 Jan 2006 22:08:41 +0100 (CET) Subject: [pypy-svn] r22811 - pypy/dist/pypy/translator/llvm Message-ID: <20060128210841.1AC5D27DDB@code1.codespeak.net> Author: rxe Date: Sat Jan 28 22:08:40 2006 New Revision: 22811 Modified: pypy/dist/pypy/translator/llvm/build_llvm_module.py Log: Since we do our own inlining, reducing the theshold gives better results. Modified: pypy/dist/pypy/translator/llvm/build_llvm_module.py ============================================================================== --- pypy/dist/pypy/translator/llvm/build_llvm_module.py (original) +++ pypy/dist/pypy/translator/llvm/build_llvm_module.py Sat Jan 28 22:08:40 2006 @@ -33,7 +33,7 @@ flags = os.popen("gccas /dev/null -o /dev/null -debug-pass=Arguments 2>&1").read()[17:-1].split() flags += "-globalopt -constmerge -ipsccp -deadargelim -inline -instcombine -scalarrepl -globalsmodref-aa -licm -load-vn -gcse -instcombine -simplifycfg -globaldce".split() -OPTIMIZATION_SWITCHES = " ".join(flags) +OPTIMIZATION_SWITCHES = " ".join(flags) + " -inline-threshold=100" def compile_module(module, source_files, object_files, library_files): open("%s_setup.py" % module, "w").write(str(py.code.Source( From rxe at codespeak.net Sat Jan 28 22:12:31 2006 From: rxe at codespeak.net (rxe at codespeak.net) Date: Sat, 28 Jan 2006 22:12:31 +0100 (CET) Subject: [pypy-svn] r22812 - pypy/dist/pypy/translator/goal Message-ID: <20060128211231.8DE5827DDC@code1.codespeak.net> Author: rxe Date: Sat Jan 28 22:12:29 2006 New Revision: 22812 Modified: pypy/dist/pypy/translator/goal/richards.py Log: tcchh - isinstance() is at applevel... think it is fairer test to not do applevel things for comparision against CPython. Last run with genllvm was at 5.2x slower. Modified: pypy/dist/pypy/translator/goal/richards.py ============================================================================== --- pypy/dist/pypy/translator/goal/richards.py (original) +++ pypy/dist/pypy/translator/goal/richards.py Sat Jan 28 22:12:29 2006 @@ -244,7 +244,7 @@ def fn(self,pkt,r): d = r - assert isinstance(d, DeviceTaskRec) + assert d.__class__ is DeviceTaskRec if pkt is None: pkt = d.pending if pkt is None: @@ -265,7 +265,7 @@ def fn(self,pkt,r): h = r - assert isinstance(h, HandlerTaskRec) + assert h.__class__ is HandlerTaskRec if pkt is not None: if pkt.kind == K_WORK: h.workInAdd(pkt) @@ -297,7 +297,7 @@ def fn(self,pkt,r): i = r - assert isinstance(i, IdleTaskRec) + assert i.__class__ is IdleTaskRec i.count -= 1 if i.count == 0: return self.hold() @@ -320,7 +320,7 @@ def fn(self,pkt,r): w = r - assert isinstance(w, WorkerTaskRec) + assert w.__class__ is WorkerTaskRec if pkt is None: return self.waitTask() From ludal at codespeak.net Sun Jan 29 02:21:23 2006 From: ludal at codespeak.net (ludal at codespeak.net) Date: Sun, 29 Jan 2006 02:21:23 +0100 (CET) Subject: [pypy-svn] r22813 - in pypy/branch/ast-experiments/pypy/interpreter/pyparser: . test Message-ID: <20060129012123.2465D27DBE@code1.codespeak.net> Author: ludal Date: Sun Jan 29 02:21:18 2006 New Revision: 22813 Added: pypy/branch/ast-experiments/pypy/interpreter/pyparser/test/test_parser.py Modified: pypy/branch/ast-experiments/pypy/interpreter/pyparser/astbuilder.py pypy/branch/ast-experiments/pypy/interpreter/pyparser/ebnfparse.py pypy/branch/ast-experiments/pypy/interpreter/pyparser/grammar.py pypy/branch/ast-experiments/pypy/interpreter/pyparser/pythonparse.py pypy/branch/ast-experiments/pypy/interpreter/pyparser/pythonutil.py pypy/branch/ast-experiments/pypy/interpreter/pyparser/test/test_astbuilder.py pypy/branch/ast-experiments/pypy/interpreter/pyparser/test/test_astcompiler.py pypy/branch/ast-experiments/pypy/interpreter/pyparser/test/test_lookahead.py pypy/branch/ast-experiments/pypy/interpreter/pyparser/test/test_pytokenizer.py pypy/branch/ast-experiments/pypy/interpreter/pyparser/tuplebuilder.py Log: refactoring III most tests pass. a problem remains with TupleBuilder messing on import xxx Modified: pypy/branch/ast-experiments/pypy/interpreter/pyparser/astbuilder.py ============================================================================== --- pypy/branch/ast-experiments/pypy/interpreter/pyparser/astbuilder.py (original) +++ pypy/branch/ast-experiments/pypy/interpreter/pyparser/astbuilder.py Sun Jan 29 02:21:18 2006 @@ -713,7 +713,7 @@ # 'is', 'is not', 'not' or 'not in' => tok.get_value() token = atoms[i] assert isinstance(token, TokenObject) - op_name = tok.tok_rpunct.get(token.name, token.get_value()) + op_name = tok.tok_rvalues.get(token.name, token.get_value()) ops.append((op_name, atoms[i+1])) builder.push(ast.Compare(atoms[0], ops, atoms[0].lineno)) @@ -1544,7 +1544,7 @@ self.lineno = lineno def get_name(self): - return tok.tok_rpunct.get(self.name, + return tok.tok_rvalues.get(self.name, tok.tok_name.get(self.name, str(self.name))) def get_value(self): Modified: pypy/branch/ast-experiments/pypy/interpreter/pyparser/ebnfparse.py ============================================================================== --- pypy/branch/ast-experiments/pypy/interpreter/pyparser/ebnfparse.py (original) +++ pypy/branch/ast-experiments/pypy/interpreter/pyparser/ebnfparse.py Sun Jan 29 02:21:18 2006 @@ -74,7 +74,7 @@ if tk.codename == self.codename: if tk.value not in self.keywords: ret = builder.token( tk.codename, tk.value, source ) - return self.debug_return( ret, tk.codename, tk.value ) + return ret source.restore( ctx ) return 0 @@ -83,7 +83,7 @@ """ if not isinstance(other, Token): raise RuntimeError("Unexpected token type") - if other is EmptyToken: + if other is self.parser.EmptyToken: return False if other.codename != self.codename: return False @@ -135,7 +135,7 @@ # If we still have a GrammarProxy associated to this codename # this means we have encountered a terminal symbol to_be_deleted[ arg.codename ] = True - rule.args[i] = Token( self.parser, arg.codename ) + rule.args[i] = self.get_token( arg.codename ) #print arg, "-> Token(",arg.rule_name,")" else: #print arg, "->", real_rule @@ -143,19 +143,20 @@ for codename in to_be_deleted.keys(): del self.parser.root_rules[codename] -## def get_token(self, codename ): -## """Returns a new or existing Token""" -## if codename in self.tokens: -## return self.tokens[codename] -## token = self.tokens[codename] = self.parser.Token(codename) -## return token + def get_token(self, codename ): + """Returns a new or existing Token""" + if codename in self.tokens: + return self.tokens[codename] + token = self.tokens[codename] = self.parser.Token(codename) + return token def get_symbolcode(self, name ): return self.parser.add_symbol( name ) def get_rule( self, name ): if name in self.parser.tokens: - return self.parser.Token_n( name ) + codename = self.parser.tokens[name] + return self.get_token( codename ) codename = self.get_symbolcode( name ) if codename in self.parser.root_rules: return self.parser.root_rules[codename] Modified: pypy/branch/ast-experiments/pypy/interpreter/pyparser/grammar.py ============================================================================== --- pypy/branch/ast-experiments/pypy/interpreter/pyparser/grammar.py (original) +++ pypy/branch/ast-experiments/pypy/interpreter/pyparser/grammar.py Sun Jan 29 02:21:18 2006 @@ -734,6 +734,7 @@ self.EmptyToken = Token( self, -1, None ) self.tok_name = {} self.tok_values = {} + self.tok_rvalues = {} self._ann_sym_count = -10 self._sym_count = 0 self.all_rules = [] @@ -775,6 +776,9 @@ self.tok_name[val] = tok if value is not None: self.tok_values[value] = val + # XXX : this reverse mapping seemed only to be used + # because of pycodegen visitAugAssign + self.tok_rvalues[val] = value return val return self.tokens[ tok ] @@ -857,3 +861,14 @@ assert value is None or isinstance( value, str) tok = Token( self, name_id, value ) return tok + + + # Debugging functions + def show_rules(self, name): + import re + rex = re.compile(name) + rules =[] + for _name, _val in self.symbols.items(): + if rex.search(_name) and _val>=0: + rules.append(self.root_rules[_val]) + return rules Modified: pypy/branch/ast-experiments/pypy/interpreter/pyparser/pythonparse.py ============================================================================== --- pypy/branch/ast-experiments/pypy/interpreter/pyparser/pythonparse.py (original) +++ pypy/branch/ast-experiments/pypy/interpreter/pyparser/pythonparse.py Sun Jan 29 02:21:18 2006 @@ -5,7 +5,6 @@ helper functions are provided that use the grammar to parse using file_input, single_input and eval_input targets """ -import autopath import sys import os from pypy.interpreter.error import OperationError, debug_print Modified: pypy/branch/ast-experiments/pypy/interpreter/pyparser/pythonutil.py ============================================================================== --- pypy/branch/ast-experiments/pypy/interpreter/pyparser/pythonutil.py (original) +++ pypy/branch/ast-experiments/pypy/interpreter/pyparser/pythonutil.py Sun Jan 29 02:21:18 2006 @@ -61,7 +61,7 @@ tuples (StackElement is only a wrapper class around these tuples) """ - builder = TupleBuilder(pysymbol._cpython_symbols, PYTHON_PARSER.rules, lineno=False) + builder = TupleBuilder(PYTHON_PARSER, lineno=False) if space is not None: builder.space = space target_rule = TARGET_DICT[mode] @@ -109,6 +109,8 @@ etc. This is to be fixed in a clean way """ tuples = pypy_parse(input, mode, True) + if 'import' in input: + toto ast = transformer.compile_node(tuples) return ast Modified: pypy/branch/ast-experiments/pypy/interpreter/pyparser/test/test_astbuilder.py ============================================================================== --- pypy/branch/ast-experiments/pypy/interpreter/pyparser/test/test_astbuilder.py (original) +++ pypy/branch/ast-experiments/pypy/interpreter/pyparser/test/test_astbuilder.py Sun Jan 29 02:21:18 2006 @@ -1,6 +1,6 @@ import os -from pypy.interpreter.pyparser.pythonparse import PYTHON_PARSER_DYN as PYTHON_PARSER +from pypy.interpreter.pyparser.pythonparse import PYTHON_PARSER from pypy.interpreter.pyparser.astbuilder import AstBuilder from pypy.interpreter.pyparser.pythonutil import ast_from_input from pypy.interpreter.stablecompiler.transformer import Transformer Modified: pypy/branch/ast-experiments/pypy/interpreter/pyparser/test/test_astcompiler.py ============================================================================== --- pypy/branch/ast-experiments/pypy/interpreter/pyparser/test/test_astcompiler.py (original) +++ pypy/branch/ast-experiments/pypy/interpreter/pyparser/test/test_astcompiler.py Sun Jan 29 02:21:18 2006 @@ -3,7 +3,6 @@ from pypy.interpreter.pyparser.astbuilder import AstBuilder from pypy.interpreter.pyparser.tuplebuilder import TupleBuilder from pypy.interpreter.pycode import PyCode -from pypy.interpreter.pyparser.pysymbol import _cpython_symbols import py.test def setup_module(mod): @@ -83,7 +82,7 @@ def compile_with_testcompiler(expr, target='exec', space=FakeSpace()): target2 = TARGET_DICT['exec'] # xxx exec: single not really tested - builder = TupleBuilder(_cpython_symbols) + builder = TupleBuilder(PYTHON_PARSER) PYTHON_PARSER.parse_source(expr, target2, builder) tuples = builder.stack[-1].as_tuple(True) from pypy.interpreter.stablecompiler import transformer, pycodegen, misc Modified: pypy/branch/ast-experiments/pypy/interpreter/pyparser/test/test_lookahead.py ============================================================================== --- pypy/branch/ast-experiments/pypy/interpreter/pyparser/test/test_lookahead.py (original) +++ pypy/branch/ast-experiments/pypy/interpreter/pyparser/test/test_lookahead.py Sun Jan 29 02:21:18 2006 @@ -1,43 +1,37 @@ from pypy.interpreter.pyparser.grammar import Alternative, Sequence, KleeneStar, \ - Token, EmptyToken, build_first_sets + Token, Parser class TestLookAheadBasics: def setup_method(self, method): - self.count = 0 - self.tok1 = Token(self.nextid(), 'foo') - self.tok2 = Token(self.nextid(), 'bar') - self.tok3 = Token(self.nextid(), 'foobar') + self.parser = Parser() + self.tok1 = self.parser.Token_n("t1", 'foo') + self.tok2 = self.parser.Token_n("t2", 'bar') + self.tok3 = self.parser.Token_n("t3", 'foobar') self.tokens = [self.tok1, self.tok2, self.tok3] - build_first_sets(self.tokens) - - def nextid(self): - self.count+=1 - return self.count + self.parser.build_first_sets() def test_basic_token(self): assert self.tok1.first_set == [self.tok1] - def test_basic_alternative(self): - alt = Alternative(self.nextid(), self.tokens) - build_first_sets([alt]) + alt = self.parser.Alternative_n("a1t", self.tokens) + self.parser.build_first_sets() assert alt.first_set == self.tokens def test_basic_sequence(self): - seq = Sequence(self.nextid(), self.tokens) - build_first_sets([seq]) + seq = self.parser.Sequence_n("seq", self.tokens) + self.parser.build_first_sets() assert seq.first_set == [self.tokens[0]] def test_basic_kleenstar(self): tok1, tok2, tok3 = self.tokens - kstar = KleeneStar(self.nextid(), 1, 3, tok1) - build_first_sets([kstar]) - assert kstar.first_set == [tok1] - kstar = KleeneStar(self.nextid(), 0, 3, tok1) - build_first_sets([kstar]) - assert kstar.first_set == [tok1, EmptyToken] + kstar1 = self.parser.KleeneStar_n("k", 1, 3, tok1) + kstar2 = self.parser.KleeneStar_n("k2", 0, 3, tok1) + self.parser.build_first_sets() + assert kstar1.first_set == [tok1] + assert kstar2.first_set == [tok1, self.parser.EmptyToken] def test_maybe_empty_sequence(self): @@ -45,11 +39,11 @@ ==> S.first_set = [tok1, tok2, EmptyToken] """ tok1, tok2, tok3 = self.tokens - k1 = KleeneStar(self.nextid(), 0, 2, tok1) - k2 = KleeneStar(self.nextid(), 0, 2, tok2) - seq = Sequence(self.nextid(), [k1, k2]) - build_first_sets([k1, k2, seq]) - assert seq.first_set == [tok1, tok2, EmptyToken] + k1 = self.parser.KleeneStar_n( "k1", 0, 2, tok1) + k2 = self.parser.KleeneStar_n("k2", 0, 2, tok2) + seq = self.parser.Sequence_n( "seq", [k1, k2]) + self.parser.build_first_sets() + assert seq.first_set == [tok1, tok2, self.parser.EmptyToken] def test_not_empty_sequence(self): @@ -57,41 +51,42 @@ ==> S.first_set = [tok1, tok2] """ tok1, tok2, tok3 = self.tokens - k1 = KleeneStar(self.nextid(), 0, 2, tok1) - k2 = KleeneStar(self.nextid(), 1, 2, tok2) - seq = Sequence(self.nextid(), [k1, k2]) - build_first_sets([k1, k2, seq]) + k1 = self.parser.KleeneStar_n("k1", 0, 2, tok1) + k2 = self.parser.KleeneStar_n("k2", 1, 2, tok2) + seq = self.parser.Sequence_n("seq", [k1, k2]) + self.parser.build_first_sets() assert seq.first_set == [tok1, tok2] -def test_token_comparison(): - assert Token(1, 'foo') == Token(1, 'foo') - assert Token(1, 'foo') != Token(2, 'foo') - assert Token(2, 'foo') != Token(2, None) + def test_token_comparison(self): + tok1 = self.parser.Token_n( "tok1", "foo" ) + tok1b = self.parser.Token_n( "tok1", "foo" ) + tok2 = self.parser.Token_n( "tok2", "foo" ) + tok3 = self.parser.Token_n( "tok2", None ) + assert tok1 == tok1b + assert tok1 != tok2 + assert tok2 != tok3 -LOW = 1 -CAP = 2 -R_A = 3 -R_B = 4 -R_C = 5 -R_k1 = 6 -R_k2 = 7 class TestLookAhead: def setup_method(self, method): - self.LOW = Token(LOW, 'low') - self.CAP = Token(CAP ,'cap') - self.A = Alternative(R_A, []) - k1 = KleeneStar(R_k1, 0, rule=self.LOW) - k2 = KleeneStar(R_k2, 0, rule=self.CAP) - self.B = Sequence(R_B, [k1, self.A]) - self.C = Sequence(R_C, [k2, self.A]) + p = self.parser = Parser() + self.LOW = p.Token_n( 'LOW', 'low') + self.CAP = p.Token_n( 'CAP' ,'cap') + self.A = p.Alternative_n( 'R_A', []) + k1 = p.KleeneStar_n( 'R_k1', 0, rule=self.LOW) + k2 = p.KleeneStar_n( 'R_k2', 0, rule=self.CAP) + self.B = p.Sequence_n( 'R_B', [k1, self.A]) + self.C = p.Sequence_n( 'R_C', [k2, self.A]) self.A.args = [self.B, self.C] - build_first_sets([self.A, self.B, self.C, self.LOW, self.CAP, k1, k2]) + p.build_first_sets() def test_S_first_set(self): - for s in [Token(LOW, 'low'), EmptyToken, Token(CAP, 'cap')]: + p = self.parser + LOW = p.tokens['LOW'] + CAP = p.tokens['CAP'] + for s in [Token(p, LOW, 'low'), p.EmptyToken, Token(p, CAP, 'cap')]: assert s in self.A.first_set assert s in self.B.first_set assert s in self.C.first_set Added: pypy/branch/ast-experiments/pypy/interpreter/pyparser/test/test_parser.py ============================================================================== --- (empty file) +++ pypy/branch/ast-experiments/pypy/interpreter/pyparser/test/test_parser.py Sun Jan 29 02:21:18 2006 @@ -0,0 +1,45 @@ + +from pypy.interpreter.pyparser.grammar import Parser + + + +def test_symbols(): + p = Parser() + x1 = p.add_symbol('sym') + x2 = p.add_token('tok') + x3 = p.add_anon_symbol(':sym') + x4 = p.add_anon_symbol(':sym1') + # test basic numbering assumption + # symbols and tokens are attributed sequentially + # using the same counter + assert x2 == x1 + 1 + # anon symbols have negative value + assert x3 != x2 + 1 + assert x4 == x3 - 1 + assert x3 < 0 + y1 = p.add_symbol('sym') + assert y1 == x1 + y2 = p.add_token('tok') + assert y2 == x2 + y3 = p.add_symbol(':sym') + assert y3 == x3 + y4 = p.add_symbol(':sym1') + assert y4 == x4 + + +def test_load(): + d = { 5 : 'sym1', + 6 : 'sym2', + 9 : 'sym3', + } + p = Parser() + p.load_symbols( d ) + v = p.add_symbol('sym4') + # check that we avoid numbering conflicts + assert v>9 + v = p.add_symbol( 'sym1' ) + assert v == 5 + v = p.add_symbol( 'sym2' ) + assert v == 6 + v = p.add_symbol( 'sym3' ) + assert v == 9 Modified: pypy/branch/ast-experiments/pypy/interpreter/pyparser/test/test_pytokenizer.py ============================================================================== --- pypy/branch/ast-experiments/pypy/interpreter/pyparser/test/test_pytokenizer.py (original) +++ pypy/branch/ast-experiments/pypy/interpreter/pyparser/test/test_pytokenizer.py Sun Jan 29 02:21:18 2006 @@ -1,17 +1,24 @@ from pypy.interpreter.pyparser.pythonlexer import Source, TokenError, \ match_encoding_declaration from pypy.interpreter.pyparser.grammar import Token, GrammarElement -from pypy.interpreter.pyparser.pytoken import EQUAL, ENDMARKER, LSQB, MINUS, NAME, NEWLINE, NULLTOKEN, NUMBER, RSQB, STRING - -from pypy.interpreter.pyparser.pytoken import tok_name, tok_punct -GrammarElement.symbols = tok_name +from pypy.interpreter.pyparser.pythonparse import PYTHON_PARSER as P +EQUAL = P.EQUAL +ENDMARKER = P.ENDMARKER +LSQB = P.LSQB +MINUS = P.MINUS +NAME = P.NAME +NEWLINE = P.NEWLINE +NULLTOKEN = P.NULLTOKEN +NUMBER = P.NUMBER +RSQB = P.RSQB +STRING = P.STRING def parse_source(source): """returns list of parsed tokens""" - lexer = Source(source.splitlines(True)) + lexer = Source( P, source.splitlines(True)) tokens = [] - last_token = Token(NULLTOKEN, None) + last_token = Token( P, NULLTOKEN, None) while last_token.codename != ENDMARKER: last_token = lexer.next() tokens.append(last_token) @@ -49,24 +56,24 @@ s = """['a' ]""" tokens = parse_source(s) - assert tokens[:4] == [Token(LSQB, None), Token(STRING, "'a'"), - Token(RSQB, None), Token(NEWLINE, '')] + assert tokens[:4] == [Token(P, LSQB, None), Token(P, STRING, "'a'"), + Token(P, RSQB, None), Token(P, NEWLINE, '')] def test_numbers(): """make sure all kind of numbers are correctly parsed""" for number in NUMBERS: - assert parse_source(number)[0] == Token(NUMBER, number) + assert parse_source(number)[0] == Token(P, NUMBER, number) neg = '-%s' % number - assert parse_source(neg)[:2] == [Token(MINUS, None), - Token(NUMBER, number)] + assert parse_source(neg)[:2] == [Token(P, MINUS, None), + Token(P, NUMBER, number)] for number in BAD_NUMBERS: - assert parse_source(number)[0] != Token(NUMBER, number) + assert parse_source(number)[0] != Token(P, NUMBER, number) def test_hex_number(): """basic pasrse""" tokens = parse_source("a = 0x12L") - assert tokens[:4] == [Token(NAME, 'a'), Token(EQUAL, None), - Token(NUMBER, '0x12L'), Token(NEWLINE, '')] + assert tokens[:4] == [Token(P, NAME, 'a'), Token(P, EQUAL, None), + Token(P, NUMBER, '0x12L'), Token(P, NEWLINE, '')] def test_punct(): """make sure each punctuation is correctly parsed""" @@ -81,7 +88,7 @@ tokens = [tok for tok, _, _, _ in error.token_stack] if prefix: tokens.pop(0) - assert tokens[0].codename == tok_punct[pstr] + assert tokens[0].codename == P.tok_values[pstr] def test_encoding_declarations_match(): Modified: pypy/branch/ast-experiments/pypy/interpreter/pyparser/tuplebuilder.py ============================================================================== --- pypy/branch/ast-experiments/pypy/interpreter/pyparser/tuplebuilder.py (original) +++ pypy/branch/ast-experiments/pypy/interpreter/pyparser/tuplebuilder.py Sun Jan 29 02:21:18 2006 @@ -1,7 +1,5 @@ -from grammar import AbstractBuilder, AbstractContext -from pytoken import tok_name, tok_rpunct, NEWLINE, INDENT, DEDENT, ENDMARKER -import pysymbol +from grammar import AbstractBuilder, AbstractContext, Parser class StackElement: """wraps TupleBuilder's tuples""" @@ -58,12 +56,14 @@ class TupleBuilder(AbstractBuilder): """A builder that directly produce the AST""" - def __init__(self, symbols, rules=None, debug=0, lineno=True): - AbstractBuilder.__init__(self, symbols, rules, debug) + def __init__(self, parser, debug=0, lineno=True): + AbstractBuilder.__init__(self, parser, debug) # This attribute is here for convenience self.source_encoding = None self.lineno = lineno self.stack = [] + self.space_token = ( self.parser.NEWLINE, self.parser.INDENT, + self.parser.DEDENT, self.parser.ENDMARKER ) def context(self): """Returns the state of the builder to be restored later""" @@ -98,8 +98,8 @@ def token(self, codename, value, source): lineno = source._token_lnum if value is None: - if codename not in ( NEWLINE, INDENT, DEDENT, ENDMARKER ): - value = tok_rpunct.get(codename, "unknown op") + if codename not in self.space_token: + value = self.parser.tok_rvalues.get(codename, "unknown op") else: value = '' self.stack.append( Terminal(codename, value, lineno) ) From arigo at codespeak.net Sun Jan 29 10:55:46 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sun, 29 Jan 2006 10:55:46 +0100 (CET) Subject: [pypy-svn] r22816 - pypy/dist/pypy/jit Message-ID: <20060129095546.9E64427B6C@code1.codespeak.net> Author: arigo Date: Sun Jan 29 10:55:44 2006 New Revision: 22816 Modified: pypy/dist/pypy/jit/hintcontainer.py Log: (pedronis, arigo) Hack for default values of container fields: we store zeroes as default but not NULL pointers -- we use SomeImpossibleValue instead for them. Makes union's life faaaar easier. Modified: pypy/dist/pypy/jit/hintcontainer.py ============================================================================== --- pypy/dist/pypy/jit/hintcontainer.py (original) +++ pypy/dist/pypy/jit/hintcontainer.py Sun Jan 29 10:55:44 2006 @@ -1,4 +1,5 @@ from pypy.annotation.listdef import ListItem +from pypy.annotation import model as annmodel from pypy.jit import hintmodel from pypy.rpython.lltypesystem import lltype @@ -18,6 +19,8 @@ if isinstance(TYPE, lltype.ContainerType): vdef = virtualcontainerdef(bookkeeper, TYPE) return hintmodel.SomeLLAbstractContainer(vdef) + elif isinstance(TYPE, lltype.Ptr): + return annmodel.s_ImpossibleValue else: return hintmodel.SomeLLAbstractConstant(TYPE, {}) From pedronis at codespeak.net Sun Jan 29 11:23:13 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Sun, 29 Jan 2006 11:23:13 +0100 (CET) Subject: [pypy-svn] r22817 - pypy/dist/pypy/jit Message-ID: <20060129102313.9860B27B6C@code1.codespeak.net> Author: pedronis Date: Sun Jan 29 11:23:12 2006 New Revision: 22817 Modified: pypy/dist/pypy/jit/hintmodel.py Log: some more ops Modified: pypy/dist/pypy/jit/hintmodel.py ============================================================================== --- pypy/dist/pypy/jit/hintmodel.py (original) +++ pypy/dist/pypy/jit/hintmodel.py Sun Jan 29 11:23:12 2006 @@ -3,10 +3,10 @@ from pypy.jit.hintbookkeeper import getbookkeeper from pypy.rpython.lltypesystem import lltype -UNARY_OPERATIONS = """same_as hint getfield setfield getsubstruct getarraysize +UNARY_OPERATIONS = """same_as hint getfield setfield getsubstruct getarraysize getarrayitem direct_call""".split() -BINARY_OPERATIONS = "int_add int_sub int_mul int_gt int_eq".split() +BINARY_OPERATIONS = "int_add int_sub int_mul int_gt int_lt int_le int_ge int_eq int_ne".split() class OriginTreeNode(object): @@ -157,6 +157,8 @@ origin.merge(hs_c2.origins) return SomeLLAbstractConstant(lltype.Bool, {origin: True}) + int_lt = int_le = int_ge = int_eq = int_ne = int_gt + def union((hs_c1, hs_c2)): assert hs_c1.concretetype == hs_c2.concretetype origins = annmodel.setunion(hs_c1.origins, hs_c2.origins) From pedronis at codespeak.net Sun Jan 29 11:37:41 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Sun, 29 Jan 2006 11:37:41 +0100 (CET) Subject: [pypy-svn] r22818 - pypy/dist/pypy/jit Message-ID: <20060129103741.CF33C27B6C@code1.codespeak.net> Author: pedronis Date: Sun Jan 29 11:37:40 2006 New Revision: 22818 Modified: pypy/dist/pypy/jit/hintmodel.py Log: some more array ops. Modified: pypy/dist/pypy/jit/hintmodel.py ============================================================================== --- pypy/dist/pypy/jit/hintmodel.py (original) +++ pypy/dist/pypy/jit/hintmodel.py Sun Jan 29 11:37:40 2006 @@ -94,6 +94,16 @@ else: return SomeLLAbstractVariable(FIELD_TYPE) + def getarrayitem(hs_c1, hs_index): + A = hs_c1.concretetype.TO + READ_TYPE = A.OF + if A._hints.get('immutable', False): + origin = getbookkeeper().myorigin() + origin.merge(hs_c1.origins) + return SomeLLAbstractConstant(READ_TYPE, {origin: True}) + else: + return SomeLLAbstractVariable(READ_TYPE) + def getsubstruct(hs_c1, hs_fieldname): S = hs_c1.concretetype.TO SUB_TYPE = getattr(S, hs_fieldname.const) @@ -125,6 +135,8 @@ getsubstruct = getfield + def getarrayitem(hs_a1, hs_index): + return hs_a1.contentdef.read_item() class __extend__(pairtype(SomeLLAbstractValue, SomeLLAbstractValue)): From pedronis at codespeak.net Sun Jan 29 11:46:14 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Sun, 29 Jan 2006 11:46:14 +0100 (CET) Subject: [pypy-svn] r22819 - pypy/extradoc/sprintinfo/mallorca Message-ID: <20060129104614.6DAE927B70@code1.codespeak.net> Author: pedronis Date: Sun Jan 29 11:46:12 2006 New Revision: 22819 Modified: pypy/extradoc/sprintinfo/mallorca/planning.txt Log: planning for sunday. Modified: pypy/extradoc/sprintinfo/mallorca/planning.txt ============================================================================== --- pypy/extradoc/sprintinfo/mallorca/planning.txt (original) +++ pypy/extradoc/sprintinfo/mallorca/planning.txt Sun Jan 29 11:46:12 2006 @@ -33,7 +33,18 @@ Designed interface (see draft-jit-ideas), Refactored llabstractinterp to use the operations Started trying to reuse the annotator framework - for off-line partial evaluation: arre, armin, samuele + for off-line partial evaluation + Progress, now trying to annotate the toy language interp. + Arre, Samuele, Armin on Virtual List + + - integration of the garbage collection toolkit with the + DONE: offsetof, sizeof work done + Started: insert GC neeeded behaviour as graph transformations. + Writing a new function generator in genc for the transformed + graphs... Added cleanup attribute to operations. + Branched, genc working again with refcounting. Next: finishing + plus Boehm policy and likely merging. + Michael (Samuele) - stackless application level interface: Christian, started from the demo coroutine impl, trying to expose @@ -44,13 +55,6 @@ IN-PROGRESS: extending interface such that coroutines, tasklet, greenlets can co-exist peacefully. Now implementing greenlets... - - integration of the garbage collection toolkit with the - DONE: offsetof, sizeof work done - Started: insert GC neeeded behaviour as graph transformations. - Writing a new function generator in genc for the transformed - graphs... Added cleanup attribute to operations. - Carl, Michael - - improving the way we interface with external functions. (also think about how exceptions fit in there): add static ctypes functionality into RPython itself From ac at codespeak.net Sun Jan 29 11:47:33 2006 From: ac at codespeak.net (ac at codespeak.net) Date: Sun, 29 Jan 2006 11:47:33 +0100 (CET) Subject: [pypy-svn] r22820 - pypy/branch/arre-experiments Message-ID: <20060129104733.42A7627B6C@code1.codespeak.net> Author: ac Date: Sun Jan 29 11:47:33 2006 New Revision: 22820 Removed: pypy/branch/arre-experiments/ Log: Remove branch (no longer needed). From pedronis at codespeak.net Sun Jan 29 11:51:39 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Sun, 29 Jan 2006 11:51:39 +0100 (CET) Subject: [pypy-svn] r22821 - pypy/dist/pypy/jit Message-ID: <20060129105139.48DCF27B70@code1.codespeak.net> Author: pedronis Date: Sun Jan 29 11:51:37 2006 New Revision: 22821 Modified: pypy/dist/pypy/jit/hintmodel.py Log: (arre, pedronis) some more ops. Modified: pypy/dist/pypy/jit/hintmodel.py ============================================================================== --- pypy/dist/pypy/jit/hintmodel.py (original) +++ pypy/dist/pypy/jit/hintmodel.py Sun Jan 29 11:51:37 2006 @@ -4,9 +4,11 @@ from pypy.rpython.lltypesystem import lltype UNARY_OPERATIONS = """same_as hint getfield setfield getsubstruct getarraysize getarrayitem - direct_call""".split() + direct_call + cast_char_to_int""".split() -BINARY_OPERATIONS = "int_add int_sub int_mul int_gt int_lt int_le int_ge int_eq int_ne".split() +BINARY_OPERATIONS = """int_add int_sub int_mul int_and + int_gt int_lt int_le int_ge int_eq int_ne""".split() class OriginTreeNode(object): @@ -124,6 +126,11 @@ return hs_res.reorigin(bookkeeper) else: return hs_res # impossible value + + def cast_char_to_int(hs_c1): + origin = getbookkeeper().myorigin() + origin.merge(hs_c1.origins) + return SomeLLAbstractConstant(lltype.Signed, {origin: True}) class __extend__(SomeLLAbstractContainer): @@ -161,7 +168,7 @@ origin.merge(hs_c2.origins) return SomeLLAbstractConstant(lltype.Signed, {origin: True}) - int_mul = int_sub = int_add + int_and = int_mul = int_sub = int_add def int_gt((hs_c1, hs_c2)): origin = getbookkeeper().myorigin() From pedronis at codespeak.net Sun Jan 29 11:58:22 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Sun, 29 Jan 2006 11:58:22 +0100 (CET) Subject: [pypy-svn] r22822 - pypy/dist/pypy/jit Message-ID: <20060129105822.8F33327B70@code1.codespeak.net> Author: pedronis Date: Sun Jan 29 11:58:21 2006 New Revision: 22822 Modified: pypy/dist/pypy/jit/hintmodel.py Log: (arre, pedronis) more ops, also getarraysize on virtual arrays. Modified: pypy/dist/pypy/jit/hintmodel.py ============================================================================== --- pypy/dist/pypy/jit/hintmodel.py (original) +++ pypy/dist/pypy/jit/hintmodel.py Sun Jan 29 11:58:21 2006 @@ -5,6 +5,7 @@ UNARY_OPERATIONS = """same_as hint getfield setfield getsubstruct getarraysize getarrayitem direct_call + int_is_true int_neg cast_char_to_int""".split() BINARY_OPERATIONS = """int_add int_sub int_mul int_and @@ -127,10 +128,18 @@ else: return hs_res # impossible value - def cast_char_to_int(hs_c1): + def int_neg(hs_c1): origin = getbookkeeper().myorigin() origin.merge(hs_c1.origins) return SomeLLAbstractConstant(lltype.Signed, {origin: True}) + + cast_char_to_int = int_neg + + def int_is_true(hs_c1): + origin = getbookkeeper().myorigin() + origin.merge(hs_c1.origins) + return SomeLLAbstractConstant(lltype.Bool, {origin: True}) + class __extend__(SomeLLAbstractContainer): @@ -145,6 +154,10 @@ def getarrayitem(hs_a1, hs_index): return hs_a1.contentdef.read_item() + def getarraysize(hs_a1): + origin = getbookkeeper().myorigin() + return SomeLLAbstractConstant(lltype.Signed, {origin: True}) + class __extend__(pairtype(SomeLLAbstractValue, SomeLLAbstractValue)): def int_add((hs_v1, hs_v2)): From pedronis at codespeak.net Sun Jan 29 12:28:24 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Sun, 29 Jan 2006 12:28:24 +0100 (CET) Subject: [pypy-svn] r22823 - in pypy/dist/pypy/jit: . test Message-ID: <20060129112824.C72D527DB5@code1.codespeak.net> Author: pedronis Date: Sun Jan 29 12:28:21 2006 New Revision: 22823 Modified: pypy/dist/pypy/jit/hintbookkeeper.py pypy/dist/pypy/jit/hintcontainer.py pypy/dist/pypy/jit/hintmodel.py pypy/dist/pypy/jit/test/test_hint_annotation.py Log: (arre, pedronis) more ops. cast_pointer for virtual structs with simple test. added HintBookkeeper.current_op_concretetype Modified: pypy/dist/pypy/jit/hintbookkeeper.py ============================================================================== --- pypy/dist/pypy/jit/hintbookkeeper.py (original) +++ pypy/dist/pypy/jit/hintbookkeeper.py Sun Jan 29 12:28:21 2006 @@ -39,6 +39,11 @@ res.const = const.value return res + def current_op_concretetype(self): + _, block, i = self.position_key + op = block.operations[i] + return op.result.concretetype + def getvirtualcontainerdef(self, TYPE): try: res = self.virtual_containers[self.position_key] Modified: pypy/dist/pypy/jit/hintcontainer.py ============================================================================== --- pypy/dist/pypy/jit/hintcontainer.py (original) +++ pypy/dist/pypy/jit/hintcontainer.py Sun Jan 29 12:28:21 2006 @@ -3,21 +3,19 @@ from pypy.jit import hintmodel from pypy.rpython.lltypesystem import lltype -def virtualcontainerdef(bookkeeper, T): +def virtualcontainerdef(bookkeeper, T, vparent=None): """Build and return a VirtualXxxDef() corresponding to a freshly allocated virtual container. """ if isinstance(T, lltype.Struct): - cls = VirtualStructDef + return VirtualStructDef(bookkeeper, T, vparent) elif isinstance(T, lltype.Array): - cls = VirtualArrayDef - else: - raise TypeError("unsupported container type %r" % (T,)) - return cls(bookkeeper, T) + return VirtualArrayDef(bookkeeper, T) + raise TypeError("unsupported container type %r" % (T,)) -def make_item_annotation(bookkeeper, TYPE): +def make_item_annotation(bookkeeper, TYPE, vparent=None): if isinstance(TYPE, lltype.ContainerType): - vdef = virtualcontainerdef(bookkeeper, TYPE) + vdef = virtualcontainerdef(bookkeeper, TYPE, vparent=vparent) return hintmodel.SomeLLAbstractContainer(vdef) elif isinstance(TYPE, lltype.Ptr): return annmodel.s_ImpossibleValue @@ -39,16 +37,31 @@ class VirtualStructDef: - def __init__(self, bookkeeper, TYPE): + def __init__(self, bookkeeper, TYPE, vparent=None): self.T = TYPE self.bookkeeper = bookkeeper self.fields = {} self.names = TYPE._names for name in self.names: FIELD_TYPE = self.fieldtype(name) - hs = make_item_annotation(bookkeeper, FIELD_TYPE) + hs = make_item_annotation(bookkeeper, FIELD_TYPE, vparent=self) fv = self.fields[name] = FieldValue(bookkeeper, name, hs) fv.itemof[self] = True + self.vparent = vparent + + def cast(self, TO): + down_or_up = lltype.castable(TO, + lltype.Ptr(self.T)) + # the following works because if a structure is virtual, then + # all its parent and inlined substructures are also virtual + vstruct = self + if down_or_up >= 0: + for n in range(down_or_up): + vstruct = vstruct.read_field(vstruct.T._names[0]).contentdef + else: + for n in range(-down_or_up): + vstruct = vstruct.vparent + return vstruct def fieldtype(self, name): return getattr(self.T, name) Modified: pypy/dist/pypy/jit/hintmodel.py ============================================================================== --- pypy/dist/pypy/jit/hintmodel.py (original) +++ pypy/dist/pypy/jit/hintmodel.py Sun Jan 29 12:28:21 2006 @@ -3,12 +3,13 @@ from pypy.jit.hintbookkeeper import getbookkeeper from pypy.rpython.lltypesystem import lltype -UNARY_OPERATIONS = """same_as hint getfield setfield getsubstruct getarraysize getarrayitem +UNARY_OPERATIONS = """same_as hint getfield setfield getsubstruct getarraysize getarrayitem setarrayitem + cast_pointer direct_call int_is_true int_neg cast_char_to_int""".split() -BINARY_OPERATIONS = """int_add int_sub int_mul int_and +BINARY_OPERATIONS = """int_add int_sub int_mul int_and int_rshift int_gt int_lt int_le int_ge int_eq int_ne""".split() class OriginTreeNode(object): @@ -158,6 +159,11 @@ origin = getbookkeeper().myorigin() return SomeLLAbstractConstant(lltype.Signed, {origin: True}) + def cast_pointer(hs_s1): + TO = getbookkeeper().current_op_concretetype() + res_vstruct =hs_s1.contentdef.cast(TO) + return SomeLLAbstractContainer(res_vstruct) + class __extend__(pairtype(SomeLLAbstractValue, SomeLLAbstractValue)): def int_add((hs_v1, hs_v2)): @@ -181,7 +187,7 @@ origin.merge(hs_c2.origins) return SomeLLAbstractConstant(lltype.Signed, {origin: True}) - int_and = int_mul = int_sub = int_add + int_rshift = int_and = int_mul = int_sub = int_add def int_gt((hs_c1, hs_c2)): origin = getbookkeeper().myorigin() Modified: pypy/dist/pypy/jit/test/test_hint_annotation.py ============================================================================== --- pypy/dist/pypy/jit/test/test_hint_annotation.py (original) +++ pypy/dist/pypy/jit/test/test_hint_annotation.py Sun Jan 29 12:28:21 2006 @@ -213,6 +213,25 @@ assert hs.concretetype == lltype.Signed assert len(hs.origins) == 1 +def test_simple_cast_pointer(): + GCS1 = lltype.GcStruct('s1', ('x', lltype.Signed)) + GCS2 = lltype.GcStruct('s2', ('sub', GCS1), ('y', lltype.Signed)) + PGCS1 = lltype.Ptr(GCS1) + PGCS2 = lltype.Ptr(GCS2) + def ll1(): + s2 = lltype.malloc(GCS2) + return lltype.cast_pointer(PGCS1, s2) + hs = hannotate(ll1, []) + assert isinstance(hs, SomeLLAbstractContainer) + assert hs.concretetype == PGCS1 + def ll1(): + s2 = lltype.malloc(GCS2) + s1 = s2.sub + return lltype.cast_pointer(PGCS2, s1) + hs = hannotate(ll1, []) + assert isinstance(hs, SomeLLAbstractContainer) + assert hs.concretetype == PGCS2 + def CUR_GOAL_test_hannotate_tl(): from pypy.jit import tl From pedronis at codespeak.net Sun Jan 29 12:33:46 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Sun, 29 Jan 2006 12:33:46 +0100 (CET) Subject: [pypy-svn] r22824 - pypy/dist/pypy/jit Message-ID: <20060129113346.43B6227B70@code1.codespeak.net> Author: pedronis Date: Sun Jan 29 12:33:44 2006 New Revision: 22824 Modified: pypy/dist/pypy/jit/hintmodel.py Log: (arre, pedronis) some more ops Modified: pypy/dist/pypy/jit/hintmodel.py ============================================================================== --- pypy/dist/pypy/jit/hintmodel.py (original) +++ pypy/dist/pypy/jit/hintmodel.py Sun Jan 29 12:33:44 2006 @@ -7,9 +7,10 @@ cast_pointer direct_call int_is_true int_neg - cast_char_to_int""".split() + cast_char_to_int + cast_bool_to_int""".split() -BINARY_OPERATIONS = """int_add int_sub int_mul int_and int_rshift +BINARY_OPERATIONS = """int_add int_sub int_mul int_and int_rshift int_floordiv int_gt int_lt int_le int_ge int_eq int_ne""".split() class OriginTreeNode(object): @@ -134,7 +135,7 @@ origin.merge(hs_c1.origins) return SomeLLAbstractConstant(lltype.Signed, {origin: True}) - cast_char_to_int = int_neg + cast_bool_to_int = cast_char_to_int = int_neg def int_is_true(hs_c1): origin = getbookkeeper().myorigin() @@ -152,6 +153,9 @@ getsubstruct = getfield + def setarrayitem(hs_a1, hs_index, hs_value): + hs_a1.contentdef.generalize_item(hs_value) + def getarrayitem(hs_a1, hs_index): return hs_a1.contentdef.read_item() @@ -187,7 +191,7 @@ origin.merge(hs_c2.origins) return SomeLLAbstractConstant(lltype.Signed, {origin: True}) - int_rshift = int_and = int_mul = int_sub = int_add + int_floordiv = int_rshift = int_and = int_mul = int_sub = int_add def int_gt((hs_c1, hs_c2)): origin = getbookkeeper().myorigin() From pedronis at codespeak.net Sun Jan 29 12:41:12 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Sun, 29 Jan 2006 12:41:12 +0100 (CET) Subject: [pypy-svn] r22825 - pypy/dist/pypy/jit Message-ID: <20060129114112.A999F27B70@code1.codespeak.net> Author: pedronis Date: Sun Jan 29 12:41:11 2006 New Revision: 22825 Modified: pypy/dist/pypy/jit/hintmodel.py Log: (arre, pedronis) some more ops. Modified: pypy/dist/pypy/jit/hintmodel.py ============================================================================== --- pypy/dist/pypy/jit/hintmodel.py (original) +++ pypy/dist/pypy/jit/hintmodel.py Sun Jan 29 12:41:11 2006 @@ -7,6 +7,7 @@ cast_pointer direct_call int_is_true int_neg + cast_int_to_uint cast_char_to_int cast_bool_to_int""".split() @@ -141,8 +142,17 @@ origin = getbookkeeper().myorigin() origin.merge(hs_c1.origins) return SomeLLAbstractConstant(lltype.Bool, {origin: True}) + +class __extend__(SomeLLConcreteValue): + + def cast_int_to_uint(hs_cv1): + return SomeLLConcreteValue(lltype.Unsigned) + + def int_neg(hs_cv1): + return SomeLLConcreteValue(lltype.Signed) + + cast_bool_to_int = cast_char_to_int = int_neg - class __extend__(SomeLLAbstractContainer): def setfield(hs_s1, hs_fieldname, hs_value): @@ -168,6 +178,9 @@ res_vstruct =hs_s1.contentdef.cast(TO) return SomeLLAbstractContainer(res_vstruct) +# ____________________________________________________________ +# binary + class __extend__(pairtype(SomeLLAbstractValue, SomeLLAbstractValue)): def int_add((hs_v1, hs_v2)): @@ -193,13 +206,13 @@ int_floordiv = int_rshift = int_and = int_mul = int_sub = int_add - def int_gt((hs_c1, hs_c2)): + def int_eq((hs_c1, hs_c2)): origin = getbookkeeper().myorigin() origin.merge(hs_c1.origins) origin.merge(hs_c2.origins) return SomeLLAbstractConstant(lltype.Bool, {origin: True}) - int_lt = int_le = int_ge = int_eq = int_ne = int_gt + int_lt = int_le = int_ge = int_ne = int_gt = int_eq def union((hs_c1, hs_c2)): assert hs_c1.concretetype == hs_c2.concretetype @@ -213,9 +226,13 @@ def int_add((hs_c1, hs_c2)): return SomeLLConcreteValue(lltype.Signed) + int_floordiv = int_rshift = int_and = int_mul = int_sub = int_add + def int_eq((hs_c1, hs_c2)): return SomeLLConcreteValue(lltype.Bool) + int_lt = int_le = int_ge = int_ne = int_gt = int_eq + class __extend__(pairtype(SomeLLAbstractContainer, SomeLLAbstractContainer)): def union((hs_cont1, hs_cont2)): From pedronis at codespeak.net Sun Jan 29 12:46:37 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Sun, 29 Jan 2006 12:46:37 +0100 (CET) Subject: [pypy-svn] r22826 - pypy/dist/pypy/jit Message-ID: <20060129114637.10A5527B6C@code1.codespeak.net> Author: pedronis Date: Sun Jan 29 12:46:35 2006 New Revision: 22826 Modified: pypy/dist/pypy/jit/hintmodel.py Log: (arre, pedronis) some uint ops. Modified: pypy/dist/pypy/jit/hintmodel.py ============================================================================== --- pypy/dist/pypy/jit/hintmodel.py (original) +++ pypy/dist/pypy/jit/hintmodel.py Sun Jan 29 12:46:35 2006 @@ -7,12 +7,14 @@ cast_pointer direct_call int_is_true int_neg + uint_is_true cast_int_to_uint cast_char_to_int cast_bool_to_int""".split() BINARY_OPERATIONS = """int_add int_sub int_mul int_and int_rshift int_floordiv - int_gt int_lt int_le int_ge int_eq int_ne""".split() + int_gt int_lt int_le int_ge int_eq int_ne + uint_gt uint_lt uint_le uint_ge uint_eq uint_ne""".split() class OriginTreeNode(object): @@ -143,6 +145,8 @@ origin.merge(hs_c1.origins) return SomeLLAbstractConstant(lltype.Bool, {origin: True}) + uint_is_true = int_is_true + class __extend__(SomeLLConcreteValue): def cast_int_to_uint(hs_cv1): @@ -152,6 +156,11 @@ return SomeLLConcreteValue(lltype.Signed) cast_bool_to_int = cast_char_to_int = int_neg + + def int_is_true(hs_cv1): + return SomeLLConcreteValue(lltype.Bool) + + uint_is_true = int_is_true class __extend__(SomeLLAbstractContainer): @@ -213,6 +222,7 @@ return SomeLLAbstractConstant(lltype.Bool, {origin: True}) int_lt = int_le = int_ge = int_ne = int_gt = int_eq + uint_lt = uint_le = uint_ge = uint_ne = uint_gt = uint_eq = int_eq def union((hs_c1, hs_c2)): assert hs_c1.concretetype == hs_c2.concretetype @@ -232,6 +242,7 @@ return SomeLLConcreteValue(lltype.Bool) int_lt = int_le = int_ge = int_ne = int_gt = int_eq + uint_lt = uint_le = uint_ge = uint_ne = uint_gt = uint_eq = int_eq class __extend__(pairtype(SomeLLAbstractContainer, SomeLLAbstractContainer)): From arigo at codespeak.net Sun Jan 29 12:49:07 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sun, 29 Jan 2006 12:49:07 +0100 (CET) Subject: [pypy-svn] r22827 - pypy/dist/pypy/rpython Message-ID: <20060129114907.67CD027B4E@code1.codespeak.net> Author: arigo Date: Sun Jan 29 12:49:06 2006 New Revision: 22827 Modified: pypy/dist/pypy/rpython/rlist.py Log: Stick the type of the list item on the LIST GcStruct definition. This gives a generic way to write 'LIST.ITEM' that works for both fixed and non-fixed kinds of lists. Modified: pypy/dist/pypy/rpython/rlist.py ============================================================================== --- pypy/dist/pypy/rpython/rlist.py (original) +++ pypy/dist/pypy/rpython/rlist.py Sun Jan 29 12:49:06 2006 @@ -210,6 +210,7 @@ "ll_length": ll_length, "ll_items": ll_items, "list_builder": self.list_builder.build, + "ITEM": ITEM, }) ) @@ -283,6 +284,7 @@ "ll_length": ll_fixed_length, "ll_items": ll_fixed_items, "list_builder": self.list_builder.build, + "ITEM": ITEM, }) self.LIST.become(ITEMARRAY) @@ -487,7 +489,7 @@ p = new_allocated - 1 while p >= 0: newitems[p] = items[p] - ITEM = typeOf(l).TO.items.TO.OF + ITEM = typeOf(l).TO.ITEM if isinstance(ITEM, Ptr): items[p] = nullptr(ITEM.TO) p -= 1 @@ -591,7 +593,7 @@ newlength = index items = l.items res = items[index] - ITEM = typeOf(l).TO.items.TO.OF + ITEM = typeOf(l).TO.ITEM if isinstance(ITEM, Ptr): items[index] = nullptr(ITEM.TO) _ll_list_resize_le(l, newlength) @@ -611,7 +613,7 @@ items[j] = items[j1] j = j1 j1 += 1 - ITEM = typeOf(l).TO.items.TO.OF + ITEM = typeOf(l).TO.ITEM if isinstance(ITEM, Ptr): items[newlength] = nullptr(ITEM.TO) _ll_list_resize_le(l, newlength) @@ -684,7 +686,7 @@ items[j] = items[j1] j = j1 j1 += 1 - ITEM = typeOf(l).TO.items.TO.OF + ITEM = typeOf(l).TO.ITEM if isinstance(ITEM, Ptr): items[newlength] = nullptr(ITEM.TO) _ll_list_resize_le(l, newlength) @@ -779,7 +781,7 @@ def ll_listdelslice_startonly(l, start): newlength = start - ITEM = typeOf(l).TO.items.TO.OF + ITEM = typeOf(l).TO.ITEM if isinstance(ITEM, Ptr): j = l.length - 1 items = l.items @@ -801,7 +803,7 @@ items[j] = items[i] i += 1 j += 1 - ITEM = typeOf(l).TO.items.TO.OF + ITEM = typeOf(l).TO.ITEM if isinstance(ITEM, Ptr): j = l.length - 1 while j >= newlength: From arigo at codespeak.net Sun Jan 29 12:53:04 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sun, 29 Jan 2006 12:53:04 +0100 (CET) Subject: [pypy-svn] r22828 - in pypy/dist/pypy/jit: . test Message-ID: <20060129115304.5BE3427B6C@code1.codespeak.net> Author: arigo Date: Sun Jan 29 12:53:00 2006 New Revision: 22828 Added: pypy/dist/pypy/jit/hintvlist.py (contents, props changed) Modified: pypy/dist/pypy/jit/hintannotator.py pypy/dist/pypy/jit/hintbookkeeper.py pypy/dist/pypy/jit/hintcontainer.py pypy/dist/pypy/jit/hintmodel.py pypy/dist/pypy/jit/test/test_hint_annotation.py Log: Hint-annotator support for virtual lists. Modified: pypy/dist/pypy/jit/hintannotator.py ============================================================================== --- pypy/dist/pypy/jit/hintannotator.py (original) +++ pypy/dist/pypy/jit/hintannotator.py Sun Jan 29 12:53:00 2006 @@ -5,8 +5,8 @@ class HintAnnotator(RPythonAnnotator): - def __init__(self): - RPythonAnnotator.__init__(self) + def __init__(self, policy=None): + RPythonAnnotator.__init__(self, policy=policy) self.bookkeeper = HintBookkeeper(self) # XXX def consider_op_malloc(self, hs_TYPE): Modified: pypy/dist/pypy/jit/hintbookkeeper.py ============================================================================== --- pypy/dist/pypy/jit/hintbookkeeper.py (original) +++ pypy/dist/pypy/jit/hintbookkeeper.py Sun Jan 29 12:53:00 2006 @@ -44,16 +44,21 @@ op = block.operations[i] return op.result.concretetype - def getvirtualcontainerdef(self, TYPE): + def getvirtualcontainerdef(self, TYPE, constructor=None): try: res = self.virtual_containers[self.position_key] assert res.T == TYPE except KeyError: - from pypy.jit.hintcontainer import virtualcontainerdef - res = virtualcontainerdef(self, TYPE) + if constructor is None: + from pypy.jit.hintcontainer import virtualcontainerdef + constructor = virtualcontainerdef + res = constructor(self, TYPE) self.virtual_containers[self.position_key] = res return res - + + def warning(self, msg): + return self.annotator.warning(msg) + # get current bookkeeper def getbookkeeper(): Modified: pypy/dist/pypy/jit/hintcontainer.py ============================================================================== --- pypy/dist/pypy/jit/hintcontainer.py (original) +++ pypy/dist/pypy/jit/hintcontainer.py Sun Jan 29 12:53:00 2006 @@ -3,6 +3,15 @@ from pypy.jit import hintmodel from pypy.rpython.lltypesystem import lltype + +class AbstractContainerDef(object): + + def __init__(self, bookkeeper, TYPE): + self.T = TYPE + self.bookkeeper = bookkeeper + +# ____________________________________________________________ + def virtualcontainerdef(bookkeeper, T, vparent=None): """Build and return a VirtualXxxDef() corresponding to a freshly allocated virtual container. @@ -35,11 +44,10 @@ vstructdef.fields[self.name] = self -class VirtualStructDef: +class VirtualStructDef(AbstractContainerDef): def __init__(self, bookkeeper, TYPE, vparent=None): - self.T = TYPE - self.bookkeeper = bookkeeper + AbstractContainerDef.__init__(self, bookkeeper, TYPE) self.fields = {} self.names = TYPE._names for name in self.names: @@ -96,11 +104,10 @@ varraydef.arrayitem = self -class VirtualArrayDef: +class VirtualArrayDef(AbstractContainerDef): def __init__(self, bookkeeper, TYPE): - self.T = TYPE - self.bookkeeper = bookkeeper + AbstractContainerDef.__init__(self, bookkeeper, TYPE) hs = make_item_annotation(bookkeeper, TYPE.OF) self.arrayitem = ArrayItem(bookkeeper, hs) self.arrayitem.itemof[self] = True Modified: pypy/dist/pypy/jit/hintmodel.py ============================================================================== --- pypy/dist/pypy/jit/hintmodel.py (original) +++ pypy/dist/pypy/jit/hintmodel.py Sun Jan 29 12:53:00 2006 @@ -126,12 +126,25 @@ def direct_call(hs_f1, *args_hs): bookkeeper = getbookkeeper() - graph = hs_f1.const._obj.graph - hs_res = bookkeeper.annotator.recursivecall(graph, bookkeeper.position_key, args_hs) + fnobj = hs_f1.const._obj + if (getattr(bookkeeper.annotator.policy, 'oopspec', False) and + hasattr(fnobj._callable, 'oopspec')): + # try to handle the call as a high-level operation + try: + return handle_highlevel_operation(bookkeeper, fnobj._callable, + *args_hs) + except NotImplementedError: + pass + # normal call + if not hasattr(fnobj, 'graph'): + raise NotImplementedError("XXX call to externals or primitives") + hs_res = bookkeeper.annotator.recursivecall(fnobj.graph, + bookkeeper.position_key, + args_hs) if isinstance(hs_res, SomeLLAbstractValue): - return hs_res.reorigin(bookkeeper) - else: - return hs_res # impossible value + hs_res = hs_res.reorigin(bookkeeper) + #else: it's a SomeImpossibleValue + return hs_res def int_neg(hs_c1): origin = getbookkeeper().myorigin() @@ -248,3 +261,41 @@ def union((hs_cont1, hs_cont2)): return SomeLLAbstractContainer(hs_cont1.contentdef.union(hs_cont2.contentdef)) + +# ____________________________________________________________ + +def handle_highlevel_operation(bookkeeper, ll_func, *args_hs): + # parse the oopspec and fill in the arguments + operation_name, args = ll_func.oopspec.split('(', 1) + assert args.endswith(')') + args = args[:-1] + ',' # trailing comma to force tuple syntax + argnames = ll_func.func_code.co_varnames[:len(args_hs)] + d = dict(zip(argnames, args_hs)) + argtuple = eval(args, d) + args_hs = [] + for hs in argtuple: + if not isinstance(hs, SomeLLAbstractValue): + hs = bookkeeper.immutablevalue(hs) + args_hs.append(hs) + # end of rather XXX'edly hackish parsing + + if operation_name == 'newlist': + from pypy.jit.hintvlist import oop_newlist + handler = oop_newlist + else: + # dispatch on the 'self' argument if it is virtual + hs_self = args_hs[0] + args_hs = args_hs[1:] + type_name, operation_name = operation_name.split('.') + if not isinstance(hs_self, SomeLLAbstractContainer): + raise NotImplementedError + if getattr(hs_self.contentdef, 'type_name', None) != type_name: + raise NotImplementedError + try: + handler = getattr(hs_self.contentdef, 'oop_' + operation_name) + except AttributeError: + bookkeeper.warning('missing handler: oop_%s' % (operation_name,)) + raise NotImplementedError + + hs_result = handler(*args_hs) # which may raise NotImplementedError + return hs_result Added: pypy/dist/pypy/jit/hintvlist.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/jit/hintvlist.py Sun Jan 29 12:53:00 2006 @@ -0,0 +1,72 @@ +from pypy.annotation.listdef import ListItem +from pypy.jit import hintmodel +from pypy.jit.hintbookkeeper import getbookkeeper +from pypy.jit.hintcontainer import AbstractContainerDef, make_item_annotation +from pypy.rpython.lltypesystem import lltype + +class VirtualListDef(AbstractContainerDef): + type_name = 'list' + + def __init__(self, bookkeeper, LIST): + AbstractContainerDef.__init__(self, bookkeeper, LIST) + hs = make_item_annotation(bookkeeper, LIST.ITEM) + self.listitem = ListItem(bookkeeper, hs) + self.listitem.itemof[self] = True + + def read_item(self): + self.listitem.read_locations[self.bookkeeper.position_key] = True + return self.listitem.s_value + + def generalize_item(self, hs_value): + assert hs_value.concretetype == self.T.ITEM + self.listitem.generalize(hs_value) + + # ________________________________________ + # OOP high-level operations + + def oop_len(self): + origin = getbookkeeper().myorigin() + return hintmodel.SomeLLAbstractConstant(lltype.Signed, {origin: True}) + + def oop_nonzero(self): + origin = getbookkeeper().myorigin() + return hintmodel.SomeLLAbstractConstant(lltype.Bool, {origin: True}) + + def oop_getitem(self, hs_index): + assert hs_index.concretetype == lltype.Signed + return self.read_item() + + def oop_setitem(self, hs_index, hs_value): + assert hs_index.concretetype == lltype.Signed + self.generalize_item(hs_value) + + def oop_delitem(self, hs_index): + assert hs_index.concretetype == lltype.Signed + + def oop_append(self, hs_value): + self.generalize_item(hs_value) + + def oop_insert(self, hs_index, hs_value): + assert hs_index.concretetype == lltype.Signed + self.generalize_item(hs_value) + + def oop_pop(self, hs_index=None): + assert hs_index is None or hs_index.concretetype == lltype.Signed + return self.read_item() + + def oop_reverse(self): + pass + + def oop_copy(self): + XXX + + def oop_concat(self, hs_other): + XXX + +# ____________________________________________________________ + +def oop_newlist(hs_numitems, hs_item=None): + bk = getbookkeeper() + LIST = bk.current_op_concretetype().TO + vlistdef = bk.getvirtualcontainerdef(LIST, VirtualListDef) + return hintmodel.SomeLLAbstractContainer(vlistdef) Modified: pypy/dist/pypy/jit/test/test_hint_annotation.py ============================================================================== --- pypy/dist/pypy/jit/test/test_hint_annotation.py (original) +++ pypy/dist/pypy/jit/test/test_hint_annotation.py Sun Jan 29 12:53:00 2006 @@ -6,8 +6,12 @@ from pypy.rpython.lltypesystem import lltype from pypy.rpython.objectmodel import hint from pypy.annotation import model as annmodel +from pypy.annotation.policy import AnnotatorPolicy -def hannotate(func, argtypes): +P_OOPSPEC = AnnotatorPolicy() +P_OOPSPEC.oopspec = True + +def hannotate(func, argtypes, policy=None): # build the normal ll graphs for ll_function t = TranslationContext() a = t.buildannotator() @@ -16,7 +20,7 @@ rtyper.specialize() graph1 = graphof(t, func) # build hint annotator types - hannotator = HintAnnotator() + hannotator = HintAnnotator(policy=policy) hs = hannotator.build_graph_types(graph1, [SomeLLAbstractConstant(v.concretetype, {OriginTreeNode(): True}) for v in graph1.getargs()]) @@ -213,6 +217,16 @@ assert hs.concretetype == lltype.Signed assert len(hs.origins) == 1 +def test_simple_list_operations(): + def ll_function(x, y, index): + l = [x] + l.append(y) + return l[index] + hs = hannotate(ll_function, [int, int, int], policy=P_OOPSPEC) + assert isinstance(hs, SomeLLAbstractConstant) + assert hs.concretetype == lltype.Signed + assert len(hs.origins) == 2 + def test_simple_cast_pointer(): GCS1 = lltype.GcStruct('s1', ('x', lltype.Signed)) GCS2 = lltype.GcStruct('s2', ('sub', GCS1), ('y', lltype.Signed)) From pedronis at codespeak.net Sun Jan 29 12:56:12 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Sun, 29 Jan 2006 12:56:12 +0100 (CET) Subject: [pypy-svn] r22829 - pypy/dist/pypy/jit Message-ID: <20060129115612.91DE327B6C@code1.codespeak.net> Author: pedronis Date: Sun Jan 29 12:56:10 2006 New Revision: 22829 Modified: pypy/dist/pypy/jit/hintmodel.py Log: (arre, pedronis) more ops. Modified: pypy/dist/pypy/jit/hintmodel.py ============================================================================== --- pypy/dist/pypy/jit/hintmodel.py (original) +++ pypy/dist/pypy/jit/hintmodel.py Sun Jan 29 12:56:10 2006 @@ -8,11 +8,14 @@ direct_call int_is_true int_neg uint_is_true + cast_int_to_char cast_int_to_uint + cast_uint_to_int cast_char_to_int cast_bool_to_int""".split() -BINARY_OPERATIONS = """int_add int_sub int_mul int_and int_rshift int_floordiv +BINARY_OPERATIONS = """int_add int_sub int_mul int_mod int_and int_rshift int_floordiv + uint_add uint_sub uint_mul uint_mod uint_and uint_rshift uint_floordiv int_gt int_lt int_le int_ge int_eq int_ne uint_gt uint_lt uint_le uint_ge uint_eq uint_ne""".split() @@ -146,12 +149,19 @@ #else: it's a SomeImpossibleValue return hs_res - def int_neg(hs_c1): + def unary_char(hs_c1): + origin = getbookkeeper().myorigin() + origin.merge(hs_c1.origins) + return SomeLLAbstractConstant(lltype.Char, {origin: True}) + + cast_int_to_char = unary_char + + def unary_int(hs_c1): origin = getbookkeeper().myorigin() origin.merge(hs_c1.origins) return SomeLLAbstractConstant(lltype.Signed, {origin: True}) - cast_bool_to_int = cast_char_to_int = int_neg + cast_uint_to_int = cast_bool_to_int = cast_char_to_int = int_neg = unary_int def int_is_true(hs_c1): origin = getbookkeeper().myorigin() @@ -165,11 +175,16 @@ def cast_int_to_uint(hs_cv1): return SomeLLConcreteValue(lltype.Unsigned) - def int_neg(hs_cv1): + def unary_int(hs_cv1): return SomeLLConcreteValue(lltype.Signed) - cast_bool_to_int = cast_char_to_int = int_neg + cast_uint_to_int = cast_bool_to_int = cast_char_to_int = int_neg = unary_int + def unary_char(hs_c1): + return SomeLLConcreteValue(lltype.Char) + + cast_int_to_char = unary_char + def int_is_true(hs_cv1): return SomeLLConcreteValue(lltype.Bool) @@ -226,7 +241,15 @@ origin.merge(hs_c2.origins) return SomeLLAbstractConstant(lltype.Signed, {origin: True}) - int_floordiv = int_rshift = int_and = int_mul = int_sub = int_add + int_floordiv = int_rshift = int_and = int_mul = int_mod = int_sub = int_add + + def uint_add((hs_c1, hs_c2)): + origin = getbookkeeper().myorigin() + origin.merge(hs_c1.origins) + origin.merge(hs_c2.origins) + return SomeLLAbstractConstant(lltype.Unsigned, {origin: True}) + + uint_floordiv = uint_rshift = uint_and = uint_mul = uint_mod = uint_sub = uint_add def int_eq((hs_c1, hs_c2)): origin = getbookkeeper().myorigin() @@ -249,7 +272,12 @@ def int_add((hs_c1, hs_c2)): return SomeLLConcreteValue(lltype.Signed) - int_floordiv = int_rshift = int_and = int_mul = int_sub = int_add + int_floordiv = int_rshift = int_and = int_mul = int_mod = int_sub = int_add + + def uint_add((hs_c1, hs_c2)): + return SomeLLConcreteValue(lltype.Unsigned) + + uint_floordiv = uint_rshift = uint_and = uint_mul = uint_mod = uint_sub = uint_add def int_eq((hs_c1, hs_c2)): return SomeLLConcreteValue(lltype.Bool) From arigo at codespeak.net Sun Jan 29 13:09:55 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sun, 29 Jan 2006 13:09:55 +0100 (CET) Subject: [pypy-svn] r22830 - in pypy/dist/pypy/jit: . test Message-ID: <20060129120955.D765727B6C@code1.codespeak.net> Author: arigo Date: Sun Jan 29 13:09:54 2006 New Revision: 22830 Modified: pypy/dist/pypy/jit/hintvlist.py pypy/dist/pypy/jit/test/test_hint_annotation.py Log: List copy and concat. Now the hint annotator has as many list operations as the llabstractinterp. Modified: pypy/dist/pypy/jit/hintvlist.py ============================================================================== --- pypy/dist/pypy/jit/hintvlist.py (original) +++ pypy/dist/pypy/jit/hintvlist.py Sun Jan 29 13:09:54 2006 @@ -58,10 +58,15 @@ pass def oop_copy(self): - XXX + bk = self.bookkeeper + vlistdef = bk.getvirtualcontainerdef(self.T, VirtualListDef) + vlistdef.generalize_item(self.read_item()) + return hintmodel.SomeLLAbstractContainer(vlistdef) def oop_concat(self, hs_other): - XXX + assert isinstance(hs_other, hintmodel.SomeLLAbstractContainer) # for now + assert hs_other.contentdef.T == self.T + return self.oop_copy() # ____________________________________________________________ Modified: pypy/dist/pypy/jit/test/test_hint_annotation.py ============================================================================== --- pypy/dist/pypy/jit/test/test_hint_annotation.py (original) +++ pypy/dist/pypy/jit/test/test_hint_annotation.py Sun Jan 29 13:09:54 2006 @@ -227,6 +227,17 @@ assert hs.concretetype == lltype.Signed assert len(hs.origins) == 2 +def test_some_more_list_operations(): + def ll_function(x, y, index): + l = [] + l.append(x) + l[0] = y + return (l+list(l))[index] + hs = hannotate(ll_function, [int, int, int], policy=P_OOPSPEC) + assert isinstance(hs, SomeLLAbstractConstant) + assert hs.concretetype == lltype.Signed + assert len(hs.origins) == 2 + def test_simple_cast_pointer(): GCS1 = lltype.GcStruct('s1', ('x', lltype.Signed)) GCS2 = lltype.GcStruct('s2', ('sub', GCS1), ('y', lltype.Signed)) From pedronis at codespeak.net Sun Jan 29 13:32:12 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Sun, 29 Jan 2006 13:32:12 +0100 (CET) Subject: [pypy-svn] r22831 - in pypy/dist/pypy/jit: . test Message-ID: <20060129123212.B494227B6C@code1.codespeak.net> Author: pedronis Date: Sun Jan 29 13:32:10 2006 New Revision: 22831 Modified: pypy/dist/pypy/jit/hintcontainer.py pypy/dist/pypy/jit/hintmodel.py pypy/dist/pypy/jit/test/test_hint_annotation.py Log: (arre, pedronis) after some pushing and pulling we can annotate the tl interpreter. Let's look at the result now... Modified: pypy/dist/pypy/jit/hintcontainer.py ============================================================================== --- pypy/dist/pypy/jit/hintcontainer.py (original) +++ pypy/dist/pypy/jit/hintcontainer.py Sun Jan 29 13:32:10 2006 @@ -29,7 +29,9 @@ elif isinstance(TYPE, lltype.Ptr): return annmodel.s_ImpossibleValue else: - return hintmodel.SomeLLAbstractConstant(TYPE, {}) + hs_c = hintmodel.SomeLLAbstractConstant(TYPE, {}) + hs_c.const = TYPE._defl() + return hs_c # ____________________________________________________________ Modified: pypy/dist/pypy/jit/hintmodel.py ============================================================================== --- pypy/dist/pypy/jit/hintmodel.py (original) +++ pypy/dist/pypy/jit/hintmodel.py Sun Jan 29 13:32:10 2006 @@ -138,6 +138,9 @@ *args_hs) except NotImplementedError: pass + # don't try to annotate suggested_primitive graphs + if getattr(getattr(fnobj, '_callable', None), 'suggested_primitive', False): + return SomeLLAbstractVariable(lltype.typeOf(fnobj).RESULT) # normal call if not hasattr(fnobj, 'graph'): raise NotImplementedError("XXX call to externals or primitives") @@ -285,6 +288,16 @@ int_lt = int_le = int_ge = int_ne = int_gt = int_eq uint_lt = uint_le = uint_ge = uint_ne = uint_gt = uint_eq = int_eq +class __extend__(pairtype(SomeLLConcreteValue, SomeLLAbstractConstant), + pairtype(SomeLLAbstractConstant, SomeLLConcreteValue)): + + def union((hs_c1, hs_c2)): + assert hs_c1.concretetype == hs_c2.concretetype + #if hasattr(hs_c1, 'const') or hasattr(hs_c2, 'const'): + return SomeLLConcreteValue(hs_c1.concretetype) # MAYBE + #else: + # raise annmodel.UnionError("%s %s don't mix, unless the constant is constant" % (hs_c1, hs_c2)) + class __extend__(pairtype(SomeLLAbstractContainer, SomeLLAbstractContainer)): def union((hs_cont1, hs_cont2)): Modified: pypy/dist/pypy/jit/test/test_hint_annotation.py ============================================================================== --- pypy/dist/pypy/jit/test/test_hint_annotation.py (original) +++ pypy/dist/pypy/jit/test/test_hint_annotation.py Sun Jan 29 13:32:10 2006 @@ -114,8 +114,13 @@ # degenerating cases py.test.raises(annmodel.UnionError, "unionof(cv1, av1)") py.test.raises(annmodel.UnionError, "unionof(av1, cv1)") - py.test.raises(annmodel.UnionError, "unionof(ac1, cv1)") - py.test.raises(annmodel.UnionError, "unionof(cv1, ac1)") + + # MAYBE... + #py.test.raises(annmodel.UnionError, "unionof(ac1, cv1)") + #py.test.raises(annmodel.UnionError, "unionof(cv1, ac1)") + assert unionof(cv1, ac1) == cv1 + assert unionof(ac1, cv1) == cv1 + # constant with values assert unionof(av1, ac1) == av1 assert unionof(ac1, av1) == av1 @@ -258,7 +263,7 @@ assert hs.concretetype == PGCS2 -def CUR_GOAL_test_hannotate_tl(): +def test_hannotate_tl(): from pypy.jit import tl hannotate(tl.interp, [str, int]) From arigo at codespeak.net Sun Jan 29 13:37:06 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sun, 29 Jan 2006 13:37:06 +0100 (CET) Subject: [pypy-svn] r22832 - pypy/dist/pypy/jit Message-ID: <20060129123706.062DE27B6C@code1.codespeak.net> Author: arigo Date: Sun Jan 29 13:37:05 2006 New Revision: 22832 Modified: pypy/dist/pypy/jit/hintcontainer.py pypy/dist/pypy/jit/hintvlist.py Log: * Missing same_as()/union() in VirtualListDef. * A nicer __repr__ showing unifications of virtual containers. Modified: pypy/dist/pypy/jit/hintcontainer.py ============================================================================== --- pypy/dist/pypy/jit/hintcontainer.py (original) +++ pypy/dist/pypy/jit/hintcontainer.py Sun Jan 29 13:37:05 2006 @@ -1,3 +1,4 @@ +import weakref from pypy.annotation.listdef import ListItem from pypy.annotation import model as annmodel from pypy.jit import hintmodel @@ -5,10 +6,26 @@ class AbstractContainerDef(object): + __counter = 0 + __cache = {} def __init__(self, bookkeeper, TYPE): self.T = TYPE self.bookkeeper = bookkeeper + # hack to try to produce a repr that shows identifications + try: + weakdict = AbstractContainerDef.__cache[TYPE] + except KeyError: + weakdict = weakref.WeakValueDictionary() + AbstractContainerDef.__cache[self.__class__, TYPE] = weakdict + weakdict[AbstractContainerDef.__counter] = self + AbstractContainerDef.__counter += 1 + + def __repr__(self): + items = AbstractContainerDef.__cache[self.__class__, self.T].items() + keys = [key for key, containerdef in items if containerdef.same_as(self)] + tag = min(keys) + return "<%s #%d>" % (self.__class__.__name__, tag) # ____________________________________________________________ @@ -93,9 +110,6 @@ def generalize_field(self, name, hs_value): self.fields[name].generalize(hs_value) - def __repr__(self): - return "" % (self.T._name,) - # ____________________________________________________________ @@ -128,6 +142,3 @@ def generalize_item(self, hs_value): self.arrayitem.generalize(hs_value) - - def __repr__(self): - return "" % (self.T.OF,) Modified: pypy/dist/pypy/jit/hintvlist.py ============================================================================== --- pypy/dist/pypy/jit/hintvlist.py (original) +++ pypy/dist/pypy/jit/hintvlist.py Sun Jan 29 13:37:05 2006 @@ -17,6 +17,14 @@ self.listitem.read_locations[self.bookkeeper.position_key] = True return self.listitem.s_value + def same_as(self, other): + return self.listitem == other.listitem + + def union(self, other): + assert self.T == other.T + self.listitem.merge(other.listitem) + return self + def generalize_item(self, hs_value): assert hs_value.concretetype == self.T.ITEM self.listitem.generalize(hs_value) From ac at codespeak.net Sun Jan 29 13:47:19 2006 From: ac at codespeak.net (ac at codespeak.net) Date: Sun, 29 Jan 2006 13:47:19 +0100 (CET) Subject: [pypy-svn] r22833 - pypy/branch/jit-refactoring Message-ID: <20060129124719.078E127B6C@code1.codespeak.net> Author: ac Date: Sun Jan 29 13:47:18 2006 New Revision: 22833 Removed: pypy/branch/jit-refactoring/ Log: Remove ended branch. From mwh at codespeak.net Sun Jan 29 13:50:33 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Sun, 29 Jan 2006 13:50:33 +0100 (CET) Subject: [pypy-svn] r22834 - in pypy/dist/pypy/rpython/memory: . test Message-ID: <20060129125033.BB4D027B82@code1.codespeak.net> Author: mwh Date: Sun Jan 29 13:50:31 2006 New Revision: 22834 Modified: pypy/dist/pypy/rpython/memory/gctransform.py pypy/dist/pypy/rpython/memory/test/test_gctransform.py Log: surprise surprise, carl and i didn't get all the details of refcounting right yesterday -- Constants that need GC handling need to be push_alived before sending them across link. Modified: pypy/dist/pypy/rpython/memory/gctransform.py ============================================================================== --- pypy/dist/pypy/rpython/memory/gctransform.py (original) +++ pypy/dist/pypy/rpython/memory/gctransform.py Sun Jan 29 13:50:31 2006 @@ -1,5 +1,5 @@ from pypy.rpython.lltypesystem import lltype -from pypy.objspace.flow.model import SpaceOperation, Variable, c_last_exception +from pypy.objspace.flow.model import SpaceOperation, Variable, Constant, c_last_exception from pypy.translator.unsimplify import insert_empty_block from pypy.rpython import rmodel import sets @@ -116,11 +116,13 @@ # exception, it can't have returned anything that # might need pop_aliving. del livecounts[livevars[-1]] - if link.last_exc_value not in link.args: - livecounts[link.last_exc_value] = 1 + livecounts[link.last_exc_value] = 1 for v in link.args: if v in livecounts: livecounts[v] -= 1 + elif var_needsgc(v): + assert isinstance(v, Constant) + livecounts[v] = -1 self.links_to_split[link] = livecounts if newops: block.operations = newops Modified: pypy/dist/pypy/rpython/memory/test/test_gctransform.py ============================================================================== --- pypy/dist/pypy/rpython/memory/test/test_gctransform.py (original) +++ pypy/dist/pypy/rpython/memory/test/test_gctransform.py Sun Jan 29 13:50:31 2006 @@ -1,5 +1,5 @@ from pypy.rpython.memory import gctransform -from pypy.objspace.flow.model import c_last_exception +from pypy.objspace.flow.model import c_last_exception, Variable from pypy.rpython.memory.gctransform import var_needsgc, var_ispyobj from pypy.translator.translator import TranslationContext, graphof from pypy.rpython.lltypesystem import lltype @@ -13,7 +13,7 @@ if block.isstartblock: refs_in = 0 else: - refs_in = len([v for v in block.inputargs if var_needsgc(v)]) + refs_in = len([v for v in block.inputargs if isinstance(v, Variable) and var_needsgc(v)]) push_alives = len([op for op in block.operations if op.opname.startswith('gc_push_alive')]) + \ len([op for op in block.operations @@ -32,8 +32,7 @@ fudge -= 1 if var_needsgc(block.operations[-1].result): fudge += 1 - refs_out = len([v for v in link.args - if isinstance(v, Variable) and var_needsgc(v)]) + refs_out = len([v for v in link.args if var_needsgc(v)]) assert refs_in + push_alives + calls - fudge == pop_alives + refs_out if block.exitswitch is c_last_exception and link.exitcase is not None: From arigo at codespeak.net Sun Jan 29 13:53:59 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sun, 29 Jan 2006 13:53:59 +0100 (CET) Subject: [pypy-svn] r22835 - pypy/dist/pypy/jit Message-ID: <20060129125359.433E027B6C@code1.codespeak.net> Author: arigo Date: Sun Jan 29 13:53:57 2006 New Revision: 22835 Modified: pypy/dist/pypy/jit/hintcontainer.py Log: oups oups, stupid bug that took me ages to figure out. Modified: pypy/dist/pypy/jit/hintcontainer.py ============================================================================== --- pypy/dist/pypy/jit/hintcontainer.py (original) +++ pypy/dist/pypy/jit/hintcontainer.py Sun Jan 29 13:53:57 2006 @@ -1,4 +1,4 @@ -import weakref +import weakref, itertools from pypy.annotation.listdef import ListItem from pypy.annotation import model as annmodel from pypy.jit import hintmodel @@ -6,20 +6,17 @@ class AbstractContainerDef(object): - __counter = 0 + __counter = itertools.count() __cache = {} def __init__(self, bookkeeper, TYPE): self.T = TYPE self.bookkeeper = bookkeeper # hack to try to produce a repr that shows identifications - try: - weakdict = AbstractContainerDef.__cache[TYPE] - except KeyError: - weakdict = weakref.WeakValueDictionary() - AbstractContainerDef.__cache[self.__class__, TYPE] = weakdict - weakdict[AbstractContainerDef.__counter] = self - AbstractContainerDef.__counter += 1 + key = (self.__class__, TYPE) + weakdict = AbstractContainerDef.__cache.setdefault(key, + weakref.WeakValueDictionary()) + weakdict[AbstractContainerDef.__counter.next()] = self def __repr__(self): items = AbstractContainerDef.__cache[self.__class__, self.T].items() From tismer at codespeak.net Sun Jan 29 13:56:26 2006 From: tismer at codespeak.net (tismer at codespeak.net) Date: Sun, 29 Jan 2006 13:56:26 +0100 (CET) Subject: [pypy-svn] r22836 - pypy/dist/pypy/module/stackless Message-ID: <20060129125626.99B6A27B6C@code1.codespeak.net> Author: tismer Date: Sun Jan 29 13:56:24 2006 New Revision: 22836 Modified: pypy/dist/pypy/module/stackless/interp_coroutine.py Log: temporary debug instrumentation. going to refactor, things can be simpler and clearer without copying any state around Modified: pypy/dist/pypy/module/stackless/interp_coroutine.py ============================================================================== --- pypy/dist/pypy/module/stackless/interp_coroutine.py (original) +++ pypy/dist/pypy/module/stackless/interp_coroutine.py Sun Jan 29 13:56:24 2006 @@ -10,7 +10,7 @@ from pypy.rpython.rstack import yield_current_frame_to_caller from pypy.module.stackless.stackless_flags import StacklessFlags -import sys +import sys, os class CoState(object): def __init__(self): @@ -31,6 +31,13 @@ def __init__(self): pass +DEBUG = True + +def D(msg, x): + if DEBUG: + txt = "%s %s\n" % (msg, hex(id(x))) + os.write(2, txt) + class Coroutine(Wrappable): def __init__(self): @@ -39,6 +46,8 @@ self.parent = self else: self.parent = costate.current + D("new coro, self", self) + D(" parent", self.parent) self.thunk = None def bind(self, thunk): @@ -70,8 +79,11 @@ # considered a programming error. # greenlets and tasklets have different ideas about this. raise CoroutineDamage + D("switch begin self", self) costate.last.frame = self._update_state(self).switch() # note that last gets updated before assignment! + D("switch end self", self) + D(" costate.last", costate.last) if costate.things_to_do: do_things_to_do(self) @@ -180,6 +192,7 @@ class AppCoroutine(Coroutine): # XXX, StacklessFlags): def __init__(self): + D("new appcoro self", self) Coroutine.__init__(self) self.flags = 0 @@ -196,15 +209,26 @@ "cannot bind a bound Coroutine")) thunk = _AppThunk(space, w_func, __args__) costate.current = appcostate.current + D("w_bind self", self) self.bind(thunk) + D("return from w_bind self", self) + D(" costate.current", costate.current) + D(" appcostate.current", appcostate.current) + appcostate.current = costate.current def w_switch(self): space = self.space if self.frame is None: raise OperationError(space.w_ValueError, space.wrap( "cannot switch to an unbound Coroutine")) + D("w_switch begin self", self) + D(" costate.current", costate.current) + D(" appcostate.current", appcostate.current) costate.current = appcostate.current + D(" --> appcostate.current before switch", appcostate.current) self.switch() + D(" after switch self", self) + D(" after switch -> appcostate.current was", appcostate.current) appcostate.current = self ret, appcostate.tempval = appcostate.tempval, space.w_None return ret @@ -300,4 +324,9 @@ a new current. After a greenlet's switch, greenlet gets a new current. +More thoughts: +-------------- +Whenever we switch, whatever object we are jumping +at, we need to save the source continuation somewhere. + """ \ No newline at end of file From pedronis at codespeak.net Sun Jan 29 14:33:44 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Sun, 29 Jan 2006 14:33:44 +0100 (CET) Subject: [pypy-svn] r22837 - in pypy/dist/pypy/jit: . test Message-ID: <20060129133344.C85ED27B68@code1.codespeak.net> Author: pedronis Date: Sun Jan 29 14:33:42 2006 New Revision: 22837 Modified: pypy/dist/pypy/jit/hintannotator.py pypy/dist/pypy/jit/hintmodel.py pypy/dist/pypy/jit/test/test_hint_annotation.py Log: (arigo, pedronis) correct dep tracking for getarrayitem Modified: pypy/dist/pypy/jit/hintannotator.py ============================================================================== --- pypy/dist/pypy/jit/hintannotator.py (original) +++ pypy/dist/pypy/jit/hintannotator.py Sun Jan 29 14:33:42 2006 @@ -5,8 +5,8 @@ class HintAnnotator(RPythonAnnotator): - def __init__(self, policy=None): - RPythonAnnotator.__init__(self, policy=policy) + def __init__(self, translator, policy=None): + RPythonAnnotator.__init__(self, translator, policy=policy) self.bookkeeper = HintBookkeeper(self) # XXX def consider_op_malloc(self, hs_TYPE): Modified: pypy/dist/pypy/jit/hintmodel.py ============================================================================== --- pypy/dist/pypy/jit/hintmodel.py (original) +++ pypy/dist/pypy/jit/hintmodel.py Sun Jan 29 14:33:42 2006 @@ -3,7 +3,7 @@ from pypy.jit.hintbookkeeper import getbookkeeper from pypy.rpython.lltypesystem import lltype -UNARY_OPERATIONS = """same_as hint getfield setfield getsubstruct getarraysize getarrayitem setarrayitem +UNARY_OPERATIONS = """same_as hint getfield setfield getsubstruct getarraysize setarrayitem cast_pointer direct_call int_is_true int_neg @@ -17,7 +17,8 @@ BINARY_OPERATIONS = """int_add int_sub int_mul int_mod int_and int_rshift int_floordiv uint_add uint_sub uint_mul uint_mod uint_and uint_rshift uint_floordiv int_gt int_lt int_le int_ge int_eq int_ne - uint_gt uint_lt uint_le uint_ge uint_eq uint_ne""".split() + uint_gt uint_lt uint_le uint_ge uint_eq uint_ne + getarrayitem""".split() class OriginTreeNode(object): @@ -50,20 +51,12 @@ self.concretetype = T assert self.__class__ != SomeLLAbstractValue - def reorigin(self, bookkeeper): - return self - class SomeLLAbstractConstant(SomeLLAbstractValue): def __init__(self, T, origins): SomeLLAbstractValue.__init__(self, T) self.origins = origins - def reorigin(self, bookkeeper): - origin = bookkeeper.myorigin() - origin.merge(self.origins) - return SomeLLAbstractConstant(self.concretetype, {origin: True}) - class SomeLLConcreteValue(SomeLLAbstractValue): pass @@ -76,6 +69,17 @@ self.contentdef = contentdef self.concretetype = lltype.Ptr(contentdef.T) +def reorigin(hs_v1, *deps_hs): + if isinstance(hs_v1, SomeLLAbstractConstant): + origin = getbookkeeper().myorigin() + origin.merge(hs_v1.origins) + for hs_dep in deps_hs: + if isinstance(hs_v1, SomeLLAbstractConstant): + origin.merge(hs_dep.origins) + return SomeLLAbstractConstant(hs_v1.concretetype, {origin: True}) + else: + return hs_v1 + # ____________________________________________________________ # operations @@ -105,16 +109,6 @@ else: return SomeLLAbstractVariable(FIELD_TYPE) - def getarrayitem(hs_c1, hs_index): - A = hs_c1.concretetype.TO - READ_TYPE = A.OF - if A._hints.get('immutable', False): - origin = getbookkeeper().myorigin() - origin.merge(hs_c1.origins) - return SomeLLAbstractConstant(READ_TYPE, {origin: True}) - else: - return SomeLLAbstractVariable(READ_TYPE) - def getsubstruct(hs_c1, hs_fieldname): S = hs_c1.concretetype.TO SUB_TYPE = getattr(S, hs_fieldname.const) @@ -147,10 +141,7 @@ hs_res = bookkeeper.annotator.recursivecall(fnobj.graph, bookkeeper.position_key, args_hs) - if isinstance(hs_res, SomeLLAbstractValue): - hs_res = hs_res.reorigin(bookkeeper) - #else: it's a SomeImpossibleValue - return hs_res + return reorigin(hs_res) def unary_char(hs_c1): origin = getbookkeeper().myorigin() @@ -206,9 +197,6 @@ def setarrayitem(hs_a1, hs_index, hs_value): hs_a1.contentdef.generalize_item(hs_value) - def getarrayitem(hs_a1, hs_index): - return hs_a1.contentdef.read_item() - def getarraysize(hs_a1): origin = getbookkeeper().myorigin() return SomeLLAbstractConstant(lltype.Signed, {origin: True}) @@ -226,6 +214,9 @@ def int_add((hs_v1, hs_v2)): return SomeLLAbstractVariable(lltype.Signed) + def getarrayitem((hs_v1, hs_v2)): + return SomeLLAbstractVariable(hs_v1.concretetype.TO.OF) + def union((hs_v1, hs_v2)): raise annmodel.UnionError("%s %s don't mix" % (hs_v1, hs_v2)) @@ -268,6 +259,17 @@ origins = annmodel.setunion(hs_c1.origins, hs_c2.origins) return SomeLLAbstractConstant(hs_c1.concretetype, origins) + def getarrayitem((hs_c1, hs_index)): + A = hs_c1.concretetype.TO + READ_TYPE = A.OF + if A._hints.get('immutable', False): + origin = getbookkeeper().myorigin() + origin.merge(hs_c1.origins) + origin.merge(hs_index.origins) + return SomeLLAbstractConstant(READ_TYPE, {origin: True}) + else: + return SomeLLAbstractVariable(READ_TYPE) + class __extend__(pairtype(SomeLLAbstractConstant, SomeLLConcreteValue), pairtype(SomeLLConcreteValue, SomeLLAbstractConstant), pairtype(SomeLLConcreteValue, SomeLLConcreteValue)): @@ -288,6 +290,9 @@ int_lt = int_le = int_ge = int_ne = int_gt = int_eq uint_lt = uint_le = uint_ge = uint_ne = uint_gt = uint_eq = int_eq + def getarrayitem((hs_c1, hs_index)): + return SomeLLConcreteValue(hs_c1.concretetype.TO.OF) + class __extend__(pairtype(SomeLLConcreteValue, SomeLLAbstractConstant), pairtype(SomeLLAbstractConstant, SomeLLConcreteValue)): @@ -303,6 +308,13 @@ def union((hs_cont1, hs_cont2)): return SomeLLAbstractContainer(hs_cont1.contentdef.union(hs_cont2.contentdef)) +class __extend__(pairtype(SomeLLAbstractContainer, SomeLLAbstractConstant)): + + def getarrayitem((hs_a1, hs_index)): + hs_res = hs_a1.contentdef.read_item() + return reorigin(hs_res, hs_index) + + # ____________________________________________________________ def handle_highlevel_operation(bookkeeper, ll_func, *args_hs): Modified: pypy/dist/pypy/jit/test/test_hint_annotation.py ============================================================================== --- pypy/dist/pypy/jit/test/test_hint_annotation.py (original) +++ pypy/dist/pypy/jit/test/test_hint_annotation.py Sun Jan 29 14:33:42 2006 @@ -11,7 +11,7 @@ P_OOPSPEC = AnnotatorPolicy() P_OOPSPEC.oopspec = True -def hannotate(func, argtypes, policy=None): +def hannotate(func, argtypes, policy=None, annotator=False): # build the normal ll graphs for ll_function t = TranslationContext() a = t.buildannotator() @@ -20,13 +20,16 @@ rtyper.specialize() graph1 = graphof(t, func) # build hint annotator types - hannotator = HintAnnotator(policy=policy) + hannotator = HintAnnotator(t, policy=policy) hs = hannotator.build_graph_types(graph1, [SomeLLAbstractConstant(v.concretetype, {OriginTreeNode(): True}) for v in graph1.getargs()]) #hannotator.translator.graphs.append(graph1) #hannotator.translator.view() - return hs + if annotator: + return hs, hannotator + else: + return hs def test_simple(): def ll_function(x, y): @@ -262,7 +265,32 @@ assert isinstance(hs, SomeLLAbstractContainer) assert hs.concretetype == PGCS2 - +def test_getarrayitem(): + A = lltype.GcArray(lltype.Signed, hints={'immutable': True}) + a = lltype.malloc(A, 10) + def ll1(n): + v = a[n] + v = hint(v, concrete=True) + return v + hs, ha = hannotate(ll1, [int], annotator=True) + assert isinstance(hs, SomeLLConcreteValue) + g1 = graphof(ha.translator, ll1) + hs_n = ha.binding(g1.getargs()[0]) + assert hs_n.origins.keys()[0].fixed + +def test_getvarrayitem(): + A = lltype.GcArray(lltype.Signed, hints={'immutable': True}) + def ll1(n): + a = lltype.malloc(A, 10) + v = a[n] + v = hint(v, concrete=True) + return v + hs, ha = hannotate(ll1, [int], annotator=True) + assert isinstance(hs, SomeLLConcreteValue) + g1 = graphof(ha.translator, ll1) + hs_n = ha.binding(g1.getargs()[0]) + assert hs_n.origins.keys()[0].fixed + def test_hannotate_tl(): from pypy.jit import tl From mwh at codespeak.net Sun Jan 29 15:03:38 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Sun, 29 Jan 2006 15:03:38 +0100 (CET) Subject: [pypy-svn] r22841 - pypy/dist/pypy/rpython/memory Message-ID: <20060129140338.9C33E27B4C@code1.codespeak.net> Author: mwh Date: Sun Jan 29 15:03:37 2006 New Revision: 22841 Modified: pypy/dist/pypy/rpython/memory/gctransform.py Log: admit that, for now, any operation can potentially raise an exception. decref last_exception if needed too (only in the pyobject case, most likely). this is broken in combination with what's on the branch now but not for long. Modified: pypy/dist/pypy/rpython/memory/gctransform.py ============================================================================== --- pypy/dist/pypy/rpython/memory/gctransform.py (original) +++ pypy/dist/pypy/rpython/memory/gctransform.py Sun Jan 29 15:03:37 2006 @@ -82,7 +82,7 @@ livevars = [var for var in block.inputargs if var_needsgc(var)] for op in block.operations: newops.extend(self.replacement_operations(op)) - if op.opname in EXCEPTION_RAISING_OPS and livevars: + if 1 or op.opname in EXCEPTION_RAISING_OPS: cleanup_on_exception = [] for var in livevars: cleanup_on_exception.extend(self.pop_alive(var)) @@ -116,7 +116,9 @@ # exception, it can't have returned anything that # might need pop_aliving. del livecounts[livevars[-1]] - livecounts[link.last_exc_value] = 1 + for v in link.last_exception, link.last_exc_value: + if var_needsgc(v): + livecounts[v] = 1 for v in link.args: if v in livecounts: livecounts[v] -= 1 From mwh at codespeak.net Sun Jan 29 15:04:56 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Sun, 29 Jan 2006 15:04:56 +0100 (CET) Subject: [pypy-svn] r22842 - pypy/branch/genc-gc-refactoring Message-ID: <20060129140456.3993F27B4C@code1.codespeak.net> Author: mwh Date: Sun Jan 29 15:04:54 2006 New Revision: 22842 Modified: pypy/branch/genc-gc-refactoring/funcgen.py pypy/branch/genc-gc-refactoring/gc.py Log: slight cleanups and de-crufting. Modified: pypy/branch/genc-gc-refactoring/funcgen.py ============================================================================== --- pypy/branch/genc-gc-refactoring/funcgen.py (original) +++ pypy/branch/genc-gc-refactoring/funcgen.py Sun Jan 29 15:04:54 2006 @@ -35,6 +35,7 @@ self.functionname = functionname # apply the gc transformation self.db.gctransformer.transform_graph(self.graph) + #self.graph.show() self.blocknum = {} # @@ -127,13 +128,18 @@ def return_with_error(self): if self.cpython_exc: + # this should most likely be done on the graph level! lltype_of_exception_value = self.db.get_lltype_of_exception_value() exc_value_typename = self.db.gettype(lltype_of_exception_value) assert self.lltypemap(self.graph.getreturnvar()) == PyObjPtr yield '{' yield '\t%s;' % cdecl(exc_value_typename, 'vanishing_exc_value') yield '\tRPyConvertExceptionToCPython(vanishing_exc_value);' - yield '\t%s' % self.pop_alive_expr('vanishing_exc_value', lltype_of_exception_value) + if lltype_of_exception_value == PyObjPtr: + yield '\tPy_XDECREF(vanishing_exc_value);' + else: + yield '\t%s' % self.gcpolicy.pop_alive_nopyobj( + 'vanishing_exc_value', lltype_of_exception_value) yield '}' yield 'return %s; ' % self.error_return_value() @@ -231,12 +237,8 @@ d = {} if isinstance(link.last_exception, Variable): d[link.last_exception] = 'exc_cls' - else: - yield '\t' + self.pop_alive_expr('exc_cls', T1) if isinstance(link.last_exc_value, Variable): d[link.last_exc_value] = 'exc_value' - else: - yield '\t' + self.pop_alive_expr('exc_value', T2) for op in self.gen_link(link, d): yield '\t' + op yield '}' @@ -296,19 +298,20 @@ " Got %r" % (TYPE,)) for i, op in list(enumerate(block.operations))[::-1]: - if not fallthrough: - yield 'err%d_%d:' % (myblocknum, i) - else: - fallthrough = False # this label was already generated - for cleanupop in getattr(op, 'cleanup', ()): - line = self.gen_op(cleanupop, 'should_never_be_jumped_to') - if '\\n' in line: - for l in line.splitlines(): - yield l + if hasattr(op, 'cleanup'): + if not fallthrough: + yield 'err%d_%d:' % (myblocknum, i) else: + fallthrough = False # this label was already generated + for cleanupop in op.cleanup: + line = self.gen_op(cleanupop, 'should_never_be_jumped_to') + if '\\n' in line: + for l in line.splitlines(): + yield l + else: + yield line + for line in self.return_with_error(): yield line - for line in self.return_with_error(): - yield line def gen_link(self, link, linklocalvars=None): "Generate the code to jump across the given Link." @@ -412,7 +415,7 @@ result = ['%s = %s;' % (newvalue, sourceexpr)] # need to adjust the refcount of the result only for PyObjects if T == PyObjPtr: - result.append(self.pyobj_incref_expr(newvalue, T)) + result.append('Py_XINCREF(%s);' % newvalue) result = '\n'.join(result) if T is Void: result = '/* %s */' % result @@ -555,7 +558,7 @@ self.expr(op.args[0]))) if TYPE == PyObjPtr: - result.append(self.pyobj_incref(op.result)) + result.append('Py_XINCREF(%s);'%(LOCAL_VAR % op.result.name)) return '\t'.join(result) def OP_CAST_INT_TO_PTR(self, op, err): @@ -572,7 +575,7 @@ result.append('%s = %s;' % (self.expr(op.result), self.expr(op.args[0]))) if TYPE == PyObjPtr: - result.append(self.pyobj_incref(op.result)) + result.append('Py_XINCREF(%s);'%(LOCAL_VAR % op.result.name)) return '\t'.join(result) def OP_HINT(self, op, err): @@ -599,19 +602,6 @@ typename = self.db.gettype(TYPE).replace("@", "*") #XXX see above return "%(result)s = *(((%(typename)s) %(addr)s ) + %(offset)s);" % locals() - def pyobj_incref(self, v): - T = self.lltypemap(v) - return self.pyobj_incref_expr(LOCALVAR % v.name, T) - - def pyobj_incref_expr(self, expr, T): - return self.gcpolicy.pyobj_incref(expr, T) - - def pop_alive(self, v, expr=None): - T = self.lltypemap(v) - return self.gcpolicy.pop_alive(expr or (LOCALVAR % v.name), T) - - def pop_alive_expr(self, expr, T): - return self.gcpolicy.pop_alive(expr, T) assert not USESLOTS or '__dict__' not in dir(FunctionCodeGenerator) Modified: pypy/branch/genc-gc-refactoring/gc.py ============================================================================== --- pypy/branch/genc-gc-refactoring/gc.py (original) +++ pypy/branch/genc-gc-refactoring/gc.py Sun Jan 29 15:04:54 2006 @@ -14,10 +14,10 @@ self.db = db self.thread_enabled = thread_enabled - def pyobj_incref(self, expr, T): + def pyobj_incref(self, expr): return 'Py_XINCREF(%s);' % expr - def pyobj_decref(self, expr, T): + def pyobj_decref(self, expr): return 'Py_XDECREF(%s);' % expr def push_alive(self, expr, T): @@ -25,7 +25,7 @@ if expr == 'NULL': # hum return '' if T.TO == PyObject: - return self.pyobj_incref(expr, T) + return self.pyobj_incref(expr) else: return self.push_alive_nopyobj(expr, T) return '' @@ -33,7 +33,7 @@ def pop_alive(self, expr, T): if isinstance(T, Ptr) and T._needsgc(): if T.TO == PyObject: - return self.pyobj_decref(expr, T) + return self.pyobj_decref(expr) else: return self.pop_alive_nopyobj(expr, T) return '' From tismer at codespeak.net Sun Jan 29 15:22:02 2006 From: tismer at codespeak.net (tismer at codespeak.net) Date: Sun, 29 Jan 2006 15:22:02 +0100 (CET) Subject: [pypy-svn] r22843 - pypy/dist/pypy/module/stackless Message-ID: <20060129142202.1DA4727B5B@code1.codespeak.net> Author: tismer Date: Sun Jan 29 15:22:00 2006 New Revision: 22843 Modified: pypy/dist/pypy/module/stackless/interp_coroutine.py Log: a little bit of abstraction often makes things much simpler. Modified: pypy/dist/pypy/module/stackless/interp_coroutine.py ============================================================================== --- pypy/dist/pypy/module/stackless/interp_coroutine.py (original) +++ pypy/dist/pypy/module/stackless/interp_coroutine.py Sun Jan 29 15:22:00 2006 @@ -14,7 +14,7 @@ class CoState(object): def __init__(self): - self.last = self.current = Coroutine() + self.last = self.current = self.main = Coroutine() self.things_to_do = False self.temp_exc = None self.del_first = None @@ -46,10 +46,15 @@ self.parent = self else: self.parent = costate.current - D("new coro, self", self) - D(" parent", self.parent) self.thunk = None + def _get_current(self): + # must be overridden for every distinguished subclass + return costate.current + def _set_current(self, new): + # must be overridden for every distinguished subclass + costate.current = new + def bind(self, thunk): if self.frame is not None: raise CoroutineDamage @@ -79,19 +84,16 @@ # considered a programming error. # greenlets and tasklets have different ideas about this. raise CoroutineDamage - D("switch begin self", self) costate.last.frame = self._update_state(self).switch() # note that last gets updated before assignment! - D("switch end self", self) - D(" costate.last", costate.last) if costate.things_to_do: do_things_to_do(self) - def _update_state(new): - costate.last, costate.current = costate.current, new + def _update_state(self, new): + costate.last = self._get_current() + self._set_current(new) frame, new.frame = new.frame, None return frame - _update_state = staticmethod(_update_state) def kill(self): # if costate.current is self: @@ -128,9 +130,8 @@ def is_zombie(self): return self.frame is not None and check_for_zombie(self) - def get_current(): - return costate.current - get_current = staticmethod(get_current) + def get_current(self): + return self._get_current() def check_for_zombie(self): @@ -192,10 +193,16 @@ class AppCoroutine(Coroutine): # XXX, StacklessFlags): def __init__(self): - D("new appcoro self", self) Coroutine.__init__(self) self.flags = 0 + def _get_current(self): + # must be overridden for every distinguished subclass + return appcostate.current + def _set_current(self, new): + # must be overridden for every distinguished subclass + appcostate.current = new + def descr_method__new__(space, w_subtype): co = space.allocate_instance(AppCoroutine, w_subtype) AppCoroutine.__init__(co) @@ -208,34 +215,18 @@ raise OperationError(space.w_ValueError, space.wrap( "cannot bind a bound Coroutine")) thunk = _AppThunk(space, w_func, __args__) - costate.current = appcostate.current - D("w_bind self", self) self.bind(thunk) - D("return from w_bind self", self) - D(" costate.current", costate.current) - D(" appcostate.current", appcostate.current) - appcostate.current = costate.current def w_switch(self): space = self.space if self.frame is None: raise OperationError(space.w_ValueError, space.wrap( "cannot switch to an unbound Coroutine")) - D("w_switch begin self", self) - D(" costate.current", costate.current) - D(" appcostate.current", appcostate.current) - costate.current = appcostate.current - D(" --> appcostate.current before switch", appcostate.current) self.switch() - D(" after switch self", self) - D(" after switch -> appcostate.current was", appcostate.current) - appcostate.current = self ret, appcostate.tempval = appcostate.tempval, space.w_None return ret def w_kill(self): - if appcostate.current is self: - costate.current = self self.kill() def __del__(self): @@ -250,7 +241,7 @@ self.space.userdel(self) def get_current(space): - return space.wrap(appcostate.current) + return space.wrap(self._get_current()) get_current = staticmethod(get_current) From mwh at codespeak.net Sun Jan 29 15:28:50 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Sun, 29 Jan 2006 15:28:50 +0100 (CET) Subject: [pypy-svn] r22844 - pypy/branch/genc-gc-refactoring Message-ID: <20060129142850.7A9DD27B5B@code1.codespeak.net> Author: mwh Date: Sun Jan 29 15:28:48 2006 New Revision: 22844 Modified: pypy/branch/genc-gc-refactoring/gc.py Log: don't try and incref NULL (grumble) Modified: pypy/branch/genc-gc-refactoring/gc.py ============================================================================== --- pypy/branch/genc-gc-refactoring/gc.py (original) +++ pypy/branch/genc-gc-refactoring/gc.py Sun Jan 29 15:28:48 2006 @@ -287,6 +287,8 @@ def OP_GC_PUSH_ALIVE(self, funcgen, op, err): expr = funcgen.expr(op.args[0]) + if expr == 'NULL': + return '' defnode = self.db.gettypedefnode(op.args[0].concretetype.TO) assert defnode.gcheader is not None return 'pypy_IncRf_%s(%s);' % (defnode.barename, expr) @@ -299,6 +301,8 @@ def OP_GC_PUSH_ALIVE_PYOBJ(self, funcgen, op, err): expr = funcgen.expr(op.args[0]) + if expr == 'NULL': + return '' return 'Py_XINCREF(%s);' % expr def OP_GC_POP_ALIVE_PYOBJ(self, funcgen, op, err): From arigo at codespeak.net Sun Jan 29 15:55:35 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sun, 29 Jan 2006 15:55:35 +0100 (CET) Subject: [pypy-svn] r22845 - in pypy/dist/pypy/jit: . test Message-ID: <20060129145535.13BC627B69@code1.codespeak.net> Author: arigo Date: Sun Jan 29 15:55:33 2006 New Revision: 22845 Modified: pypy/dist/pypy/jit/hintbookkeeper.py pypy/dist/pypy/jit/hintmodel.py pypy/dist/pypy/jit/hintvlist.py pypy/dist/pypy/jit/test/test_hint_annotation.py Log: (arre, pedronis, arigo) Refactoring: removed the OriginTreeNode and replaced it with an empty box OriginFlags. Now the SomeLLAbstractConstant annotations contain a family of such boxes, which no longer point to each other directly. Fixed the tests, added a nice __repr__. The TL interpreter seems to get correct-looking annotations now :-) Modified: pypy/dist/pypy/jit/hintbookkeeper.py ============================================================================== --- pypy/dist/pypy/jit/hintbookkeeper.py (original) +++ pypy/dist/pypy/jit/hintbookkeeper.py Sun Jan 29 15:55:33 2006 @@ -6,7 +6,7 @@ def __init__(self, hannotator): self.pending_specializations = [] - self.origins = {} + self.originflags = {} self.virtual_containers = {} self.annotator = hannotator @@ -23,11 +23,11 @@ def myorigin(self): try: - origin = self.origins[self.position_key] + origin = self.originflags[self.position_key] except KeyError: from pypy.jit import hintmodel - origin = hintmodel.OriginTreeNode() - self.origins[self.position_key] = origin + origin = hintmodel.OriginFlags() + self.originflags[self.position_key] = origin return origin def compute_at_fixpoint(self): Modified: pypy/dist/pypy/jit/hintmodel.py ============================================================================== --- pypy/dist/pypy/jit/hintmodel.py (original) +++ pypy/dist/pypy/jit/hintmodel.py Sun Jan 29 15:55:33 2006 @@ -20,31 +20,17 @@ uint_gt uint_lt uint_le uint_ge uint_eq uint_ne getarrayitem""".split() -class OriginTreeNode(object): +class OriginFlags(object): fixed = False - def __init__(self, origins=None): - if origins is None: - origins = {} - self.origins = origins - - def merge(self, nodes): - self.origins.update(nodes) - - def visit(self, seen=None): - if seen is None: - seen = {} - yield self - for o in self.origins: - if o not in seen: - seen[o] = True - for o1 in o.visit(seen): - yield o1 - def __repr__(self): - return "O" + (self.fixed and "f" or "") - + if self.fixed: + s = "fixed " + else: + s = "" + return "<%sorigin>" % (s,) + class SomeLLAbstractValue(annmodel.SomeObject): def __init__(self, T): @@ -57,6 +43,22 @@ SomeLLAbstractValue.__init__(self, T) self.origins = origins + def fmt_origins(self, origins): + counts = {} + for o in origins: + x = repr(o) + counts[x] = counts.get(x, 0) + 1 + items = counts.items() + items.sort() + lst = [] + for key, count in items: + s = '' + if count > 1: + s += '%d*' % count + s += key + lst.append(s) + return '<%s>' % (', '.join(lst),) + class SomeLLConcreteValue(SomeLLAbstractValue): pass @@ -69,14 +71,29 @@ self.contentdef = contentdef self.concretetype = lltype.Ptr(contentdef.T) + +setunion = annmodel.setunion + +def setadd(set, newitem): + if newitem not in set: + set = set.copy() + set[newitem] = True + return set + +def newset(set, *sets): + set = set.copy() + for s2 in sets: + set.update(s2) + return set + def reorigin(hs_v1, *deps_hs): if isinstance(hs_v1, SomeLLAbstractConstant): - origin = getbookkeeper().myorigin() - origin.merge(hs_v1.origins) - for hs_dep in deps_hs: - if isinstance(hs_v1, SomeLLAbstractConstant): - origin.merge(hs_dep.origins) - return SomeLLAbstractConstant(hs_v1.concretetype, {origin: True}) + deps_origins = [hs_dep.origins for hs_dep in deps_hs + if isinstance(hs_dep, SomeLLAbstractConstant)] + d = newset(hs_v1.origins, + {getbookkeeper().myorigin(): True}, + *deps_origins) + return SomeLLAbstractConstant(hs_v1.concretetype, d) else: return hs_v1 @@ -95,31 +112,27 @@ return SomeLLAbstractVariable(hs_c1.concretetype) assert hs_flags.const['concrete'] for o in hs_c1.origins: - for o1 in o.visit(): - o1.fixed = True + o.fixed = True return SomeLLConcreteValue(hs_c1.concretetype) def getfield(hs_c1, hs_fieldname): S = hs_c1.concretetype.TO FIELD_TYPE = getattr(S, hs_fieldname.const) if S._hints.get('immutable', False): - origin = getbookkeeper().myorigin() - origin.merge(hs_c1.origins) - return SomeLLAbstractConstant(FIELD_TYPE, {origin: True}) + d = setadd(hs_c1.origins, getbookkeeper().myorigin()) + return SomeLLAbstractConstant(FIELD_TYPE, d) else: return SomeLLAbstractVariable(FIELD_TYPE) def getsubstruct(hs_c1, hs_fieldname): S = hs_c1.concretetype.TO SUB_TYPE = getattr(S, hs_fieldname.const) - origin = getbookkeeper().myorigin() - origin.merge(hs_c1.origins) - return SomeLLAbstractConstant(lltype.Ptr(SUB_TYPE), {origin: True}) + d = setadd(hs_c1.origins, getbookkeeper().myorigin()) + return SomeLLAbstractConstant(lltype.Ptr(SUB_TYPE), d) def getarraysize(hs_c1): - origin = getbookkeeper().myorigin() - origin.merge(hs_c1.origins) - return SomeLLAbstractConstant(lltype.Signed, {origin: True}) + d = setadd(hs_c1.origins, getbookkeeper().myorigin()) + return SomeLLAbstractConstant(lltype.Signed, d) def direct_call(hs_f1, *args_hs): bookkeeper = getbookkeeper() @@ -144,23 +157,20 @@ return reorigin(hs_res) def unary_char(hs_c1): - origin = getbookkeeper().myorigin() - origin.merge(hs_c1.origins) - return SomeLLAbstractConstant(lltype.Char, {origin: True}) + d = setadd(hs_c1.origins, getbookkeeper().myorigin()) + return SomeLLAbstractConstant(lltype.Char, d) cast_int_to_char = unary_char def unary_int(hs_c1): - origin = getbookkeeper().myorigin() - origin.merge(hs_c1.origins) - return SomeLLAbstractConstant(lltype.Signed, {origin: True}) + d = setadd(hs_c1.origins, getbookkeeper().myorigin()) + return SomeLLAbstractConstant(lltype.Signed, d) cast_uint_to_int = cast_bool_to_int = cast_char_to_int = int_neg = unary_int def int_is_true(hs_c1): - origin = getbookkeeper().myorigin() - origin.merge(hs_c1.origins) - return SomeLLAbstractConstant(lltype.Bool, {origin: True}) + d = setadd(hs_c1.origins, getbookkeeper().myorigin()) + return SomeLLAbstractConstant(lltype.Bool, d) uint_is_true = int_is_true @@ -230,43 +240,39 @@ class __extend__(pairtype(SomeLLAbstractConstant, SomeLLAbstractConstant)): def int_add((hs_c1, hs_c2)): - origin = getbookkeeper().myorigin() - origin.merge(hs_c1.origins) - origin.merge(hs_c2.origins) - return SomeLLAbstractConstant(lltype.Signed, {origin: True}) + d = newset(hs_c1.origins, hs_c2.origins, + {getbookkeeper().myorigin(): True}) + return SomeLLAbstractConstant(lltype.Signed, d) int_floordiv = int_rshift = int_and = int_mul = int_mod = int_sub = int_add def uint_add((hs_c1, hs_c2)): - origin = getbookkeeper().myorigin() - origin.merge(hs_c1.origins) - origin.merge(hs_c2.origins) - return SomeLLAbstractConstant(lltype.Unsigned, {origin: True}) + d = newset(hs_c1.origins, hs_c2.origins, + {getbookkeeper().myorigin(): True}) + return SomeLLAbstractConstant(lltype.Unsigned, d) uint_floordiv = uint_rshift = uint_and = uint_mul = uint_mod = uint_sub = uint_add def int_eq((hs_c1, hs_c2)): - origin = getbookkeeper().myorigin() - origin.merge(hs_c1.origins) - origin.merge(hs_c2.origins) - return SomeLLAbstractConstant(lltype.Bool, {origin: True}) + d = newset(hs_c1.origins, hs_c2.origins, + {getbookkeeper().myorigin(): True}) + return SomeLLAbstractConstant(lltype.Bool, d) int_lt = int_le = int_ge = int_ne = int_gt = int_eq uint_lt = uint_le = uint_ge = uint_ne = uint_gt = uint_eq = int_eq def union((hs_c1, hs_c2)): assert hs_c1.concretetype == hs_c2.concretetype - origins = annmodel.setunion(hs_c1.origins, hs_c2.origins) - return SomeLLAbstractConstant(hs_c1.concretetype, origins) + d = newset(hs_c1.origins, hs_c2.origins) + return SomeLLAbstractConstant(hs_c1.concretetype, d) def getarrayitem((hs_c1, hs_index)): A = hs_c1.concretetype.TO READ_TYPE = A.OF if A._hints.get('immutable', False): - origin = getbookkeeper().myorigin() - origin.merge(hs_c1.origins) - origin.merge(hs_index.origins) - return SomeLLAbstractConstant(READ_TYPE, {origin: True}) + d = newset(hs_c1.origins, hs_index.origins, + {getbookkeeper().myorigin(): True}) + return SomeLLAbstractConstant(READ_TYPE, d) else: return SomeLLAbstractVariable(READ_TYPE) Modified: pypy/dist/pypy/jit/hintvlist.py ============================================================================== --- pypy/dist/pypy/jit/hintvlist.py (original) +++ pypy/dist/pypy/jit/hintvlist.py Sun Jan 29 15:55:33 2006 @@ -1,5 +1,6 @@ from pypy.annotation.listdef import ListItem -from pypy.jit import hintmodel +from pypy.jit.hintmodel import SomeLLAbstractConstant +from pypy.jit.hintmodel import SomeLLAbstractContainer, reorigin from pypy.jit.hintbookkeeper import getbookkeeper from pypy.jit.hintcontainer import AbstractContainerDef, make_item_annotation from pypy.rpython.lltypesystem import lltype @@ -34,15 +35,15 @@ def oop_len(self): origin = getbookkeeper().myorigin() - return hintmodel.SomeLLAbstractConstant(lltype.Signed, {origin: True}) + return SomeLLAbstractConstant(lltype.Signed, {origin: True}) def oop_nonzero(self): origin = getbookkeeper().myorigin() - return hintmodel.SomeLLAbstractConstant(lltype.Bool, {origin: True}) + return SomeLLAbstractConstant(lltype.Bool, {origin: True}) def oop_getitem(self, hs_index): assert hs_index.concretetype == lltype.Signed - return self.read_item() + return reorigin(self.read_item(), hs_index) def oop_setitem(self, hs_index, hs_value): assert hs_index.concretetype == lltype.Signed @@ -60,7 +61,7 @@ def oop_pop(self, hs_index=None): assert hs_index is None or hs_index.concretetype == lltype.Signed - return self.read_item() + return reorigin(self.read_item(), hs_index) def oop_reverse(self): pass @@ -69,10 +70,10 @@ bk = self.bookkeeper vlistdef = bk.getvirtualcontainerdef(self.T, VirtualListDef) vlistdef.generalize_item(self.read_item()) - return hintmodel.SomeLLAbstractContainer(vlistdef) + return SomeLLAbstractContainer(vlistdef) def oop_concat(self, hs_other): - assert isinstance(hs_other, hintmodel.SomeLLAbstractContainer) # for now + assert isinstance(hs_other, SomeLLAbstractContainer) # for now assert hs_other.contentdef.T == self.T return self.oop_copy() @@ -82,4 +83,4 @@ bk = getbookkeeper() LIST = bk.current_op_concretetype().TO vlistdef = bk.getvirtualcontainerdef(LIST, VirtualListDef) - return hintmodel.SomeLLAbstractContainer(vlistdef) + return SomeLLAbstractContainer(vlistdef) Modified: pypy/dist/pypy/jit/test/test_hint_annotation.py ============================================================================== --- pypy/dist/pypy/jit/test/test_hint_annotation.py (original) +++ pypy/dist/pypy/jit/test/test_hint_annotation.py Sun Jan 29 15:55:33 2006 @@ -21,11 +21,11 @@ graph1 = graphof(t, func) # build hint annotator types hannotator = HintAnnotator(t, policy=policy) + t.annotator = hannotator # XXX? hs = hannotator.build_graph_types(graph1, [SomeLLAbstractConstant(v.concretetype, - {OriginTreeNode(): True}) + {OriginFlags(): True}) for v in graph1.getargs()]) - #hannotator.translator.graphs.append(graph1) - #hannotator.translator.view() + #t.view() if annotator: return hs, hannotator else: @@ -36,7 +36,7 @@ return x + y hs = hannotate(ll_function, [int, int]) assert isinstance(hs, SomeLLAbstractConstant) - assert len(hs.origins) == 1 + assert len(hs.origins) == 3 assert hs.concretetype == lltype.Signed def test_join(): @@ -48,7 +48,7 @@ return z hs = hannotate(ll_function, [bool, int, int]) assert isinstance(hs, SomeLLAbstractConstant) - assert len(hs.origins) == 2 + assert len(hs.origins) == 4 assert hs.concretetype == lltype.Signed @@ -74,13 +74,9 @@ return z # origin of z1 hs = hannotate(ll_function, [bool, int, int]) assert isinstance(hs, SomeLLAbstractConstant) - assert len(hs.origins) == 2 + assert len(hs.origins) == 4 for o in hs.origins: assert o.fixed - assert len(o.origins) == 2 - for o in o.origins: - assert o.fixed - assert not o.origins assert hs.concretetype == lltype.Signed def test_simple_variable(): @@ -156,7 +152,7 @@ hs = hannotate(ll_function, [int, int]) assert isinstance(hs, SomeLLAbstractConstant) assert hs.concretetype == lltype.Signed - assert len(hs.origins) == 2 + assert len(hs.origins) == 4 def test_loop1(): def ll_function(x, y): @@ -169,7 +165,7 @@ hs = hannotate(ll_function, [int, int]) assert isinstance(hs, SomeLLAbstractConstant) assert hs.concretetype == lltype.Signed - assert len(hs.origins) == 2 + assert len(hs.origins) == 4 def test_simple_struct(): S = lltype.GcStruct('helloworld', ('hello', lltype.Signed), @@ -180,8 +176,7 @@ hs = hannotate(ll_function, [annmodel.SomePtr(lltype.Ptr(S))]) assert isinstance(hs, SomeLLAbstractConstant) assert hs.concretetype == lltype.Signed - assert len(hs.origins) == 1 - assert len(hs.origins.keys()[0].origins) == 2 + assert len(hs.origins) == 4 def test_simple_struct_malloc(): S = lltype.GcStruct('helloworld', ('hello', lltype.Signed), @@ -194,8 +189,7 @@ hs = hannotate(ll_function, [int]) assert isinstance(hs, SomeLLAbstractConstant) assert hs.concretetype == lltype.Signed - assert len(hs.origins) == 1 - assert len(hs.origins.keys()[0].origins) == 1 + assert len(hs.origins) == 2 def test_container_union(): S = lltype.GcStruct('helloworld', ('hello', lltype.Signed), @@ -212,8 +206,7 @@ hs = hannotate(ll_function, [bool, int, int]) assert isinstance(hs, SomeLLAbstractConstant) assert hs.concretetype == lltype.Signed - assert len(hs.origins) == 1 - assert len(hs.origins.keys()[0].origins) == 2 + assert len(hs.origins) == 3 def test_simple_call(): def ll2(x, y): @@ -223,7 +216,7 @@ hs = hannotate(ll1, [int, int, int]) assert isinstance(hs, SomeLLAbstractConstant) assert hs.concretetype == lltype.Signed - assert len(hs.origins) == 1 + assert len(hs.origins) == 7 def test_simple_list_operations(): def ll_function(x, y, index): @@ -233,7 +226,7 @@ hs = hannotate(ll_function, [int, int, int], policy=P_OOPSPEC) assert isinstance(hs, SomeLLAbstractConstant) assert hs.concretetype == lltype.Signed - assert len(hs.origins) == 2 + assert len(hs.origins) == 4 def test_some_more_list_operations(): def ll_function(x, y, index): @@ -244,7 +237,7 @@ hs = hannotate(ll_function, [int, int, int], policy=P_OOPSPEC) assert isinstance(hs, SomeLLAbstractConstant) assert hs.concretetype == lltype.Signed - assert len(hs.origins) == 2 + assert len(hs.origins) == 4 def test_simple_cast_pointer(): GCS1 = lltype.GcStruct('s1', ('x', lltype.Signed)) @@ -293,5 +286,4 @@ def test_hannotate_tl(): from pypy.jit import tl - - hannotate(tl.interp, [str, int]) + hannotate(tl.interp, [str, int], policy=P_OOPSPEC) From arigo at codespeak.net Sun Jan 29 16:19:24 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sun, 29 Jan 2006 16:19:24 +0100 (CET) Subject: [pypy-svn] r22846 - in pypy/dist/pypy: annotation jit translator/tool translator/tool/pygame Message-ID: <20060129151924.2C79E27B69@code1.codespeak.net> Author: arigo Date: Sun Jan 29 16:19:19 2006 New Revision: 22846 Modified: pypy/dist/pypy/annotation/model.py pypy/dist/pypy/jit/hintmodel.py pypy/dist/pypy/translator/tool/graphpage.py pypy/dist/pypy/translator/tool/pygame/drawgraph.py pypy/dist/pypy/translator/tool/pygame/graphdisplay.py Log: (arre, pedronis, arigo) Nice colors in the pygame flow graph viewer :-) Modified: pypy/dist/pypy/annotation/model.py ============================================================================== --- pypy/dist/pypy/annotation/model.py (original) +++ pypy/dist/pypy/annotation/model.py Sun Jan 29 16:19:19 2006 @@ -447,6 +447,7 @@ """The empty set. Instances are placeholders for objects that will never show up at run-time, e.g. elements of an empty list.""" immutable = True + annotationcolor = (160,160,160) def can_be_none(self): return False Modified: pypy/dist/pypy/jit/hintmodel.py ============================================================================== --- pypy/dist/pypy/jit/hintmodel.py (original) +++ pypy/dist/pypy/jit/hintmodel.py Sun Jan 29 16:19:19 2006 @@ -59,13 +59,24 @@ lst.append(s) return '<%s>' % (', '.join(lst),) + def annotationcolor(self): + """Compute the color of the variables with this annotation + for the pygame viewer + """ + for o in self.origins: + if not o.fixed: + return None + return (50,140,0) + annotationcolor = property(annotationcolor) + class SomeLLConcreteValue(SomeLLAbstractValue): - pass + annotationcolor = (0,100,0) class SomeLLAbstractVariable(SomeLLAbstractValue): pass class SomeLLAbstractContainer(SomeLLAbstractValue): + annotationcolor = (0,60,160) def __init__(self, contentdef): self.contentdef = contentdef Modified: pypy/dist/pypy/translator/tool/graphpage.py ============================================================================== --- pypy/dist/pypy/translator/tool/graphpage.py (original) +++ pypy/dist/pypy/translator/tool/graphpage.py Sun Jan 29 16:19:19 2006 @@ -143,7 +143,8 @@ if self.annotator: for var, s_value in self.annotator.bindings.items(): info = '%s: %s' % (var.name, s_value) - self.links[var.name] = info + annotationcolor = getattr(s_value, 'annotationcolor', None) + self.links[var.name] = info, annotationcolor self.current_value[var.name] = s_value self.caused_by[var.name] = ( self.annotator.binding_caused_by.get(var)) @@ -151,6 +152,10 @@ cause_history = ( self.annotator.binding_cause_history.get(var, [])) self.binding_history[var.name] = zip(history, cause_history) + + from pypy.jit import hintannotator + if isinstance(self.annotator, hintannotator.HintAnnotator): + return def visit(node): if isinstance(node, Block): @@ -165,11 +170,6 @@ #info = '(%s) %s' % (var.concretetype, info) info = str(var.concretetype) self.links[var.name] = info - - from pypy.jit import hintannotator - - if isinstance(self.annotator, hintannotator.HintAnnotator): - return for graph in graphs: traverse(visit, graph) Modified: pypy/dist/pypy/translator/tool/pygame/drawgraph.py ============================================================================== --- pypy/dist/pypy/translator/tool/pygame/drawgraph.py (original) +++ pypy/dist/pypy/translator/tool/pygame/drawgraph.py Sun Jan 29 16:19:19 2006 @@ -264,10 +264,17 @@ self.visibleedges = [] def wordcolor(self, word): + info = self.highlightwords[word] + if isinstance(info, tuple) and len(info) >= 2: + color = info[1] + else: + color = None + if color is None: + color = (128,0,0) if word == self.highlight_word: - return ((255,255,80), (128,0,0)) + return ((255,255,80), color) else: - return ((128,0,0), None) + return (color, None) def setscale(self, scale): scale = max(min(scale, self.SCALEMAX), self.SCALEMIN) Modified: pypy/dist/pypy/translator/tool/pygame/graphdisplay.py ============================================================================== --- pypy/dist/pypy/translator/tool/pygame/graphdisplay.py (original) +++ pypy/dist/pypy/translator/tool/pygame/graphdisplay.py Sun Jan 29 16:19:19 2006 @@ -451,6 +451,8 @@ word = self.viewer.at_position(pos) if word in self.layout.links: info = self.layout.links[word] + if isinstance(info, tuple): + info = info[0] self.setstatusbar(info) self.sethighlight(word) return From arigo at codespeak.net Sun Jan 29 17:18:16 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sun, 29 Jan 2006 17:18:16 +0100 (CET) Subject: [pypy-svn] r22847 - in pypy/dist/pypy/jit: . test Message-ID: <20060129161816.43BBC27B4C@code1.codespeak.net> Author: arigo Date: Sun Jan 29 17:18:14 2006 New Revision: 22847 Modified: pypy/dist/pypy/jit/hintmodel.py pypy/dist/pypy/jit/test/test_hint_annotation.py Log: (pedronis, arigo) Bouh. Removed the origin tracking through containers. This has the effect of producing the correct annotations for the TL, but we have to think more about whether it's the completely correct thing to do -- and also about direct_calls(). Modified: pypy/dist/pypy/jit/hintmodel.py ============================================================================== --- pypy/dist/pypy/jit/hintmodel.py (original) +++ pypy/dist/pypy/jit/hintmodel.py Sun Jan 29 17:18:14 2006 @@ -98,11 +98,13 @@ return set def reorigin(hs_v1, *deps_hs): + """Make a copy of hs_v1 with its origins removed and replaced by myorigin(). + Optionally, the origins of other annotations can also be added. + """ if isinstance(hs_v1, SomeLLAbstractConstant): deps_origins = [hs_dep.origins for hs_dep in deps_hs if isinstance(hs_dep, SomeLLAbstractConstant)] - d = newset(hs_v1.origins, - {getbookkeeper().myorigin(): True}, + d = newset({getbookkeeper().myorigin(): True}, *deps_origins) return SomeLLAbstractConstant(hs_v1.concretetype, d) else: @@ -165,7 +167,8 @@ hs_res = bookkeeper.annotator.recursivecall(fnobj.graph, bookkeeper.position_key, args_hs) - return reorigin(hs_res) + # for now, keep the origins of 'hs_res' in the new result: + return reorigin(hs_res, hs_res) def unary_char(hs_c1): d = setadd(hs_c1.origins, getbookkeeper().myorigin()) Modified: pypy/dist/pypy/jit/test/test_hint_annotation.py ============================================================================== --- pypy/dist/pypy/jit/test/test_hint_annotation.py (original) +++ pypy/dist/pypy/jit/test/test_hint_annotation.py Sun Jan 29 17:18:14 2006 @@ -226,7 +226,7 @@ hs = hannotate(ll_function, [int, int, int], policy=P_OOPSPEC) assert isinstance(hs, SomeLLAbstractConstant) assert hs.concretetype == lltype.Signed - assert len(hs.origins) == 4 + assert len(hs.origins) == 2 def test_some_more_list_operations(): def ll_function(x, y, index): @@ -237,7 +237,7 @@ hs = hannotate(ll_function, [int, int, int], policy=P_OOPSPEC) assert isinstance(hs, SomeLLAbstractConstant) assert hs.concretetype == lltype.Signed - assert len(hs.origins) == 4 + assert len(hs.origins) == 2 def test_simple_cast_pointer(): GCS1 = lltype.GcStruct('s1', ('x', lltype.Signed)) From gromit at codespeak.net Sun Jan 29 17:47:31 2006 From: gromit at codespeak.net (gromit at codespeak.net) Date: Sun, 29 Jan 2006 17:47:31 +0100 (CET) Subject: [pypy-svn] r22849 - in pypy/dist/pypy/rpython/rctypes: . test Message-ID: <20060129164731.1323A27B58@code1.codespeak.net> Author: gromit Date: Sun Jan 29 17:47:28 2006 New Revision: 22849 Modified: pypy/dist/pypy/rpython/rctypes/implementation.py pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py Log: ADD: We now can compile ctypes structures that manage their own memory. Modified: pypy/dist/pypy/rpython/rctypes/implementation.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/implementation.py (original) +++ pypy/dist/pypy/rpython/rctypes/implementation.py Sun Jan 29 17:47:28 2006 @@ -11,7 +11,11 @@ from pypy.annotation.model import SomeInteger, SomeCTypesObject, \ SomeString, SomeFloat from pypy.rpython.lltypesystem.lltype import Signed, SignedLongLong, \ - Unsigned, UnsignedLongLong, Char, Float + Unsigned, UnsignedLongLong, Char, Float, Ptr, GcStruct, \ + Void +from pypy.rpython.rmodel import Repr, IntegerRepr, inputconst +from pypy.rpython.error import TyperError + # ctypes_annotation_list contains various attributes that # are used by the pypy annotation. @@ -118,8 +122,12 @@ return SomeCTypesObject(cls) compute_annotation = classmethod(compute_annotation) - def specialize(self, hop): - raise NotImplementedError + def specialize( cls, highLevelOperation ): + ctypesStructureType = highLevelOperation.r_result.lowleveltype + return highLevelOperation.llops.genop( + "malloc", [ inputconst( Void, ctypesStructureType ) ], + highLevelOperation.r_result ) + specialize = classmethod(specialize) def compute_result_annotation(cls, *args_s): """ @@ -128,6 +136,21 @@ return SomeCTypesObject(cls,SomeCTypesObject.OWNSMEMORY) compute_result_annotation = classmethod(compute_result_annotation) + def createLowLevelRepresentation( rtyper, annotationObject ): + """ + Answer the correspondending low level object. + """ + if annotationObject.memorystate == annotationObject.OWNSMEMORY: + return CtypesMemoryOwningStructureRepresentation( + rtyper, annotationObject ) + elif annotationObject.memorystate == annotationObject.MEMORYALIAS: + return CtypesMemoryAliasStructureRepresentation( + rtyper, annotationObject ) + else: + raise TyperError( "Unkown memory state in %r" % annotationObject ) + createLowLevelRepresentation = staticmethod( createLowLevelRepresentation ) + + class RByrefObj(object): default_memorystate = SomeCTypesObject.MEMORYALIAS @@ -158,6 +181,9 @@ assert answer is cls return SomeCTypesObject(cls) answer.compute_result_annotation = classmethod(compute_result_annotation) + # We specialcased accessing pointers be getting their contents attribute + # because we can't use the memory state from 'cls'. + # So the obvious way to do it is obsolete (#o#). #o#answer._fields_def_ = {"contents": cls} answer.default_memorystate = SomeCTypesObject.MEMORYALIAS return answer @@ -202,3 +228,61 @@ return answer +class AbstractCtypesRepresentation( Repr ): + """ + The abstract base class of all ctypes low level representations. + """ + + +class AbstractCtypesStructureRepresentation( AbstractCtypesRepresentation ): + """ + The abstract base class of ctypes structures' low level representation. + """ + + def __init__( self, rtyper, annotationObject ): + # XXX This .ll_type may not work for pointers or structures + # conating structures + fields = [ ( name, ctypesType.ll_type ) + for name, ctypesType in annotationObject.knowntype._fields_ ] + self.lowleveltype = Ptr( + GcStruct( + 'CtypesStructure_%s' % + annotationObject.knowntype.__name__, *fields ) ) + + +class CtypesMemoryOwningStructureRepresentation( AbstractCtypesStructureRepresentation ): + """ + The lowlevel representation of a ctypes structure that owns its memory. + """ + + def rtype_setattr( self, highLevelOperation ): + #d#print highLevelOperation.args_v + #d#print highLevelOperation.args_r + #d#print highLevelOperation.args_s + highLevelOperation.genop( + "setfield", + highLevelOperation.inputargs( + *highLevelOperation.args_r[ :3 ] ) ) + + def rtype_getattr( self, highLevelOperation ): + return highLevelOperation.genop( + "getfield", + highLevelOperation.inputargs( + *highLevelOperation.args_r[ :2 ] ), + highLevelOperation.r_result ) + + + +class CtypesMemoryAliasStructureRepresentation( AbstractCtypesStructureRepresentation ): + """ + The lowlevel representation of a ctypes structure that is an alias to + someone else's memory. + """ + +class __extend__( SomeCTypesObject ): + def rtyper_makerepr( self, rtyper ): + return self.knowntype.createLowLevelRepresentation( rtyper, self ) + + def rtyper_makekey( self ): + return self.__class__, self.knowntype, self.memorystate + Modified: pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py Sun Jan 29 17:47:28 2006 @@ -1,10 +1,34 @@ import py.test from pypy.annotation.annrpython import RPythonAnnotator from pypy.translator.translator import TranslationContext -from pypy.translator.c.test.test_genc import compile +from pypy.translator.c.test.test_genc import compile, compile_db +#o#from pypy.translator.c import compile from pypy.translator.tool.cbuild import compile_c_module from pypy.annotation.model import SomeCTypesObject, SomeObject import sys + + +def compile(fn, argtypes, view=False): + from pypy.translator.c.database import LowLevelDatabase + from pypy.rpython.lltypesystem.lltype import pyobjectptr + t = TranslationContext() + a = t.buildannotator() + a.build_types(fn, argtypes) + t.buildrtyper().specialize() + if view: + t.view() + #t#backend_optimizations(t) + db = LowLevelDatabase(t) + entrypoint = db.get(pyobjectptr(fn)) + db.complete() + module = compile_db(db) + compiled_fn = getattr(module, entrypoint) + def checking_fn(*args, **kwds): + res = compiled_fn(*args, **kwds) + mallocs, frees = module.malloc_counters() + assert mallocs == frees + return res + return checking_fn try: import ctypes @@ -174,6 +198,15 @@ return p.x +def _py_test_compile_struct( p, x, y ): + p.x = x + p.y = y + + return p + +def py_test_compile_struct( x, y ): + return _py_test_compile_struct( tagpoint(), x, y ).x + class Test_rctypes: @@ -321,14 +354,34 @@ s = a.build_types(py_test_simple_ctypes_non_const,[]) assert s.knowntype == c_float - def x_test_specialize_struct(self): + def test_specialize_struct(self): t = TranslationContext() a = t.buildannotator() s = a.build_types(py_test_specialize_struct, []) # result should be an integer assert s.knowntype == int - t.buildrtyper().specialize() + try: + t.buildrtyper().specialize() + finally: + #d#t.view() + pass + + def test_specialize_struct_1(self): + t = TranslationContext() + a = t.buildannotator() + s = a.build_types( py_test_compile_struct, [ int, int ] ) #d#t.view() + try: + t.buildrtyper().specialize() + finally: + #d#t.view() + pass + + def test_compile_struct(self): + fn = compile( py_test_compile_struct, [ int, int ], True ) + res = fn( 42, -42 ) + assert res == 42 + class Test_array: @@ -357,13 +410,13 @@ a = t.buildannotator() s = a.build_types(py_test_annotate_pointer_content,[]) assert s.knowntype == int - #t#t.view() + #d#t.view() def test_annotate_array_slice_access(self): t = TranslationContext() a = t.buildannotator() s = a.build_types(py_test_annotate_array_slice_content,[]) - #t#t.view() + #d#t.view() #d#print "v90:", s, type(s) assert s.knowntype == list s.listdef.listitem.s_value.knowntype == int From arigo at codespeak.net Sun Jan 29 17:59:20 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sun, 29 Jan 2006 17:59:20 +0100 (CET) Subject: [pypy-svn] r22850 - in pypy/dist/pypy/objspace/flow: . test Message-ID: <20060129165920.06BCA27B64@code1.codespeak.net> Author: arigo Date: Sun Jan 29 17:59:17 2006 New Revision: 22850 Modified: pypy/dist/pypy/objspace/flow/model.py pypy/dist/pypy/objspace/flow/test/test_model.py Log: (arre, pedronis, arigo) A copygraph() function to make a copy of an existing flow graph. Modified: pypy/dist/pypy/objspace/flow/model.py ============================================================================== --- pypy/dist/pypy/objspace/flow/model.py (original) +++ pypy/dist/pypy/objspace/flow/model.py Sun Jan 29 17:59:17 2006 @@ -460,6 +460,67 @@ lst.append(link) return result +def copygraph(graph): + "Make a copy of a flow graph." + blockmap = {} + varmap = {} + + def copyvar(v): + if isinstance(v, Variable): + try: + return varmap[v] + except KeyError: + v2 = varmap[v] = Variable(v) + if hasattr(v, 'concretetype'): + v2.concretetype = v.concretetype + return v2 + else: + return v + + def copyblock(block): + newblock = Block([copyvar(v) for v in block.inputargs]) + if block.operations == (): + newblock.operations = () + else: + def copyoplist(oplist): + result = [] + for op in oplist: + copyop = SpaceOperation(op.opname, + [copyvar(v) for v in op.args], + copyvar(op.result)) + copyop.offset = op.offset + if hasattr(op, 'cleanup'): + copyop.cleanup = copyoplist(op.cleanup) + result.append(copyop) + return result + newblock.operations = copyoplist(block.operations) + newblock.exitswitch = copyvar(block.exitswitch) + newblock.exc_handler = block.exc_handler + return newblock + + for block in graph.iterblocks(): + blockmap[block] = copyblock(block) + + if graph.returnblock not in blockmap: + blockmap[graph.returnblock] = copyblock(graph.returnblock) + if graph.exceptblock not in blockmap: + blockmap[graph.exceptblock] = copyblock(graph.exceptblock) + + for block, newblock in blockmap.items(): + newlinks = [] + for link in block.exits: + newlink = link.copy(copyvar) + newlink.target = blockmap[link.target] + newlinks.append(newlink) + newblock.closeblock(*newlinks) + + newstartblock = blockmap[graph.startblock] + newstartblock.isstartblock = True + newgraph = FunctionGraph(graph.name, newstartblock) + newgraph.returnblock = blockmap[graph.returnblock] + newgraph.exceptblock = blockmap[graph.exceptblock] + return newgraph + def checkgraph(graph): "Check the consistency of a flow graph." if not __debug__: Modified: pypy/dist/pypy/objspace/flow/test/test_model.py ============================================================================== --- pypy/dist/pypy/objspace/flow/test/test_model.py (original) +++ pypy/dist/pypy/objspace/flow/test/test_model.py Sun Jan 29 17:59:17 2006 @@ -48,6 +48,10 @@ def test_checkgraph(): checkgraph(graph) +def test_copygraph(): + graph2 = copygraph(graph) + checkgraph(graph2) + def test_graphattributes(): assert graph.startblock is pieces.startblock assert graph.returnblock is pieces.headerblock.exits[0].target From arigo at codespeak.net Sun Jan 29 18:45:58 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sun, 29 Jan 2006 18:45:58 +0100 (CET) Subject: [pypy-svn] r22852 - in pypy/dist/pypy/jit: . test Message-ID: <20060129174558.814B927B68@code1.codespeak.net> Author: arigo Date: Sun Jan 29 18:45:55 2006 New Revision: 22852 Modified: pypy/dist/pypy/jit/hintannotator.py pypy/dist/pypy/jit/hintbookkeeper.py pypy/dist/pypy/jit/hintmodel.py pypy/dist/pypy/jit/test/test_hint_annotation.py Log: (arre, pedronis, arigo) Started on specialization of graphs in the hint annotator. Modified: pypy/dist/pypy/jit/hintannotator.py ============================================================================== --- pypy/dist/pypy/jit/hintannotator.py (original) +++ pypy/dist/pypy/jit/hintannotator.py Sun Jan 29 18:45:55 2006 @@ -5,10 +5,15 @@ class HintAnnotator(RPythonAnnotator): - def __init__(self, translator, policy=None): + def __init__(self, translator=None, policy=None): RPythonAnnotator.__init__(self, translator, policy=policy) self.bookkeeper = HintBookkeeper(self) # XXX + def build_types(self, origgraph, input_args_hs): + desc = self.bookkeeper.getdesc(origgraph) + flowgraph = desc.specialize(input_args_hs) + return self.build_graph_types(flowgraph, input_args_hs) + def consider_op_malloc(self, hs_TYPE): TYPE = hs_TYPE.const vstructdef = self.bookkeeper.getvirtualcontainerdef(TYPE) Modified: pypy/dist/pypy/jit/hintbookkeeper.py ============================================================================== --- pypy/dist/pypy/jit/hintbookkeeper.py (original) +++ pypy/dist/pypy/jit/hintbookkeeper.py Sun Jan 29 18:45:55 2006 @@ -1,25 +1,74 @@ from pypy.tool.tls import tlsobject +from pypy.objspace.flow.model import copygraph TLS = tlsobject() + +class GraphDesc(object): + + def __init__(self, bookkeeper, origgraph): + self.bookkeeper = bookkeeper + self.origgraph = origgraph + self._cache = {} + + def specialize(self, input_args_hs): + from pypy.jit import hintmodel + # get the specialized graph -- for now, no specialization + graph = self.cachedgraph(None) + + # modify input_args_hs in-place to change their origin + for i in range(len(input_args_hs)): + old = self.bookkeeper.enter((graph, i)) + try: + input_args_hs[i] = hintmodel.reorigin(input_args_hs[i]) + finally: + self.bookkeeper.leave(old) + + return graph + + def cachedgraph(self, key, alt_name=None): + try: + return self._cache[key] + except KeyError: + graph = copygraph(self.origgraph) + if alt_name is not None: + graph.name = alt_name + self._cache[key] = graph + self.bookkeeper.annotator.translator.graphs.append(graph) + return graph + + class HintBookkeeper(object): def __init__(self, hannotator): self.pending_specializations = [] self.originflags = {} self.virtual_containers = {} + self.descs = {} self.annotator = hannotator + def getdesc(self, graph): + try: + return self.descs[graph] + except KeyError: + self.descs[graph] = desc = GraphDesc(self, graph) + return desc + def enter(self, position_key): """Start of an operation. The operation is uniquely identified by the given key.""" + res = getattr(self, 'position_key', None) self.position_key = position_key TLS.bookkeeper = self + return res - def leave(self): + def leave(self, old=None): """End of an operation.""" - del TLS.bookkeeper - del self.position_key + if old is None: + del TLS.bookkeeper + del self.position_key + else: + self.position_key = old def myorigin(self): try: Modified: pypy/dist/pypy/jit/hintmodel.py ============================================================================== --- pypy/dist/pypy/jit/hintmodel.py (original) +++ pypy/dist/pypy/jit/hintmodel.py Sun Jan 29 18:45:55 2006 @@ -164,11 +164,23 @@ # normal call if not hasattr(fnobj, 'graph'): raise NotImplementedError("XXX call to externals or primitives") - hs_res = bookkeeper.annotator.recursivecall(fnobj.graph, + desc = bookkeeper.getdesc(fnobj.graph) + input_args_hs = list(args_hs) + graph = desc.specialize(input_args_hs) + hs_res = bookkeeper.annotator.recursivecall(graph, bookkeeper.position_key, - args_hs) - # for now, keep the origins of 'hs_res' in the new result: - return reorigin(hs_res, hs_res) + input_args_hs) + # look on which input args the hs_res result depends on + if isinstance(hs_res, SomeLLAbstractConstant): + deps_hs = [] + for hs_inputarg, hs_arg in zip(input_args_hs, args_hs): + if isinstance(hs_inputarg, SomeLLAbstractConstant): + assert len(hs_inputarg.origins) == 1 + [o] = hs_inputarg.origins.keys() + if o in hs_res.origins: + deps_hs.append(hs_arg) + hs_res = reorigin(hs_res, *deps_hs) + return hs_res def unary_char(hs_c1): d = setadd(hs_c1.origins, getbookkeeper().myorigin()) Modified: pypy/dist/pypy/jit/test/test_hint_annotation.py ============================================================================== --- pypy/dist/pypy/jit/test/test_hint_annotation.py (original) +++ pypy/dist/pypy/jit/test/test_hint_annotation.py Sun Jan 29 18:45:55 2006 @@ -20,11 +20,11 @@ rtyper.specialize() graph1 = graphof(t, func) # build hint annotator types - hannotator = HintAnnotator(t, policy=policy) - t.annotator = hannotator # XXX? - hs = hannotator.build_graph_types(graph1, [SomeLLAbstractConstant(v.concretetype, - {OriginFlags(): True}) - for v in graph1.getargs()]) + hannotator = HintAnnotator(policy=policy) + hs = hannotator.build_types(graph1, [SomeLLAbstractConstant(v.concretetype, + {OriginFlags(): True}) + for v in graph1.getargs()]) + t = hannotator.translator #t.view() if annotator: return hs, hannotator @@ -209,14 +209,14 @@ assert len(hs.origins) == 3 def test_simple_call(): - def ll2(x, y): + def ll2(x, y, z): return x + (y + 42) def ll1(x, y, z): - return ll2(x, y - z) + return ll2(x, y - z, x + y + z) hs = hannotate(ll1, [int, int, int]) assert isinstance(hs, SomeLLAbstractConstant) assert hs.concretetype == lltype.Signed - assert len(hs.origins) == 7 + assert len(hs.origins) == 5 def test_simple_list_operations(): def ll_function(x, y, index): From arigo at codespeak.net Sun Jan 29 19:23:42 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sun, 29 Jan 2006 19:23:42 +0100 (CET) Subject: [pypy-svn] r22855 - pypy/dist/pypy/objspace/flow Message-ID: <20060129182342.3137D27B69@code1.codespeak.net> Author: arigo Date: Sun Jan 29 19:23:38 2006 New Revision: 22855 Modified: pypy/dist/pypy/objspace/flow/model.py Log: Forgot to check this in. Modified: pypy/dist/pypy/objspace/flow/model.py ============================================================================== --- pypy/dist/pypy/objspace/flow/model.py (original) +++ pypy/dist/pypy/objspace/flow/model.py Sun Jan 29 19:23:38 2006 @@ -519,6 +519,8 @@ newgraph = FunctionGraph(graph.name, newstartblock) newgraph.returnblock = blockmap[graph.returnblock] newgraph.exceptblock = blockmap[graph.exceptblock] + for key, value in graph.__dict__.items(): + newgraph.__dict__.setdefault(key, value) return newgraph def checkgraph(graph): From pedronis at codespeak.net Sun Jan 29 19:31:38 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Sun, 29 Jan 2006 19:31:38 +0100 (CET) Subject: [pypy-svn] r22856 - pypy/dist/pypy/jit/test Message-ID: <20060129183138.5583327B69@code1.codespeak.net> Author: pedronis Date: Sun Jan 29 19:31:36 2006 New Revision: 22856 Modified: pypy/dist/pypy/jit/test/test_hint_annotation.py Log: test for fixed version of function for which some result somewhere is fixed Modified: pypy/dist/pypy/jit/test/test_hint_annotation.py ============================================================================== --- pypy/dist/pypy/jit/test/test_hint_annotation.py (original) +++ pypy/dist/pypy/jit/test/test_hint_annotation.py Sun Jan 29 19:31:36 2006 @@ -21,6 +21,7 @@ graph1 = graphof(t, func) # build hint annotator types hannotator = HintAnnotator(policy=policy) + hannotator.base_translator = t hs = hannotator.build_types(graph1, [SomeLLAbstractConstant(v.concretetype, {OriginFlags(): True}) for v in graph1.getargs()]) @@ -283,6 +284,30 @@ g1 = graphof(ha.translator, ll1) hs_n = ha.binding(g1.getargs()[0]) assert hs_n.origins.keys()[0].fixed + +def INPROG_test_simple_fixed_call(): + def ll_help(cond, x, y): + if cond: + z = x+y + else: + z = x-y + return z + def ll_function(cond, x,y, y1): + z1 = ll_help(cond, x, y1) + z = ll_help(cond, x, y) + z = hint(z, concrete=True) + return z + hs, ha = hannotate(ll_function, [bool, int, int, int], annotator=True) + assert isinstance(hs, SomeLLConcreteValue) + assert hs.concretetype == lltype.Signed + ll_help_graph = graphof(ha.base_translator, ll_help) + gdesc = ha.bookkeeper.getdesc(ll_help_graph) + for o in ha.binding(gdesc._cache[None].getreturnvar()).origins: + assert not o.fixed + assert len(gdesc._cache) == 2 + for o in ha.binding(gdesc._cache['fixed'].getreturnvar()).origins: + assert o.fixed + def test_hannotate_tl(): from pypy.jit import tl From arigo at codespeak.net Sun Jan 29 19:36:00 2006 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sun, 29 Jan 2006 19:36:00 +0100 (CET) Subject: [pypy-svn] r22857 - in pypy/dist/pypy/jit: . test Message-ID: <20060129183600.BCB7B27B69@code1.codespeak.net> Author: arigo Date: Sun Jan 29 19:35:58 2006 New Revision: 22857 Modified: pypy/dist/pypy/jit/hintbookkeeper.py pypy/dist/pypy/jit/hintmodel.py pypy/dist/pypy/jit/test/test_hint_annotation.py Log: (pedronis, arre, arigo) Make two versions of each called graph: one normal, and one identical but used for the case where the result is then fixed in the caller. Modified: pypy/dist/pypy/jit/hintbookkeeper.py ============================================================================== --- pypy/dist/pypy/jit/hintbookkeeper.py (original) +++ pypy/dist/pypy/jit/hintbookkeeper.py Sun Jan 29 19:35:58 2006 @@ -11,10 +11,10 @@ self.origgraph = origgraph self._cache = {} - def specialize(self, input_args_hs): + def specialize(self, input_args_hs, key=None, alt_name=None): from pypy.jit import hintmodel # get the specialized graph -- for now, no specialization - graph = self.cachedgraph(None) + graph = self.cachedgraph(key, alt_name) # modify input_args_hs in-place to change their origin for i in range(len(input_args_hs)): Modified: pypy/dist/pypy/jit/hintmodel.py ============================================================================== --- pypy/dist/pypy/jit/hintmodel.py (original) +++ pypy/dist/pypy/jit/hintmodel.py Sun Jan 29 19:35:58 2006 @@ -23,6 +23,7 @@ class OriginFlags(object): fixed = False + read_positions = None def __repr__(self): if self.fixed: @@ -31,6 +32,20 @@ s = "" return "<%sorigin>" % (s,) + def read_fixed(self): + if self.read_positions is None: + self.read_positions = {} + self.read_positions[getbookkeeper().position_key] = True + return self.fixed + + def set_fixed(self): + if not self.fixed: + self.fixed = True + if self.read_positions: + annotator = getbookkeeper().annotator + for p in self.read_positions: + annotator.reflowfromposition(p) + class SomeLLAbstractValue(annmodel.SomeObject): def __init__(self, T): @@ -125,7 +140,7 @@ return SomeLLAbstractVariable(hs_c1.concretetype) assert hs_flags.const['concrete'] for o in hs_c1.origins: - o.fixed = True + o.set_fixed() return SomeLLConcreteValue(hs_c1.concretetype) def getfield(hs_c1, hs_fieldname): @@ -165,8 +180,13 @@ if not hasattr(fnobj, 'graph'): raise NotImplementedError("XXX call to externals or primitives") desc = bookkeeper.getdesc(fnobj.graph) + key = None + alt_name = None + if bookkeeper.myorigin().read_fixed(): + key = 'fixed' + alt_name = fnobj.graph.name + '_HFixed' input_args_hs = list(args_hs) - graph = desc.specialize(input_args_hs) + graph = desc.specialize(input_args_hs, key=key, alt_name=alt_name) hs_res = bookkeeper.annotator.recursivecall(graph, bookkeeper.position_key, input_args_hs) @@ -179,6 +199,8 @@ [o] = hs_inputarg.origins.keys() if o in hs_res.origins: deps_hs.append(hs_arg) + if key == 'fixed': + deps_hs.append(hs_res) hs_res = reorigin(hs_res, *deps_hs) return hs_res Modified: pypy/dist/pypy/jit/test/test_hint_annotation.py ============================================================================== --- pypy/dist/pypy/jit/test/test_hint_annotation.py (original) +++ pypy/dist/pypy/jit/test/test_hint_annotation.py Sun Jan 29 19:35:58 2006 @@ -285,7 +285,7 @@ hs_n = ha.binding(g1.getargs()[0]) assert hs_n.origins.keys()[0].fixed -def INPROG_test_simple_fixed_call(): +def test_simple_fixed_call(): def ll_help(cond, x, y): if cond: z = x+y From mwh at codespeak.net Sun Jan 29 19:46:38 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Sun, 29 Jan 2006 19:46:38 +0100 (CET) Subject: [pypy-svn] r22858 - pypy/dist/pypy/rpython Message-ID: <20060129184638.6B6F527B64@code1.codespeak.net> Author: mwh Date: Sun Jan 29 19:46:36 2006 New Revision: 22858 Modified: pypy/dist/pypy/rpython/annlowlevel.py pypy/dist/pypy/rpython/rspecialcase.py Log: a way for a function to have special things happen when you annotate/rtype it begin called in a low level helper Modified: pypy/dist/pypy/rpython/annlowlevel.py ============================================================================== --- pypy/dist/pypy/rpython/annlowlevel.py (original) +++ pypy/dist/pypy/rpython/annlowlevel.py Sun Jan 29 19:46:36 2006 @@ -44,6 +44,10 @@ allow_someobjects = False def default_specialize(pol, funcdesc, args_s): + if hasattr(funcdesc, 'pyobj') and hasattr(funcdesc.pyobj, 'llresult'): + # XXX bug mwh to write some tests for this stuff + funcdesc.overridden = True + return annmodel.lltype_to_annotation(funcdesc.pyobj.llresult) key = [] new_args_s = [] for s_obj in args_s: Modified: pypy/dist/pypy/rpython/rspecialcase.py ============================================================================== --- pypy/dist/pypy/rpython/rspecialcase.py (original) +++ pypy/dist/pypy/rpython/rspecialcase.py Sun Jan 29 19:46:36 2006 @@ -7,6 +7,9 @@ if len(s_pbc.descriptions) != 1: raise TyperError("not monomorphic call_specialcase") desc, = s_pbc.descriptions + if hasattr(desc.pyobj, 'compute_ll_ops'): + # XXX bug mwh to write some tests for this stuff + return desc.pyobj.compute_ll_ops(hop) tag = desc.pyobj._annspecialcase_ if not tag.startswith("override:"): raise TyperError("call_specialcase only supports 'override:' functions") From mwh at codespeak.net Sun Jan 29 19:47:17 2006 From: mwh at codespeak.net (mwh at codespeak.net) Date: Sun, 29 Jan 2006 19:47:17 +0100 (CET) Subject: [pypy-svn] r22859 - in pypy/dist/pypy/rpython/memory: . test Message-ID: <20060129184717.EF04927B64@code1.codespeak.net> Author: mwh Date: Sun Jan 29 19:47:15 2006 New Revision: 22859 Modified: pypy/dist/pypy/rpython/memory/gctransform.py pypy/dist/pypy/rpython/memory/test/test_gctransform.py Log: some very nearly insane code to generate the graph of the deallocator for an lltype. Modified: pypy/dist/pypy/rpython/memory/gctransform.py ============================================================================== --- pypy/dist/pypy/rpython/memory/gctransform.py (original) +++ pypy/dist/pypy/rpython/memory/gctransform.py Sun Jan 29 19:47:15 2006 @@ -1,5 +1,6 @@ from pypy.rpython.lltypesystem import lltype -from pypy.objspace.flow.model import SpaceOperation, Variable, Constant, c_last_exception +from pypy.objspace.flow.model import SpaceOperation, Variable, Constant, \ + c_last_exception, FunctionGraph, Block, Link, checkgraph from pypy.translator.unsimplify import insert_empty_block from pypy.rpython import rmodel import sets @@ -168,3 +169,47 @@ result.concretetype = lltype.Void return [SpaceOperation("gc_pop_alive_pyobj", [var], result)] + # ---------------------------------------------------------------- + + def _deallocator_body_for_type(self, v, TYPE, depth=1): + if isinstance(TYPE, lltype.Array): + + inner = list(self._deallocator_body_for_type('v_%i'%depth, TYPE.OF, depth+1)) + if inner: + yield ' '*depth + 'i_%d = 0'%(depth,) + yield ' '*depth + 'l_%d = len(%s)'%(depth, v) + yield ' '*depth + 'while i_%d < l_%d:'%(depth, depth) + yield ' '*depth + ' v_%d = %s[i_%d]'%(depth, v, depth) + for line in inner: + yield line + yield ' '*depth + ' i_%d += 1'%(depth,) + elif isinstance(TYPE, lltype.Struct): + for name in TYPE._names: + inner = list(self._deallocator_body_for_type( + v + '_' + name, TYPE._flds[name], depth)) + if inner: + yield ' '*depth + v + '_' + name + ' = ' + v + '.' + name + for line in inner: + yield line + elif isinstance(TYPE, lltype.Ptr): + yield ' '*depth + 'pop_alive(%s)'%v + + def deallocation_graph_for_type(self, translator, TYPE, var): + def compute_ll_ops(hop): + hop.llops.extend(self.pop_alive(hop.args_v[1])) + return hop.inputconst(hop.r_result.lowleveltype, hop.s_result.const) + def pop_alive(var): + pass + pop_alive.compute_ll_ops = compute_ll_ops + pop_alive.llresult = lltype.Void + + body = '\n'.join(self._deallocator_body_for_type('v', TYPE)) + if not body: + return + src = 'def deallocator(v):\n' + body + d = {'pop_alive':pop_alive} + exec src in d + this = d['deallocator'] + g = translator.rtyper.annotate_helper(this, [lltype.Ptr(TYPE)]) + translator.rtyper.specialize_more_blocks() + return g Modified: pypy/dist/pypy/rpython/memory/test/test_gctransform.py ============================================================================== --- pypy/dist/pypy/rpython/memory/test/test_gctransform.py (original) +++ pypy/dist/pypy/rpython/memory/test/test_gctransform.py Sun Jan 29 19:47:15 2006 @@ -240,3 +240,40 @@ return 0 return 1 t = rtype_and_transform(f, [], gctransform.GCTransformer) + +# ---------------------------------------------------------------------- + +def make_deallocator(TYPE, view=False): + def f(): + pass + t = TranslationContext() + t.buildannotator().build_types(f, []) + t.buildrtyper().specialize(t) + + transformer = gctransform.GCTransformer() + v = Variable() + v.concretetype = TYPE + graph = transformer.deallocation_graph_for_type(t, TYPE, v) + if view: + t.view() + return graph + +def dont_test_deallocator_simple(): + S = lltype.GcStruct("S", ('x', lltype.Signed)) + dgraph = make_deallocator(S) + assert dgraph is None + +def test_deallocator_less_simple(): + TPtr = lltype.Ptr(lltype.GcStruct("T", ('a', lltype.Signed))) + S = lltype.GcStruct( + "S", + ('x', lltype.Signed), + ('y', TPtr), + ('z', TPtr), + ) + dgraph = make_deallocator(S) + +def test_deallocator_less_simple2(): + S = lltype.GcArray(lltype.Ptr(lltype.GcStruct("S", ('x', lltype.Signed)))) + dgraph = make_deallocator(S) + From tismer at codespeak.net Sun Jan 29 19:55:00 2006 From: tismer at codespeak.net (tismer at codespeak.net) Date: Sun, 29 Jan 2006 19:55:00 +0100 (CET) Subject: [pypy-svn] r22860 - pypy/dist/pypy/translator/goal Message-ID: <20060129185500.E3E5F27B69@code1.codespeak.net> Author: tismer Date: Sun Jan 29 19:54:58 2006 New Revision: 22860 Modified: pypy/dist/pypy/translator/goal/targetpypystandalone.py Log: automatically include the stackless module if --stackless is selected Modified: pypy/dist/pypy/translator/goal/targetpypystandalone.py ============================================================================== --- pypy/dist/pypy/translator/goal/targetpypystandalone.py (original) +++ pypy/dist/pypy/translator/goal/targetpypystandalone.py Sun Jan 29 19:54:58 2006 @@ -84,6 +84,8 @@ usemodules = [] if options.thread: usemodules.append('thread') + if options.stackless: + usemodules.append('stackless') space = StdObjSpace(nofaking=True, compiler="ast", # interpreter/astcompiler From tismer at codespeak.net Sun Jan 29 20:42:57 2006 From: tismer at codespeak.net (tismer at codespeak.net) Date: Sun, 29 Jan 2006 20:42:57 +0100 (CET) Subject: [pypy-svn] r22861 - pypy/dist/pypy/module/stackless Message-ID: <20060129194257.4AEF227B69@code1.codespeak.net> Author: tismer Date: Sun Jan 29 20:42:55 2006 New Revision: 22861 Modified: pypy/dist/pypy/module/stackless/interp_coroutine.py Log: temporary check-in, looking for an endless loop in parent finding Modified: pypy/dist/pypy/module/stackless/interp_coroutine.py ============================================================================== --- pypy/dist/pypy/module/stackless/interp_coroutine.py (original) +++ pypy/dist/pypy/module/stackless/interp_coroutine.py Sun Jan 29 20:42:55 2006 @@ -20,6 +20,18 @@ self.del_first = None self.del_last = None + def cast_current(self, current): + if self.current is not current: + c = self.current + c.frame, current.frame = current.frame, c.frame + self.current = current + + def cast_main(self, main): + if self.main is not main: + c = self.main + c.frame, main.frame = main.frame, c.frame + self.main = main + costate = None class CoroutineDamage(SystemError): @@ -48,13 +60,6 @@ self.parent = costate.current self.thunk = None - def _get_current(self): - # must be overridden for every distinguished subclass - return costate.current - def _set_current(self, new): - # must be overridden for every distinguished subclass - costate.current = new - def bind(self, thunk): if self.frame is not None: raise CoroutineDamage @@ -89,11 +94,11 @@ if costate.things_to_do: do_things_to_do(self) - def _update_state(self, new): - costate.last = self._get_current() - self._set_current(new) + def _update_state(new): + costate.last, costate.current = costate.current, new frame, new.frame = new.frame, None return frame + _update_state = staticmethod(_update_state) def kill(self): # if costate.current is self: @@ -130,8 +135,13 @@ def is_zombie(self): return self.frame is not None and check_for_zombie(self) - def get_current(self): - return self._get_current() + def getcurrent(): + return costate.current + getcurrent = staticmethod(getcurrent) + + def getmain(): + return costate.main + getmain = staticmethod(getmain) def check_for_zombie(self): @@ -196,13 +206,6 @@ Coroutine.__init__(self) self.flags = 0 - def _get_current(self): - # must be overridden for every distinguished subclass - return appcostate.current - def _set_current(self, new): - # must be overridden for every distinguished subclass - appcostate.current = new - def descr_method__new__(space, w_subtype): co = space.allocate_instance(AppCoroutine, w_subtype) AppCoroutine.__init__(co) @@ -215,18 +218,24 @@ raise OperationError(space.w_ValueError, space.wrap( "cannot bind a bound Coroutine")) thunk = _AppThunk(space, w_func, __args__) + costate.current = appcostate.current self.bind(thunk) + appcostate.current = costate.current def w_switch(self): space = self.space if self.frame is None: raise OperationError(space.w_ValueError, space.wrap( "cannot switch to an unbound Coroutine")) + costate.current = appcostate.current self.switch() + appcostate.current = self ret, appcostate.tempval = appcostate.tempval, space.w_None return ret def w_kill(self): + if appcostate.current is self: + costate.current = self self.kill() def __del__(self): @@ -240,9 +249,15 @@ self.set_is_zombie(True) self.space.userdel(self) - def get_current(space): - return space.wrap(self._get_current()) - get_current = staticmethod(get_current) + def getcurrent(space): + costate.cast_current(appcostate.current) + return space.wrap(appcostate.current) + getcurrent = staticmethod(getcurrent) + + def getmain(space): + costate.cast_main(appcostate.main) + return space.wrap(appcostate.main) + getmain = staticmethod(getmain) # _mixin_ did not work @@ -267,7 +282,8 @@ def post_install(module): appcostate.post_install(module.space) - makeStaticMethod(module, 'Coroutine', 'get_current') + makeStaticMethod(module, 'Coroutine', 'getcurrent') + makeStaticMethod(module, 'Coroutine', 'getmain') # space.appexec("""() : @@ -280,12 +296,13 @@ switch = interp2app(AppCoroutine.w_switch), kill = interp2app(AppCoroutine.w_kill), is_zombie = GetSetProperty(AppCoroutine.w_get_is_zombie, doc=AppCoroutine.get_is_zombie.__doc__), - get_current = interp2app(AppCoroutine.get_current), + getcurrent = interp2app(AppCoroutine.getcurrent), + getmain = interp2app(AppCoroutine.getmain), ) class AppCoState(object): def __init__(self): - self.current = AppCoroutine() + self.current = self.main = AppCoroutine() def post_install(self, space): appcostate.current.space = space From cfbolz at codespeak.net Mon Jan 30 22:30:44 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Mon, 30 Jan 2006 22:30:44 +0100 (CET) Subject: [pypy-svn] r22884 - pypy/dist/pypy/rpython/rctypes/test Message-ID: <20060130213044.E227510053@code0.codespeak.net> Author: cfbolz Date: Mon Jan 30 22:30:43 2006 New Revision: 22884 Modified: pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py Log: make test use absolute paths to the testfile -- necessary when running all tests together Modified: pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py ============================================================================== --- pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py (original) +++ pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py Mon Jan 30 22:30:43 2006 @@ -7,6 +7,7 @@ from pypy.annotation.model import SomeCTypesObject, SomeObject import sys +thisdir = py.magic.autopath().dirpath() def compile(fn, argtypes, view=False): from pypy.translator.c.database import LowLevelDatabase @@ -61,12 +62,12 @@ _fields_ = [("x", c_int), ("y", c_int)] # compile and load our local test C file -compile_c_module([py.path.local("_rctypes_test.c")], "_rctypes_test") +compile_c_module([thisdir.join("_rctypes_test.c")], "_rctypes_test") if sys.platform == "win32": _rctypes_test = cdll.LoadLibrary("_rctypes_test.pyd") else: - _rctypes_test = cdll.LoadLibrary("_rctypes_test.so") + _rctypes_test = cdll.LoadLibrary(str(thisdir.join("_rctypes_test.so"))) # _testfunc_byval testfunc_byval = _rctypes_test._testfunc_byval @@ -247,7 +248,7 @@ if sys.platform == "win32": dll = cdll.LoadLibrary("_rctypes_test.pyd") else: - dll = cdll.LoadLibrary("_rctypes_test.so") + dll = cdll.LoadLibrary(str(thisdir.join("_rctypes_test.so"))) in_point = tagpoint() in_point.x = 42 in_point.y = 17 @@ -369,7 +370,7 @@ def test_specialize_struct_1(self): t = TranslationContext() a = t.buildannotator() - s = a.build_types( py_test_compile_struct, [ int, int ] ) + s = a.build_types(py_test_compile_struct, [int, int]) #d#t.view() try: t.buildrtyper().specialize() @@ -378,7 +379,7 @@ pass def test_compile_struct(self): - fn = compile( py_test_compile_struct, [ int, int ], True ) + fn = compile(py_test_compile_struct, [int, int]) res = fn( 42, -42 ) assert res == 42 @@ -387,13 +388,13 @@ def test_annotate_array(self): a = RPythonAnnotator() - s = a.build_types(py_test_annotate_array,[]) + s = a.build_types(py_test_annotate_array, []) assert s.knowntype == c_int_10 def test_annotate_array_access(self): t = TranslationContext() a = t.buildannotator() - s = a.build_types(py_test_annotate_array_content,[]) + s = a.build_types(py_test_annotate_array_content, []) assert s.knowntype == int #d#t.view() @@ -408,14 +409,14 @@ """ t = TranslationContext() a = t.buildannotator() - s = a.build_types(py_test_annotate_pointer_content,[]) + s = a.build_types(py_test_annotate_pointer_content, []) assert s.knowntype == int #d#t.view() def test_annotate_array_slice_access(self): t = TranslationContext() a = t.buildannotator() - s = a.build_types(py_test_annotate_array_slice_content,[]) + s = a.build_types(py_test_annotate_array_slice_content, []) #d#t.view() #d#print "v90:", s, type(s) assert s.knowntype == list @@ -424,7 +425,7 @@ def test_annotate_array_access_variable(self): t = TranslationContext() a = t.buildannotator() - s = a.build_types(py_test_annotate_array_content_variable_index,[]) + s = a.build_types(py_test_annotate_array_content_variable_index, []) assert s.knowntype == int #t#t.view() From auc at codespeak.net Tue Jan 31 11:54:49 2006 From: auc at codespeak.net (auc at codespeak.net) Date: Tue, 31 Jan 2006 11:54:49 +0100 (CET) Subject: [pypy-svn] r22886 - pypy/dist/pypy/lib/logic/basic_store Message-ID: <20060131105449.8BD6C1008C@code0.codespeak.net> Author: auc Date: Tue Jan 31 11:54:26 2006 New Revision: 22886 Modified: pypy/dist/pypy/lib/logic/basic_store/test_unification.py pypy/dist/pypy/lib/logic/basic_store/unification.py Log: * unification status update * test for threads waitind on unbound variable Modified: pypy/dist/pypy/lib/logic/basic_store/test_unification.py ============================================================================== --- pypy/dist/pypy/lib/logic/basic_store/test_unification.py (original) +++ pypy/dist/pypy/lib/logic/basic_store/test_unification.py Tue Jan 31 11:54:26 2006 @@ -188,6 +188,27 @@ assert (t2.raised and not t1.raised) or \ (t1.raised and not t2.raised) + def test_threads_waiting_for_unbound_var(self): + import time + start_time = time.time() + + def wait_on_unbound(thread, var, start_time): + thread.val = var.get() + thread.waited = time.time() - start_time + + x = u.var('x') + t1, t2 = (FunThread(wait_on_unbound, x, start_time), + FunThread(wait_on_unbound, x, start_time)) + t1.start() + t2.start() + time.sleep(1) + u.bind(x, 42) + t1.join() + t2.join() + assert t1.val == 42 + assert t2.val == 42 + assert t1.waited > 1 + assert t2.waited > 1 def test_set_var_domain(self): x = u.var('x') Modified: pypy/dist/pypy/lib/logic/basic_store/unification.py ============================================================================== --- pypy/dist/pypy/lib/logic/basic_store/unification.py (original) +++ pypy/dist/pypy/lib/logic/basic_store/unification.py Tue Jan 31 11:54:26 2006 @@ -112,11 +112,7 @@ # * understand this : # http://www.mozart-oz.org/papers/abstracts/ProgrammingConstraintServices.html # * support '_' as shown above -# * turn Var into some dataflow-ish thing (as far as Python allows) -# * ensure that the store supports concurrent access -# (using the implicit blocking provided by dataflow vars) # * add entailment checks -# * add constraints support import threading From auc at codespeak.net Tue Jan 31 12:03:33 2006 From: auc at codespeak.net (auc at codespeak.net) Date: Tue, 31 Jan 2006 12:03:33 +0100 (CET) Subject: [pypy-svn] r22887 - pypy/dist/pypy/lib/logic/basic_store Message-ID: <20060131110333.74A831008E@code0.codespeak.net> Author: auc Date: Tue Jan 31 12:03:32 2006 New Revision: 22887 Modified: pypy/dist/pypy/lib/logic/basic_store/test_unification.py Log: more carefully assert timings Modified: pypy/dist/pypy/lib/logic/basic_store/test_unification.py ============================================================================== --- pypy/dist/pypy/lib/logic/basic_store/test_unification.py (original) +++ pypy/dist/pypy/lib/logic/basic_store/test_unification.py Tue Jan 31 12:03:32 2006 @@ -4,6 +4,9 @@ from py.test import raises, skip from threading import Thread +def near(v1, v2, err): + return abs(v1 - v2) < err + class FunThread(Thread): def __init__(self, fun, *args): @@ -207,8 +210,8 @@ t2.join() assert t1.val == 42 assert t2.val == 42 - assert t1.waited > 1 - assert t2.waited > 1 + assert near(t1.waited, 1, .1) + assert near(t2.waited, 1, .1) def test_set_var_domain(self): x = u.var('x') From ericvrp at codespeak.net Tue Jan 31 13:18:04 2006 From: ericvrp at codespeak.net (ericvrp at codespeak.net) Date: Tue, 31 Jan 2006 13:18:04 +0100 (CET) Subject: [pypy-svn] r22888 - in pypy/dist/pypy: rpython/raisingops translator/backendopt translator/c/src translator/c/test Message-ID: <20060131121804.83ECD10093@code0.codespeak.net> Author: ericvrp Date: Tue Jan 31 13:18:00 2006 New Revision: 22888 Modified: pypy/dist/pypy/rpython/raisingops/raisingops.py pypy/dist/pypy/translator/backendopt/all.py pypy/dist/pypy/translator/backendopt/raisingop2direct_call.py pypy/dist/pypy/translator/c/src/int.h pypy/dist/pypy/translator/c/test/test_backendoptimized.py pypy/dist/pypy/translator/c/test/test_stackless.py pypy/dist/pypy/translator/c/test/test_typed.py Log: Refactored raising operations out of the external functions C code. There are some XXX's left in raisingops.py that I would very much like someone to have a look at. (I hope this is not too much work) Related to there XXX's is probably the currently skipped test_int_mod_ovf_zer test in test-typed.py Modified: pypy/dist/pypy/rpython/raisingops/raisingops.py ============================================================================== --- pypy/dist/pypy/rpython/raisingops/raisingops.py (original) +++ pypy/dist/pypy/rpython/raisingops/raisingops.py Tue Jan 31 13:18:00 2006 @@ -1,26 +1,252 @@ -from pypy.rpython.rarithmetic import r_longlong +from pypy.rpython.rarithmetic import r_longlong, r_uint, intmask -def _pypyop_divmod_adj(x, y, p_rem=None): - xdivy = r_longlong(x / y) - xmody = r_longlong(x - xdivy * y) +#XXX original SIGNED_RIGHT_SHIFT_ZERO_FILLS not taken into account +#XXX assuming HAVE_LONG_LONG (int_mul_ovf) +#XXX should int_mod and int_floordiv return an intmask(...) instead? + +def int_floordiv_zer(x, y): + '''#define OP_INT_FLOORDIV_ZER(x,y,r,err) \ + if ((y)) { OP_INT_FLOORDIV(x,y,r,err); } \ + else FAIL_ZER(err, "integer division") + ''' + if y: + return int_floordiv(x, y) + else: + raise ZeroDivisionError("integer division") + +def uint_floordiv_zer(x, y): + '''#define OP_UINT_FLOORDIV_ZER(x,y,r,err) \ + if ((y)) { OP_UINT_FLOORDIV(x,y,r,err); } \ + else FAIL_ZER(err, "unsigned integer division") + ''' + if y: + return x / y + else: + raise ZeroDivisionError("unsigned integer division") + +def int_neg_ovf(x): + '''#define OP_INT_NEG_OVF(x,r,err) \ + OP_INT_NEG(x,r,err); \ + if ((x) >= 0 || (x) != -(x)); \ + else FAIL_OVF(err, "integer negate") + ''' + r = -x + if x >= 0 or x != r: + return r + else: + raise OverflowError("integer negate") + +def int_abs_ovf(x): + '''#define OP_INT_ABS_OVF(x,r,err) \ + OP_INT_ABS(x,r,err); \ + if ((x) >= 0 || (x) != -(x)); \ + else FAIL_OVF(err, "integer absolute") + ''' + if x >= 0: + r = x + else: + r = -x + if x >= 0 or x != r: + return r + else: + raise OverflowError("integer absolute") + +def int_add_ovf(x, y): + '''#define OP_INT_ADD_OVF(x,y,r,err) \ + OP_INT_ADD(x,y,r,err); \ + if ((r^(x)) >= 0 || (r^(y)) >= 0); \ + else FAIL_OVF(err, "integer addition") + ''' + r = x + y + if r^x >= 0 or r^y >= 0: + return r + else: + raise OverflowError("integer addition") + +def int_sub_ovf(x, y): + '''#define OP_INT_SUB_OVF(x,y,r,err) \ + OP_INT_SUB(x,y,r,err); \ + if ((r^(x)) >= 0 || (r^~(y)) >= 0); \ + else FAIL_OVF(err, "integer subtraction") + ''' + r = x - y + if r^x >= 0 or r^~y >= 0: + return r + else: + raise OverflowError("integer subtraction") + +def int_lshift_ovf(x, y): + '''#define OP_INT_LSHIFT_OVF(x,y,r,err) \ + OP_INT_LSHIFT(x,y,r,err); \ + if ((x) != Py_ARITHMETIC_RIGHT_SHIFT(long, r, (y))) \ + FAIL_OVF(err, "x<= 0) { OP_INT_RSHIFT(x,y,r,err); } \ + else FAIL_VAL(err, "negative shift count") + ''' + if y >= 0: + return _Py_ARITHMETIC_RIGHT_SHIFT(x, y) + else: + raise ValueError("negative shift count") + +def int_lshift_val(x, y): + '''#define OP_INT_LSHIFT_VAL(x,y,r,err) \ + if ((y) >= 0) { OP_INT_LSHIFT(x,y,r,err); } \ + else FAIL_VAL(err, "negative shift count") + ''' + if y >= 0: + return x << y + else: + raise ValueError("negative shift count") + +def int_lshift_ovf_val(x, y): + '''#define OP_INT_LSHIFT_OVF_VAL(x,y,r,err) \ + if ((y) >= 0) { OP_INT_LSHIFT_OVF(x,y,r,err); } \ + else FAIL_VAL(err, "negative shift count") + ''' + if y >= 0: + return int_lshift_ovf(x, y) + else: + raise ValueError("negative shift count") + +def int_floordiv_ovf(x, y): + '''#define OP_INT_FLOORDIV_OVF(x,y,r,err) \ + if ((y) == -1 && (x) < 0 && ((unsigned long)(x) << 1) == 0) \ + FAIL_OVF(err, "integer division"); \ + OP_INT_FLOORDIV(x,y,r,err) + ''' + if y == -1 and x < 0 and (r_uint(x) << 1) == 0: + raise OverflowError("integer division") + else: + return int_floordiv(x, y) + +def int_floordiv_ovf_zer(x, y): + '''#define OP_INT_FLOORDIV_OVF_ZER(x,y,r,err) \ + if ((y)) { OP_INT_FLOORDIV_OVF(x,y,r,err); } \ + else FAIL_ZER(err, "integer division") + ''' + if y: + return int_floordiv_ovf(x, y) + else: + raise ZeroDivisionError("integer division") + +def int_mod_ovf(x, y): + '''#define OP_INT_MOD_OVF(x,y,r,err) \ + if ((y) == -1 && (x) < 0 && ((unsigned long)(x) << 1) == 0) \ + FAIL_OVF(err, "integer modulo"); \ + OP_INT_MOD(x,y,r,err) + ''' + if y == -1 and x < 0 and (r_uint(x) << 1) == 0: + raise OverflowError("integer modulo") + else: + return int_mod(x, y) + +def int_mod_zer(x, y): + '''#define OP_INT_MOD_ZER(x,y,r,err) \ + if ((y)) { OP_INT_MOD(x,y,r,err); } \ + else FAIL_ZER(err, "integer modulo") + ''' + if y: + return int_mod(x, y) + else: + raise ZeroDivisionError("integer modulo") + +def uint_mod_zer(x, y): + '''#define OP_UINT_MOD_ZER(x,y,r,err) \ + if ((y)) { OP_UINT_MOD(x,y,r,err); } \ + else FAIL_ZER(err, "unsigned integer modulo") + ''' + if y: + return x % y + else: + raise ZeroDivisionError("unsigned integer modulo") + +def int_mod_ovf_zer(x, y): + '''#define OP_INT_MOD_OVF_ZER(x,y,r,err) \ + if ((y)) { OP_INT_MOD_OVF(x,y,r,err); } \ + else FAIL_ZER(err, "integer modulo") + ''' + if y: + return int_mod_ovf(x, y) + else: + ZeroDivisionError("integer modulo") + +# Helpers... + +def int_floordiv(x, y): + xdivy = r_longlong(x / y) + xmody = r_longlong(x - xdivy * y) - # If the signs of x and y differ, and the remainder is non-0, - # C89 doesn't define whether xdivy is now the floor or the - # ceiling of the infinitely precise quotient. We want the floor, - # and we have it iff the remainder's sign matches y's. - if xmody and ((y ^ xmody) < 0): + # If the signs of x and y differ, and the remainder is non-0, + # C89 doesn't define whether xdivy is now the floor or the + # ceiling of the infinitely precise quotient. We want the floor, + # and we have it iff the remainder's sign matches y's. + if xmody and ((y ^ xmody) < 0): + xmody += y + xdivy -= 1 + assert xmody and ((y ^ xmody) >= 0) + + return xdivy + +def int_mod(x, y): + xdivy = r_longlong(x / y) + xmody = r_longlong(x - xdivy * y) - xmody += y - xdivy -= 1 - assert xmody and ((y ^ xmody) >= 0) - - #XXX was a pointer - # if p_rem - # p_rem = xmody; - return xdivy - -def int_floordiv_zer(x,y): - if y: - return _pypyop_divmod_adj(x, y) - else: - raise ZeroDivisionError("integer division") + # If the signs of x and y differ, and the remainder is non-0, + # C89 doesn't define whether xdivy is now the floor or the + # ceiling of the infinitely precise quotient. We want the floor, + # and we have it iff the remainder's sign matches y's. + if xmody and ((y ^ xmody) < 0): + xmody += y + xdivy -= 1 + assert xmody and ((y ^ xmody) >= 0) + + return xmody + +def _Py_ARITHMETIC_RIGHT_SHIFT(i, j): + ''' +// Py_ARITHMETIC_RIGHT_SHIFT +// C doesn't define whether a right-shift of a signed integer sign-extends +// or zero-fills. Here a macro to force sign extension: +// Py_ARITHMETIC_RIGHT_SHIFT(TYPE, I, J) +// Return I >> J, forcing sign extension. +// Requirements: +// I is of basic signed type TYPE (char, short, int, long, or long long). +// TYPE is one of char, short, int, long, or long long, although long long +// must not be used except on platforms that support it. +// J is an integer >= 0 and strictly less than the number of bits in TYPE +// (because C doesn't define what happens for J outside that range either). +// Caution: +// I may be evaluated more than once. + +#ifdef SIGNED_RIGHT_SHIFT_ZERO_FILLS + #define Py_ARITHMETIC_RIGHT_SHIFT(TYPE, I, J) \ + ((I) < 0 ? ~((~(unsigned TYPE)(I)) >> (J)) : (I) >> (J)) +#else + #define Py_ARITHMETIC_RIGHT_SHIFT(TYPE, I, J) ((I) >> (J)) +#endif + ''' + return i >> j + +def int_mul_ovf(x, y): + '''{ \ + PY_LONG_LONG lr = (PY_LONG_LONG)(x) * (PY_LONG_LONG)(y); \ + r = (long)lr; \ + if ((PY_LONG_LONG)r == lr); \ + else FAIL_OVF(err, "integer multiplication"); \ + } + ''' + lr = r_longlong(x) * r_longlong(y); + r = intmask(lr) + if r_longlong(r) == lr: + return r + else: + raise OverflowError("integer multiplication") Modified: pypy/dist/pypy/translator/backendopt/all.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/all.py (original) +++ pypy/dist/pypy/translator/backendopt/all.py Tue Jan 31 13:18:00 2006 @@ -9,7 +9,7 @@ from pypy.translator.backendopt.escape import malloc_to_stack -def backend_optimizations(translator, raisingop2direct_call_all=False, +def backend_optimizations(translator, raisingop2direct_call_all=True, inline_threshold=1, mallocs=True, ssa_form=True, Modified: pypy/dist/pypy/translator/backendopt/raisingop2direct_call.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/raisingop2direct_call.py (original) +++ pypy/dist/pypy/translator/backendopt/raisingop2direct_call.py Tue Jan 31 13:18:00 2006 @@ -6,13 +6,17 @@ """search for operations that could raise an exception and change that operation into a direct_call to a function from the raisingops directory. This function also needs to be annotated and specialized. + + note: this could be extended to allow for any operation to be changed into + a direct_call to a (RPython) function! """ seen = {} for op in all_operations(translator): s = op.opname if not s.startswith('int_') and not s.startswith('uint_') and not s.startswith('float_'): continue - if not s.endswith('_zer') and not s.endswith('_ovf') and not s.endswith('_val'): + if not s.endswith('_zer') and not s.endswith('_ovf') and not s.endswith('_val') and \ + not s in ('int_floordiv', 'int_mod'): continue func = getattr(pypy.rpython.raisingops.raisingops, s, None) assert func, "exception raising operation %s was not found" % s Modified: pypy/dist/pypy/translator/c/src/int.h ============================================================================== --- pypy/dist/pypy/translator/c/src/int.h (original) +++ pypy/dist/pypy/translator/c/src/int.h Tue Jan 31 13:18:00 2006 @@ -1,4 +1,3 @@ - /************************************************************/ /*** C header subsection: operations between ints ***/ @@ -12,19 +11,9 @@ #define OP_INT_NEG(x,r,err) r = -(x) -#define OP_INT_NEG_OVF(x,r,err) \ - OP_INT_NEG(x,r,err); \ - if ((x) >= 0 || (x) != -(x)); \ - else FAIL_OVF(err, "integer negate") - #define OP_INT_ABS(x,r,err) r = (x) >= 0 ? x : -(x) #define OP_UINT_ABS(x,r,err) r = (x) -#define OP_INT_ABS_OVF(x,r,err) \ - OP_INT_ABS(x,r,err); \ - if ((x) >= 0 || (x) != -(x)); \ - else FAIL_OVF(err, "integer absolute") - /*** binary operations ***/ #define OP_INT_EQ(x,y,r,err) r = ((x) == (y)) @@ -41,37 +30,10 @@ #define OP_INT_ADD(x,y,r,err) r = (x) + (y) -#define OP_INT_ADD_OVF(x,y,r,err) \ - OP_INT_ADD(x,y,r,err); \ - if ((r^(x)) >= 0 || (r^(y)) >= 0); \ - else FAIL_OVF(err, "integer addition") - #define OP_INT_SUB(x,y,r,err) r = (x) - (y) -#define OP_INT_SUB_OVF(x,y,r,err) \ - OP_INT_SUB(x,y,r,err); \ - if ((r^(x)) >= 0 || (r^~(y)) >= 0); \ - else FAIL_OVF(err, "integer subtraction") - #define OP_INT_MUL(x,y,r,err) r = (x) * (y) -#ifndef HAVE_LONG_LONG - -#define OP_INT_MUL_OVF(x,y,r,err) \ - if (op_int_mul_ovf(x,y,&r)); \ - else FAIL_OVF(err, "integer multiplication") - -#else - -#define OP_INT_MUL_OVF(x,y,r,err) \ - { \ - PY_LONG_LONG lr = (PY_LONG_LONG)(x) * (PY_LONG_LONG)(y); \ - r = (long)lr; \ - if ((PY_LONG_LONG)r == lr); \ - else FAIL_OVF(err, "integer multiplication"); \ - } -#endif - /* shifting */ /* NB. shifting has same limitations as C: the shift count must be @@ -82,68 +44,18 @@ #define OP_INT_LSHIFT(x,y,r,err) r = (x) << (y) #define OP_UINT_LSHIFT(x,y,r,err) r = (x) << (y) -#define OP_INT_LSHIFT_OVF(x,y,r,err) \ - OP_INT_LSHIFT(x,y,r,err); \ - if ((x) != Py_ARITHMETIC_RIGHT_SHIFT(long, r, (y))) \ - FAIL_OVF(err, "x<= 0) { OP_INT_RSHIFT(x,y,r,err); } \ - else FAIL_VAL(err, "negative shift count") - -#define OP_INT_LSHIFT_VAL(x,y,r,err) \ - if ((y) >= 0) { OP_INT_LSHIFT(x,y,r,err); } \ - else FAIL_VAL(err, "negative shift count") - -#define OP_INT_LSHIFT_OVF_VAL(x,y,r,err) \ - if ((y) >= 0) { OP_INT_LSHIFT_OVF(x,y,r,err); } \ - else FAIL_VAL(err, "negative shift count") - - /* floor division */ -#define OP_INT_FLOORDIV(x,y,r,err) r = op_divmod_adj(x, y, NULL) +/* NB: OP_INT_FLOORDIV and OP_INT_MOD are the operations that end up in the graphs + after the raisingop2direct_call transformation has been done. */ +#define OP_INT_FLOORDIV(x,y,r,err) r = (x) / (y) #define OP_UINT_FLOORDIV(x,y,r,err) r = (x) / (y) -#define OP_INT_FLOORDIV_OVF(x,y,r,err) \ - if ((y) == -1 && (x) < 0 && ((unsigned long)(x) << 1) == 0) \ - FAIL_OVF(err, "integer division"); \ - OP_INT_FLOORDIV(x,y,r,err) - -#define OP_INT_FLOORDIV_ZER(x,y,r,err) \ - if ((y)) { OP_INT_FLOORDIV(x,y,r,err); } \ - else FAIL_ZER(err, "integer division") -#define OP_UINT_FLOORDIV_ZER(x,y,r,err) \ - if ((y)) { OP_UINT_FLOORDIV(x,y,r,err); } \ - else FAIL_ZER(err, "unsigned integer division") - -#define OP_INT_FLOORDIV_OVF_ZER(x,y,r,err) \ - if ((y)) { OP_INT_FLOORDIV_OVF(x,y,r,err); } \ - else FAIL_ZER(err, "integer division") - /* modulus */ -#define OP_INT_MOD(x,y,r,err) op_divmod_adj(x, y, &r) +#define OP_INT_MOD(x,y,r,err) r = (x) % (y) #define OP_UINT_MOD(x,y,r,err) r = (x) % (y) -#define OP_INT_MOD_OVF(x,y,r,err) \ - if ((y) == -1 && (x) < 0 && ((unsigned long)(x) << 1) == 0) \ - FAIL_OVF(err, "integer modulo"); \ - OP_INT_MOD(x,y,r,err) - -#define OP_INT_MOD_ZER(x,y,r,err) \ - if ((y)) { OP_INT_MOD(x,y,r,err); } \ - else FAIL_ZER(err, "integer modulo") -#define OP_UINT_MOD_ZER(x,y,r,err) \ - if ((y)) { OP_UINT_MOD(x,y,r,err); } \ - else FAIL_ZER(err, "unsigned integer modulo") - -#define OP_INT_MOD_OVF_ZER(x,y,r,err) \ - if ((y)) { OP_INT_MOD_OVF(x,y,r,err); } \ - else FAIL_ZER(err, "integer modulo") - /* bit operations */ #define OP_INT_AND(x,y,r,err) r = (x) & (y) @@ -172,85 +84,6 @@ /* _________________ certain implementations __________________ */ -#ifndef HAVE_LONG_LONG -/* adjusted from intobject.c, Python 2.3.3 */ - -/* prototypes */ - -op_int_mul_ovf(long a, long b, long *longprod); - -/* implementations */ - -#ifndef PYPY_NOT_MAIN_FILE - -int -op_int_mul_ovf(long a, long b, long *longprod) -{ - double doubled_longprod; /* (double)longprod */ - double doubleprod; /* (double)a * (double)b */ - - *longprod = a * b; - doubleprod = (double)a * (double)b; - doubled_longprod = (double)*longprod; - - /* Fast path for normal case: small multiplicands, and no info - is lost in either method. */ - if (doubled_longprod == doubleprod) - return 1; - - /* Somebody somewhere lost info. Close enough, or way off? Note - that a != 0 and b != 0 (else doubled_longprod == doubleprod == 0). - The difference either is or isn't significant compared to the - true value (of which doubleprod is a good approximation). - */ - { - const double diff = doubled_longprod - doubleprod; - const double absdiff = diff >= 0.0 ? diff : -diff; - const double absprod = doubleprod >= 0.0 ? doubleprod : - -doubleprod; - /* absdiff/absprod <= 1/32 iff - 32 * absdiff <= absprod -- 5 good bits is "close enough" */ - if (32.0 * absdiff <= absprod) - return 1; - return 0; - } -} - -#endif /* PYPY_NOT_MAIN_FILE */ - -#endif /* HAVE_LONG_LONG */ - -/* XXX we might probe the compiler whether it does what we want */ - -/* prototypes */ - -long op_divmod_adj(long x, long y, long *p_rem); - -/* implementations */ - -#ifndef PYPY_NOT_MAIN_FILE - -long op_divmod_adj(long x, long y, long *p_rem) -{ - long xdivy = x / y; - long xmody = x - xdivy * y; - /* If the signs of x and y differ, and the remainder is non-0, - * C89 doesn't define whether xdivy is now the floor or the - * ceiling of the infinitely precise quotient. We want the floor, - * and we have it iff the remainder's sign matches y's. - */ - if (xmody && ((y ^ xmody) < 0) /* i.e. and signs differ */) { - xmody += y; - --xdivy; - assert(xmody && ((y ^ xmody) >= 0)); - } - if (p_rem) - *p_rem = xmody; - return xdivy; -} - -#endif /* PYPY_NOT_MAIN_FILE */ - /* no editing below this point */ /* following lines are generated by mkuint.py */ Modified: pypy/dist/pypy/translator/c/test/test_backendoptimized.py ============================================================================== --- pypy/dist/pypy/translator/c/test/test_backendoptimized.py (original) +++ pypy/dist/pypy/translator/c/test/test_backendoptimized.py Tue Jan 31 13:18:00 2006 @@ -9,7 +9,7 @@ def process(self, t): _TestTypedTestCase.process(self, t) self.t = t - backend_optimizations(t, merge_if_blocks_to_switch=False) + backend_optimizations(t, raisingop2direct_call_all=False, merge_if_blocks_to_switch=False) def test_remove_same_as(self): def f(n=bool): @@ -180,7 +180,7 @@ def process(self, t): _TestTypedTestCase.process(self, t) self.t = t - backend_optimizations(t, raisingop2direct_call_all=True) + backend_optimizations(t) def test_int_floordiv_zer(self): def f(x=int): Modified: pypy/dist/pypy/translator/c/test/test_stackless.py ============================================================================== --- pypy/dist/pypy/translator/c/test/test_stackless.py (original) +++ pypy/dist/pypy/translator/c/test/test_stackless.py Tue Jan 31 13:18:00 2006 @@ -4,6 +4,7 @@ from pypy.annotation.listdef import ListDef from pypy.rpython.rstack import stack_unwind, stack_frames_depth, stack_too_big from pypy.rpython.rstack import yield_current_frame_to_caller +from pypy.translator.backendopt.raisingop2direct_call import raisingop2direct_call import os def wrap_stackless_function(fn): @@ -16,6 +17,7 @@ t = TranslationContext() t.buildannotator().build_types(entry_point, [s_list_of_strings]) t.buildrtyper().specialize() + raisingop2direct_call(t) cbuilder = CStandaloneBuilder(t, entry_point) cbuilder.stackless = True Modified: pypy/dist/pypy/translator/c/test/test_typed.py ============================================================================== --- pypy/dist/pypy/translator/c/test/test_typed.py (original) +++ pypy/dist/pypy/translator/c/test/test_typed.py Tue Jan 31 13:18:00 2006 @@ -3,7 +3,8 @@ import py from py.test import raises from pypy.translator.test import snippet -from pypy.rpython.rarithmetic import r_uint, intmask +from pypy.rpython.rarithmetic import r_uint, r_longlong, intmask +from pypy.translator.backendopt.raisingop2direct_call import raisingop2direct_call from pypy.translator.c.test.test_annotated import TestAnnotatedTestCase as _TestAnnotatedTestCase @@ -13,6 +14,7 @@ def process(self, t): _TestAnnotatedTestCase.process(self, t) t.buildrtyper().specialize() + raisingop2direct_call(t) def test_call_five(self): # -- the result of call_five() isn't a real list, but an rlist @@ -205,12 +207,23 @@ fn = self.getcompiled(snippet.add_func) raises(OverflowError, fn, sys.maxint) - def test_int_div_ovf_zer(self): # + def test_int_div_ovf_zer(self): fn = self.getcompiled(snippet.div_func) raises(OverflowError, fn, -1) raises(ZeroDivisionError, fn, 0) + def test_int_mul_ovf(self): + fn = self.getcompiled(snippet.mul_func) + for y in range(-5, 5): + for x in range(-5, 5): + assert fn(x, y) == snippet.mul_func(x, y) + n = sys.maxint / 4 + assert fn(n, 3) == snippet.mul_func(n, 3) + assert fn(n, 4) == snippet.mul_func(n, 4) + raises(OverflowError, fn, n, 5) + def test_int_mod_ovf_zer(self): + py.test.skip("XXX does not annotate anymore after raisingops2direct_call transformation") fn = self.getcompiled(snippet.mod_func) raises(OverflowError, fn, -1) raises(ZeroDivisionError, fn, 0) From ericvrp at codespeak.net Tue Jan 31 13:23:22 2006 From: ericvrp at codespeak.net (ericvrp at codespeak.net) Date: Tue, 31 Jan 2006 13:23:22 +0100 (CET) Subject: [pypy-svn] r22889 - pypy/dist/pypy/translator/test Message-ID: <20060131122322.BD4C51008E@code0.codespeak.net> Author: ericvrp Date: Tue Jan 31 13:23:21 2006 New Revision: 22889 Modified: pypy/dist/pypy/translator/test/snippet.py Log: forgot to check this in Modified: pypy/dist/pypy/translator/test/snippet.py ============================================================================== --- pypy/dist/pypy/translator/test/snippet.py (original) +++ pypy/dist/pypy/translator/test/snippet.py Tue Jan 31 13:23:21 2006 @@ -1144,6 +1144,12 @@ except (OverflowError, ZeroDivisionError): raise +def mul_func(x=numtype, y=numtype): + try: + return ovfcheck(x * y) + except OverflowError: + raise + def mod_func(i=numtype): try: return ovfcheck((-maxint-1) % i) From ericvrp at codespeak.net Tue Jan 31 13:38:34 2006 From: ericvrp at codespeak.net (ericvrp at codespeak.net) Date: Tue, 31 Jan 2006 13:38:34 +0100 (CET) Subject: [pypy-svn] r22890 - pypy/dist/pypy/translator/c/test Message-ID: <20060131123834.912BE1008E@code0.codespeak.net> Author: ericvrp Date: Tue Jan 31 13:38:32 2006 New Revision: 22890 Modified: pypy/dist/pypy/translator/c/test/test_backendoptimized.py Log: this fixes the segfault. Apperently it's al too easy to run raisingops2direct_call multiple times, which does not work. The problem is that the newly added functions contain identicly named operations which should not be converted to direct_calls the 2nd time the transformation is called. It's probably better to rename int_floordiv and int_mod in the newly added functions so this transformation can be called multiple times without a problem. Modified: pypy/dist/pypy/translator/c/test/test_backendoptimized.py ============================================================================== --- pypy/dist/pypy/translator/c/test/test_backendoptimized.py (original) +++ pypy/dist/pypy/translator/c/test/test_backendoptimized.py Tue Jan 31 13:38:32 2006 @@ -180,7 +180,6 @@ def process(self, t): _TestTypedTestCase.process(self, t) self.t = t - backend_optimizations(t) def test_int_floordiv_zer(self): def f(x=int): From pedronis at codespeak.net Tue Jan 31 14:29:38 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Tue, 31 Jan 2006 14:29:38 +0100 (CET) Subject: [pypy-svn] r22891 - in pypy/dist/pypy/jit: . test Message-ID: <20060131132938.1637B10092@code0.codespeak.net> Author: pedronis Date: Tue Jan 31 14:29:25 2006 New Revision: 22891 Modified: pypy/dist/pypy/jit/hintmodel.py pypy/dist/pypy/jit/test/test_hint_annotation.py Log: try at specialisation for functions during hint-annotation separating signatures involving concretes and fixed and the rest. Modified: pypy/dist/pypy/jit/hintmodel.py ============================================================================== --- pypy/dist/pypy/jit/hintmodel.py (original) +++ pypy/dist/pypy/jit/hintmodel.py Tue Jan 31 14:29:25 2006 @@ -74,14 +74,20 @@ lst.append(s) return '<%s>' % (', '.join(lst),) + def is_fixed(self): + for o in self.origins: + if not o.fixed: + return False + return True + def annotationcolor(self): """Compute the color of the variables with this annotation for the pygame viewer """ - for o in self.origins: - if not o.fixed: - return None - return (50,140,0) + if self.is_fixed(): + return (50,140,0) + else: + return None annotationcolor = property(annotationcolor) class SomeLLConcreteValue(SomeLLAbstractValue): @@ -182,11 +188,49 @@ desc = bookkeeper.getdesc(fnobj.graph) key = None alt_name = None + to_fix = [] if bookkeeper.myorigin().read_fixed(): key = 'fixed' alt_name = fnobj.graph.name + '_HFixed' + else: + key = [] + specialize = False + for i, arg_hs in enumerate(args_hs): + if isinstance(arg_hs, SomeLLConcreteValue): + key.append('C') + specialize = True + elif isinstance(arg_hs, SomeLLAbstractConstant): + if arg_hs.origins: + fixed = True + for o in arg_hs.origins: + if not o.read_fixed(): + fixed = False + break + if fixed: + key.append('f') + # fix the corresponding SomeLLAbstractConstant + # in the input signature of the specialized callee + to_fix.append(i) + specialize = True + continue + + key.append('x') + else: + key.append('x') + if specialize: + key = ''.join(key) + alt_name = fnobj.graph.name + '_H'+key + else: + key = None + input_args_hs = list(args_hs) graph = desc.specialize(input_args_hs, key=key, alt_name=alt_name) + + for i in to_fix: + inp_arg_hs = input_args_hs[i] + assert len(inp_arg_hs.origins) == 1 + inp_arg_hs.origins.keys()[0].set_fixed() + hs_res = bookkeeper.annotator.recursivecall(graph, bookkeeper.position_key, input_args_hs) @@ -355,7 +399,7 @@ #if hasattr(hs_c1, 'const') or hasattr(hs_c2, 'const'): return SomeLLConcreteValue(hs_c1.concretetype) # MAYBE #else: - # raise annmodel.UnionError("%s %s don't mix, unless the constant is constant" % (hs_c1, hs_c2)) + #raise annmodel.UnionError("%s %s don't mix, unless the constant is constant" % (hs_c1, hs_c2)) class __extend__(pairtype(SomeLLAbstractContainer, SomeLLAbstractContainer)): Modified: pypy/dist/pypy/jit/test/test_hint_annotation.py ============================================================================== --- pypy/dist/pypy/jit/test/test_hint_annotation.py (original) +++ pypy/dist/pypy/jit/test/test_hint_annotation.py Tue Jan 31 14:29:25 2006 @@ -76,8 +76,7 @@ hs = hannotate(ll_function, [bool, int, int]) assert isinstance(hs, SomeLLAbstractConstant) assert len(hs.origins) == 4 - for o in hs.origins: - assert o.fixed + assert hs.is_fixed() assert hs.concretetype == lltype.Signed def test_simple_variable(): @@ -292,21 +291,46 @@ else: z = x-y return z - def ll_function(cond, x,y, y1): - z1 = ll_help(cond, x, y1) + def ll_function(cond, x,y, x1, y1): + z1 = ll_help(cond, x1, y1) z = ll_help(cond, x, y) z = hint(z, concrete=True) return z - hs, ha = hannotate(ll_function, [bool, int, int, int], annotator=True) + hs, ha = hannotate(ll_function, [bool, int, int, int, int], annotator=True) assert isinstance(hs, SomeLLConcreteValue) assert hs.concretetype == lltype.Signed ll_help_graph = graphof(ha.base_translator, ll_help) gdesc = ha.bookkeeper.getdesc(ll_help_graph) - for o in ha.binding(gdesc._cache[None].getreturnvar()).origins: - assert not o.fixed + assert not ha.binding(gdesc._cache[None].getreturnvar()).is_fixed() assert len(gdesc._cache) == 2 - for o in ha.binding(gdesc._cache['fixed'].getreturnvar()).origins: - assert o.fixed + assert ha.binding(gdesc._cache['fixed'].getreturnvar()).is_fixed() + +def test_specialize_calls(): + def ll_add(x, y): + return x+y + def ll_function(x,y): + z0 = ll_add(y, 2) + z1 = ll_add(x, y) + x1 = hint(x, concrete=True) + z2 = ll_add(x1, y) + return z2 + hs, ha = hannotate(ll_function, [int, int], annotator=True) + assert isinstance(hs, SomeLLConcreteValue) + assert hs.concretetype == lltype.Signed + ll_add_graph = graphof(ha.base_translator, ll_add) + gdesc = ha.bookkeeper.getdesc(ll_add_graph) + assert len(gdesc._cache) == 3 + assert 'Cx' in gdesc._cache + assert 'fx' in gdesc._cache + v1, v2 = gdesc._cache['Cx'].getargs() + assert isinstance(ha.binding(v1), SomeLLConcreteValue) + assert isinstance(ha.binding(v2), SomeLLAbstractConstant) + assert not ha.binding(v2).is_fixed() + v1, v2 = gdesc._cache['fx'].getargs() + assert isinstance(ha.binding(v1), SomeLLAbstractConstant) + assert isinstance(ha.binding(v2), SomeLLAbstractConstant) + assert ha.binding(v1).is_fixed() + assert not ha.binding(v2).is_fixed() def test_hannotate_tl(): From jum at codespeak.net Tue Jan 31 15:05:06 2006 From: jum at codespeak.net (jum at codespeak.net) Date: Tue, 31 Jan 2006 15:05:06 +0100 (CET) Subject: [pypy-svn] r22892 - pypy/dist/pypy/doc Message-ID: <20060131140506.098E110093@code0.codespeak.net> Author: jum Date: Tue Jan 31 15:05:06 2006 New Revision: 22892 Modified: pypy/dist/pypy/doc/svn-help.txt Log: Change the paragraph about proxy problems to use SSL instead of the previous codepeak implementation detail of having the subversion server also running on port 8080. Modified: pypy/dist/pypy/doc/svn-help.txt ============================================================================== --- pypy/dist/pypy/doc/svn-help.txt (original) +++ pypy/dist/pypy/doc/svn-help.txt Tue Jan 31 15:05:06 2006 @@ -116,17 +116,9 @@ ---------------------------- Some proxies don't let extended HTTP commands through. If you have an -error complaining about a bad request, you can ask subversion to use the -alternate port 8080 to connect to codespeak.net by adding the following -lines in a file ``~/.subversion/servers`` (on Unix) or -``%APPDATA%\Subversion\servers`` (on Windows):: - - [groups] - codespeak = codespeak.net - - [codespeak] - http-proxy-host = codespeak.net - http-proxy-port = 8080 +error complaining about a bad request, you should use https: instead of +http: in the subversion URL. This will make use of SSL encryption, which +cannot be intercepted by proxies. How to Avoid Line-ending Hell ----------------------------- From auc at codespeak.net Tue Jan 31 15:33:16 2006 From: auc at codespeak.net (auc at codespeak.net) Date: Tue, 31 Jan 2006 15:33:16 +0100 (CET) Subject: [pypy-svn] r22893 - pypy/dist/pypy/lib/logic Message-ID: <20060131143316.8550E1009B@code0.codespeak.net> Author: auc Date: Tue Jan 31 15:33:15 2006 New Revision: 22893 Modified: pypy/dist/pypy/lib/logic/variable.py Log: add ridiculous comment Modified: pypy/dist/pypy/lib/logic/variable.py ============================================================================== --- pypy/dist/pypy/lib/logic/variable.py (original) +++ pypy/dist/pypy/lib/logic/variable.py Tue Jan 31 15:33:15 2006 @@ -35,6 +35,7 @@ if name in cs.names: raise AlreadyInStore(name) self.name = name + # the creation-time space self.cs = cs # top-level 'commited' binding self._val = NoValue From auc at codespeak.net Tue Jan 31 15:33:53 2006 From: auc at codespeak.net (auc at codespeak.net) Date: Tue, 31 Jan 2006 15:33:53 +0100 (CET) Subject: [pypy-svn] r22894 - pypy/dist/pypy/lib/logic Message-ID: <20060131143353.EC3641009B@code0.codespeak.net> Author: auc Date: Tue Jan 31 15:33:52 2006 New Revision: 22894 Modified: pypy/dist/pypy/lib/logic/problems.py Log: fix return value Modified: pypy/dist/pypy/lib/logic/problems.py ============================================================================== --- pypy/dist/pypy/lib/logic/problems.py (original) +++ pypy/dist/pypy/lib/logic/problems.py Tue Jan 31 15:33:52 2006 @@ -49,4 +49,4 @@ def dummy_problem(computation_space): ret = computation_space.var('__dummy__') ret.dom = c.FiniteDomain([1, 2]) - return ret + return (ret) From auc at codespeak.net Tue Jan 31 15:34:58 2006 From: auc at codespeak.net (auc at codespeak.net) Date: Tue, 31 Jan 2006 15:34:58 +0100 (CET) Subject: [pypy-svn] r22895 - pypy/dist/pypy/lib/logic Message-ID: <20060131143458.571541009C@code0.codespeak.net> Author: auc Date: Tue Jan 31 15:34:57 2006 New Revision: 22895 Modified: pypy/dist/pypy/lib/logic/computationspace.py Log: get rid of global stuff (doesn't help any more) Modified: pypy/dist/pypy/lib/logic/computationspace.py ============================================================================== --- pypy/dist/pypy/lib/logic/computationspace.py (original) +++ pypy/dist/pypy/lib/logic/computationspace.py Tue Jan 31 15:34:57 2006 @@ -476,7 +476,6 @@ else: # 1. both are unbound self._merge(var, val) else: # 3. val is really a value - print "%s, is that you ?" % var if var._is_bound(): raise AlreadyBound(var.name) self._bind(var.val, val) @@ -783,32 +782,32 @@ #-- #-- the global store -from problems import dummy_problem -_cs = ComputationSpace(dummy_problem) +#from problems import dummy_problem +#_cs = ComputationSpace(dummy_problem) #-- global accessor functions -def var(name): - v = Var(name, _cs) - _cs.add_unbound(v) - return v +## def var(name): +## v = Var(name, _cs) +## _cs.add_unbound(v) +## return v -def set_domain(var, dom): - return _cs.set_domain(var, dom) +## def set_domain(var, dom): +## return _cs.set_domain(var, dom) -def add_constraint(constraint): - return _cs.add_constraint(constraint) +## def add_constraint(constraint): +## return _cs.add_constraint(constraint) -def satisfiable(constraint): - return _cs.satisfiable(constraint) +## def satisfiable(constraint): +## return _cs.satisfiable(constraint) -def get_satisfying_domains(constraint): - return _cs.get_satisfying_domains(constraint) +## def get_satisfying_domains(constraint): +## return _cs.get_satisfying_domains(constraint) -def satisfy(constraint): - return _cs.satisfy(constraint) +## def satisfy(constraint): +## return _cs.satisfy(constraint) -def bind(var, val): - return _cs.bind(var, val) +## def bind(var, val): +## return _cs.bind(var, val) -def unify(var1, var2): - return _cs.unify(var1, var2) +## def unify(var1, var2): +## return _cs.unify(var1, var2) From auc at codespeak.net Tue Jan 31 15:35:49 2006 From: auc at codespeak.net (auc at codespeak.net) Date: Tue, 31 Jan 2006 15:35:49 +0100 (CET) Subject: [pypy-svn] r22896 - pypy/dist/pypy/lib/logic Message-ID: <20060131143549.3D3891009C@code0.codespeak.net> Author: auc Date: Tue Jan 31 15:35:45 2006 New Revision: 22896 Modified: pypy/dist/pypy/lib/logic/test_computationspace.py Log: don't use global stuff from computationspace.py Modified: pypy/dist/pypy/lib/logic/test_computationspace.py ============================================================================== --- pypy/dist/pypy/lib/logic/test_computationspace.py (original) +++ pypy/dist/pypy/lib/logic/test_computationspace.py Tue Jan 31 15:35:45 2006 @@ -2,13 +2,12 @@ import variable as v import constraint as c -import computationspace as cs +import computationspace as space import distributor as di -from problems import * +import problems from py.test import raises - -#-- meat ------------------------ +#-- utility --------------------- class FunThread(Thread): @@ -20,127 +19,139 @@ def run(self): self.fun(self, *self.args) +#-- meat ------------------------ + class TestStoreUnification: - def setup_method(self, meth): - cs._cs = cs.ComputationSpace(dummy_problem) def test_already_in_store(self): - x = cs.var('x') - raises(v.AlreadyInStore, cs.var, 'x') + sp = space.ComputationSpace(problems.dummy_problem) + x = sp.var('x') + raises(v.AlreadyInStore, sp.var, 'x') def test_already_bound(self): - x = cs.var('x') - cs.bind(x, 42) - raises(cs.AlreadyBound, cs.bind, x, 42) + sp = space.ComputationSpace(problems.dummy_problem) + x = sp.var('x') + sp.bind(x, 42) + raises(space.AlreadyBound, sp.bind, x, 42) def test_bind_var_var(self): - x = cs.var('x') - y = cs.var('y') - z = cs.var('z') - cs.bind(x, z) - assert x.val == cs.EqSet([x, z]) - assert y.val == cs.EqSet([y]) - assert z.val == cs.EqSet([x, z]) + sp = space.ComputationSpace(problems.dummy_problem) + x = sp.var('x') + y = sp.var('y') + z = sp.var('z') + sp.bind(x, z) + assert x.val == space.EqSet([x, z]) + assert y.val == space.EqSet([y]) + assert z.val == space.EqSet([x, z]) def test_bind_var_val(self): - x, y, z = cs.var('x'), cs.var('y'), cs.var('z') - cs.bind(x, z) - cs.bind(y, 42) - cs.bind(z, 3.14) + sp = space.ComputationSpace(problems.dummy_problem) + x, y, z = sp.var('x'), sp.var('y'), sp.var('z') + sp.bind(x, z) + sp.bind(y, 42) + sp.bind(z, 3.14) assert x.val == 3.14 assert y.val == 42 assert z.val == 3.14 def test_unify_same(self): - x,y,z,w = (cs.var('x'), cs.var('y'), - cs.var('z'), cs.var('w')) - cs.bind(x, [42, z]) - cs.bind(y, [z, 42]) - cs.bind(w, [z, 43]) - raises(cs.UnificationFailure, cs.unify, x, w) - cs.unify(x, y) + sp = space.ComputationSpace(problems.dummy_problem) + x,y,z,w = (sp.var('x'), sp.var('y'), + sp.var('z'), sp.var('w')) + sp.bind(x, [42, z]) + sp.bind(y, [z, 42]) + sp.bind(w, [z, 43]) + raises(space.UnificationFailure, sp.unify, x, w) + sp.unify(x, y) assert z.val == 42 def test_double_unification(self): - x, y, z = (cs.var('x'), cs.var('y'), - cs.var('z')) - cs.bind(x, 42) - cs.bind(y, z) - cs.unify(x, y) + sp = space.ComputationSpace(problems.dummy_problem) + x, y, z = (sp.var('x'), sp.var('y'), + sp.var('z')) + sp.bind(x, 42) + sp.bind(y, z) + sp.unify(x, y) assert z.val == 42 - cs.unify(x, y) + sp.unify(x, y) assert (z.val == x.val) and (x.val == y.val) def test_unify_values(self): - x, y = cs.var('x'), cs.var('y') - cs.bind(x, [1, 2, 3]) - cs.bind(y, [1, 2, 3]) - cs.unify(x, y) + sp = space.ComputationSpace(problems.dummy_problem) + x, y = sp.var('x'), sp.var('y') + sp.bind(x, [1, 2, 3]) + sp.bind(y, [1, 2, 3]) + sp.unify(x, y) assert x.val == [1, 2, 3] assert y.val == [1, 2, 3] def test_unify_lists_success(self): - x,y,z,w = (cs.var('x'), cs.var('y'), - cs.var('z'), cs.var('w')) - cs.bind(x, [42, z]) - cs.bind(y, [w, 44]) - cs.unify(x, y) + sp = space.ComputationSpace(problems.dummy_problem) + x,y,z,w = (sp.var('x'), sp.var('y'), + sp.var('z'), sp.var('w')) + sp.bind(x, [42, z]) + sp.bind(y, [w, 44]) + sp.unify(x, y) assert x.val == [42, z] assert y.val == [w, 44] assert z.val == 44 assert w.val == 42 def test_unify_dicts_success(self): - x,y,z,w = (cs.var('x'), cs.var('y'), - cs.var('z'), cs.var('w')) - cs.bind(x, {1:42, 2:z}) - cs.bind(y, {1:w, 2:44}) - cs.unify(x, y) + sp = space.ComputationSpace(problems.dummy_problem) + x,y,z,w = (sp.var('x'), sp.var('y'), + sp.var('z'), sp.var('w')) + sp.bind(x, {1:42, 2:z}) + sp.bind(y, {1:w, 2:44}) + sp.unify(x, y) assert x.val == {1:42, 2:z} assert y.val == {1:w, 2:44} assert z.val == 44 assert w.val == 42 def test_unify_failure(self): - x,y,z = cs.var('x'), cs.var('y'), cs.var('z') - cs.bind(x, [42, z]) - cs.bind(y, [z, 44]) - raises(cs.UnificationFailure, cs.unify, x, y) + sp = space.ComputationSpace(problems.dummy_problem) + x,y,z = sp.var('x'), sp.var('y'), sp.var('z') + sp.bind(x, [42, z]) + sp.bind(y, [z, 44]) + raises(space.UnificationFailure, sp.unify, x, y) # check store consistency assert x.val == [42, z] assert y.val == [z, 44] - assert z.val == cs.EqSet([z]) + assert z.val == space.EqSet([z]) def test_unify_failure2(self): - x,y,z,w = (cs.var('x'), cs.var('y'), - cs.var('z'), cs.var('w')) - cs.bind(x, [42, z]) - cs.bind(y, [w, 44]) - cs.bind(z, w) - assert cs._cs.in_transaction == False - raises(cs.UnificationFailure, cs.unify, x, y) - assert cs._cs.in_transaction == False + sp = space.ComputationSpace(problems.dummy_problem) + x,y,z,w = (sp.var('x'), sp.var('y'), + sp.var('z'), sp.var('w')) + sp.bind(x, [42, z]) + sp.bind(y, [w, 44]) + sp.bind(z, w) + assert sp.in_transaction == False + raises(space.UnificationFailure, sp.unify, x, y) + assert sp.in_transaction == False # check store consistency assert x.val == [42, z] assert y.val == [w, 44] - assert z.val == cs.EqSet([z,w]) - assert w.val == cs.EqSet([z,w]) + assert z.val == space.EqSet([z,w]) + assert w.val == space.EqSet([z,w]) def test_unify_circular(self): - x, y, z, w, a, b = (cs.var('x'), cs.var('y'), - cs.var('z'), cs.var('w'), - cs.var('a'), cs.var('b')) - cs.bind(x, [y]) - cs.bind(y, [x]) - raises(cs.UnificationFailure, cs.unify, x, y) - cs.bind(z, [1, w]) - cs.bind(w, [z, 2]) - raises(cs.UnificationFailure, cs.unify, z, w) - cs.bind(a, {1:42, 2:b}) - cs.bind(b, {1:a, 2:42}) - raises(cs.UnificationFailure, cs.unify, a, b) + sp = space.ComputationSpace(problems.dummy_problem) + x, y, z, w, a, b = (sp.var('x'), sp.var('y'), + sp.var('z'), sp.var('w'), + sp.var('a'), sp.var('b')) + sp.bind(x, [y]) + sp.bind(y, [x]) + raises(space.UnificationFailure, sp.unify, x, y) + sp.bind(z, [1, w]) + sp.bind(w, [z, 2]) + raises(space.UnificationFailure, sp.unify, z, w) + sp.bind(a, {1:42, 2:b}) + sp.bind(b, {1:a, 2:42}) + raises(space.UnificationFailure, sp.unify, a, b) # check store consistency assert x.val == [y] assert y.val == [x] @@ -151,11 +162,12 @@ def test_threads_creating_vars(self): + sp = space.ComputationSpace(problems.dummy_problem) def create_var(thread, *args): - x = cs.var('x') + x = sp.var('x') def create_var2(thread, *args): - raises(v.AlreadyExists, cs.var, 'x') + raises(v.AlreadyExists, sp.var, 'x') t1, t2 = (FunThread(create_var), FunThread(create_var2)) @@ -164,27 +176,28 @@ def test_threads_binding_vars(self): + sp = space.ComputationSpace(problems.dummy_problem) def do_stuff(thread, var, val): thread.raised = False try: # pb. with TLS (thread-local-stuff) in # cs class - cs.bind(var, val) + sp.bind(var, val) except Exception, e: print e thread.raised = True - assert isinstance(e, cs.AlreadyBound) + assert isinstance(e, space.AlreadyBound) - x = cs.var('x') + x = sp.var('x') vars_ = [] for nvar in range(1000): - v = cs.var('x-'+str(nvar)) - cs.bind(x, v) + v = sp.var('x-'+str(nvar)) + sp.bind(x, v) vars_.append(v) for var in vars_: - assert var in cs._cs.vars + assert var in sp.vars assert var.val == x.val t1, t2 = (FunThread(do_stuff, x, 42), @@ -195,144 +208,183 @@ t2.join() #check that every var is really bound to 42 or 43 for var in vars_: - assert var in cs._cs.vars + assert var in sp.vars assert var.val == x.val assert (t2.raised and not t1.raised) or \ (t1.raised and not t2.raised) + def test_threads_waiting_for_unbound_var(self): + sp = space.ComputationSpace(problems.dummy_problem) + import time + + def near(v1, v2, err): + return abs(v1 - v2) < err + + start_time = time.time() + + def wait_on_unbound(thread, var, start_time): + thread.val = var.get() + thread.waited = time.time() - start_time + + x = sp.var('x') + t1, t2 = (FunThread(wait_on_unbound, x, start_time), + FunThread(wait_on_unbound, x, start_time)) + t1.start() + t2.start() + time.sleep(1) + sp.bind(x, 42) + t1.join() + t2.join() + assert t1.val == 42 + assert t2.val == 42 + assert near(t1.waited, 1, .1) + assert near(t2.waited, 1, .1) + + def test_set_var_domain(self): - x = cs.var('x') - cs.set_domain(x, [1, 3, 5]) - assert x.cs_get_dom(cs._cs) == c.FiniteDomain([1, 3, 5]) + sp = space.ComputationSpace(problems.dummy_problem) + x = sp.var('x') + sp.set_domain(x, [1, 3, 5]) + assert x.cs_get_dom(sp) == c.FiniteDomain([1, 3, 5]) def test_bind_with_domain(self): - x = cs.var('x') - cs.set_domain(x, [1, 2, 3]) - raises(cs.OutOfDomain, cs.bind, x, 42) - cs.bind(x, 3) + sp = space.ComputationSpace(problems.dummy_problem) + x = sp.var('x') + sp.set_domain(x, [1, 2, 3]) + raises(space.OutOfDomain, sp.bind, x, 42) + sp.bind(x, 3) assert x.val == 3 def test_bind_with_incompatible_domains(self): - x, y = cs.var('x'), cs.var('y') - cs.set_domain(x, [1, 2]) - cs.set_domain(y, [3, 4]) - raises(cs.IncompatibleDomains, cs.bind, x, y) - cs.set_domain(y, [2, 4]) - cs.bind(x, y) + sp = space.ComputationSpace(problems.dummy_problem) + x, y = sp.var('x'), sp.var('y') + sp.set_domain(x, [1, 2]) + sp.set_domain(y, [3, 4]) + raises(space.IncompatibleDomains, sp.bind, x, y) + sp.set_domain(y, [2, 4]) + sp.bind(x, y) # check x and y are in the same equiv. set assert x.val == y.val def test_unify_with_domains(self): - x,y,z = cs.var('x'), cs.var('y'), cs.var('z') - cs.bind(x, [42, z]) - cs.bind(y, [z, 42]) - cs.set_domain(z, [1, 2, 3]) - raises(cs.UnificationFailure, cs.unify, x, y) - cs.set_domain(z, [41, 42, 43]) - cs.unify(x, y) + sp = space.ComputationSpace(problems.dummy_problem) + x,y,z = sp.var('x'), sp.var('y'), sp.var('z') + sp.bind(x, [42, z]) + sp.bind(y, [z, 42]) + sp.set_domain(z, [1, 2, 3]) + raises(space.UnificationFailure, sp.unify, x, y) + sp.set_domain(z, [41, 42, 43]) + sp.unify(x, y) assert z.val == 42 - assert z.cs_get_dom(cs._cs) == c.FiniteDomain([41, 42, 43]) + assert z.cs_get_dom(sp) == c.FiniteDomain([41, 42, 43]) def test_add_constraint(self): - x,y,z = cs.var('x'), cs.var('y'), cs.var('z') + sp = space.ComputationSpace(problems.dummy_problem) + x,y,z = sp.var('x'), sp.var('y'), sp.var('z') raises(c.DomainlessVariables, - c.Expression, cs._cs, [x, y, z], 'x == y + z') - x.cs_set_dom(cs._cs, c.FiniteDomain([1, 2])) - y.cs_set_dom(cs._cs, c.FiniteDomain([2, 3])) - z.cs_set_dom(cs._cs, c.FiniteDomain([3, 4])) - k = c.Expression(cs._cs, [x, y, z], 'x == y + z') - cs.add_constraint(k) - assert k in cs._cs.constraints + c.Expression, sp, [x, y, z], 'x == y + z') + x.cs_set_dom(sp, c.FiniteDomain([1, 2])) + y.cs_set_dom(sp, c.FiniteDomain([2, 3])) + z.cs_set_dom(sp, c.FiniteDomain([3, 4])) + k = c.Expression(sp, [x, y, z], 'x == y + z') + sp.add_constraint(k) + assert k in sp.constraints def test_narrowing_domains_failure(self): - x,y,z = cs.var('x'), cs.var('y'), cs.var('z') - x.cs_set_dom(cs._cs, c.FiniteDomain([1, 2])) - y.cs_set_dom(cs._cs, c.FiniteDomain([2, 3])) - z.cs_set_dom(cs._cs, c.FiniteDomain([3, 4])) - k = c.Expression(cs._cs, [x, y, z], 'x == y + z') + sp = space.ComputationSpace(problems.dummy_problem) + x,y,z = sp.var('x'), sp.var('y'), sp.var('z') + x.cs_set_dom(sp, c.FiniteDomain([1, 2])) + y.cs_set_dom(sp, c.FiniteDomain([2, 3])) + z.cs_set_dom(sp, c.FiniteDomain([3, 4])) + k = c.Expression(sp, [x, y, z], 'x == y + z') raises(c.ConsistencyFailure, k.narrow) def test_narrowing_domains_success(self): - x,y,z = cs.var('x'), cs.var('y'), cs.var('z') - x.cs_set_dom(cs._cs, c.FiniteDomain([1, 2, 5])) - y.cs_set_dom(cs._cs, c.FiniteDomain([2, 3])) - z.cs_set_dom(cs._cs, c.FiniteDomain([3, 4])) - k = c.Expression(cs._cs, [x, y, z], 'x == y + z') + sp = space.ComputationSpace(problems.dummy_problem) + x,y,z = sp.var('x'), sp.var('y'), sp.var('z') + x.cs_set_dom(sp, c.FiniteDomain([1, 2, 5])) + y.cs_set_dom(sp, c.FiniteDomain([2, 3])) + z.cs_set_dom(sp, c.FiniteDomain([3, 4])) + k = c.Expression(sp, [x, y, z], 'x == y + z') k.narrow() - assert x.cs_get_dom(cs._cs) == c.FiniteDomain([5]) - assert y.cs_get_dom(cs._cs) == c.FiniteDomain([2]) - assert z.cs_get_dom(cs._cs) == c.FiniteDomain([3]) + assert x.cs_get_dom(sp) == c.FiniteDomain([5]) + assert y.cs_get_dom(sp) == c.FiniteDomain([2]) + assert z.cs_get_dom(sp) == c.FiniteDomain([3]) def test_compute_dependant_vars(self): - x,y,z,w = (cs.var('x'), cs.var('y'), - cs.var('z'), cs.var('w')) - x.cs_set_dom(cs._cs, c.FiniteDomain([1, 2, 5])) - y.cs_set_dom(cs._cs, c.FiniteDomain([2, 3])) - z.cs_set_dom(cs._cs, c.FiniteDomain([3, 4])) - w.cs_set_dom(cs._cs, c.FiniteDomain([1, 4, 5])) - k1 = c.Expression(cs._cs, [x, y, z], 'x == y + z') - k2 = c.Expression(cs._cs, [z, w], 'z < w') - cs.add_constraint(k1) - cs.add_constraint(k2) + sp = space.ComputationSpace(problems.dummy_problem) + x,y,z,w = (sp.var('x'), sp.var('y'), + sp.var('z'), sp.var('w')) + x.cs_set_dom(sp, c.FiniteDomain([1, 2, 5])) + y.cs_set_dom(sp, c.FiniteDomain([2, 3])) + z.cs_set_dom(sp, c.FiniteDomain([3, 4])) + w.cs_set_dom(sp, c.FiniteDomain([1, 4, 5])) + k1 = c.Expression(sp, [x, y, z], 'x == y + z') + k2 = c.Expression(sp, [z, w], 'z < w') + sp.add_constraint(k1) + sp.add_constraint(k2) varset = set() constset = set() - cs._cs._compute_dependant_vars(k1, varset, constset) + sp._compute_dependant_vars(k1, varset, constset) assert varset == set([x, y, z, w]) assert constset == set([k1, k2]) def test_store_satisfiable_success(self): - x,y,z = cs.var('x'), cs.var('y'), cs.var('z') - x.cs_set_dom(cs._cs, c.FiniteDomain([1, 2, 5])) - y.cs_set_dom(cs._cs, c.FiniteDomain([2, 3])) - z.cs_set_dom(cs._cs, c.FiniteDomain([3, 4])) - k = c.Expression(cs._cs, [x, y, z], 'x == y + z') - cs.add_constraint(k) - assert cs.satisfiable(k) == True - assert x.cs_get_dom(cs._cs) == c.FiniteDomain([1, 2, 5]) - assert y.cs_get_dom(cs._cs) == c.FiniteDomain([2, 3]) - assert z.cs_get_dom(cs._cs) == c.FiniteDomain([3, 4]) + sp = space.ComputationSpace(problems.dummy_problem) + x,y,z = sp.var('x'), sp.var('y'), sp.var('z') + x.cs_set_dom(sp, c.FiniteDomain([1, 2, 5])) + y.cs_set_dom(sp, c.FiniteDomain([2, 3])) + z.cs_set_dom(sp, c.FiniteDomain([3, 4])) + k = c.Expression(sp, [x, y, z], 'x == y + z') + sp.add_constraint(k) + assert sp.satisfiable(k) == True + assert x.cs_get_dom(sp) == c.FiniteDomain([1, 2, 5]) + assert y.cs_get_dom(sp) == c.FiniteDomain([2, 3]) + assert z.cs_get_dom(sp) == c.FiniteDomain([3, 4]) def test_store_satisfiable_failure(self): - x,y,z = cs.var('x'), cs.var('y'), cs.var('z') - x.cs_set_dom(cs._cs, c.FiniteDomain([1, 2])) - y.cs_set_dom(cs._cs, c.FiniteDomain([2, 3])) - z.cs_set_dom(cs._cs, c.FiniteDomain([3, 4])) - k = c.Expression(cs._cs, [x, y, z], 'x == y + z') - cs.add_constraint(k) - assert cs.satisfiable(k) == False - assert x.cs_get_dom(cs._cs) == c.FiniteDomain([1, 2]) - assert y.cs_get_dom(cs._cs) == c.FiniteDomain([2, 3]) - assert z.cs_get_dom(cs._cs) == c.FiniteDomain([3, 4]) + sp = space.ComputationSpace(problems.dummy_problem) + x,y,z = sp.var('x'), sp.var('y'), sp.var('z') + x.cs_set_dom(sp, c.FiniteDomain([1, 2])) + y.cs_set_dom(sp, c.FiniteDomain([2, 3])) + z.cs_set_dom(sp, c.FiniteDomain([3, 4])) + k = c.Expression(sp, [x, y, z], 'x == y + z') + sp.add_constraint(k) + assert sp.satisfiable(k) == False + assert x.cs_get_dom(sp) == c.FiniteDomain([1, 2]) + assert y.cs_get_dom(sp) == c.FiniteDomain([2, 3]) + assert z.cs_get_dom(sp) == c.FiniteDomain([3, 4]) def test_satisfiable_many_const_success(self): - x,y,z,w = (cs.var('x'), cs.var('y'), - cs.var('z'), cs.var('w')) - x.cs_set_dom(cs._cs, c.FiniteDomain([1, 2, 5])) - y.cs_set_dom(cs._cs, c.FiniteDomain([2, 3])) - z.cs_set_dom(cs._cs, c.FiniteDomain([3, 4])) - w.cs_set_dom(cs._cs, c.FiniteDomain([1, 4, 5])) - k1 = c.Expression(cs._cs, [x, y, z], 'x == y + z') - k2 = c.Expression(cs._cs, [z, w], 'z < w') - cs.add_constraint(k1) - cs.add_constraint(k2) - assert cs.satisfiable(k1) == True - assert x.cs_get_dom(cs._cs) == c.FiniteDomain([1, 2, 5]) - assert y.cs_get_dom(cs._cs) == c.FiniteDomain([2, 3]) - assert z.cs_get_dom(cs._cs) == c.FiniteDomain([3, 4]) - assert w.cs_get_dom(cs._cs) == c.FiniteDomain([1, 4, 5]) - assert cs.satisfiable(k2) == True - assert x.cs_get_dom(cs._cs) == c.FiniteDomain([1, 2, 5]) - assert y.cs_get_dom(cs._cs) == c.FiniteDomain([2, 3]) - assert z.cs_get_dom(cs._cs) == c.FiniteDomain([3, 4]) - assert w.cs_get_dom(cs._cs) == c.FiniteDomain([1, 4, 5]) - narrowed_doms = cs.get_satisfying_domains(k1) + sp = space.ComputationSpace(problems.dummy_problem) + x,y,z,w = (sp.var('x'), sp.var('y'), + sp.var('z'), sp.var('w')) + x.cs_set_dom(sp, c.FiniteDomain([1, 2, 5])) + y.cs_set_dom(sp, c.FiniteDomain([2, 3])) + z.cs_set_dom(sp, c.FiniteDomain([3, 4])) + w.cs_set_dom(sp, c.FiniteDomain([1, 4, 5])) + k1 = c.Expression(sp, [x, y, z], 'x == y + z') + k2 = c.Expression(sp, [z, w], 'z < w') + sp.add_constraint(k1) + sp.add_constraint(k2) + assert sp.satisfiable(k1) == True + assert x.cs_get_dom(sp) == c.FiniteDomain([1, 2, 5]) + assert y.cs_get_dom(sp) == c.FiniteDomain([2, 3]) + assert z.cs_get_dom(sp) == c.FiniteDomain([3, 4]) + assert w.cs_get_dom(sp) == c.FiniteDomain([1, 4, 5]) + assert sp.satisfiable(k2) == True + assert x.cs_get_dom(sp) == c.FiniteDomain([1, 2, 5]) + assert y.cs_get_dom(sp) == c.FiniteDomain([2, 3]) + assert z.cs_get_dom(sp) == c.FiniteDomain([3, 4]) + assert w.cs_get_dom(sp) == c.FiniteDomain([1, 4, 5]) + narrowed_doms = sp.get_satisfying_domains(k1) assert narrowed_doms == {x:c.FiniteDomain([5]), y:c.FiniteDomain([2]), z:c.FiniteDomain([3]), w:c.FiniteDomain([4, 5])} - narrowed_doms = cs.get_satisfying_domains(k2) + narrowed_doms = sp.get_satisfying_domains(k2) assert narrowed_doms == {x:c.FiniteDomain([5]), y:c.FiniteDomain([2]), z:c.FiniteDomain([3]), @@ -340,70 +392,73 @@ def test_satisfiable_many_const_failure(self): - x,y,z,w = (cs.var('x'), cs.var('y'), - cs.var('z'), cs.var('w')) - x.cs_set_dom(cs._cs, c.FiniteDomain([1, 2, 5])) - y.cs_set_dom(cs._cs, c.FiniteDomain([2, 3])) - z.cs_set_dom(cs._cs, c.FiniteDomain([3, 4])) - w.cs_set_dom(cs._cs, c.FiniteDomain([1])) - k1 = c.Expression(cs._cs, [x, y, z], 'x == y + z') - k2 = c.Expression(cs._cs, [z, w], 'z < w') - cs.add_constraint(k1) - cs.add_constraint(k2) - assert cs.satisfiable(k1) == False - assert x.cs_get_dom(cs._cs) == c.FiniteDomain([1, 2, 5]) - assert y.cs_get_dom(cs._cs) == c.FiniteDomain([2, 3]) - assert z.cs_get_dom(cs._cs) == c.FiniteDomain([3, 4]) - assert w.cs_get_dom(cs._cs) == c.FiniteDomain([1]) - assert cs.satisfiable(k2) == False - assert x.cs_get_dom(cs._cs) == c.FiniteDomain([1, 2, 5]) - assert y.cs_get_dom(cs._cs) == c.FiniteDomain([2, 3]) - assert z.cs_get_dom(cs._cs) == c.FiniteDomain([3, 4]) - assert w.cs_get_dom(cs._cs) == c.FiniteDomain([1]) - narrowed_doms = cs.get_satisfying_domains(k1) + sp = space.ComputationSpace(problems.dummy_problem) + x,y,z,w = (sp.var('x'), sp.var('y'), + sp.var('z'), sp.var('w')) + x.cs_set_dom(sp, c.FiniteDomain([1, 2, 5])) + y.cs_set_dom(sp, c.FiniteDomain([2, 3])) + z.cs_set_dom(sp, c.FiniteDomain([3, 4])) + w.cs_set_dom(sp, c.FiniteDomain([1])) + k1 = c.Expression(sp, [x, y, z], 'x == y + z') + k2 = c.Expression(sp, [z, w], 'z < w') + sp.add_constraint(k1) + sp.add_constraint(k2) + assert sp.satisfiable(k1) == False + assert x.cs_get_dom(sp) == c.FiniteDomain([1, 2, 5]) + assert y.cs_get_dom(sp) == c.FiniteDomain([2, 3]) + assert z.cs_get_dom(sp) == c.FiniteDomain([3, 4]) + assert w.cs_get_dom(sp) == c.FiniteDomain([1]) + assert sp.satisfiable(k2) == False + assert x.cs_get_dom(sp) == c.FiniteDomain([1, 2, 5]) + assert y.cs_get_dom(sp) == c.FiniteDomain([2, 3]) + assert z.cs_get_dom(sp) == c.FiniteDomain([3, 4]) + assert w.cs_get_dom(sp) == c.FiniteDomain([1]) + narrowed_doms = sp.get_satisfying_domains(k1) assert narrowed_doms == {} - narrowed_doms = cs.get_satisfying_domains(k2) + narrowed_doms = sp.get_satisfying_domains(k2) assert narrowed_doms == {} def test_satisfy_many_const_failure(self): - x,y,z,w = (cs.var('x'), cs.var('y'), - cs.var('z'), cs.var('w')) - x.cs_set_dom(cs._cs, c.FiniteDomain([1, 2, 5])) - y.cs_set_dom(cs._cs, c.FiniteDomain([2, 3])) - z.cs_set_dom(cs._cs, c.FiniteDomain([3, 4])) - w.cs_set_dom(cs._cs, c.FiniteDomain([1])) - k1 = c.Expression(cs._cs, [x, y, z], 'x == y + z') - k2 = c.Expression(cs._cs, [z, w], 'z < w') - cs.add_constraint(k1) - cs.add_constraint(k2) - raises(cs.ConsistencyFailure, cs.satisfy, k1) - assert x.cs_get_dom(cs._cs) == c.FiniteDomain([1, 2, 5]) - assert y.cs_get_dom(cs._cs) == c.FiniteDomain([2, 3]) - assert z.cs_get_dom(cs._cs) == c.FiniteDomain([3, 4]) - assert w.cs_get_dom(cs._cs) == c.FiniteDomain([1]) - raises(cs.ConsistencyFailure, cs.satisfy, k2) - assert x.cs_get_dom(cs._cs) == c.FiniteDomain([1, 2, 5]) - assert y.cs_get_dom(cs._cs) == c.FiniteDomain([2, 3]) - assert z.cs_get_dom(cs._cs) == c.FiniteDomain([3, 4]) - assert w.cs_get_dom(cs._cs) == c.FiniteDomain([1]) + sp = space.ComputationSpace(problems.dummy_problem) + x,y,z,w = (sp.var('x'), sp.var('y'), + sp.var('z'), sp.var('w')) + x.cs_set_dom(sp, c.FiniteDomain([1, 2, 5])) + y.cs_set_dom(sp, c.FiniteDomain([2, 3])) + z.cs_set_dom(sp, c.FiniteDomain([3, 4])) + w.cs_set_dom(sp, c.FiniteDomain([1])) + k1 = c.Expression(sp, [x, y, z], 'x == y + z') + k2 = c.Expression(sp, [z, w], 'z < w') + sp.add_constraint(k1) + sp.add_constraint(k2) + raises(space.ConsistencyFailure, sp.satisfy, k1) + assert x.cs_get_dom(sp) == c.FiniteDomain([1, 2, 5]) + assert y.cs_get_dom(sp) == c.FiniteDomain([2, 3]) + assert z.cs_get_dom(sp) == c.FiniteDomain([3, 4]) + assert w.cs_get_dom(sp) == c.FiniteDomain([1]) + raises(space.ConsistencyFailure, sp.satisfy, k2) + assert x.cs_get_dom(sp) == c.FiniteDomain([1, 2, 5]) + assert y.cs_get_dom(sp) == c.FiniteDomain([2, 3]) + assert z.cs_get_dom(sp) == c.FiniteDomain([3, 4]) + assert w.cs_get_dom(sp) == c.FiniteDomain([1]) def test_satisfy_many_const_success(self): - x,y,z,w = (cs.var('x'), cs.var('y'), - cs.var('z'), cs.var('w')) - x.cs_set_dom(cs._cs, c.FiniteDomain([1, 2, 5])) - y.cs_set_dom(cs._cs, c.FiniteDomain([2, 3])) - z.cs_set_dom(cs._cs, c.FiniteDomain([3, 4])) - w.cs_set_dom(cs._cs, c.FiniteDomain([1, 4, 5])) - k1 = c.Expression(cs._cs, [x, y, z], 'x == y + z') - k2 = c.Expression(cs._cs, [z, w], 'z < w') - cs.add_constraint(k1) - cs.add_constraint(k2) - cs.satisfy(k2) - print x.cs_get_dom(cs._cs) - assert x.cs_get_dom(cs._cs) == c.FiniteDomain([5]) - assert y.cs_get_dom(cs._cs) == c.FiniteDomain([2]) - assert z.cs_get_dom(cs._cs) == c.FiniteDomain([3]) - assert w.cs_get_dom(cs._cs) == c.FiniteDomain([4, 5]) + sp = space.ComputationSpace(problems.dummy_problem) + x,y,z,w = (sp.var('x'), sp.var('y'), + sp.var('z'), sp.var('w')) + x.cs_set_dom(sp, c.FiniteDomain([1, 2, 5])) + y.cs_set_dom(sp, c.FiniteDomain([2, 3])) + z.cs_set_dom(sp, c.FiniteDomain([3, 4])) + w.cs_set_dom(sp, c.FiniteDomain([1, 4, 5])) + k1 = c.Expression(sp, [x, y, z], 'x == y + z') + k2 = c.Expression(sp, [z, w], 'z < w') + sp.add_constraint(k1) + sp.add_constraint(k2) + sp.satisfy(k2) + print x.cs_get_dom(sp) + assert x.cs_get_dom(sp) == c.FiniteDomain([5]) + assert y.cs_get_dom(sp) == c.FiniteDomain([2]) + assert z.cs_get_dom(sp) == c.FiniteDomain([3]) + assert w.cs_get_dom(sp) == c.FiniteDomain([4, 5]) class TestComputationSpace: @@ -412,30 +467,30 @@ pass def test_bind_cs_root(self): - spc = cs.ComputationSpace(satisfiable_problem) + spc = space.ComputationSpace(problems.satisfiable_problem) assert '__root__' in spc.names assert set(['x', 'y', 'w']) == \ set([var.name for var in spc.root.val]) def test_ask_success(self): - spc = cs.ComputationSpace(one_solution_problem) - assert spc.ask() == cs.Succeeded + spc = space.ComputationSpace(problems.one_solution_problem) + assert spc.ask() == space.Succeeded def test_double_ask(self): - spc = cs.ComputationSpace(one_solution_problem) - assert spc.ask() == cs.Succeeded - assert spc.ask() == cs.Succeeded + spc = space.ComputationSpace(problems.one_solution_problem) + assert spc.ask() == space.Succeeded + assert spc.ask() == space.Succeeded def test_ask_failure(self): - spc = cs.ComputationSpace(unsatisfiable_problem) - assert spc.ask() == cs.Failed + spc = space.ComputationSpace(problems.unsatisfiable_problem) + assert spc.ask() == space.Failed def test_ask_alternatives(self): - spc = cs.ComputationSpace(satisfiable_problem) - assert spc.ask() == cs.Alternatives(2) + spc = space.ComputationSpace(problems.satisfiable_problem) + assert spc.ask() == space.Alternatives(2) def test_distribute(self): - spc = cs.ComputationSpace(satisfiable_problem) + spc = space.ComputationSpace(problems.satisfiable_problem) new_domains = [tuple(d.items()) for d in spc.distributor.distribute()] x, y, z, w = (spc.get_var_by_name('x'), @@ -464,14 +519,14 @@ #assert set(new_domains) == set(expected_domains) def test_clone(self): - spc = cs.ComputationSpace(satisfiable_problem) + spc = space.ComputationSpace(problems.satisfiable_problem) w = spc.get_var_by_name('w') - assert spc.ask() == cs.Alternatives(2) + assert spc.ask() == space.Alternatives(2) new_spc = spc.clone() new_spc.add_constraint(c.Expression(new_spc, [w], 'w == 5')) new_spc._process() - assert spc.ask() == cs.Alternatives(2) - assert new_spc.ask() == cs.Succeeded + assert spc.ask() == space.Alternatives(2) + assert new_spc.ask() == space.Succeeded assert w.cs_get_dom(spc) == c.FiniteDomain([5, 6, 7]) assert w.cs_get_dom(new_spc) == c.FiniteDomain([5]) spc.commit(0) From cfbolz at codespeak.net Tue Jan 31 15:36:31 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Tue, 31 Jan 2006 15:36:31 +0100 (CET) Subject: [pypy-svn] r22897 - in pypy/dist/pypy: objspace/flow objspace/flow/test tool tool/test Message-ID: <20060131143631.02A311009C@code0.codespeak.net> Author: cfbolz Date: Tue Jan 31 15:36:29 2006 New Revision: 22897 Added: pypy/dist/pypy/objspace/flow/test/test_picklegraph.py pypy/dist/pypy/tool/picklesupport.py pypy/dist/pypy/tool/test/test_picklesupport.py Modified: pypy/dist/pypy/objspace/flow/model.py Log: add __getstate__ and __setstate__ to some of the classes in flow/model.py to make bare graphs (no lltypes attached) pickleable. Modified: pypy/dist/pypy/objspace/flow/model.py ============================================================================== --- pypy/dist/pypy/objspace/flow/model.py (original) +++ pypy/dist/pypy/objspace/flow/model.py Tue Jan 31 15:36:29 2006 @@ -6,6 +6,7 @@ from __future__ import generators from pypy.tool.uid import uid, Hashable from pypy.tool.sourcetools import PY_IDENTIFIER, nice_repr_for_func +from pypy.tool.picklesupport import getstate_with_slots, setstate_with_slots """ memory size before and after introduction of __slots__ @@ -151,6 +152,9 @@ def __repr__(self): return "link from %s to %s" % (str(self.prevblock), str(self.target)) + __getstate__ = getstate_with_slots + __setstate__ = setstate_with_slots + class Block(object): __slots__ = """isstartblock inputargs operations exitswitch @@ -222,6 +226,9 @@ exit.prevblock = self self.exits = exits + __getstate__ = getstate_with_slots + __setstate__ = setstate_with_slots + class Variable(object): __slots__ = ["_name", "_nr", "concretetype"] Added: pypy/dist/pypy/objspace/flow/test/test_picklegraph.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/objspace/flow/test/test_picklegraph.py Tue Jan 31 15:36:29 2006 @@ -0,0 +1,18 @@ +from pypy.objspace.flow.test import test_model +from pypy.objspace.flow.model import checkgraph + +import pickle + + +def test_pickle_block(): + # does not raise + s = pickle.dumps(test_model.graph.startblock) + # does not raise + block = pickle.loads(s) + +def test_pickle_graph(): + # does not raise + s = pickle.dumps(test_model.graph) + # does not raise + newgraph = pickle.loads(s) + checkgraph(newgraph) Added: pypy/dist/pypy/tool/picklesupport.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/tool/picklesupport.py Tue Jan 31 15:36:29 2006 @@ -0,0 +1,50 @@ +import weakref + +def getstate_with_slots(self): + result = [] + for cls in self.__class__.__mro__: + if not hasattr(cls, '__slots__'): + continue + for attr in cls.__slots__: + if attr in ("__weakref__", "__dict__"): + continue + if hasattr(self, attr): + result.append((True, attr, getattr(self, attr))) + else: + result.append((False, attr, None)) + if hasattr(self, "__dict__"): + return result, self.__dict__ + else: + return result, None + +def setstate_with_slots(self, (state, __dict__)): + for i, (is_set, attr, value) in enumerate(state): + if is_set: + # circumvent eventual __setattr__ + desc = getattr(self.__class__, attr, None) + if desc is None: + setattr(self, attr, value) + else: + desc.__set__(self, value) + if __dict__ is not None: + self.__dict__ = __dict__ + + +class pickleable_weakref(object): + __slots__ = "ref" + + def __init__(self, obj): + self.ref = weakref.ref(obj) + + def __call__(self): + return self.ref() + + def __getstate__(self): + return self.ref() + + def __setstate__(self, obj): + self.ref = weakref.ref(obj) + + def __repr__(self): + return repr(self.ref) + Added: pypy/dist/pypy/tool/test/test_picklesupport.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/tool/test/test_picklesupport.py Tue Jan 31 15:36:29 2006 @@ -0,0 +1,104 @@ +from pypy.tool.picklesupport import * + +import pickle +import py + +class A(object): + __slots__ = ("a", "b") + __getstate__ = getstate_with_slots + __setstate__ = setstate_with_slots + +class B(A): + __slots__ = ("c", "d") + +class C(B): + pass + +def test_slot_getstate_setstate(): + b = B() + b.a, b.c = range(2) + s = pickle.dumps(b) + new_b = pickle.loads(s) + assert new_b.a == b.a + py.test.raises(AttributeError, "new_b.b") + assert new_b.c == b.c + py.test.raises(AttributeError, "new_b.d") + +def test_slot_getstate_setstate_with_dict(): + c = C() + assert '__dict__' in dir(c) + c.a, c.b, c.c, c.d, c.e = range(5) + s = pickle.dumps(c) + new_c = pickle.loads(s) + assert new_c.a == c.a + assert new_c.b == c.b + assert new_c.c == c.c + assert new_c.d == c.d + assert new_c.e == c.e + +class D(B): + __slots__ = '__weakref__' + +def test_slot_getstate_setstate_with_weakref(): + d = D() + s = pickle.dumps(d) + new_d = pickle.loads(s) + assert new_d.__weakref__ is None + +def test_pickleable_weakref(): + d = D() + ref = pickleable_weakref(d) + s = pickle.dumps((d, ref)) + new_d, new_ref = pickle.loads(s) + assert new_ref() is new_d + del new_d + assert new_ref() is None + +def test_pickleable_weakref_dieing(): + d = D() + ref = pickleable_weakref(d) + s = pickle.dumps(ref) + new_ref = pickle.loads(s) + assert new_ref() is None + +class E(B): + __slots__ = () + def __init__(self, a, b, c, d): + self.__class__.a.__set__(self, a) + self.__class__.b.__set__(self, b) + self.__class__.c.__set__(self, c) + self.__class__.d.__set__(self, d) + def __getattr__(self, attr): + raise AttributeError("not found") + +def test_getsetstate_getattr(): + e = E(1, 2, 3, 4) + s = pickle.dumps(e) + new_e = pickle.loads(s) + assert new_e.a == 1 + assert new_e.b == 2 + assert new_e.c == 3 + assert new_e.d == 4 + +class F(B): + __slots__ = "__dict__" + def __init__(self, a, b, c, d, e): + self.__class__.a.__set__(self, a) + self.__class__.b.__set__(self, b) + self.__class__.c.__set__(self, c) + self.__class__.d.__set__(self, d) + self.__dict__['e'] = e + def __getattr__(self, attr): + raise AttributeError("not found") + +def test_getsetstate_getattr_withdict(): + f = F(1, 2, 3, 4, 5) + assert '__dict__' in dir(f) + s = pickle.dumps(f) + new_f = pickle.loads(s) + assert new_f.a == 1 + assert new_f.b == 2 + assert new_f.c == 3 + assert new_f.d == 4 + assert new_f.e == 5 + From auc at codespeak.net Tue Jan 31 15:36:45 2006 From: auc at codespeak.net (auc at codespeak.net) Date: Tue, 31 Jan 2006 15:36:45 +0100 (CET) Subject: [pypy-svn] r22898 - pypy/dist/pypy/lib/logic Message-ID: <20060131143645.8D4FA100A0@code0.codespeak.net> Author: auc Date: Tue Jan 31 15:36:42 2006 New Revision: 22898 Modified: pypy/dist/pypy/lib/logic/distributor.py Log: fix ref. to choice Modified: pypy/dist/pypy/lib/logic/distributor.py ============================================================================== --- pypy/dist/pypy/lib/logic/distributor.py (original) +++ pypy/dist/pypy/lib/logic/distributor.py Tue Jan 31 15:36:42 2006 @@ -145,8 +145,8 @@ nb_subspaces = self.nb_subdomains() values = variable.cs_get_dom(self.cs).get_values() nb_elts = max(1, len(values)*1./nb_subspaces) - start, end = (int(math.floor(index * nb_elts)), - int(math.floor((index + 1) * nb_elts))) + start, end = (int(math.floor(choice * nb_elts)), + int(math.floor((choice + 1) * nb_elts))) variable.cs_get_dom(self.cs).remove_values(values[:start]) variable.cs_get_dom(self.cs).remove_values(values[end:]) From auc at codespeak.net Tue Jan 31 15:37:46 2006 From: auc at codespeak.net (auc at codespeak.net) Date: Tue, 31 Jan 2006 15:37:46 +0100 (CET) Subject: [pypy-svn] r22899 - in pypy/dist/pypy/lib/logic: . computation_space Message-ID: <20060131143746.EBA67100A0@code0.codespeak.net> Author: auc Date: Tue Jan 31 15:37:44 2006 New Revision: 22899 Added: pypy/dist/pypy/lib/logic/computation_space/ pypy/dist/pypy/lib/logic/computation_space/computationspace.py - copied unchanged from r22895, pypy/dist/pypy/lib/logic/computationspace.py pypy/dist/pypy/lib/logic/computation_space/constraint.py - copied unchanged from r22885, pypy/dist/pypy/lib/logic/constraint.py pypy/dist/pypy/lib/logic/computation_space/distributor.py - copied unchanged from r22898, pypy/dist/pypy/lib/logic/distributor.py pypy/dist/pypy/lib/logic/computation_space/problems.py - copied unchanged from r22894, pypy/dist/pypy/lib/logic/problems.py pypy/dist/pypy/lib/logic/computation_space/state.py - copied unchanged from r22885, pypy/dist/pypy/lib/logic/state.py pypy/dist/pypy/lib/logic/computation_space/test_computationspace.py - copied unchanged from r22896, pypy/dist/pypy/lib/logic/test_computationspace.py pypy/dist/pypy/lib/logic/computation_space/test_variable.py - copied unchanged from r22885, pypy/dist/pypy/lib/logic/test_variable.py pypy/dist/pypy/lib/logic/computation_space/variable.py - copied unchanged from r22893, pypy/dist/pypy/lib/logic/variable.py Removed: pypy/dist/pypy/lib/logic/computationspace.py pypy/dist/pypy/lib/logic/constraint.py pypy/dist/pypy/lib/logic/distributor.py pypy/dist/pypy/lib/logic/problems.py pypy/dist/pypy/lib/logic/state.py pypy/dist/pypy/lib/logic/test_computationspace.py pypy/dist/pypy/lib/logic/test_variable.py pypy/dist/pypy/lib/logic/variable.py Log: move computation space stuff into its own directory From cfbolz at codespeak.net Tue Jan 31 15:38:30 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Tue, 31 Jan 2006 15:38:30 +0100 (CET) Subject: [pypy-svn] r22900 - in pypy/dist/pypy/rpython/lltypesystem: . test Message-ID: <20060131143830.AB2F7100A0@code0.codespeak.net> Author: cfbolz Date: Tue Jan 31 15:38:29 2006 New Revision: 22900 Added: pypy/dist/pypy/rpython/lltypesystem/test/test_picklelltype.py Modified: pypy/dist/pypy/rpython/lltypesystem/lltype.py Log: make some of the classes in lltype pickleable Modified: pypy/dist/pypy/rpython/lltypesystem/lltype.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/lltype.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/lltype.py Tue Jan 31 15:38:29 2006 @@ -1,9 +1,9 @@ -import weakref import py from pypy.rpython.rarithmetic import r_uint, r_ulonglong, r_longlong from pypy.rpython.objectmodel import Symbolic from pypy.tool.uid import Hashable from pypy.tool.tls import tlsobject +from pypy.tool.picklesupport import getstate_with_slots, setstate_with_slots, pickleable_weakref from types import NoneType from sys import maxint @@ -118,6 +118,9 @@ def _is_varsize(self): return False + __getstate__ = getstate_with_slots + __setstate__ = setstate_with_slots + NFOUND = object() class ContainerType(LowLevelType): @@ -647,7 +650,7 @@ obj0 = pointing_to else: self._set_weak(True) - obj0 = weakref.ref(pointing_to) + obj0 = pickleable_weakref(pointing_to) self._set_solid(solid) self._set_obj0(obj0) @@ -751,6 +754,9 @@ return callb(*args) raise TypeError("%r instance is not a function" % (self._T,)) + __getstate__ = getstate_with_slots + __setstate__ = setstate_with_slots + assert not '__dict__' in dir(_ptr) class _parentable(object): @@ -766,7 +772,7 @@ self._TYPE = TYPE def _setparentstructure(self, parent, parentindex): - self._wrparent = weakref.ref(parent) + self._wrparent = pickleable_weakref(parent) self._parent_type = typeOf(parent) self._parent_index = parentindex if (isinstance(self._parent_type, Struct) @@ -789,6 +795,8 @@ def _check(self): self._parentstructure() + __getstate__ = getstate_with_slots + __setstate__ = setstate_with_slots def _struct_variety(flds, cache={}): flds = list(flds) @@ -801,7 +809,12 @@ __slots__ = flds cache[tag] = _struct1 return _struct1 - + +#for pickling support: +def _get_empty_instance_of_struct_variety(flds): + cls = _struct_variety(flds) + return object.__new__(cls) + class _struct(_parentable): _kind = "structure" @@ -843,6 +856,11 @@ def __str__(self): return 'struct %s { %s }' % (self._TYPE._name, self._str_fields()) + def __reduce__(self): + return _get_empty_instance_of_struct_variety, (self.__slots__, ), getstate_with_slots(self) + + __setstate__ = setstate_with_slots + class _array(_parentable): _kind = "array" Added: pypy/dist/pypy/rpython/lltypesystem/test/test_picklelltype.py ============================================================================== --- (empty file) +++ pypy/dist/pypy/rpython/lltypesystem/test/test_picklelltype.py Tue Jan 31 15:38:29 2006 @@ -0,0 +1,408 @@ +from pypy.rpython.lltypesystem.lltype import * + +import pickle + +def isweak(p, T): + return p._weak and typeOf(p).TO == T + +def test_pickle_types(): + S0 = GcStruct("s0", ('a', Signed), ('b', Signed)) + p_S0 = pickle.dumps(S0) + r_S0 = pickle.loads(p_S0) + assert S0 == r_S0 + assert hash(S0) == hash(r_S0) + s0 = malloc(S0) + print s0 + s0.a = 1 + s0.b = s0.a + p_s0 = pickle.dumps(s0) + r_s0 = pickle.loads(p_s0) + # simple array + Ar = GcArray(('v', Signed)) + p_Ar = pickle.dumps(Ar) + r_Ar = pickle.loads(p_Ar) + assert Ar == r_Ar + x = malloc(Ar,0) + p_x = pickle.dumps(x) + r_x = pickle.loads(p_x) + assert len(r_x) == 0 + x = malloc(Ar,3) + x[0].v = 1 + x[1].v = 2 + x[2].v = 3 + p_x = pickle.dumps(x) + r_x = pickle.loads(p_x) + assert typeOf(x) == Ptr(Ar) + assert isweak(x[0], Ar.OF) + assert typeOf(x[0].v) == Signed + assert [x[z].v for z in range(3)] == [1, 2, 3] + # + def define_list(T): + List_typ = GcStruct("list", + ("items", Ptr(GcArray(('item',T))))) + def newlist(): + l = malloc(List_typ) + items = malloc(List_typ.items.TO, 0) + l.items = items + return l + + def append(l, newitem): + length = len(l.items) + newitems = malloc(List_typ.items.TO, length+1) + i = 0 + while i Author: auc Date: Tue Jan 31 15:49:53 2006 New Revision: 22901 Modified: pypy/dist/pypy/lib/logic/computation_space/test_variable.py Log: last round of fix removing globals (should be) Modified: pypy/dist/pypy/lib/logic/computation_space/test_variable.py ============================================================================== --- pypy/dist/pypy/lib/logic/computation_space/test_variable.py (original) +++ pypy/dist/pypy/lib/logic/computation_space/test_variable.py Tue Jan 31 15:49:53 2006 @@ -2,9 +2,9 @@ from py.test import raises -import computationspace as u +import computationspace as space import variable as v -from problems import * +from problems import dummy_problem class Consumer(Thread): @@ -24,46 +24,48 @@ class TestVariable: - def setup_method(self, meth): - u._cs = u.ComputationSpace(dummy_problem) - def test_no_same_name(self): - x = u.var('x') - raises(u.AlreadyInStore, u.var, 'x') + sp = space.ComputationSpace(dummy_problem) + x = sp.var('x') + raises(space.AlreadyInStore, sp.var, 'x') def test_get_by_name(self): - x = u.var('x') - assert x == u._cs.get_var_by_name('x') - raises(u.NotInStore, u._cs.get_var_by_name, 'y') + sp = space.ComputationSpace(dummy_problem) + x = sp.var('x') + assert x == sp.get_var_by_name('x') + raises(space.NotInStore, sp.get_var_by_name, 'y') def test_one_thread_reading_one_var(self): + sp = space.ComputationSpace(dummy_problem) cons = Consumer() - x = u.var('x') + x = sp.var('x') cons.give_var(x) cons.start() - u.bind(x, 42) + sp.bind(x, 42) cons.join() assert cons.var.val == 42 def test_many_threads_reading_one_var(self): + sp = space.ComputationSpace(dummy_problem) conss = [Consumer() for i in range(10)] - x = u.var('x') + x = sp.var('x') for cons in conss: cons.give_var(x) cons.start() - u.bind(x, 42) + sp.bind(x, 42) for cons in conss: cons.join() assert cons.var.val == 42 def test_many_thread_reading_many_var(self): + sp = space.ComputationSpace(dummy_problem) conss = [NConsumer() for i in range(10)] - vars_ = [u.var(str(i)) for i in range(10)] + vars_ = [sp.var(str(i)) for i in range(10)] for cons in conss: cons.give_vars(vars_) cons.start() for var in vars_: - u.bind(var, var.name) + sp.bind(var, var.name) for cons in conss: cons.join() for i in range(10): From cfbolz at codespeak.net Tue Jan 31 15:57:04 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Tue, 31 Jan 2006 15:57:04 +0100 (CET) Subject: [pypy-svn] r22902 - pypy/dist/pypy/rpython/lltypesystem/test Message-ID: <20060131145704.BB24C10095@code0.codespeak.net> Author: cfbolz Date: Tue Jan 31 15:54:06 2006 New Revision: 22902 Modified: pypy/dist/pypy/rpython/lltypesystem/test/test_picklelltype.py Log: add another test for correct hashes of unpickled lltypes Modified: pypy/dist/pypy/rpython/lltypesystem/test/test_picklelltype.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/test/test_picklelltype.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/test/test_picklelltype.py Tue Jan 31 15:54:06 2006 @@ -216,7 +216,7 @@ assert not p10 -def DONOTtest_hash(): +def test_hash(): S = ForwardReference() S.become(Struct('S', ('p', Ptr(S)))) assert S == S @@ -224,6 +224,8 @@ S1 = Struct('S', ('p', Ptr(S))) assert S1 == S assert S == S1 + r_S = pickle.loads(pickle.dumps(S)) + r_S1 = pickle.loads(pickle.dumps(S1)) assert hash(S1) == hash(S) def DONOTtest_array_with_non_container_elements(): From auc at codespeak.net Tue Jan 31 18:05:23 2006 From: auc at codespeak.net (auc at codespeak.net) Date: Tue, 31 Jan 2006 18:05:23 +0100 (CET) Subject: [pypy-svn] r22903 - pypy/dist/pypy/lib/logic/computation_space Message-ID: <20060131170523.66B32100A9@code0.codespeak.net> Author: auc Date: Tue Jan 31 18:05:22 2006 New Revision: 22903 Modified: pypy/dist/pypy/lib/logic/computation_space/computationspace.py pypy/dist/pypy/lib/logic/computation_space/distributor.py pypy/dist/pypy/lib/logic/computation_space/test_computationspace.py Log: * add inject (to further constrain a space) * fix space/distributor relationship * remove dead code Modified: pypy/dist/pypy/lib/logic/computation_space/computationspace.py ============================================================================== --- pypy/dist/pypy/lib/logic/computation_space/computationspace.py (original) +++ pypy/dist/pypy/lib/logic/computation_space/computationspace.py Tue Jan 31 18:05:22 2006 @@ -160,23 +160,6 @@ ## then it returns alternatives(N), where N is the number of ## alternatives. -## An example specifying how to use a computation space : - -## def my_problem(store): -## #declare variables, their domain -## x, y, z = var('x'), var('y'), var('z') -## #declare constraints -## set_domain(x, FiniteDomain([1, 2])) -## set_domain(y, FiniteDomain([2, 3])) -## set_domain(z, FiniteDomain([42, 43])) -## add_constraint(c.Expression([x, y, z], 'x == y + z')) -## add_constraint(c.Expression([z, w], 'z < w')) -## #set up a distribution strategy -## ???? -## return (x, y, z) - -## space = ComputationSpace(fun=my_problem) - from threading import Thread, Condition, RLock, local from state import Succeeded, Distributable, Failed, Merged @@ -249,9 +232,10 @@ * variables bound to a number, record or procedure (also called determined variables).""" + # we have to enforce only one distributor + # thread running in one space at the same time _nb_choices = 0 - def __init__(self, problem, parent=None): # consistency-preserving stuff self.in_transaction = False @@ -644,15 +628,15 @@ spc = ComputationSpace(NoProblem, parent=self) for var in spc.vars: var.cs_set_dom(spc, var.cs_get_dom(self).copy()) - # check satisfiability of the space - spc._process() + spc.distributor.set_space(spc) if spc.status == Distributable: spc.distributor.start() return spc def inject(self, restricting_problem): - """add additional stuff into a space""" - pass + """add additional entities into a space""" + restricting_problem(self) + self._process() def commit(self, choice): """if self is distributable, causes the Choose call in the @@ -682,39 +666,6 @@ ComputationSpace._nb_choices += 1 return self.var('__choice__'+str(self._nb_choices)) - def make_children(self): - for dommap in self.distributor.distribute(): - cs = ComputationSpace(lambda cs : True, - parent=self) - self.children.add(cs) - for var, dom in dommap.items(): - var.cs_set_dom(cs, dom) - - def solve_all(self): - """recursively solves the problem - """ - if self.status == Unprocessed: - self.process() - if self.status == Succeeded: return self.root - if self.status == Failed: raise Failed - self.make_children() - results = set() # to be merged/committed ? - for cs in self.children: - try: - results.add(cs.solve_all()) - except Failed: - pass - for result in results: - # Q: do we (a) merge our children results right now - # or (b) do we pass results up the call chain ? - # (b) makes sense for a SolveAll kind of method on cs's - # (a) might be more Oz-ish, maybe allowing some very fancy - # stuff with distribution or whatever - self.do_something_with(result) - - - - #-- Unifiability checks--------------------------------------- #-- #-- quite costly & could be merged back in unify @@ -778,36 +729,3 @@ def _both_are_bound(v1, v2): return v1._is_bound() and v2._is_bound() - - -#-- -#-- the global store -#from problems import dummy_problem -#_cs = ComputationSpace(dummy_problem) - -#-- global accessor functions -## def var(name): -## v = Var(name, _cs) -## _cs.add_unbound(v) -## return v - -## def set_domain(var, dom): -## return _cs.set_domain(var, dom) - -## def add_constraint(constraint): -## return _cs.add_constraint(constraint) - -## def satisfiable(constraint): -## return _cs.satisfiable(constraint) - -## def get_satisfying_domains(constraint): -## return _cs.get_satisfying_domains(constraint) - -## def satisfy(constraint): -## return _cs.satisfy(constraint) - -## def bind(var, val): -## return _cs.bind(var, val) - -## def unify(var1, var2): -## return _cs.unify(var1, var2) Modified: pypy/dist/pypy/lib/logic/computation_space/distributor.py ============================================================================== --- pypy/dist/pypy/lib/logic/computation_space/distributor.py (original) +++ pypy/dist/pypy/lib/logic/computation_space/distributor.py Tue Jan 31 18:05:22 2006 @@ -18,6 +18,9 @@ self.nb_subspaces = nb_subspaces self.cs = c_space self.verbose = 0 + + def set_space(self, space): + self.cs = space def findSmallestDomain(self): """returns the variable having the smallest domain. @@ -132,8 +135,6 @@ def run(self): while self.cs.status == Distributable: choice = self.cs.choose(self.nb_subdomains()) - if self.cs.status == Failed: - return self.new_distribute(choice) self.cs._process() Modified: pypy/dist/pypy/lib/logic/computation_space/test_computationspace.py ============================================================================== --- pypy/dist/pypy/lib/logic/computation_space/test_computationspace.py (original) +++ pypy/dist/pypy/lib/logic/computation_space/test_computationspace.py Tue Jan 31 18:05:22 2006 @@ -489,7 +489,7 @@ spc = space.ComputationSpace(problems.satisfiable_problem) assert spc.ask() == space.Alternatives(2) - def test_distribute(self): + def test_old_distribute(self): spc = space.ComputationSpace(problems.satisfiable_problem) new_domains = [tuple(d.items()) for d in spc.distributor.distribute()] @@ -512,13 +512,8 @@ for (e1, e2) in zip(d1, d2): print e1, '=?', e2 assert e1 == e2 - # the following assertion fails for mysterious reasons - # have we discovered a bug in CPython ? - #print hash(new_domains[0]), hash(new_domains[1]) - #print hash(expected_domains[0]), hash(expected_domains[1]) - #assert set(new_domains) == set(expected_domains) - def test_clone(self): + def test_clone_and_distribute(self): spc = space.ComputationSpace(problems.satisfiable_problem) w = spc.get_var_by_name('w') assert spc.ask() == space.Alternatives(2) @@ -529,5 +524,17 @@ assert new_spc.ask() == space.Succeeded assert w.cs_get_dom(spc) == c.FiniteDomain([5, 6, 7]) assert w.cs_get_dom(new_spc) == c.FiniteDomain([5]) - spc.commit(0) - new_spc.commit(0) + + def test_inject(self): + def more_constraints(space): + space.add_constraint(c.Expression(space, [w], 'w == 5')) + spc = space.ComputationSpace(problems.satisfiable_problem) + w = spc.get_var_by_name('w') + assert spc.ask() == space.Alternatives(2) + new_spc = spc.clone() + new_spc.inject(more_constraints) + assert spc.ask() == space.Alternatives(2) + assert new_spc.ask() == space.Succeeded + assert w.cs_get_dom(spc) == c.FiniteDomain([5, 6, 7]) + assert w.cs_get_dom(new_spc) == c.FiniteDomain([5]) + From cfbolz at codespeak.net Tue Jan 31 20:04:45 2006 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Tue, 31 Jan 2006 20:04:45 +0100 (CET) Subject: [pypy-svn] r22904 - in pypy/dist/pypy/rpython/lltypesystem: . test Message-ID: <20060131190445.AD97E100AD@code0.codespeak.net> Author: cfbolz Date: Tue Jan 31 20:04:43 2006 New Revision: 22904 Modified: pypy/dist/pypy/rpython/lltypesystem/lltype.py pypy/dist/pypy/rpython/lltypesystem/test/test_picklelltype.py Log: enable pickling of function ptrs. works only for functions that are globals in a module. Modified: pypy/dist/pypy/rpython/lltypesystem/lltype.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/lltype.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/lltype.py Tue Jan 31 20:04:43 2006 @@ -927,6 +927,19 @@ def __hash__(self): return hash(frozendict(self.__dict__)) + def __getstate__(self): + import pickle, types + __dict__ = self.__dict__.copy() + try: + pickle.dumps(self._callable) + except pickle.PicklingError: + __dict__['_callable'] = None + return __dict__ + + def __setstate__(self, __dict__): + import new + self.__dict__ = __dict__ + class _opaque(_parentable): def __init__(self, TYPE, parent=None, parentindex=None, **attrs): _parentable.__init__(self, TYPE) Modified: pypy/dist/pypy/rpython/lltypesystem/test/test_picklelltype.py ============================================================================== --- pypy/dist/pypy/rpython/lltypesystem/test/test_picklelltype.py (original) +++ pypy/dist/pypy/rpython/lltypesystem/test/test_picklelltype.py Tue Jan 31 20:04:43 2006 @@ -180,13 +180,20 @@ gc.collect() py.test.raises(RuntimeError, "r_p1_5.v") -def DONOTtest_functions(): - F = FuncType((Signed,), Signed) - py.test.raises(TypeError, "Struct('x', ('x', F))") +def example_func(x): + return x + 1 +example_func.suggested_primitive = True +def test_functions(): + F = FuncType((Signed,), Signed) PF = Ptr(F) - pf = PF._example() - assert pf(0) == 0 + r_PF = pickle.loads(pickle.dumps(PF)) + assert PF == r_PF + pf = functionptr(F, 'example_func', _callable=example_func, graph="dummy") + r_pf = pickle.loads(pickle.dumps(pf)) + assert pf(0) == 1 + assert pf._obj._callable.suggested_primitive + assert pf._obj.graph == "dummy" py.test.raises(TypeError, pf, 0, 0) py.test.raises(TypeError, pf, 'a') @@ -215,7 +222,6 @@ assert typeOf(p10) == Ptr(S1) assert not p10 - def test_hash(): S = ForwardReference() S.become(Struct('S', ('p', Ptr(S)))) @@ -328,12 +334,13 @@ s1.sub.x = 1 assert runtime_type_info(s1.sub) == getRuntimeTypeInfo(S1) -def DONOTtest_flavor_malloc(): +def test_flavor_malloc(): S = Struct('s', ('x', Signed)) py.test.raises(TypeError, malloc, S) p = malloc(S, flavor="raw") - assert typeOf(p).TO == S - assert not isweak(p, S) + r_p = pickle.loads(pickle.dumps(p)) + assert typeOf(r_p).TO == S + assert not isweak(r_p, S) def DONOTtest_opaque(): O = OpaqueType('O') @@ -347,22 +354,6 @@ assert typeOf(p2.stuff) == Ptr(O) assert parentlink(p2.stuff._obj) == (p2._obj, 'stuff') -def DONOTtest_is_atomic(): - U = Struct('inlined', ('z', Signed)) - A = Ptr(RuntimeTypeInfo) - P = Ptr(GcStruct('p')) - Q = GcStruct('q', ('i', Signed), ('u', U), ('p', P)) - O = OpaqueType('O') - F = GcForwardReference() - assert A._is_atomic() is True - assert P._is_atomic() is False - assert Q.i._is_atomic() is True - assert Q.u._is_atomic() is True - assert Q.p._is_atomic() is False - assert Q._is_atomic() is False - assert O._is_atomic() is False - assert F._is_atomic() is False - def DONOTtest_adtmeths(): def h_newstruct(): return malloc(S) From pedronis at codespeak.net Tue Jan 31 20:26:38 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Tue, 31 Jan 2006 20:26:38 +0100 (CET) Subject: [pypy-svn] r22907 - pypy/dist/pypy/translator/goal Message-ID: <20060131192638.055E11007F@code0.codespeak.net> Author: pedronis Date: Tue Jan 31 20:26:37 2006 New Revision: 22907 Removed: pypy/dist/pypy/translator/goal/richards.py Log: reverting last change to richards, they brake targetrichards From pedronis at codespeak.net Tue Jan 31 20:28:39 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Tue, 31 Jan 2006 20:28:39 +0100 (CET) Subject: [pypy-svn] r22908 - pypy/dist/pypy/translator/goal Message-ID: <20060131192839.E0C1D1007F@code0.codespeak.net> Author: pedronis Date: Tue Jan 31 20:28:38 2006 New Revision: 22908 Added: pypy/dist/pypy/translator/goal/richards.py - copied unchanged from r18050, pypy/dist/pypy/translator/goal/richards.py Log: revert. From pedronis at codespeak.net Tue Jan 31 22:26:02 2006 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Tue, 31 Jan 2006 22:26:02 +0100 (CET) Subject: [pypy-svn] r22913 - pypy/dist/pypy/module/stackless/test Message-ID: <20060131212602.B944210086@code0.codespeak.net> Author: pedronis Date: Tue Jan 31 22:26:01 2006 New Revision: 22913 Modified: pypy/dist/pypy/module/stackless/test/test_interp_coroutine.py Log: reset coro state before trying to translate coro code. Modified: pypy/dist/pypy/module/stackless/test/test_interp_coroutine.py ============================================================================== --- pypy/dist/pypy/module/stackless/test/test_interp_coroutine.py (original) +++ pypy/dist/pypy/module/stackless/test/test_interp_coroutine.py Tue Jan 31 22:26:01 2006 @@ -5,6 +5,8 @@ import os from pypy.module.stackless.interp_coroutine import costate, Coroutine +costate.__init__() + def wrap_stackless_function(fn): from pypy.translator.translator import TranslationContext from pypy.translator.c.genc import CStandaloneBuilder From ericvrp at codespeak.net Tue Jan 31 23:52:10 2006 From: ericvrp at codespeak.net (ericvrp at codespeak.net) Date: Tue, 31 Jan 2006 23:52:10 +0100 (CET) Subject: [pypy-svn] r22915 - in pypy/dist/pypy/translator: backendopt c/src c/test Message-ID: <20060131225210.0353D10093@code0.codespeak.net> Author: ericvrp Date: Tue Jan 31 23:51:48 2006 New Revision: 22915 Modified: pypy/dist/pypy/translator/backendopt/all.py pypy/dist/pypy/translator/c/src/int.h pypy/dist/pypy/translator/c/test/test_typed.py Log: Disabling raisingop2direct_call transformation until it no longer breaks translation. Reverting int.h until that time too. Modified: pypy/dist/pypy/translator/backendopt/all.py ============================================================================== --- pypy/dist/pypy/translator/backendopt/all.py (original) +++ pypy/dist/pypy/translator/backendopt/all.py Tue Jan 31 23:51:48 2006 @@ -9,7 +9,7 @@ from pypy.translator.backendopt.escape import malloc_to_stack -def backend_optimizations(translator, raisingop2direct_call_all=True, +def backend_optimizations(translator, raisingop2direct_call_all=False, inline_threshold=1, mallocs=True, ssa_form=True, Modified: pypy/dist/pypy/translator/c/src/int.h ============================================================================== --- pypy/dist/pypy/translator/c/src/int.h (original) +++ pypy/dist/pypy/translator/c/src/int.h Tue Jan 31 23:51:48 2006 @@ -1,3 +1,4 @@ + /************************************************************/ /*** C header subsection: operations between ints ***/ @@ -11,9 +12,19 @@ #define OP_INT_NEG(x,r,err) r = -(x) +#define OP_INT_NEG_OVF(x,r,err) \ + OP_INT_NEG(x,r,err); \ + if ((x) >= 0 || (x) != -(x)); \ + else FAIL_OVF(err, "integer negate") + #define OP_INT_ABS(x,r,err) r = (x) >= 0 ? x : -(x) #define OP_UINT_ABS(x,r,err) r = (x) +#define OP_INT_ABS_OVF(x,r,err) \ + OP_INT_ABS(x,r,err); \ + if ((x) >= 0 || (x) != -(x)); \ + else FAIL_OVF(err, "integer absolute") + /*** binary operations ***/ #define OP_INT_EQ(x,y,r,err) r = ((x) == (y)) @@ -30,10 +41,37 @@ #define OP_INT_ADD(x,y,r,err) r = (x) + (y) +#define OP_INT_ADD_OVF(x,y,r,err) \ + OP_INT_ADD(x,y,r,err); \ + if ((r^(x)) >= 0 || (r^(y)) >= 0); \ + else FAIL_OVF(err, "integer addition") + #define OP_INT_SUB(x,y,r,err) r = (x) - (y) +#define OP_INT_SUB_OVF(x,y,r,err) \ + OP_INT_SUB(x,y,r,err); \ + if ((r^(x)) >= 0 || (r^~(y)) >= 0); \ + else FAIL_OVF(err, "integer subtraction") + #define OP_INT_MUL(x,y,r,err) r = (x) * (y) +#ifndef HAVE_LONG_LONG + +#define OP_INT_MUL_OVF(x,y,r,err) \ + if (op_int_mul_ovf(x,y,&r)); \ + else FAIL_OVF(err, "integer multiplication") + +#else + +#define OP_INT_MUL_OVF(x,y,r,err) \ + { \ + PY_LONG_LONG lr = (PY_LONG_LONG)(x) * (PY_LONG_LONG)(y); \ + r = (long)lr; \ + if ((PY_LONG_LONG)r == lr); \ + else FAIL_OVF(err, "integer multiplication"); \ + } +#endif + /* shifting */ /* NB. shifting has same limitations as C: the shift count must be @@ -44,18 +82,68 @@ #define OP_INT_LSHIFT(x,y,r,err) r = (x) << (y) #define OP_UINT_LSHIFT(x,y,r,err) r = (x) << (y) +#define OP_INT_LSHIFT_OVF(x,y,r,err) \ + OP_INT_LSHIFT(x,y,r,err); \ + if ((x) != Py_ARITHMETIC_RIGHT_SHIFT(long, r, (y))) \ + FAIL_OVF(err, "x<= 0) { OP_INT_RSHIFT(x,y,r,err); } \ + else FAIL_VAL(err, "negative shift count") + +#define OP_INT_LSHIFT_VAL(x,y,r,err) \ + if ((y) >= 0) { OP_INT_LSHIFT(x,y,r,err); } \ + else FAIL_VAL(err, "negative shift count") + +#define OP_INT_LSHIFT_OVF_VAL(x,y,r,err) \ + if ((y) >= 0) { OP_INT_LSHIFT_OVF(x,y,r,err); } \ + else FAIL_VAL(err, "negative shift count") + + /* floor division */ -/* NB: OP_INT_FLOORDIV and OP_INT_MOD are the operations that end up in the graphs - after the raisingop2direct_call transformation has been done. */ -#define OP_INT_FLOORDIV(x,y,r,err) r = (x) / (y) +#define OP_INT_FLOORDIV(x,y,r,err) r = op_divmod_adj(x, y, NULL) #define OP_UINT_FLOORDIV(x,y,r,err) r = (x) / (y) +#define OP_INT_FLOORDIV_OVF(x,y,r,err) \ + if ((y) == -1 && (x) < 0 && ((unsigned long)(x) << 1) == 0) \ + FAIL_OVF(err, "integer division"); \ + OP_INT_FLOORDIV(x,y,r,err) + +#define OP_INT_FLOORDIV_ZER(x,y,r,err) \ + if ((y)) { OP_INT_FLOORDIV(x,y,r,err); } \ + else FAIL_ZER(err, "integer division") +#define OP_UINT_FLOORDIV_ZER(x,y,r,err) \ + if ((y)) { OP_UINT_FLOORDIV(x,y,r,err); } \ + else FAIL_ZER(err, "unsigned integer division") + +#define OP_INT_FLOORDIV_OVF_ZER(x,y,r,err) \ + if ((y)) { OP_INT_FLOORDIV_OVF(x,y,r,err); } \ + else FAIL_ZER(err, "integer division") + /* modulus */ -#define OP_INT_MOD(x,y,r,err) r = (x) % (y) +#define OP_INT_MOD(x,y,r,err) op_divmod_adj(x, y, &r) #define OP_UINT_MOD(x,y,r,err) r = (x) % (y) +#define OP_INT_MOD_OVF(x,y,r,err) \ + if ((y) == -1 && (x) < 0 && ((unsigned long)(x) << 1) == 0) \ + FAIL_OVF(err, "integer modulo"); \ + OP_INT_MOD(x,y,r,err) + +#define OP_INT_MOD_ZER(x,y,r,err) \ + if ((y)) { OP_INT_MOD(x,y,r,err); } \ + else FAIL_ZER(err, "integer modulo") +#define OP_UINT_MOD_ZER(x,y,r,err) \ + if ((y)) { OP_UINT_MOD(x,y,r,err); } \ + else FAIL_ZER(err, "unsigned integer modulo") + +#define OP_INT_MOD_OVF_ZER(x,y,r,err) \ + if ((y)) { OP_INT_MOD_OVF(x,y,r,err); } \ + else FAIL_ZER(err, "integer modulo") + /* bit operations */ #define OP_INT_AND(x,y,r,err) r = (x) & (y) @@ -84,6 +172,85 @@ /* _________________ certain implementations __________________ */ +#ifndef HAVE_LONG_LONG +/* adjusted from intobject.c, Python 2.3.3 */ + +/* prototypes */ + +op_int_mul_ovf(long a, long b, long *longprod); + +/* implementations */ + +#ifndef PYPY_NOT_MAIN_FILE + +int +op_int_mul_ovf(long a, long b, long *longprod) +{ + double doubled_longprod; /* (double)longprod */ + double doubleprod; /* (double)a * (double)b */ + + *longprod = a * b; + doubleprod = (double)a * (double)b; + doubled_longprod = (double)*longprod; + + /* Fast path for normal case: small multiplicands, and no info + is lost in either method. */ + if (doubled_longprod == doubleprod) + return 1; + + /* Somebody somewhere lost info. Close enough, or way off? Note + that a != 0 and b != 0 (else doubled_longprod == doubleprod == 0). + The difference either is or isn't significant compared to the + true value (of which doubleprod is a good approximation). + */ + { + const double diff = doubled_longprod - doubleprod; + const double absdiff = diff >= 0.0 ? diff : -diff; + const double absprod = doubleprod >= 0.0 ? doubleprod : + -doubleprod; + /* absdiff/absprod <= 1/32 iff + 32 * absdiff <= absprod -- 5 good bits is "close enough" */ + if (32.0 * absdiff <= absprod) + return 1; + return 0; + } +} + +#endif /* PYPY_NOT_MAIN_FILE */ + +#endif /* HAVE_LONG_LONG */ + +/* XXX we might probe the compiler whether it does what we want */ + +/* prototypes */ + +long op_divmod_adj(long x, long y, long *p_rem); + +/* implementations */ + +#ifndef PYPY_NOT_MAIN_FILE + +long op_divmod_adj(long x, long y, long *p_rem) +{ + long xdivy = x / y; + long xmody = x - xdivy * y; + /* If the signs of x and y differ, and the remainder is non-0, + * C89 doesn't define whether xdivy is now the floor or the + * ceiling of the infinitely precise quotient. We want the floor, + * and we have it iff the remainder's sign matches y's. + */ + if (xmody && ((y ^ xmody) < 0) /* i.e. and signs differ */) { + xmody += y; + --xdivy; + assert(xmody && ((y ^ xmody) >= 0)); + } + if (p_rem) + *p_rem = xmody; + return xdivy; +} + +#endif /* PYPY_NOT_MAIN_FILE */ + /* no editing below this point */ /* following lines are generated by mkuint.py */ Modified: pypy/dist/pypy/translator/c/test/test_typed.py ============================================================================== --- pypy/dist/pypy/translator/c/test/test_typed.py (original) +++ pypy/dist/pypy/translator/c/test/test_typed.py Tue Jan 31 23:51:48 2006 @@ -14,7 +14,7 @@ def process(self, t): _TestAnnotatedTestCase.process(self, t) t.buildrtyper().specialize() - raisingop2direct_call(t) + #raisingop2direct_call(t) def test_call_five(self): # -- the result of call_five() isn't a real list, but an rlist @@ -223,7 +223,7 @@ raises(OverflowError, fn, n, 5) def test_int_mod_ovf_zer(self): - py.test.skip("XXX does not annotate anymore after raisingops2direct_call transformation") + #py.test.skip("XXX does not annotate anymore after raisingops2direct_call transformation") fn = self.getcompiled(snippet.mod_func) raises(OverflowError, fn, -1) raises(ZeroDivisionError, fn, 0)