From tismer at tismer.com  Mon Sep  1 18:16:05 2003
From: tismer at tismer.com (Christian Tismer)
Date: Mon, 01 Sep 2003 18:16:05 +0200
Subject: [pypy-dev] Actions have started (was:  funding and so)
In-Reply-To: <20030730131425.Z32350@prim.han.de>
References: <20030730131425.Z32350@prim.han.de>
Message-ID: <3F5370C5.5060602@tismer.com>

holger krekel wrote:

...

> Some people understood that i am just plainly against getting funding 
> which is not true.  In fact i want to help to make funding possible but 
> i am not going to lead much efforts there.  Laura and Jacob has been 
> active in preparing the start of a proposal but i don't know if they 
> find enough time to move it forward.  I sure hope so.  Also Anders
> has been investing time and contacting people which is great.  Still we 
> should make ours minds up if and how we want to go for the 
> October 15th deadline.

We are working on that proposal right now and will get
it done in time. I've submitted a call for input
of ideas on the pypy-funding list.
Everybody who's interested to help and discuss is
invited to subscribe to pypy-funding at codespeak.net .
Especially, we need opinions about possible goals for PyPy
which make sense for the EU -- see my post topypy-funding
from today.

see you there -- chris

-- 
Christian Tismer             :^)   <mailto:tismer at tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  pager +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/



From jacob at strakt.com  Mon Sep  1 21:14:19 2003
From: jacob at strakt.com (Jacob =?iso-8859-1?q?Hall=E9n?=)
Date: Mon, 1 Sep 2003 21:14:19 +0200
Subject: [pypy-dev] Request for viewcvs
Message-ID: <200309011914.h81JEtbZ004406@theraft.strakt.com>

I found this request in the IRC. Seems reasonable to do this.

Jacob

20:12:18] <azazel> hi guys!
[20:16:34] <azazel> you are doing a real intersting piece of work, but please 
install viewcvs from its cvs repository, it supports Subversion repositories 
as well, take a look here 
[20:16:46] <azazel> http://nautilus.homeip.net/cgi-bin/viewcvs.cgi/
[20:17:23] <azazel> if you want to take a look at viewcvs subversion support!
[

From hpk at trillke.net  Mon Sep  1 23:23:00 2003
From: hpk at trillke.net (holger krekel)
Date: Mon, 1 Sep 2003 23:23:00 +0200
Subject: [pypy-dev] Request for viewcvs
In-Reply-To: <200309011914.h81JEtbZ004406@theraft.strakt.com>;
	from jacob@strakt.com on Mon, Sep 01, 2003 at 09:14:19PM +0200
References: <200309011914.h81JEtbZ004406@theraft.strakt.com>
Message-ID: <20030901232300.W32350@prim.han.de>

Hello Jacob,

[Jacob Hall?n Mon, Sep 01, 2003 at 09:14:19PM +0200]
> I found this request in the IRC. Seems reasonable to do this.
> 
> Jacob
> 
> 20:12:18] <azazel> hi guys!
> [20:16:34] <azazel> you are doing a real intersting piece of work, but please 
> install viewcvs from its cvs repository, it supports Subversion repositories 
> as well, take a look here 
> [20:16:46] <azazel> http://nautilus.homeip.net/cgi-bin/viewcvs.cgi/
> [20:17:23] <azazel> if you want to take a look at viewcvs subversion support!

some svn-viewer makes sense.  I am not sure i get to it soonish and
setting up an issue for it wouldn't hurt so that i don't forget it.

Actually i don't particularly like the interface of viewcvs but
that's another issue.  Jum and me have written some svn-access
libraries which should turn this into a pretty easy task. But
until then, installing viewcvs for viewing svn-repos makes sense.

btw, will you be attending the Berlin-Sprint?  

cheers,

    holger

From arigo at tunes.org  Tue Sep  2 15:54:56 2003
From: arigo at tunes.org (Armin Rigo)
Date: Tue, 2 Sep 2003 14:54:56 +0100
Subject: [pypy-dev] "Unwrap" Concidered Harmful
In-Reply-To: <193BFA2F.746B29ED.9ADE5C6A@netscape.net>
References: <193BFA2F.746B29ED.9ADE5C6A@netscape.net>
Message-ID: <20030902135456.GA11770@vicky.ecs.soton.ac.uk>

Hello Rocco,

On Sat, Aug 23, 2003 at 09:00:58PM -0400, Rocco Moretti wrote:
> Maybe I overstate the subject line, but there are serious limitations
> on the .unwrap() concept.

Yes, I agree with your analysis. I actually think that we are using unwrap() 
for different purposes at different places, and we should clearly mark these 
differences...

> all information about value. So there is no way we can say that
> s.unwrap(s.wrap(x)) == x.

I am now coming closer to the idea that wrap() should be redefined as well, 
because it is used not only on simple types but also on "internal interpreter 
structures", like code objects plus others more internals than that.

I now tend to think about wrap(x) as 'I want you, object space, to build 
something that acts as reference to my object x'. The unwrap() operation would 
then only be defined on objects that have been wrapped, and the object space 
should somehow guarantee that "unwrap(wrap(x)) is x".

Now this is not exactly the way wrap() and unwrap() are used in the current
code base. We also use wrap(x) when x is a simple immutable object, to build 
simple blackboxed objects (ints, strs...). And we also use unwrap(x) when we 
expect 'x' to be such an object, to get a plain int or str out of a blackbox.

Moreover, in the light of the current refactoring on branch 'builtinrefactor',
I think it would make sense to say that the object spaces (all of them,
whatever their semantics are) have to support a particular kind of
interpreter-controlled semantics on complex objects created by wrap(). In
other words, the interpreter would control what occurs when object space
operations involve such an object. So, for example, space.getattr() on a
wrapped code object would force the object space to call back the interpreter,
which would perform the necessary attribute-reading operation. (I think it is
reasonable to say that the object space should not be responsible for that
because it is essentially an interpreter-controlled object. The same would go
for, say, wrapped frames, tracebacks, whatever, possibly module objects too. 
There is no need to code logic for these in each and every object space, as we
are now doing in std and ann).

In summary we'd have two usages for wrap()/unwrap() -- let's try to figure out
some better names:

 * wrap(x) -> create a blackboxed reference for the interpreter object x
 * unwrap(w_x,ExpectedType) -> inverse of the previous operation
 * newint(i), newstr(s)... -> create simple object space objects
 * unwrap(w_x,int), unwrap(w_x,str) -> inverse of the previous operation

In this idea, unwrap() would be a bit like PyArg_ParseTuple in that it would
check what type of interpreter-level object it is supposed to return, possibly
raising a TypeError.

Still this doesn't directly address the whole issue of application-level
helpers. I tend to think now that we should be very careful before using
app-level helpers. All the troubles we had with the argument-parsing one were
solved when Holger finally convinced me to just try to write the damn thing
directly at interpreter-level, which I did without too much trouble after all. 
I'd currently advice to reserve app-level helpers to tasks that could really
be thought of as part of a user program. For example I think it is ok to keep
at app-level the logic that decodes what a 'raise' statement really meant,
e.g. 'raise X, Y' -> 'raise X(Y)', because this is the kind of thing that a
user could write in a small general-purpose function of his program if it
wasn't built in the interpreter.


A bient?t,

Armin.


From mwh at python.net  Tue Sep  2 16:10:06 2003
From: mwh at python.net (Michael Hudson)
Date: Tue, 02 Sep 2003 15:10:06 +0100
Subject: [pypy-dev] Re: "Unwrap" Concidered Harmful
References: <193BFA2F.746B29ED.9ADE5C6A@netscape.net>
	<20030902135456.GA11770@vicky.ecs.soton.ac.uk>
Message-ID: <2mn0dns2m9.fsf@starship.python.net>

Armin Rigo <arigo at tunes.org> writes:

> Hello Rocco,

Oops, I've been meaning to reply to this too...

> On Sat, Aug 23, 2003 at 09:00:58PM -0400, Rocco Moretti wrote:
>> Maybe I overstate the subject line, but there are serious limitations
>> on the .unwrap() concept.
>
> Yes, I agree with your analysis. I actually think that we are using unwrap() 
> for different purposes at different places, and we should clearly mark these 
> differences...

Yup.

>> all information about value. So there is no way we can say that
>> s.unwrap(s.wrap(x)) == x.
>
> I am now coming closer to the idea that wrap() should be redefined
> as well, because it is used not only on simple types but also on
> "internal interpreter structures", like code objects plus others
> more internals than that.

[snippety]

> In summary we'd have two usages for wrap()/unwrap() -- let's try to
> figure out some better names:
>
>  * wrap(x) -> create a blackboxed reference for the interpreter object x
>  * unwrap(w_x,ExpectedType) -> inverse of the previous operation
>  * newint(i), newstr(s)... -> create simple object space objects

(note this is like a bit like Py_BuildValue in the C API)

I'd *much* rather write space.build(1) than space.newint(1).

This would be a convenience method that takes a fixed set of types
(maybe just integers and strings...).

>  * unwrap(w_x,int), unwrap(w_x,str) -> inverse of the previous operation

unbuild?

If we have different calls to unwrap doing radically different things,
I think they should have different names...

> In this idea, unwrap() would be a bit like PyArg_ParseTuple in that
> it would check what type of interpreter-level object it is supposed
> to return, possibly raising a TypeError.
>
> Still this doesn't directly address the whole issue of
> application-level helpers. I tend to think now that we should be
> very careful before using app-level helpers.

It seems that way.

Cheers,
mwh

-- 
  Good? Bad? Strap him into the IETF-approved witch-dunking
  apparatus immediately!                        -- NTK now, 21/07/2000


From arigo at tunes.org  Wed Sep  3 15:50:29 2003
From: arigo at tunes.org (Armin Rigo)
Date: Wed, 3 Sep 2003 14:50:29 +0100
Subject: [pypy-dev] Re: "Unwrap" Concidered Harmful
In-Reply-To: <2mn0dns2m9.fsf@starship.python.net>
References: <193BFA2F.746B29ED.9ADE5C6A@netscape.net>
	<20030902135456.GA11770@vicky.ecs.soton.ac.uk>
	<2mn0dns2m9.fsf@starship.python.net>
Message-ID: <20030903135029.GA27744@vicky.ecs.soton.ac.uk>

Hello Michael,

On Tue, Sep 02, 2003 at 03:10:06PM +0100, Michael Hudson wrote:
> >  * wrap(x) -> create a blackboxed reference for the interpreter object x
> >  * unwrap(w_x,ExpectedType) -> inverse of the previous operation
> >  * newint(i), newstr(s)... -> create simple object space objects
> 
> (note this is like a bit like Py_BuildValue in the C API)
> 
> I'd *much* rather write space.build(1) than space.newint(1).

Ok. Maybe the whole issue should be sorted out in the general context of how 
to declare "gateways" between interpreter- and app-level. For example, given 
an interpreter-level class

class X:
  def __init__(self, frame):
    self.w_stuff = space.newdict([])
    self.n = 5
    self.frame = frame
  def dosomething(self, w_x, i):
    return self.n

how could we cleanly specify that we want 'n' and 'dosomething' be
app-level-visible as, respectively, an integer object and a method taking two
arguments the second of which must be an integer? In other words the whole
Py_BuildValue / Py_BuildTuple / PyArg_ParseTuple business, plus
structmember.c.

The point of giving the same name to wrap/unwrap and to the proposed 
build/unbuild is to have a uniform way of exposing attributes of different 
kinds, as shown on the above example:

 * w_stuff is a wrapped app-level object all along, no conversion required.
 * n is an integer, must use build/unbuild.
 * frame is another interp-level internal object, must use wrap/unwrap.

This is the reason why we might want a single function pair for all three 
cases. Alternatively we could use 'type descriptor' objects:

 * t_wrapped.wrap(space, w_stuff)   -> w_stuff
 * t_int.wrap(space, 5)             -> w_5
 * t_interplevel.wrap(space, frame) -> wrapped frame

 * t_wrapped.unwrap(space, w_stuff)     -> w_stuff
 * t_int.unwrap(space, w_5)             -> 5
 * t_interplevel.unwrap(space, w_frame) -> frame

The advantage is that we can declare function signatures explicitely, say
appmethod([t_wrapped, t_int], t_int) for the above dosomething() method, and
it is clear that the wrap/unwrap methods of the specified type descriptors
will be used for the conversion at appropriate times. (Type descriptors are
also a good place to store extra information and functionality if needed, e.g. 
the corresponding C type. The idea comes from Thomas Heller's ctypes module.)

The t_xxx objects above could also be classes instead, with wrap/unwrap class 
or static methods. This would let us use the interpreter-level class hierarchy 
directly instead of the t_interplevel trick:

  Frame.unwrap(space, w_frame) -> frame

This is more explicit because it can check that we actually have, not just an 
interpreter-level object, but a Frame instance.

We could also put the space argument at the end, so that we can write

  Frame.wrap(frame, space) -> w_frame

or

  frame.wrap(space) -> w_frame

and thus wrap (but not unwrap) could actually be a regular method.

As a final note, there would then be a lot of different wrap/unwrap methods
defined all around which handle different types -- certainly a good thing for 
flexibility and also for the annotation object space, which would not get 
confused by the fact that a single build() or unwrap() could accept a whole 
lot of different types.


A bient?t,

Armin.


From lac at strakt.com  Thu Sep  4 15:07:53 2003
From: lac at strakt.com (Laura Creighton)
Date: Thu, 4 Sep 2003 15:07:53 +0200
Subject: [pypy-dev] 
 needed 18 month plan of what we would do in developing PyPy if we
 got funding.
Message-ID: <200309041307.h84D7rQb029849@ratthing-b246.strakt.com>


you know:

if we were writing what we did we would say:

+ add support for builtins
+ make translation object space
+ produce first running code
+ speed up code generation

Ideally, what would come out is 18 things to do, one per month, because
that is really easy for the non-technical reviewers to understand.  This
is an impossible goal, since everything depends on everything else, but
it would help if things looked a little more linear than they actually
can be in practice.  Can people start brainstorming a list, please.

Thank you,
Laura

From lac at strakt.com  Thu Sep  4 16:32:22 2003
From: lac at strakt.com (Laura Creighton)
Date: Thu, 4 Sep 2003 16:32:22 +0200
Subject: [pypy-dev] pypy home page says 'no date set for the Berlin Sprint'
Message-ID: <200309041432.h84EWMWQ030259@ratthing-b246.strakt.com>

please update it when you get a chance.

Thanks very much,
Laura

From pedronis at bluewin.ch  Thu Sep  4 16:52:49 2003
From: pedronis at bluewin.ch (Samuele Pedroni)
Date: Thu, 04 Sep 2003 16:52:49 +0200
Subject: [pypy-dev]  needed 18 month plan of what we would do in
  developing PyPy if we got funding.
In-Reply-To: <200309041307.h84D7rQb029849@ratthing-b246.strakt.com>
Message-ID: <5.2.1.1.0.20030904161323.026e5a80@pop.bluewin.ch>

At 15:07 04.09.2003 +0200, Laura Creighton wrote:

>you know:
>
>if we were writing what we did we would say:
>
>+ add support for builtins
>+ make translation object space
>+ produce first running code
>+ speed up code generation
>
>Ideally, what would come out is 18 things to do, one per month, because
>that is really easy for the non-technical reviewers to understand.  This
>is an impossible goal, since everything depends on everything else, but
>it would help if things looked a little more linear than they actually
>can be in practice.  Can people start brainstorming a list, please.

this is more a list of "milestones" and todos and research topics, that 
exactly what you asked. I hope this is at least a starting point. 
Disclaimer: my POV, some bits can be controversial etc..., I have not 
looked at Psyco recently, some of my comments can be off-base

1) manage to translate the body of some of our multimethods to C or pyrex ...
see whether the annotation space approach works and is maintainable, or  an 
AST based
approach is more suitable or some mixed approach - AST based for 
control-flow level, annotation space for basic blocks/exprs

2) how to attack/translate the whole thing: CPython hosted PyPy
- make the design of the app-level/interp-level interface converge
- parts of the produced intepreter vs. our PyPy Python codebase correspondence,
   how global piece-wise translation should work
- object layout (reuse CPython's one or roll our own)
- write needed/missing parts for runtime: especially code that implement 
multimethods

Note: threading/memory management from/like CPython

3) high-performance standalone PyPy-Python:
- decide object layout/memory management strategy (ref counting/Boehm/our 
own...)
- decide stragety wrt threading (messy issue)
- what part of CPython we lose: strategy for reuse vs. rewrite directly in 
C vs. rewrite in Python plus translation vs. rewrite as app-level Python
- write runtime parts we need
- adjust translation of the rest

- integration of Psyco technology (rewrite to translated Python plus 
automatic code generation of all what can be deduced from our PyPy codebase 
or ???)
- integration of Stackless

- remove some constraints imposed by Psyco on dynamic behavior
- need to improve profiling approach to find hotspots for Psyco ?  (I don't 
know
how Psyco is doing about this now, Armin?)

- optimizations for lookup/globals while
supporting dynamic changes (things AFAIK Psyco does not address completely 
? is that
correct)

- extension interface ?
- what to do for legacy CPython extensions

- further  improvements ...

4) small devices PyPy-Python ? (do we need a partner in that area?)

( 5) Java/.NET-Python from PyPy
   ...
)

3-4(-5) can proceed in parallel 


From arigo at tunes.org  Sun Sep  7 20:29:05 2003
From: arigo at tunes.org (Armin Rigo)
Date: Sun, 7 Sep 2003 19:29:05 +0100
Subject: [pypy-dev]  needed 18 month plan of what we would do in
	developing PyPy if we got funding.
In-Reply-To: <5.2.1.1.0.20030904161323.026e5a80@pop.bluewin.ch>
References: <200309041307.h84D7rQb029849@ratthing-b246.strakt.com>
	<5.2.1.1.0.20030904161323.026e5a80@pop.bluewin.ch>
Message-ID: <20030907182905.GD17209@vicky.ecs.soton.ac.uk>

Hello again,

Sorry, forgot to attach the document.


Armin
-------------- next part --------------
=========================
Draft of a PyPy work plan
=========================


1. The PyPy Interpreter
-----------------------

The goal is to make a complete Python interpreter that runs under any
existing Python implementation.

 a) develop and complete the PyPy interpreter itself, as a regular
Python program, until it contains all the parts of CPython that we don't
want to move to (b). Further investigate the unorthodox multimethod
concepts that the standard object space is based on, and how to hook in
the bytecode compiler.

 b) translate all other parts of CPython into regular Python libraries.
These ones should also work without PyPy, being just plain-Python
replacements for existing CPython functionality. This includes the
bytecode compiler.


2. Translation of RPython
-------------------------

The goal is to be able to translate arbitrary RPython source code (e.g. 
the one produced in 1a) into low-level code (C, Pyrex, Java, others). 
This includes making a stand-alone, not-PyPy-related tool for general
optimization of arbitrary but suitably restricted Python application or
parts thereof.

 a) analyse code to produce the relevant typing information. Investigate
if we can use the annotation object space only or if additional
AST-based control flow analysis is needed.

 b) produce low-level code out of the data gathered in (a). Again
investigate how this is best done (AST-guided translation or
reverse-engeneering of the low-level control flow gathered by the
annotation object space). Compare different low-level environment that
we could target (C, Pyrex, others?).


3. Bootstrapping PyPy
---------------------

The goal is to put (1) and (2) together.

 a) investigate the particular problems specific to the global
translation of PyPy, as opposed to general to any RPython program. 
According to the requirements and insights of (2) we will probably have
to redesign specific parts of PyPy, e.g. make the various
app-level/interp-level interface designs converge.

 b) build the low-level-specific run-time components of PyPy, most
notably the object layout, the memory management, possibly threading
support, and multimethod dispatch. Here, if we target C code, important
parts can be directly re-used from CPython.


4. High-performance PyPy-Python
-------------------------------

The goal is to optimize (3) in possibly various ways, building on its
flexibility to go beyond CPython.

 a) develop several object implementations for the same types, as
explicitely allowed by the standard object space, and develop heuristics
to switch between implementations during execution.

 b) identify which optimizations would benefit from support from the
translator (2). These are the optimizations not easily available to
CPython because they would require large-scale code rewrites.

 c) for each issue, work on several solutions when no one is obviously
better than the other ones. The meta-programming underlying (b) --
namely the work on the translator instead of on the resulting code -- is
what gives us the possibility of actually implementing several very
different schemes.

 d) integrate existing technology that traditionally depended on closely
following CPython's code base, notably Psyco and Stackless. Rewrite each
one as a meta-component that hooks into the translator (2) plus a
dedicated run-time component (3b). Further develop these technologies
based on the results gathered in (c), e.g. identify when these
technologies would guide specific choices among the solutions developed 
in (a) and (b).


Annex to (a)
~~~~~~~~~~~~

Some major uses for several implementations of the built-in types:

 * dictionaries as hash-table vs. plain (key, value) lists vs. b-trees, 
or with string-only or integer-only keys. Dictionaries with specific 
support for "on-change" callbacks (useful for Psyco).

 * strings as plain immutable memory buffers vs. immutable but more 
complex data structures (see functional languages) vs. internally 
mutable data structures (e.g. Psyco's concatenated strings)

 * ints as machine words vs. two machine words vs. internal longs vs. 
external bignum library (investigate if completely unifying ints and
longs is possible in the Python language at this stage).

 * etc. (lists as range() or chained lists, ...)

The above are mostly independent from any particular low-level run-time 
environment.


Annex to (b)
~~~~~~~~~~~~

Here are some of the main issues and tricks. Note that compatibility
with legacy C extensions can be acheived by choosing, for each of the
following issues, the same one as CPython did.

 * object layout and memory management strategy or strategies, e.g.
reference counting vs. Boehm garbage collection vs. our own. Includes
speed vs. data size trade-offs.

 * code size vs. speed trade-offs (e.g. whether the final interpreter
should still include compact precompiled bytecode or be completely
translated into C).

 * the complex issue of threading (global interpreter lock vs.
alternatives).

 * multimethod dispatching

 * pointer tagging, e.g. encoding an integer object as a pointer with a 
special value instead of a real pointer to a data structure representing 
the integer.

The above are mostly specific to a particular low-level run-time.


5. Low-level targets, tools and releases
----------------------------------------

The goal is to identify, among those low-level targets that are in
widespread use (e.g. workstation usage vs. web server vs.
high-performance computing vs. memory-starved hand-held device; C/Unix
vs. Java vs. .NET environment), which ones would benefit most from a
high-performance Python interpreter. For each of these, focus will be
given to:

 a) develop the translation process, run-time and those optimizations
that depend on low-level details.

 b) design interfaces for extension modules. Some can be very general
(e.g. a pure Python one that should allow generic third-party code to
hook into the PyPy interpreter source code without worrying about the
translation process). Others depend on the low-level environment and on
the choices made for the issues of (4).

 c) combine different solutions for the different issues discussed in
(4). Gather statistics with real-work Python application. Compare the
results. This is where the flexibility of the whole project is vital.
Typically, very different trade-offs need to be made on different
environments.

 d) most importantly, develop tools to easily allow third-parties to
repeat (c) in their own domain and build their own tailored versions of
PyPy.

 e) release a few official versions pre-tailored for various common
environments. Develop in particular a version whose goal is to simulate
the existing CPython interpreter to support legacy extension modules. 
Investigate if the PyPy core can make internal choices that are very
different from CPython's without sacrifying legacy extension modules
compatibility.


6. Infrastructure
-----------------

The goal is to address the development and maintenance issues.

 a) PyPy's own development needs an infrastructure that must
continuously be kept up-to-date and further developed.

 b) write tests. All parts of PyPy should be extensively covered by
stress tests. Investigate the use of test-coverage analysers.

 c) investigate means of keeping PyPy in sync with the future
developments of CPython, e.g. ways to relate pieces of PyPy source and
pieces of CPython source. Look for existing solutions.


7. Extension of PyPy
--------------------

The goal is to add functionalities in PyPy that are not present in
existing Python implementations. This is an open goal. We only list a
few promizing directions:

 a) build alternate object spaces provides features that are essentially
language-transparent, e.g. distributed computing (via a network proxy
object space), compatibility layers (e.g. a Python-1.5.2-compliant
object space), persistance (via a persistant object space).

 b) build language features that rely on translator support (2), i.e. 
which can be turned on or off during the production of individual 
versions of PyPy, e.g. Stackless and continuations.

 c) work on the interaction between the compiler and the main loop to 
allow custom opcodes to be defined, generated by the compiler, and 
interpreted by the main loop, thus allowing syntactic extension of the 
language by user code

 d) conversely, develop interfaces to use object spaces without the main
loop to provide Python-like object semantics to other programming
languages, using their own syntax and execution environment, e.g. Java.

From arigo at tunes.org  Sun Sep  7 19:08:42 2003
From: arigo at tunes.org (Armin Rigo)
Date: Sun, 7 Sep 2003 18:08:42 +0100
Subject: [pypy-dev]  needed 18 month plan of what we would do in
	developing PyPy if we got funding.
In-Reply-To: <5.2.1.1.0.20030904161323.026e5a80@pop.bluewin.ch>
References: <200309041307.h84D7rQb029849@ratthing-b246.strakt.com>
	<5.2.1.1.0.20030904161323.026e5a80@pop.bluewin.ch>
Message-ID: <20030907170842.GA17209@vicky.ecs.soton.ac.uk>

Hello Samuele,

Let me try to fill some bits in your list in an attempt to make it more nicely
linear...

The resulting document is in the svn repository at doc/EU_funding/plan.txt. 
I'm attaching a copy to this e-mail. Wwow, I somehow managed to make it look
very nicely structured with clear categories :-)

Everything in there is wide open for discussion of course! That's also quite
my own point of view (as I guess you'll see if only by the fact that
"infrastructure" is the last of the 6 categories ;-) but it should maybe be
put much earlier, if the categories are supposed to be ordered by starting
date).


A few Psyco-specific questions from Samuele...

> - integration of Psyco technology (rewrite to translated Python plus 
> automatic code generation of all what can be deduced from our PyPy codebase 
> or ???)

For both Psyco and Stackless I expect that we will divide them in two parts, 
the one that we tediously maintained from the Python source, and the one that 
is really unique to the project. For the first part we will try as much as 
possible to replace it by hooking into the translator. For the second part, it 
depends, I guess it can also be divided into fairly generic stuff that can be 
written in Python (interp- or even app-level) and stuff that really depends on 
the low-level.

More specifically (this only goes as far as I understand Stackless):


+ in PyPy's ---------- Psyco ------------------- Stackless -------

translator     translate the PyPy           translate regular functions
               interpreter into             into Continuation-Passing
               an abstract interpreter      Style (CPS) (Stackless 1.0)

pure Python    binding logic,               high-level tasklet aspects,
               profiler heuristics          scheduling
               
run-time       machine code emission,       C stack switching
               run-time support for         (Stackless 2.0)
               the abstact interpreter

In both case it might be possible to reduce the "run-time" part drastically,
thus making the tool essentially independent from a low-level run-time -- it
would work in Java too :-)  For Psyco this would be done using a generic 
virtual machine as the one I just finished (yup, Psyco works on PowerPC now, 
and probably on any 32-bit machine :-)  -- it would be done by writing the 
virtual machine's implementation itself in interpreter-level Python. For 
Stackless you'd have full continuations using the translator-to-CPS, which 
means essentially that you don't need C stack switching if you don't want it 
(you'd get a full Stackless 1.0).

Oh yes, BTW, thread pickling is another example of what the translator can
help make possible :-) It would be possible because to pickle a thread state
cleanly you really need all information you can get about the data structures
and where they are on the stack. A translator could emit special C code that
keeps track of that in a way that would be extremely tedious to do manually.

> - need to improve profiling approach to find hotspots for Psyco ?  (I don't 
> know how Psyco is doing about this now, Armin?)

The profiler is already reasonable, I think, but more testing would be cool.

> - optimizations for lookup/globals while
> supporting dynamic changes (things AFAIK Psyco does not address completely 
> ? is that correct)

Yes. Global lookups are cached but still work correctly in presence of dynamic
changes, but what doesn't at the moment is lookups of methods in new-style
classes.


A bient?t,

Armin.


From pedronis at bluewin.ch  Sun Sep  7 21:36:37 2003
From: pedronis at bluewin.ch (Samuele Pedroni)
Date: Sun, 07 Sep 2003 21:36:37 +0200
Subject: [pypy-dev]  needed 18 month plan of what we would do in
  developing PyPy if we got funding.
In-Reply-To: <20030907170842.GA17209@vicky.ecs.soton.ac.uk>
References: <5.2.1.1.0.20030904161323.026e5a80@pop.bluewin.ch>
 <200309041307.h84D7rQb029849@ratthing-b246.strakt.com>
 <5.2.1.1.0.20030904161323.026e5a80@pop.bluewin.ch>
Message-ID: <5.2.1.1.0.20030907212655.026b55b0@pop.bluewin.ch>

At 18:08 07.09.2003 +0100, Armin Rigo wrote:

>In both case it might be possible to reduce the "run-time" part drastically,
>thus making the tool essentially independent from a low-level run-time -- it
>would work in Java too :-)

it would be an interesting experiment, although the JVM has its own quirks
in term of performance and limitations.

OTOH a faster "Jython" with this or other techniques is achievable,
but just a less costly to mantain codbase would be already good.


>Oh yes, BTW, thread pickling is another example of what the translator can
>help make possible :-) It would be possible because to pickle a thread state
>cleanly you really need all information you can get about the data structures
>and where they are on the stack. A translator could emit special C code that
>keeps track of that in a way that would be extremely tedious to do manually.

yes, a whole range of things that are tedious/hard to do manually, becomes 
possible and
opens up.


> > - optimizations for lookup/globals while
> > supporting dynamic changes (things AFAIK Psyco does not address completely
> > ? is that correct)
>
>Yes. Global lookups are cached but still work correctly in presence of dynamic
>changes, but what doesn't at the moment is lookups of methods in new-style
>classes.

yes lookup was distinguished from globals, and was referring to 
attribute/method
lookup, is an area where I have some ideas and where we can borrow from 
smalltalk/self...
research.

Samuele.



From pedronis at bluewin.ch  Sun Sep  7 23:12:28 2003
From: pedronis at bluewin.ch (Samuele Pedroni)
Date: Sun, 07 Sep 2003 23:12:28 +0200
Subject: [pypy-dev]  needed 18 month plan of what we would do in
  developing PyPy if we got funding.
In-Reply-To: <20030907182905.GD17209@vicky.ecs.soton.ac.uk>
References: <5.2.1.1.0.20030904161323.026e5a80@pop.bluewin.ch>
 <200309041307.h84D7rQb029849@ratthing-b246.strakt.com>
 <5.2.1.1.0.20030904161323.026e5a80@pop.bluewin.ch>
Message-ID: <5.2.1.1.0.20030907230643.026b5240@pop.bluewin.ch>


about

"""
2. Translation of RPython
-------------------------

The goal is to be able to translate arbitrary RPython source code (e.g.
the one produced in 1a) into low-level code (C, Pyrex, Java, others).
This includes making a stand-alone, not-PyPy-related tool for general
optimization of arbitrary but suitably restricted Python application or
parts thereof.

  a) analyse code to produce the relevant typing information. Investigate
if we can use the annotation object space only or if additional
AST-based control flow analysis is needed.
"""

part of this would be also to define what precisely RPython is, right?

I imagine that as long as what we translate is PyPy code , the definition, 
whatever
our tool accept, works.

OTOH if we want as result to target arbitrary RPython source code, we need 
a "formal"
definition of RPython and of the translation process.


From hpk at trillke.net  Mon Sep  8 00:50:00 2003
From: hpk at trillke.net (holger krekel)
Date: Mon, 8 Sep 2003 00:50:00 +0200
Subject: [pypy-dev] PyPy "Berlin" sprint (29th Sep - 4th Oct 2003)
Message-ID: <20030908005000.D22698@prim.han.de>

hello python developers, 

the fourth PyPy coding sprint will take place in Berlin (Germany) from

    (morning of) 29th of September to (evening of) 4th of October 2003. 

All Python developers are welcome to join the sprint. 
See below for participation details. 


What is PyPy?
-------------

PyPy is a remimplementation of Python in the Python language itself. 
Simplicity and flexibilty are the foremost goals.  With PyPy we want it
to be easy to extend the language or generate a more minimal language 
implementation.

One key technique to making this feasible is to specialize our abstract/general 
PyPy-Python implementation into native code (e.g. C-code).  Eventually the
concepts of PSYCO and Stackless are to be integrated into PyPy. 

There is, of course, much more to it so feel free to consult some
documents especially our Oscon-2003 paper on this page:

    http://codespeak.net/pypy/index.cgi?doc

We know that many areas and issues are still somewhat vague
although PyPy can already run many python programs [*]. Fortunately, Python 
is pretty tolerant about expressing ideas in vague ways.  After all, 
hacking at PyPy aims (and already proved!) to preserve the fun of 
hacking in Python. 


History, status and Berlin goals
--------------------------------

The project started in Jan/Feb 2003 with an initiative from Christian 
Tismer, Holger Krekel and Armin Rigo. During the course many more
developers joined our sprints and there are now between 5 and 15 people
involved in developing PyPy.  

The three sprints resulted in a fully working interpreter, some development 
infrastructure and the "standard object space" which is an abstraction for 
operations on standard (CPython) objects. 

There is no public release yet, you more or less have to install
a subversion-client and checkout the trunk.  Here is our 'howtosvn' 
which contains up-to-date clients for multiple platforms along
with some instructions how to get started:

    http://codespeak.net/pypy/index.cgi?doc/devel/howtosvn.html

Main goals for the Berlin sprint are 

- integration/enhancement of a Python parser and the compiler package 

- generating native code from our Python interpreter/types implementations

- enhancing/correcting/completing what we have (various tasks)


How to participate in Berlin (29th of Sept. to 4th of October)
--------------------------------------------------------------

If you are interested to participate please subscribe at our 
sprint organization list 

    http://codespeak.net/mailman/pypy-sprint

and please list yourself at 

    http://codespeak.net/moin/pypy/moin.cgi/SprintAttendants

so that we can organize accomodation and arrange details. Costs
should be pretty low (except from travelling costs, of course).
Currently nine people are scheduled to come.  

Btw, the sprint starts early on the 29th of september so it's best to
arrive on sunday, 28th. Also we probably want to have one day
off during the sprint week to do some sight (or cafe+pub) seeing. 
Actually it's much better to participate the whole week to better
get into it. 

cheers,

    holger 
    

[*] and currently only 20-40 thousand times slower than CPython! which clearly 
    shows that we are successfully follow the "premature optimization is the root 
    of all evil" maxime. 

From lac at strakt.com  Wed Sep 10 17:47:41 2003
From: lac at strakt.com (Laura Creighton)
Date: Wed, 10 Sep 2003 17:47:41 +0200
Subject: [pypy-dev] I cannot connect to codespeak.  
Message-ID: <200309101547.h8AFlfLg024131@ratthing-b246.strakt.com>

I get connection refused.

From hpk at trillke.net  Wed Sep 10 17:50:12 2003
From: hpk at trillke.net (holger krekel)
Date: Wed, 10 Sep 2003 17:50:12 +0200
Subject: [pypy-dev] Re: I cannot connect to codespeak.
In-Reply-To: <200309101547.h8AFlfLg024131@ratthing-b246.strakt.com>;
	from lac@strakt.com on Wed, Sep 10, 2003 at 05:47:41PM +0200
References: <200309101547.h8AFlfLg024131@ratthing-b246.strakt.com>
Message-ID: <20030910175012.V22698@prim.han.de>

[Laura Creighton Wed, Sep 10, 2003 at 05:47:41PM +0200]
> I get connection refused.

hello laura, 

you really are hitting the second we are changing something on
codespeak and this (unexpectedly, unfortunately) affected the 
apache service. 

i guess this might take up to 30 minutes to fix, would that be ok? 

    holger

From lac at strakt.com  Wed Sep 10 17:52:42 2003
From: lac at strakt.com (Laura Creighton)
Date: Wed, 10 Sep 2003 17:52:42 +0200
Subject: [pypy-dev] Re: I cannot connect to codespeak. 
In-Reply-To: Message from holger krekel <hpk@trillke.net> 
   of "Wed, 10 Sep 2003 17:50:12 +0200." <20030910175012.V22698@prim.han.de> 
References: <200309101547.h8AFlfLg024131@ratthing-b246.strakt.com>
	<20030910175012.V22698@prim.han.de> 
Message-ID: <200309101552.h8AFqgq3024179@ratthing-b246.strakt.com>

In a message of Wed, 10 Sep 2003 17:50:12 +0200, holger krekel writes:
>[Laura Creighton Wed, Sep 10, 2003 at 05:47:41PM +0200]
>> I get connection refused.
>
>hello laura, 
>
>you really are hitting the second we are changing something on
>codespeak and this (unexpectedly, unfortunately) affected the 
>apache service. 
>
>i guess this might take up to 30 minutes to fix, would that be ok? 
>
>    holger

No problem.  Just wanted to let you know in case it was unexpectedly down.

Laura


From hpk at trillke.net  Wed Sep 10 18:01:17 2003
From: hpk at trillke.net (holger krekel)
Date: Wed, 10 Sep 2003 18:01:17 +0200
Subject: [pypy-dev] Re: I cannot connect to codespeak.
In-Reply-To: <200309101552.h8AFqgq3024179@ratthing-b246.strakt.com>;
	from lac@strakt.com on Wed, Sep 10, 2003 at 05:52:42PM +0200
References: <200309101547.h8AFlfLg024131@ratthing-b246.strakt.com>
	<20030910175012.V22698@prim.han.de> <hpk@trillke.net>
	<200309101552.h8AFqgq3024179@ratthing-b246.strakt.com>
Message-ID: <20030910180117.W22698@prim.han.de>

[Laura Creighton Wed, Sep 10, 2003 at 05:52:42PM +0200]
> In a message of Wed, 10 Sep 2003 17:50:12 +0200, holger krekel writes:
> >[Laura Creighton Wed, Sep 10, 2003 at 05:47:41PM +0200]
> >> I get connection refused.
> >
> >hello laura, 
> >
> >you really are hitting the second we are changing something on
> >codespeak and this (unexpectedly, unfortunately) affected the 
> >apache service. 
> >
> >i guess this might take up to 30 minutes to fix, would that be ok? 
> >
> >    holger
> 
> No problem.  Just wanted to let you know in case it was unexpectedly down.

ok. apache is up again, subversion soon to follow (that is you cannot
work with svn for the moment). The good thing is that we got rid of some
hacks regarding bsd-db (v3<->v4) integration. 

cheers,

    holger

From hpk at trillke.net  Wed Sep 10 18:31:49 2003
From: hpk at trillke.net (holger krekel)
Date: Wed, 10 Sep 2003 18:31:49 +0200
Subject: [pypy-dev] Re: I cannot connect to codespeak.
In-Reply-To: <20030910180117.W22698@prim.han.de>;
	from hpk@trillke.net on Wed, Sep 10, 2003 at 06:01:17PM +0200
References: <200309101547.h8AFlfLg024131@ratthing-b246.strakt.com>
	<20030910175012.V22698@prim.han.de> <hpk@trillke.net>
	<200309101552.h8AFqgq3024179@ratthing-b246.strakt.com>
	<20030910180117.W22698@prim.han.de>
Message-ID: <20030910183149.X22698@prim.han.de>

[holger krekel Wed, Sep 10, 2003 at 06:01:17PM +0200]
> [Laura Creighton Wed, Sep 10, 2003 at 05:52:42PM +0200]
> > In a message of Wed, 10 Sep 2003 17:50:12 +0200, holger krekel writes:
> > >[Laura Creighton Wed, Sep 10, 2003 at 05:47:41PM +0200]
> > >> I get connection refused.
> > >
> > >hello laura, 
> > >
> > >you really are hitting the second we are changing something on
> > >codespeak and this (unexpectedly, unfortunately) affected the 
> > >apache service. 
> > >
> > >i guess this might take up to 30 minutes to fix, would that be ok? 
> > >
> > >    holger
> > 
> > No problem.  Just wanted to let you know in case it was unexpectedly down.
> 
> ok. apache is up again, subversion soon to follow (that is you cannot
> work with svn for the moment). The good thing is that we got rid of some
> hacks regarding bsd-db (v3<->v4) integration. 

now it's all working again.  If you experience any problems today please
1) recheck if it still doesn't work after 5 minutes 
2) sent a mail to root at codespeak.net

thanks,

    holger

From tismer at tismer.com  Wed Sep 10 19:14:20 2003
From: tismer at tismer.com (Christian Tismer)
Date: Wed, 10 Sep 2003 19:14:20 +0200
Subject: [pypy-dev] Tortoise SVN probs
Message-ID: <3F5F5BEC.3030005@tismer.com>

Hi Holger,

I can check out the complete pypy repository
(which tells me btw. about "external items" whatever
that may be), but when I try to do an update
with TortoisSVN, I get error messages like

"REPORT response handling failed to complete the editor drive"

I had that problem yesterday, already.
Is this a known problem, do I have to upgrade something?

ciao - chris

-- 
Christian Tismer             :^)   <mailto:tismer at tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  pager +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/



From jum at anubis.han.de  Wed Sep 10 19:49:25 2003
From: jum at anubis.han.de (Jens-Uwe Mager)
Date: Wed, 10 Sep 2003 19:49:25 +0200
Subject: [pypy-dev] Tortoise SVN probs
In-Reply-To: <3F5F5BEC.3030005@tismer.com>
References: <3F5F5BEC.3030005@tismer.com>
Message-ID: <20030910174924.GC3460@anubis>

On Wed, Sep 10, 2003 at 19:14 +0200, Christian Tismer wrote:

> I can check out the complete pypy repository
> (which tells me btw. about "external items" whatever
> that may be), but when I try to do an update
> with TortoisSVN, I get error messages like
> 
> "REPORT response handling failed to complete the editor drive"
> 
> I had that problem yesterday, already.
> Is this a known problem, do I have to upgrade something?

I just tried it from my windows box using TortoiseSVN 0.16 and I do not
get any such messages. I would recommend to upgrade to the latest
TortoiseSVN and see if the problem goes away.

-- 
Jens-Uwe Mager	<pgp-mailto:F476EBC2>

From tismer at tismer.com  Wed Sep 10 21:25:54 2003
From: tismer at tismer.com (Christian Tismer)
Date: Wed, 10 Sep 2003 21:25:54 +0200
Subject: [pypy-dev] Tortoise SVN probs
In-Reply-To: <20030910174924.GC3460@anubis>
References: <3F5F5BEC.3030005@tismer.com> <20030910174924.GC3460@anubis>
Message-ID: <3F5F7AC2.8030208@tismer.com>

Jens-Uwe Mager wrote:
...

> I just tried it from my windows box using TortoiseSVN 0.16 and I do not
> get any such messages. I would recommend to upgrade to the latest
> TortoiseSVN and see if the problem goes away.

Ok, I upgraded both svn and Tortoise and did
a fresh check-out. This even worked nicely
with Tortoise.
But now, I can do no update at all from
the explorer. I always get the complaint
". is not a working copy".
Are there some path settings or other magic involved
which I might have missed?
Well, I have Cygwin installed, and it is in the path.
Might that be a problem?

ciao & thx - chris
-- 
Christian Tismer             :^)   <mailto:tismer at tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  pager +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/



From tismer at tismer.com  Thu Sep 11 13:00:55 2003
From: tismer at tismer.com (Christian Tismer)
Date: Thu, 11 Sep 2003 13:00:55 +0200
Subject: [pypy-dev] Tortoise SVN probs
In-Reply-To: <3F5F7AC2.8030208@tismer.com>
References: <3F5F5BEC.3030005@tismer.com> <20030910174924.GC3460@anubis>
	<3F5F7AC2.8030208@tismer.com>
Message-ID: <3F6055E7.2040206@tismer.com>

Christian Tismer wrote:

> Jens-Uwe Mager wrote:
> ...
> 
>> I just tried it from my windows box using TortoiseSVN 0.16 and I do not
>> get any such messages. I would recommend to upgrade to the latest
>> TortoiseSVN and see if the problem goes away.

There seems to be one problem with upgrading Tortoise:
You really need to uninstall the stuff before you upgrade.
I wasn't sure if I did this in the first place.

First, I installed the new Tortoise.
When I had trouble with checking out, I upgraded
the new cmd line svn. No change in behavior.

Then, I uninstalled Tortoise completely, rebooted,
and,to be safe, removed every registry entry that
had Tortoise in it.
Then I installed Tortoise again.
It works fine now, and the context menu has changed,
completely. So that was probably the problem.

back to working on the proposal -- chris

-- 
Christian Tismer             :^)   <mailto:tismer at tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  pager +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/



From tismer at tismer.com  Thu Sep 11 13:32:19 2003
From: tismer at tismer.com (Christian Tismer)
Date: Thu, 11 Sep 2003 13:32:19 +0200
Subject: [pypy-dev] 
 Extended goals (was: needed 18 month plan of what we would do in
 developing PyPy if we got funding.)
In-Reply-To: <5.2.1.1.0.20030907230643.026b5240@pop.bluewin.ch>
References: <5.2.1.1.0.20030904161323.026e5a80@pop.bluewin.ch>
	<200309041307.h84D7rQb029849@ratthing-b246.strakt.com>
	<5.2.1.1.0.20030904161323.026e5a80@pop.bluewin.ch>
	<5.2.1.1.0.20030907230643.026b5240@pop.bluewin.ch>
Message-ID: <3F605D43.4030505@tismer.com>

Howdy,

In addition to the goals in plan.txt, I think we need
to do some more things.
So far, the draft tells about PyPy, how to develop
and maintain it, and how it can be extended.
This is just fine from the view of a PyPy developer.

The problem is to convince other parties to support
us. They probably don't have a vital interest in
PyPy for itself, but would like to see goals which
go towards realy usable applications, which can be
developed further.

Therefore, besides the theoretical things which are
great, I'm trying to add prototypical implementations
of application-like stuff, as there is for example:

- creating tiny, specialized code for embedded devices

- distributed processing and load-balancing

I think, these goals would be interesting for some
companies which might participate, and they could
support us by providing hardware and other support.

In extent to this, I just realized a few days ago, that
we have James Tauber in the PyPy list.
He is working on an operating system in Python, see
the cleese project at http://www.jtauber.com/cleese .

In a similar way as we intend to integrate Psyco and
Stackless in some way, I'd like to extend this to
Cleese. I'd like to add

- write a small, prototypical operating system in PyPy

to the list of application goals.

Question: Is there any objection to this, or can I continue?

Are there other ideas of applications which we would
like to include as "working horses" to create interest in PyPy?

cheers - chris

-- 
Christian Tismer             :^)   <mailto:tismer at tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  pager +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/



From bob at redivi.com  Thu Sep 11 13:55:17 2003
From: bob at redivi.com (Bob Ippolito)
Date: Thu, 11 Sep 2003 07:55:17 -0400
Subject: [pypy-dev]  Extended goals (was: needed 18 month plan of what we
	would do in developing PyPy if we got funding.)
In-Reply-To: <3F605D43.4030505@tismer.com>
Message-ID: <CFF66FE3-E44E-11D7-BE2C-000A95686CD8@redivi.com>


On Thursday, Sep 11, 2003, at 07:32 America/New_York, Christian Tismer 
wrote:

> Howdy,
>
> In addition to the goals in plan.txt, I think we need
> to do some more things.
> So far, the draft tells about PyPy, how to develop
> and maintain it, and how it can be extended.
> This is just fine from the view of a PyPy developer.
>
> The problem is to convince other parties to support
> us. They probably don't have a vital interest in
> PyPy for itself, but would like to see goals which
> go towards realy usable applications, which can be
> developed further.
>
> Therefore, besides the theoretical things which are
> great, I'm trying to add prototypical implementations
> of application-like stuff, as there is for example:
>
> - creating tiny, specialized code for embedded devices
>
> - distributed processing and load-balancing
>
> I think, these goals would be interesting for some
> companies which might participate, and they could
> support us by providing hardware and other support.
>
> In extent to this, I just realized a few days ago, that
> we have James Tauber in the PyPy list.
> He is working on an operating system in Python, see
> the cleese project at http://www.jtauber.com/cleese .
>
> In a similar way as we intend to integrate Psyco and
> Stackless in some way, I'd like to extend this to
> Cleese. I'd like to add
>
> - write a small, prototypical operating system in PyPy
>
> to the list of application goals.
>
> Question: Is there any objection to this, or can I continue?
>
> Are there other ideas of applications which we would
> like to include as "working horses" to create interest in PyPy?

What about a focus on scientific applications, number crunching?  There 
seems to be a pretty good community around SciPy, Numeric, numarray, 
etc.  Imagine tweaking a few settings in the PyPy JIT such that it 
aligns all of your ints, doubles, or floats to a particular memory 
boundary so the code could be vectorized.. and there could be add-ons 
to the JIT that allow it to take advantage of AltiVec or SSE2?

Personally, I'm really looking forward to a PPC JIT for Python (even if 
I have to help write it myself).. especially now that I have a dual 
2ghz g5 on order :)

-bob


From hpk at trillke.net  Thu Sep 11 14:17:34 2003
From: hpk at trillke.net (holger krekel)
Date: Thu, 11 Sep 2003 14:17:34 +0200
Subject: [pypy-dev] Extended goals (was: needed 18 month plan of what we
	would do in developing PyPy if we got funding.)
In-Reply-To: <3F605D43.4030505@tismer.com>;
	from tismer@tismer.com on Thu, Sep 11, 2003 at 01:32:19PM +0200
References: <5.2.1.1.0.20030904161323.026e5a80@pop.bluewin.ch>
	<200309041307.h84D7rQb029849@ratthing-b246.strakt.com>
	<5.2.1.1.0.20030904161323.026e5a80@pop.bluewin.ch>
	<5.2.1.1.0.20030907230643.026b5240@pop.bluewin.ch>
	<3F605D43.4030505@tismer.com>
Message-ID: <20030911141734.Z22698@prim.han.de>

Hi Christian,

[Christian Tismer Thu, Sep 11, 2003 at 01:32:19PM +0200]
> Howdy,
> 
> In addition to the goals in plan.txt, I think we need
> to do some more things.
> So far, the draft tells about PyPy, how to develop
> and maintain it, and how it can be extended.
> This is just fine from the view of a PyPy developer.

Which is currently (and for some time to come) the main
user group :-)

> ...

> In extent to this, I just realized a few days ago, that
> we have James Tauber in the PyPy list.
> He is working on an operating system in Python, see
> the cleese project at http://www.jtauber.com/cleese .
> 
> In a similar way as we intend to integrate Psyco and
> Stackless in some way, I'd like to extend this to
> Cleese. I'd like to add
> 
> - write a small, prototypical operating system in PyPy
> 
> to the list of application goals.
> 
> Question: Is there any objection to this, or can I continue?

to the opposite.  There were already some related discussions 
about PyPy and kernel interaction on this thread

    http://codespeak.net/pipermail/pypy-dev/2003q3/000832.html

where the main idea was to have PyPy perform syscalls directly
(without any glibc etc.).  this would make for a very efficient
and small runtime platform.  Driving hardware directly (without the
help of some linux-kernel) is certainly possible but i wouldn' t 
propose this for the next years unless someone like James Tauber is 
to join the project :-) 

cheers,

    holger

From tismer at tismer.com  Thu Sep 11 14:48:20 2003
From: tismer at tismer.com (Christian Tismer)
Date: Thu, 11 Sep 2003 14:48:20 +0200
Subject: [pypy-dev]  Extended goals (was: needed 18 month plan of what
 we would do in developing PyPy if we got funding.)
In-Reply-To: <CFF66FE3-E44E-11D7-BE2C-000A95686CD8@redivi.com>
References: <CFF66FE3-E44E-11D7-BE2C-000A95686CD8@redivi.com>
Message-ID: <3F606F14.8090109@tismer.com>

Bob Ippolito wrote:

> 
> On Thursday, Sep 11, 2003, at 07:32 America/New_York, Christian Tismer 
> wrote:

[extensions, cleese, ...]

>> Are there other ideas of applications which we would
>> like to include as "working horses" to create interest in PyPy?
> 
> 
> What about a focus on scientific applications, number crunching?  There 
> seems to be a pretty good community around SciPy, Numeric, numarray, 
> etc.  Imagine tweaking a few settings in the PyPy JIT such that it 
> aligns all of your ints, doubles, or floats to a particular memory 
> boundary so the code could be vectorized.. and there could be add-ons to 
> the JIT that allow it to take advantage of AltiVec or SSE2?

Ok, this opens several directions.
One is that we probably will want to (at least I do:) write
code generators for many architectures. This should be
great fun in a high level language like Python, with the
flexibility of PyPy.
The platforms you are addressing are special in the sense
that they use the SIMD model (my load balancing idea between
machines would be MIMD of course, if we do number crunching).

Remarks:
AltiVec is a joint development of Motorola, IBM, and Apple.
There are some document worth reading on their web site:
http://www.simdtech.org/apps/group_public/documents.php

SSE2 (Streaming SIMD Extension 2) is Intel's development.
There is some information here:
http://cedar.intel.com/cgi-bin/ids.dll/topic.jsp?catCode=DCG

I like the idea to support special processor extensions.
What I need is to formulate the reason why this should
particularily be done with PyPy.

Here an attempt:
PyPy is extremely flexible, since it supports static
and dynamic code generation by pluggable processor
architecture modules. PyPy can use special hardware
by itself, or it can cross-compile code for target
machines which will run only a downsized PyPy
instantiation which is runtime only.

One advantage of using PyPy instead of other compiled
libraries is its ability to not only support the
special hardware, but also to adjust to the given
machine properties, like cache sizes, number of
parallel FPUs, memory access speed and memory size.
PyPy is able to probe these parameters and to choose
an optimum implementation for the particular hardware
configuration during startup time.

[please give me more advantages here!]

Working package 0:
Provide a way to align certain data arrays in a way
that supports vectorizing.
Write PyPy implementations of numarray.
[Q.: Should be adopt existing number crunching C
      libraries, or better lift them up into PyPy?
      A whole bunch of new targets might come out here ...]

possible = [
   ( 1, "AltiVec"),
   ( 2, "SSE2"),
   # add some more here as you like
]

for i, name in possible:
     print """
Working package %d:
Implement a code generator that supports the SIMD
instructions of %s.
Provide demonstration code, exhaustive testing
and sample application which show how the numerical
packages take full advantage of the supported harware.
     """ % (i, name)

> Personally, I'm really looking forward to a PPC JIT for Python (even if 
> I have to help write it myself).. especially now that I have a dual 2ghz 
> g5 on order :)

I'd be more than happy to chime such stuff into PyPy,
and I know who'd love to work on it :-)

ciao - chris

-- 
Christian Tismer             :^)   <mailto:tismer at tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  pager +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/



From tismer at tismer.com  Thu Sep 11 14:57:35 2003
From: tismer at tismer.com (Christian Tismer)
Date: Thu, 11 Sep 2003 14:57:35 +0200
Subject: [pypy-dev] Extended goals
In-Reply-To: <20030911141734.Z22698@prim.han.de>
References: <5.2.1.1.0.20030904161323.026e5a80@pop.bluewin.ch>
	<200309041307.h84D7rQb029849@ratthing-b246.strakt.com>
	<5.2.1.1.0.20030904161323.026e5a80@pop.bluewin.ch>
	<5.2.1.1.0.20030907230643.026b5240@pop.bluewin.ch>
	<3F605D43.4030505@tismer.com> <20030911141734.Z22698@prim.han.de>
Message-ID: <3F60713F.5010507@tismer.com>

holger krekel wrote:

> Hi Christian,
> 
> [Christian Tismer Thu, Sep 11, 2003 at 01:32:19PM +0200]
> 
>>Howdy,
>>
>>In addition to the goals in plan.txt, I think we need
>>to do some more things.
>>So far, the draft tells about PyPy, how to develop
>>and maintain it, and how it can be extended.
>>This is just fine from the view of a PyPy developer.
> 
> Which is currently (and for some time to come) the main
> user group :-)

Oops, I'm sorry, somehow the discussion moved
out of pypy-funding.

Well, on the other hand it was ok to ask this here, since
I'm probably puting something into the proposal
that extends the PyPy project, which needs to be
discussed by all PyPy members, not only those who
are interested in funding.
I will send any specific, funding related
stuff to pypy-funding only. I hope there are enough
people subscribed, of course. :-)

[PyPy OS - Cleese?]

> to the opposite.  There were already some related discussions 
> about PyPy and kernel interaction on this thread
> 
>     http://codespeak.net/pipermail/pypy-dev/2003q3/000832.html
> 
> where the main idea was to have PyPy perform syscalls directly
> (without any glibc etc.).  this would make for a very efficient
> and small runtime platform.  Driving hardware directly (without the
> help of some linux-kernel) is certainly possible but i wouldn' t 
> propose this for the next years unless someone like James Tauber is 
> to join the project :-) 

Well, may this could be started using the Linux kernel,
and gradually move over to direct hardware support.
This also gives us the opportunity to extend the
project quite, if funding allows and there is interest.

ciao - chris

-- 
Christian Tismer             :^)   <mailto:tismer at tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  pager +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/



From pedronis at bluewin.ch  Thu Sep 11 15:20:25 2003
From: pedronis at bluewin.ch (Samuele Pedroni)
Date: Thu, 11 Sep 2003 15:20:25 +0200
Subject: [pypy-dev]  Extended goals (was: needed 18 month plan of
  what we would do in developing PyPy if we got funding.)
In-Reply-To: <3F605D43.4030505@tismer.com>
References: <5.2.1.1.0.20030907230643.026b5240@pop.bluewin.ch>
 <5.2.1.1.0.20030904161323.026e5a80@pop.bluewin.ch>
 <200309041307.h84D7rQb029849@ratthing-b246.strakt.com>
 <5.2.1.1.0.20030904161323.026e5a80@pop.bluewin.ch>
 <5.2.1.1.0.20030907230643.026b5240@pop.bluewin.ch>
Message-ID: <5.2.1.1.0.20030911150634.027701a0@pop.bluewin.ch>

At 13:32 11.09.2003 +0200, Christian Tismer wrote:
>Howdy,
>
>In addition to the goals in plan.txt, I think we need
>to do some more things.
>So far, the draft tells about PyPy, how to develop
>and maintain it, and how it can be extended.
>This is just fine from the view of a PyPy developer.
>
>The problem is to convince other parties to support
>us. They probably don't have a vital interest in
>PyPy for itself, but would like to see goals which
>go towards realy usable applications, which can be
>developed further.
>
>Therefore, besides the theoretical things which are
>great, I'm trying to add prototypical implementations
>of application-like stuff, as there is for example:
>
>- creating tiny, specialized code for embedded devices
>
>- distributed processing and load-balancing
>
>I think, these goals would be interesting for some
>companies which might participate, and they could
>support us by providing hardware and other support.
>
>In extent to this, I just realized a few days ago, that
>we have James Tauber in the PyPy list.
>He is working on an operating system in Python, see
>the cleese project at http://www.jtauber.com/cleese .
>
>In a similar way as we intend to integrate Psyco and
>Stackless in some way, I'd like to extend this to
>Cleese. I'd like to add
>
>- write a small, prototypical operating system in PyPy
>
>to the list of application goals.
>
>Question: Is there any objection to this, or can I continue?
>

one thing I think we should not forget is that merit of Python success is 
also because it is a good glue language. While PyPy performance change the 
balance of the equation, I think it will still be very important.

I have not thought this trough completely but consider this scenario (I 
think it is related to base functionality we want to provide, but has also 
a distinct "applicative" flavor):

someone has a large C++ codebase and - yes - he knows it is a pain to 
maintain and to add new functionality, PyPy-Python seems cool but what can 
it do for him?

1) well he can rewrite all his code in Python

  bzzt, wrong answer ( :) )

2) well he can embed PyPy in his code, PyPy extension mechanism and 
taylorability make
that easy and the overhead for PyPy to C++ calls very low, some PyPy 
classes can also work as C++ subclasses (<- this two thing is something we 
would have to work on, extending what is already there for CPython and beyond)

he can add new functionality in Python, and write unit tests in Python 
(also for old code),

he can slowly and progressevely refactor his code, and convert old things 
to Python as he
see fit.

this seems more reasonable because the there will always be a working system.

Samuele. 


From arigo at tunes.org  Fri Sep 12 12:41:51 2003
From: arigo at tunes.org (Armin Rigo)
Date: Fri, 12 Sep 2003 11:41:51 +0100
Subject: [pypy-dev]  Extended goals (was: needed 18 month plan of what we
	would do in developing PyPy if we got funding.)
In-Reply-To: <CFF66FE3-E44E-11D7-BE2C-000A95686CD8@redivi.com>
References: <3F605D43.4030505@tismer.com>
	<CFF66FE3-E44E-11D7-BE2C-000A95686CD8@redivi.com>
Message-ID: <20030912104151.GB18552@vicky.ecs.soton.ac.uk>

Hello Bob,

On Thu, Sep 11, 2003 at 07:55:17AM -0400, Bob Ippolito wrote:
> Personally, I'm really looking forward to a PPC JIT for Python (even if 
> I have to help write it myself).. especially now that I have a dual 
> 2ghz g5 on order :)

As a side note, Psyco now runs on PowerPCs using a custom virtual machine 
back-end.  http://psyco.sourceforge.net


Armin


From arigo at tunes.org  Fri Sep 12 13:15:07 2003
From: arigo at tunes.org (Armin Rigo)
Date: Fri, 12 Sep 2003 12:15:07 +0100
Subject: [pypy-dev]  Extended goals (was: needed 18 month plan of what we
	would do in developing PyPy if we got funding.)
In-Reply-To: <5.2.1.1.0.20030911150634.027701a0@pop.bluewin.ch>
References: <5.2.1.1.0.20030907230643.026b5240@pop.bluewin.ch>
	<5.2.1.1.0.20030904161323.026e5a80@pop.bluewin.ch>
	<200309041307.h84D7rQb029849@ratthing-b246.strakt.com>
	<5.2.1.1.0.20030904161323.026e5a80@pop.bluewin.ch>
	<5.2.1.1.0.20030907230643.026b5240@pop.bluewin.ch>
	<5.2.1.1.0.20030911150634.027701a0@pop.bluewin.ch>
Message-ID: <20030912111507.GC18552@vicky.ecs.soton.ac.uk>

Hello,

First, thanks to Christian for starting this thread -- I guess PyPy
applications could make another group of workpackages at least as big as the
"core PyPy" workpackages I listed in plan.txt.

On Thu, Sep 11, 2003 at 03:20:25PM +0200, Samuele Pedroni wrote:
> one thing I think we should not forget is that merit of Python success is 
> also because it is a good glue language. While PyPy performance change the 
> balance of the equation, I think it will still be very important.

Samuele has a point here. We should consider some of the major application
areas of today's Python (glue language, scientific/numeric, Zope) and argue
what PyPy can offer in addition to CPython. In general it's all based on 
flexibility; for example:

> 2) well he can embed PyPy in his code, PyPy extension mechanism and 
> taylorability make
> that easy and the overhead for PyPy to C++ calls very low, some PyPy 
> classes can also work as C++ subclasses (<- this two thing is something we 
> would have to work on, extending what is already there for CPython and 
> beyond)

Here was can target C++ for the translation, representing Python objects as
classes which could be tailored to be compatible with the existing
application's class model.

More generally, we can put forward as a goal the reduction of the boilerplate
interfacing code. This boilerplate isn't so huge right now, which is in part
why Python is good as being a glue language, but still it is large enough that
there are specific tools like SWIG to help generate it. We can help there,
both to reduce the amount of code and to actually reduce the corresponding
run-time overhead.

For numeric applications, there are several senarios that come to mind, based
on Christian's and Holger's messages. However I don't know Numeric well enough
to see what could really be done, so take this with a grain of salt. We could
rewrite Numeric in RPython and regenerate good come from that, which has the
advantage that the code can indeed be very good because it can be tailored to
use (or not) particular hardware features. So far it can more or less be done
with a C library full of #defines. The difference is in its integration: this
Numeric can influence PyPy's own object layout and alignment if needed, for
example. It can also be translated into something that checks at run-time for
what features are present; in C it is harder to do without recompilation. But
more importantly, if it is written in RPython instead of C it can be subjected
to other transformations like Psyco-ification, the net result of which would
be a Psyco that knows about Numeric types and can directly generate
processor-optimized code like SSE2.


A bientot,

Armin


From arigo at tunes.org  Fri Sep 12 16:28:34 2003
From: arigo at tunes.org (Armin Rigo)
Date: Fri, 12 Sep 2003 15:28:34 +0100
Subject: [pypy-dev] Re: [Python-Dev] Making python C-API thread safe (try 2)
In-Reply-To: <3F615217.8060106@sci.fi>
References: <5.1.1.6.0.20030911142317.02b88640@telecommunity.com>
	<5.1.1.6.0.20030911130607.02426ec0@telecommunity.com>
	<5.1.1.6.0.20030911130607.02426ec0@telecommunity.com>
	<5.1.1.6.0.20030911142317.02b88640@telecommunity.com>
	<5.1.1.6.0.20030911162016.02027750@telecommunity.com>
	<3F615217.8060106@sci.fi>
Message-ID: <20030912142834.GD18552@vicky.ecs.soton.ac.uk>

Hello Harri,

Sorry for the private e-mail about the Python threading issue you raied in
pypy-dev; I guess you got quite some of these. I'm sending a copy of this mail
to the pypy-dev mailing list -- Christian, this is one more application not to
forget to list in the goals :-)

I wanted to point you to the PyPy project, whose goal is to create a more
flexible Python interpreter, by writing it completely in Python instead of C.

The kind of trivial-but-tedious global transformation that you suggest to make
multiple independent interpreters in the same process is exactly the kind of
change that we aim for, thanks to extra flexibility. We are writing the
interpreter in such a way that it can be automatically translated into
efficient C code. This translator can be tailored to different usages, for
example to add an extra "PyInterpreterState*" argument to each C function it
produces.

So if you are genuinely considering patching the whole CPython source code,
consider that it is not only tedious but is a nightmare to maintain if you
want to keep up-to-date with CPython's evolution. There are several people in
the PyPy project that have tried to do exactly that for other kind of changes
(Psyco/Stackless). It is one of the motivations behind PyPy. And besides the
new flexibility, it is much more fun to write a Python interpreter in Python
than in C :-)

http://codespeak.net/pypy/


Armin


From lac at strakt.com  Sat Sep 13 12:02:39 2003
From: lac at strakt.com (Laura Creighton)
Date: Sat, 13 Sep 2003 12:02:39 +0200
Subject: [pypy-dev] NOBODY expects the Spanish Inquistion!
Message-ID: <200309131002.h8DA2dX8005651@ratthing-b246.strakt.com>


See: http://www.codespeak.net/pypy/index.cgi?lists

(JARRING CHORD)
(The door flies open and Cardinal Ximinez of Spain (Palin) enters, flanked by
two junior cardinals.  Cardinal Biggles (Jones) has goggles pushed over his
forehead.  Cardinal Fang (Gilliam) is just Cardinal Fang)
 
Ximinez: NOBODY expects the Spanish Inquisition!  Our chief weapon is
         suprise...surprise and fear...fear and surprise....  Our two
         weapons are fear and surprise...and ruthless efficiency....  Our
         *three* weapons are fear, surprise, and ruthless efficiency...and an
         almost fanatical devotion to the Pope....  Our *four*...no...
         *Amongst* our weapons....  Amongst our weaponry...are such elements as
         fear, surprise....  I'll come in again.  (Exit and exeunt)

From lac at strakt.com  Sat Sep 13 16:51:47 2003
From: lac at strakt.com (Laura Creighton)
Date: Sat, 13 Sep 2003 16:51:47 +0200
Subject: [pypy-dev] history of PyPy
Message-ID: <200309131451.h8DEplu5006885@ratthing-b246.strakt.com>

Holger wrote something about 'how Armin, Christian and Holger
started discussing this on the german python mailing list' for
the history of the project.  I've since lost it.  Anybody
have that, also the exact dates for the Sprints?

Laura

From roccomoretti at netscape.net  Sat Sep 13 20:24:51 2003
From: roccomoretti at netscape.net (Rocco Moretti)
Date: Sat, 13 Sep 2003 14:24:51 -0400
Subject: [pypy-dev] "Unwrap" Concidered Harmful
Message-ID: <4614FE29.6C52EA96.9ADE5C6A@netscape.net>

Armin Rigo <arigo at tunes.org> wrote:

>On Sat, Aug 23, 2003 at 09:00:58PM -0400, Rocco Moretti wrote:

>> all information about value. So there is no way we can say that
>> s.unwrap(s.wrap(x)) == x.
>
>I am now coming closer to the idea that wrap() should be redefined as well,
>because it is used not only on simple types but also on "internal interpreter
>structures", like code objects plus others more internals than that.

Let me see if I can clarify my present understanding. (Thinking out loud follows...)

When we use wrap(x), we mean one of two things:

1) Give me an application level object that is equivalent to x.
2) Make the interpreter level object x available at application level.

Case 1
------

Michael advocates calling use 1) "build". For the build case, it is
wrong to assume that unwrap(wrap(x)) == x, as, in general, an
application level object may not have a 1-to-1 correspondence with an
interpreter level object.

My opinion is that "unbuild" should not exist. Once an object is built,
there is no guarantee that it will ever again correspond directly to
an interpreter level object. If we need an interpreter level object out
of an application level one, we should do it in a very restricted way,
for a specified purpose. If we want an interpreter level object, it is
so that we can do manipulations on it, but in general, unwrap and
manipulations are not associative: 
    space.unwrap(space.str(w_x)) != str(space.unwrap(w_x))
    
space.is_true() is an example of such a limited use function. In my mind,
is_true() exists for a single purpose - to determine if a wrapped
object is true so that the interpreter can determine if a
JUMP_IF_TRUE type branch should be taken. Any other use would be
considered a "hack".

I can perhaps see the need for a .to_string(x) function, with the
semantics that "the results of this function are going to be presented
directly to the user - I don't care what sort of bizarre and twisted
data structure 'x' actually represents, just make a reasonable string
so that the human at the keyboard has some kind of feedback for
debugging purposes." Use of the output for any other purpose would
be highly discouraged. - I would hesitate to add any more such functions
until a compelling need for one is demonstrated.

Case 2
------

This is what we are looking for when we say .wrap(frame). Here the role
of unwrap depends on how wrap behaves. [If unwrap_2 exists, it should
*always* have the semantics that unwrap(wrap(x)) == x] - Possible
alternative names for this functionality include shadow() or proxy().

As I interpret it, Armin indicates wrap_2() should create an application
level object that shadows an interpreter level object.

That is:
    y = space.wrap(x)
    print x # <'xyz'>
    space.fubar(y) #.fubar alters y
    print x # <'yzw'>
and also:
    y = space.wrap(x)
    print y # <'xyz'>
    x.fubar()
    print y # <'npq'>

In implementation, y is just a proxy which passes along the
modifications or attribute acesses to the underlying interpreter object,
and (unwrap(wrap(x)) is x) is True.

An alternative is to have wrap_2() create a new object which retains
enough information to recreate an interpreter level object when unwrap()
is called. In this case there would be no differences in output for the
print statements above, and (unwrap(wrap(x)) is not x) is True.

We still have a problem with unwrap if we allow the application level
to modify objects produced by wrap_2. Say we have a function object.
What happens when someone says:

    fun.func_name = name

in application level code? What if name is generated from the application
level and contains an object that represents "92% chance of the string
'MyFun', 7% chance of a class with the .__str__ method hooked up to the
fortunes generator, and 1% chance of rain."?

The only solution I see is to limit the rvalue to objects that were
also created with wrap_2 and thus have their interpreter level values
still attached. So in effect, our black box objects become a black hole -
values go in, but can never come back out to interpreter level.

-Rocco

P.S. On further consideration, there is a third solution. We *can* allow
an "unbuild" function, with the directive that it is nervous, and
raises an exception every time it is the least bit confused at what to
do. This allows for free flow of information in the regular cases when
we aren't doing anything tricky with the object spaces.

My concern would be that we would become dependant on a well behaved
unbuild, and require it in a critical piece of code, making development
of the clever object spaces difficult to impossible. I'd disapprove of
it's use in any code that would be exercised in interpreting the majority
of Python scripts. Discourage its use and save it for implementing the
dusty introspection/self referential corners of Python only - and be
sure to document the expected behavior of the return values well, so
any conflicts can be anticipated.


__________________________________________________________________
McAfee VirusScan Online from the Netscape Network.
Comprehensive protection for your entire computer. Get your free trial today!
http://channels.netscape.com/ns/computing/mcafee/index.jsp?promo=393397

Get AOL Instant Messenger 5.1 free of charge.  Download Now!
http://aim.aol.com/aimnew/Aim/register.adp?promo=380455

From roccomoretti at netscape.net  Sat Sep 13 20:42:19 2003
From: roccomoretti at netscape.net (Rocco Moretti)
Date: Sat, 13 Sep 2003 14:42:19 -0400
Subject: [pypy-dev] Extended goals
Message-ID: <12FCA545.20E83982.9ADE5C6A@netscape.net>

Christian Tismer <tismer at tismer.com> wrote:

>holger krekel wrote:
>
>> where the main idea was to have PyPy perform syscalls directly
>> (without any glibc etc.). ?this would make for a very efficient
>> and small runtime platform. ?Driving hardware directly (without the
>> help of some linux-kernel) is certainly possible but i wouldn' t 
>> propose this for the next years unless someone like James Tauber is 
>> to join the project :-) 
>
>Well, may this could be started using the Linux kernel,
>and gradually move over to direct hardware support.
>This also gives us the opportunity to extend the
>project quite, if funding allows and there is interest.

I hate to be the one to bring up licensing issues, but ...

I might recommend using a *BSD instead of Linux, to avoid having to manage
GPL code portions. If an (admittedly optimistic) goal of PyPy is to 
become the reference implementation of Python, licensing concerns need be 
addressed, as I get the impression that Guido et al. would much prefer 
not to have to deal with the GPL/FSF.

It's not that I'm against taking advantage of existing work when 
availible, its just that interoperability with the core Python developers 
(as well as the Python-using community in general) would be enhanced by 
sticking to the BSD/MIT (i.e. original Python) licensing scheme as much as possible.

Next-week-I'll-bring-up-Emacs-vs-Vi-ly'rs, ;-)
-Rocco

__________________________________________________________________
McAfee VirusScan Online from the Netscape Network.
Comprehensive protection for your entire computer. Get your free trial today!
http://channels.netscape.com/ns/computing/mcafee/index.jsp?promo=393397

Get AOL Instant Messenger 5.1 free of charge.  Download Now!
http://aim.aol.com/aimnew/Aim/register.adp?promo=380455

From roccomoretti at netscape.net  Sat Sep 13 20:51:39 2003
From: roccomoretti at netscape.net (Rocco Moretti)
Date: Sat, 13 Sep 2003 14:51:39 -0400
Subject: [pypy-dev]
	=?ISO-8859-1?B?oEV4dGVuZGVkIGdvYWxzICh3YXM6IG5lZWRlZCAxOCBtb250aCBwbGFuIG9md2hhdCB3ZSB3b3VsZCBkbyBpbiBkZXZlbG9waW5nIFB5UHkgaWYgd2UgZ290IGZ1bmRpbmcuKQ==?=
Message-ID: <6618AC7D.69F39C5A.9ADE5C6A@netscape.net>

Samuele Pedroni <pedronis at bluewin.ch> wrote:

>someone has a large C++ codebase and - yes - he knows it is a pain to 
>maintain and to add new functionality, PyPy-Python seems cool but what can 
>it do for him?
>
>1) well he can rewrite all his code in Python
>
> ?bzzt, wrong answer ( :) )
>

Or, he could have PyPy rewrite it for him:

If we continue on the track of maximum generality, we should be able to 
hook a C/C++ parser/code generator into the front end of PyPy. And with a 
flexible enough backend, instead of PyPy-Psyco emitting machine code or 
C, it should be able to emit Python code.

The trick is to write such a beast such that comments are preserved and 
the emitted python code is readable.

-Rocco

__________________________________________________________________
McAfee VirusScan Online from the Netscape Network.
Comprehensive protection for your entire computer. Get your free trial today!
http://channels.netscape.com/ns/computing/mcafee/index.jsp?promo=393397

Get AOL Instant Messenger 5.1 free of charge.  Download Now!
http://aim.aol.com/aimnew/Aim/register.adp?promo=380455

From pedronis at bluewin.ch  Sat Sep 13 21:41:58 2003
From: pedronis at bluewin.ch (Samuele Pedroni)
Date: Sat, 13 Sep 2003 21:41:58 +0200
Subject:  =?iso-8859-1?Q?Re:_[pypy-dev]_=A0Extended_goals_(was:_needed?=
  18 month plan ofwhat we would do in developing PyPy if we got funding
 .)
In-Reply-To: <6618AC7D.69F39C5A.9ADE5C6A@netscape.net>
Message-ID: <5.2.1.1.0.20030913212139.02777060@pop.bluewin.ch>

At 14:51 13.09.2003 -0400, Rocco Moretti wrote:
>Samuele Pedroni <pedronis at bluewin.ch> wrote:
>
> >someone has a large C++ codebase and - yes - he knows it is a pain to
> >maintain and to add new functionality, PyPy-Python seems cool but what can
> >it do for him?
> >
> >1) well he can rewrite all his code in Python
> >
> >  bzzt, wrong answer ( :) )
> >
>
>Or, he could have PyPy rewrite it for him:
>
>If we continue on the track of maximum generality, we should be able to
>hook a C/C++ parser/code generator into the front end of PyPy. And with a
>flexible enough backend, instead of PyPy-Psyco emitting machine code or
>C, it should be able to emit Python code.
>
>The trick is to write such a beast such that comments are preserved and
>the emitted python code is readable.

there is a tension between converting struct/classes to classes vs them
as views on memory regions and access through pointer arithmetic etc...

The lisp machines had a C compiler, using lisp arrays for memory allocation,
but I doubt it tried to produce readable lisp code. And I don't know how
it fared with non-standard-conforming pointer arithmetic and comparisons...

That's why technically, I think, Microsoft invented Managed C++ for .NET 
plus the
capability to still mix and escape to/from non-managed (vanilla) C++.
It's also their way to allow piece-wise conversion.








From hpk at trillke.net  Sun Sep 14 09:32:15 2003
From: hpk at trillke.net (holger krekel)
Date: Sun, 14 Sep 2003 09:32:15 +0200
Subject: [pypy-dev] history of PyPy
In-Reply-To: <200309131451.h8DEplu5006885@ratthing-b246.strakt.com>;
	from lac@strakt.com on Sat, Sep 13, 2003 at 04:51:47PM +0200
References: <200309131451.h8DEplu5006885@ratthing-b246.strakt.com>
Message-ID: <20030914093215.V12635@prim.han.de>

Hello Laura,

[Laura Creighton Sat, Sep 13, 2003 at 04:51:47PM +0200]
> Holger wrote something about 'how Armin, Christian and Holger
> started discussing this on the german python mailing list' for
> the history of the project.  I've since lost it.  Anybody
> have that, also the exact dates for the Sprints?

Here is the posting where i offer to organize the first "Minimal
Python" sprint which quickly turned into "PyPy".  It is a response
to Christian Tismer who said in a "P.S." [1] to Gerson Kurz that 
writing "a totally minimzed python" would be quite interesting. 

http://starship.python.net/pipermail/python-de/2003q1/002925.html

Christian and me involved Armin (in private mails) in the next weeks 
and we quickly agreed to send out the first sprint announcement and 
invitation to the "Hildesheim" sprint (containing the exact dates):

http://mail.python.org/pipermail/python-dev/2003-January/032427.html

Actually, i came to the idea of doing that sprint because i attended 
Zope3 sprints and liked several aspects about them.  The problem here
obviously was that we started out from nothing which is not 
common but proved to be workable. 

hope that helps, if not, ask for more detais. 

cheers,

    holger


[1] Never underestimate the potential of a P.S. :-)

From lac at strakt.com  Sun Sep 14 10:14:34 2003
From: lac at strakt.com (Laura Creighton)
Date: Sun, 14 Sep 2003 10:14:34 +0200
Subject: [pypy-dev] history of PyPy 
In-Reply-To: Message from holger krekel <hpk@trillke.net> 
   of "Sun, 14 Sep 2003 09:32:15 +0200." <20030914093215.V12635@prim.han.de> 
References: <200309131451.h8DEplu5006885@ratthing-b246.strakt.com>
	<20030914093215.V12635@prim.han.de> 
Message-ID: <200309140814.h8E8EYq3010505@ratthing-b246.strakt.com>

In a message of Sun, 14 Sep 2003 09:32:15 +0200, holger krekel writes:
>Hello Laura,
>
>[Laura Creighton Sat, Sep 13, 2003 at 04:51:47PM +0200]
>> Holger wrote something about 'how Armin, Christian and Holger
>> started discussing this on the german python mailing list' for
>> the history of the project.  I've since lost it.  Anybody
>> have that, also the exact dates for the Sprints?
>
>Here is the posting where i offer to organize the first "Minimal
>Python" sprint which quickly turned into "PyPy".  It is a response
>to Christian Tismer who said in a "P.S." [1] to Gerson Kurz that 
>writing "a totally minimzed python" would be quite interesting. 
>
>http://starship.python.net/pipermail/python-de/2003q1/002925.html
>
>Christian and me involved Armin (in private mails) in the next weeks 
>and we quickly agreed to send out the first sprint announcement and 
>invitation to the "Hildesheim" sprint (containing the exact dates):
>
>http://mail.python.org/pipermail/python-dev/2003-January/032427.html
>
>Actually, i came to the idea of doing that sprint because i attended 
>Zope3 sprints and liked several aspects about them.  The problem here
>obviously was that we started out from nothing which is not 
>common but proved to be workable. 
>
>hope that helps, if not, ask for more detais. 
>
>cheers,
>
>    holger
>
>
>[1] Never underestimate the potential of a P.S. :-)

Oh man, this is perfect.  They want Agile Methodologies!!!  

------------

New paragraph should be something like:

----------

Meanwhile, on the German Python mailing list,
http://starship.python.net/mailman/listinfo/python-de an unrelated
discussion was about to bear fruit.   FOOTNOTE:
Here's the mail that started it all, 
http://starship.python.net/pipermail/python-de/2003q1/002925.html 
include full text.

Academically trained Python
programmers were pondering writing an implementation of Python in
itself.  This group included Armin Rigo, author of Psyco
http://psyco.sourceforge.net/introduction.html, a specialising
Just-In-Time compiler for Python, and Christian Tismer, author of
Stackless Python, http://www.stackless.com/ an implementation of
Python which does not use the C stack.  These people knew and were
interested in doing exactly that as part of the design.

It is useful to quote Armin Rigo, stating the goals of Psyco in full,
from its website:

	My goal in programming Psyco is to contribute to reduce the
	following wide gap between academic computer science and industrial
	programming tools.

	While the former develops a number of programming languages
	with very cool semantics and features, the latter stick with low-level
	languages principally for performance reasons, on the ground that the
	higher the level of a language, the slower it is. Althought clearly
	justified in practice, this belief is theoretically false, and even
	completely inverted --- for large, evolving systems like a whole
	operating system and its applications, high-level programming can
	deliver much higher performances.

	The new class of languages called "dynamic scripting
	languages", of which Python is an example, is semantically close to
	long-studied languages like Lisp. The constrains behind their designs
	are however different: some high-level languages can be relatively
	well statically compiled, we can do some type inference, and so on,
	whereas with Python it is much harder --- the design goal was
	different. We now have powerful machines to stick with interpretation
	for a number of applications. This, of course, contributes to the
	common belief that high-level languages are terribly slow.

	Psyco is both an academic and an industrial project. It is an 
	academic experiment testing some new techniques in the field of 
	on-line specialization. It develops an industrially useful 
	performance benefit for Python. And first of all it is a modest 
	step towards:

		High-level languages are faster than low-level ones!

FIXME Footnote to: http://psyco.sourceforge.net/introduction.html

PyPy would be a more ambitious step, of course.  

Also instrumental was Holger Krekel, at TrillkeGut a (what the heck
do you call it?? in Hildesheim).  Holger Krekel was experienced 
in one of the latest in Agile Software methodologies, Sprint-driven
development, from his experience in the Zope   HOLGER
WRITE ME A PARAGRAPH HERE PLEASE ABOUT WHY THIS IS COOL


This was precisely what the Python Business Forum members had been
looking for.  Academics conversant in the latest in compiler theory
who were interested in producing the Python interpreter which they
needed.  We began discussing the possibility of collaboration, both
on mailing lists,   FIXME list our mailing list and in our first
face-to-face meeting at Hildesheim, Germany Feb 17-23 2003.

We decided to make a prototype to test the proof of concept, and our
ability to work together.  We would get together every six weeks in
our own time, and work on the prototype in intense work sessions which
are known as 'Sprints'.  FIXME Etymology of Sprint?  After the first
Sprint at TrillkeGut in Hildesheim, we held subsequent Sprints at AB
Strakt in Gothenburg, Sweden, and at the University of
Louvain-La-Neuve in Belgium.

By the time the Louvain-La-Neuve Sprint, which was held June 20-24,
ended, the PyPy project had produced a working prototype.  There
existed a working interpreter and standard implementation of 90% of
the python types, as well as such language features as nested scopes,
generators and metaclasses.  We had begun work on a type inference
engine.  Perhaps most importantly, we had the enthusiastic support of
the Python community.  Guido van Rossum, author of Python, not only
endorsed the PyPy project, but attended our Louvain-La-Neuve Sprint.
No work had been done on actually Optimising the code, so it ran
FIXME  30,000 times(?) slower than existing Python.  But for a
proof of concept, approximately three weeks work total, it was
a success.

It was time to look for funding.

From lac at strakt.com  Sun Sep 14 10:16:30 2003
From: lac at strakt.com (Laura Creighton)
Date: Sun, 14 Sep 2003 10:16:30 +0200
Subject: [pypy-dev] history of PyPy 
In-Reply-To: Your message of "Sun, 14 Sep 2003 09:32:15 +0200."
             <20030914093215.V12635@prim.han.de> 
References: <200309131451.h8DEplu5006885@ratthing-b246.strakt.com>
	<20030914093215.V12635@prim.han.de> 
Message-ID: <200309140816.h8E8GUq3010524@ratthing-b246.strakt.com>

Im going to put the background doc into our doc dir so its easier
for us to all make changes.

Laura

From lac at strakt.com  Sun Sep 14 10:35:20 2003
From: lac at strakt.com (Laura Creighton)
Date: Sun, 14 Sep 2003 10:35:20 +0200
Subject: [pypy-dev] HELP!  I can't do svn up!
Message-ID: <200309140835.h8E8ZK3c010604@ratthing-b246.strakt.com>


Error follows:
At revision 1289.

Fetching external item into trunk/www/service/vpath
svn: RA layer request failed
svn: REPORT response handling failed to complete the editor drive

From hpk at trillke.net  Sun Sep 14 11:23:13 2003
From: hpk at trillke.net (holger krekel)
Date: Sun, 14 Sep 2003 11:23:13 +0200
Subject: [pypy-dev] HELP!  I can't do svn up!
In-Reply-To: <200309140835.h8E8ZK3c010604@ratthing-b246.strakt.com>;
	from lac@strakt.com on Sun, Sep 14, 2003 at 10:35:20AM +0200
References: <200309140835.h8E8ZK3c010604@ratthing-b246.strakt.com>
Message-ID: <20030914112313.Y12635@prim.han.de>

[Laura Creighton Sun, Sep 14, 2003 at 10:35:20AM +0200]
> 
> Error follows:
> At revision 1289.
> 
> Fetching external item into trunk/www/service/vpath
> svn: RA layer request failed
> svn: REPORT response handling failed to complete the editor drive

hmmm. this works for me.  

1) Does it repeatedly fail? 

2) does anyone else have this problem? 

cheers,
    
    holger

From lac at strakt.com  Sun Sep 14 12:16:32 2003
From: lac at strakt.com (Laura Creighton)
Date: Sun, 14 Sep 2003 12:16:32 +0200
Subject: [pypy-dev] HELP! I can't do svn up! 
In-Reply-To: Message from holger krekel <hpk@trillke.net> 
   of "Sun, 14 Sep 2003 11:23:13 +0200." <20030914112313.Y12635@prim.han.de> 
References: <200309140835.h8E8ZK3c010604@ratthing-b246.strakt.com>
	<20030914112313.Y12635@prim.han.de> 
Message-ID: <200309141016.h8EAGWq3010942@ratthing-b246.strakt.com>

In a message of Sun, 14 Sep 2003 11:23:13 +0200, holger krekel writes:
>[Laura Creighton Sun, Sep 14, 2003 at 10:35:20AM +0200]
>> 
>> Error follows:
>> At revision 1289.
>> 
>> Fetching external item into trunk/www/service/vpath
>> svn: RA layer request failed
>> svn: REPORT response handling failed to complete the editor drive
>
>hmmm. this works for me.  
>
>1) Does it repeatedly fail? 
>
>2) does anyone else have this problem? 
>
>cheers,
>    
>    holger

I only have it on my laptop.  My repository on my machine at work is fine.
(which is why even though I am at home, I am hacking now by telnetting
to work, rather than writing stuff on my laptop).

Laura



From lac at strakt.com  Mon Sep 15 12:34:03 2003
From: lac at strakt.com (Laura Creighton)
Date: Mon, 15 Sep 2003 12:34:03 +0200
Subject: [pypy-dev] new section B4.2_other_countries.txt
Message-ID: <200309151034.h8FAY3mt016090@ratthing-b246.strakt.com>

Don't be impressed, it's just a list of questions not to forget to ask 
Stockholm.
You are free to add your own, of course.  I wish you _would_.
Make us a list so we don't forget stuff.  Call it Stockholm_Questions.txt
or some such and stick it in the doc directory.

Thanks, 
Laura

From arigo at tunes.org  Mon Sep 15 14:15:17 2003
From: arigo at tunes.org (Armin Rigo)
Date: Mon, 15 Sep 2003 13:15:17 +0100
Subject: [pypy-dev] "Unwrap" Concidered Harmful
In-Reply-To: <4614FE29.6C52EA96.9ADE5C6A@netscape.net>
References: <4614FE29.6C52EA96.9ADE5C6A@netscape.net>
Message-ID: <20030915121517.GB17488@vicky.ecs.soton.ac.uk>

Hello Rocco,

On Sat, Sep 13, 2003 at 02:24:51PM -0400, Rocco Moretti wrote:
> space.is_true() is an example of such a limited use function. In my mind,
> is_true() exists for a single purpose - to determine if a wrapped
> object is true so that the interpreter can determine if a
> JUMP_IF_TRUE type branch should be taken. Any other use would be
> considered a "hack".

Right. However there are a few other cases in which the interpreter may be 
interested in "probing" information about a wrapped object. Here are two I can 
think of:

 * getting an integer. We need this at a few points, e.g. to know the exact
length of a tuple (which is needed to interpret something like "f(*args)").

 * getting a (short) string. Again there is an example in arguments decoding, 
to interpret "f(**kw)" we need to manipulate the keywords.

It may be possible to remove all these uses of unwrap (e.g. the length of a 
tuple can be obtained by iterating over it and counting), but I'd say that it 
would only make matters more obscure. Instead, just as we have "is_true()" we 
could do with a few other methods to get the integer value of an object, or 
its string value.

> Case 2
> ------
> (...)
> In implementation, y is just a proxy which passes along the
> modifications or attribute acesses to the underlying interpreter object,
> and (unwrap(wrap(x)) is x) is True.

Yes, that's the intended purpose. Application-level code that tries to 
manipulate such a "proxy" wrapped object, as in

>     fun.func_name = name

will make the object space call back to the interpreter to perform the 
operation. So the interpreter itself is free to implement the operation as it 
wishes using unwrap(fun) to get back to its own original object. This is 
similar to CPython's attribute and method tables, in which C code simulates 
what is seen at application-level as attribute reading/writing.


A bientot,

Armin.


From arigo at tunes.org  Mon Sep 15 14:17:27 2003
From: arigo at tunes.org (Armin Rigo)
Date: Mon, 15 Sep 2003 13:17:27 +0100
Subject: [pypy-dev] Extended goals
In-Reply-To: <12FCA545.20E83982.9ADE5C6A@netscape.net>
References: <12FCA545.20E83982.9ADE5C6A@netscape.net>
Message-ID: <20030915121727.GC17488@vicky.ecs.soton.ac.uk>

Hello Rocco,

On Sat, Sep 13, 2003 at 02:42:19PM -0400, Rocco Moretti wrote:
> I might recommend using a *BSD instead of Linux, to avoid having to manage
> GPL code portions.

PyPy is under the MIT license, it was from the beginning.


Armin


From hpk at trillke.net  Mon Sep 15 15:02:54 2003
From: hpk at trillke.net (holger krekel)
Date: Mon, 15 Sep 2003 15:02:54 +0200
Subject: [pypy-dev] how to help with PyPy right now
Message-ID: <20030915150254.P12635@prim.han.de>

Hello PyPy developers and observers,

currently we are setting up (led by Laura Creighton) some documents 
in preparation for applying to EU funding.  We know that this is 
far-too-close timing especially considering the private workload 
of everybody and the approaching "Berlin" sprint and its organization. 

That's why everyone is warmly welcomed to read the evolving 
documents in the "funding" section of the website, e.g. 

http://codespeak.net/pypy/index.cgi?doc/funding/B2.0.0_background.html

If you have any suggestions, corrections or additions to contribute
that would most certainly help.  Btw, with each document you find a 
"ReST-source" link at a ReST-text file which is the source for each 
documentation page.  Please feel free to send corrected/extended 
versions to pypy-funding at codespeak.net so we can incorporate them. 
Alterternatively if you have an svn-client you can check out the 
documents at 

    http://codespeak.net/svn/pypy/trunk/doc/funding

thanks for any help,

    Holger & Laura


P.S.: 
    Laura is overworked, and may not answer you as quickly as 
    you deserve, but is greatful for any help. 

From Nicolas.Chauvat at logilab.fr  Tue Sep 16 00:10:29 2003
From: Nicolas.Chauvat at logilab.fr (Nicolas Chauvat)
Date: Tue, 16 Sep 2003 00:10:29 +0200
Subject: [pypy-dev] Extended goals (was: needed 18 month plan of what we
	would do in developing PyPy if we got funding.)
In-Reply-To: <3F605D43.4030505@tismer.com>
References: <5.2.1.1.0.20030904161323.026e5a80@pop.bluewin.ch>
	<200309041307.h84D7rQb029849@ratthing-b246.strakt.com>
	<5.2.1.1.0.20030904161323.026e5a80@pop.bluewin.ch>
	<5.2.1.1.0.20030907230643.026b5240@pop.bluewin.ch>
	<3F605D43.4030505@tismer.com>
Message-ID: <20030915221029.GI7608@logilab.fr>

On Thu, Sep 11, 2003 at 01:32:19PM +0200, Christian Tismer wrote:

> The problem is to convince other parties to support
> us. They probably don't have a vital interest in
> PyPy for itself, but would like to see goals which
> go towards realy usable applications, which can be
> developed further.

Yes.

Ideally, we should have technical/research partners and end-user partners.

Do we have both at the moment ? Who are the end users? Having on board
new end-users will be close to impossible in the upcoming month, but we
can try.

By the way, do we have both small and big partners ?


-- 
Nicolas Chauvat

http://www.logilab.com - "Mais o? est donc Ornicar ?" - LOGILAB, Paris (France)

From Nicolas.Chauvat at logilab.fr  Tue Sep 16 00:17:19 2003
From: Nicolas.Chauvat at logilab.fr (Nicolas Chauvat)
Date: Tue, 16 Sep 2003 00:17:19 +0200
Subject: [pypy-dev]  Extended goals (was: needed 18 month plan of what we
	would do in developing PyPy if we got funding.)
In-Reply-To: <20030912111507.GC18552@vicky.ecs.soton.ac.uk>
References: <5.2.1.1.0.20030907230643.026b5240@pop.bluewin.ch>
	<5.2.1.1.0.20030904161323.026e5a80@pop.bluewin.ch>
	<200309041307.h84D7rQb029849@ratthing-b246.strakt.com>
	<5.2.1.1.0.20030904161323.026e5a80@pop.bluewin.ch>
	<5.2.1.1.0.20030907230643.026b5240@pop.bluewin.ch>
	<5.2.1.1.0.20030911150634.027701a0@pop.bluewin.ch>
	<20030912111507.GC18552@vicky.ecs.soton.ac.uk>
Message-ID: <20030915221719.GJ7608@logilab.fr>

On Fri, Sep 12, 2003 at 12:15:07PM +0100, Armin Rigo wrote:

> Samuele has a point here. We should consider some of the major application
> areas of today's Python (glue language, scientific/numeric, Zope) and argue
> what PyPy can offer in addition to CPython. In general it's all based on 
> flexibility; for example:

+1

Explain that largely succesfull projects : Python, Python for Scientific apps, Zope will be even more successfull thanks to PyPy.

Reuse success stories from python.org and pythonology.org to show that Python is successfull and that this success has to be pushed further.

-- 
Nicolas Chauvat

http://www.logilab.com - "Mais o? est donc Ornicar ?" - LOGILAB, Paris (France)

From Dorinewarm at 03v.com  Tue Sep 16 01:50:44 2003
From: Dorinewarm at 03v.com (Female Trends)
Date: Mon, 15 Sep 2003 22:50:44 -0100
Subject: [pypy-dev] Boost your sexiness-boost your L|ps size
Message-ID: <b0dudum15l$6e1z$hth7u$229-x@j5c416i6hqf3c>

An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20030915/93a4b28c/attachment.html>

From tismer at tismer.com  Tue Sep 16 11:35:35 2003
From: tismer at tismer.com (Christian Tismer)
Date: Tue, 16 Sep 2003 11:35:35 +0200
Subject: [pypy-dev] Extended goals
In-Reply-To: <20030915221029.GI7608@logilab.fr>
References: <5.2.1.1.0.20030904161323.026e5a80@pop.bluewin.ch>
	<200309041307.h84D7rQb029849@ratthing-b246.strakt.com>
	<5.2.1.1.0.20030904161323.026e5a80@pop.bluewin.ch>
	<5.2.1.1.0.20030907230643.026b5240@pop.bluewin.ch>
	<3F605D43.4030505@tismer.com> <20030915221029.GI7608@logilab.fr>
Message-ID: <3F66D967.7060608@tismer.com>

Nicolas Chauvat wrote:
...

> Ideally, we should have technical/research partners and end-user partners.
> 
> Do we have both at the moment ? Who are the end users? Having on board
> new end-users will be close to impossible in the upcoming month, but we
> can try.
> 
> By the way, do we have both small and big partners ?

I'm not sure. As I understood Laura, there are two
not so small companies interested.
That's why I'm talking about load balancing in distributed
networks, and maybe creating a tiny OS in PyPy.

ciao - chris
-- 
Christian Tismer             :^)   <mailto:tismer at tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  pager +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/



From lac at strakt.com  Tue Sep 16 12:38:49 2003
From: lac at strakt.com (Laura Creighton)
Date: Tue, 16 Sep 2003 12:38:49 +0200
Subject: [pypy-dev] Extended goals 
In-Reply-To: Message from Christian Tismer <tismer@tismer.com> 
   of "Tue, 16 Sep 2003 11:35:35 +0200." <3F66D967.7060608@tismer.com> 
References: <5.2.1.1.0.20030904161323.026e5a80@pop.bluewin.ch>
	<200309041307.h84D7rQb029849@ratthing-b246.strakt.com>
	<5.2.1.1.0.20030904161323.026e5a80@pop.bluewin.ch>
	<5.2.1.1.0.20030907230643.026b5240@pop.bluewin.ch>
	<3F605D43.4030505@tismer.com> <20030915221029.GI7608@logilab.fr>
	<3F66D967.7060608@tismer.com> 
Message-ID: <200309161038.h8GAcnq3022017@ratthing-b246.strakt.com>

In a message of Tue, 16 Sep 2003 11:35:35 +0200, Christian Tismer writes:
>Nicolas Chauvat wrote:
>...
>
>> Ideally, we should have technical/research partners and end-user partne
>rs.
>> 
>> Do we have both at the moment ? Who are the end users? Having on board
>> new end-users will be close to impossible in the upcoming month, but we
>> can try.
>> 
>> By the way, do we have both small and big partners ?
>
>I'm not sure. As I understood Laura, there are two
>not so small companies interested.
>That's why I'm talking about load balancing in distributed
>networks, and maybe creating a tiny OS in PyPy.
>
>ciao - chris
>-- 

They are interested but we cannot count on them.  3 months is hellishly
fast for them to make a decision.  (And you wonder why SMEs generate
the most wealth for a nation).

Laura

From gherman at darwin.in-berlin.de  Tue Sep 16 13:08:09 2003
From: gherman at darwin.in-berlin.de (Dinu Gherman)
Date: Tue, 16 Sep 2003 13:08:09 +0200
Subject: [pypy-dev] Extended goals
In-Reply-To: <3F66D967.7060608@tismer.com>
Message-ID: <0DE9BCFA-E836-11D7-8C5C-00039345C610@darwin.in-berlin.de>

Christian Tismer:

> I'm not sure. As I understood Laura, there are two
> not so small companies interested.
> That's why I'm talking about load balancing in distributed
> networks, and maybe creating a tiny OS in PyPy.

I just tried having a quick loook at this to get some orientation,
but then decided it'll take too long. Therefore I'm adding my two
cents without checking if they are already listed.

When I hear "embedded devices" I can't help but subclass this with
"mobile devices" (embedded in my pocket) where I used to work for
a while and which is a huge market that has become tighter in the
recent years. Still it's always looking for the *ultimate* appli-
cation everybody wants to have (for a killer application it would
need some kind of an array of nano-tube machine guns...).

So what? Recently, I've been on the IFA fair in Berlin (consumer
electronics) and it seems like mobile phones are there and waiting
to be programmed by script-kiddies (which should be quite useful
at school). The only challenge for them to face is learning J2ME,
a 'castrated' version of Java to write so-called Midlets for such
devices [1]_.

It should be clear what I'm trying to say... which is why I think
that adding "supporting mobile devices" explicitly to the list of
goals might be worth it, since this is very eye-catching.

Who knows, maybe we'll be cheerfully using services provided by
mobile Python agents via UMTS, soon (if it doesn't die before that
can happen...)?

Dinu

.. [1] http://www.forum.nokia.com/nds_for_j2me.html

--
Dinu C. Gherman
......................................................................
"The whole point of brainwashing, is that those being brainwashed
don't know it." (Graham Haley)


From gherman at darwin.in-berlin.de  Tue Sep 16 13:24:12 2003
From: gherman at darwin.in-berlin.de (Dinu Gherman)
Date: Tue, 16 Sep 2003 13:24:12 +0200
Subject: [pypy-dev] Extended goals
In-Reply-To: <3F66D967.7060608@tismer.com>
Message-ID: <4C1FF457-E838-11D7-8C5C-00039345C610@darwin.in-berlin.de>

I forgot to mention: if you really want EU money, you should, under
all circumstances, find a good explanation for how PyPy only can
fuel the world's most sophisticated future E-Government platforms!

No joke! Combine this with the mobile devices I mentioned before! ;-)

Dinu

--
Dinu C. Gherman
......................................................................
"Any sufficiently advanced technology is indistinguishable
from magic." (Arthur C. Clarke)


From hpk at trillke.net  Tue Sep 16 13:46:26 2003
From: hpk at trillke.net (holger krekel)
Date: Tue, 16 Sep 2003 13:46:26 +0200
Subject: [pypy-dev] Extended goals
In-Reply-To: <4C1FF457-E838-11D7-8C5C-00039345C610@darwin.in-berlin.de>;
	from gherman@darwin.in-berlin.de on Tue, Sep 16, 2003 at 01:24:12PM +0200
References: <3F66D967.7060608@tismer.com>
	<4C1FF457-E838-11D7-8C5C-00039345C610@darwin.in-berlin.de>
Message-ID: <20030916134626.O12635@prim.han.de>

[Dinu Gherman Tue, Sep 16, 2003 at 01:24:12PM +0200]
> I forgot to mention: if you really want EU money, you should, under
> all circumstances, find a good explanation for how PyPy only can
> fuel the world's most sophisticated future E-Government platforms!
> 
> No joke! Combine this with the mobile devices I mentioned before! ;-)

And here we should mention Zope (and it's speed problems :-).  Maybe
we/i can get some prominent Zope-developers to state that PyPy 
could help Zope etc.  At least in germany government agencies 
are using Zope a lot ... What do you think?

cheers,

    holger

From lac at strakt.com  Tue Sep 16 14:39:39 2003
From: lac at strakt.com (Laura Creighton)
Date: Tue, 16 Sep 2003 14:39:39 +0200
Subject: [pypy-dev] Extended goals 
In-Reply-To: Message from Dinu Gherman <gherman@darwin.in-berlin.de> 
	of "Tue, 16 Sep 2003 13:24:12 +0200."
	<4C1FF457-E838-11D7-8C5C-00039345C610@darwin.in-berlin.de> 
References: <4C1FF457-E838-11D7-8C5C-00039345C610@darwin.in-berlin.de> 
Message-ID: <200309161239.h8GCddq3022482@ratthing-b246.strakt.com>

In a message of Tue, 16 Sep 2003 13:24:12 +0200, Dinu Gherman writes:
>I forgot to mention: if you really want EU money, you should, under
>all circumstances, find a good explanation for how PyPy only can
>fuel the world's most sophisticated future E-Government platforms!
>
>No joke! Combine this with the mobile devices I mentioned before! ;-)
>
>Dinu

Well, given that next month AB Strakt is announcing at the scandianavian
government conference -- ITOS, where we are the gold sponsor -- 
http://www.itos.se/  a total procurement system for the EU,  and we
want PyPy, in part, for reasons of scalability, that shouldn't be
too hard.  I can drown them in the marketing materials we are making
for the conference ....

Laura

From lac at strakt.com  Tue Sep 16 14:45:10 2003
From: lac at strakt.com (Laura Creighton)
Date: Tue, 16 Sep 2003 14:45:10 +0200
Subject: [pypy-dev] Extended goals 
In-Reply-To: Message from holger krekel <hpk@trillke.net> 
   of "Tue, 16 Sep 2003 13:46:26 +0200." <20030916134626.O12635@prim.han.de> 
References: <3F66D967.7060608@tismer.com>
	<4C1FF457-E838-11D7-8C5C-00039345C610@darwin.in-berlin.de>
	<20030916134626.O12635@prim.han.de> 
Message-ID: <200309161245.h8GCjAq3022581@ratthing-b246.strakt.com>

In a message of Tue, 16 Sep 2003 13:46:26 +0200, holger krekel writes:
>[Dinu Gherman Tue, Sep 16, 2003 at 01:24:12PM +0200]
>> I forgot to mention: if you really want EU money, you should, under
>> all circumstances, find a good explanation for how PyPy only can
>> fuel the world's most sophisticated future E-Government platforms!
>> 
>> No joke! Combine this with the mobile devices I mentioned before! ;-)
>
>And here we should mention Zope (and it's speed problems :-).  Maybe
>we/i can get some prominent Zope-developers to state that PyPy 
>could help Zope etc.  At least in germany government agencies 
>are using Zope a lot ... What do you think?
>
>cheers,
>
>    holger

Good idea. I will ask Paul Everitt as well.

Laura


From tismer at tismer.com  Tue Sep 16 15:20:59 2003
From: tismer at tismer.com (Christian Tismer)
Date: Tue, 16 Sep 2003 15:20:59 +0200
Subject: [pypy-dev] Extended goals
In-Reply-To: <200309161245.h8GCjAq3022581@ratthing-b246.strakt.com>
References: <3F66D967.7060608@tismer.com>
	<4C1FF457-E838-11D7-8C5C-00039345C610@darwin.in-berlin.de>
	<20030916134626.O12635@prim.han.de>
	<200309161245.h8GCjAq3022581@ratthing-b246.strakt.com>
Message-ID: <3F670E3B.8050509@tismer.com>

Laura Creighton wrote:

> In a message of Tue, 16 Sep 2003 13:46:26 +0200, holger krekel writes:

[Zope speed]

> Good idea. I will ask Paul Everitt as well.

Absolutely!
Zope would not only become much faster, but
could also get rid of *all its C extensions*
(which are not too many, btw).
-- 
Christian Tismer             :^)   <mailto:tismer at tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  pager +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/



From arigo at tunes.org  Tue Sep 16 16:22:26 2003
From: arigo at tunes.org (Armin Rigo)
Date: Tue, 16 Sep 2003 15:22:26 +0100
Subject: [pypy-dev] HELP! I can't do svn up!
In-Reply-To: <200309141016.h8EAGWq3010942@ratthing-b246.strakt.com>
References: <200309140835.h8E8ZK3c010604@ratthing-b246.strakt.com>
	<20030914112313.Y12635@prim.han.de>
	<200309141016.h8EAGWq3010942@ratthing-b246.strakt.com>
Message-ID: <20030916142226.GA3390@vicky.ecs.soton.ac.uk>

Hello Laura,

On Sun, Sep 14, 2003 at 12:16:32PM +0200, Laura Creighton wrote:
> >> Fetching external item into trunk/www/service/vpath
> >> svn: RA layer request failed
> >> svn: REPORT response handling failed to complete the editor drive

I have the same problem for possibly different reasons (I'm behind a firewall 
which prevents me from accessing codespeak.net on port 80 with svn, so I'm 
using port 8080, and the "external item" link seem to be pointing to port 80). 
Note however that this error doesn't prevent the "svn up" from completing. 
It's fine, you just don't have that "external item" subbranch in your working 
copy.


A bientot,

Armin.


From hpk at trillke.net  Tue Sep 16 16:31:45 2003
From: hpk at trillke.net (holger krekel)
Date: Tue, 16 Sep 2003 16:31:45 +0200
Subject: [pypy-dev] HELP! I can't do svn up!
In-Reply-To: <20030916142226.GA3390@vicky.ecs.soton.ac.uk>;
	from arigo@tunes.org on Tue, Sep 16, 2003 at 03:22:26PM +0100
References: <200309140835.h8E8ZK3c010604@ratthing-b246.strakt.com>
	<20030914112313.Y12635@prim.han.de>
	<200309141016.h8EAGWq3010942@ratthing-b246.strakt.com>
	<20030916142226.GA3390@vicky.ecs.soton.ac.uk>
Message-ID: <20030916163145.R12635@prim.han.de>

[Armin Rigo Tue, Sep 16, 2003 at 03:22:26PM +0100]
> Hello Laura,
> 
> On Sun, Sep 14, 2003 at 12:16:32PM +0200, Laura Creighton wrote:
> > >> Fetching external item into trunk/www/service/vpath
> > >> svn: RA layer request failed
> > >> svn: REPORT response handling failed to complete the editor drive
> 
> I have the same problem for possibly different reasons (I'm behind a firewall 
> which prevents me from accessing codespeak.net on port 80 with svn, so I'm 
> using port 8080, and the "external item" link seem to be pointing to port 80). 
> Note however that this error doesn't prevent the "svn up" from completing. 
> It's fine, you just don't have that "external item" subbranch in your working 
> copy.

Probably it's not only a firewall but a http-proxy and those tend to not
understand DAV/DELTAV although it's a standard protocol over http. Maybe we can 
try to completly mirror 80 to 8080 (might be a task for Jum during the sprint)
as a workaround. 

It's not a big problem because currently only the www-part of pypy 
uses external items.  I hope to give a short talk during the BerlinSprint why
svn's external items are incredibly useful, btw.  

cheers,

    holger

From pypy-issues at codespeak.net  Tue Sep 16 16:38:04 2003
From: pypy-issues at codespeak.net (holger krekel)
Date: Tue, 16 Sep 2003 14:38:04 +0000
Subject: [pypy-dev] [issue28] proxy codespeak.net:80 to codespeak.net:8080
Message-ID: <1063723084.35.0.860930864508.issue@codespeak.net>


New submission from holger krekel <hpk at trillke.net>:

It seems that people with a (transparent) http-proxy 
cannot connect properly via DAV/DELTAV to subversion. We 
might want to *proxy* codespeak.net:8080 to 
codspeak.net:80 (with url-translation) to keep the 
repository accessible for these people.


----------
messages: 63
nosy: hpk, jum
priority: feature
status: unread
title: proxy codespeak.net:80 to codespeak.net:8080
__________________________________________________
PyPython issue tracker <pypy-issues at codespeak.net>
http://codespeak.net/issues/pypy/issue28
__________________________________________________

From bob at redivi.com  Tue Sep 16 16:41:24 2003
From: bob at redivi.com (Bob Ippolito)
Date: Tue, 16 Sep 2003 10:41:24 -0400
Subject: [pypy-dev] Extended goals
In-Reply-To: <3F670E3B.8050509@tismer.com>
Message-ID: <D8AC753A-E853-11D7-A81E-000A95686CD8@redivi.com>


On Tuesday, Sep 16, 2003, at 09:20 America/New_York, Christian Tismer 
wrote:

> Laura Creighton wrote:
>
>> In a message of Tue, 16 Sep 2003 13:46:26 +0200, holger krekel writes:
>
> [Zope speed]
>
>> Good idea. I will ask Paul Everitt as well.
>
> Absolutely!
> Zope would not only become much faster, but
> could also get rid of *all its C extensions*
> (which are not too many, btw).

Does Zope really need any C extensions to do what it needs to do (not 
considering performance) if they were allowed to use new style classes? 
  I was under the impression that the C extension(s) in Zope were 
developed to get around some of the shortfalls of the old class system, 
and was part of the reason for type/class unification?

-bob


From tismer at tismer.com  Tue Sep 16 17:02:24 2003
From: tismer at tismer.com (Christian Tismer)
Date: Tue, 16 Sep 2003 17:02:24 +0200
Subject: [pypy-dev] Extended goals
In-Reply-To: <D8AC753A-E853-11D7-A81E-000A95686CD8@redivi.com>
References: <D8AC753A-E853-11D7-A81E-000A95686CD8@redivi.com>
Message-ID: <3F672600.2000202@tismer.com>

Bob Ippolito wrote:

> 
> On Tuesday, Sep 16, 2003, at 09:20 America/New_York, Christian Tismer 
> wrote:

...

>> Zope would not only become much faster, but
>> could also get rid of *all its C extensions*
>> (which are not too many, btw).
> 
> 
> Does Zope really need any C extensions to do what it needs to do (not 
> considering performance) if they were allowed to use new style classes? 
>  I was under the impression that the C extension(s) in Zope were 
> developed to get around some of the shortfalls of the old class system, 
> and was part of the reason for type/class unification?

Well, I think they should be able to trash ExtensionClass,
but they have a bit more:

D:\pypy\trunk\doc\funding>dir D:\Zope-2.7.0-b2\lib\python\*.pyd
  Datentr?ger in Laufwerk D: ist Daten
  Datentr?gernummer: 8838-4458

  Verzeichnis von D:\Zope-2.7.0-b2\lib\python

26.08.2003  14:36               28,672 Acquisition.pyd
26.08.2003  14:36               28,672 BTree.pyd
26.08.2003  14:36               20,480 ComputedAttribute.pyd
26.08.2003  14:36               61,440 ExtensionClass.pyd
26.08.2003  14:36               28,672 IIBTree.pyd
26.08.2003  14:37               20,480 initgroups.pyd
26.08.2003  14:36               20,480 intSet.pyd
26.08.2003  14:36               28,672 IOBTree.pyd
26.08.2003  14:36               20,480 MethodObject.pyd
26.08.2003  14:36               20,480 Missing.pyd
26.08.2003  14:36               20,480 MultiMapping.pyd
26.08.2003  14:36               28,672 OIBTree.pyd
26.08.2003  14:36               20,480 Record.pyd
26.08.2003  14:36               20,480 ThreadLock.pyd
               14 Datei(en)        368,640 Bytes
                0 Verzeichnis(se),     267,665,408 Bytes frei

ciao - chris
-- 
Christian Tismer             :^)   <mailto:tismer at tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  pager +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/



From hpk at trillke.net  Tue Sep 16 17:19:41 2003
From: hpk at trillke.net (holger krekel)
Date: Tue, 16 Sep 2003 17:19:41 +0200
Subject: [pypy-dev] Extended goals
In-Reply-To: <3F672600.2000202@tismer.com>;
	from tismer@tismer.com on Tue, Sep 16, 2003 at 05:02:24PM +0200
References: <D8AC753A-E853-11D7-A81E-000A95686CD8@redivi.com>
	<3F672600.2000202@tismer.com>
Message-ID: <20030916171941.S12635@prim.han.de>

[Christian Tismer Tue, Sep 16, 2003 at 05:02:24PM +0200]
> 26.08.2003  14:36               28,672 Acquisition.pyd
> 26.08.2003  14:36               28,672 BTree.pyd
> 26.08.2003  14:36               20,480 ComputedAttribute.pyd
> 26.08.2003  14:36               61,440 ExtensionClass.pyd
> 26.08.2003  14:36               28,672 IIBTree.pyd
> 26.08.2003  14:37               20,480 initgroups.pyd
> 26.08.2003  14:36               20,480 intSet.pyd
> 26.08.2003  14:36               28,672 IOBTree.pyd
> 26.08.2003  14:36               20,480 MethodObject.pyd
> 26.08.2003  14:36               20,480 Missing.pyd
> 26.08.2003  14:36               20,480 MultiMapping.pyd
> 26.08.2003  14:36               28,672 OIBTree.pyd
> 26.08.2003  14:36               20,480 Record.pyd
> 26.08.2003  14:36               20,480 ThreadLock.pyd

btw, i forgot all about it: this is mainly the Zope-Database (ZODB) 
which is independent from Zope and has quite some parts written 
in C, for performance reasons ASFAIK.  I'd say they are
applying a common technique with high-level-languages:
rapidly prototype in Python and reimplement the 
speed-relevant parts in C (and maintain it).  
It's rather obvious how PyPy could help here :-)

    holger

From tismer at tismer.com  Tue Sep 16 17:46:55 2003
From: tismer at tismer.com (Christian Tismer)
Date: Tue, 16 Sep 2003 17:46:55 +0200
Subject: [pypy-dev] Extended goals
In-Reply-To: <20030916171941.S12635@prim.han.de>
References: <D8AC753A-E853-11D7-A81E-000A95686CD8@redivi.com>
	<3F672600.2000202@tismer.com> <20030916171941.S12635@prim.han.de>
Message-ID: <3F67306F.10704@tismer.com>

holger krekel wrote:

> [Christian Tismer Tue, Sep 16, 2003 at 05:02:24PM +0200]

[many more binaries than expected]

> btw, i forgot all about it: this is mainly the Zope-Database (ZODB) 
> which is independent from Zope and has quite some parts written 
> in C, for performance reasons ASFAIK.  I'd say they are
> applying a common technique with high-level-languages:
> rapidly prototype in Python and reimplement the 
> speed-relevant parts in C (and maintain it).  
> It's rather obvious how PyPy could help here :-)

Absolutely. Zope can reverse its path and move
its core back to where it belongs: Python.

This is what is going to happen if we really
get the project funded, and a few of us might
work almost fulltime on PyPy:
All C extensions can probably envision their
death, sooner or later.
And this is such a huge achievement, since the
amount of man hours saved in the future
is countable infinite. :-)

Probably a selling argument which should get
into pypy-funding...

ciao - chris

-- 
Christian Tismer             :^)   <mailto:tismer at tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  pager +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/



From arigo at tunes.org  Wed Sep 17 22:51:29 2003
From: arigo at tunes.org (Armin Rigo)
Date: Wed, 17 Sep 2003 21:51:29 +0100
Subject: [pypy-dev] the new shining Builtinrefactor branch :-)
Message-ID: <20030917205129.GA7690@vicky.ecs.soton.ac.uk>

Hello everybody !

The 'builtinrefactor' branch that Holger and Armin(*) have been working on is
finally stable again. We are merging it back with the trunk. (This only
affects the src/pypy subdirectory.) Take yourself a cup of tea and read on :-)

(*) this is a join e-mail written in a common 'screen' session :-)

The main change is a rewrite of the interactions between interpreter-
and app-level code with the following design goals:

- allow to freely "mix" app-level and interp-level code in one file
  (thus no need to have xxx_app.py files anymore)

- implement functions and modules uniformly for all object spaces 

- provide mechanisms (interpreter/gateway.py) for transparently 

    - calling app-level code from interp-level code 

    - calling interp-level code from app-level code 

    - accessing interp-level defined attributes from app-level 

- argument parsing is now at interpreter level to once and for all get 
  rid of bootstrapping and debugging nightmares :-)

- reorganized the Code and Frame classes and introduced subclasses of these
  to reduce redunant code dealing with those. 

Here is a more detailled description of the changes.


Application-Level and Interpreter-Level Interaction
---------------------------------------------------

Part one: invoke app-level code from interpreter level
------------------------------------------------------

We no longer need 'xxx_app.py' files for helpers for 'xxx.py'. An app-level
helper function can be written in-line in the source. See for example the
function normalize_exception() in interpreter/pyframe.py:

def app_normalize_exception(etype, evalue):
    ...plain Python app-level code...
normalize_exception = gateway.app2interp(app_normalize_exception)

This makes 'normalize_exception' callable from interpreter-level as if it had
been defined at interpreter-level with the following signature:

def normalize_exception(space, w_etype, w_evalue):
    ...

App-level helpers can also be used as methods. In pyopcode.py,
app_prepare_exec() is the app-level definition of prepare_exec(), a method of
the new class PyInterpFrame (more about it below).

All these helpers can be called normally from the rest of the interpreter
code. Global functions must be called with an extra first argument, 'space'.
For methods the space is read from 'self.space'. All other arguments must be
wrapped objects, and so is the result.

If you have many 'app_*' functions you can register them "en masse" via a call
to

    gateway.importall(globals())   # app_xxx() -> xxx()

For other examples, see objspace/std/dicttype.py, which contains the code that
used to be in the separae dictobject_app.py. Note how all the functions are
defined with app_ prefixes, and then gateway.importall() is used to make the
non-app_ interpreter-level-callable gateways, and finally register_all()
registers the latter into the multimethod tables.

Part two: make interpreter-level objects accessible from app-level 
------------------------------------------------------------------

Conversely, 'gateway' contains code to allow interpreter-level functions to be
visible from app-level. This can be done manually with gateway.interp2app(),
but most of the time it is done for you by the ExtModule base class. See
module/builtin.py:

class __builtin__(ExtModule):
    ...

Although such extension-modules are defined as classes there should only be
one instance per object space.  We are not only reusing the class-statement to
provide a regular way of defining modules that contain code and attributes
defined both at app-level and interp-level. Actually we need them to be a
class because there might be several object spaces alive around there. 

Now the new module/builtin.py contains interp-level code like globals(),
locals(), __import__()..., and app-level code like app_execfile(),
app_range()...  The ExtModule parent class will make sure all these appear on
the instance, both at app-level and at interp-level. The app-level will see an
ExtModule-instance as a plain module. It is better to start with the
interpreter-level view to see how this works. 

You can call all these methods from *interpreter level*; for example, in
'def __import__(self, ...)' we do:

   self.execfile(space.wrap(filename), w_globals, w_locals)

but execfile() is actually defined like so:

   def app_execfile(self, filename, glob=None, loc=None):
	...

and because this is an app-level definition all the arguments are of course
wrapped (we are executing app_execfile() at app-level and the body of
app_execfile refers to the parameters). Please note that app_* functions must
always be called with wrapped arguments even though you don't see a 'w_'
prefix. 
 
Of course you can also call 'execfile' from outside the instance methods, e.g. 
you can do

     space.builtin.execfile(space.wrap('somefile.py'), w_globals, w_locals)

One more thing is interesting to discuss here. Methods like app_execfile()
have a 'self' argument but what does it mean in the context of a module?  The
bottom line is that this provides an explicit way to access objects in the
same module (instance):

    class somemodule(ExtModule):
        def app_y(self):
            ...

        def app_x(self):
            r = self.y()
    
Calling 'y()' directly wouldn't usually work because this would try to access
the globals of the CPython-module where 'somemodule' is defined.  This is
unrelated to the "globals namespace" of the app-level module, which is (behind
the scene) implemented as the content of the 'self' instance.  So app_x()
above is in all respects a method of a class.  The important thing to remember
here is that app-level modules are implemented as class-instances at
interpreter-level -- and the usual scoping rules apply. 

>From the user's point of view (i.e. app-level outside the definition of the
module), an expression like 'sys.displayhook' actually gets you a *bound
method* of the 'sys' instance. This is true even when accessing 'displayhook'
with other syntaxes like 'from sys import displayhook'. This is also true for
the __builtin__ module, so e.g. when you type 'len' into the pypy interpreter
you see

    <pypy.interpreter.function.Method object at 0x4032c424>

A bit weird, but it works as expected and we could later change the
representation string :-)

    >>>> import sys
    >>>> sys.displayhook
    <pypy.interpreter.function.Method object at 0xe44f7cc>
    >>>> def f(): 
    ....   pass
    .... 
    >>>> sys.f = f
    >>>> sys.f
    <pypy.interpreter.function.Function object at 0xe4ed3cc>
    >>>> 

The above should start to make sense if you really think of 'sys' as an
instance :-) And this brings us to the next change.


Functions (and friends) and Modules  moved off objectspaces
-----------------------------------------------------------
 
Function, Method, Generator, Module (and probably more) classes are now part
of the interpreter; an object space is no longer responsible for providing
them. It makes sense because these classes are straightforward structures
anyway, and almost only created and used by the interpreter (like code
objects, who where already in an interpreter-level class of their own). See
interpreter/function.py and interpreter/module.py.

An object space now needs to callback into interpreter-level objects to carry
out operations on them. This allows the interpreter-level to control its own
internal classes.  For example in interpreter/pyframe.py the following method
controls visibility of attributes of a Frame instance:

    def pypy_getattr(self, w_attr):
        attr = self.space.unwrap(w_attr)
        if attr == 'f_locals':   return self.w_locals
        if attr == 'f_globals':  return self.w_globals
        if attr == 'f_builtins': return self.w_builtins
        if attr == 'f_code':     return self.space.wrap(self.code)
        raise OperationError(self.space.w_AttributeError, w_attr)

(For reading attributes we'll probably design some better interface later :-)

Note the preliminary interface to tell the object space how the
interpreter-level objects should react to operations: if an interpreter-level
class defines methods like e.g. pypy_getattr(), pypy_call(), pypy_iter(), etc
then every objectspace is required to call those methods when it encounters a
wrap()ed interpreter-level object for the getattr(), call(), iter() operations
respectively.  When the object space must wrap() one of these objects it does
so in a special structure (for stdobjspace it is CPythonObject) which knows
that it should look for these pypy_* method names.

For example, Function objects have a pypy_call() method that is called
whenever the space.call() operation is issued with a wrapped Function as first
argument. At some point all these wrappable classes should have a
pypy_getattr() as well and we might eventually allow only subclasses of
baseobjspace.Wrappable to be wrap()ed. 

The new interface to wrap()&co that we are discussing in pypy-dev is not
implemented yet, but essentially the one remaining usage for wrap() would be
exactly that: make a "black-box" proxy that just calls back to the pypy_*
methods. It is clearly better to require that an objspace honours the 'pypy_*'
methods instead of requiring that every objectspace implements modules and
functions on its own. 


Code and Frame classes reorganized
----------------------------------

The classes involved in code execution have been reorganized a bit. There are
now two abstract base classes: Code and Frame, plus a few concrete classes:
Function, Method, and Gateway.

A Function object, like CPython's, is essentially a container for a code
object with references to a globals dict and default arguments, and possibly
other closure stuff. As it is already the case since the Gothenburg sprint, in
PyPy there is only one Function class for builtin and user-defined functions,
the difference being in the Code object. A Method object is just the same as
in CPython. And Gateway objects are what app2interp() and interp2app() return:
essentially a Function that isn't bound to any particular object space yet. It
isn't supposed to show up at app-level: whenever it gets associated with a
space, it becomes a Function.

A Code object is some structure that knows its 'signature' (what arguments it
expects), and also knows how to build a Frame to run itself in. A Frame
represents the execution of a Code. It has an abstract run() method, and a few
methods to get and set its locals as a dictionary or as a plain list ("fast
locals"). To see how this fits together, see the method call() of Function
objects in function.py:

    def call(self, w_args, w_kwds=None):
        scope_w = self.parse_args(w_args, w_kwds)
        frame = self.func_code.create_frame(self.space, self.w_globals,
                                            self.closure)
        frame.setfastscope(scope_w)
        return frame.run()

Argument parsing is done in parse_args() according to
self.func_code.signature(); then the Code is asked to create a Frame; the
Frame is sent the decoded list of arguments to initialize its locals; and then
it is run.

The other interface to running code is the method exec_code() of Code in
eval.py, which creates a frame and sets its locals as a dictionary before
running it:

    def exec_code(self, space, w_globals, w_locals):
        "Implements the 'exec' statement."
        frame = self.create_frame(space, w_globals)
        frame.setdictscope(w_locals)
        return frame.run()

PyCode is a subclass of Code with all the co_xxx attributes of regular CPython
code objects. Quite expectedly, it runs in a PyFrame, which is like a regular
CPython frame. Unexpectedly, however, there are now several subclasses of
PyFrame, depending on what particular opcodes we need.

* The regular one is PyInterpFrame in pyopcode.py. All opcode
  definitions are now methods of this class instead of being global
  functions.

* There is also PyNestedScopeFrame in nestedscope.py, which adds a few
  more opcodes, namely the ones related to nested scopes.

* And there is GeneratorFrame in generator.py which overrides
  RETURN_VALUE and defines YIELD_VALUE to provide generator-like behavior.

The PyCode class knows which frame to create by inspecting its co_xxx
attributes (co_flags tells if we are a generator, and co_cellvars/co_freevars
are empty unless we need nested scopes).  The nice thing is that apart from
these checks, everything about nested scopes is in nestedscope.py, and
everything about generators is in generator.py. Contrast this with CPython's
huge ceval.c file which mixes all these features throughout the code.

Later on we plan to extend that model even more to allow more flexibility,
like custom opcodes. 

There are other, simpler subclasses of Code with their corresponding
subclasses of Frame:

* in gateway.py, BuiltinCode and BuiltinFrame just call back to an
  interpreter-level function. This is the code object found in Functions
  created out of Gateways built by gateway.interp2app(). Such Functions
  are the equivalent of CPython's built-in functions. Note that
  BuiltinCode, like all Code subclasses, exposes a signature for Function
  to be able to decode arguments. Thus, when a built-in function is 
  called: 

  (1) the arguments are parsed in the normal way (e.g. passing
      arguments by keyword is always allowed) and turned into a "fast locals"
      list;
 
  (2) a BuiltinFrame is created and assigned the "fast locals"; 

  (3) the BuiltinFrame.run() method is called, which just calls the
      interpreter-level function using the "fast locals" list as arguments.

* in objspace/std/typeobject.py, MultimethodCode and a few XxxMmFrame
  classes implement multimethod-calls. This is the type of the Code
  objects that you would see by typing '[].append.im_func.func_code'
  (expect that 'im_func' isn't implemented right now).


Enjoy !

Armin & Holger
Holger & Armin
whatever


From faassen at vet.uu.nl  Fri Sep 19 11:58:40 2003
From: faassen at vet.uu.nl (Martijn Faassen)
Date: Fri, 19 Sep 2003 11:58:40 +0200
Subject: [pypy-dev] Extended goals
In-Reply-To: <20030916134626.O12635@prim.han.de>
References: <3F66D967.7060608@tismer.com>
	<4C1FF457-E838-11D7-8C5C-00039345C610@darwin.in-berlin.de>
	<20030916134626.O12635@prim.han.de>
Message-ID: <20030919095839.GA14612@vet.uu.nl>

holger krekel wrote:
> And here we should mention Zope (and it's speed problems :-).  Maybe
> we/i can get some prominent Zope-developers to state that PyPy 
> could help Zope etc.  At least in germany government agencies 
> are using Zope a lot ... What do you think?

There also seems to be quite a bit of government-level Zope use in
France (Nuxeo comes to mind).

Anyway, for what it's worth:

"PyPy could help Zope."

:)

Regards,

Martijn


From faassen at vet.uu.nl  Fri Sep 19 12:00:51 2003
From: faassen at vet.uu.nl (Martijn Faassen)
Date: Fri, 19 Sep 2003 12:00:51 +0200
Subject: [pypy-dev] Extended goals
In-Reply-To: <D8AC753A-E853-11D7-A81E-000A95686CD8@redivi.com>
References: <3F670E3B.8050509@tismer.com>
	<D8AC753A-E853-11D7-A81E-000A95686CD8@redivi.com>
Message-ID: <20030919100051.GB14612@vet.uu.nl>

Bob Ippolito wrote:
> Does Zope really need any C extensions to do what it needs to do (not 
> considering performance) if they were allowed to use new style classes? 
>  I was under the impression that the C extension(s) in Zope were 
> developed to get around some of the shortfalls of the old class system, 
> and was part of the reason for type/class unification?

In Zope 3 which can use new style classes we still see some C-extensions,
but I imagine those are there mostly for performance reasons. I'm not sure
about that though; I'll ask around.

Regards,

Martijn


From hpk at trillke.net  Fri Sep 19 12:09:13 2003
From: hpk at trillke.net (holger krekel)
Date: Fri, 19 Sep 2003 12:09:13 +0200
Subject: [pypy-dev] Extended goals
In-Reply-To: <20030919095839.GA14612@vet.uu.nl>;
	from faassen@vet.uu.nl on Fri, Sep 19, 2003 at 11:58:40AM +0200
References: <3F66D967.7060608@tismer.com>
	<4C1FF457-E838-11D7-8C5C-00039345C610@darwin.in-berlin.de>
	<20030916134626.O12635@prim.han.de> <20030919095839.GA14612@vet.uu.nl>
Message-ID: <20030919120913.F12635@prim.han.de>

[Martijn Faassen Fri, Sep 19, 2003 at 11:58:40AM +0200]
> holger krekel wrote:
> > And here we should mention Zope (and it's speed problems :-).  Maybe
> > we/i can get some prominent Zope-developers to state that PyPy 
> > could help Zope etc.  At least in germany government agencies 
> > are using Zope a lot ... What do you think?
> 
> There also seems to be quite a bit of government-level Zope use in
> France (Nuxeo comes to mind).
> 
> Anyway, for what it's worth:
> 
> "PyPy could help Zope."
> 
> :)

We'll elaborate that a bit when i am in R'dam next week :-)

cheers,

    holger

From lac at strakt.com  Sat Sep 20 16:38:36 2003
From: lac at strakt.com (Laura Creighton)
Date: Sat, 20 Sep 2003 16:38:36 +0200
Subject: [pypy-dev] anybody have an email addr for Michael Hudson that isn't
Message-ID: <200309201438.h8KEcaEk016810@ratthing-b246.strakt.com>

on the Starship?  I want to ask him something and I can't mail him.

Laura

From gherman at darwin.in-berlin.de  Sat Sep 20 18:45:14 2003
From: gherman at darwin.in-berlin.de (Dinu Gherman)
Date: Sat, 20 Sep 2003 18:45:14 +0200
Subject: [pypy-dev] anybody have an email addr for Michael Hudson that
	isn't
In-Reply-To: <200309201438.h8KEcaEk016810@ratthing-b246.strakt.com>
Message-ID: <CEEE453B-EB89-11D7-B583-00039345C610@darwin.in-berlin.de>

Laura Creighton:

> on the Starship?  I want to ask him something and I can't mail him.

I was about to suggest mwh at python2.net, but gherman at python2.net
doesn't work for me either. You should see http://python2.net,
though, although this is no help for the email, sorry!

Stefan Drees told be there is some trouble with DNS routing or
whateber on Zope Corp's site. I'd really like to see that fixed
since it happens pretty often and it happens for entire days in
a row, like just now...

Christian, can you intervene somehow?

Dinu

--
Dinu C. Gherman
......................................................................
"Who controls the past, controls the future: who controls the present
controls the past." (George Orwell, 1984)


From lac at strakt.com  Sat Sep 20 18:52:06 2003
From: lac at strakt.com (Laura Creighton)
Date: Sat, 20 Sep 2003 18:52:06 +0200
Subject: [pypy-dev] anybody have an email addr for Michael Hudson that
	isn't 
In-Reply-To: Message from Dinu Gherman <gherman@darwin.in-berlin.de> 
	of "Sat, 20 Sep 2003 18:45:14 +0200."
	<CEEE453B-EB89-11D7-B583-00039345C610@darwin.in-berlin.de> 
References: <CEEE453B-EB89-11D7-B583-00039345C610@darwin.in-berlin.de> 
Message-ID: <200309201652.h8KGq6h9017450@ratthing-b246.strakt.com>

In a message of Sat, 20 Sep 2003 18:45:14 +0200, Dinu Gherman writes:
>Laura Creighton:
>
>> on the Starship?  I want to ask him something and I can't mail him.
>
>I was about to suggest mwh at python2.net, but gherman at python2.net
>doesn't work for me either. You should see http://python2.net,
>though, although this is no help for the email, sorry!
>
>Stefan Drees told be there is some trouble with DNS routing or
>whateber on Zope Corp's site. I'd really like to see that fixed
>since it happens pretty often and it happens for entire days in
>a row, like just now...
>
>Christian, can you intervene somehow?
>
>Dinu

Christian has tried, and failed.  And we have sunk somewhat more time
into this problem today than we had available, so next attempt to fix
has got to be done by somebody else.

Laura

From robin at reportlab.com  Sat Sep 20 19:52:32 2003
From: robin at reportlab.com (Robin Becker)
Date: Sat, 20 Sep 2003 18:52:32 +0100
Subject: [pypy-dev] svn error help please
Message-ID: <d9u24LAgPJb$Ewtp@jessikat.fsnet.co.uk>

I tried 
        svn up 

in my pypy tree and see this

Fetching external item into www/service/vpath
svn: RA layer request failed
svn: REPORT response handling failed to complete the editor drive.

Anyone know what it means?
-- 
Robin Becker

From lac at strakt.com  Sat Sep 20 20:01:42 2003
From: lac at strakt.com (Laura Creighton)
Date: Sat, 20 Sep 2003 20:01:42 +0200
Subject: [pypy-dev] svn error help please 
In-Reply-To: Message from Robin Becker <robin@reportlab.com> 
	of "Sat, 20 Sep 2003 18:52:32 BST."
	<d9u24LAgPJb$Ewtp@jessikat.fsnet.co.uk> 
References: <d9u24LAgPJb$Ewtp@jessikat.fsnet.co.uk> 
Message-ID: <200309201801.h8KI1gh9017799@ratthing-b246.strakt.com>

In a message of Sat, 20 Sep 2003 18:52:32 BST, Robin Becker writes:
>I tried 
>        svn up 
>
>in my pypy tree and see this
>
>Fetching external item into www/service/vpath
>svn: RA layer request failed
>svn: REPORT response handling failed to complete the editor drive.
>
>Anyone know what it means?
>-- 
>Robin Becker

Armin and I are getting it as well.  It isn't stopping us from
getting the sources.  Armin says his is because he is behind a
firewall.  Mine is 'just because'.

Laura

From hpk at trillke.net  Sat Sep 20 20:17:25 2003
From: hpk at trillke.net (holger krekel)
Date: Sat, 20 Sep 2003 20:17:25 +0200
Subject: [pypy-dev] svn error help please
In-Reply-To: <d9u24LAgPJb$Ewtp@jessikat.fsnet.co.uk>;
	from robin@reportlab.com on Sat, Sep 20, 2003 at 06:52:32PM +0100
References: <d9u24LAgPJb$Ewtp@jessikat.fsnet.co.uk>
Message-ID: <20030920201725.Y12635@prim.han.de>

[Robin Becker Sat, Sep 20, 2003 at 06:52:32PM +0100]
> I tried 
>         svn up 
> 
> in my pypy tree and see this
> 
> Fetching external item into www/service/vpath
> svn: RA layer request failed
> svn: REPORT response handling failed to complete the editor drive.
> 
> Anyone know what it means?

There are two possible causes i know off:

1. wrong subversion version, 0.28 and 0.29 work and 
   can be downloaded from

  http://codespeak.net/pypy/index.cgi?doc/devel/howtosvn.html

2. a (transparent) http-proxy which is not able to 
   proxy WEBDAV requests.  In this case pointing your 
   browser to http://codespeak.net:8080/svn/... 
   might help. 

I guess it's 1. in your case. 

cheers,

    holger

From robin at reportlab.com  Sat Sep 20 20:20:30 2003
From: robin at reportlab.com (Robin Becker)
Date: Sat, 20 Sep 2003 19:20:30 +0100
Subject: [pypy-dev] svn error help please
In-Reply-To: <20030920201725.Y12635@prim.han.de>
References: <d9u24LAgPJb$Ewtp@jessikat.fsnet.co.uk>
 <20030920201725.Y12635@prim.han.de>
Message-ID: <8d9+AXAupJb$Ewrk@jessikat.fsnet.co.uk>

In article <20030920201725.Y12635 at prim.han.de>, holger krekel
<hpk at trillke.net> writes
>[Robin Becker Sat, Sep 20, 2003 at 06:52:32PM +0100]

I'll upgrade to 0.29 and see what happens.

>> Fetching external item into www/service/vpath
>> svn: RA layer request failed
>> svn: REPORT response handling failed to complete the editor drive.
>> 
>> Anyone know what it means?
>
>There are two possible causes i know off:
>
>1. wrong subversion version, 0.28 and 0.29 work and 
>   can be downloaded from
>
>  http://codespeak.net/pypy/index.cgi?doc/devel/howtosvn.html
>
>2. a (transparent) http-proxy which is not able to 
>   proxy WEBDAV requests.  In this case pointing your 
>   browser to http://codespeak.net:8080/svn/... 
>   might help. 
>
>I guess it's 1. in your case. 
>
>cheers,
>
>    holger
>

-- 
Robin Becker

From robin at reportlab.com  Sat Sep 20 20:39:17 2003
From: robin at reportlab.com (Robin Becker)
Date: Sat, 20 Sep 2003 19:39:17 +0100
Subject: [pypy-dev] svn error help please
In-Reply-To: <20030920201725.Y12635@prim.han.de>
References: <d9u24LAgPJb$Ewtp@jessikat.fsnet.co.uk>
 <20030920201725.Y12635@prim.han.de>
Message-ID: <jtD5sgAV7Jb$Ewqa@jessikat.fsnet.co.uk>

In article <20030920201725.Y12635 at prim.han.de>, holger krekel
<hpk at trillke.net> writes
>There are two possible causes i know off:
>
>1. wrong subversion version, 0.28 and 0.29 work and 
>   can be downloaded from
>
>  http://codespeak.net/pypy/index.cgi?doc/devel/howtosvn.html
>
>2. a (transparent) http-proxy which is not able to 
>   proxy WEBDAV requests.  In this case pointing your 
>   browser to http://codespeak.net:8080/svn/... 
>   might help. 
>
>I guess it's 1. in your case. 
>
>cheers,
>
>    holger
>
you were right 0.29 fixed it.
-- 
Robin Becker

From reply at WindowFlower.com  Sun Sep 21 15:04:56 2003
From: reply at WindowFlower.com (Window Flower)
Date: Sun, 21 Sep 2003 21:04:56 +0800
Subject: [pypy-dev] The Magic of Chinese Folk Art - Paper-cutting!
Message-ID: <20030921130258.E69BC5A271@thoth.codespeak.net>

An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20030921/4b8dd31b/attachment.html>

From arigo at tunes.org  Mon Sep 22 15:08:45 2003
From: arigo at tunes.org (Armin Rigo)
Date: Mon, 22 Sep 2003 14:08:45 +0100
Subject: [pypy-dev] Re: fixeol
In-Reply-To: <3F6B081B.80001@tismer.com>
References: <3F673C08.9060502@tismer.com>
	<20030917125138.GA24502@vicky.ecs.soton.ac.uk> <3F6AF99F.9030602@tismer.com>
	<20030919130501.GA7823@vicky.ecs.soton.ac.uk> <3F6B081B.80001@tismer.com>
Message-ID: <20030922130845.GA5469@vicky.ecs.soton.ac.uk>

Hello Christian,

> >Argh ! The files look completely broken now !! Warning !

I hopefully fixed the script now. You might try it (carefully!) on some new
text files.

I'm still a bit worried that if fixeol actually changes content, as would be
the case if it is run by someone on another platform than the one used to
create it, then svn will record it as completely changed (ideal source for
conflicts). The only solution I could imagine would be to make sure that
people specify files to be text when they first add it to the repository. 
Somehow the default "files-are-binary" of SVN creates different, new kind of
problems than the default "files-are-text" of CVS. Maybe a no-default
you-must-specify-it-explicitely wouldn't have been such a bad idea.


A bientot,

Armin.


From hpk at trillke.net  Mon Sep 22 15:15:10 2003
From: hpk at trillke.net (holger krekel)
Date: Mon, 22 Sep 2003 15:15:10 +0200
Subject: [pypy-dev] Re: fixeol
In-Reply-To: <20030922130845.GA5469@vicky.ecs.soton.ac.uk>;
	from arigo@tunes.org on Mon, Sep 22, 2003 at 02:08:45PM +0100
References: <3F673C08.9060502@tismer.com>
	<20030917125138.GA24502@vicky.ecs.soton.ac.uk> <3F6AF99F.9030602@tismer.com>
	<20030919130501.GA7823@vicky.ecs.soton.ac.uk> <3F6B081B.80001@tismer.com>
	<20030922130845.GA5469@vicky.ecs.soton.ac.uk>
Message-ID: <20030922151510.F12635@prim.han.de>

[Armin Rigo Mon, Sep 22, 2003 at 02:08:45PM +0100]
> Hello Christian,
> 
> > >Argh ! The files look completely broken now !! Warning !
> 
> I hopefully fixed the script now. You might try it (carefully!) on some new
> text files.
> 
> I'm still a bit worried that if fixeol actually changes content, as would be
> the case if it is run by someone on another platform than the one used to
> create it, then svn will record it as completely changed (ideal source for
> conflicts). The only solution I could imagine would be to make sure that
> people specify files to be text when they first add it to the repository. 
> Somehow the default "files-are-binary" of SVN creates different, new kind of
> problems than the default "files-are-text" of CVS. Maybe a no-default
> you-must-specify-it-explicitely wouldn't have been such a bad idea.

we could enforce this on a pre-commit hook and maybe other things
like e.g. that REST-files compile without errors.  The latter would make
a checkin of documents slower but i think it's worth it IMO.

cheers,

    holger

From lac at strakt.com  Mon Sep 22 18:27:40 2003
From: lac at strakt.com (Laura Creighton)
Date: Mon, 22 Sep 2003 18:27:40 +0200
Subject: [pypy-dev] Re: fixeol 
In-Reply-To: Message from holger krekel <hpk@trillke.net> 
   of "Mon, 22 Sep 2003 15:15:10 +0200." <20030922151510.F12635@prim.han.de> 
References: <3F673C08.9060502@tismer.com>
	<20030917125138.GA24502@vicky.ecs.soton.ac.uk> <3F6AF99F.9030602@tismer.com>
	<20030919130501.GA7823@vicky.ecs.soton.ac.uk> <3F6B081B.80001@tismer.com>
	<20030922130845.GA5469@vicky.ecs.soton.ac.uk>
	<20030922151510.F12635@prim.han.de> 
Message-ID: <200309221627.h8MGReh9028566@ratthing-b246.strakt.com>

In a message of Mon, 22 Sep 2003 15:15:10 +0200, holger krekel writes:
>[Armin Rigo Mon, Sep 22, 2003 at 02:08:45PM +0100]
>> Hello Christian,
>> 
>> > >Argh ! The files look completely broken now !! Warning !
>> 
>> I hopefully fixed the script now. You might try it (carefully!) on some
> new
>> text files.
>> 
>> I'm still a bit worried that if fixeol actually changes content, as wou
>ld be
>> the case if it is run by someone on another platform than the one used 
>to
>> create it, then svn will record it as completely changed (ideal source 
>for
>> conflicts). The only solution I could imagine would be to make sure tha
>t
>> people specify files to be text when they first add it to the repositor
>y. 
>> Somehow the default "files-are-binary" of SVN creates different, new ki
>nd of
>> problems than the default "files-are-text" of CVS. Maybe a no-default
>> you-must-specify-it-explicitely wouldn't have been such a bad idea.
>
>we could enforce this on a pre-commit hook and maybe other things
>like e.g. that REST-files compile without errors.  The latter would make
>a checkin of documents slower but i think it's worth it IMO.
>
>cheers,
>
>    holger

I want to be able to stick broken ReST into the depository at any
time.  When your sysadmin is about to take your machine at work down
for the weekend, is the time you need to stuff your half done work
into the repository so you can go home and keep working, not discover
that your work is marooned on a disk that is running a weekend long
memory error diagnostic.  Now BITCHING at me that it is broken, at
commit time, preferably with a reminder of how to run the tool that
tells you whats not working, that is a fine idea, I think.

Laura


From tinuviel at sparcs.kaist.ac.kr  Tue Sep 23 18:38:37 2003
From: tinuviel at sparcs.kaist.ac.kr (Seo Sanghyeon)
Date: Wed, 24 Sep 2003 01:38:37 +0900
Subject: [pypy-dev] My first try: dir()
Message-ID: <20030923163837.GA32413@sparcs.kaist.ac.kr>

Hello, everybody,

I followed the mailing list for some time, but haven't
touched the code yet.

Here's my first try, dir(), from your confused newbie.
For now, everything from Objectspace to Gateway is just
incomprehensible for me, but I will try...

I'm requesting for some comments. My try seems to work,
but I want to know how would you write the following,
and how to implement the rest of dir(), etc.

Index: builtin.py
===================================================================
--- builtin.py  (revision 1415)
+++ builtin.py  (working copy)
@@ -383,6 +383,20 @@
         except AttributeError:
             return False
 
+    def dir(self, w_object=None):
+        space = self.space
+        ob = space.unwrap(w_object)
+        if ob is None:
+            dic = self.locals()
+        elif type(ob) is Module:
+            dic = ob.w_dict
+        else:
+            w_exc = space.w_ValueError("Not implemented yet")
+            raise OperationError(space.w_ValueError, w_exc)
+        result = space.call_method(dic, 'keys')
+        space.call_method(result, 'sort')
+        return result
+
 # source code for the builtin xrange-class
 xrange_appsource = """if 1: 
     class xrange:

From jeremy at zope.com  Wed Sep 24 03:50:59 2003
From: jeremy at zope.com (Jeremy Hylton)
Date: Tue, 23 Sep 2003 21:50:59 -0400
Subject: [pypy-dev] Extended goals
In-Reply-To: <3F67306F.10704@tismer.com>
References: <D8AC753A-E853-11D7-A81E-000A95686CD8@redivi.com>
	 <3F672600.2000202@tismer.com> <20030916171941.S12635@prim.han.de>
	 <3F67306F.10704@tismer.com>
Message-ID: <1064368258.2072.83.camel@localhost.localdomain>

On Tue, 2003-09-16 at 11:46, Christian Tismer wrote:
> [many more binaries than expected]
> 
> > btw, i forgot all about it: this is mainly the Zope-Database (ZODB) 
> > which is independent from Zope and has quite some parts written 
> > in C, for performance reasons ASFAIK.  I'd say they are
> > applying a common technique with high-level-languages:
> > rapidly prototype in Python and reimplement the 
> > speed-relevant parts in C (and maintain it).  
> > It's rather obvious how PyPy could help here :-)
> 
> Absolutely. Zope can reverse its path and move
> its core back to where it belongs: Python.

In fact, Zope3 started with all Python implements of ZODB.  A few parts
of it have been rewritten in C over time.  Performance is the main goal,
but another noteworthy reason to use C is to reduce memory usage.  Pure
Python objects consume a lot of memory.  C extension types can be fairly
small.  (datetime is a good example.)

Does pypy have a plan for memory or is performance the primary goal?

Jeremy



From hpk at trillke.net  Wed Sep 24 08:41:16 2003
From: hpk at trillke.net (holger krekel)
Date: Wed, 24 Sep 2003 08:41:16 +0200
Subject: [pypy-dev] Extended goals
In-Reply-To: <1064368258.2072.83.camel@localhost.localdomain>;
	from jeremy@zope.com on Tue, Sep 23, 2003 at 09:50:59PM -0400
References: <D8AC753A-E853-11D7-A81E-000A95686CD8@redivi.com>
	<3F672600.2000202@tismer.com> <20030916171941.S12635@prim.han.de>
	<3F67306F.10704@tismer.com> <1064368258.2072.83.camel@localhost.localdomain>
Message-ID: <20030924084116.E20157@prim.han.de>

[Jeremy Hylton Tue, Sep 23, 2003 at 09:50:59PM -0400]
> On Tue, 2003-09-16 at 11:46, Christian Tismer wrote:
> > [many more binaries than expected]
> > 
> > > btw, i forgot all about it: this is mainly the Zope-Database (ZODB) 
> > > which is independent from Zope and has quite some parts written 
> > > in C, for performance reasons ASFAIK.  I'd say they are
> > > applying a common technique with high-level-languages:
> > > rapidly prototype in Python and reimplement the 
> > > speed-relevant parts in C (and maintain it).  
> > > It's rather obvious how PyPy could help here :-)
> > 
> > Absolutely. Zope can reverse its path and move
> > its core back to where it belongs: Python.
> 
> In fact, Zope3 started with all Python implements of ZODB.  A few parts
> of it have been rewritten in C over time.  Performance is the main goal,
> but another noteworthy reason to use C is to reduce memory usage.  Pure
> Python objects consume a lot of memory.  C extension types can be fairly
> small.  (datetime is a good example.)
> 
> Does pypy have a plan for memory or is performance the primary goal?

allowing to have a small memory footprint is certainly a goal. 
PyPy aims to make it a lot easier to make decisions regarding
space<->time tradeoffs. So Zope3 could e.g. use their own completely 
compliant python interpreter optimized for memory consumption. 

Do the parts of Zope3 that got rewritten in C still have a complete
python counterpart? Do the unit-tests run on both of them? 

cheers,

    holger

From jeremy at zope.com  Wed Sep 24 15:46:06 2003
From: jeremy at zope.com (Jeremy Hylton)
Date: Wed, 24 Sep 2003 09:46:06 -0400
Subject: [pypy-dev] Extended goals
In-Reply-To: <20030924084116.E20157@prim.han.de>
References: <D8AC753A-E853-11D7-A81E-000A95686CD8@redivi.com>
	 <3F672600.2000202@tismer.com> <20030916171941.S12635@prim.han.de>
	 <3F67306F.10704@tismer.com>
	 <1064368258.2072.83.camel@localhost.localdomain>
	 <20030924084116.E20157@prim.han.de>
Message-ID: <1064411165.2072.104.camel@localhost.localdomain>

On Wed, 2003-09-24 at 02:41, holger krekel wrote:
> [Jeremy Hylton Tue, Sep 23, 2003 at 09:50:59PM -0400]
> > In fact, Zope3 started with all Python implements of ZODB.  A few parts
> > of it have been rewritten in C over time.  Performance is the main goal,
> > but another noteworthy reason to use C is to reduce memory usage.  Pure
> > Python objects consume a lot of memory.  C extension types can be fairly
> > small.  (datetime is a good example.)
> > 
> > Does pypy have a plan for memory or is performance the primary goal?
> 
> allowing to have a small memory footprint is certainly a goal. 
> PyPy aims to make it a lot easier to make decisions regarding
> space<->time tradeoffs. So Zope3 could e.g. use their own completely 
> compliant python interpreter optimized for memory consumption. 
> 
> Do the parts of Zope3 that got rewritten in C still have a complete
> python counterpart? Do the unit-tests run on both of them? 

ZODB4 has very little C code and most of it is also available in
Python.  There is a timestamp that was implemented in C from the get-go,
but it would be trivial to rewrite in Python.  The tests used to run
with only Python code, but no one has run the tests that way for a long
time.

I think Zope3 has more C code without a Python counterpart.  There are
several extensions that started in Python, migrated to C, then evolved. 
So I wouldn't expect Zope to be straightforward, but the ZODB kernel
should be.

Jeremy



From paoloinvernizzi at dmsware.com  Thu Sep 25 14:20:34 2003
From: paoloinvernizzi at dmsware.com (Paolo Invernizzi)
Date: Thu, 25 Sep 2003 14:20:34 +0200
Subject: [pypy-dev] Re: Extended goals
In-Reply-To: <1064411165.2072.104.camel@localhost.localdomain>
References: <D8AC753A-E853-11D7-A81E-000A95686CD8@redivi.com>
	<3F672600.2000202@tismer.com> <20030916171941.S12635@prim.han.de>
	<3F67306F.10704@tismer.com>	 <1064368258.2072.83.camel@localhost.localdomain>
	<20030924084116.E20157@prim.han.de>
	<1064411165.2072.104.camel@localhost.localdomain>
Message-ID: <bkumii$p1a$1@sea.gmane.org>

Jeremy Hylton wrote:

  > ZODB4 has very little C code and most of it is also available in
> Python.

*mumble*

Is not ZODB4 development *frozen*?

---
Paolo Invernizzi



From jeremy at alum.mit.edu  Thu Sep 25 17:24:30 2003
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Thu, 25 Sep 2003 11:24:30 -0400
Subject: [pypy-dev] Re: Extended goals
In-Reply-To: <bkumii$p1a$1@sea.gmane.org>
References: <D8AC753A-E853-11D7-A81E-000A95686CD8@redivi.com>
	 <3F672600.2000202@tismer.com> <20030916171941.S12635@prim.han.de>
	 <3F67306F.10704@tismer.com>
	 <1064368258.2072.83.camel@localhost.localdomain>
	 <20030924084116.E20157@prim.han.de>
	 <1064411165.2072.104.camel@localhost.localdomain>
	 <bkumii$p1a$1@sea.gmane.org>
Message-ID: <1064503470.2072.191.camel@localhost.localdomain>

On Thu, 2003-09-25 at 08:20, Paolo Invernizzi wrote:
> Jeremy Hylton wrote:
> 
>   > ZODB4 has very little C code and most of it is also available in
> > Python.
> 
> *mumble*
> 
> Is not ZODB4 development *frozen*?

Yes, and who knows when it will thaw.  But it's a better target than
ZODB3, at least for now.

Jeremy



From arigo at tunes.org  Thu Sep 25 21:07:20 2003
From: arigo at tunes.org (Armin Rigo)
Date: Thu, 25 Sep 2003 20:07:20 +0100
Subject: [pypy-dev] Pyrex back-end
Message-ID: <20030925190720.GC20013@vicky.ecs.soton.ac.uk>

Hello everybody,

Concensus seems to araise that we should try during the sprint to obtain a
basic translator-to-low-level-code, and that Pyrex would be a nice try.

Remember that AnnObjSpace is supposed to do a complete type analysis of all
the RPython code. With Pyrex we can seriously work on the different problem of
actually emiting low-level code, before we have a really complete analysis. 
The idea is to simulate bytecode-by-bytecode the execution of the RPython
source code, and instead of actually executing the instructions, we write a
line in a .pyx (Pyrex) file. Whenever we are unsure about the type of a 
specific variable we can always fall back to generic Python objects.

Pyrex don't have gotos -- they are essential for the low-level kind of code we
can emit by blindly recording operation in an object space, because the object
space doesn't see the control structures at all (only the operations). So
Holger implemented (in 29 minutes!) a new Pyrex statement:

    cinline "anything"

which causes Pyrex to emit the given string literally in the C code it 
produces. So we can say

    cinline "Label1:"
    ...
    if condition:
        cinline "goto Label1;"

which is probably nicer than explicit "label" and "goto" statements because it
denotes clearly that it is a hack :-) It also let us emit any other C code
that we couldn't easily express in the Pyrex language.

    http://codespeak.net/svn/user/hpk/pyrex-0.8.2-hacked


A bientot,

Armin


From gherman at darwin.in-berlin.de  Fri Sep 26 20:40:11 2003
From: gherman at darwin.in-berlin.de (Dinu Gherman)
Date: Fri, 26 Sep 2003 20:40:11 +0200
Subject: [pypy-dev] Fwd: Lightweight Languages 2003 (LL3) CFP
Message-ID: <DC078FE5-F050-11D7-985D-00039345C610@darwin.in-berlin.de>

Sounds like this could be of interest for PyPy development (and/or
funding?), especially since the program committee is highly aware
of Python, already! ;-)

Dinu

> Von: Jeremy Hylton <jeremy at alum.mit.edu>
> Datum: Fr, 26. Sep 2003  16:14:18 Europe/Berlin
> An: python-announce-list at python.org
> Betreff: Lightweight Languages 2003 (LL3) CFP
> Antwort an: jeremy at alum.mit.edu
>
>           Lightweight Languages Workshop 2003 (LL3)
>           ----------------------------------
>
>        Saturday, November 8, 2003, MIT, Cambridge, MA
>        http://ll3.ai.mit.edu/   mailto:ll3 at ai.mit.edu
>
>             CALL FOR PRESENTATIONS
>
> LL3 will be an intense, exciting, one-day forum bringing together the
> best programming language implementors and researchers, from both
> academia and industry, to exchange ideas and information, to challenge
> one another, and to learn from one another.
>
> The workshop series focuses on programming languages, tools, and
> processes that are usable and useful.  Lightweight languages have been
> an effective vehicle for introducing new features to mainstream
> programmers.
>
> We encourage presentations on topics of interest to the community of
> lightweight language users and designers.  We prefer topics that will
> interest a broad audience.  We do not require the presentation of
> novel research ideas.
>
> Proposal Abstracts
> ------------------
> Due Friday Oct. 17
>
>   We seek two to five page abstracts of talks to be given at the
>   workshop.  Talks will be 30 minutes long, including time for
>   questions and answers.  Presenters are not expected to submit
>   papers, but slides will be published on the workshop web site.
>   We will also consider proposals for talks of different lengths.
>
>   Some suggested topics are:
>
>     * Language design: New language features, type systems and other
>       reasoning tools, critiques of existing languages.
>
>     * Implementation techniques: Interpreters and virtual machines,
>       optimizations, cross-language interactions.
>
>     * From scripts to programs: Software engineering with
>       lightweight and/or a mix of languages.  How to (or not to!)
>       scale from little scripts to large, complex applications.
>       Application war stories.
>
>     * From ivory tower to cubicle -- what language implementors need
>       to know from academic programming languages research -- and
>       back.
>
>     * Lightweight pearls:  Elegant and instructive examples of
>       programs in lightweight languages.
>
>     * Programming tools and development processes.
>
>   Send submissions to: ll3-submit at ai.mit.edu
>
> We want presentations that will inspire, motivate, and educate. We
> want language implementors and researchers to leave the workshop fired
> up with ideas for future languages, features, and implementation
> tricks.  We want language users to leave the workshop fired up with
> new ideas and new tools.
>
> In addition to submitted presentations, there will be two invited
> talks. After the workshop, there will be an evening social event and
> dinner.
>
> Dates:  Tuesday, Oct. 17 -- submissions due by end of day.
>         Tuesday, Oct. 24 -- notification of acceptance or rejection
>         Saturday, Nov. 8 -- 9am-7pm, LL3!
>
> Program Committee:
>
>   Ken Anderson, BBN (co-chair)
>   Jeremy Hylton, Python Software Foundation (co-chair)
>   Geoffrey Knauth, BAE Systems
>   Shriram Krishnamurthi, Brown University
>   Erik Meijer, Microsoft Research
>   Dan Sugalski, Perl Foundation
>   Greg Sullivan, MIT CSAIL


From lac at strakt.com  Fri Sep 26 20:43:16 2003
From: lac at strakt.com (Laura Creighton)
Date: Fri, 26 Sep 2003 20:43:16 +0200
Subject: [pypy-dev] Fwd: Lightweight Languages 2003 (LL3) CFP 
In-Reply-To: Message from Dinu Gherman <gherman@darwin.in-berlin.de> 
	of "Fri, 26 Sep 2003 20:40:11 +0200."
	<DC078FE5-F050-11D7-985D-00039345C610@darwin.in-berlin.de> 
References: <DC078FE5-F050-11D7-985D-00039345C610@darwin.in-berlin.de> 
Message-ID: <200309261843.h8QIhGh9019598@ratthing-b246.strakt.com>

In a message of Fri, 26 Sep 2003 20:40:11 +0200, Dinu Gherman writes:
>Sounds like this could be of interest for PyPy development (and/or
>funding?), especially since the program committee is highly aware
>of Python, already! ;-)
>
>Dinu

If we only had teleporters ....  Wonder if Jonathan wants to give a talk.
Where in the USA is he from anyhow?

Laura


From jriehl at cs.uchicago.edu  Sat Sep 27 02:23:04 2003
From: jriehl at cs.uchicago.edu (Jonathan David Riehl)
Date: Fri, 26 Sep 2003 19:23:04 -0500 (CDT)
Subject: [pypy-dev] Fwd: Lightweight Languages 2003 (LL3) CFP 
In-Reply-To: <200309261843.h8QIhGh9019598@ratthing-b246.strakt.com>
Message-ID: <Pine.LNX.4.44.0309261914580.22575-100000@pennsylvanian.cs.uchicago.edu>

I'm in Chicago, more specifically at the University of Chicago, studying
under Dave Beazley (just to name drop :)).

Actually, I attended LL2, and was giving serious consideration to
attending LL3 (there are some DrScheme people here and I get jealous when
they get to go cool places and I don't.)  I'd be game, especially if anyone
wants to help me come up with a juicy topic to write about, maybe at the
sprint (hint hint).

-Jon

On Fri, 26 Sep 2003, Laura Creighton wrote:

> In a message of Fri, 26 Sep 2003 20:40:11 +0200, Dinu Gherman writes:
> >Sounds like this could be of interest for PyPy development (and/or
> >funding?), especially since the program committee is highly aware
> >of Python, already! ;-)
> >
> >Dinu
>
> If we only had teleporters ....  Wonder if Jonathan wants to give a talk.
> Where in the USA is he from anyhow?
>
> Laura
>


From tismer at tismer.com  Mon Sep 29 02:29:04 2003
From: tismer at tismer.com (Christian Tismer)
Date: Mon, 29 Sep 2003 02:29:04 +0200
Subject: [pypy-dev] OT: abs(x) with 4 assembly insns
Message-ID: <3F777CD0.5000409@tismer.com>

Hi friends,

today, Armin presented me a simple brain-teaser:

You have X86 assembly, you have only 4 insns,
and you don't want to use a jump.
You have a register, loaded with a value, and
you should produce its abs, in another register,
while preserving the argument register.

Hmm. 4 insns.

Here is the standard C solution for doing abs without a jump,
whioch you can find on the internet:

int int_abs(int a) {
     return a - ((a+a) & (a>>31));
}

This clearly computes the bas, quite nicely.
The resulting assembly code looks like so:

; 2    :     return a - ((a+a) & (a>>31));

   00000	8b 44 24 04	 mov	 eax, DWORD PTR _a$[esp-4]
   00004	8b c8		 mov	 ecx, eax
   00006	c1 f9 1f	 sar	 ecx, 31			; 0000001fH
   00009	8d 14 00	 lea	 edx, DWORD PTR [eax+eax]
   0000c	23 ca		 and	 ecx, edx
   0000e	2b c1		 sub	 eax, ecx

; 3    : }

This is clearly some solution. But, ignoring the first insn,
which is just moving the argument, this really takes
five instructions, one too much of course.

Actually, there is a much better solution.
X86 has the cdq insn, which does a sign extension from
32 bit signed eax to 64 bit signed edx/eax.

This in fact puts a -1 into edx, whenever the eax is negative,
or a 0, if eax is >= 0.

Now, what is taking the abs() of a value? Right, it is a two's
complement, if the argument is negative, or a do-nothing if
it is not negative.
And the name 'two's complement' comes from the fact
that you need to do two operations for the complement:
First, take the NOT of the value, then add one.

So, here comes the trick:
NOT is equivalent to XORing with -1.
ADD 1 is the same as SUB -1.

Fortunately, this can all be carried out with the result of
the CDQ sign extension, since we got the -1 from it:

Do the CDQ.
Load eax into another register
XOR it with edx   (which is either 0 or -1)
SUB edx from it   (which is either adding one or zero)

So, here the solution(TM):

int other_abs(int a) {
     __asm {
         mov eax, a;
         /* from here... */
         cdq;
         mov ebx, eax;
         xor ebx, edx;
         sub ebx, edx;
         /*... to here*/
         xchg eax, ebx;
     }
}

The meat of the program is exactly four insns, the rest
is just needed to make the C calling conventions happy.

Here the full program, you can step through it with a
debugger, to see that it really works:


int int_abs(int a) {
     return a - ((a+a) & (a>>31));
}


int other_abs(int a) {
     __asm {
         mov eax, a;                /* C: load arg */
         /* from here... */
         cdq;                       /* 1: edx has either 0 or -1 */
         mov ebx, eax;              /* 2: get a copy */
         xor ebx, edx;              /* 3: either identity, or NOT */
         sub ebx, edx;              /* 4: either add 0, or 1 */
         /* ... to here */
         xchg eax, ebx;             /* C: store result */
     }
}


int main(int argc, char **argv) {
     int a;
     a = int_abs(5);
     a = int_abs(-5);
     a = other_abs(5);
     a = other_abs(-5);
     return 0;
}


Hoping-that-everybody-trusts-me-now-that-I-can-write-a-small-OS
     -- ly y'rs - chris

-- 
Christian Tismer             :^)   <mailto:tismer at tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  mobile +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/




From bob at redivi.com  Mon Sep 29 04:41:56 2003
From: bob at redivi.com (Bob Ippolito)
Date: Sun, 28 Sep 2003 22:41:56 -0400
Subject: [pypy-dev] OT: abs(x) with 4 assembly insns
In-Reply-To: <3F777CD0.5000409@tismer.com>
Message-ID: <7D9FD136-F226-11D7-B287-000A95686CD8@redivi.com>

On Sunday, Sep 28, 2003, at 20:29 America/New_York, Christian Tismer 
wrote:

> Hi friends,
>
> today, Armin presented me a simple brain-teaser:
>
> You have X86 assembly, you have only 4 insns,
> and you don't want to use a jump.
> You have a register, loaded with a value, and
> you should produce its abs, in another register,
> while preserving the argument register.
>
> Hmm. 4 insns.

If you are using a PowerPC with Altivec, you can get the absolute value 
of four 32bit integers in three instructions from C using vector signed 
int vec_abs(vector signed int a), which is a special compiler macro 
that turns into:
					# v1 is argument vector, v0 is result vector
	vspltisw	v0,0		# v0 = [0] * 4
	vsubuwm	v0,v0,v1	# v0 = map(int.__sub__, v0, v1)
	vmaxsw	v0,v1,v0	# v0 = map(max, v1, v0)

With conventional PowerPC instructions (also three instructions):
					# r0 is argument register, r1 is result, r2 is scratch
	srawi	r1,r0,31	# see below, python can't do this nicely
	add		r2,r1,r0	# r2 = r1 + r0
	xor		r1,r2,r1	# r1 = r2 ^ r1

Here's an Python example of the second, mainly to demonstrate how srawi 
works (working in unsigned longs because python 2.3 is inconvenient 
about bit twiddling signed integers):

def binary(v):
	return ''.join([(v & (1L << i)) and '1' or '0' for i in 
range(32)[::-1]])+'b'

def srawi(v, num):
	if v & 0x80000000L:
		mask = 0xFFFFFFFFL ^ ((1L << (32L - num)) - 1)
	else:
		mask = 0x00000000L
	return (v >> num) | mask

def new_abs(r0):
	if r0 < 0:
		r0 = 0x100000000L + r0
	print 'input is %s' % (binary(r0),)
	r1 = srawi(r0, 31)
	print 'r1 = %s' % (binary(r1),)
	r2 = r1 + r0
	print 'r2 = %s' % (binary(r2),)
	r1 = (r2 ^ r1) & 0xFFFFFFFFL
	print 'r1 = %s' % (binary(r1),)
	return r1

 >>> new_abs(1000)
input is 00000000000000000000001111101000b
r1 = 00000000000000000000000000000000b
r2 = 00000000000000000000001111101000b
r1 = 00000000000000000000001111101000b
1000L
 >>> new_abs(-1000)
input is 11111111111111111111110000011000b
r1 = 11111111111111111111111111111111b
r2 = 11111111111111111111110000010111b
r1 = 00000000000000000000001111101000b
1000L

-bob


From mwh at python.net  Mon Sep 29 13:15:23 2003
From: mwh at python.net (Michael Hudson)
Date: Mon, 29 Sep 2003 12:15:23 +0100
Subject: [pypy-dev] Re: OT: abs(x) with 4 assembly insns
References: <3F777CD0.5000409@tismer.com>
Message-ID: <2mr81z26d0.fsf@starship.python.net>

Christian Tismer <tismer at tismer.com> writes:

> Hi friends,
>
> today, Armin presented me a simple brain-teaser:
>
> You have X86 assembly, you have only 4 insns,
> and you don't want to use a jump.
> You have a register, loaded with a value, and
> you should produce its abs, in another register,
> while preserving the argument register.
>
> Hmm. 4 insns.

Or you can cheat, and remember that you saw something like this in
some psyco header file somewhere...

  /* as you can check the following takes the absolute value of (say) EAX:      \
       ADD EAX, EAX                                                             \
       SBB EAX, sourcecopy                                                      \
       SBB EDX, EDX                                                             \
       XOR EAX, EDX                                                             \
    (note: although the idea is not original, the above code might be           \
     original as it has been found by an exhaustive search on *all*             \
     short codes :-)                                                            \
  */                                                                            \

I don't know x86 assembler, so I have no idea which of the two code
sequences are actually likely to run quicker on a modern processor.

I've been reading the PPC architecture manual, though, so I can tell
you that Bob's 3 instructions are all single cycle instructions on a
G3...

Cheers,
mwh

-- 
  There are two kinds of large software systems: those that evolved
  from small systems and those that don't work.
                           -- Seen on slashdot.org, then quoted by amk


From bob at redivi.com  Mon Sep 29 13:33:56 2003
From: bob at redivi.com (Bob Ippolito)
Date: Mon, 29 Sep 2003 07:33:56 -0400
Subject: [pypy-dev] Re: OT: abs(x) with 4 assembly insns
In-Reply-To: <2mr81z26d0.fsf@starship.python.net>
Message-ID: <CF85DA91-F270-11D7-B287-000A95686CD8@redivi.com>


On Monday, Sep 29, 2003, at 07:15 America/New_York, Michael Hudson  
wrote:

> Christian Tismer <tismer at tismer.com> writes:
>
>> Hi friends,
>>
>> today, Armin presented me a simple brain-teaser:
>>
>> You have X86 assembly, you have only 4 insns,
>> and you don't want to use a jump.
>> You have a register, loaded with a value, and
>> you should produce its abs, in another register,
>> while preserving the argument register.
>>
>> Hmm. 4 insns.
>
> Or you can cheat, and remember that you saw something like this in
> some psyco header file somewhere...
>
>   /* as you can check the following takes the absolute value of (say)  
> EAX:      \
>        ADD EAX, EAX                                                     
>          \
>        SBB EAX, sourcecopy                                              
>          \
>        SBB EDX, EDX                                                     
>          \
>        XOR EAX, EDX                                                     
>          \
>     (note: although the idea is not original, the above code might be   
>          \
>      original as it has been found by an exhaustive search on *all*     
>          \
>      short codes :-)                                                    
>          \
>   */                                                                    
>          \
>
> I don't know x86 assembler, so I have no idea which of the two code
> sequences are actually likely to run quicker on a modern processor.
>
> I've been reading the PPC architecture manual, though, so I can tell
> you that Bob's 3 instructions are all single cycle instructions on a
> G3...

You should also check out IBM's PowerPC Compiler Writer's Guide, if you  
haven't already:
http://www-3.ibm.com/chips/techlib/techlib.nsf/techdocs/ 
852569B20050FF7785256996007558C6

Hard copies of this document can be obtained by calling IBM at (800)  
879-2755 and requesting document number SA14-2094, I hear that they  
used to (maybe still do) charge $1.00 for the book :)

-bob


From mwh at python.net  Mon Sep 29 14:23:04 2003
From: mwh at python.net (Michael Hudson)
Date: Mon, 29 Sep 2003 13:23:04 +0100
Subject: [pypy-dev] Re: OT: abs(x) with 4 assembly insns
References: <2mr81z26d0.fsf@starship.python.net>
	<CF85DA91-F270-11D7-B287-000A95686CD8@redivi.com>
Message-ID: <2mn0cn2387.fsf@starship.python.net>

Bob Ippolito <bob at redivi.com> writes:

>> I've been reading the PPC architecture manual, though, so I can tell
>> you that Bob's 3 instructions are all single cycle instructions on a
>> G3...
>
> You should also check out IBM's PowerPC Compiler Writer's Guide, if
> you  haven't already:
> http://www-3.ibm.com/chips/techlib/techlib.nsf/techdocs/
> 852569B20050FF7785256996007558C6
>
> Hard copies of this document can be obtained by calling IBM at (800)
> 879-2755 and requesting document number SA14-2094, I hear that they
> used to (maybe still do) charge $1.00 for the book :)

Our Apologies

You have attempted to enter a controlled access area that you are not
currently entitled to view. IBM Microelectronics uses a entitlement
process to protect potentially sensitive data.

... oh well.  I'll dig around for it, though.  And I guess s&h might
be more that $1 to the UK...

Cheers,
mwh


-- 
  if-you-need-your-own-xxx.py-you-know-where-to-shove-it<wink>-ly 
  y'rs - tim
              -- Tim Peters dishes out versioning advice on python-dev


From lac at strakt.com  Mon Sep 29 14:34:50 2003
From: lac at strakt.com (Laura Creighton)
Date: Mon, 29 Sep 2003 14:34:50 +0200
Subject: [pypy-dev] Re: OT: abs(x) with 4 assembly insns 
In-Reply-To: Message from Bob Ippolito <bob@redivi.com> 
	of "Mon, 29 Sep 2003 07:33:56 EDT."
	<CF85DA91-F270-11D7-B287-000A95686CD8@redivi.com> 
References: <CF85DA91-F270-11D7-B287-000A95686CD8@redivi.com> 
Message-ID: <200309291234.h8TCYoh9002415@ratthing-b246.strakt.com>

In a message of Mon, 29 Sep 2003 07:33:56 EDT, Bob Ippolito writes:
>
>
>You should also check out IBM's PowerPC Compiler Writer's Guide, if you  
>haven't already:
>http://www-3.ibm.com/chips/techlib/techlib.nsf/techdocs/ 
>852569B20050FF7785256996007558C6
>
>Hard copies of this document can be obtained by calling IBM at (800)  
>879-2755 and requesting document number SA14-2094, I hear that they  
>used to (maybe still do) charge $1.00 for the book :)
>
>-bob

800 numbers don't work outside of the US, or maybe outside of the US
and Canada.  Do you have a regular number?

Laura

From tismer at tismer.com  Mon Sep 29 15:13:57 2003
From: tismer at tismer.com (Christian Tismer)
Date: Mon, 29 Sep 2003 15:13:57 +0200
Subject: [pypy-dev] Re: OT: abs(x) with 4 assembly insns
In-Reply-To: <2mr81z26d0.fsf@starship.python.net>
References: <3F777CD0.5000409@tismer.com> <2mr81z26d0.fsf@starship.python.net>
Message-ID: <3F783015.4060402@tismer.com>

Michael Hudson wrote:

> Christian Tismer <tismer at tismer.com> writes:
...

>>Hmm. 4 insns.
> 
> 
> Or you can cheat, and remember that you saw something like this in
> some psyco header file somewhere...

No, I couldn't. Armin was referring to it, and I wanted
to find out by myself. He told me about the exhaustive
search, which I found funny.

>   /* as you can check the following takes the absolute value of (say) EAX:      \
>        ADD EAX, EAX                                                             \
>        SBB EAX, sourcecopy                                                      \
>        SBB EDX, EDX                                                             \
>        XOR EAX, EDX                                                             \
>     (note: although the idea is not original, the above code might be           \
>      original as it has been found by an exhaustive search on *all*             \
>      short codes :-)                                                            \
>   */                                                                            \

Actually, the above is not correct since it needs
5 insns. Armin didn't count the sourcecopy.

My solution really has four insns, but it is
cheating since it *needs* the register eax.

> I don't know x86 assembler, so I have no idea which of the two code
> sequences are actually likely to run quicker on a modern processor.

I guess they are almost the same speed.

ciao - chris

-- 
Christian Tismer             :^)   <mailto:tismer at tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  pager +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/



From bob at redivi.com  Mon Sep 29 15:19:58 2003
From: bob at redivi.com (Bob Ippolito)
Date: Mon, 29 Sep 2003 09:19:58 -0400
Subject: [pypy-dev] Re: OT: abs(x) with 4 assembly insns
In-Reply-To: <2mn0cn2387.fsf@starship.python.net>
Message-ID: <9FAEA22E-F27F-11D7-B287-000A95686CD8@redivi.com>


On Monday, Sep 29, 2003, at 08:23 America/New_York, Michael Hudson 
wrote:

> Bob Ippolito <bob at redivi.com> writes:
>
>>> I've been reading the PPC architecture manual, though, so I can tell
>>> you that Bob's 3 instructions are all single cycle instructions on a
>>> G3...
>>
>> You should also check out IBM's PowerPC Compiler Writer's Guide, if
>> you  haven't already:
>> http://www-3.ibm.com/chips/techlib/techlib.nsf/techdocs/
>> 852569B20050FF7785256996007558C6
>>
>> Hard copies of this document can be obtained by calling IBM at (800)
>> 879-2755 and requesting document number SA14-2094, I hear that they
>> used to (maybe still do) charge $1.00 for the book :)
>
> Our Apologies
>
> You have attempted to enter a controlled access area that you are not
> currently entitled to view. IBM Microelectronics uses a entitlement
> process to protect potentially sensitive data.
>
> ... oh well.  I'll dig around for it, though.  And I guess s&h might
> be more that $1 to the UK...

Maybe it's restricted to US citizens only?  That would be odd..  I 
tried it in another browser just to make sure it wasn't a baked cookie 
from when I downloaded IBM's PowerPC compilers, and it worked just fine.

As for the 800 number not working outside of the US.. I have no idea, I 
don't work at IBM.

I don't think I can mirror the pdf publically on an archived mailing 
list according to the copyright of the book, but you all know what my 
email address is if you are interested in it ;)

-bob


From mwh at python.net  Mon Sep 29 15:23:37 2003
From: mwh at python.net (Michael Hudson)
Date: Mon, 29 Sep 2003 14:23:37 +0100
Subject: [pypy-dev] Re: OT: abs(x) with 4 assembly insns
References: <2mn0cn2387.fsf@starship.python.net>
	<9FAEA22E-F27F-11D7-B287-000A95686CD8@redivi.com>
Message-ID: <2misnb20fa.fsf@starship.python.net>

Bob Ippolito <bob at redivi.com> writes:

> On Monday, Sep 29, 2003, at 08:23 America/New_York, Michael Hudson
> wrote:
>
>> Bob Ippolito <bob at redivi.com> writes:
>>
>>>> I've been reading the PPC architecture manual, though, so I can tell
>>>> you that Bob's 3 instructions are all single cycle instructions on a
>>>> G3...
>>>
>>> You should also check out IBM's PowerPC Compiler Writer's Guide, if
>>> you  haven't already:
>>> http://www-3.ibm.com/chips/techlib/techlib.nsf/techdocs/
>>> 852569B20050FF7785256996007558C6
>>>
>>> Hard copies of this document can be obtained by calling IBM at (800)
>>> 879-2755 and requesting document number SA14-2094, I hear that they
>>> used to (maybe still do) charge $1.00 for the book :)
>>
>> Our Apologies
>>
>> You have attempted to enter a controlled access area that you are not
>> currently entitled to view. IBM Microelectronics uses a entitlement
>> process to protect potentially sensitive data.
>>
>> ... oh well.  I'll dig around for it, though.  And I guess s&h might
>> be more that $1 to the UK...
>
> Maybe it's restricted to US citizens only?  That would be odd..  I
> tried it in another browser just to make sure it wasn't a baked cookie
> from when I downloaded IBM's PowerPC compilers, and it worked just
> fine.

Using the site search got me the PDF in the end.  Maybe it checks
Referrer headers?  It's not like I care now...

Goody, more reading.

Cheers,
mwh


-- 
  Of the four project development variables - scope, cost, time and
  quality - quality isn't really a free variable.  The only possible
  values are "excellent" and "insanely excellent", depending on
  whether lives are at stake.              -- Kent Beck, XP Explained


From bob at redivi.com  Mon Sep 29 15:23:44 2003
From: bob at redivi.com (Bob Ippolito)
Date: Mon, 29 Sep 2003 09:23:44 -0400
Subject: [pypy-dev] Re: OT: abs(x) with 4 assembly insns
In-Reply-To: <3F783015.4060402@tismer.com>
Message-ID: <26269F0C-F280-11D7-B287-000A95686CD8@redivi.com>


On Monday, Sep 29, 2003, at 09:13 America/New_York, Christian Tismer 
wrote:

> Michael Hudson wrote:
>
>> Christian Tismer <tismer at tismer.com> writes:
> ...
>
>>> Hmm. 4 insns.
>> Or you can cheat, and remember that you saw something like this in
>> some psyco header file somewhere...
>
> No, I couldn't. Armin was referring to it, and I wanted
> to find out by myself. He told me about the exhaustive
> search, which I found funny.
>
>>   /* as you can check the following takes the absolute value of (say) 
>> EAX:      \
>>        ADD EAX, EAX                                                   
>>           \
>>        SBB EAX, sourcecopy                                            
>>           \
>>        SBB EDX, EDX                                                   
>>           \
>>        XOR EAX, EDX                                                   
>>           \
>>     (note: although the idea is not original, the above code might be 
>>           \
>>      original as it has been found by an exhaustive search on *all*   
>>           \
>>      short codes :-)                                                  
>>           \
>>   */                                                                  
>>           \
>
> Actually, the above is not correct since it needs
> 5 insns. Armin didn't count the sourcecopy.
>
> My solution really has four insns, but it is
> cheating since it *needs* the register eax.
>
>> I don't know x86 assembler, so I have no idea which of the two code
>> sequences are actually likely to run quicker on a modern processor.
>
> I guess they are almost the same speed.

Ah, the joys of x86.  It's amazing that so many people still use such a 
braindead ISA :)

-bob


From lac at strakt.com  Mon Sep 29 16:27:30 2003
From: lac at strakt.com (Laura Creighton)
Date: Mon, 29 Sep 2003 16:27:30 +0200
Subject: [pypy-dev] Re: OT: abs(x) with 4 assembly insns 
In-Reply-To: Message from Bob Ippolito <bob@redivi.com> 
	of "Mon, 29 Sep 2003 09:19:58 EDT."
	<9FAEA22E-F27F-11D7-B287-000A95686CD8@redivi.com> 
References: <9FAEA22E-F27F-11D7-B287-000A95686CD8@redivi.com> 
Message-ID: <200309291427.h8TERUh9002842@ratthing-b246.strakt.com>

In a message of Mon, 29 Sep 2003 09:19:58 EDT, Bob Ippolito writes:
>
>On Monday, Sep 29, 2003, at 08:23 America/New_York, Michael Hudson 
>wrote:
>
>> Bob Ippolito <bob at redivi.com> writes:
>>
>>>> I've been reading the PPC architecture manual, though, so I can tell
>>>> you that Bob's 3 instructions are all single cycle instructions on a
>>>> G3...
>>>
>>> You should also check out IBM's PowerPC Compiler Writer's Guide, if
>>> you  haven't already:
>>> http://www-3.ibm.com/chips/techlib/techlib.nsf/techdocs/
>>> 852569B20050FF7785256996007558C6
>>>
>>> Hard copies of this document can be obtained by calling IBM at (800)
>>> 879-2755 and requesting document number SA14-2094, I hear that they
>>> used to (maybe still do) charge $1.00 for the book :)
>>
>> Our Apologies
>>
>> You have attempted to enter a controlled access area that you are not
>> currently entitled to view. IBM Microelectronics uses a entitlement
>> process to protect potentially sensitive data.
>>
>> ... oh well.  I'll dig around for it, though.  And I guess s&h might
>> be more that $1 to the UK...
>
>Maybe it's restricted to US citizens only?  That would be odd..  I 
>tried it in another browser just to make sure it wasn't a baked cookie 
>from when I downloaded IBM's PowerPC compilers, and it worked just fine.

I can read it just fine as well.  How very odd.

Laura

From robin at reportlab.com  Mon Sep 29 16:49:32 2003
From: robin at reportlab.com (Robin Becker)
Date: Mon, 29 Sep 2003 15:49:32 +0100
Subject: [pypy-dev] Re: OT: abs(x) with 4 assembly insns
In-Reply-To: <2misnb20fa.fsf@starship.python.net>
References: <2mn0cn2387.fsf@starship.python.net>
 <9FAEA22E-F27F-11D7-B287-000A95686CD8@redivi.com>
 <2misnb20fa.fsf@starship.python.net>
Message-ID: <ZwsnbDB8ZEe$EwVX@jessikat.fsnet.co.uk>

http://www-3.ibm.com/chips/techlib/techlib.nsf/techdocs/852569B20050FF77
85256996007558C6/$file/cwg.pdf

seems to work.
-- 
Robin Becker

From damien-morton at nyc.rr.com  Tue Sep 30 06:59:16 2003
From: damien-morton at nyc.rr.com (Damien Morton)
Date: Tue, 30 Sep 2003 00:59:16 -0400
Subject: [pypy-dev] OT: abs(x) with 4 assembly insns  
Message-ID: <000001c3870f$9a276c80$6501a8c0@damien>

>From http://www.hackersdelight.org/basics.pdf

If your machine does not have an instruction for computing the absolute
value,
this computation can usually be done in three or four branch-free
instructions.
First, compute:

signed int x
signed int y = x >> 31

and then one of the following:
abs = (x ^ y) - y
abs = (x + y) ^ y
abs = x - ((x<<1) & y)