From cfbolz at gmx.de  Thu Feb  1 12:00:32 2007
From: cfbolz at gmx.de (Carl Friedrich Bolz)
Date: Thu, 01 Feb 2007 12:00:32 +0100
Subject: [pypy-dev] pypy 1.0 release meeting invitation
In-Reply-To: <87veiowkla.fsf@starship.python.net>
References: <87veiowkla.fsf@starship.python.net>
Message-ID: <45C1C850.9090309@gmx.de>

Hi all!

Michael Hudson wrote:
 > Task Planning
 > =============
 >
 > This topic is to collect tasks to be done for the release and divide them
 > among people. Some tasks that we could think of:
 >
 > * work out a timeframe for jit usefulness
 > * fix llvm(?)
 > * test pypy-cs, run benchmarks.
 >   * windows testing!
 > * documentation!!!
 >   * pypy.net
 >   * object optimizations (WP6)
 >   * release announcement
 >   * improve getting-started
 >   * transparent proxies
 >   * security prototype
 > * make build-tool useable
 > * polish js backend?

Another task that I came up with: we should try to document our config
options. Maybe not the more obscure ones, though :-). Imo this is quite
important, since the list of options is very big and makes it hard to
find what you are looking for.

There should also be more "high-level" options like --allworkingmodules
and --faassen (which should be renamed for the release, I guess :-) ).
And maybe a small "guide through the option jungle" or something like
this.

Cheers,

Carl Friedrich




From paul.degrandis at gmail.com  Thu Feb  1 15:12:09 2007
From: paul.degrandis at gmail.com (Paul deGrandis)
Date: Thu, 1 Feb 2007 09:12:09 -0500
Subject: [pypy-dev] pypy 1.0 release meeting invitation
In-Reply-To: <45C1C850.9090309@gmx.de>
References: <87veiowkla.fsf@starship.python.net> <45C1C850.9090309@gmx.de>
Message-ID: <9c0bb8a00702010612j3c347503j87dba823daa7f79f@mail.gmail.com>

I think this is a good idea.
Also, not so much part of the PyPy 1.0 release, but I'll have the command
line compiler done this weekend.  I'll host to source on my servers and
email the list when it's up.

Paul


On 2/1/07, Carl Friedrich Bolz <cfbolz at gmx.de> wrote:
>
> Hi all!
>
> Michael Hudson wrote:
> > Task Planning
> > =============
> >
> > This topic is to collect tasks to be done for the release and divide
> them
> > among people. Some tasks that we could think of:
> >
> > * work out a timeframe for jit usefulness
> > * fix llvm(?)
> > * test pypy-cs, run benchmarks.
> >   * windows testing!
> > * documentation!!!
> >   * pypy.net
> >   * object optimizations (WP6)
> >   * release announcement
> >   * improve getting-started
> >   * transparent proxies
> >   * security prototype
> > * make build-tool useable
> > * polish js backend?
>
> Another task that I came up with: we should try to document our config
> options. Maybe not the more obscure ones, though :-). Imo this is quite
> important, since the list of options is very big and makes it hard to
> find what you are looking for.
>
> There should also be more "high-level" options like --allworkingmodules
> and --faassen (which should be renamed for the release, I guess :-) ).
> And maybe a small "guide through the option jungle" or something like
> this.
>
> Cheers,
>
> Carl Friedrich
>
>
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20070201/ddd2c6ba/attachment.html>

From santagada at gmail.com  Sat Feb  3 14:01:35 2007
From: santagada at gmail.com (Leonardo Santagada)
Date: Sat, 3 Feb 2007 14:01:35 +0100
Subject: [pypy-dev] JS Interpreter and the mozilla testing suite
Message-ID: <B6AF6827-C8DB-4315-AFB8-582F39DAC442@gmail.com>

We are able to parse most of the mozilla suite of tests, so I made a  
little driver (a thing that runs the mozilla suite) to run the tests  
against our interpreter. Most of the tests suite breaks after the  
parsing on some missing opcode or when trying to access some non  
implemented function (or object, or prototype, or constructor). The  
parts that get as far as finishing building the test cases are here:

6-2.js passed 5 of 5 tests
7.1-1.js passed 4 of 4 tests
7.1-2.js passed 3 of 3 tests
7.7.2.js passed 1 of 1 tests
7.7.3-1.js passed 25 of 25 tests
7.7.3.js passed 166 of 184 tests
10.1.3.js passed 15 of 15 tests
10.1.6.js passed 0 of 16 tests
10.2.3-1.js passed 0 of 2 tests
11.12-1.js passed 6 of 6 tests
11.13.2-2.js passed 2 of 58 tests
11.13.2-3.js passed 2 of 76 tests
11.13.2-4.js passed 14 of 36 tests
11.13.2-5.js passed 6 of 44 tests
11.13.js passed 6 of 6 tests
11.14-1.js passed 2 of 2 tests
11.2.2-11.js passed 1 of 1 tests
11.5.1.js passed 26 of 26 tests
11.6.3.js passed 20 of 24 tests
11.9.1.js passed 29 of 63 tests
11.9.2.js passed 29 of 63 tests
11.9.3.js passed 29 of 63 tests
12.5-1.js passed 3 of 7 tests
12.5-2.js passed 3 of 7 tests
15.1.1.1.js passed 2 of 2 tests
15.1.1.2.js passed 2 of 2 tests
15.2.2.2.js passed 2 of 2 tests
15.3.5.1.js passed 0 of 2 tests
15.4.1.3.js passed 1 of 5 tests
15.4.2.2-2.js passed 0 of 14 tests
15.6.1.js passed 19 of 19 tests
15.6.3.1-4.js passed 0 of 2 tests
15.6.3.js passed 0 of 2 tests
15.6.4.2-1.js passed 0 of 46 tests
15.7.3.1-2.js passed 0 of 2 tests
15.8-1.js passed 0 of 2 tests

Not much but this is just the start. If someone wants to run all  
tests on their machines just checkout the latest revision and run  
driver.py... it will try to run all the test suite (yep, i'm planning  
on improving this) and write the results to result.txt while on  
screen you get what tests are running. Warning: this might take some  
hours

Somehow my code crashed the python interpreter (2.4.4) on one of the  
test files... will try to reproduce the crash and send a bug report  
upstream.

Now I will work on some simple stuff like the MOD operator and this  
kind of stuff to make more tests run :)

abra?os,
Leonardo Santagada




From nytrokiss at gmail.com  Sun Feb  4 01:13:56 2007
From: nytrokiss at gmail.com (James Matthews)
Date: Sat, 3 Feb 2007 19:13:56 -0500
Subject: [pypy-dev] JS Interpreter and the mozilla testing suite
In-Reply-To: <B6AF6827-C8DB-4315-AFB8-582F39DAC442@gmail.com>
References: <B6AF6827-C8DB-4315-AFB8-582F39DAC442@gmail.com>
Message-ID: <8a6b8e350702031613v72c218f5x412c8368fe5baccd@mail.gmail.com>

Great!

On 2/3/07, Leonardo Santagada <santagada at gmail.com> wrote:
>
> We are able to parse most of the mozilla suite of tests, so I made a
> little driver (a thing that runs the mozilla suite) to run the tests
> against our interpreter. Most of the tests suite breaks after the
> parsing on some missing opcode or when trying to access some non
> implemented function (or object, or prototype, or constructor). The
> parts that get as far as finishing building the test cases are here:
>
> 6-2.js passed 5 of 5 tests
> 7.1-1.js passed 4 of 4 tests
> 7.1-2.js passed 3 of 3 tests
> 7.7.2.js passed 1 of 1 tests
> 7.7.3-1.js passed 25 of 25 tests
> 7.7.3.js passed 166 of 184 tests
> 10.1.3.js passed 15 of 15 tests
> 10.1.6.js passed 0 of 16 tests
> 10.2.3-1.js passed 0 of 2 tests
> 11.12-1.js passed 6 of 6 tests
> 11.13.2-2.js passed 2 of 58 tests
> 11.13.2-3.js passed 2 of 76 tests
> 11.13.2-4.js passed 14 of 36 tests
> 11.13.2-5.js passed 6 of 44 tests
> 11.13.js passed 6 of 6 tests
> 11.14-1.js passed 2 of 2 tests
> 11.2.2-11.js passed 1 of 1 tests
> 11.5.1.js passed 26 of 26 tests
> 11.6.3.js passed 20 of 24 tests
> 11.9.1.js passed 29 of 63 tests
> 11.9.2.js passed 29 of 63 tests
> 11.9.3.js passed 29 of 63 tests
> 12.5-1.js passed 3 of 7 tests
> 12.5-2.js passed 3 of 7 tests
> 15.1.1.1.js passed 2 of 2 tests
> 15.1.1.2.js passed 2 of 2 tests
> 15.2.2.2.js passed 2 of 2 tests
> 15.3.5.1.js passed 0 of 2 tests
> 15.4.1.3.js passed 1 of 5 tests
> 15.4.2.2-2.js passed 0 of 14 tests
> 15.6.1.js passed 19 of 19 tests
> 15.6.3.1-4.js passed 0 of 2 tests
> 15.6.3.js passed 0 of 2 tests
> 15.6.4.2-1.js passed 0 of 46 tests
> 15.7.3.1-2.js passed 0 of 2 tests
> 15.8-1.js passed 0 of 2 tests
>
> Not much but this is just the start. If someone wants to run all
> tests on their machines just checkout the latest revision and run
> driver.py... it will try to run all the test suite (yep, i'm planning
> on improving this) and write the results to result.txt while on
> screen you get what tests are running. Warning: this might take some
> hours
>
> Somehow my code crashed the python interpreter (2.4.4) on one of the
> test files... will try to reproduce the crash and send a bug report
> upstream.
>
> Now I will work on some simple stuff like the MOD operator and this
> kind of stuff to make more tests run :)
>
> abra?os,
> Leonardo Santagada
>
>
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev
>



-- 
http://www.goldwatches.com
http://www.wazoozle.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20070203/9f0a93f8/attachment.html>

From Ben.Young at risk.sungard.com  Thu Feb  8 09:24:16 2007
From: Ben.Young at risk.sungard.com (Ben.Young at risk.sungard.com)
Date: Thu, 8 Feb 2007 08:24:16 +0000
Subject: [pypy-dev] [pypy-svn] r38126 - pypy/dist/pypy
In-Reply-To: <20070207222934.68B941007F@code0.codespeak.net>
Message-ID: <OF810C682B.FD7543B7-ON8025727C.002DFEF3-0025727C.002E2B08@stars.internal.sungard.corp>

Hi Carl,

Did you mean to return a dictionary here? Or am I missing something 
obvious :) ?

Good luck to everyone with the release!

Cheers,
Ben

Ben Young - Senior Software Engineer
SunGard - Enterprise House, Vision Park, Histon, Cambridge, CB24 9ZR
Tel +44 1223 266042 - Main +44 1223 266000 - http://www.sungard.com/

CONFIDENTIALITY:  This email (including any attachments) may contain 
confidential, proprietary and privileged information, and unauthorized 
disclosure or use is prohibited.  If you received this email in error, 
please notify the sender and delete this email from your system.  Thank 
you.



cfbolz at codespeak.net 
Sent by: pypy-svn-bounces at codespeak.net
07/02/2007 22:29
Please respond to
pypy-dev at codespeak.net


To
pypy-svn at codespeak.net
cc

Subject
[pypy-svn] r38126 - pypy/dist/pypy






Author: cfbolz
Date: Wed Feb  7 23:29:30 2007
New Revision: 38126

Modified:
   pypy/dist/pypy/conftest.py
Log:
add space.newlist op (to give parsing test a chance to pass).


Modified: pypy/dist/pypy/conftest.py
==============================================================================
--- pypy/dist/pypy/conftest.py           (original)
+++ pypy/dist/pypy/conftest.py           Wed Feb  7 23:29:30 2007
@@ -93,6 +93,9 @@
     def is_true(self, obj):
         return bool(obj)
 
+    def newlist(self):
+        return {}
+
 
 class OpErrKeyboardInterrupt(KeyboardInterrupt):
     pass
_______________________________________________
pypy-svn mailing list
pypy-svn at codespeak.net
http://codespeak.net/mailman/listinfo/pypy-svn





From cfbolz at gmx.de  Thu Feb  8 09:46:14 2007
From: cfbolz at gmx.de (Carl Friedrich Bolz)
Date: Thu, 08 Feb 2007 09:46:14 +0100
Subject: [pypy-dev] [pypy-svn] r38126 - pypy/dist/pypy
In-Reply-To: <OF810C682B.FD7543B7-ON8025727C.002DFEF3-0025727C.002E2B08@stars.internal.sungard.corp>
References: <OF810C682B.FD7543B7-ON8025727C.002DFEF3-0025727C.002E2B08@stars.internal.sungard.corp>
Message-ID: <45CAE356.308@gmx.de>

Hi Ben!

Ben.Young at risk.sungard.com wrote:
> Did you mean to return a dictionary here? Or am I missing something 
> obvious :) ?

No, I meant to write newdict instead of newslist. Thanks for watching!

> Good luck to everyone with the release!

Cheers,

Carl Friedrich



From arigo at tunes.org  Thu Feb  8 12:23:21 2007
From: arigo at tunes.org (Armin Rigo)
Date: Thu, 8 Feb 2007 12:23:21 +0100
Subject: [pypy-dev] JIT status
Message-ID: <20070208112320.GA31391@code0.codespeak.net>

Hi all,

The JIT has now reached the state it will have in the upcoming PyPy
release (it's still on a branch, but it will be merged).

    What it does is produce machine code in memory, and run it.
    What it does not do is gain much speed by doing so.
    Yet.

It's just bad timing that the release and EU-driven deadlines mean that
we have no time left to put the pieces together now - but essentially
they are there already.  We expect the jit to become useful during
March, and then we'll in all likeliness do another release - if only
because we promized to the EU numbers like "50% of the speed of C" for
the most extremely algorithmic examples.

Back to the present.  Let me describe the current status.  The produced
machine code is kind of good (the backend performs some liftime analysis
and register allocation).  The "timeshifter", which produces the jit
frontend, is able to handle rather incredibly tricky situations
successfully.  From a user point of view the jit has some rough edges
but nothing fundamental, and by construction it can jit absolutely any
kind of Python code - generators, nested scopes, exec statements,
sys._getframe().f_back.f_back.f_locals, etc.

At the moment only the interpreter's main dispatch loop (i.e. mostly
only pyopcode.py) is fed to the timeshifter, so the produced jit can
remove the bytecode interpretation overhead, but cannot look inside the
implementation of space.add(), for example.  So it gives results that
look like Python2C+gcc: it produces machine code that is essentially a
sequence of calls to space.add(), space.cmp(), space.is_true(), etc.  As
we all know, Python2C gives a ~2x speed-up; we get about the same with
our jit, except that the start-up overhead is greater - calling small
functions is still quite slower with the jit than without.  On the other
hand, Python2C doesn't emulate frames, for example, so AFAIK pypy
contains the first Python compiler ever where sys._getframe() completely
works (Psyco emulates frames but not at 100%).

Only the 386 backend is complete; mwh guesses that there is some degree
of likeliness that the ppc backend will be completed in time for the
release too.


How to compile a pypy-c with a jit: use translate.py on targetjit
(located in pypy/jit/goal/).  Then:

   export PYPYJITLOG=log         # logs machine code into file 'log'
   pypy-c
   >>> def f(x): return x*5
   >>> import pypyjit
   >>> pypyjit.enable(f.func_code)
   >>> f(7)

   ..../jit/codegen/i386/viewcode.py  log


A bientot,

Armin.


From arigo at tunes.org  Thu Feb  8 13:26:30 2007
From: arigo at tunes.org (Armin Rigo)
Date: Thu, 8 Feb 2007 13:26:30 +0100
Subject: [pypy-dev] release planning
Message-ID: <20070208122630.GA11511@code0.codespeak.net>

Hi,

I've created extradoc/planning/1.0/ to put txt files planning the tasks
of the upcoming release.  If you want to help with the documentation,
check if I missed something and possibly assign yourself to the items
there.  (I've put my name on two items only so far but I plan to do
more.)


Armin


From arigo at tunes.org  Thu Feb  8 20:35:12 2007
From: arigo at tunes.org (Armin Rigo)
Date: Thu, 8 Feb 2007 20:35:12 +0100
Subject: [pypy-dev] pypy/dist => pypy/trunk
Message-ID: <20070208193512.GA27892@code0.codespeak.net>

Hi !

As part of the preparation for the upcoming release, we are going to
implement the idea that 'pypy/trunk' will be for the future a better
name for normal development to go on, and 'pypy/dist' is a better name
for the place where people can get something stable.  So we will soon
copy pypy/dist to pypy/trunk, to get:

* http://codespeak.net/svn/pypy/trunk => the normal development repo
* http://codespeak.net/svn/pypy/dist => a stable snapshot

The idea is that people that would like to get a known-working but
recent PyPy can checkout or update the pypy/dist path, and people
working on PyPy itself can work in pypy/trunk.

When this is done (due sometime between tomorrow and Monday) most
checkins should go to pypy/trunk.  Holger will write-protect pypy/dist
to avoid accidental checkins.

Reminder: you can switch your existing working copy by doing:

  svn info
  # look at URL, which must be exactly http://codespeak.net/svn/pypy/dist
  # and then do:
  svn switch http://codespeak.net/svn/pypy/trunk

It really works like an update, so it should keep your local changes,
which you can then checkin (and they go to the new location,
pypy/trunk).  But never fully trust svn: if you have something important
better make a backup of the working copy first.


A bientot,

Armin


From holger at merlinux.de  Fri Feb  9 01:14:37 2007
From: holger at merlinux.de (holger krekel)
Date: Fri, 9 Feb 2007 01:14:37 +0100
Subject: [pypy-dev] changes from py-trunk / dist merge (38224)
Message-ID: <20070209001437.GT16146@solar.trillke>

Hi all, 

so i just merged the py-trunk branch to py-dist
(which is the one used as an external by pypy-dist currently)
and also did a few changes to pypy-dist itself to accomodate
for the new situation. 

The update includes a number of internal cleanups
and refactorings, most of which should not be visible from 
the user side (well, we also cleaned up all py.* namespaces 
to only provide what is supposed to be maintained in the
future, that might catch you).  And lots of documentation 
updates, including docstrings. 

For distributed testing the options you can put into
conftest.py files have now more systematic names and meanings: 

* `dist_hosts`: a required list of host specifications
* `dist_rsync_roots` - a list of relative locations to copy to the remote machines.
* `dist_rsync_ignore` - a list of relative locations to ignore for rsyncing 
* `dist_remotepython` - the remote python executable to run.
* `dist_nicelevel` - process priority of remote nodes. 
* `dist_boxed` - will run each single test in a separate process 
  (allowing to survive segfaults for example) 
* `dist_taskspernode` - Maximum number of tasks being queued to remote nodes 

NOTE that 

    pypy/conftest.py 

defines dist_rsync_roots (and dist_rsync_ignore) itself so 
you don't need to do that yourself anymore.  Ah, 
and in case we didn't mention it yet, you invoke 
distributed testing with one of: 

    py.test -d    
    py.test --dist 

Noticeable is also that Stdout/Stderr capturing
is now done FD-based instead of only patching sys.stdout/stderr. 

In other non-pypy related news, the tkinter and py.path.extpy
classes got removed, and some other not-really-used stuff
(or so i presume :). 

Please let us know if you experience any regressions/problems
as the current py-dist is soon to become the 0.9 release
(there still is a bit of documentation and packaging 
work to be done, and fixing bugs).  

best, 

holger with lots of help/work from fijal, guido and cf

P.S.: Armin, we didn't fix htmlconftest yet. 

-- 
merlinux GmbH       Steinbergstr. 42    31139 Hildesheim   
http://merlinux.de  tel +49 5121 20800 75 (fax 77) 


From anto.cuni at gmail.com  Fri Feb  9 14:10:48 2007
From: anto.cuni at gmail.com (Antonio Cuni)
Date: Fri, 09 Feb 2007 14:10:48 +0100
Subject: [pypy-dev] Status of pypy-c under Windows
Message-ID: <45CC72D8.2090006@gmail.com>

Hi all!

As we agreed on #pypy-sync, I've spent the last days in investigating & 
testing the translation of pypy-c under Windows.

My box runs Windows XP SP 1 under vmware. To get a C compiler I 
installed the C++ package of Visual Studio .NET 2003; maybe it could 
work also with other version of Visual Studio, I don't know.

Surprisingly enough, the translation of pypy-c worked out of the box 
:-). I only had to install the windows version of boehm using the howto 
in the docstring of goal/win32/gc_path_windows.py, then ./translate.

Then, I ran the lib-python tests against pypy-c.exe. The results are here:

http://codespeak.net/~antocuni/pypy-c-win32-results/

Some of the failures are quite easy to explain (for example, test_thread 
fails because threading was disabled :-)), some other are very strange. 
Btw, more than 90% of tests passes.

To run the tests I had to manually disable test_mailbox.py, because it 
couldn't cleanup the @test directory it creates and this caused next 
tests to fail. I tried to investigate a bit in that direction, then I 
remembered that windows sucks and I gave up :-).

ciao Anto


From joe at merlinux.de  Fri Feb  9 19:03:23 2007
From: joe at merlinux.de (Martin Pajak)
Date: Fri, 09 Feb 2007 19:03:23 +0100
Subject: [pypy-dev] codespeak.net downtime due to hosts switch
Message-ID: <45CCB76B.4030104@merlinux.de>

Hello all,

in recent weeks, codespeak.net has been causing some problems,
mostly internal ones but ones to be taken seriously
(kernel oops - mostly resulting in the need to restart).
So we plan to switch to another host system, and then
investigate further on the to-be-old system.  As
there are releases pending, i'd like to get the
migration done before that and plan to do it:

     Feb 10th, saturday, around 10 AM GMT+2

if nobody objects.  In case of problems i will
switch back to the old system by means of a DNS
switch (and i'll take special care that no mails
or commits go astray)

greetings, Martin


From scott+pypy-dev at scottdial.com  Fri Feb  9 22:11:16 2007
From: scott+pypy-dev at scottdial.com (Scott Dial)
Date: Fri, 09 Feb 2007 16:11:16 -0500
Subject: [pypy-dev] Status of pypy-c under Windows
In-Reply-To: <45CC72D8.2090006@gmail.com>
References: <45CC72D8.2090006@gmail.com>
Message-ID: <45CCE374.4080402@scottdial.com>

Antonio Cuni wrote:
> Hi all!
> 
> As we agreed on #pypy-sync, I've spent the last days in investigating & 
> testing the translation of pypy-c under Windows.
> 

My apologies for having let my automated testing fall to pieces.
Somewhere about a month ago the working copy got damaged and "svn up"
stopped working bringing a halt to the whole process. I attempted to
revive it last night, however I can't seem to get Armin's scripts to
work anymore.. my only guess is that it is related to the "P.S.: Armin,
we didn't fix htmlconftest yet." from holger. Because I can run the test
suite just fine, however no html files are being generated. If anyone
has a solution, then I'll get the win32 testing back up and running.
Unless you are planning to run it on a regular basis, in which case I
can just resign my box from doing that.

Along the same lines as you, it was relatively painless. The only
difference was that I am using the free VCToolkit compiler and Platform
SDK instead of a full-fledge install of Visual Studio.

(I accidentally sent this to only Antonio before, so sorry to Antonio 
for the dup.)

-- 
Scott Dial
scott at scottdial.com
scodial at cs.indiana.edu



From holger at merlinux.de  Fri Feb  9 22:25:57 2007
From: holger at merlinux.de (holger krekel)
Date: Fri, 9 Feb 2007 22:25:57 +0100
Subject: [pypy-dev] Status of pypy-c under Windows
In-Reply-To: <45CCE374.4080402@scottdial.com>
References: <45CC72D8.2090006@gmail.com> <45CCE374.4080402@scottdial.com>
Message-ID: <20070209212557.GE16146@solar.trillke>

Hi Scott!

On Fri, Feb 09, 2007 at 16:11 -0500, Scott Dial wrote:
> Antonio Cuni wrote:
> > Hi all!
> > 
> > As we agreed on #pypy-sync, I've spent the last days in investigating & 
> > testing the translation of pypy-c under Windows.
> > 
> 
> My apologies for having let my automated testing fall to pieces.
> Somewhere about a month ago the working copy got damaged and "svn up"
> stopped working bringing a halt to the whole process. I attempted to
> revive it last night, however I can't seem to get Armin's scripts to
> work anymore.. my only guess is that it is related to the "P.S.: Armin,
> we didn't fix htmlconftest yet." from holger. Because I can run the test
> suite just fine, however no html files are being generated. If anyone
> has a solution, then I'll get the win32 testing back up and running.

i think it would help.  Indeed, last night was unfortunate timing ...
fijal tried to fix htmlconftest, does it work for you now maybe? 
If not, it will be fixed in the next days, i am sure. 

> Unless you are planning to run it on a regular basis, in which case I
> can just resign my box from doing that.

if you can keep it up for now, that'd be good, i think. 
 
> Along the same lines as you, it was relatively painless. The only
> difference was that I am using the free VCToolkit compiler and Platform
> SDK instead of a full-fledge install of Visual Studio.

could you provide a few url-references how to do this oneself? 

best & thanks!

holger


From scott+pypy-dev at scottdial.com  Sat Feb 10 00:41:52 2007
From: scott+pypy-dev at scottdial.com (Scott Dial)
Date: Fri, 09 Feb 2007 18:41:52 -0500
Subject: [pypy-dev] Status of pypy-c under Windows
In-Reply-To: <20070209212557.GE16146@solar.trillke>
References: <45CC72D8.2090006@gmail.com> <45CCE374.4080402@scottdial.com>
	<20070209212557.GE16146@solar.trillke>
Message-ID: <45CD06C0.3000902@scottdial.com>

holger krekel wrote:
> On Fri, Feb 09, 2007 at 16:11 -0500, Scott Dial wrote:
>> My apologies for having let my automated testing fall to pieces.
>> Somewhere about a month ago the working copy got damaged and "svn up"
>> stopped working bringing a halt to the whole process. I attempted to
>> revive it last night, however I can't seem to get Armin's scripts to
>> work anymore.. my only guess is that it is related to the "P.S.: Armin,
>> we didn't fix htmlconftest yet." from holger. Because I can run the test
>> suite just fine, however no html files are being generated. If anyone
>> has a solution, then I'll get the win32 testing back up and running.
> 
> i think it would help.  Indeed, last night was unfortunate timing ...
> fijal tried to fix htmlconftest, does it work for you now maybe? 
> If not, it will be fixed in the next days, i am sure. 

It does indeed appear to be working. I'm currently in the middle of a 
run. I had to revert from using svnwcrevert to update the working copy. 
I kept getting an AttributeError about not having a 'locked' attribute 
on something or other. My problem in the past was the SVN data getting 
out of whack, so I dunno even despite the fresh checkout. I've decided 
to let TortoiseSVN handle the updating.. we'll see how well that lasts.

I would share with you my patches to the autotest.py but they feel very 
homegrown and not very generalized. Let me know if you are interested at 
all in that.

>> Unless you are planning to run it on a regular basis, in which case I
>> can just resign my box from doing that.
> 
> if you can keep it up for now, that'd be good, i think. 

Will do.

>> Along the same lines as you, it was relatively painless. The only
>> difference was that I am using the free VCToolkit compiler and Platform
>> SDK instead of a full-fledge install of Visual Studio.
> 
> could you provide a few url-references how to do this oneself? 
> 

My build environment is derived from the tools need to build CPython 
extensions[1] with an extra patch[2] to keep the PATH a sane length 
(duplicates PATH entries). I don't remember what hackery was required to 
get gc6.8 to build other than info about boehm already available. For 
reference, I've made a patch reflecting my changes[3].

Unfortunately (as has been hashed out on python-dev quite a bit), the VC 
Toolkit 2003 is not made available by Microsoft anymore. I've not 
attempted to use the 2005 Express Edition compiler, but for the purposes 
of compiling pypy, it should work fine unless you are trying to build a 
module to load into CPython. I retain a copy of the VCToolkitSetup.exe 
install and it can still be found 3rd-party (MD5 
90d8b963ca196aa9855b2ca6c3174c14).

[1] http://www.vrplumber.com/programming/mstoolkit/
[2] http://scottdial.com/pypy-dev/msvccompiler.py.diff
[3] http://scottdial.com/pypy-dev/NT_THREADS_MAKEFILE.diff

-- 
Scott Dial
scott at scottdial.com
scodial at cs.indiana.edu


From scott+pypy-dev at scottdial.com  Sat Feb 10 02:06:44 2007
From: scott+pypy-dev at scottdial.com (Scott Dial)
Date: Fri, 09 Feb 2007 20:06:44 -0500
Subject: [pypy-dev] Status of pypy-c under Windows
In-Reply-To: <45CD06C0.3000902@scottdial.com>
References: <45CC72D8.2090006@gmail.com>
	<45CCE374.4080402@scottdial.com>	<20070209212557.GE16146@solar.trillke>
	<45CD06C0.3000902@scottdial.com>
Message-ID: <45CD1AA4.40306@scottdial.com>

Scott Dial wrote:
> It does indeed appear to be working. I'm currently in the middle of a 
> run.

The run completely successfully, it can be found at the same URL it was 
at before: http://scottdial.com/pypytest/

-- 
Scott Dial
scott at scottdial.com
scodial at cs.indiana.edu


From holger at merlinux.de  Sat Feb 10 11:20:37 2007
From: holger at merlinux.de (holger krekel)
Date: Sat, 10 Feb 2007 11:20:37 +0100
Subject: [pypy-dev] codespeak.net downtime due to hosts switch
In-Reply-To: <45CCB76B.4030104@merlinux.de>
References: <45CCB76B.4030104@merlinux.de>
Message-ID: <20070210102037.GM16146@solar.trillke>

Hi again, 

Martin has done the switch successfully 
(the new codespeak.net IP address is 88.198.148.44, btw). 
and svn and mail seems to be in order.  please report
any regressions or problems to admin at codespeak.net or 
support at merlinux.de or ping joe_mp_ on #pypy. 

best, 

holger


On Fri, Feb 09, 2007 at 19:03 +0100, Martin Pajak wrote:
> Hello all,
> 
> in recent weeks, codespeak.net has been causing some problems,
> mostly internal ones but ones to be taken seriously
> (kernel oops - mostly resulting in the need to restart).
> So we plan to switch to another host system, and then
> investigate further on the to-be-old system.  As
> there are releases pending, i'd like to get the
> migration done before that and plan to do it:
> 
>      Feb 10th, saturday, around 10 AM GMT+2
> 
> if nobody objects.  In case of problems i will
> switch back to the old system by means of a DNS
> switch (and i'll take special care that no mails
> or commits go astray)
> 
> greetings, Martin
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev
> 

-- 
merlinux GmbH       Steinbergstr. 42    31139 Hildesheim   
http://merlinux.de  tel +49 5121 20800 75 (fax 77) 


From anto.cuni at gmail.com  Sat Feb 10 10:21:31 2007
From: anto.cuni at gmail.com (Antonio Cuni)
Date: Sat, 10 Feb 2007 10:21:31 +0100
Subject: [pypy-dev] Status of pypy-c under Windows
In-Reply-To: <45CD1AA4.40306@scottdial.com>
References: <45CC72D8.2090006@gmail.com>	<45CCE374.4080402@scottdial.com>	<20070209212557.GE16146@solar.trillke>	<45CD06C0.3000902@scottdial.com>
	<45CD1AA4.40306@scottdial.com>
Message-ID: <45CD8E9B.4040306@gmail.com>

Scott Dial wrote:
> Scott Dial wrote:
>> It does indeed appear to be working. I'm currently in the middle of a 
>> run.
> 
> The run completely successfully, it can be found at the same URL it was 
> at before: http://scottdial.com/pypytest/

This is very cool, thank you!
I've noticed that most of CLI tests are skipped because the .NET SDK is 
not installed: could you install it please, if it's not a problem? I've 
got a windows box under vmware but running all the CLI tests there on a 
regular basis is not very convenient: il would be very useful to have 
the daily tests also on windows.

ciao Anto


From scott+pypy-dev at scottdial.com  Sat Feb 10 21:56:31 2007
From: scott+pypy-dev at scottdial.com (Scott Dial)
Date: Sat, 10 Feb 2007 15:56:31 -0500
Subject: [pypy-dev] rlib.parsing.test.test_ebnfparse / test_jsonparse
Message-ID: <45CE317F.5090202@scottdial.com>

This test is invoking pygame despite what seems to be an attempt to have 
that be an option of the testing suite. This is happening on snake and 
my win32 box, but unfortunately my box actually has pygame installed. 
Thus, the testing gets halted waiting on me to click the [X]. I don't 
know the appropriate change to make, so I will defer patching it to 
whoever knows that.

-- 
Scott Dial
scott at scottdial.com
scodial at cs.indiana.edu


From holger at merlinux.de  Sat Feb 10 22:15:50 2007
From: holger at merlinux.de (holger krekel)
Date: Sat, 10 Feb 2007 22:15:50 +0100
Subject: [pypy-dev] rlib.parsing.test.test_ebnfparse / test_jsonparse
In-Reply-To: <45CE317F.5090202@scottdial.com>
References: <45CE317F.5090202@scottdial.com>
Message-ID: <20070210211550.GR16146@solar.trillke>

Hi Scott!

On Sat, Feb 10, 2007 at 15:56 -0500, Scott Dial wrote:
> This test is invoking pygame despite what seems to be an attempt to have 
> that be an option of the testing suite. This is happening on snake and 
> my win32 box, but unfortunately my box actually has pygame installed. 
> Thus, the testing gets halted waiting on me to click the [X]. I don't 
> know the appropriate change to make, so I will defer patching it to 
> whoever knows that.

that pygame-invocation should be fixed/removed now. 

holger


From cfbolz at gmx.de  Sat Feb 10 22:18:07 2007
From: cfbolz at gmx.de (Carl Friedrich Bolz)
Date: Sat, 10 Feb 2007 22:18:07 +0100
Subject: [pypy-dev] rlib.parsing.test.test_ebnfparse / test_jsonparse
In-Reply-To: <20070210211550.GR16146@solar.trillke>
References: <45CE317F.5090202@scottdial.com>
	<20070210211550.GR16146@solar.trillke>
Message-ID: <45CE368F.2030703@gmx.de>

Hi!

holger krekel wrote:
> On Sat, Feb 10, 2007 at 15:56 -0500, Scott Dial wrote:
>> This test is invoking pygame despite what seems to be an attempt to have 
>> that be an option of the testing suite. This is happening on snake and 
>> my win32 box, but unfortunately my box actually has pygame installed. 
>> Thus, the testing gets halted waiting on me to click the [X]. I don't 
>> know the appropriate change to make, so I will defer patching it to 
>> whoever knows that.
> 
> that pygame-invocation should be fixed/removed now. 

it is indeed now. sorry for the trouble, it was a checkin of mine.

cheers,

Carl Friedrich


From scott+pypy-dev at scottdial.com  Sun Feb 11 06:55:57 2007
From: scott+pypy-dev at scottdial.com (Scott Dial)
Date: Sun, 11 Feb 2007 00:55:57 -0500
Subject: [pypy-dev] Status of pypy-c under Windows
In-Reply-To: <45CD8E9B.4040306@gmail.com>
References: <45CC72D8.2090006@gmail.com>	<45CCE374.4080402@scottdial.com>	<20070209212557.GE16146@solar.trillke>	<45CD06C0.3000902@scottdial.com>	<45CD1AA4.40306@scottdial.com>
	<45CD8E9B.4040306@gmail.com>
Message-ID: <45CEAFED.8080300@scottdial.com>

Antonio Cuni wrote:
> I've noticed that most of CLI tests are skipped because the .NET SDK is 
> not installed: could you install it please, if it's not a problem?

Not a problem. Done. Most of the tests that were being skipped are 
causing failures now.. but I was able to translate a pypy-cli without 
error, so I am convinced I haven't made a mistake. However let me know 
if I am wrong about that.

Otherwise notable is the benchmark results I received:

executable                 richards           pystone          size (MB)
pypy-cli-38437             75124  123.0x      495.2  109.5x       8.398
pypy-c-38437                3148    5.2x    11345.0    4.8x       2.527
python 2.4.3                 610    1.0x    54227.7    1.0x       1.790

-- 
Scott Dial
scott at scottdial.com
scodial at cs.indiana.edu


From tismer at stackless.com  Tue Feb 13 06:10:20 2007
From: tismer at stackless.com (Christian Tismer)
Date: Mon, 12 Feb 2007 21:10:20 -0800
Subject: [pypy-dev] pypy won't compile
Message-ID: <45D1483C.8090209@stackless.com>

Hi folks,

I just tried to build pypy on my mac, but I get an annotation
error.

[translation:ERROR]  AssertionError': built-in class <type 'Struct'> not 
found in typename_mapping while compiling _compile
[translation:ERROR]     .. v500 = getattr(self_30, ('w_dict'))
[translation:ERROR]     .. 
'(pypy.interpreter.mixedmodule:34)MixedModule.getdictvalue'
[translation:ERROR] Processing block:
[translation:ERROR]  block at 3 is a <class 
'pypy.objspace.flow.flowcontext.SpamBlock'>
[translation:ERROR]  in 
(pypy.interpreter.mixedmodule:34)MixedModule.getdictvalue
[translation:ERROR]  containing the following operations:
[translation:ERROR]        v501 = getattr(space_0, ('finditem'))
[translation:ERROR]        v500 = getattr(self_30, ('w_dict'))
[translation:ERROR]        v502 = simple_call(v501, v500, w_name_0)
[translation:ERROR]        v503 = getattr(self_30, ('lazy'))
[translation:ERROR]        v504 = is_true(v503)
[translation:ERROR]  --end--

rev. 38657 checked out on OSX, Python 2.5

I installed nothing yet. Maybe something missing?

cheers - chris


From scott+pypy-dev at scottdial.com  Tue Feb 13 14:16:52 2007
From: scott+pypy-dev at scottdial.com (Scott Dial)
Date: Tue, 13 Feb 2007 08:16:52 -0500
Subject: [pypy-dev] pypy won't compile
In-Reply-To: <45D1483C.8090209@stackless.com>
References: <45D1483C.8090209@stackless.com>
Message-ID: <45D1BA44.9070300@scottdial.com>

Christian Tismer wrote:
> Hi folks,
> 
> I just tried to build pypy on my mac, but I get an annotation
> error.
> 
> rev. 38657 checked out on OSX, Python 2.5
> 
> I installed nothing yet. Maybe something missing?
> 

AFAIK, pypy is still targeted at Python 2.4 and there are still 
lingering bugs when using Python 2.5.

-- 
Scott Dial
scott at scottdial.com
scodial at cs.indiana.edu


From tismer at stackless.com  Tue Feb 13 21:55:49 2007
From: tismer at stackless.com (Christian Tismer)
Date: Tue, 13 Feb 2007 12:55:49 -0800
Subject: [pypy-dev] pypy won't compile
In-Reply-To: <45D1BA44.9070300@scottdial.com>
References: <45D1483C.8090209@stackless.com> <45D1BA44.9070300@scottdial.com>
Message-ID: <45D225D5.2010100@stackless.com>

Scott Dial wrote:
> Christian Tismer wrote:
>> Hi folks,
>>
>> I just tried to build pypy on my mac, but I get an annotation
>> error.
>>
>> rev. 38657 checked out on OSX, Python 2.5
>>
>> I installed nothing yet. Maybe something missing?
>>
> 
> AFAIK, pypy is still targeted at Python 2.4 and there are still 
> lingering bugs when using Python 2.5.

Oh, thanks!
That I did not expect, thought that the upcoming release
must be 2.5 for sure.

ciao - chris


From cfbolz at gmx.de  Wed Feb 14 10:16:36 2007
From: cfbolz at gmx.de (Carl Friedrich Bolz)
Date: Wed, 14 Feb 2007 10:16:36 +0100
Subject: [pypy-dev] documentation age
Message-ID: <45D2D374.3070102@gmx.de>

Hi all!

Just a pointer to a wonderful hack [1] that Samuele implemented
yesterday evening:

http://wyvern.cs.uni-duesseldorf.de/~cfbolz/pypy-dist/pypy/doc/

This is a copy of parts of the PyPy documentation, but with different
coloring than usual: The title gives a general age of the document in
shades of gray (with white being very young and gray being very old).
All the text is colored in shades of yellow, giving the *relative* age
of the lines making up a paragraph (using the median of all the
revisions). In addition, every paragraph has a tooltip showing its age
as a revision number.

In my opinion it is quite useful for finding outdated documentation. It
can be improved of course, but I think even the current status is very
useful.

Cheers,

Carl Friedrich



[1] http://codespeak.net/svn/user/pedronis/docage.py


From alexandre.fayolle at logilab.fr  Wed Feb 14 12:05:42 2007
From: alexandre.fayolle at logilab.fr (Alexandre Fayolle)
Date: Wed, 14 Feb 2007 12:05:42 +0100
Subject: [pypy-dev] news from the logilab internal sprint
Message-ID: <20070214110542.GB29454@crater.logilab.fr>

Hi,

We are having an internal sprint at logilab this week. This mail is to
keep you posted about our progress. 

The WP10 work is in good shape. Adrien and Sylvain managed to translate
pypy with the grammar modifications, but the merging will take some
time, I think. They are also working on a pylint checker for rpython which
hopefully will help beginners trying to produce rpython catch some
problems faster than by trying to translate their code. Coming up for
the second half of the sprint is the definition of an API for the
grammar changes, more stuff for "rpylint" and work on the interim
report.

WP09 is making rapid progress (with the writing of a rpython
module for the constraint library, which will be the basis for a mixed
module for CSP) (Aur?lien, Alexandre).

Regarding WP11, Ludovic and David have made good progress on the report.
Ludovic has done some work to write a busybox-ish tool in rpython and
an HTTP server. 

-- 
Alexandre Fayolle                              LOGILAB, Paris (France)
Formations Python, Zope, Plone, Debian:  http://www.logilab.fr/formations
D?veloppement logiciel sur mesure:       http://www.logilab.fr/services
Informatique scientifique:               http://www.logilab.fr/science
Reprise et maintenance de sites CPS:     http://www.migration-cms.com/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 481 bytes
Desc: Digital signature
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20070214/a54e3b2b/attachment.pgp>

From holger at merlinux.de  Wed Feb 14 17:51:16 2007
From: holger at merlinux.de (holger krekel)
Date: Wed, 14 Feb 2007 17:51:16 +0100
Subject: [pypy-dev] py lib 0.9.0: py.test, distributed execution,
	microthreads ...
Message-ID: <20070214165116.GI16146@solar.trillke>

py lib 0.9.0: py.test, distributed execution, greenlets and more
======================================================================

Welcome to the 0.9.0 py lib release - a library aiming to 
support agile and test-driven python development on various levels. 

Main API/Tool Features: 

* py.test: cross-project testing tool with many advanced features
* py.execnet: ad-hoc code distribution to SSH, Socket and local sub processes
* py.magic.greenlet: micro-threads on standard CPython ("stackless-light")
* py.path: path abstractions over local and subversion files 
* rich documentation of py's exported API 
* tested against Linux, OSX and partly against Win32, python 2.3-2.5

All these features and their API have extensive documentation,
generated with the new "apigen", which we intend to make accessible
for other python projects as well.  

Download/Install:   http://codespeak.net/py/0.9.0/download.html
Documentation/API:  http://codespeak.net/py/0.9.0/index.html

Work on the py lib has been partially funded by the 
European Union IST programme and by http://merlinux.de 
within the PyPy project. 

best, have fun and let us know what you think!

holger krekel, Maciej Fijalkowski, 
Guido Wesdorp, Carl Friedrich Bolz



From anto.cuni at gmail.com  Wed Feb 14 18:44:31 2007
From: anto.cuni at gmail.com (Antonio Cuni)
Date: Wed, 14 Feb 2007 18:44:31 +0100
Subject: [pypy-dev] Status of pypy-c under Windows
In-Reply-To: <45CEAFED.8080300@scottdial.com>
References: <45CC72D8.2090006@gmail.com>	<45CCE374.4080402@scottdial.com>	<20070209212557.GE16146@solar.trillke>	<45CD06C0.3000902@scottdial.com>	<45CD1AA4.40306@scottdial.com>
	<45CD8E9B.4040306@gmail.com> <45CEAFED.8080300@scottdial.com>
Message-ID: <45D34A7F.9040201@gmail.com>

Scott Dial wrote:
> Antonio Cuni wrote:
>> I've noticed that most of CLI tests are skipped because the .NET SDK 
>> is not installed: could you install it please, if it's not a problem?
> 
> Not a problem. Done. Most of the tests that were being skipped are 
> causing failures now.. but I was able to translate a pypy-cli without 
> error, so I am convinced I haven't made a mistake. However let me know 
> if I am wrong about that.
> 
> Otherwise notable is the benchmark results I received:
> 
> executable                 richards           pystone          size (MB)
> pypy-cli-38437             75124  123.0x      495.2  109.5x       8.398
> pypy-c-38437                3148    5.2x    11345.0    4.8x       2.527
> python 2.4.3                 610    1.0x    54227.7    1.0x       1.790

Hello Scott,
sorry for the late answer.

I've looked at the failures of gencli under windows: most of them should 
be easy to fix, I'll do as soon as possibile (not before next week, btw).

About the benchmark, I think that the result in wrong, because it's very 
likely that the build couldn't be performed because of a bug in MS ilasm 
I've not yet found a workaround for.

ciao Anto


From gbowyer at fastmail.co.uk  Thu Feb 15 00:05:15 2007
From: gbowyer at fastmail.co.uk (Greg Bowyer)
Date: Wed, 14 Feb 2007 23:05:15 +0000
Subject: [pypy-dev] pypy won't compile
In-Reply-To: <45D225D5.2010100@stackless.com>
References: <45D1483C.8090209@stackless.com> <45D1BA44.9070300@scottdial.com>
	<45D225D5.2010100@stackless.com>
Message-ID: <er04jc$v9h$1@sea.gmane.org>

Christian Tismer wrote:
> Scott Dial wrote:
>> Christian Tismer wrote:
>>> Hi folks,
>>>
>>> I just tried to build pypy on my mac, but I get an annotation
>>> error.
>>>
>>> rev. 38657 checked out on OSX, Python 2.5
>>>
>>> I installed nothing yet. Maybe something missing?
>>>
>> AFAIK, pypy is still targeted at Python 2.4 and there are still 
>> lingering bugs when using Python 2.5.
> 
> Oh, thanks!
> That I did not expect, thought that the upcoming release
> must be 2.5 for sure.
> 
> ciao - chris
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev
> 
Whats left in terms of getting pypy happy with CPython 2.5 ?



From pedronis at openendsystems.com  Sat Feb 17 20:11:31 2007
From: pedronis at openendsystems.com (Samuele Pedroni)
Date: Sat, 17 Feb 2007 20:11:31 +0100
Subject: [pypy-dev] PyPy 0.99 is out: new object spaces, optimizations,
 configuration ...
Message-ID: <45D75363.8060504@openendsystems.com>

First thanks to all the people that helped with the release; and to the 
people that contributed to PyPy so far.

=================================================================
pypy-0.99.0: new object spaces, optimizations, configuration ...
=================================================================
Welcome to the PyPy 0.99.0 release - a major snapshot
and milestone of the last 8 months of work and contributions
since PyPy-0.9.0 came out in June 2006!

Main entry point for getting-started/download and documentation:

    http://codespeak.net/pypy/dist/pypy/doc/index.html

Further below you'll find some notes about PyPy,
the 0.99.0 highlights and our aims for PyPy 1.0.

have fun,

    the PyPy team,
    Samuele Pedroni, Carl Friedrich Bolz, Armin Rigo, Michael Hudson,
    Maciej Fijalkowski, Anders Chrigstroem, Holger Krekel,
    Guido Wesdorp

    and many others:
    http://codespeak.net/pypy/dist/pypy/doc/contributor.html


What is PyPy?
================================

Technically, PyPy is both a Python Interpreter implementation
and an advanced Compiler, actually a framework for implementing
dynamic languages and generating virtual machines for them.
The Framework allows for alternative frontends and
for alternative backends, currently C, LLVM and .NET. 
For our main target "C", we can can "mix in" different Garbage
Collectors and threading models, including micro-threads aka
"Stackless".  The inherent complexity that arises from this
ambitious approach is mostly kept away from the Python
interpreter implementation, our main frontend.

Socially, PyPy is a collaborative effort of many individuals
working together in a distributed and sprint-driven way since
2003.  PyPy would not have gotten as far without the coding,
feedback and general support from numerous people.

Formally, many of the current developers are involved in
executing an EU contract with the goal of exploring and
researching new approaches to Language/Compiler development and
software engineering.  This contract's duration is about to
end March 2007 and we are working and preparing the according
final review which is scheduled for May 2007. 


Key 0.99.0 Features
=====================

* new object spaces:

  - Tainting: a 270-line proxy object space tracking
    and boxing sensitive information within an application.
    A tainted object is completely barred from crossing
    an I/O barrier, such as writing to files, databases
    or sockets.  This allows to significantly reduce the
    effort of e.g. security reviews to the few places where
    objects are "declassified" in order to send information
    across I/O barriers.

  - Transparent proxies: allow to customize both application and
    builtin objects from application level code.  Works as an addition
    to the Standard Object Space (and is translatable). For details see
    http://codespeak.net/pypy/dist/pypy/doc/proxy.html
 
* optimizations:

  - Experimental new optimized implementations for various built in Python
    types (strings, dicts, lists)

  - Optimized builtin lookups to not require any dictionary lookups if the
    builtin is not shadowed by a name in the global dictionary.

  - Improved inlining (now also working for higher level
    backends) and malloc removal.

  - twice the speed of the 0.9 release, overall 2-3 slower than CPython

* High level backends:

  - It is now possible to translate the PyPy interpreter to run on the .NET
    platform, which gives a very compliant (but somewhat slow) Python
    interpreter.

  - the JavaScript backend has evolved to a point where it can be used 
to write
    AJAX web applications with it. This is still an experimental technique,
    though. For demo applications see:
    http://play1.codespeak.net:8008/

* new configuration system:
  There is a new comprehensive configuration system that allows
  fine-grained configuration of the PyPy standard interpreter and the
  translation process.

* new and improved modules: Since the last release, the signal, mmap, bz2
  and fcntl standard library modules have been implemented for PyPy. The 
socket,
  _sre and os modules have been greatly improved. In addition we added a the
  pypymagic module that contains PyPy-specific functionality.

* improved file implementation: Our file implementation was ported to 
RPython
  and is therefore faster (and not based on libc).

* The stability of stackless features was greatly improved. For more details
  see: http://codespeak.net/pypy/dist/pypy/doc/stackless.html

* RPython library: The release contains our emerging RPython library 
that tries
  to make programming in RPython more pleasant. It contains an experimental
  parser generator framework. For more details see:
  http://codespeak.net/pypy/dist/pypy/doc/rlib.html

* improved documentation:
 
  - extended documentation about stackless features:
    http://codespeak.net/pypy/dist/pypy/doc/stackless.html
 
  - PyPy video documentation: eight hours of talks, interviews and features:
    http://codespeak.net/pypy/dist/pypy/doc/video-index.html

  - technical reports about various aspects of PyPy:
    http://codespeak.net/pypy/dist/pypy/doc/index-report.html
   
  The entry point to all our documentation is:
  http://codespeak.net/pypy/dist/pypy/doc/index.html



What about 1.0?
======================

In the last week leading up to the release, we decided
to go for tagging the release as 0.99.0, mainly because
we have some efforts pending to integrate and complete
research and coding work:

* the JIT Compiler Generator is ready, but not fully integrated
  with the PyPy interpreter.  As a result, the JIT does not give
  actual speed improvements yet, so we chose to leave it out of the
  0.99 release: the result doesn't meet yet the speed expectations
  that we set for ourselves - and which some blogs and people
  have chosen as the main criterium for looking at PyPy.

* the extension enabling runtime changes of the Python grammar is not
  yet integrated. This will be used to provide Aspect-Oriented
  Programming extensions and Design by Contract facilities in PyPy.

* the Logic object space, which provides Logic Variables in PyPy,
  needs to undergo a bit more testing. A constraint problem solver
  extension module is ready, and needs to be integrated with the codebase.

PyPy 0.99 is the start for getting to 1.0 end of March 2007,
which we intend to become a base for a longer (and more relaxed :)
time to come.



Funding partners and organisations
=====================================================
   
PyPy development and activities happen as an open source project 
and with the support of a consortium partially funded by a 28 months
European Union IST research grant. The full partners of that
consortium are:
       
    Heinrich-Heine University (Germany), Open End (Sweden)
    merlinux GmbH (Germany), tismerysoft GmbH (Germany)
    Logilab Paris (France), DFKI GmbH (Germany)
    ChangeMaker (Sweden), Impara (Germany)




From johnsmithjohnxxx at yahoo.com  Sat Feb 17 22:23:02 2007
From: johnsmithjohnxxx at yahoo.com (John Smith)
Date: Sat, 17 Feb 2007 13:23:02 -0800 (PST)
Subject: [pypy-dev] translating classes to javascript
Message-ID: <766761.99865.qm@web62311.mail.re1.yahoo.com>

Hi list,

I'm fairly new to pypy and was only doing some experimenting with the javascript translator so far. First of all let me say that the whole project is pretty amazing!

After translating a couple of functions I tried translating a class, but was unsuccessful so far but since the docs mention supporting inheritance I guess this should be possible. Is it?

For instance if I have in RPython the following:

class test:
    def __init__( self, value ):
        self.value = value
    def meth1( self ):
        return self.value
    def meth2( self ):
        do_something_which_is_translatable( )

then how do I get the corresponding javascript code? Since the docs say jscompile should be invoked by 'jscompile module function_names' I'm kind of lost. The same holds for rpython2javascript it expects a list of functions.

Any insight or comment would be very helpful.
 
 
---------------------------------
Need Mail bonding?
Go to the Yahoo! Mail Q&A for great tips from Yahoo! Answers users.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20070217/031097b6/attachment.html>

From fijal at genesilico.pl  Sat Feb 17 22:41:06 2007
From: fijal at genesilico.pl (Maciek Fijalkowski)
Date: Sat, 17 Feb 2007 22:41:06 +0100
Subject: [pypy-dev] translating classes to javascript
In-Reply-To: <766761.99865.qm@web62311.mail.re1.yahoo.com>
References: <766761.99865.qm@web62311.mail.re1.yahoo.com>
Message-ID: <45D77672.4090704@genesilico.pl>

John Smith wrote:
> Hi list,
>
> I'm fairly new to pypy and was only doing some experimenting with the 
> javascript translator so far. First of all let me say that the whole 
> project is pretty amazing!
>
> After translating a couple of functions I tried translating a class, 
> but was unsuccessful so far but since the docs mention supporting 
> inheritance I guess this should be possible. Is it?
>
> For instance if I have in RPython the following:
>
> class test:
>     def __init__( self, value ):
>        self.value = value
>     def meth1( self ):
>        return self.value
>     def meth2( self ):
>        do_something_which_is_translatable( )
>
> then how do I get the corresponding javascript code? Since the docs 
> say jscompile should be invoked by 'jscompile module function_names' 
> I'm kind of lost. The same holds for rpython2javascript it expects a 
> list of functions.
>
> Any insight or comment would be very helpful.
>
> ------------------------------------------------------------------------
> Need Mail bonding?
> Go to the Yahoo! Mail Q&A 
> <http://answers.yahoo.com/dir/index;_ylc=X3oDMTFvbGNhMGE3BF9TAzM5NjU0NTEwOARfcwMzOTY1NDUxMDMEc2VjA21haWxfdGFnbGluZQRzbGsDbWFpbF90YWcx?link=ask&sid=396546091> 
> for great tips from Yahoo! Answers 
> <http://answers.yahoo.com/dir/index;_ylc=X3oDMTFvbGNhMGE3BF9TAzM5NjU0NTEwOARfcwMzOTY1NDUxMDMEc2VjA21haWxfdGFnbGluZQRzbGsDbWFpbF90YWcx?link=ask&sid=396546091> 
> users.
> ------------------------------------------------------------------------
>
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev
Hi John!


Because usual way of invoking javascript on your browser is to call a 
function, you need a function where you'll begin. Like:


# class test as above

def f():
   test_instance = test(3)
   test_instance.meth()

and translate function f. Your class will magically appear in translated 
javascript.

Cheers,
fijal



From johnsmithjohnxxx at yahoo.com  Sat Feb 17 23:33:25 2007
From: johnsmithjohnxxx at yahoo.com (John Smith)
Date: Sat, 17 Feb 2007 14:33:25 -0800 (PST)
Subject: [pypy-dev] translating classes to javascript
In-Reply-To: <45D77672.4090704@genesilico.pl>
Message-ID: <599581.71585.qm@web62315.mail.re1.yahoo.com>


> I'm fairly new to pypy and was only doing some experimenting with the 
> javascript translator so far. First of all let me say that the whole 
> project is pretty amazing!
>
> After translating a couple of functions I tried translating a class, 
> but was unsuccessful so far but since the docs mention supporting 
> inheritance I guess this should be possible. Is it?
>
> For instance if I have in RPython the following:
>
> class test:
>         def __init__( self, value ):
>                self.value = value
>         def meth1( self ):
>                return self.value
>         def meth2( self ):
>                do_something_which_is_translatable( )
>
> then how do I get the corresponding javascript code? Since the docs 
> say jscompile should be invoked by 'jscompile module function_names' 
> I'm kind of lost. The same holds for rpython2javascript it expects a 
> list of functions.
>
> Any insight or comment would be very helpful.
>

Because usual way of invoking javascript on your browser is to call a 
function, you need a function where you'll begin. Like:


# class test as above

def f():
    test_instance = test(3)
    test_instance.meth()

and translate function f. Your class will magically appear in translated 
javascript.


Thanks very much for the reply, I can certainly do things this way however being able to call methods directly sounds like a reasonable thing to me (both in python and in JS). Are there plans for adding this kind of functionality or there are reasons why you think this is not necessary?


Cheers,
John

 
---------------------------------
Bored stiff? Loosen up...
Download and play hundreds of games for free on Yahoo! Games.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20070217/2dcdca30/attachment.html>

From johnsmithjohnxxx at yahoo.com  Sun Feb 18 00:01:04 2007
From: johnsmithjohnxxx at yahoo.com (John Smith)
Date: Sat, 17 Feb 2007 15:01:04 -0800 (PST)
Subject: [pypy-dev] translating classes to javascript
In-Reply-To: <45D77672.4090704@genesilico.pl>
Message-ID: <612588.6200.qm@web62305.mail.re1.yahoo.com>

<OT>

I'm really sorry for the messed up layout, yahoo mail sucks big time....

</OT>


 
---------------------------------
Access over 1 million songs - Yahoo! Music Unlimited.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20070217/e6700ffd/attachment.html>

From fijal at genesilico.pl  Sun Feb 18 00:49:01 2007
From: fijal at genesilico.pl (Maciek Fijalkowski)
Date: Sun, 18 Feb 2007 00:49:01 +0100
Subject: [pypy-dev] translating classes to javascript
In-Reply-To: <599581.71585.qm@web62315.mail.re1.yahoo.com>
References: <599581.71585.qm@web62315.mail.re1.yahoo.com>
Message-ID: <45D7946D.7000304@genesilico.pl>

John Smith wrote:
>
>     > I'm fairly new to pypy and was only doing some experimenting
>     with the
>     > javascript translator so far. First of all let me say that the
>     whole
>     > project is pretty amazing!
>     >
>     > After translating a couple of functions I tried translating a
>     class,
>     > but was unsuccessful so far but since the docs mention supporting
>     > inheritance I guess this should be possible. Is it?
>     >
>     > For instance if I have in RPython the following:
>     >
>     > class test:
>     >     def __init__( self, value ):
>     >         self.value = value
>     >     def meth1( self ):
>     >         return self.value
>     >     def meth2( self ):
>     >         do_something_which_is_translatable( )
>     >
>     > then how do I get the corresponding javascript code? Since the docs
>     > say jscompile should be invoked by 'jscompile module
>     function_names'
>     > I'm kind of lost. The same holds for rpython2javascript it
>     expects a
>     > list of functions.
>     >
>     > Any insight or comment would be very helpful.
>     >
>
>     Because usual way of invoking javascript on your browser is to call a
>     function, you need a function where you'll begin. Like:
>
>
>     # class test as above
>
>     def f():
>         test_instance = test(3)
>         test_instance.meth()
>
>     and translate function f. Your class will magically appear in
>     translated
>     javascript.
>
>
> Thanks very much for the reply, I can certainly do things this way 
> however being able to call methods directly sounds like a reasonable 
> thing to me (both in python and in JS). Are there plans for adding 
> this kind of functionality or there are reasons why you think this is 
> not necessary?
>
Yes there are plans for such support, while there was no time to 
implement it yet :-( The list of ideas lay down here: 
http://codespeak.net/pypy/dist/pypy/doc/js/todo.html

Cheers,
fijal



From tismer at stackless.com  Sun Feb 18 04:00:59 2007
From: tismer at stackless.com (Christian Tismer)
Date: Sat, 17 Feb 2007 19:00:59 -0800
Subject: [pypy-dev] PyPy 0.99 is out: new object spaces, optimizations,
 configuration ...
In-Reply-To: <45D75363.8060504@openendsystems.com>
References: <45D75363.8060504@openendsystems.com>
Message-ID: <45D7C16B.6070109@stackless.com>

Samuele Pedroni wrote:
...

Well, this is all great and nice to read.
What I'm missing in the announcement is any mention
of our current supported version of CPython.

In the documentation, I also could nowhere find out that PyPy
does not translate with Python 2.5.
Maybe this should be put somewhere since it might be quite
confusing to new users?

ciao - chris


From holger at merlinux.de  Sun Feb 18 10:07:37 2007
From: holger at merlinux.de (holger krekel)
Date: Sun, 18 Feb 2007 10:07:37 +0100
Subject: [pypy-dev] PyPy 0.99 is out: new object spaces, optimizations,
	configuration ...
In-Reply-To: <45D7C16B.6070109@stackless.com>
References: <45D75363.8060504@openendsystems.com>
	<45D7C16B.6070109@stackless.com>
Message-ID: <20070218090737.GM16146@solar.trillke>

Hi Christian! 

On Sat, Feb 17, 2007 at 19:00 -0800, Christian Tismer wrote:
> Samuele Pedroni wrote:
> ...
> 
> Well, this is all great and nice to read.
> What I'm missing in the announcement is any mention
> of our current supported version of CPython.
> 
> In the documentation, I also could nowhere find out that PyPy
> does not translate with Python 2.5.
>
> Maybe this should be put somewhere since it might be quite
> confusing to new users?

It's in the FAQ in the first "General" section, that PyPy aims
to support Python 2.4 and that that the translation 
process does not fully work with 2.5: 

    http://codespeak.net/pypy/dist/pypy/doc/faq.html

I also just added an according note in the "getting-started" in the 
"Trying out the translator" section, as it is the main entry
point apart from the release announcement which could
also have mentioned it, but generally refers to documentation 
for technical details. 

best, 

holger


From tismer at stackless.com  Sun Feb 18 23:45:44 2007
From: tismer at stackless.com (Christian Tismer)
Date: Sun, 18 Feb 2007 14:45:44 -0800
Subject: [pypy-dev] PyPy 0.99 is out: new object spaces, optimizations,
 configuration ...
In-Reply-To: <20070218090737.GM16146@solar.trillke>
References: <45D75363.8060504@openendsystems.com>
	<45D7C16B.6070109@stackless.com>
	<20070218090737.GM16146@solar.trillke>
Message-ID: <45D8D718.2010407@stackless.com>

Hi Holger,

holger krekel wrote:

> I also just added an according note in the "getting-started" in the 
> "Trying out the translator" section, as it is the main entry
> point apart from the release announcement which could
> also have mentioned it, but generally refers to documentation 
> for technical details. 

Yeah, that's good. From the getting started POV, nobody
expects that Python 2.4 is required.

thanks & ciao - chris


From nytrokiss at gmail.com  Mon Feb 19 03:06:56 2007
From: nytrokiss at gmail.com (James Matthews)
Date: Sun, 18 Feb 2007 21:06:56 -0500
Subject: [pypy-dev] PyPy 0.99 is out: new object spaces, optimizations,
	configuration ...
In-Reply-To: <45D8D718.2010407@stackless.com>
References: <45D75363.8060504@openendsystems.com>
	<45D7C16B.6070109@stackless.com>
	<20070218090737.GM16146@solar.trillke>
	<45D8D718.2010407@stackless.com>
Message-ID: <8a6b8e350702181806g2bec56cfs65f27626e7a4a3a1@mail.gmail.com>

Thanks This is great keep up the good work!

On 2/18/07, Christian Tismer <tismer at stackless.com> wrote:
>
> Hi Holger,
>
> holger krekel wrote:
>
> > I also just added an according note in the "getting-started" in the
> > "Trying out the translator" section, as it is the main entry
> > point apart from the release announcement which could
> > also have mentioned it, but generally refers to documentation
> > for technical details.
>
> Yeah, that's good. From the getting started POV, nobody
> expects that Python 2.4 is required.
>
> thanks & ciao - chris
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev
>



-- 
http://www.goldwatches.com/watches.asp?Brand=39
http://www.wazoozle.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20070218/d3d93b31/attachment.html>

From vinjvinj at yahoo.com  Mon Feb 19 17:05:08 2007
From: vinjvinj at yahoo.com (Vinj Vinj)
Date: Mon, 19 Feb 2007 08:05:08 -0800 (PST)
Subject: [pypy-dev] Running untrusted code in pypy
Message-ID: <165008.79233.qm@web33610.mail.mud.yahoo.com>

Hi,

I've been following pypy dev for several years and
this is my first post here. Thanks for all the hard
work you guys have put in, it is truly exciting to see
what pypy has accomplished so far. 

I'm currently building a distributing financial
trading application that allows users to write trading
models in python and lua. I had to introduce lua,
since there is no way to completely "secure" user
models written in cPython. 

I'm working with a modified Lua core which allows me
to restrict the max memory and max CPU cycles
available for each vm. I then have a python-lua bridge
code that allows you exchange data and function calls
between the two. 

Would I be able to do something similar with pypy?
Would I be able to ensure that no malicious user is
able to bring my hosted application down?

Are any of you guys going to be at pycon-2007?

Thanks,

Vineet





 
____________________________________________________________________________________
Bored stiff? Loosen up... 
Download and play hundreds of games for free on Yahoo! Games.
http://games.yahoo.com/games/front


From holger at merlinux.de  Mon Feb 19 17:27:41 2007
From: holger at merlinux.de (holger krekel)
Date: Mon, 19 Feb 2007 17:27:41 +0100
Subject: [pypy-dev] Running untrusted code in pypy
In-Reply-To: <165008.79233.qm@web33610.mail.mud.yahoo.com>
References: <165008.79233.qm@web33610.mail.mud.yahoo.com>
Message-ID: <20070219162741.GU16146@solar.trillke>

Hi Vinj, 

On Mon, Feb 19, 2007 at 08:05 -0800, Vinj Vinj wrote:
> I've been following pypy dev for several years and
> this is my first post here. Thanks for all the hard
> work you guys have put in, it is truly exciting to see
> what pypy has accomplished so far. 
> 
> I'm currently building a distributing financial
> trading application that allows users to write trading
> models in python and lua. I had to introduce lua,
> since there is no way to completely "secure" user
> models written in cPython. 
> 
> I'm working with a modified Lua core which allows me
> to restrict the max memory and max CPU cycles
> available for each vm. I then have a python-lua bridge
> code that allows you exchange data and function calls
> between the two. 
> 
> Would I be able to do something similar with pypy?

PyPy does not (currently) aim at offering cpu/mem
restrictions, but you could use virtual hosts (XEN or vserver)
for that, both offer such restriction settings.  PyPy itself
may help with the Taint Object Space: 

http://codespeak.net/pypy/dist/pypy/doc/objspace-proxies.html#the-taint-object-space

to track sensitive data flows in your application and prevent
it from accidentally leaking. 

> Would I be able to ensure that no malicious user is
> able to bring my hosted application down?

I'd probably use kernel-level security for that, maybe
in combination with VM provided features. (not sure
if you refer to processing of user-input or to 
DOS attacks or some other security aspects, it
obviously all depends a bit on concrete use cases
and intentions). 
 
> Are any of you guys going to be at pycon-2007?

Michael and Christian are going to be there,
myself i am busy preparing for the upcoming sprints, 
but i am happy to discuss possibilities some time. 

best, 

holger

--
merlinux GmbH       Steinbergstr. 42    31139 Hildesheim   
http://merlinux.de  tel +49 5121 20800 75 (fax 77) 


From vinjvinj at yahoo.com  Mon Feb 19 17:47:04 2007
From: vinjvinj at yahoo.com (Vinj Vinj)
Date: Mon, 19 Feb 2007 08:47:04 -0800 (PST)
Subject: [pypy-dev] Running untrusted code in pypy
In-Reply-To: <20070219162741.GU16146@solar.trillke>
Message-ID: <214627.24084.qm@web33613.mail.mud.yahoo.com>

PyPy does not (currently) aim at offering cpu/mem
> restrictions, but you could use virtual hosts (XEN
or vserver)
> for that, both offer such restriction settings. 
PyPy itself
> may help with the Taint Object Space: 
> 
Unfortunately, for my use cases, using virtual hosts
will not work. All the user models work on time series
price data. This data can get very large and has to be
shared by all user models. It is not practical for
each vmware client to have its own copy of user data.
With lua, I'm able to share this time series with all
user models and still ensure that all the models are
run securely. 


obviously all depends a bit on concrete use cases
> and intentions). 
> 
Among many things, will I be able to restrict the user
from doing:

1. a = []*10000000000000000000000000
2. a = 23**3294832098980989898
3. disable recursive calls
4. writing while loops which never end
5. etc. 

I'm trying to get a feel for whether this kind of
thing would be (in the future) possible with pypy.
With cPython, I've been told that it is just not going
to be possible. Which is why, I moved to lua for user
models. I would much rather use python so that I don't
have to maintain the python-lua bridge. 

Michael and Christian are going to be there,
> 
Cool. 

Thanks,

Vineet



 
____________________________________________________________________________________
Looking for earth-friendly autos? 
Browse Top Cars by "Green Rating" at Yahoo! Autos' Green Center.
http://autos.yahoo.com/green_center/


From vinjvinj at yahoo.com  Mon Feb 19 17:54:03 2007
From: vinjvinj at yahoo.com (Vinj Vinj)
Date: Mon, 19 Feb 2007 08:54:03 -0800 (PST)
Subject: [pypy-dev] Running untrusted code in pypy
In-Reply-To: <214627.24084.qm@web33613.mail.mud.yahoo.com>
Message-ID: <800769.16237.qm@web33608.mail.mud.yahoo.com>

> 1. a = []*10000000000000000000000000

What I meant was:

a = [None]*10000000000000000000000000


 
____________________________________________________________________________________
Need a quick answer? Get one in minutes from people who know.
Ask your question on www.Answers.yahoo.com


From cfbolz at gmx.de  Mon Feb 19 18:51:14 2007
From: cfbolz at gmx.de (Carl Friedrich Bolz)
Date: Mon, 19 Feb 2007 18:51:14 +0100
Subject: [pypy-dev] Running untrusted code in pypy
In-Reply-To: <214627.24084.qm@web33613.mail.mud.yahoo.com>
References: <214627.24084.qm@web33613.mail.mud.yahoo.com>
Message-ID: <45D9E392.5060004@gmx.de>

Hi Vineet!

Vinj Vinj wrote:
 > PyPy does not (currently) aim at offering cpu/mem
 >> restrictions, but you could use virtual hosts (XEN
 > or vserver)
 >> for that, both offer such restriction settings.
 > PyPy itself
 >> may help with the Taint Object Space:
 >>
 > Unfortunately, for my use cases, using virtual hosts
 > will not work. All the user models work on time series
 > price data. This data can get very large and has to be
 > shared by all user models. It is not practical for
 > each vmware client to have its own copy of user data.
 > With lua, I'm able to share this time series with all
 > user models and still ensure that all the models are
 > run securely.

How is the data shared? Using files or somehow differently?

 >> obviously all depends a bit on concrete use cases
 >> and intentions).
 >>
 > Among many things, will I be able to restrict the user
 > from doing:
 >
 > 1. a = []*10000000000000000000000000

Already CPython notices that this will use too much memory. But yes, in
PyPy you could impose an upper limit to memory used, by using our
custom mark-and-sweep garbage collector. This collector collects quite a
bit information while it is running, especially how much non-dead memory
is used currently. This would make it possible to impose a hard limit there.

 > 2. a = 23**3294832098980989898

Doable, but harder (and I guess you mean this in a more general way than
just checking during long computations). You would need a transformation
that inserts checks into the PyPy graphs to see whether something is
running too long without ever reaching the interpreter main loop. You
might be better served with having the program run with a timeout.

 > 3. disable recursive calls

You could fix the recursion limit.

 > 4. writing while loops which never end

You could limit the maximum numbers of bytecode instructions executed.
Again, a timeout might be the better solution.

 > 5. etc.

Another thing I can see there is accessing the file system in malicious
ways. Can be fixed on the OS level, I guess. You could not include
things like socket into your PyPy interpreter executable.


 > I'm trying to get a feel for whether this kind of
 > thing would be (in the future) possible with pypy.

As Holger said, it depends very much on what exactly you want. In PyPy
usually many things are possible and you have to choose the right
possibilities.

 > With cPython, I've been told that it is just not going
 > to be possible. Which is why, I moved to lua for user
 > models. I would much rather use python so that I don't
 > have to maintain the python-lua bridge.

Do you know about lunatic Python?  http://labix.org/lunatic-python

Cheers,

Carl Friedrich


From vinjvinj at yahoo.com  Mon Feb 19 19:49:45 2007
From: vinjvinj at yahoo.com (Vinj Vinj)
Date: Mon, 19 Feb 2007 10:49:45 -0800 (PST)
Subject: [pypy-dev] Running untrusted code in pypy
In-Reply-To: <45D9E392.5060004@gmx.de>
Message-ID: <584771.9632.qm@web33611.mail.mud.yahoo.com>

> How is the data shared? Using files or somehow
differently?

Slices of numeric python arrays

> custom mark-and-sweep garbage collector. This
collector collects quite a
> bit information while it is running, especially how
much non-dead memory
> is used currently. This would make it possible to
impose a hard limit there.

Ok. But this limit would be for the entire app and not
per user model. This should be fine, I would just take
the penalty of the OS/interpretor than releasing back
all the unused memory. 


> Doable, but harder (and I guess you mean this in a
more general way than
> just checking during long computations). You would
need a transformation
> that inserts checks into the PyPy graphs to see
whether something is
> running too long without ever reaching the
interpreter main loop. You
> might be better served with having the program run
with a timeout.
> 

I think again os based timeout interrupts would work
fine? Do you see
any downside of using os level interrupts? Any way
that the application
would not be able to catch them?

> You could fix the recursion limit.

Again this would be for the entire application and not
per user model. 

> Another thing I can see there is accessing the file
system in malicious
> ways. Can be fixed on the OS level, I guess. You
could not include
> things like socket into your PyPy interpreter
executable.

This is the tricky part. The main python application
used a lot of
cPython libraries, so not including them in the
interpreter was not an
option. I was hoping that there would be some other
way which could tell
the pypy interpreter, before it executes a certain
piece of code, that
access to the following list of modules ([x, y, z...])
is allowed.

> As Holger said, it depends very much on what exactly
you want. In PyPy
> usually many things are possible and you have to
choose the right
> possibilities.

> Do you know about lunatic Python? 
http://labix.org/lunatic-python

Yes. I'm using a modified version of this library. 

Vineet




 
____________________________________________________________________________________
Don't pick lemons.
See all the new 2007 cars at Yahoo! Autos.
http://autos.yahoo.com/new_cars.html 


From cfbolz at gmx.de  Mon Feb 19 21:27:02 2007
From: cfbolz at gmx.de (Carl Friedrich Bolz)
Date: Mon, 19 Feb 2007 21:27:02 +0100
Subject: [pypy-dev] Running untrusted code in pypy
In-Reply-To: <584771.9632.qm@web33611.mail.mud.yahoo.com>
References: <584771.9632.qm@web33611.mail.mud.yahoo.com>
Message-ID: <45DA0816.9010000@gmx.de>

Vinj Vinj wrote:
 >> How is the data shared? Using files or somehow
 >> differently?
 >
 > Slices of numeric python arrays

Then the foremost problem will probably be that PyPy is not close to
supporting numeric arrays :-). I guess that will change at one point.

 >> custom mark-and-sweep garbage collector. This
 >> collector collects quite a
 >> bit information while it is running, especially how
 >> much non-dead memory
 >> is used currently. This would make it possible to
 >> impose a hard limit there.
 >
 > Ok. But this limit would be for the entire app and not
 > per user model. This should be fine, I would just take
 > the penalty of the OS/interpretor than releasing back
 > all the unused memory.

There can be more advanced solutions: The GC has "memory pools" and you
could have a solution where the main app has its own (unlimited) pool
and the user models have limited pools.

 > I think again os based timeout interrupts would work
 > fine? Do you see
 > any downside of using os level interrupts? Any way
 > that the application
 > would not be able to catch them?

There is a downside to os-level interrupts (both in CPython and in
PyPy): Only the intepreter main loop checks for interrupts, that means
that this does not work against 2**(something big) if I see it
correctly. I cannot think of a good way to fix this, I fear.

 >> You could fix the recursion limit.
 >
 > Again this would be for the entire application and not
 > per user model.
 >
 >> Another thing I can see there is accessing the file
 >> system in malicious
 >> ways. Can be fixed on the OS level, I guess. You
 >> could not include
 >> things like socket into your PyPy interpreter
 >> executable.
 >
 > This is the tricky part. The main python application
 > used a lot of
 > cPython libraries, so not including them in the
 > interpreter was not an
 > option. I was hoping that there would be some other
 > way which could tell
 > the pypy interpreter, before it executes a certain
 > piece of code, that
 > access to the following list of modules ([x, y, z...])
 > is allowed.

This is something which is quite hard to enforce, given Python's very
introspective nature. There are some ideas to support a rather strict
distinction between two different sorts of code within the same process
with PyPy. You would have two interpreters in the same executable, one
for trusted, one for sandboxed code. The sandboxed interpreter would
only get access to a very limited set of modules and builtins. The
trusted interpreter could somehow "control" what sort of operations the
untrusted part would be allowed to do.

This is quite a mess to implement correctly (but easier than in CPython,
I suppose), but might give a general solution to this set of problems.

Cheers,

Carl Friedrich


From jeff at taupro.com  Mon Feb 19 21:30:47 2007
From: jeff at taupro.com (Jeff Rush)
Date: Mon, 19 Feb 2007 14:30:47 -0600
Subject: [pypy-dev] Running untrusted code in pypy
In-Reply-To: <214627.24084.qm@web33613.mail.mud.yahoo.com>
References: <214627.24084.qm@web33613.mail.mud.yahoo.com>
Message-ID: <45DA08F7.1030607@taupro.com>

Vinj Vinj wrote:
> 
> With cPython, I've been told that it is just not going
> to be possible. Which is why, I moved to lua for user
> models.

It sounds like you're going to be at PyCon, so be sure not to miss the talk on 
Saturday afternoon:

Securing Python: "Protecting the interpreter from code wielding fresh fruit." 
(#41) by Brett Cannon

"Python currently has no security model. This talk discusses why this is and 
how I am fixing the problem."

-Jeff



From nytrokiss at gmail.com  Tue Feb 20 03:51:08 2007
From: nytrokiss at gmail.com (James Matthews)
Date: Mon, 19 Feb 2007 21:51:08 -0500
Subject: [pypy-dev] Running untrusted code in pypy
In-Reply-To: <45DA08F7.1030607@taupro.com>
References: <214627.24084.qm@web33613.mail.mud.yahoo.com>
	<45DA08F7.1030607@taupro.com>
Message-ID: <8a6b8e350702191851g621c2984kbbe03f1bdbd8ccd@mail.gmail.com>

How can you detect such code running ( all the bad code)

On 2/19/07, Jeff Rush <jeff at taupro.com> wrote:
>
> Vinj Vinj wrote:
> >
> > With cPython, I've been told that it is just not going
> > to be possible. Which is why, I moved to lua for user
> > models.
>
> It sounds like you're going to be at PyCon, so be sure not to miss the
> talk on
> Saturday afternoon:
>
> Securing Python: "Protecting the interpreter from code wielding fresh
> fruit."
> (#41) by Brett Cannon
>
> "Python currently has no security model. This talk discusses why this is
> and
> how I am fixing the problem."
>
> -Jeff
>
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev
>



-- 
http://www.goldwatches.com/watches.asp?Brand=39
http://www.wazoozle.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20070219/7c373336/attachment.html>

From jeff at taupro.com  Tue Feb 20 04:14:05 2007
From: jeff at taupro.com (Jeff Rush)
Date: Mon, 19 Feb 2007 21:14:05 -0600
Subject: [pypy-dev] Running untrusted code in pypy
In-Reply-To: <8a6b8e350702191851g621c2984kbbe03f1bdbd8ccd@mail.gmail.com>
References: <214627.24084.qm@web33613.mail.mud.yahoo.com>	<45DA08F7.1030607@taupro.com>
	<8a6b8e350702191851g621c2984kbbe03f1bdbd8ccd@mail.gmail.com>
Message-ID: <45DA677D.8090407@taupro.com>

James Matthews wrote:
> How can you detect such code running ( all the bad code)

A complicated topic that can't be covered in a brief email but Python used to 
have a security model/features:

   http://www.python.org/doc/2.3.5/lib/restricted.html

but there were ways to escape the sandbox.  Perhaps they could be closed but 
no one had the time to carefully study the matter, so it was disabled in 2.3 
and I believe removed in 2.5.

Brett Cannon is re-opening the matter.  You can read about his approach at:

   http://tinyurl.com/2sh55f

Many in the Python community are excited because it will finally bring 
capability-based security to Python, if it works.

There is also some cross-pollination of ideas re capabilities with the 
one-laptop-per-child project, who recently published their security model. 
They have a lot of Python code to secure, in a potentially hostile 
laptop/network environment.  You can read about their model at:

   http://wiki.laptop.org/go/Bitfrost

Ask Ivan Krsti? about Bitfrost, whose development he led.  He is giving the 
opening keynote at PyCon on Friday morning.

-Jeff


> On 2/19/07, *Jeff Rush* <jeff at taupro.com <mailto:jeff at taupro.com>> wrote:
> 
>     Vinj Vinj wrote:
>      >
>      > With cPython, I've been told that it is just not going
>      > to be possible. Which is why, I moved to lua for user
>      > models.
> 
>     It sounds like you're going to be at PyCon, so be sure not to miss
>     the talk on
>     Saturday afternoon:
> 
>     Securing Python: "Protecting the interpreter from code wielding
>     fresh fruit."
>     (#41) by Brett Cannon
> 
>     "Python currently has no security model. This talk discusses why
>     this is and
>     how I am fixing the problem."



From npk.gangadhar at gmail.com  Tue Feb 20 19:19:37 2007
From: npk.gangadhar at gmail.com (Gangadhar NPK)
Date: Tue, 20 Feb 2007 23:49:37 +0530
Subject: [pypy-dev] Running untrusted code in pypy
In-Reply-To: <45DA677D.8090407@taupro.com>
References: <214627.24084.qm@web33613.mail.mud.yahoo.com>
	<45DA08F7.1030607@taupro.com>
	<8a6b8e350702191851g621c2984kbbe03f1bdbd8ccd@mail.gmail.com>
	<45DA677D.8090407@taupro.com>
Message-ID: <61b4e70f0702201019u28d1bd2flbdc59930bcb7f6cd@mail.gmail.com>

Vineet,
Though not directly related to this post, I would be interested to
know how you have modified the lua core to provide individual VMs for
users to operate within. The problem you are trying to solve is to
enable pythonic access within Lua, but I am more interested in
restricted user level execution model, and would like to know more
about it. May be you can drop a note (either to the list or to my mail
id) when you get some time.

Thank You
Gangadhar

On 2/20/07, Jeff Rush <jeff at taupro.com> wrote:
> James Matthews wrote:
> > How can you detect such code running ( all the bad code)
>
> A complicated topic that can't be covered in a brief email but Python used to
> have a security model/features:
>
>    http://www.python.org/doc/2.3.5/lib/restricted.html
>
> but there were ways to escape the sandbox.  Perhaps they could be closed but
> no one had the time to carefully study the matter, so it was disabled in 2.3
> and I believe removed in 2.5.
>
> Brett Cannon is re-opening the matter.  You can read about his approach at:
>
>    http://tinyurl.com/2sh55f
>
> Many in the Python community are excited because it will finally bring
> capability-based security to Python, if it works.
>
> There is also some cross-pollination of ideas re capabilities with the
> one-laptop-per-child project, who recently published their security model.
> They have a lot of Python code to secure, in a potentially hostile
> laptop/network environment.  You can read about their model at:
>
>    http://wiki.laptop.org/go/Bitfrost
>
> Ask Ivan Krsti? about Bitfrost, whose development he led.  He is giving the
> opening keynote at PyCon on Friday morning.
>
> -Jeff
>
>
> > On 2/19/07, *Jeff Rush* <jeff at taupro.com <mailto:jeff at taupro.com>> wrote:
> >
> >     Vinj Vinj wrote:
> >      >
> >      > With cPython, I've been told that it is just not going
> >      > to be possible. Which is why, I moved to lua for user
> >      > models.
> >
> >     It sounds like you're going to be at PyCon, so be sure not to miss
> >     the talk on
> >     Saturday afternoon:
> >
> >     Securing Python: "Protecting the interpreter from code wielding
> >     fresh fruit."
> >     (#41) by Brett Cannon
> >
> >     "Python currently has no security model. This talk discusses why
> >     this is and
> >     how I am fixing the problem."
>
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev

From micahel at gmail.com  Wed Feb 21 12:44:44 2007
From: micahel at gmail.com (Michael Hudson)
Date: Wed, 21 Feb 2007 11:44:44 +0000
Subject: [pypy-dev] python.net server issues
Message-ID: <aa0dfd860702210344l57a2a4c8v63073b08c50f209e@mail.gmail.com>

The python.net server is down currently.  I'd hoped it would only be a
brief downtime, but it's now looking more likely that it'll be down
for some time, so can I ask anyone who needs to email me to use this
address instead.

Cheers,
mwh
(not happy)


From faassen at startifact.com  Thu Feb 22 11:41:16 2007
From: faassen at startifact.com (Martijn Faassen)
Date: Thu, 22 Feb 2007 11:41:16 +0100
Subject: [pypy-dev] py lib 0.9.0: py.test, distributed execution,
 microthreads ...
In-Reply-To: <20070214165116.GI16146@solar.trillke>
References: <20070214165116.GI16146@solar.trillke>
Message-ID: <erjs0f$963$1@sea.gmane.org>

holger krekel wrote:
> py lib 0.9.0: py.test, distributed execution, greenlets and more
> ======================================================================
> 
> Welcome to the 0.9.0 py lib release - a library aiming to 
> support agile and test-driven python development on various levels. 
> 
> Main API/Tool Features: 
> 
> * py.test: cross-project testing tool with many advanced features
> * py.execnet: ad-hoc code distribution to SSH, Socket and local sub processes
> * py.magic.greenlet: micro-threads on standard CPython ("stackless-light")
> * py.path: path abstractions over local and subversion files 
> * rich documentation of py's exported API 
> * tested against Linux, OSX and partly against Win32, python 2.3-2.5
> 
> All these features and their API have extensive documentation,
> generated with the new "apigen", which we intend to make accessible
> for other python projects as well.  
> 
> Download/Install:   http://codespeak.net/py/0.9.0/download.html

Cool, a pylib release!

Is a cheeseshop entry planned? I checked the download page but there's 
no mention of it. Perhaps there's a cheeseshop entry that I didn't see? 
People might want to do 'easy_install pylib'. Having a cheeseshop entry 
also makes it more easy to use pylib in any buildouts.

Regards,

Martijn



From santagada at gmail.com  Fri Feb 23 06:01:07 2007
From: santagada at gmail.com (Leonardo Santagada)
Date: Fri, 23 Feb 2007 03:01:07 -0200
Subject: [pypy-dev] Help to put the test results on my user directory
Message-ID: <B64EEA69-A110-447C-B751-3FFB7CBF743E@gmail.com>

I want to post the results of the test run of the mozilla test suite  
against my js interpreter... do anyone have any good idea how to do  
that? My plan was to run the tests outputing to reST and then using  
rst2html to generate a web page, but any html file that I put on my  
user dir on svn get shown as a text file on browsers. Another problem  
is that I would love to be able to present the information in  
diferent ways, isn't there a way to put all the data collected on the  
run on a xml file or a database so I can use a template language to  
format my data?

Thanks a lot in advance,
Leonardo Santagada


From micahel at gmail.com  Fri Feb 23 15:42:40 2007
From: micahel at gmail.com (Michael Hudson)
Date: Fri, 23 Feb 2007 08:42:40 -0600
Subject: [pypy-dev] Help to put the test results on my user directory
In-Reply-To: <B64EEA69-A110-447C-B751-3FFB7CBF743E@gmail.com>
References: <B64EEA69-A110-447C-B751-3FFB7CBF743E@gmail.com>
Message-ID: <aa0dfd860702230642j7585f79ai4c554bea5e4c3bd6@mail.gmail.com>

On 2/22/07, Leonardo Santagada <santagada at gmail.com> wrote:
> I want to post the results of the test run of the mozilla test suite
> against my js interpreter... do anyone have any good idea how to do
> that? My plan was to run the tests outputing to reST and then using
> rst2html to generate a web page, but any html file that I put on my
> user dir on svn get shown as a text file on browsers.

Do you really need to put them in subversion?  If not, you can just
copy them to ~santagada/public_html/jstestsresults/ or whatever.

If there is, then you probably still want to serve the html out of
~santagada, but you can use a ~/.svntrigger file to update the copy in
your home dir whenever new results get checked in.

> Another problem
> is that I would love to be able to present the information in
> diferent ways, isn't there a way to put all the data collected on the
> run on a xml file or a database so I can use a template language to
> format my data?

Um, probably :-)

If you're asking for a prebuilt solution, I don't think so.  If you're
asking for recommendations, I don't really have any either, but it
doesn't sound like the most daunting of tasks, really...

Cheers,
mwh


From lac at openend.se  Sun Feb 25 09:12:56 2007
From: lac at openend.se (Laura Creighton)
Date: Sun, 25 Feb 2007 09:12:56 +0100
Subject: [pypy-dev] GvR having fun in edu-sig
Message-ID: <200702250812.l1P8CuVV021341@theraft.openend.se>

http://mail.python.org/pipermail/edu-sig/2007-February/007787.html


From arigo at tunes.org  Sun Feb 25 10:52:11 2007
From: arigo at tunes.org (Armin Rigo)
Date: Sun, 25 Feb 2007 10:52:11 +0100
Subject: [pypy-dev] pypy won't compile
In-Reply-To: <er04jc$v9h$1@sea.gmane.org>
References: <45D1483C.8090209@stackless.com> <45D1BA44.9070300@scottdial.com>
	<45D225D5.2010100@stackless.com> <er04jc$v9h$1@sea.gmane.org>
Message-ID: <20070225095210.GA14909@code0.codespeak.net>

Hi Greg,

On Wed, Feb 14, 2007 at 11:05:15PM +0000, Greg Bowyer wrote:
> Whats left in terms of getting pypy happy with CPython 2.5 ?

We don't know for sure.  Someone would have to look at each test failure
on 2.5 and decide what to do about it.  We already occasionally did that
(e.g. the annotator should now be happy with new-style classes based on
BaseException instead of old-style classes based on Exception), so there
is probably nothing quite deep left.


A bientot,

Armin