From renesd at gmail.com  Mon Mar  1 11:15:22 2010
From: renesd at gmail.com (=?ISO-8859-1?Q?Ren=E9_Dudfield?=)
Date: Mon, 1 Mar 2010 10:15:22 +0000
Subject: [pypy-dev] speed.pypy.org launched
In-Reply-To: <hm862m$cer$1@dough.gmane.org>
References: <c561346d1002250710h5df95a1ao8eaaf314ef34d963@mail.gmail.com>
	<4B86B588.4000601@gmx.de> <hm862m$cer$1@dough.gmane.org>
Message-ID: <64ddb72c1003010215t50a3b620n1f541c851894684b@mail.gmail.com>

On Fri, Feb 26, 2010 at 9:59 AM, Stefan Behnel <stefan_ml at behnel.de> wrote:
> Carl Friedrich Bolz, 25.02.2010 18:38:
>> On 02/25/2010 04:10 PM, Miquel Torres wrote:
>>> As some of you already know, there is a new performance site online:
>>> speed.pypy.org.
>> [...]
>>
>> I'm quite impressed, this is very cool work and a good improvement over
>> the current plots. Thanks for doing this!
>>
>> One thing that I would really find useful are error bars in the timeline
>> view. This would help us judge whether up-and-down movements are within
>> the margin of randomness, or whether it is a real effect. I don't know
>> how annoying they are to implement though, no clue whether the plot lib
>> supports that. There should be enough information about errors in the
>> json files, as far as I remember.
>
> The standard deviation is commonly considered meaningless for benchmark
> results. All that really matters is the fastest run, everything else is
> just fog.
>
> Read the docs on timeit, for example.
>
> Stefan
>


hi,

For some sets of problems, the first run is very important.  For
example, where you only want to process the particular data once.

For others the Worst performance is the most important.  For example
'real time' like programs which require the runtime only takes at
maximum N where N is measured.

Running something 10 times and removing outliers fails for these two
important cases.


cya.


From renesd at gmail.com  Mon Mar  1 11:18:07 2010
From: renesd at gmail.com (=?ISO-8859-1?Q?Ren=E9_Dudfield?=)
Date: Mon, 1 Mar 2010 10:18:07 +0000
Subject: [pypy-dev] speed.pypy.org launched
In-Reply-To: <64ddb72c1003010215t50a3b620n1f541c851894684b@mail.gmail.com>
References: <c561346d1002250710h5df95a1ao8eaaf314ef34d963@mail.gmail.com>
	<4B86B588.4000601@gmx.de> <hm862m$cer$1@dough.gmane.org>
	<64ddb72c1003010215t50a3b620n1f541c851894684b@mail.gmail.com>
Message-ID: <64ddb72c1003010218l73a9779j17709e7b4275be23@mail.gmail.com>

On Mon, Mar 1, 2010 at 10:15 AM, Ren? Dudfield <renesd at gmail.com> wrote:
>
> For others the Worst performance is the most important. ?For example
> 'real time' like programs which require the runtime only takes at
> maximum N where N is measured.

um, this made no sense at all... oops!  Sorry, let me try again...


For others the Worst performance is the most important.  For example
'real time' like programs which require the runtime only takes at
maximum N.  Where N is an allocated time budget for that code.


From fijall at gmail.com  Tue Mar  2 05:50:36 2010
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Mon, 1 Mar 2010 21:50:36 -0700
Subject: [pypy-dev] Release and tags
Message-ID: <693bc9ab1003012050q39f4a2a8odd009c58b9375c67@mail.gmail.com>

Hello.

Can we copy trunk -> dist for the release, so we don't block any new
development because of the release? There was the point of dist, even
if we forgot it completely.

Cheers,
fijal


From renesd at gmail.com  Tue Mar  2 16:51:38 2010
From: renesd at gmail.com (=?ISO-8859-1?Q?Ren=E9_Dudfield?=)
Date: Tue, 2 Mar 2010 15:51:38 +0000
Subject: [pypy-dev] gsoc for pypy...
Message-ID: <64ddb72c1003020751r2bc5d26hfd0f3f0eff80c47a@mail.gmail.com>

hi,

gsoc is on again... for your information the python mailing list for it is
here:
    http://mail.python.org/mailman/listinfo/soc2010-mentors

It would be good to have a wiki page(or blog post? ... err a text file
available via http) of project ideas for pypy... if there are any pypy
people mentoring that is.  With the pygame project last year, we found
having a project page for suggestions of acceptable projects helped students
pick things they were interested in, and things the project would be happy
with.



cu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20100302/c14e67a2/attachment.html>

From renesd at gmail.com  Tue Mar  2 19:06:25 2010
From: renesd at gmail.com (=?ISO-8859-1?Q?Ren=E9_Dudfield?=)
Date: Tue, 2 Mar 2010 18:06:25 +0000
Subject: [pypy-dev] gsoc for pypy...
In-Reply-To: <64ddb72c1003020751r2bc5d26hfd0f3f0eff80c47a@mail.gmail.com>
References: <64ddb72c1003020751r2bc5d26hfd0f3f0eff80c47a@mail.gmail.com>
Message-ID: <64ddb72c1003021006r6dc70afdj23e804490909ea@mail.gmail.com>

hello again.

To start the gsoc ideas list off...

- port jit to 64 bit (maybe too short for 3 month project, if so start on
ARM after amd64?).
- speed up ctypes wrapper.
- GIL removal work.
- python2.6/2.7/py3k features.
- ctypes bindings for database adaptors (would be good for ironpython,
jython, and even cpython too).
- faster ctypes module.  (by using jit?)
- cpython bridge module.  Either load pypy as cpython extension or the
otherway around.  To allow gradual porting.
- ironclad port to pypy.  To allow loading cpython C extensions.  eg,
ironpython can run numpy with this.
- revive javascript/flex backend.
- improvements to java/.net backend.
- AOT compilation research - to allow compiling to iphone for example.

I guess this list could also be put on a page looking for sponsors?  To give
sponsors something concrete they can pay a contract for.  eg, 'approximately
2-3 months work(24,000 euros) to get X benefit, as well as a logo+link on
our sponsors page... etc.'


Anyway... those are some of the areas I can think of from past discussions.
I'm sure pypy project members have a better idea of what would be good for
gsoc students to work on?


cu,




On Tue, Mar 2, 2010 at 3:51 PM, Ren? Dudfield <renesd at gmail.com> wrote:

> hi,
>
> gsoc is on again... for your information the python mailing list for it is
> here:
>     http://mail.python.org/mailman/listinfo/soc2010-mentors
>
> It would be good to have a wiki page(or blog post? ... err a text file
> available via http) of project ideas for pypy... if there are any pypy
> people mentoring that is.  With the pygame project last year, we found
> having a project page for suggestions of acceptable projects helped students
> pick things they were interested in, and things the project would be happy
> with.
>
>
>
> cu
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20100302/89e0d4bf/attachment.html>

From fijall at gmail.com  Tue Mar  2 19:38:33 2010
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Tue, 2 Mar 2010 11:38:33 -0700
Subject: [pypy-dev] gsoc for pypy...
In-Reply-To: <64ddb72c1003021006r6dc70afdj23e804490909ea@mail.gmail.com>
References: <64ddb72c1003020751r2bc5d26hfd0f3f0eff80c47a@mail.gmail.com> 
	<64ddb72c1003021006r6dc70afdj23e804490909ea@mail.gmail.com>
Message-ID: <693bc9ab1003021038h6ac15c0dqcbe25ca07413026f@mail.gmail.com>

Hey.

Thanks for posting that! I think it makes sense to discuss a bit
whether stuff makes or does not make sense.

On Tue, Mar 2, 2010 at 11:06 AM, Ren? Dudfield <renesd at gmail.com> wrote:
> hello again.
>
> To start the gsoc ideas list off...
>
> - port jit to 64 bit (maybe too short for 3 month project, if so start on
> ARM after amd64?).

Not sure if it's too short. It's too short for core dev 3 month
project, but for student it's ideal.

> - speed up ctypes wrapper.

A bit hard, but also JIT-cooperation.

> - GIL removal work.

Hard/researchy

> - python2.6/2.7/py3k features.
> - ctypes bindings for database adaptors (would be good for ironpython,
> jython, and even cpython too).

that's not strictly pypy-related. I would like such projects to be
under PSF umbrella. Also there is a question of maintainability.

> - cpython bridge module.? Either load pypy as cpython extension or the
> otherway around.? To allow gradual porting.
> - ironclad port to pypy.? To allow loading cpython C extensions.? eg,
> ironpython can run numpy with this.

This is probably interesting, might also be a bit hard.

> - revive javascript/flex backend.

please no :)

> - improvements to java/.net backend.

Honestly, we had problems with maintability of new backends. Also .net
is sort of done (except .net bindings).

> - AOT compilation research - to allow compiling to iphone for example.

that's research, possibly interesting though.

>
> I guess this list could also be put on a page looking for sponsors?? To give
> sponsors something concrete they can pay a contract for.? eg, 'approximately
> 2-3 months work(24,000 euros) to get X benefit, as well as a logo+link on
> our sponsors page... etc.'

Sure. Although note that so far we did not get any feedback on
sponsoring blog post.

>
>
> Anyway... those are some of the areas I can think of from past discussions.
> I'm sure pypy project members have a better idea of what would be good for
> gsoc students to work on?

How about investigating (again) llvm jit backend?

Thanks for starting the discussion!

Cheers,
fijal


From fijall at gmail.com  Tue Mar  2 20:01:23 2010
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Tue, 2 Mar 2010 12:01:23 -0700
Subject: [pypy-dev] gsoc for pypy...
In-Reply-To: <40f323d01003021056l7badac4ft99f57c971a8507e6@mail.gmail.com>
References: <64ddb72c1003020751r2bc5d26hfd0f3f0eff80c47a@mail.gmail.com> 
	<64ddb72c1003021006r6dc70afdj23e804490909ea@mail.gmail.com> 
	<693bc9ab1003021038h6ac15c0dqcbe25ca07413026f@mail.gmail.com> 
	<40f323d01003021056l7badac4ft99f57c971a8507e6@mail.gmail.com>
Message-ID: <693bc9ab1003021101s5ea358f8s3705fb40bf489271@mail.gmail.com>

On Tue, Mar 2, 2010 at 11:56 AM, Benoit Boissinot
<bboissin+pypy at gmail.com> wrote:
> On Tue, Mar 2, 2010 at 7:38 PM, Maciej Fijalkowski <fijall at gmail.com> wrote:
>> Hey.
>>
>>> Anyway... those are some of the areas I can think of from past discussions.
>>> I'm sure pypy project members have a better idea of what would be good for
>>> gsoc students to work on?
>>
>> How about investigating (again) llvm jit backend?
>
> Is there a summary of what the problems were? I guess the lack of
> maturity of the JIT engine was one (as experienced by
> unladed-swallow).
>
> cheers,
>
> Benoit
>

Not written. Bugs and immaturity, yes. The precise issue we run into
was tail calls not working, but this one is supposed to be fixed by
now. Having US team on LLVM JIT team is certainly a big advantage.

I suppose next step is to try to push it and see what's next.

Cheers,
fijal


From bboissin+pypy at gmail.com  Tue Mar  2 19:56:59 2010
From: bboissin+pypy at gmail.com (Benoit Boissinot)
Date: Tue, 2 Mar 2010 19:56:59 +0100
Subject: [pypy-dev] gsoc for pypy...
In-Reply-To: <693bc9ab1003021038h6ac15c0dqcbe25ca07413026f@mail.gmail.com>
References: <64ddb72c1003020751r2bc5d26hfd0f3f0eff80c47a@mail.gmail.com>
	<64ddb72c1003021006r6dc70afdj23e804490909ea@mail.gmail.com>
	<693bc9ab1003021038h6ac15c0dqcbe25ca07413026f@mail.gmail.com>
Message-ID: <40f323d01003021056l7badac4ft99f57c971a8507e6@mail.gmail.com>

On Tue, Mar 2, 2010 at 7:38 PM, Maciej Fijalkowski <fijall at gmail.com> wrote:
> Hey.
>
>> Anyway... those are some of the areas I can think of from past discussions.
>> I'm sure pypy project members have a better idea of what would be good for
>> gsoc students to work on?
>
> How about investigating (again) llvm jit backend?

Is there a summary of what the problems were? I guess the lack of
maturity of the JIT engine was one (as experienced by
unladed-swallow).

cheers,

Benoit


From arigo at tunes.org  Tue Mar  2 20:22:51 2010
From: arigo at tunes.org (Armin Rigo)
Date: Tue, 2 Mar 2010 20:22:51 +0100
Subject: [pypy-dev] gsoc for pypy...
In-Reply-To: <693bc9ab1003021101s5ea358f8s3705fb40bf489271@mail.gmail.com>
References: <64ddb72c1003020751r2bc5d26hfd0f3f0eff80c47a@mail.gmail.com>
	<64ddb72c1003021006r6dc70afdj23e804490909ea@mail.gmail.com>
	<693bc9ab1003021038h6ac15c0dqcbe25ca07413026f@mail.gmail.com>
	<40f323d01003021056l7badac4ft99f57c971a8507e6@mail.gmail.com>
	<693bc9ab1003021101s5ea358f8s3705fb40bf489271@mail.gmail.com>
Message-ID: <20100302192251.GA9739@code0.codespeak.net>

Hi,

On Tue, Mar 02, 2010 at 12:01:23PM -0700, Maciej Fijalkowski wrote:
> >> How about investigating (again) llvm jit backend?
> (...)
> I suppose next step is to try to push it and see what's next.

Yes, but I bet from experience that we're going to run after a few days'
work into the next issue.  We've already done that 3 or 4 times with
llvm, so now I'm honestly a bit fed up.  For comparison, gcc gives code
that is a bit slower but at least it is really a solid bug-free project.


A bientot,

Armin.


From arigo at tunes.org  Tue Mar  2 20:35:48 2010
From: arigo at tunes.org (Armin Rigo)
Date: Tue, 2 Mar 2010 20:35:48 +0100
Subject: [pypy-dev] gsoc for pypy...
In-Reply-To: <20100302192251.GA9739@code0.codespeak.net>
References: <64ddb72c1003020751r2bc5d26hfd0f3f0eff80c47a@mail.gmail.com>
	<64ddb72c1003021006r6dc70afdj23e804490909ea@mail.gmail.com>
	<693bc9ab1003021038h6ac15c0dqcbe25ca07413026f@mail.gmail.com>
	<40f323d01003021056l7badac4ft99f57c971a8507e6@mail.gmail.com>
	<693bc9ab1003021101s5ea358f8s3705fb40bf489271@mail.gmail.com>
	<20100302192251.GA9739@code0.codespeak.net>
Message-ID: <20100302193547.GA12258@code0.codespeak.net>

Re-hi,

On Tue, Mar 02, 2010 at 08:22:51PM +0100, Armin Rigo wrote:
> Yes, but I bet from experience that we're going to run after a few days'
> work into the next issue.  We've already done that 3 or 4 times with
> llvm, so now I'm honestly a bit fed up.  For comparison, gcc gives code
> that is a bit slower but at least it is really a solid bug-free project.

Let me make this statement more precise: I mean for the non-JIT backends
of PyPy, using the llvm backend of PyPy gaves results that were
typically a little bit faster than using the C backend, when compiling
with llvm-gcc.  However, with the various generations of our JIT, using
a JIT llvm backend always ran into llvm bugs.


A bientot,

Armin.


From bboissin+pypy at gmail.com  Tue Mar  2 20:40:49 2010
From: bboissin+pypy at gmail.com (Benoit Boissinot)
Date: Tue, 2 Mar 2010 20:40:49 +0100
Subject: [pypy-dev] gsoc for pypy...
In-Reply-To: <20100302193547.GA12258@code0.codespeak.net>
References: <64ddb72c1003020751r2bc5d26hfd0f3f0eff80c47a@mail.gmail.com>
	<64ddb72c1003021006r6dc70afdj23e804490909ea@mail.gmail.com>
	<693bc9ab1003021038h6ac15c0dqcbe25ca07413026f@mail.gmail.com>
	<40f323d01003021056l7badac4ft99f57c971a8507e6@mail.gmail.com>
	<693bc9ab1003021101s5ea358f8s3705fb40bf489271@mail.gmail.com>
	<20100302192251.GA9739@code0.codespeak.net>
	<20100302193547.GA12258@code0.codespeak.net>
Message-ID: <40f323d01003021140y5ee5f5a6i3ca65a9f4ddf2b4b@mail.gmail.com>

On Tue, Mar 2, 2010 at 8:35 PM, Armin Rigo <arigo at tunes.org> wrote:
> Re-hi,
>
> On Tue, Mar 02, 2010 at 08:22:51PM +0100, Armin Rigo wrote:
>> Yes, but I bet from experience that we're going to run after a few days'
>> work into the next issue. ?We've already done that 3 or 4 times with
>> llvm, so now I'm honestly a bit fed up. ?For comparison, gcc gives code
>> that is a bit slower but at least it is really a solid bug-free project.
>
> Let me make this statement more precise: I mean for the non-JIT backends
> of PyPy, using the llvm backend of PyPy gaves results that were
> typically a little bit faster than using the C backend, when compiling
> with llvm-gcc. ?However, with the various generations of our JIT, using
> a JIT llvm backend always ran into llvm bugs.

On the other hand, it looks like the US guys made it production
quality, isn't it?
(At least it seems they generate correct code)

Benoit


From arigo at tunes.org  Tue Mar  2 20:51:35 2010
From: arigo at tunes.org (Armin Rigo)
Date: Tue, 2 Mar 2010 20:51:35 +0100
Subject: [pypy-dev] gsoc for pypy...
In-Reply-To: <40f323d01003021140y5ee5f5a6i3ca65a9f4ddf2b4b@mail.gmail.com>
References: <64ddb72c1003020751r2bc5d26hfd0f3f0eff80c47a@mail.gmail.com>
	<64ddb72c1003021006r6dc70afdj23e804490909ea@mail.gmail.com>
	<693bc9ab1003021038h6ac15c0dqcbe25ca07413026f@mail.gmail.com>
	<40f323d01003021056l7badac4ft99f57c971a8507e6@mail.gmail.com>
	<693bc9ab1003021101s5ea358f8s3705fb40bf489271@mail.gmail.com>
	<20100302192251.GA9739@code0.codespeak.net>
	<20100302193547.GA12258@code0.codespeak.net>
	<40f323d01003021140y5ee5f5a6i3ca65a9f4ddf2b4b@mail.gmail.com>
Message-ID: <20100302195134.GA13608@code0.codespeak.net>

Hi,

On Tue, Mar 02, 2010 at 08:40:49PM +0100, Benoit Boissinot wrote:
> On the other hand, it looks like the US guys made it production
> quality, isn't it?
> (At least it seems they generate correct code)

Yes.  Well, I suppose it should be ok if a student wants to work on it,
though I remain a bit skeptical (maybe wrongly).


A bientot,

Armin.


From benjamin at python.org  Tue Mar  2 22:46:24 2010
From: benjamin at python.org (Benjamin Peterson)
Date: Tue, 2 Mar 2010 15:46:24 -0600
Subject: [pypy-dev] Release and tags
In-Reply-To: <693bc9ab1003012050q39f4a2a8odd009c58b9375c67@mail.gmail.com>
References: <693bc9ab1003012050q39f4a2a8odd009c58b9375c67@mail.gmail.com>
Message-ID: <1afaf6161003021346x5c4af40dxf411226f816f096c@mail.gmail.com>

2010/3/1 Maciej Fijalkowski <fijall at gmail.com>:
> Hello.
>
> Can we copy trunk -> dist for the release, so we don't block any new
> development because of the release? There was the point of dist, even
> if we forgot it completely.

Maybe we should copy it to branch/release-1.2maint or some such name?
We could back port critical bug fixes.



-- 
Regards,
Benjamin


From fijall at gmail.com  Tue Mar  2 22:51:33 2010
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Tue, 2 Mar 2010 14:51:33 -0700
Subject: [pypy-dev] Release and tags
In-Reply-To: <1afaf6161003021346x5c4af40dxf411226f816f096c@mail.gmail.com>
References: <693bc9ab1003012050q39f4a2a8odd009c58b9375c67@mail.gmail.com> 
	<1afaf6161003021346x5c4af40dxf411226f816f096c@mail.gmail.com>
Message-ID: <693bc9ab1003021351k55248aa9mdf29cec3b92690db@mail.gmail.com>

This is already done. However, I wonder what's the purpose of dist.

On Tue, Mar 2, 2010 at 2:46 PM, Benjamin Peterson <benjamin at python.org> wrote:
> 2010/3/1 Maciej Fijalkowski <fijall at gmail.com>:
>> Hello.
>>
>> Can we copy trunk -> dist for the release, so we don't block any new
>> development because of the release? There was the point of dist, even
>> if we forgot it completely.
>
> Maybe we should copy it to branch/release-1.2maint or some such name?
> We could back port critical bug fixes.
>
>
>
> --
> Regards,
> Benjamin
>


From arigo at tunes.org  Tue Mar  2 22:56:33 2010
From: arigo at tunes.org (Armin Rigo)
Date: Tue, 2 Mar 2010 22:56:33 +0100
Subject: [pypy-dev] Release and tags
In-Reply-To: <693bc9ab1003021351k55248aa9mdf29cec3b92690db@mail.gmail.com>
References: <693bc9ab1003012050q39f4a2a8odd009c58b9375c67@mail.gmail.com>
	<1afaf6161003021346x5c4af40dxf411226f816f096c@mail.gmail.com>
	<693bc9ab1003021351k55248aa9mdf29cec3b92690db@mail.gmail.com>
Message-ID: <20100302215633.GA23901@code0.codespeak.net>

Hi Maciej,

On Tue, Mar 02, 2010 at 02:51:33PM -0700, Maciej Fijalkowski wrote:
> This is already done. However, I wonder what's the purpose of dist.

I'll copy from release/1.2.x into both release/1.2.0 and dist when
we really do the release.  That's supposedly the purpose of dist,
I guess.


A bientot,

Armin.


From fijall at gmail.com  Wed Mar  3 06:54:43 2010
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Tue, 2 Mar 2010 22:54:43 -0700
Subject: [pypy-dev] buildbot question
Message-ID: <693bc9ab1003022154g5f181018y1b4600f33271b0a8@mail.gmail.com>

any particular reason why pypy-c tests are run twice each night? once
for app tests and once for pypy-c jit tests?


From arigo at tunes.org  Wed Mar  3 09:40:36 2010
From: arigo at tunes.org (Armin Rigo)
Date: Wed, 3 Mar 2010 09:40:36 +0100
Subject: [pypy-dev] buildbot question
In-Reply-To: <693bc9ab1003022154g5f181018y1b4600f33271b0a8@mail.gmail.com>
References: <693bc9ab1003022154g5f181018y1b4600f33271b0a8@mail.gmail.com>
Message-ID: <20100303084036.GA8881@code0.codespeak.net>

Hi Maciek,

On Tue, Mar 02, 2010 at 10:54:43PM -0700, Maciej Fijalkowski wrote:
> any particular reason why pypy-c tests are run twice each night? once
> for app tests and once for pypy-c jit tests?

Once with a pypy-c and once with a pypy-c-jit, obviously.  It used to
make sense, and probably still does a bit.


Armin


From arigo at tunes.org  Wed Mar  3 10:14:35 2010
From: arigo at tunes.org (Armin Rigo)
Date: Wed, 3 Mar 2010 10:14:35 +0100
Subject: [pypy-dev] =?iso-8859-1?q?cobra_server_at_the_Uni_D=FCsseldorf?=
In-Reply-To: <4B73F4D7.1040508@gmx.de>
References: <4B73F4D7.1040508@gmx.de>
Message-ID: <20100303091434.GA11020@code0.codespeak.net>

Hi Carl Friedrich,

On Thu, Feb 11, 2010 at 01:15:19PM +0100, Carl Friedrich Bolz wrote:
> kept and move them elsewhere? We plan to shut Cobra down on the 19th of 
> February, if somebody cannot make it till then, please send me a mail.

Cobra is still around, but wyvern's hard drive seems to have died
yesterday.  At least it ended up disconnecting itself from the buildbot
metaserver.  You (cfbolz) can always try restarting the machine and see
if it helps, but I doubt it a bit.  We need a solution for that issue; I
suppose that waiting for the upcoming server at OpenEnd would work.  In
the meantime I listed cobra as an alternate server on which to run these
tests (it was listed on some builds, but not all).


A bientot,

Armin.


From cfbolz at gmx.de  Wed Mar  3 10:47:10 2010
From: cfbolz at gmx.de (Carl Friedrich Bolz)
Date: Wed, 03 Mar 2010 10:47:10 +0100
Subject: [pypy-dev] =?iso-8859-1?q?cobra_server_at_the_Uni_D=FCsseldorf?=
In-Reply-To: <20100303091434.GA11020@code0.codespeak.net>
References: <4B73F4D7.1040508@gmx.de>
	<20100303091434.GA11020@code0.codespeak.net>
Message-ID: <4B8E301E.30303@gmx.de>

Hi Armin,

On 03/03/2010 10:14 AM, Armin Rigo wrote:
> On Thu, Feb 11, 2010 at 01:15:19PM +0100, Carl Friedrich Bolz wrote:
>> kept and move them elsewhere? We plan to shut Cobra down on the 19th of
>> February, if somebody cannot make it till then, please send me a mail.
>
> Cobra is still around, but wyvern's hard drive seems to have died
> yesterday.  At least it ended up disconnecting itself from the buildbot
> metaserver.  You (cfbolz) can always try restarting the machine and see
> if it helps, but I doubt it a bit.  We need a solution for that issue; I
> suppose that waiting for the upcoming server at OpenEnd would work.  In
> the meantime I listed cobra as an alternate server on which to run these
> tests (it was listed on some builds, but not all).

I asked Jens to wait a bit with the Cobra migration, so that we can do 
the release and have at least one machines that still runs our tests 
nightly. We should try to get a new hard drive for wyvern and reinstall 
it, but I guess only after the release.

Cheers,

Carl Friedrich


From arigo at tunes.org  Thu Mar  4 13:18:13 2010
From: arigo at tunes.org (Armin Rigo)
Date: Thu, 4 Mar 2010 13:18:13 +0100
Subject: [pypy-dev] next Leysin sprint?
Message-ID: <20100304121813.GA896@code0.codespeak.net>

Hi all,

This is a quick poll to know if there would be people interested in
attending the next sprint in Switzerland, one week after Easter, roughly
10-17th April 2010.

I guess the conditions for it to happen are that I am not the only "core
developer" (cfbolz?), and there are at least a couple of extra people
with whom to sprint.  The sprint topics are completely open.  The basic
idea is to do the sprint in Leysin, but I am also open to the idea of
making a 2nd sprint in Bern with a topic like "let's apply the JIT to
the Smalltalk interpreter" :-)


A bientot,

Armin.


From fuzzyman at gmail.com  Thu Mar  4 14:42:25 2010
From: fuzzyman at gmail.com (Michael Foord)
Date: Thu, 4 Mar 2010 13:42:25 +0000
Subject: [pypy-dev] next Leysin sprint?
In-Reply-To: <20100304121813.GA896@code0.codespeak.net>
References: <20100304121813.GA896@code0.codespeak.net>
Message-ID: <6f4025011003040542u6fdbe0fet1554bd78bba39802@mail.gmail.com>

On 4 March 2010 12:18, Armin Rigo <arigo at tunes.org> wrote:

> Hi all,
>
> This is a quick poll to know if there would be people interested in
> attending the next sprint in Switzerland, one week after Easter, roughly
> 10-17th April 2010.
>


I can't commit to it, but am definitely interested. Especially if Carl makes
it. :-)

Michael


>
> I guess the conditions for it to happen are that I am not the only "core
> developer" (cfbolz?), and there are at least a couple of extra people
> with whom to sprint.  The sprint topics are completely open.  The basic
> idea is to do the sprint in Leysin, but I am also open to the idea of
> making a 2nd sprint in Bern with a topic like "let's apply the JIT to
> the Smalltalk interpreter" :-)
>
>
> A bientot,
>
> Armin.
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev
>



-- 
http://www.ironpythoninaction.com/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20100304/f4c2494b/attachment.html>

From arigo at tunes.org  Fri Mar  5 11:47:41 2010
From: arigo at tunes.org (Armin Rigo)
Date: Fri, 5 Mar 2010 11:47:41 +0100
Subject: [pypy-dev] Getting anywhere?
Message-ID: <20100305104741.GA17903@code0.codespeak.net>

Hi Maciek,

It's now been two days since you last checked in some text in
svn/user/fijal/website/.  Can you please confirm that you are still
working on it?  I have nothing against general hacking, but just to make
it clear, I plan to not merge most of what we do now inside
'release/1.2.x', so that's not release work.


A bientot,

Armin.


From arigo at tunes.org  Sat Mar  6 22:27:45 2010
From: arigo at tunes.org (Armin Rigo)
Date: Sat, 6 Mar 2010 22:27:45 +0100
Subject: [pypy-dev] speed.pypy.org update
Message-ID: <20100306212745.GA29817@code0.codespeak.net>

Hi all,

Fijal may comment on this more, but note that he tweaked the number of
runs of the benchmarks displayed by speed.pypy.org, which explains the
jumps we see in some of them.  The positive-for-us jumps, I should add
:-)

I think that we should remove the history up to today, as it makes
little sense to compare...


A bientot,

Armin.


From fijall at gmail.com  Sun Mar  7 05:09:35 2010
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Sat, 6 Mar 2010 21:09:35 -0700
Subject: [pypy-dev] speed.pypy.org update
In-Reply-To: <20100306212745.GA29817@code0.codespeak.net>
References: <20100306212745.GA29817@code0.codespeak.net>
Message-ID: <693bc9ab1003062009p6d682ddfg68fd4a94d6b97b9f@mail.gmail.com>

Hey.

Yes, sorry about not saying that upfront. I've removed --fast from
run, so our numbers looks better (because warmup is more spread over).
It makes sense to remove history IMO.

However, I plan to work a bit more on that, to completely remove
warmup time from some benchmark and to include warmup for each run for
others (like richards, html5lib), so that would be yet another removal
of history.

Right now the "average" is not that meaningful (and stddev even less,
although it's not displayed on speed), since it averages warmup across
all the run. I would like to have warmup either completely included
for each run (for benchmarks that it makes sense) or completely
separated and reported as some other variable.

Cheers,
fijal

On Sat, Mar 6, 2010 at 2:27 PM, Armin Rigo <arigo at tunes.org> wrote:
> Hi all,
>
> Fijal may comment on this more, but note that he tweaked the number of
> runs of the benchmarks displayed by speed.pypy.org, which explains the
> jumps we see in some of them. ?The positive-for-us jumps, I should add
> :-)
>
> I think that we should remove the history up to today, as it makes
> little sense to compare...
>
>
> A bientot,
>
> Armin.
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev
>


From tobami at googlemail.com  Mon Mar  8 11:29:18 2010
From: tobami at googlemail.com (Miquel Torres)
Date: Mon, 8 Mar 2010 11:29:18 +0100
Subject: [pypy-dev] speed.pypy.org update
In-Reply-To: <693bc9ab1003062009p6d682ddfg68fd4a94d6b97b9f@mail.gmail.com>
References: <20100306212745.GA29817@code0.codespeak.net>
	<693bc9ab1003062009p6d682ddfg68fd4a94d6b97b9f@mail.gmail.com>
Message-ID: <c561346d1003080229r44647a49vbd1eabc9f20d0d0f@mail.gmail.com>

Yeah, history is now not very meaningful.

While we are at it, I think we should complete the job and define a
standard set of benchmarks together with the python.org and unladen
guys. So that we can keep the data this time :-)

How were the chats about that developing, fijal?



2010/3/7 Maciej Fijalkowski <fijall at gmail.com>:
> Hey.
>
> Yes, sorry about not saying that upfront. I've removed --fast from
> run, so our numbers looks better (because warmup is more spread over).
> It makes sense to remove history IMO.
>
> However, I plan to work a bit more on that, to completely remove
> warmup time from some benchmark and to include warmup for each run for
> others (like richards, html5lib), so that would be yet another removal
> of history.
>
> Right now the "average" is not that meaningful (and stddev even less,
> although it's not displayed on speed), since it averages warmup across
> all the run. I would like to have warmup either completely included
> for each run (for benchmarks that it makes sense) or completely
> separated and reported as some other variable.
>
> Cheers,
> fijal
>
> On Sat, Mar 6, 2010 at 2:27 PM, Armin Rigo <arigo at tunes.org> wrote:
>> Hi all,
>>
>> Fijal may comment on this more, but note that he tweaked the number of
>> runs of the benchmarks displayed by speed.pypy.org, which explains the
>> jumps we see in some of them. ?The positive-for-us jumps, I should add
>> :-)
>>
>> I think that we should remove the history up to today, as it makes
>> little sense to compare...
>>
>>
>> A bientot,
>>
>> Armin.
>> _______________________________________________
>> pypy-dev at codespeak.net
>> http://codespeak.net/mailman/listinfo/pypy-dev
>>
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev


From fijall at gmail.com  Mon Mar  8 18:10:16 2010
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Mon, 8 Mar 2010 10:10:16 -0700
Subject: [pypy-dev] speed.pypy.org update
In-Reply-To: <c561346d1003080229r44647a49vbd1eabc9f20d0d0f@mail.gmail.com>
References: <20100306212745.GA29817@code0.codespeak.net>
	<693bc9ab1003062009p6d682ddfg68fd4a94d6b97b9f@mail.gmail.com> 
	<c561346d1003080229r44647a49vbd1eabc9f20d0d0f@mail.gmail.com>
Message-ID: <693bc9ab1003080910u5ab34afeq87d4f8871606fc00@mail.gmail.com>

On Mon, Mar 8, 2010 at 3:29 AM, Miquel Torres <tobami at googlemail.com> wrote:
> Yeah, history is now not very meaningful.
>
> While we are at it, I think we should complete the job and define a
> standard set of benchmarks together with the python.org and unladen
> guys. So that we can keep the data this time :-)
>
> How were the chats about that developing, fijal?
>

Progressing, but slowly. We all have slightly different goals I
believe. For example I think pypy is aiming also at people who are
doing stuff in C and would like to do python instead. US is aiming
exclusively at python programmers. (I think)

Cheers,
fijal

>
>
> 2010/3/7 Maciej Fijalkowski <fijall at gmail.com>:
>> Hey.
>>
>> Yes, sorry about not saying that upfront. I've removed --fast from
>> run, so our numbers looks better (because warmup is more spread over).
>> It makes sense to remove history IMO.
>>
>> However, I plan to work a bit more on that, to completely remove
>> warmup time from some benchmark and to include warmup for each run for
>> others (like richards, html5lib), so that would be yet another removal
>> of history.
>>
>> Right now the "average" is not that meaningful (and stddev even less,
>> although it's not displayed on speed), since it averages warmup across
>> all the run. I would like to have warmup either completely included
>> for each run (for benchmarks that it makes sense) or completely
>> separated and reported as some other variable.
>>
>> Cheers,
>> fijal
>>
>> On Sat, Mar 6, 2010 at 2:27 PM, Armin Rigo <arigo at tunes.org> wrote:
>>> Hi all,
>>>
>>> Fijal may comment on this more, but note that he tweaked the number of
>>> runs of the benchmarks displayed by speed.pypy.org, which explains the
>>> jumps we see in some of them. ?The positive-for-us jumps, I should add
>>> :-)
>>>
>>> I think that we should remove the history up to today, as it makes
>>> little sense to compare...
>>>
>>>
>>> A bientot,
>>>
>>> Armin.
>>> _______________________________________________
>>> pypy-dev at codespeak.net
>>> http://codespeak.net/mailman/listinfo/pypy-dev
>>>
>> _______________________________________________
>> pypy-dev at codespeak.net
>> http://codespeak.net/mailman/listinfo/pypy-dev
>


From pedronis at openend.se  Wed Mar 10 20:38:56 2010
From: pedronis at openend.se (Samuele Pedroni)
Date: Wed, 10 Mar 2010 20:38:56 +0100
Subject: [pypy-dev] interesting thread on Lambda The Ultimate on tracing
	trade-offs
Message-ID: <4B97F550.8000707@openend.se>


http://lambda-the-ultimate.org/node/3851

in case people missed it,

Samuele


From tobami at googlemail.com  Wed Mar 10 21:14:36 2010
From: tobami at googlemail.com (Miquel Torres)
Date: Wed, 10 Mar 2010 21:14:36 +0100
Subject: [pypy-dev] speed.pypy.org quick update
Message-ID: <c561346d1003101214q6dd04f51lb13defb2791d64ec@mail.gmail.com>

Hi!

I wanted to explain a couple of things about the speed website:

- New feature: the Timeline view now defaults to a plot grid, showing
all benchmarks at the same time. It was a feature request made more
than once, so depending on personal tastes, you can bookmark either
/overview/ or /timeline/. Thanks go to nsf for helping with the
implementation.
- The code has now moved to github as Codespeed, a benchmark
visualization framework (http://github.com/tobami/codespeed)
- I have updated speed.pypy.org with version 0.3. Much of the work has
been under the hood to make it feasible for other projects to use
codespeed as a framework.

For those interested in further development you can go to the releases
wiki (still a work in progress):
http://wiki.github.com/tobami/codespeed/releases

Next in the line are some DB changes to be able to save standard
deviation data and the like. Long term goals besides world domination
are integration with buildbot and similarly unrealistic things.
Feedback is always welcome.

Cheers,
Miquel


From bokr at oz.net  Thu Mar 11 00:45:50 2010
From: bokr at oz.net (Bengt Richter)
Date: Wed, 10 Mar 2010 15:45:50 -0800
Subject: [pypy-dev] speed.pypy.org quick update
In-Reply-To: <c561346d1003101214q6dd04f51lb13defb2791d64ec@mail.gmail.com>
References: <c561346d1003101214q6dd04f51lb13defb2791d64ec@mail.gmail.com>
Message-ID: <4B982F2E.4030304@oz.net>

On 03/10/2010 12:14 PM Miquel Torres wrote:
> Hi!
> 
> I wanted to explain a couple of things about the speed website:
> 
> - New feature: the Timeline view now defaults to a plot grid, showing
> all benchmarks at the same time. It was a feature request made more
> than once, so depending on personal tastes, you can bookmark either
> /overview/ or /timeline/. Thanks go to nsf for helping with the
> implementation.
> - The code has now moved to github as Codespeed, a benchmark
> visualization framework (http://github.com/tobami/codespeed)
> - I have updated speed.pypy.org with version 0.3. Much of the work has
> been under the hood to make it feasible for other projects to use
> codespeed as a framework.
> 
> For those interested in further development you can go to the releases
> wiki (still a work in progress):
> http://wiki.github.com/tobami/codespeed/releases
> 
> Next in the line are some DB changes to be able to save standard
> deviation data and the like. Long term goals besides world domination
> are integration with buildbot and similarly unrealistic things.
> Feedback is always welcome.

Nice looking stuff. But a couple comments:

1. IMO standard deviation is too often worse than useless, since it hides
    the true nature of the distribution. I think the assumption of normalcy
    is highly suspect for benchmark timings, and pruning may hide interesting clusters.

    I prefer to look at scattergrams, where things like clustering and correlations
    are easily apparent to the eye, as well as the amount of data (assuming a good
    mapping of density to visuals).

2. IMO benchmark timings are like travel times, comparing different vehicles.
    (pypy with jit being a vehicle capable of dynamic self-modification ;-)
    E.g., which part of travel from Stockholm to Paris would you concentrate
    on improving to improve the overall result? How about travel from Brussels to Paris?
    Or Paris to Sydney? ;-P Different things come into play in different benchmarks/trips.
    A Porsche Turbo and a 2CV will both have to wait for a ferry, if that's part of the trip.

    IOW, it would be nice to see total time broken down somehow, to see what's really
    happening.

    Don't get me wrong, the total times are certainly useful indicators of progress
    (which has been amazing).

3. Speed is ds/dt and you are showing the integral of dt/ds over the trip distance to get time.
    A 25% improvement in total time is not a 25% improvement in speed. I.e., (if you define
    improvement as a percentage change in a desired direction), for e.g. 25%:
    distance/(0.75*time) != 1.25*(distance/time).

    IMO 'speed' (the implication to me in the name speed.pypy.org) would be benchmarks/time
    more appropriately than time/benchmark.

    Both measures are useful, but time percentages are easy to mis{use,construe} ;-)

4. Is there any memory footprint data?

Regards,
Bengt Richter



From fijall at gmail.com  Thu Mar 11 01:32:52 2010
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Wed, 10 Mar 2010 17:32:52 -0700
Subject: [pypy-dev] speed.pypy.org quick update
In-Reply-To: <4B982F2E.4030304@oz.net>
References: <c561346d1003101214q6dd04f51lb13defb2791d64ec@mail.gmail.com> 
	<4B982F2E.4030304@oz.net>
Message-ID: <693bc9ab1003101632p79f9254cj3a2e1c244314fc9b@mail.gmail.com>

Hey.

I'll answer questions that are relevant to benchmarks themselves and
not running.

On Wed, Mar 10, 2010 at 4:45 PM, Bengt Richter <bokr at oz.net> wrote:
> On 03/10/2010 12:14 PM Miquel Torres wrote:
>> Hi!
>>
>> I wanted to explain a couple of things about the speed website:
>>
>> - New feature: the Timeline view now defaults to a plot grid, showing
>> all benchmarks at the same time. It was a feature request made more
>> than once, so depending on personal tastes, you can bookmark either
>> /overview/ or /timeline/. Thanks go to nsf for helping with the
>> implementation.
>> - The code has now moved to github as Codespeed, a benchmark
>> visualization framework (http://github.com/tobami/codespeed)
>> - I have updated speed.pypy.org with version 0.3. Much of the work has
>> been under the hood to make it feasible for other projects to use
>> codespeed as a framework.
>>
>> For those interested in further development you can go to the releases
>> wiki (still a work in progress):
>> http://wiki.github.com/tobami/codespeed/releases
>>
>> Next in the line are some DB changes to be able to save standard
>> deviation data and the like. Long term goals besides world domination
>> are integration with buildbot and similarly unrealistic things.
>> Feedback is always welcome.
>
> Nice looking stuff. But a couple comments:
>
> 1. IMO standard deviation is too often worse than useless, since it hides
> ? ?the true nature of the distribution. I think the assumption of normalcy
> ? ?is highly suspect for benchmark timings, and pruning may hide interesting clusters.
>
> ? ?I prefer to look at scattergrams, where things like clustering and correlations
> ? ?are easily apparent to the eye, as well as the amount of data (assuming a good
> ? ?mapping of density to visuals).

That's true. In general a benchmark run over time is a period of
warmup, when JIT compiles assembler followed by thing that can be
described by average and std devation. Personally I would like to have
those 3 measures separated, but didn't implement that yet (it has also
some interesting statistical questions involved). Std deviation is
useful to get whether a difference was meaningful of certain checkin
or just noise.

>
> 2. IMO benchmark timings are like travel times, comparing different vehicles.
> ? ?(pypy with jit being a vehicle capable of dynamic self-modification ;-)
> ? ?E.g., which part of travel from Stockholm to Paris would you concentrate
> ? ?on improving to improve the overall result? How about travel from Brussels to Paris?
> ? ?Or Paris to Sydney? ;-P Different things come into play in different benchmarks/trips.
> ? ?A Porsche Turbo and a 2CV will both have to wait for a ferry, if that's part of the trip.
>
> ? ?IOW, it would be nice to see total time broken down somehow, to see what's really
> ? ?happening.

I can't agree more with that. We already do split time when we perform
benchmarks by hand, but they're not yet integrated into the whole
nightly run. Total time is what users see though, that's why our
public site is focused on that. I want more information available, but
we have only limited amount of manpower and miquel already did quite
amazing job in my opinion :-) We'll probably go into more details.

The part we want to focus on past-release is speeding up certain parts
of tracing as well as limiting it's GC pressure. As you can see, the
split would be very useful for our development.

>
> ? ?Don't get me wrong, the total times are certainly useful indicators of progress
> ? ?(which has been amazing).
>
> 3. Speed is ds/dt and you are showing the integral of dt/ds over the trip distance to get time.
> ? ?A 25% improvement in total time is not a 25% improvement in speed. I.e., (if you define
> ? ?improvement as a percentage change in a desired direction), for e.g. 25%:
> ? ?distance/(0.75*time) != 1.25*(distance/time).
>
> ? ?IMO 'speed' (the implication to me in the name speed.pypy.org) would be benchmarks/time
> ? ?more appropriately than time/benchmark.
>
> ? ?Both measures are useful, but time percentages are easy to mis{use,construe} ;-)

That's correct.

Benchmarks are in general very easy to lie about and they're by
definition flawed. That's why I always include raw data when I publish
stuff on the blog, so people can work on it themselves.

>
> 4. Is there any memory footprint data?
>

No. Memory measurment is hard and it's even less useful without
breaking down. Those particular benchmarks are not very good basis for
memory measurment - in case of pypy you would mostly observe the
default allocated memory (which is roughly 10M for the interpreter +
16M for semispace GC + cache for nursery).

Also our GC is of a kind that it can run faster if you give it more
memory (not that we use this feature, but it's possible).

Cheers,
fijal


From fijall at gmail.com  Thu Mar 11 16:45:37 2010
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Thu, 11 Mar 2010 08:45:37 -0700
Subject: [pypy-dev] [pypy-svn] r72100 - pypy/trunk/pypy/tool
In-Reply-To: <20100311131745.993F9282BD7@codespeak.net>
References: <20100311131745.993F9282BD7@codespeak.net>
Message-ID: <693bc9ab1003110745k480dc624kddb988605b9d5bbb@mail.gmail.com>

This comes with no test.

On Thu, Mar 11, 2010 at 6:17 AM,  <arigo at codespeak.net> wrote:
> Author: arigo
> Date: Thu Mar 11 14:17:44 2010
> New Revision: 72100
>
> Modified:
> ? pypy/trunk/pypy/tool/package.py
> Log:
> Accept a third argument: the name to give to the pypy-c.
>
>
> Modified: pypy/trunk/pypy/tool/package.py
> ==============================================================================
> --- pypy/trunk/pypy/tool/package.py ? ? (original)
> +++ pypy/trunk/pypy/tool/package.py ? ? Thu Mar 11 14:17:44 2010
> @@ -2,7 +2,7 @@
> ?""" A sample script that packages PyPy, provided that it's already built.
> ?Usage:
>
> -package.py pypydir [name-of-archive]
> +package.py pypydir [name-of-archive] [name-of-pypy-c]
> ?"""
>
> ?import autopath
> @@ -31,7 +31,7 @@
> ?class PyPyCNotFound(Exception):
> ? ? pass
>
> -def main(basedir, name='pypy-nightly'):
> +def main(basedir, name='pypy-nightly', rename_pypy_c='pypy-c'):
> ? ? basedir = py.path.local(basedir)
> ? ? pypy_c = basedir.join('pypy', 'translator', 'goal', 'pypy-c')
> ? ? if not pypy_c.check():
> @@ -50,11 +50,12 @@
> ? ? for file in ['LICENSE', 'README']:
> ? ? ? ? shutil.copy(str(basedir.join(file)), str(pypydir))
> ? ? pypydir.ensure('bin', dir=True)
> - ? ?shutil.copy(str(pypy_c), str(pypydir.join('bin', 'pypy-c')))
> + ? ?archive_pypy_c = pypydir.join('bin', rename_pypy_c)
> + ? ?shutil.copy(str(pypy_c), str(archive_pypy_c))
> ? ? old_dir = os.getcwd()
> ? ? try:
> ? ? ? ? os.chdir(str(builddir))
> - ? ? ? ?os.system("strip " + str(pypydir.join('bin', 'pypy-c')))
> + ? ? ? ?os.system("strip " + str(archive_pypy_c))
> ? ? ? ? os.system('tar cvjf ' + str(builddir.join(name + '.tar.bz2')) +
> ? ? ? ? ? ? ? ? ? " " + name)
> ? ? finally:
> @@ -62,10 +63,8 @@
> ? ? return builddir # for tests
>
> ?if __name__ == '__main__':
> - ? ?if len(sys.argv) == 1 or len(sys.argv) > 3:
> + ? ?if len(sys.argv) == 1 or len(sys.argv) > 4:
> ? ? ? ? print >>sys.stderr, __doc__
> ? ? ? ? sys.exit(1)
> - ? ?elif len(sys.argv) == 2:
> - ? ? ? ?main(sys.argv[1])
> ? ? else:
> - ? ? ? ?main(sys.argv[1], sys.argv[2])
> + ? ? ? ?main(*sys.argv[1:])
> _______________________________________________
> pypy-svn mailing list
> pypy-svn at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-svn
>


From renesd at gmail.com  Fri Mar 12 11:49:54 2010
From: renesd at gmail.com (=?ISO-8859-1?Q?Ren=E9_Dudfield?=)
Date: Fri, 12 Mar 2010 10:49:54 +0000
Subject: [pypy-dev] speed.pypy.org quick update
In-Reply-To: <693bc9ab1003101632p79f9254cj3a2e1c244314fc9b@mail.gmail.com>
References: <c561346d1003101214q6dd04f51lb13defb2791d64ec@mail.gmail.com>
	<4B982F2E.4030304@oz.net>
	<693bc9ab1003101632p79f9254cj3a2e1c244314fc9b@mail.gmail.com>
Message-ID: <64ddb72c1003120249pe469a7bv73a10a674b9c5771@mail.gmail.com>

btw, for python memory usage on linux
/proc/PID/status

Here is some code for linux...

wget http://rene.f0o.com/~rene/stuff/memory_usage.py

>>> import memory_usage
>>> bytes_of_resident_memory = memory_usage.resident()


Should be easy enough to add that to benchmarks at the start and end?  Maybe
calling it in the middle would be a little harder... but not too hard.


TODO: Would need to be updated for other platforms, and support measuring
child processes, tests, and code cleanup :)

cu,



On Thu, Mar 11, 2010 at 12:32 AM, Maciej Fijalkowski <fijall at gmail.com>wrote:

> Hey.
>
> I'll answer questions that are relevant to benchmarks themselves and
> not running.
>
> On Wed, Mar 10, 2010 at 4:45 PM, Bengt Richter <bokr at oz.net> wrote:
> > On 03/10/2010 12:14 PM Miquel Torres wrote:
> >> Hi!
> >>
> >> I wanted to explain a couple of things about the speed website:
> >>
> >> - New feature: the Timeline view now defaults to a plot grid, showing
> >> all benchmarks at the same time. It was a feature request made more
> >> than once, so depending on personal tastes, you can bookmark either
> >> /overview/ or /timeline/. Thanks go to nsf for helping with the
> >> implementation.
> >> - The code has now moved to github as Codespeed, a benchmark
> >> visualization framework (http://github.com/tobami/codespeed)
> >> - I have updated speed.pypy.org with version 0.3. Much of the work has
> >> been under the hood to make it feasible for other projects to use
> >> codespeed as a framework.
> >>
> >> For those interested in further development you can go to the releases
> >> wiki (still a work in progress):
> >> http://wiki.github.com/tobami/codespeed/releases
> >>
> >> Next in the line are some DB changes to be able to save standard
> >> deviation data and the like. Long term goals besides world domination
> >> are integration with buildbot and similarly unrealistic things.
> >> Feedback is always welcome.
> >
> > Nice looking stuff. But a couple comments:
> >
> > 1. IMO standard deviation is too often worse than useless, since it hides
> >    the true nature of the distribution. I think the assumption of
> normalcy
> >    is highly suspect for benchmark timings, and pruning may hide
> interesting clusters.
> >
> >    I prefer to look at scattergrams, where things like clustering and
> correlations
> >    are easily apparent to the eye, as well as the amount of data
> (assuming a good
> >    mapping of density to visuals).
>
> That's true. In general a benchmark run over time is a period of
> warmup, when JIT compiles assembler followed by thing that can be
> described by average and std devation. Personally I would like to have
> those 3 measures separated, but didn't implement that yet (it has also
> some interesting statistical questions involved). Std deviation is
> useful to get whether a difference was meaningful of certain checkin
> or just noise.
>
> >
> > 2. IMO benchmark timings are like travel times, comparing different
> vehicles.
> >    (pypy with jit being a vehicle capable of dynamic self-modification
> ;-)
> >    E.g., which part of travel from Stockholm to Paris would you
> concentrate
> >    on improving to improve the overall result? How about travel from
> Brussels to Paris?
> >    Or Paris to Sydney? ;-P Different things come into play in different
> benchmarks/trips.
> >    A Porsche Turbo and a 2CV will both have to wait for a ferry, if
> that's part of the trip.
> >
> >    IOW, it would be nice to see total time broken down somehow, to see
> what's really
> >    happening.
>
> I can't agree more with that. We already do split time when we perform
> benchmarks by hand, but they're not yet integrated into the whole
> nightly run. Total time is what users see though, that's why our
> public site is focused on that. I want more information available, but
> we have only limited amount of manpower and miquel already did quite
> amazing job in my opinion :-) We'll probably go into more details.
>
> The part we want to focus on past-release is speeding up certain parts
> of tracing as well as limiting it's GC pressure. As you can see, the
> split would be very useful for our development.
>
> >
> >    Don't get me wrong, the total times are certainly useful indicators of
> progress
> >    (which has been amazing).
> >
> > 3. Speed is ds/dt and you are showing the integral of dt/ds over the trip
> distance to get time.
> >    A 25% improvement in total time is not a 25% improvement in speed.
> I.e., (if you define
> >    improvement as a percentage change in a desired direction), for e.g.
> 25%:
> >    distance/(0.75*time) != 1.25*(distance/time).
> >
> >    IMO 'speed' (the implication to me in the name speed.pypy.org) would
> be benchmarks/time
> >    more appropriately than time/benchmark.
> >
> >    Both measures are useful, but time percentages are easy to
> mis{use,construe} ;-)
>
> That's correct.
>
> Benchmarks are in general very easy to lie about and they're by
> definition flawed. That's why I always include raw data when I publish
> stuff on the blog, so people can work on it themselves.
>
> >
> > 4. Is there any memory footprint data?
> >
>
> No. Memory measurment is hard and it's even less useful without
> breaking down. Those particular benchmarks are not very good basis for
> memory measurment - in case of pypy you would mostly observe the
> default allocated memory (which is roughly 10M for the interpreter +
> 16M for semispace GC + cache for nursery).
>
> Also our GC is of a kind that it can run faster if you give it more
> memory (not that we use this feature, but it's possible).
>
> Cheers,
> fijal
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20100312/b5d4f914/attachment.html>

From renesd at gmail.com  Fri Mar 12 14:52:02 2010
From: renesd at gmail.com (=?ISO-8859-1?Q?Ren=E9_Dudfield?=)
Date: Fri, 12 Mar 2010 13:52:02 +0000
Subject: [pypy-dev] memory recording... Re:  speed.pypy.org quick update
Message-ID: <64ddb72c1003120552x77fb215rd6d4b36a3ccf7dc9@mail.gmail.com>

Hi again,

trying to do some research on ways to record memory usage in an X-platform
way...

keeping my notes here:

http://renesd.blogspot.com/2010/03/memory-usage-of-processes-from-python.html

So far people have come up with these two useful projects so far:
    http://code.google.com/p/psutil/
    http://code.google.com/p/pympler/

I think psutil will have most info needed to construct a decent memory
recording module for benchmarks.  However, it includes C code, so will
probably have to rip some of the memory parts out, and maybe reimplement
with ctypes.


cu,




On Fri, Mar 12, 2010 at 10:49 AM, Ren? Dudfield <renesd at gmail.com> wrote:

> btw, for python memory usage on linux
> /proc/PID/status
>
> Here is some code for linux...
>
> wget http://rene.f0o.com/~rene/stuff/memory_usage.py<http://rene.f0o.com/%7Erene/stuff/memory_usage.py>
>
> >>> import memory_usage
> >>> bytes_of_resident_memory = memory_usage.resident()
>
>
> Should be easy enough to add that to benchmarks at the start and end?
> Maybe calling it in the middle would be a little harder... but not too hard.
>
>
> TODO: Would need to be updated for other platforms, and support measuring
> child processes, tests, and code cleanup :)
>
> cu,
>
>
>
>
> On Thu, Mar 11, 2010 at 12:32 AM, Maciej Fijalkowski <fijall at gmail.com>wrote:
>
>> Hey.
>>
>> I'll answer questions that are relevant to benchmarks themselves and
>> not running.
>>
>> On Wed, Mar 10, 2010 at 4:45 PM, Bengt Richter <bokr at oz.net> wrote:
>> > On 03/10/2010 12:14 PM Miquel Torres wrote:
>> >> Hi!
>> >>
>> >> I wanted to explain a couple of things about the speed website:
>> >>
>> >> - New feature: the Timeline view now defaults to a plot grid, showing
>> >> all benchmarks at the same time. It was a feature request made more
>> >> than once, so depending on personal tastes, you can bookmark either
>> >> /overview/ or /timeline/. Thanks go to nsf for helping with the
>> >> implementation.
>> >> - The code has now moved to github as Codespeed, a benchmark
>> >> visualization framework (http://github.com/tobami/codespeed)
>> >> - I have updated speed.pypy.org with version 0.3. Much of the work has
>> >> been under the hood to make it feasible for other projects to use
>> >> codespeed as a framework.
>> >>
>> >> For those interested in further development you can go to the releases
>> >> wiki (still a work in progress):
>> >> http://wiki.github.com/tobami/codespeed/releases
>> >>
>> >> Next in the line are some DB changes to be able to save standard
>> >> deviation data and the like. Long term goals besides world domination
>> >> are integration with buildbot and similarly unrealistic things.
>> >> Feedback is always welcome.
>> >
>> > Nice looking stuff. But a couple comments:
>> >
>> > 1. IMO standard deviation is too often worse than useless, since it
>> hides
>> >    the true nature of the distribution. I think the assumption of
>> normalcy
>> >    is highly suspect for benchmark timings, and pruning may hide
>> interesting clusters.
>> >
>> >    I prefer to look at scattergrams, where things like clustering and
>> correlations
>> >    are easily apparent to the eye, as well as the amount of data
>> (assuming a good
>> >    mapping of density to visuals).
>>
>> That's true. In general a benchmark run over time is a period of
>> warmup, when JIT compiles assembler followed by thing that can be
>> described by average and std devation. Personally I would like to have
>> those 3 measures separated, but didn't implement that yet (it has also
>> some interesting statistical questions involved). Std deviation is
>> useful to get whether a difference was meaningful of certain checkin
>> or just noise.
>>
>> >
>> > 2. IMO benchmark timings are like travel times, comparing different
>> vehicles.
>> >    (pypy with jit being a vehicle capable of dynamic self-modification
>> ;-)
>> >    E.g., which part of travel from Stockholm to Paris would you
>> concentrate
>> >    on improving to improve the overall result? How about travel from
>> Brussels to Paris?
>> >    Or Paris to Sydney? ;-P Different things come into play in different
>> benchmarks/trips.
>> >    A Porsche Turbo and a 2CV will both have to wait for a ferry, if
>> that's part of the trip.
>> >
>> >    IOW, it would be nice to see total time broken down somehow, to see
>> what's really
>> >    happening.
>>
>> I can't agree more with that. We already do split time when we perform
>> benchmarks by hand, but they're not yet integrated into the whole
>> nightly run. Total time is what users see though, that's why our
>> public site is focused on that. I want more information available, but
>> we have only limited amount of manpower and miquel already did quite
>> amazing job in my opinion :-) We'll probably go into more details.
>>
>> The part we want to focus on past-release is speeding up certain parts
>> of tracing as well as limiting it's GC pressure. As you can see, the
>> split would be very useful for our development.
>>
>> >
>> >    Don't get me wrong, the total times are certainly useful indicators
>> of progress
>> >    (which has been amazing).
>> >
>> > 3. Speed is ds/dt and you are showing the integral of dt/ds over the
>> trip distance to get time.
>> >    A 25% improvement in total time is not a 25% improvement in speed.
>> I.e., (if you define
>> >    improvement as a percentage change in a desired direction), for e.g.
>> 25%:
>> >    distance/(0.75*time) != 1.25*(distance/time).
>> >
>> >    IMO 'speed' (the implication to me in the name speed.pypy.org) would
>> be benchmarks/time
>> >    more appropriately than time/benchmark.
>> >
>> >    Both measures are useful, but time percentages are easy to
>> mis{use,construe} ;-)
>>
>> That's correct.
>>
>> Benchmarks are in general very easy to lie about and they're by
>> definition flawed. That's why I always include raw data when I publish
>> stuff on the blog, so people can work on it themselves.
>>
>> >
>> > 4. Is there any memory footprint data?
>> >
>>
>> No. Memory measurment is hard and it's even less useful without
>> breaking down. Those particular benchmarks are not very good basis for
>> memory measurment - in case of pypy you would mostly observe the
>> default allocated memory (which is roughly 10M for the interpreter +
>> 16M for semispace GC + cache for nursery).
>>
>> Also our GC is of a kind that it can run faster if you give it more
>> memory (not that we use this feature, but it's possible).
>>
>> Cheers,
>> fijal
>> _______________________________________________
>> pypy-dev at codespeak.net
>> http://codespeak.net/mailman/listinfo/pypy-dev
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20100312/a13891e0/attachment.html>

From arigo at tunes.org  Fri Mar 12 19:27:51 2010
From: arigo at tunes.org (Armin Rigo)
Date: Fri, 12 Mar 2010 19:27:51 +0100
Subject: [pypy-dev] Released: PyPy 1.2, JIT included
Message-ID: <20100312182751.GA17278@code0.codespeak.net>

==================================
PyPy 1.2: Just-in-Time Compilation
==================================

Welcome to the PyPy 1.2 release.  The highlight of this release is
to be the first that ships with a Just-in-Time compiler that is
known to be faster than CPython (and unladen swallow) on some
real-world applications (or the best benchmarks we could get for
them).  The main theme for the 1.2 release is speed.

Main site:

    http://pypy.org/

The JIT is stable and we don't observe crashes. Nevertheless we
would recommend you to treat it as beta software and as a way to try
out the JIT to see how it works for you.


Highlights of This Release
==========================

* The JIT compiler.

* Various interpreter optimizations that improve performance
  as well as help save memory.

* Introducing a new PyPy website at http://pypy.org/ , made by
  tav and improved by the PyPy team.

* Introducing http://speed.pypy.org/ , a new service that
  monitors our performance nightly, made by Miquel Torres.

* There will be ubuntu packages on "PyPy's PPA" made by
  Bartosz Skowron; however various troubles prevented us from
  having them as of now.


Known JIT problems (or why you should consider this beta software):

* The only supported platform is 32bit x86 for now, we're
  looking for help with other platforms.

* It is still memory-hungry.  There is no limit on the amount
  of RAM that the assembler can consume; it is thus possible
  (although unlikely) that the assembler ends up using
  unreasonable amounts of memory.


If you want to try PyPy, go to the "download page" on our excellent
new site at http://pypy.org/download.html and find the binary for
your platform. If the binary does not work (e.g. on Linux, because
of different versions of external .so dependencies), or if your
platform is not supported, you can try building from the source.


What is PyPy?
=============

Technically, PyPy is both a Python interpreter implementation and an
advanced compiler, or more precisely a framework for implementing
dynamic languages and generating virtual machines for them.  The
focus of this release is the introduction of a new transformation,
the JIT Compiler Generator, and its application to the Python
interpreter.

Socially, PyPy is a collaborative effort of many individuals working
together in a distributed and sprint-driven way since 2003.  PyPy
would not have gotten as far as it has without the coding, feedback
and general support from numerous people.


The PyPy release team,
    Armin Rigo, Maciej Fijalkowski and Amaury Forgeot d'Arc

Together with
    Antonio Cuni, Carl Friedrich Bolz, Holger Krekel and
    Samuele Pedroni

and many others:
    http://codespeak.net/pypy/dist/pypy/doc/contributor.html


From fredrik.johansson at gmail.com  Fri Mar 12 22:01:52 2010
From: fredrik.johansson at gmail.com (Fredrik Johansson)
Date: Fri, 12 Mar 2010 22:01:52 +0100
Subject: [pypy-dev] Two bugs in PyPy 1.2
In-Reply-To: <3d0cebfb1003121144i2a00ef6asf497389fe36bdddb@mail.gmail.com>
References: <3d0cebfb1003121144i2a00ef6asf497389fe36bdddb@mail.gmail.com>
Message-ID: <3d0cebfb1003121301w772a8973m6aa9994e217bee98@mail.gmail.com>

(Note: Trying a second time, since the mail does not seem to have arrived on
the first try. Sorry if it's a duplicate.)


Dear PyPy developers,

I have found two bugs while trying to run mpmath on pypy-1.2. This is with
the Windows binary:

Python 2.5.2 (72130, Mar 12 2010, 12:25:40)
[PyPy 1.2.0] on win32

My system is an HP 530, Intel Celeron M 520 @ 1.60 GHz, 1024 MB RAM running
Windows XP.

The first bug is an incompatibility with CPython. In CPython, builtin float
functions accept arbitrary objects as long as they have a __float__ method.
Consider the following code:

class x(object):
    def __float__(self): return 1.0

class y(object):
    def __complex__(self): return 1.0j

round(x())
import math; math.log(x())
import cmath; cmath.log(x())
import cmath; cmath.log(y())


CPython 2.6 gives:
1.0
0.0
0j
1.5707963267948966j

CPython 2.5 gives:
1.0
0.0
0j
TypeError: a float is required

PyPy gives:
TypeError: expected float, got x object
TypeError: expected float, got x object
0j
1.5707963267948966j

Presumably PyPy should do the same thing as 2.6 here.


Secondly, there is an error somewhere in the long multiplication code:

C:\pypy12>pypy
Python 2.5.2 (72130, Mar 12 2010, 12:25:40)
[PyPy 1.2.0] on win32
Type "help", "copyright", "credits" or "license" for more information.
And now for something completely different: ``no random bugs in random,
after
all''
>>>> import mpmath
>>>> mpmath.runtests()
mpmath imported from C:\pypy12\lib-python\2.5.2\mpmath
mpmath backend: python
mpmath mp class: <mpmath.ctx_mp.MPContext object at 0x0000001a>
mpmath version: 0.15-svn
Python version: 2.5.2 (72130, Mar 12 2010, 12:25:40)
[PyPy 1.2.0]

[...]

test_functions2
    agm                       ok        0.0446171 s
    airy                      ok        0.1957078 s
    appellf1                  ok        1.1814778 s
    bessel                    ok        0.6132425 s
    coulomb                   ok        0.4926749 s
    e1                        ok        0.0109151 s
    ei                        ok        0.0346402 s
    elliptic_integrals        ok        0.2575911 s
    erf                       ok        0.3858091 s
    exp_integrals             ok        0.0715859 s
    RPython traceback:
  File "implement_4.c", line 35008, in
opcode_impl_for_mul__AccessDirect_star_2
  File "implement_9.c", line 47885, in __mm_mul_W_LongObject_W_LongObject
  File "implement_13.c", line 3727, in _k_lopsided_mul
  File "implement_9.c", line 38027, in _k_mul
Fatal RPython error: AssertionError


The test that triggers the failure is "from mpmath import expint;
expint('1.01', '1e-1000')".

For your convenience, I have tracked down the precise numbers. Since they
have several hundred digits, I put them in an attachment.

Thank you for your hard work!

Fredrik
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20100312/3e7c664f/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: mulbug.py
Type: application/octet-stream
Size: 4110 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20100312/3e7c664f/attachment.obj>

From alex.gaynor at gmail.com  Fri Mar 12 22:08:10 2010
From: alex.gaynor at gmail.com (Alex Gaynor)
Date: Fri, 12 Mar 2010 15:08:10 -0600
Subject: [pypy-dev] speed.pypy.org quick update
In-Reply-To: <c561346d1003101214q6dd04f51lb13defb2791d64ec@mail.gmail.com>
References: <c561346d1003101214q6dd04f51lb13defb2791d64ec@mail.gmail.com>
Message-ID: <ae6584d31003121308s5ff1e3dtdd8348d25fdc1a59@mail.gmail.com>

So one tiny pony I have is that on the tablular timeline page
(http://speed.pypy.org/timeline/) that when you mouseover a graph it
doesn't show the "coordinates" on graphs of those sizes I don't think
it adds any value, and it's farily distracting.

Thanks for your work,
Alex


-- 
"I disapprove of what you say, but I will defend to the death your
right to say it." -- Voltaire
"The people's good is the highest law." -- Cicero
"Code can always be simpler than you think, but never as simple as you
want" -- Me


From santagada at gmail.com  Fri Mar 12 22:37:49 2010
From: santagada at gmail.com (Leonardo Santagada)
Date: Fri, 12 Mar 2010 18:37:49 -0300
Subject: [pypy-dev] speed.pypy.org quick update
In-Reply-To: <ae6584d31003121308s5ff1e3dtdd8348d25fdc1a59@mail.gmail.com>
References: <c561346d1003101214q6dd04f51lb13defb2791d64ec@mail.gmail.com>
	<ae6584d31003121308s5ff1e3dtdd8348d25fdc1a59@mail.gmail.com>
Message-ID: <16870A89-5284-4BBA-934A-DF3CCE12D7C2@gmail.com>


On Mar 12, 2010, at 6:08 PM, Alex Gaynor wrote:

> So one tiny pony I have is that on the tablular timeline page
> (http://speed.pypy.org/timeline/) that when you mouseover a graph it
> doesn't show the "coordinates" on graphs of those sizes I don't think
> it adds any value, and it's farily distracting.

For a start it could be removed (that should be pretty easy) but as a second step it would be interesting to highlight and maybe show the revision or time of the closest point (if revision then highlight all points of that revision).


> Thanks for your work,
> Alex
--
Leonardo Santagada
santagada at gmail.com





From arigo at tunes.org  Sat Mar 13 04:00:16 2010
From: arigo at tunes.org (Armin Rigo)
Date: Sat, 13 Mar 2010 04:00:16 +0100
Subject: [pypy-dev] Two bugs in PyPy 1.2
In-Reply-To: <3d0cebfb1003121301w772a8973m6aa9994e217bee98@mail.gmail.com>
References: <3d0cebfb1003121144i2a00ef6asf497389fe36bdddb@mail.gmail.com>
	<3d0cebfb1003121301w772a8973m6aa9994e217bee98@mail.gmail.com>
Message-ID: <20100313030015.GA24400@code0.codespeak.net>

Hi Fredrik,

On Fri, Mar 12, 2010 at 10:01:52PM +0100, Fredrik Johansson wrote:
> I have found two bugs while trying to run mpmath on pypy-1.2.

Thank you a lot for the precise bug reports :-)  We will fix it.


A bientot,

Armin.


From fijall at gmail.com  Sat Mar 13 07:23:24 2010
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Fri, 12 Mar 2010 23:23:24 -0700
Subject: [pypy-dev] Two bugs in PyPy 1.2
In-Reply-To: <20100313030015.GA24400@code0.codespeak.net>
References: <3d0cebfb1003121144i2a00ef6asf497389fe36bdddb@mail.gmail.com> 
	<3d0cebfb1003121301w772a8973m6aa9994e217bee98@mail.gmail.com> 
	<20100313030015.GA24400@code0.codespeak.net>
Message-ID: <693bc9ab1003122223u1b88d3f1p3b88afa3e3476214@mail.gmail.com>

On Fri, Mar 12, 2010 at 8:00 PM, Armin Rigo <arigo at tunes.org> wrote:
> Hi Fredrik,
>
> On Fri, Mar 12, 2010 at 10:01:52PM +0100, Fredrik Johansson wrote:
>> I have found two bugs while trying to run mpmath on pypy-1.2.
>
> Thank you a lot for the precise bug reports :-) ?We will fix it.
>

Actually I fixed the one about fatal rpython assertion. There is also
a problem, which seems to be windows only that I can't reproduce.

(you need mpmath for that).

from mpmath import fp
for i in range(1000): fp.zeta(0.5+37235j)


From wkornewald at freenet.de  Sat Mar 13 12:50:32 2010
From: wkornewald at freenet.de (Waldemar Kornewald)
Date: Sat, 13 Mar 2010 12:50:32 +0100
Subject: [pypy-dev] slow benchmark?
Message-ID: <a8d220611003130350x2b8d6896ldd1c95d0e24809a3@mail.gmail.com>

Hi,
maybe I'm doing something totally wrong, but the attached trivial
benchmark runs several times slower than on CPython. I've tested it
with the Windows binary from your website. Your other benchmarks run
fast, BTW. Is there something wrong with my code?

Bye,
Waldemar Kornewald

--
http://www.allbuttonspressed.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: fak.py
Type: application/octet-stream
Size: 598 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20100313/aa76d454/attachment.obj>

From cfbolz at gmx.de  Sat Mar 13 13:15:20 2010
From: cfbolz at gmx.de (Carl Friedrich Bolz)
Date: Sat, 13 Mar 2010 13:15:20 +0100
Subject: [pypy-dev] slow benchmark?
In-Reply-To: <a8d220611003130350x2b8d6896ldd1c95d0e24809a3@mail.gmail.com>
References: <a8d220611003130350x2b8d6896ldd1c95d0e24809a3@mail.gmail.com>
Message-ID: <348899051003130415s312e584claddf3c8954d073ef@mail.gmail.com>

Hi Waldemar,

2010/3/13 Waldemar Kornewald <wkornewald at freenet.de>:
> Hi,
> maybe I'm doing something totally wrong, but the attached trivial
> benchmark runs several times slower than on CPython. I've tested it
> with the Windows binary from your website. Your other benchmarks run
> fast, BTW. Is there something wrong with my code?

This benchmark is completely dominated by multiplying long integers.
This is nothing that our JIT can speed up, since the multiplication is
implemented in RPython (in CPython it is equivalently implemented in
C). While we use the same algorithm as CPython, we probably have some
overheads that make us slower by a constant factor. This might be
fixable, if we find somebody who is interested in looking at it.

Cheers,

Carl Friedrich


From marius at pov.lt  Sun Mar 14 02:28:08 2010
From: marius at pov.lt (Marius Gedminas)
Date: Sun, 14 Mar 2010 03:28:08 +0200
Subject: [pypy-dev] speed.pypy.org quick update
In-Reply-To: <16870A89-5284-4BBA-934A-DF3CCE12D7C2@gmail.com>
References: <c561346d1003101214q6dd04f51lb13defb2791d64ec@mail.gmail.com>
	<ae6584d31003121308s5ff1e3dtdd8348d25fdc1a59@mail.gmail.com>
	<16870A89-5284-4BBA-934A-DF3CCE12D7C2@gmail.com>
Message-ID: <20100314012808.GA441@fridge.pov.lt>

On Fri, Mar 12, 2010 at 06:37:49PM -0300, Leonardo Santagada wrote:
> On Mar 12, 2010, at 6:08 PM, Alex Gaynor wrote:
> > So one tiny pony I have is that on the tablular timeline page
> > (http://speed.pypy.org/timeline/) that when you mouseover a graph it
> > doesn't show the "coordinates" on graphs of those sizes I don't think
> > it adds any value, and it's farily distracting.
> 
> For a start it could be removed (that should be pretty easy)

Actually, if you added units to those numbers, they could answer
important questions like "is a higher line better or is a lower line
better?"

> but as a
> second step it would be interesting to highlight and maybe show the
> revision or time of the closest point (if revision then highlight all
> points of that revision).

Some kind of rounding would be nice, as seeing "0.6 seconds in revision
71807.4" is a bit weird.

Very shiny website, BTW, I love it.

Marius Gedminas
-- 
Q:  How many IBM CPU's does it take to execute a job?
A:  Four; three to hold it down, and one to rip its head off.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20100314/33fb45b4/attachment.pgp>

From fijall at gmail.com  Sun Mar 14 08:04:45 2010
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Sun, 14 Mar 2010 00:04:45 -0700
Subject: [pypy-dev] issue tracker
Message-ID: <693bc9ab1003132304i7b3d82f5q9b5103835cd385eb@mail.gmail.com>

Hello.

I'm considering changing issue tracker as apparently a lot of people
gave feedback after the release. For me the current incarnation is
unbearable - I can't easily track things like "submitted" or "triaged"
from "windows build requires superuser" which always pop at the top or
"polish objspace initialization" which is probably important but I
already decided not to deal with it.

It might be the inherent problem with roundup or just our
implementation. pylib for example moved to bitbucket, probably also
for similar reasons.

Do you have any thoughts or obvious choices?

Cheers,
fijal


From alex.gaynor at gmail.com  Sun Mar 14 08:09:34 2010
From: alex.gaynor at gmail.com (Alex Gaynor)
Date: Sun, 14 Mar 2010 01:09:34 -0600
Subject: [pypy-dev] issue tracker
In-Reply-To: <693bc9ab1003132304i7b3d82f5q9b5103835cd385eb@mail.gmail.com>
References: <693bc9ab1003132304i7b3d82f5q9b5103835cd385eb@mail.gmail.com>
Message-ID: <ae6584d31003132309u94dce06u482111ea3cb8ed4b@mail.gmail.com>

On Sun, Mar 14, 2010 at 1:04 AM, Maciej Fijalkowski <fijall at gmail.com> wrote:
> Hello.
>
> I'm considering changing issue tracker as apparently a lot of people
> gave feedback after the release. For me the current incarnation is
> unbearable - I can't easily track things like "submitted" or "triaged"
> from "windows build requires superuser" which always pop at the top or
> "polish objspace initialization" which is probably important but I
> already decided not to deal with it.
>
> It might be the inherent problem with roundup or just our
> implementation. pylib for example moved to bitbucket, probably also
> for similar reasons.
>
> Do you have any thoughts or obvious choices?
>
> Cheers,
> fijal
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev
>

I'd say trac is the "obvious" choice since we are on svn.

Alex

-- 
"I disapprove of what you say, but I will defend to the death your
right to say it." -- Voltaire
"The people's good is the highest law." -- Cicero
"Code can always be simpler than you think, but never as simple as you
want" -- Me


From micahel at gmail.com  Sun Mar 14 08:20:47 2010
From: micahel at gmail.com (Michael Hudson-Doyle)
Date: Sun, 14 Mar 2010 20:20:47 +1300
Subject: [pypy-dev] issue tracker
In-Reply-To: <693bc9ab1003132304i7b3d82f5q9b5103835cd385eb@mail.gmail.com>
References: <693bc9ab1003132304i7b3d82f5q9b5103835cd385eb@mail.gmail.com>
Message-ID: <aa0dfd861003132320w763e9d6dg2c1326676296215@mail.gmail.com>

On 14 March 2010 20:04, Maciej Fijalkowski <fijall at gmail.com> wrote:
> Hello.
>
> I'm considering changing issue tracker as apparently a lot of people
> gave feedback after the release. For me the current incarnation is
> unbearable - I can't easily track things like "submitted" or "triaged"
> from "windows build requires superuser" which always pop at the top or
> "polish objspace initialization" which is probably important but I
> already decided not to deal with it.
>
> It might be the inherent problem with roundup or just our
> implementation. pylib for example moved to bitbucket, probably also
> for similar reasons.
>
> Do you have any thoughts or obvious choices?

There's always Launchpad; it's not perfect but if it works for ubuntu
it must be at least OK :-)  Also, someone else runs the server, which
probably isn't to be discounted, and can probably be relied on to
improve over time -- there's a lot of development happening on it.  It
does all the usual things reasonably well, with sensible statuses and
tagging and assignment and so on. Possibly a disadvantage is that
because it has a focus on collaboration between projects, it's not
tweakable to meet any particular project's needs.

It should be possible to import the bugs from roundup into lp,
although roundup's 'build your own schema' thing probably means some
hand-holding will be required.

Cheers,
mwh


From santagada at gmail.com  Sun Mar 14 11:11:56 2010
From: santagada at gmail.com (Leonardo Santagada)
Date: Sun, 14 Mar 2010 07:11:56 -0300
Subject: [pypy-dev] speed.pypy.org quick update
In-Reply-To: <20100314012808.GA441@fridge.pov.lt>
References: <c561346d1003101214q6dd04f51lb13defb2791d64ec@mail.gmail.com>
	<ae6584d31003121308s5ff1e3dtdd8348d25fdc1a59@mail.gmail.com>
	<16870A89-5284-4BBA-934A-DF3CCE12D7C2@gmail.com>
	<20100314012808.GA441@fridge.pov.lt>
Message-ID: <2364806D-9111-43DB-95AB-370B35D3696C@gmail.com>


On Mar 13, 2010, at 10:28 PM, Marius Gedminas wrote:

> On Fri, Mar 12, 2010 at 06:37:49PM -0300, Leonardo Santagada wrote:
>> On Mar 12, 2010, at 6:08 PM, Alex Gaynor wrote:
>>> So one tiny pony I have is that on the tablular timeline page
>>> (http://speed.pypy.org/timeline/) that when you mouseover a graph it
>>> doesn't show the "coordinates" on graphs of those sizes I don't think
>>> it adds any value, and it's farily distracting.
>> 
>> For a start it could be removed (that should be pretty easy)
> 
> Actually, if you added units to those numbers, they could answer
> important questions like "is a higher line better or is a lower line
> better?"

Yes but axis should be named in a always visible place, so when people see the graphs they know what they mean.

>> but as a
>> second step it would be interesting to highlight and maybe show the
>> revision or time of the closest point (if revision then highlight all
>> points of that revision).
> 
> Some kind of rounding would be nice, as seeing "0.6 seconds in revision
> 71807.4" is a bit weird.

No rounding but actually showing the data for the closest point and not where the mouse is over.

> Very shiny website, BTW, I love it.

I think this is a clear way to show performance for non developers and is great even for developers, it is a win win website :)

--
Leonardo Santagada
santagada at gmail.com





From tobami at googlemail.com  Sun Mar 14 14:14:25 2010
From: tobami at googlemail.com (Miquel Torres)
Date: Sun, 14 Mar 2010 14:14:25 +0100
Subject: [pypy-dev] memory recording... Re: speed.pypy.org quick update
In-Reply-To: <64ddb72c1003120552x77fb215rd6d4b36a3ccf7dc9@mail.gmail.com>
References: <64ddb72c1003120552x77fb215rd6d4b36a3ccf7dc9@mail.gmail.com>
Message-ID: <c561346d1003140614w580b1d1cp889c5be0d6704bf5@mail.gmail.com>

Yeah, we really need to sit down and find an acceptable way to measure and
save memory consumption.data.


2010/3/12 Ren? Dudfield <renesd at gmail.com>

> Hi again,
>
> trying to do some research on ways to record memory usage in an X-platform
> way...
>
> keeping my notes here:
>
> http://renesd.blogspot.com/2010/03/memory-usage-of-processes-from-python.html
>
> So far people have come up with these two useful projects so far:
>     http://code.google.com/p/psutil/
>     http://code.google.com/p/pympler/
>
> I think psutil will have most info needed to construct a decent memory
> recording module for benchmarks.  However, it includes C code, so will
> probably have to rip some of the memory parts out, and maybe reimplement
> with ctypes.
>
>
> cu,
>
>
>
>
> On Fri, Mar 12, 2010 at 10:49 AM, Ren? Dudfield <renesd at gmail.com> wrote:
>
>> btw, for python memory usage on linux
>> /proc/PID/status
>>
>> Here is some code for linux...
>>
>> wget http://rene.f0o.com/~rene/stuff/memory_usage.py<http://rene.f0o.com/%7Erene/stuff/memory_usage.py>
>>
>> >>> import memory_usage
>> >>> bytes_of_resident_memory = memory_usage.resident()
>>
>>
>> Should be easy enough to add that to benchmarks at the start and end?
>> Maybe calling it in the middle would be a little harder... but not too hard.
>>
>>
>> TODO: Would need to be updated for other platforms, and support measuring
>> child processes, tests, and code cleanup :)
>>
>> cu,
>>
>>
>>
>>
>> On Thu, Mar 11, 2010 at 12:32 AM, Maciej Fijalkowski <fijall at gmail.com>wrote:
>>
>>> Hey.
>>>
>>> I'll answer questions that are relevant to benchmarks themselves and
>>> not running.
>>>
>>> On Wed, Mar 10, 2010 at 4:45 PM, Bengt Richter <bokr at oz.net> wrote:
>>> > On 03/10/2010 12:14 PM Miquel Torres wrote:
>>> >> Hi!
>>> >>
>>> >> I wanted to explain a couple of things about the speed website:
>>> >>
>>> >> - New feature: the Timeline view now defaults to a plot grid, showing
>>> >> all benchmarks at the same time. It was a feature request made more
>>> >> than once, so depending on personal tastes, you can bookmark either
>>> >> /overview/ or /timeline/. Thanks go to nsf for helping with the
>>> >> implementation.
>>> >> - The code has now moved to github as Codespeed, a benchmark
>>> >> visualization framework (http://github.com/tobami/codespeed)
>>> >> - I have updated speed.pypy.org with version 0.3. Much of the work
>>> has
>>> >> been under the hood to make it feasible for other projects to use
>>> >> codespeed as a framework.
>>> >>
>>> >> For those interested in further development you can go to the releases
>>> >> wiki (still a work in progress):
>>> >> http://wiki.github.com/tobami/codespeed/releases
>>> >>
>>> >> Next in the line are some DB changes to be able to save standard
>>> >> deviation data and the like. Long term goals besides world domination
>>> >> are integration with buildbot and similarly unrealistic things.
>>> >> Feedback is always welcome.
>>> >
>>> > Nice looking stuff. But a couple comments:
>>> >
>>> > 1. IMO standard deviation is too often worse than useless, since it
>>> hides
>>> >    the true nature of the distribution. I think the assumption of
>>> normalcy
>>> >    is highly suspect for benchmark timings, and pruning may hide
>>> interesting clusters.
>>> >
>>> >    I prefer to look at scattergrams, where things like clustering and
>>> correlations
>>> >    are easily apparent to the eye, as well as the amount of data
>>> (assuming a good
>>> >    mapping of density to visuals).
>>>
>>> That's true. In general a benchmark run over time is a period of
>>> warmup, when JIT compiles assembler followed by thing that can be
>>> described by average and std devation. Personally I would like to have
>>> those 3 measures separated, but didn't implement that yet (it has also
>>> some interesting statistical questions involved). Std deviation is
>>> useful to get whether a difference was meaningful of certain checkin
>>> or just noise.
>>>
>>> >
>>> > 2. IMO benchmark timings are like travel times, comparing different
>>> vehicles.
>>> >    (pypy with jit being a vehicle capable of dynamic self-modification
>>> ;-)
>>> >    E.g., which part of travel from Stockholm to Paris would you
>>> concentrate
>>> >    on improving to improve the overall result? How about travel from
>>> Brussels to Paris?
>>> >    Or Paris to Sydney? ;-P Different things come into play in different
>>> benchmarks/trips.
>>> >    A Porsche Turbo and a 2CV will both have to wait for a ferry, if
>>> that's part of the trip.
>>> >
>>> >    IOW, it would be nice to see total time broken down somehow, to see
>>> what's really
>>> >    happening.
>>>
>>> I can't agree more with that. We already do split time when we perform
>>> benchmarks by hand, but they're not yet integrated into the whole
>>> nightly run. Total time is what users see though, that's why our
>>> public site is focused on that. I want more information available, but
>>> we have only limited amount of manpower and miquel already did quite
>>> amazing job in my opinion :-) We'll probably go into more details.
>>>
>>> The part we want to focus on past-release is speeding up certain parts
>>> of tracing as well as limiting it's GC pressure. As you can see, the
>>> split would be very useful for our development.
>>>
>>> >
>>> >    Don't get me wrong, the total times are certainly useful indicators
>>> of progress
>>> >    (which has been amazing).
>>> >
>>> > 3. Speed is ds/dt and you are showing the integral of dt/ds over the
>>> trip distance to get time.
>>> >    A 25% improvement in total time is not a 25% improvement in speed.
>>> I.e., (if you define
>>> >    improvement as a percentage change in a desired direction), for e.g.
>>> 25%:
>>> >    distance/(0.75*time) != 1.25*(distance/time).
>>> >
>>> >    IMO 'speed' (the implication to me in the name speed.pypy.org)
>>> would be benchmarks/time
>>> >    more appropriately than time/benchmark.
>>> >
>>> >    Both measures are useful, but time percentages are easy to
>>> mis{use,construe} ;-)
>>>
>>> That's correct.
>>>
>>> Benchmarks are in general very easy to lie about and they're by
>>> definition flawed. That's why I always include raw data when I publish
>>> stuff on the blog, so people can work on it themselves.
>>>
>>> >
>>> > 4. Is there any memory footprint data?
>>> >
>>>
>>> No. Memory measurment is hard and it's even less useful without
>>> breaking down. Those particular benchmarks are not very good basis for
>>> memory measurment - in case of pypy you would mostly observe the
>>> default allocated memory (which is roughly 10M for the interpreter +
>>> 16M for semispace GC + cache for nursery).
>>>
>>> Also our GC is of a kind that it can run faster if you give it more
>>> memory (not that we use this feature, but it's possible).
>>>
>>> Cheers,
>>> fijal
>>> _______________________________________________
>>> pypy-dev at codespeak.net
>>> http://codespeak.net/mailman/listinfo/pypy-dev
>>>
>>
>>
>
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20100314/167ff28e/attachment.html>

From benjamin at python.org  Sun Mar 14 15:03:45 2010
From: benjamin at python.org (Benjamin Peterson)
Date: Sun, 14 Mar 2010 08:03:45 -0600
Subject: [pypy-dev] issue tracker
In-Reply-To: <693bc9ab1003132304i7b3d82f5q9b5103835cd385eb@mail.gmail.com>
References: <693bc9ab1003132304i7b3d82f5q9b5103835cd385eb@mail.gmail.com>
Message-ID: <1afaf6161003140703p47ae704av88e09d7de00546f3@mail.gmail.com>

2010/3/14 Maciej Fijalkowski <fijall at gmail.com>:
> Hello.
>
> I'm considering changing issue tracker as apparently a lot of people
> gave feedback after the release. For me the current incarnation is
> unbearable - I can't easily track things like "submitted" or "triaged"
> from "windows build requires superuser" which always pop at the top or
> "polish objspace initialization" which is probably important but I
> already decided not to deal with it.
>
> It might be the inherent problem with roundup or just our
> implementation. pylib for example moved to bitbucket, probably also
> for similar reasons.
>
> Do you have any thoughts or obvious choices?

Perhaps we can try and get a python.org roundup tracker. Jython does
this, and the CPython one is quite bearable.



-- 
Regards,
Benjamin


From marius at pov.lt  Sun Mar 14 15:06:26 2010
From: marius at pov.lt (Marius Gedminas)
Date: Sun, 14 Mar 2010 16:06:26 +0200
Subject: [pypy-dev] speed.pypy.org quick update
In-Reply-To: <2364806D-9111-43DB-95AB-370B35D3696C@gmail.com>
References: <c561346d1003101214q6dd04f51lb13defb2791d64ec@mail.gmail.com>
	<ae6584d31003121308s5ff1e3dtdd8348d25fdc1a59@mail.gmail.com>
	<16870A89-5284-4BBA-934A-DF3CCE12D7C2@gmail.com>
	<20100314012808.GA441@fridge.pov.lt>
	<2364806D-9111-43DB-95AB-370B35D3696C@gmail.com>
Message-ID: <20100314140625.GA7406@fridge.pov.lt>

On Sun, Mar 14, 2010 at 07:11:56AM -0300, Leonardo Santagada wrote:
> On Mar 13, 2010, at 10:28 PM, Marius Gedminas wrote:
> > Some kind of rounding would be nice, as seeing "0.6 seconds in revision
> > 71807.4" is a bit weird.
> 
> No rounding but actually showing the data for the closest point and
> not where the mouse is over.

Sorry, yes, I meant rounding of coordinates to the nearest data point,
not rounding the interpolated data to appear more truthy.

Marius Gedminas
-- 
UNIX is user friendly. It's just selective about who its friends are.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20100314/19f1dd0d/attachment.pgp>

From tobami at googlemail.com  Sun Mar 14 19:04:22 2010
From: tobami at googlemail.com (Miquel Torres)
Date: Sun, 14 Mar 2010 19:04:22 +0100
Subject: [pypy-dev] speed.pypy.org quick update
In-Reply-To: <2364806D-9111-43DB-95AB-370B35D3696C@gmail.com>
References: <c561346d1003101214q6dd04f51lb13defb2791d64ec@mail.gmail.com>
	<ae6584d31003121308s5ff1e3dtdd8348d25fdc1a59@mail.gmail.com>
	<16870A89-5284-4BBA-934A-DF3CCE12D7C2@gmail.com>
	<20100314012808.GA441@fridge.pov.lt>
	<2364806D-9111-43DB-95AB-370B35D3696C@gmail.com>
Message-ID: <c561346d1003141104s1fecc7dey40b87c274e50a113@mail.gmail.com>

Hey, thanks for the feedback.

- In the timeline grid, I agree showing coordinates is not useful. I have
disabled that
- The rounding: yes, it is pretty stupid to show "revision 71807.4". That
goes away once coordinates are disabled though.

>I think this is a clear way to show performance for non developers and is
great even for developers, it is a win
> win website :)

I'm very glad to hear that. That was exactly my intention when starting the
project :-D
I had to scratch my itch of wanting to better follow pypy's performance as a
common python developer, but I also recognized that being such a performance
oriented project, pypy badly needed good performance regression monitoring
and progress tracking.

Cheers!
Miquel


2010/3/14 Leonardo Santagada <santagada at gmail.com>

>
> On Mar 13, 2010, at 10:28 PM, Marius Gedminas wrote:
>
> > On Fri, Mar 12, 2010 at 06:37:49PM -0300, Leonardo Santagada wrote:
> >> On Mar 12, 2010, at 6:08 PM, Alex Gaynor wrote:
> >>> So one tiny pony I have is that on the tablular timeline page
> >>> (http://speed.pypy.org/timeline/) that when you mouseover a graph it
> >>> doesn't show the "coordinates" on graphs of those sizes I don't think
> >>> it adds any value, and it's farily distracting.
> >>
> >> For a start it could be removed (that should be pretty easy)
> >
> > Actually, if you added units to those numbers, they could answer
> > important questions like "is a higher line better or is a lower line
> > better?"
>
> Yes but axis should be named in a always visible place, so when people see
> the graphs they know what they mean.
>
> >> but as a
> >> second step it would be interesting to highlight and maybe show the
> >> revision or time of the closest point (if revision then highlight all
> >> points of that revision).
> >
> > Some kind of rounding would be nice, as seeing "0.6 seconds in revision
> > 71807.4" is a bit weird.
>
> No rounding but actually showing the data for the closest point and not
> where the mouse is over.
>
> > Very shiny website, BTW, I love it.
>
> I think this is a clear way to show performance for non developers and is
> great even for developers, it is a win win website :)
>
> --
> Leonardo Santagada
> santagada at gmail.com
>
>
>
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20100314/ea7f5a80/attachment.html>

From tobami at googlemail.com  Sun Mar 14 19:06:42 2010
From: tobami at googlemail.com (Miquel Torres)
Date: Sun, 14 Mar 2010 19:06:42 +0100
Subject: [pypy-dev] speed.pypy.org quick update
In-Reply-To: <c561346d1003141104s1fecc7dey40b87c274e50a113@mail.gmail.com>
References: <c561346d1003101214q6dd04f51lb13defb2791d64ec@mail.gmail.com>
	<ae6584d31003121308s5ff1e3dtdd8348d25fdc1a59@mail.gmail.com>
	<16870A89-5284-4BBA-934A-DF3CCE12D7C2@gmail.com>
	<20100314012808.GA441@fridge.pov.lt>
	<2364806D-9111-43DB-95AB-370B35D3696C@gmail.com>
	<c561346d1003141104s1fecc7dey40b87c274e50a113@mail.gmail.com>
Message-ID: <c561346d1003141106g66738a36reafeeb22bb27606b@mail.gmail.com>

- About showing axis labels, it can be done, of course, as they are for a
single timeline plot. But please bear in mind that the timeline grid is a
bit of a compromise UI-wise. It currently shows 20 plots (!) at the same
time (probably more in the future if Maciej keeps adding more). Showing a
"seconds" or "lower is better" in all of them would be a big waste of screen
real state. You also have to assume that people who access this site are
knowledgeable enough for being able to interpret plots correctly, specially
if the single plot view does specify units and ALL yaxis are in seconds. It
is not like they change to flops and then to mips and seconds again...



2010/3/14 Miquel Torres <tobami at googlemail.com>

> Hey, thanks for the feedback.
>
> - In the timeline grid, I agree showing coordinates is not useful. I have
> disabled that
> - The rounding: yes, it is pretty stupid to show "revision 71807.4". That
> goes away once coordinates are disabled though.
>
>
> >I think this is a clear way to show performance for non developers and is
> great even for developers, it is a win
> > win website :)
>
> I'm very glad to hear that. That was exactly my intention when starting the
> project :-D
> I had to scratch my itch of wanting to better follow pypy's performance as
> a common python developer, but I also recognized that being such a
> performance oriented project, pypy badly needed good performance regression
> monitoring and progress tracking.
>
> Cheers!
> Miquel
>
>
> 2010/3/14 Leonardo Santagada <santagada at gmail.com>
>
>
>> On Mar 13, 2010, at 10:28 PM, Marius Gedminas wrote:
>>
>> > On Fri, Mar 12, 2010 at 06:37:49PM -0300, Leonardo Santagada wrote:
>> >> On Mar 12, 2010, at 6:08 PM, Alex Gaynor wrote:
>> >>> So one tiny pony I have is that on the tablular timeline page
>> >>> (http://speed.pypy.org/timeline/) that when you mouseover a graph it
>> >>> doesn't show the "coordinates" on graphs of those sizes I don't think
>> >>> it adds any value, and it's farily distracting.
>> >>
>> >> For a start it could be removed (that should be pretty easy)
>> >
>> > Actually, if you added units to those numbers, they could answer
>> > important questions like "is a higher line better or is a lower line
>> > better?"
>>
>> Yes but axis should be named in a always visible place, so when people see
>> the graphs they know what they mean.
>>
>> >> but as a
>> >> second step it would be interesting to highlight and maybe show the
>> >> revision or time of the closest point (if revision then highlight all
>> >> points of that revision).
>> >
>> > Some kind of rounding would be nice, as seeing "0.6 seconds in revision
>> > 71807.4" is a bit weird.
>>
>> No rounding but actually showing the data for the closest point and not
>> where the mouse is over.
>>
>> > Very shiny website, BTW, I love it.
>>
>> I think this is a clear way to show performance for non developers and is
>> great even for developers, it is a win win website :)
>>
>> --
>> Leonardo Santagada
>> santagada at gmail.com
>>
>>
>>
>> _______________________________________________
>> pypy-dev at codespeak.net
>> http://codespeak.net/mailman/listinfo/pypy-dev
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20100314/70259e7b/attachment.html>

From fijall at gmail.com  Sun Mar 14 19:18:15 2010
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Sun, 14 Mar 2010 11:18:15 -0700
Subject: [pypy-dev] speed.pypy.org quick update
In-Reply-To: <c561346d1003141106g66738a36reafeeb22bb27606b@mail.gmail.com>
References: <c561346d1003101214q6dd04f51lb13defb2791d64ec@mail.gmail.com> 
	<ae6584d31003121308s5ff1e3dtdd8348d25fdc1a59@mail.gmail.com> 
	<16870A89-5284-4BBA-934A-DF3CCE12D7C2@gmail.com>
	<20100314012808.GA441@fridge.pov.lt> 
	<2364806D-9111-43DB-95AB-370B35D3696C@gmail.com>
	<c561346d1003141104s1fecc7dey40b87c274e50a113@mail.gmail.com> 
	<c561346d1003141106g66738a36reafeeb22bb27606b@mail.gmail.com>
Message-ID: <693bc9ab1003141118n74267c25g8dffe67d61233140@mail.gmail.com>

Hello.

Please keep in mind that I'm using that on 12' monitor and that makes
screen space even more valuable ;-)

Cheers,
fijal

On Sun, Mar 14, 2010 at 11:06 AM, Miquel Torres <tobami at googlemail.com> wrote:
> - About showing axis labels, it can be done, of course, as they are for a
> single timeline plot. But please bear in mind that the timeline grid is a
> bit of a compromise UI-wise. It currently shows 20 plots (!) at the same
> time (probably more in the future if Maciej keeps adding more). Showing a
> "seconds" or "lower is better" in all of them would be a big waste of screen
> real state. You also have to assume that people who access this site are
> knowledgeable enough for being able to interpret plots correctly, specially
> if the single plot view does specify units and ALL yaxis are in seconds. It
> is not like they change to flops and then to mips and seconds again...
>
>
>
> 2010/3/14 Miquel Torres <tobami at googlemail.com>
>>
>> Hey, thanks for the feedback.
>>
>> - In the timeline grid, I agree showing coordinates is not useful. I have
>> disabled that
>> - The rounding: yes, it is pretty stupid to show "revision 71807.4". That
>> goes away once coordinates are disabled though.
>>
>> >I think this is a clear way to show performance for non developers and is
>> > great even for developers, it is a win
>> > win website :)
>>
>> I'm very glad to hear that. That was exactly my intention when starting
>> the project :-D
>> I had to scratch my itch of wanting to better follow pypy's performance as
>> a common python developer, but I also recognized that being such a
>> performance oriented project, pypy badly needed good performance regression
>> monitoring and progress tracking.
>>
>> Cheers!
>> Miquel
>>
>>
>> 2010/3/14 Leonardo Santagada <santagada at gmail.com>
>>>
>>> On Mar 13, 2010, at 10:28 PM, Marius Gedminas wrote:
>>>
>>> > On Fri, Mar 12, 2010 at 06:37:49PM -0300, Leonardo Santagada wrote:
>>> >> On Mar 12, 2010, at 6:08 PM, Alex Gaynor wrote:
>>> >>> So one tiny pony I have is that on the tablular timeline page
>>> >>> (http://speed.pypy.org/timeline/) that when you mouseover a graph it
>>> >>> doesn't show the "coordinates" on graphs of those sizes I don't think
>>> >>> it adds any value, and it's farily distracting.
>>> >>
>>> >> For a start it could be removed (that should be pretty easy)
>>> >
>>> > Actually, if you added units to those numbers, they could answer
>>> > important questions like "is a higher line better or is a lower line
>>> > better?"
>>>
>>> Yes but axis should be named in a always visible place, so when people
>>> see the graphs they know what they mean.
>>>
>>> >> but as a
>>> >> second step it would be interesting to highlight and maybe show the
>>> >> revision or time of the closest point (if revision then highlight all
>>> >> points of that revision).
>>> >
>>> > Some kind of rounding would be nice, as seeing "0.6 seconds in revision
>>> > 71807.4" is a bit weird.
>>>
>>> No rounding but actually showing the data for the closest point and not
>>> where the mouse is over.
>>>
>>> > Very shiny website, BTW, I love it.
>>>
>>> I think this is a clear way to show performance for non developers and is
>>> great even for developers, it is a win win website :)
>>>
>>> --
>>> Leonardo Santagada
>>> santagada at gmail.com
>>>
>>>
>>>
>>> _______________________________________________
>>> pypy-dev at codespeak.net
>>> http://codespeak.net/mailman/listinfo/pypy-dev
>>
>
>
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev
>


From micahel at gmail.com  Mon Mar 15 08:21:57 2010
From: micahel at gmail.com (Michael Hudson-Doyle)
Date: Mon, 15 Mar 2010 20:21:57 +1300
Subject: [pypy-dev] bzr import of pypy from svn
Message-ID: <aa0dfd861003150021l1f197cd3u4e87503dbe15df49@mail.gmail.com>

As I've mentioned a couple of times on #pypy, during the codespeak
outage I imported pypy trunk from Subversion into bzr (from a local
tarball of the repo I had).  Since codespeak came back up again I've
updated it, and Alexander asked for an import of all the branches
rather than just trunk.  This is now done and I've just finishing
pushing them up to Launchpad:

https://code.edge.launchpad.net/~mwhudson/pypy/

I can keep this up to date by hand, but automating it is a bit tricky
given that uploading the end result requires my ssh key.  I'll try to
think of something.

Launchpad can import subversion branches automatically of course, but
this is currently broken by a subversion bug
(http://subversion.tigris.org/issues/show_bug.cgi?id=3480).  A
workaround would be setting up read-only svnserve access or maaaybe
upgrading svn.

Cheers,
mwh


From fijall at gmail.com  Mon Mar 15 20:22:35 2010
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Mon, 15 Mar 2010 13:22:35 -0600
Subject: [pypy-dev] someone is wrong on the internet
Message-ID: <693bc9ab1003151222j2ab003c0j8f8efe7f4bfb15c5@mail.gmail.com>

This is particularly good hate mail I found:

http://www.linuxtoday.com/news_story.php3?ltsn=2010-03-15-013-35-NW-RL-0000

Enjoy :)


From fredrik.johansson at gmail.com  Fri Mar 12 20:44:35 2010
From: fredrik.johansson at gmail.com (Fredrik Johansson)
Date: Fri, 12 Mar 2010 20:44:35 +0100
Subject: [pypy-dev] Two bugs in PyPy 1.2
Message-ID: <3d0cebfb1003121144i2a00ef6asf497389fe36bdddb@mail.gmail.com>

Dear PyPy developers,

I have found two bugs while trying to run mpmath on pypy-1.2. This is with
the Windows binary:

Python 2.5.2 (72130, Mar 12 2010, 12:25:40)
[PyPy 1.2.0] on win32

My system is an HP 530, Intel Celeron M 520 @ 1.60 GHz, 1024 MB RAM running
Windows XP.

The first bug is an incompatibility with CPython. In CPython, builtin float
functions accept arbitrary objects as long as they have a __float__ method.
Consider the following code:

class x(object):
    def __float__(self): return 1.0

class y(object):
    def __complex__(self): return 1.0j

round(x())
import math; math.log(x())
import cmath; cmath.log(x())
import cmath; cmath.log(y())


CPython 2.6 gives:
1.0
0.0
0j
1.5707963267948966j

CPython 2.5 gives:
1.0
0.0
0j
TypeError: a float is required

PyPy gives:
TypeError: expected float, got x object
TypeError: expected float, got x object
0j
1.5707963267948966j

Presumably PyPy should do the same thing as 2.6 here.


Secondly, there is an error somewhere in the long multiplication code:

C:\pypy12>pypy
Python 2.5.2 (72130, Mar 12 2010, 12:25:40)
[PyPy 1.2.0] on win32
Type "help", "copyright", "credits" or "license" for more information.
And now for something completely different: ``no random bugs in random,
after
all''
>>>> import mpmath
>>>> mpmath.runtests()
mpmath imported from C:\pypy12\lib-python\2.5.2\mpmath
mpmath backend: python
mpmath mp class: <mpmath.ctx_mp.MPContext object at 0x0000001a>
mpmath version: 0.15-svn
Python version: 2.5.2 (72130, Mar 12 2010, 12:25:40)
[PyPy 1.2.0]

[...]

test_functions2
    agm                       ok        0.0446171 s
    airy                      ok        0.1957078 s
    appellf1                  ok        1.1814778 s
    bessel                    ok        0.6132425 s
    coulomb                   ok        0.4926749 s
    e1                        ok        0.0109151 s
    ei                        ok        0.0346402 s
    elliptic_integrals        ok        0.2575911 s
    erf                       ok        0.3858091 s
    exp_integrals             ok        0.0715859 s
    RPython traceback:
  File "implement_4.c", line 35008, in
opcode_impl_for_mul__AccessDirect_star_2
  File "implement_9.c", line 47885, in __mm_mul_W_LongObject_W_LongObject
  File "implement_13.c", line 3727, in _k_lopsided_mul
  File "implement_9.c", line 38027, in _k_mul
Fatal RPython error: AssertionError


The test that triggers the failure is "from mpmath import expint;
expint('1.01', '1e-1000')".

For your convenience, I have tracked down the precise numbers. Since they
have several hundred digits, I put them in an attachment.

Thank you for your hard work!

Fredrik
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20100312/7f6663fc/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: mulbug.py
Type: application/octet-stream
Size: 4110 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20100312/7f6663fc/attachment.obj>

From santagada at gmail.com  Mon Mar 15 22:11:56 2010
From: santagada at gmail.com (Leonardo Santagada)
Date: Mon, 15 Mar 2010 18:11:56 -0300
Subject: [pypy-dev] someone is wrong on the internet
In-Reply-To: <693bc9ab1003151222j2ab003c0j8f8efe7f4bfb15c5@mail.gmail.com>
References: <693bc9ab1003151222j2ab003c0j8f8efe7f4bfb15c5@mail.gmail.com>
Message-ID: <1439437F-104D-4D23-9FAB-2ED44E8BBE63@gmail.com>

Mega troll win. He is probably in engadget and slashdot trolling all day. Answering to him would only provoke the troll.

On Mar 15, 2010, at 4:22 PM, Maciej Fijalkowski wrote:

> This is particularly good hate mail I found:
> 
> http://www.linuxtoday.com/news_story.php3?ltsn=2010-03-15-013-35-NW-RL-0000
> 
> Enjoy :)
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev

--
Leonardo Santagada
santagada at gmail.com





From arigo at tunes.org  Tue Mar 16 10:01:47 2010
From: arigo at tunes.org (Armin Rigo)
Date: Tue, 16 Mar 2010 10:01:47 +0100
Subject: [pypy-dev] Two bugs in PyPy 1.2
In-Reply-To: <693bc9ab1003122223u1b88d3f1p3b88afa3e3476214@mail.gmail.com>
References: <3d0cebfb1003121144i2a00ef6asf497389fe36bdddb@mail.gmail.com>
	<3d0cebfb1003121301w772a8973m6aa9994e217bee98@mail.gmail.com>
	<20100313030015.GA24400@code0.codespeak.net>
	<693bc9ab1003122223u1b88d3f1p3b88afa3e3476214@mail.gmail.com>
Message-ID: <20100316090147.GA17016@code0.codespeak.net>

Hi Maciek, hi Fredrik,

On Fri, Mar 12, 2010 at 11:23:24PM -0700, Maciej Fijalkowski wrote:
> Actually I fixed the one about fatal rpython assertion. There is also
> a problem, which seems to be windows only that I can't reproduce.
> 
> from mpmath import fp
> for i in range(1000): fp.zeta(0.5+37235j)

I fixed this in r72263.  The issue was hard to pinpoint because it was
caused by a fast-path malloc: on the slow path (i.e. rarely), it would
call random C code without saving the XMM registers.  That's wrong:
there is -- on Windows only? -- a path through that C code that trashes
the XMM registers.


A bientot,

Armin.


From fredrik.johansson at gmail.com  Tue Mar 16 19:44:54 2010
From: fredrik.johansson at gmail.com (Fredrik Johansson)
Date: Tue, 16 Mar 2010 19:44:54 +0100
Subject: [pypy-dev] Two bugs in PyPy 1.2
In-Reply-To: <20100316090147.GA17016@code0.codespeak.net>
References: <3d0cebfb1003121144i2a00ef6asf497389fe36bdddb@mail.gmail.com>
	<3d0cebfb1003121301w772a8973m6aa9994e217bee98@mail.gmail.com>
	<20100313030015.GA24400@code0.codespeak.net>
	<693bc9ab1003122223u1b88d3f1p3b88afa3e3476214@mail.gmail.com>
	<20100316090147.GA17016@code0.codespeak.net>
Message-ID: <3d0cebfb1003161144k1788f7acraf669db179668f72@mail.gmail.com>

On Tue, Mar 16, 2010 at 10:01 AM, Armin Rigo <arigo at tunes.org> wrote:

> Hi Maciek, hi Fredrik,
>
> On Fri, Mar 12, 2010 at 11:23:24PM -0700, Maciej Fijalkowski wrote:
> > Actually I fixed the one about fatal rpython assertion. There is also
> > a problem, which seems to be windows only that I can't reproduce.
> >
> > from mpmath import fp
> > for i in range(1000): fp.zeta(0.5+37235j)
>
> I fixed this in r72263.  The issue was hard to pinpoint because it was
> caused by a fast-path malloc: on the slow path (i.e. rarely), it would
> call random C code without saving the XMM registers.  That's wrong:
> there is -- on Windows only? -- a path through that C code that trashes
> the XMM registers.
>
>
> A bientot,
>
> Armin.
>

Thank you!

Fredrik
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20100316/069f9833/attachment.html>

From fijall at gmail.com  Wed Mar 17 02:30:34 2010
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Tue, 16 Mar 2010 19:30:34 -0600
Subject: [pypy-dev] autonosiness in roundup
Message-ID: <693bc9ab1003161830x2381726ay7b1189fb77530181@mail.gmail.com>

Anyone knows how to set up autonosy on roundup? I only get information
about new tickets and then I don't get followups.


From micahel at gmail.com  Wed Mar 17 02:36:17 2010
From: micahel at gmail.com (Michael Hudson-Doyle)
Date: Wed, 17 Mar 2010 14:36:17 +1300
Subject: [pypy-dev] autonosiness in roundup
In-Reply-To: <693bc9ab1003161830x2381726ay7b1189fb77530181@mail.gmail.com>
References: <693bc9ab1003161830x2381726ay7b1189fb77530181@mail.gmail.com>
Message-ID: <aa0dfd861003161836q7ba9daafq26a0fd61d18d0b25@mail.gmail.com>

On 17 March 2010 14:30, Maciej Fijalkowski <fijall at gmail.com> wrote:
> Anyone knows how to set up autonosy on roundup? I only get information
> about new tickets and then I don't get followups.

I think it involves editing a particular file on codespeak.  Yay!

Cheers,
mwh


From benjamin at python.org  Wed Mar 17 02:36:23 2010
From: benjamin at python.org (Benjamin Peterson)
Date: Tue, 16 Mar 2010 20:36:23 -0500
Subject: [pypy-dev] autonosiness in roundup
In-Reply-To: <693bc9ab1003161830x2381726ay7b1189fb77530181@mail.gmail.com>
References: <693bc9ab1003161830x2381726ay7b1189fb77530181@mail.gmail.com>
Message-ID: <1afaf6161003161836u4b543766xa6c897185ebb22e9@mail.gmail.com>

2010/3/16 Maciej Fijalkowski <fijall at gmail.com>:
> Anyone knows how to set up autonosy on roundup? I only get information
> about new tickets and then I don't get followups.

When that's figured out, I'd also like to be on autonosy.



-- 
Regards,
Benjamin


From micahel at gmail.com  Wed Mar 17 02:38:29 2010
From: micahel at gmail.com (Michael Hudson-Doyle)
Date: Wed, 17 Mar 2010 14:38:29 +1300
Subject: [pypy-dev] autonosiness in roundup
In-Reply-To: <aa0dfd861003161836q7ba9daafq26a0fd61d18d0b25@mail.gmail.com>
References: <693bc9ab1003161830x2381726ay7b1189fb77530181@mail.gmail.com>
	<aa0dfd861003161836q7ba9daafq26a0fd61d18d0b25@mail.gmail.com>
Message-ID: <aa0dfd861003161838y49e1fa55m70b8ca82505ef9b1@mail.gmail.com>

On 17 March 2010 14:36, Michael Hudson-Doyle <micahel at gmail.com> wrote:
> On 17 March 2010 14:30, Maciej Fijalkowski <fijall at gmail.com> wrote:
>> Anyone knows how to set up autonosy on roundup? I only get information
>> about new tickets and then I don't get followups.
>
> I think it involves editing a particular file on codespeak. ?Yay!

In particular, /www/codespeak.net/issue/pypy-dev/detectors/nosyreaction.py

Cheers,
mwh


From anto.cuni at gmail.com  Wed Mar 17 10:09:11 2010
From: anto.cuni at gmail.com (Antonio Cuni)
Date: Wed, 17 Mar 2010 10:09:11 +0100
Subject: [pypy-dev] autonosiness in roundup
In-Reply-To: <1afaf6161003161836u4b543766xa6c897185ebb22e9@mail.gmail.com>
References: <693bc9ab1003161830x2381726ay7b1189fb77530181@mail.gmail.com>
	<1afaf6161003161836u4b543766xa6c897185ebb22e9@mail.gmail.com>
Message-ID: <4BA09C37.8080400@gmail.com>

Benjamin Peterson wrote:
> 2010/3/16 Maciej Fijalkowski <fijall at gmail.com>:
>> Anyone knows how to set up autonosy on roundup? I only get information
>> about new tickets and then I don't get followups.
> 
> When that's figured out, I'd also like to be on autonosy.

me too :-)


From amauryfa at gmail.com  Wed Mar 17 10:19:19 2010
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Wed, 17 Mar 2010 10:19:19 +0100
Subject: [pypy-dev] autonosiness in roundup
In-Reply-To: <4BA09C37.8080400@gmail.com>
References: <693bc9ab1003161830x2381726ay7b1189fb77530181@mail.gmail.com>
	<1afaf6161003161836u4b543766xa6c897185ebb22e9@mail.gmail.com>
	<4BA09C37.8080400@gmail.com>
Message-ID: <e27efe131003170219r5e8d3e85ieed3b4e9b553dfd0@mail.gmail.com>

2010/3/17 Antonio Cuni <anto.cuni at gmail.com>

> Benjamin Peterson wrote:
> > 2010/3/16 Maciej Fijalkowski <fijall at gmail.com>:
> >> Anyone knows how to set up autonosy on roundup? I only get information
> >> about new tickets and then I don't get followups.
> >
> > When that's figured out, I'd also like to be on autonosy.
>
> me too :-)


What about a mailing list, similar to Python-bugs-list?

-- 
Amaury Forgeot d'Arc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20100317/c9fd3df0/attachment.html>

From micktwomey at gmail.com  Fri Mar 19 15:09:27 2010
From: micktwomey at gmail.com (Michael Twomey)
Date: Fri, 19 Mar 2010 14:09:27 +0000
Subject: [pypy-dev] Python Ireland Conference
Message-ID: <50a522ca1003190709k5f26aa67s31f0502c6791dd70@mail.gmail.com>

Hi all,

I'm one of the folks involved in organising the first python Ireland
conference, to be held in Dublin on Saturday 17th of July (with
sprints on Sunday).

We'd love it if someone from PyPy could make it over to give a talk.
The audience is your typical spread of python developers, many of whom
haven't been exposed to PyPy and what it can do. An introduction to
PyPy and the benefits it offers would go down very well.

cheers,
  Michael


From arigo at tunes.org  Tue Mar 23 12:49:31 2010
From: arigo at tunes.org (Armin Rigo)
Date: Tue, 23 Mar 2010 12:49:31 +0100
Subject: [pypy-dev] autonosiness in roundup
In-Reply-To: <e27efe131003170219r5e8d3e85ieed3b4e9b553dfd0@mail.gmail.com>
References: <693bc9ab1003161830x2381726ay7b1189fb77530181@mail.gmail.com>
	<1afaf6161003161836u4b543766xa6c897185ebb22e9@mail.gmail.com>
	<4BA09C37.8080400@gmail.com>
	<e27efe131003170219r5e8d3e85ieed3b4e9b553dfd0@mail.gmail.com>
Message-ID: <20100323114931.GA15591@code0.codespeak.net>

Hi,

On Wed, Mar 17, 2010 at 10:19:19AM +0100, Amaury Forgeot d'Arc wrote:
> What about a mailing list, similar to Python-bugs-list?

That's a very good idea, but unfortunately I don't know if there are
issues, like what occurs when you reply to an e-mail in that list.  It
probably just works if the mailing list is configured so that replies go
by default to the author of the mail instead of to the list's address.
If that's not an issue then I could make the mailing list (provided
maybe with some reminder of how I'm supposed to do that on
codespeak...).


A bientot,

Armin.


From holger at merlinux.eu  Tue Mar 23 17:28:38 2010
From: holger at merlinux.eu (holger krekel)
Date: Tue, 23 Mar 2010 17:28:38 +0100
Subject: [pypy-dev] autonosiness in roundup
In-Reply-To: <20100323114931.GA15591@code0.codespeak.net>
References: <693bc9ab1003161830x2381726ay7b1189fb77530181@mail.gmail.com>
	<1afaf6161003161836u4b543766xa6c897185ebb22e9@mail.gmail.com>
	<4BA09C37.8080400@gmail.com>
	<e27efe131003170219r5e8d3e85ieed3b4e9b553dfd0@mail.gmail.com>
	<20100323114931.GA15591@code0.codespeak.net>
Message-ID: <20100323162838.GU26514@trillke.net>

Hi, 

On Tue, Mar 23, 2010 at 12:49 +0100, Armin Rigo wrote:
> Hi,
> 
> On Wed, Mar 17, 2010 at 10:19:19AM +0100, Amaury Forgeot d'Arc wrote:
> > What about a mailing list, similar to Python-bugs-list?
> 
> That's a very good idea, but unfortunately I don't know if there are
> issues, like what occurs when you reply to an e-mail in that list.  It
> probably just works if the mailing list is configured so that replies go
> by default to the author of the mail instead of to the list's address.
> If that's not an issue then I could make the mailing list (provided
> maybe with some reminder of how I'm supposed to do that on
> codespeak...).

head to http://codespeak.net/mailman/create
and look into /root/.ssh/mmsitepass for password (requires root)
H.


From zejn at kiberpipa.org  Tue Mar 23 18:02:59 2010
From: zejn at kiberpipa.org (Gasper Zejn)
Date: Tue, 23 Mar 2010 18:02:59 +0100
Subject: [pypy-dev] Nightly builds are unavailable
Message-ID: <201003231803.00045.zejn@kiberpipa.org>

The links at the URL http://buildbot.pypy.org/nightly/ are failing with 
"resource unavailable"; misconfiguration?

Just to make sure it's a known problem.

KR,
Gasper


From arigo at tunes.org  Tue Mar 23 18:38:32 2010
From: arigo at tunes.org (Armin Rigo)
Date: Tue, 23 Mar 2010 18:38:32 +0100
Subject: [pypy-dev] autonosiness in roundup
In-Reply-To: <20100323162838.GU26514@trillke.net>
References: <693bc9ab1003161830x2381726ay7b1189fb77530181@mail.gmail.com>
	<1afaf6161003161836u4b543766xa6c897185ebb22e9@mail.gmail.com>
	<4BA09C37.8080400@gmail.com>
	<e27efe131003170219r5e8d3e85ieed3b4e9b553dfd0@mail.gmail.com>
	<20100323114931.GA15591@code0.codespeak.net>
	<20100323162838.GU26514@trillke.net>
Message-ID: <20100323173832.GA10910@code0.codespeak.net>

Hi,

On Tue, Mar 23, 2010 at 05:28:38PM +0100, holger krekel wrote:
> > > What about a mailing list, similar to Python-bugs-list?

Done.  The list is there:

    http://codespeak.net/mailman/listinfo/pypy-issue

Note that I did not remove the existing hacks.  Please ask me.  One
issue is that I'm not sure how to make "pypy-issue" autonosy on the
already-existing tracker entries -- so far I fear that it's going to
receive only the new entries (including follow-ups to that).


A bientot,

Armin.


From anto.cuni at gmail.com  Tue Mar 23 19:23:45 2010
From: anto.cuni at gmail.com (Antonio Cuni)
Date: Tue, 23 Mar 2010 19:23:45 +0100
Subject: [pypy-dev] Nightly builds are unavailable
In-Reply-To: <201003231803.00045.zejn@kiberpipa.org>
References: <201003231803.00045.zejn@kiberpipa.org>
Message-ID: <58e316a41003231123x9d6ce03v765c2d2b55caf772@mail.gmail.com>

On Tue, Mar 23, 2010 at 6:02 PM, Gasper Zejn <zejn at kiberpipa.org> wrote:
> The links at the URL http://buildbot.pypy.org/nightly/ are failing with
> "resource unavailable"; misconfiguration?
>
> Just to make sure it's a known problem.

It's working for me right now. Maybe it has just been down for a while?


From arigo at tunes.org  Tue Mar 23 19:42:15 2010
From: arigo at tunes.org (Armin Rigo)
Date: Tue, 23 Mar 2010 19:42:15 +0100
Subject: [pypy-dev] autonosiness in roundup
In-Reply-To: <20100323173832.GA10910@code0.codespeak.net>
References: <693bc9ab1003161830x2381726ay7b1189fb77530181@mail.gmail.com>
	<1afaf6161003161836u4b543766xa6c897185ebb22e9@mail.gmail.com>
	<4BA09C37.8080400@gmail.com>
	<e27efe131003170219r5e8d3e85ieed3b4e9b553dfd0@mail.gmail.com>
	<20100323114931.GA15591@code0.codespeak.net>
	<20100323162838.GU26514@trillke.net>
	<20100323173832.GA10910@code0.codespeak.net>
Message-ID: <20100323184215.GA17031@code0.codespeak.net>

Re-hi,

On Tue, Mar 23, 2010 at 06:38:32PM +0100, Armin Rigo wrote:
> Note that I did not remove the existing hacks.  Please ask me.  One
> issue is that I'm not sure how to make "pypy-issue" autonosy on the
> already-existing tracker entries -- so far I fear that it's going to
> receive only the new entries (including follow-ups to that).

That's done too now.  A number of people should be removed from the
"nosy" field as soon as there is a new posting to a existing or new
issue.  Additionally, "pypy-issue" should show up in the "nosy" field.
It goes to the new mailing list, to which I already subscribed precisely
the same list of people.  (This means that you're still getting all
mails from the tracker, but now coming from pypy-issue at codespeak.net, so
your mail filters may need some re-tweaking.)

For everybody else, please feel free to subscribe at
http://codespeak.net/mailman/listinfo/pypy-issue .


A bientot,

Arin.


From zejn at kiberpipa.org  Tue Mar 23 19:43:33 2010
From: zejn at kiberpipa.org (Gasper Zejn)
Date: Tue, 23 Mar 2010 19:43:33 +0100
Subject: [pypy-dev] Nightly builds are unavailable
In-Reply-To: <58e316a41003231123x9d6ce03v765c2d2b55caf772@mail.gmail.com>
References: <201003231803.00045.zejn@kiberpipa.org>
	<58e316a41003231123x9d6ce03v765c2d2b55caf772@mail.gmail.com>
Message-ID: <201003231943.33616.zejn@kiberpipa.org>

On Tuesday 23 March 2010 19:23:45 you wrote:
> On Tue, Mar 23, 2010 at 6:02 PM, Gasper Zejn <zejn at kiberpipa.org> wrote:
> > The links at the URL http://buildbot.pypy.org/nightly/ are failing with
> > "resource unavailable"; misconfiguration?
> >
> > Just to make sure it's a known problem.
> 
> It's working for me right now. Maybe it has just been down for a while?
> 

Nope, not working. The "directory listing" works, but not the links. Did you 
click on any of the links provided on that url?

KR,
Gasper


From arigo at tunes.org  Tue Mar 23 22:43:55 2010
From: arigo at tunes.org (Armin Rigo)
Date: Tue, 23 Mar 2010 22:43:55 +0100
Subject: [pypy-dev] Branches
Message-ID: <20100323214355.GA30680@code0.codespeak.net>

Hi all,

Can we decide if the following branches have been merged, are useful
enough to be merged, need more work, "maybe one day", are very outdated,
or turned out just to be a plain bad idea?  The numbers are the rev
number of the last change.

Also, what about separating in-development branches that we expect to
merge or kill soon, from the branches that are for "some other idea",
e.g. "effect-analysis", which may not be merged in a long while but
still we may want to keep around?

    * 71164  abort-no-asm (fijal, cfbolz)
    * 70752  bridges-experimental (cfbolz)
    * 55732  build-external (amaury)
    * 63757  classdeco (benjamin)
    * 68723  effect-analysis (verte)
    * 55751  eval-loop-experiments (antocuni)
    * 68703  gc-arena (arigo)
    * 70970  gc-huge-list (fijal, arigo)
    * 70689  jit-profiling (pedronis, fijal)
    * 55472  judy-trees (fijal)
    * 71270  kill-can-inline (cfbolz)
    * 49688  newdotviewer (misto)
    * 64690  pypycpp (igorto)
    * 70786  separate-compilation (amaury)
    * 70024  sepcomp (xoraxax)
    * 66009  type-celldict (cfbolz)
    * 64645  unicode_filename (amaury)
    * 70890  unroll-safe-if-const-arg (fijal, cfbolz, arigo)

Interesting to see the wide range of authors :-)


A bientot,

Armin.


From sl at scrooge.dk  Tue Mar 23 22:46:50 2010
From: sl at scrooge.dk (=?ISO-8859-1?Q?S=F8ren_Laursen?=)
Date: Tue, 23 Mar 2010 22:46:50 +0100
Subject: [pypy-dev] Sandboxing pypy
Message-ID: <ed932062c4806901e1c4caaccf6abd0f@mail.gmail.com>

Hi,


I have been following  the pypy project for over a year. And I have been
playing around with it for some time.



The project I am working with Minimum Intrusion Grid : MiG (
http://sites.google.com/site/minimumintrusiongrid/) are looking into using
pypy.



I would like to use it for sandboxing user code in MiG, more specific allow
the users to develop their own ?scheduler? .



The MiG, it self is written in Python so we might even be able to run it on
pypy.  But right now it is the sandboxing that I am working with.



For me it is a bit unclear in the documentation/website, but for that I read
into ?An attacker that tries to escape the sandbox is stuck within a C
program that contains no external function calls at all except for writing
to stdout and reading from stdin.? means that I have to write functions that
emulates file read/write operations.



I have tried different things using the pypy_interact.py (just love the
?timeout parameter J), looked at code but have not been able to read files
or write files using the pypy_interact.py.



What have I missed?


Regards,



S?ren Laursen
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20100323/a022a9bd/attachment.html>

From benjamin at python.org  Tue Mar 23 22:57:43 2010
From: benjamin at python.org (Benjamin Peterson)
Date: Tue, 23 Mar 2010 16:57:43 -0500
Subject: [pypy-dev] Sandboxing pypy
In-Reply-To: <ed932062c4806901e1c4caaccf6abd0f@mail.gmail.com>
References: <ed932062c4806901e1c4caaccf6abd0f@mail.gmail.com>
Message-ID: <1afaf6161003231457p7fd6eb0age0904e61f9909bba@mail.gmail.com>

2010/3/23 S?ren Laursen <sl at scrooge.dk>:
> I have tried different things using the pypy_interact.py (just love the
> ?timeout parameter J), looked at code but have not been able to read files
> or write files using the pypy_interact.py.

That's probably because pypy_interact.py puts the sandboxed process in a VFS.

Feel free to drop by #pypy IRC anytime.


-- 
Regards,
Benjamin


From ehsanamiri at gmail.com  Wed Mar 24 00:17:07 2010
From: ehsanamiri at gmail.com (Ehsan Amiri)
Date: Tue, 23 Mar 2010 16:17:07 -0700
Subject: [pypy-dev] GSoC projects
Message-ID: <fe13bba81003231617h46f6443dya1f47184fe9b0a33@mail.gmail.com>

Hello all

I am a graduate student interested in participation in GSoC 2010. I found
some PyPy projects closely related to my background. In particular writing a
backend for Intel 64, for JIT compiler, sounds interesting. Also I would
like to know more about projects on stackless features. I would be happy to
hear any comment on these projects and pointers to documents/paper/code that
might be helpful.

Last year I participated in GSoC as well. I developed a prototype of an
architecture independent SIMD library for Python. This library provides an
interface for programmers to write SIMD code in Python and then translates
it to low level architecture specific SIMD code using CorePy. The prototype
that I developed last year supports intel 64 as I did not have enough time
to develop code generators for any other architecture.

Best
Ehsan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20100323/d92fe45f/attachment.html>

From benjamin at python.org  Wed Mar 24 02:31:50 2010
From: benjamin at python.org (Benjamin Peterson)
Date: Tue, 23 Mar 2010 20:31:50 -0500
Subject: [pypy-dev] GSoC projects
In-Reply-To: <fe13bba81003231617h46f6443dya1f47184fe9b0a33@mail.gmail.com>
References: <fe13bba81003231617h46f6443dya1f47184fe9b0a33@mail.gmail.com>
Message-ID: <1afaf6161003231831x70fa6394pacd874434f290627@mail.gmail.com>

Hi Ehsan!

2010/3/23 Ehsan Amiri <ehsanamiri at gmail.com>:
> Hello all
> I am a graduate student interested in participation in GSoC 2010. I found
> some PyPy projects closely related to my background. In particular writing a
> backend for Intel 64, for JIT compiler, sounds interesting. Also I would
> like to know more about projects on stackless features. I would be happy to
> hear any comment on these projects and pointers to documents/paper/code that
> might be helpful.

We have somebody who has voiced interest in doing a 64 bit backend,
but there's no reason you couldn't assist them or work on some other
aspect of PyPy.

Feel free to drop by the #pypy channel on Freenode anytime.


-- 
Regards,
Benjamin


From william.leslie.ttg at gmail.com  Wed Mar 24 03:00:29 2010
From: william.leslie.ttg at gmail.com (William Leslie)
Date: Wed, 24 Mar 2010 13:00:29 +1100
Subject: [pypy-dev] Branches
In-Reply-To: <20100323214355.GA30680@code0.codespeak.net>
References: <20100323214355.GA30680@code0.codespeak.net>
Message-ID: <f45df7231003231900l33beeac5q2234cbf1e26fbe19@mail.gmail.com>

On 24 March 2010 08:43, Armin Rigo <arigo at tunes.org> wrote:
> Hi all,
>
> Can we decide if the following branches have been merged, are useful
> enough to be merged, need more work, "maybe one day", are very outdated,
> or turned out just to be a plain bad idea? ?The numbers are the rev
> number of the last change.
>
> Also, what about separating in-development branches that we expect to
> merge or kill soon, from the branches that are for "some other idea",
> e.g. "effect-analysis", which may not be merged in a long while but
> still we may want to keep around?

Effect analysis can probably disappear for the moment. I've been
working on it locally, but do not have a regular internet connection.
SVN doesn't have anything useful in it anyway.

-- 
William Leslie


From arigo at tunes.org  Wed Mar 24 10:38:55 2010
From: arigo at tunes.org (Armin Rigo)
Date: Wed, 24 Mar 2010 10:38:55 +0100
Subject: [pypy-dev] Sandboxing pypy
In-Reply-To: <ed932062c4806901e1c4caaccf6abd0f@mail.gmail.com>
References: <ed932062c4806901e1c4caaccf6abd0f@mail.gmail.com>
Message-ID: <20100324093854.GA18762@code0.codespeak.net>

Hi,

On Tue, Mar 23, 2010 at 10:46:50PM +0100, S?ren Laursen wrote:
> I have tried different things using the pypy_interact.py (just love the
> ?timeout parameter J), looked at code but have not been able to read files
> or write files using the pypy_interact.py.

You get indeed a VFS (virtual file system) which is read-only so far.
You can read any file from the virtual path "/tmp/xxx" if you start
pypy_interact.py with the option "--tmp=some/path".  There is no support
yet to allow writes.  It could be easily added by editing vfs.py.


A bientot,

Armin.


From Ronny.Pfannschmidt at gmx.de  Wed Mar 24 10:47:38 2010
From: Ronny.Pfannschmidt at gmx.de (Ronny Pfannschmidt)
Date: Wed, 24 Mar 2010 10:47:38 +0100
Subject: [pypy-dev] dvcs conversation of pypy and maybe all the other
 projects in the repo, too
Message-ID: <1269424058.16631.83.camel@localhost>

Hi,

since pypy's history itself is kinda unsuitable for practically all
converters/importers out there, i started to experiment with analysis of
it in order to eventually convert to a dag based dvcs (most likely hg).

Since that repository carries various other projects as well i wonder if
anyone else is interested in converting to a dvcs.

The needs of pypy's  history alone create the need of a very powerfull
tool anyway, so converting the other projects would be a nice
sideeffect.

-- Ronny




From anto.cuni at gmail.com  Wed Mar 24 10:47:37 2010
From: anto.cuni at gmail.com (Antonio Cuni)
Date: Wed, 24 Mar 2010 10:47:37 +0100
Subject: [pypy-dev] Nightly builds are unavailable
In-Reply-To: <201003231943.33616.zejn@kiberpipa.org>
References: <201003231803.00045.zejn@kiberpipa.org>
	<58e316a41003231123x9d6ce03v765c2d2b55caf772@mail.gmail.com>
	<201003231943.33616.zejn@kiberpipa.org>
Message-ID: <58e316a41003240247u58ca76bfy19da2dde635b0e6f@mail.gmail.com>

On Tue, Mar 23, 2010 at 7:43 PM, Gasper Zejn <zejn at kiberpipa.org> wrote:

> Nope, not working. The "directory listing" works, but not the links. Did you
> click on any of the links provided on that url?

Ah no, sorry. My fault, the links indeed don't work.

ciao,
Anto


From anto.cuni at gmail.com  Wed Mar 24 10:51:15 2010
From: anto.cuni at gmail.com (Antonio Cuni)
Date: Wed, 24 Mar 2010 10:51:15 +0100
Subject: [pypy-dev] Branches
In-Reply-To: <20100323214355.GA30680@code0.codespeak.net>
References: <20100323214355.GA30680@code0.codespeak.net>
Message-ID: <58e316a41003240251q6f56d768t4398fcc5d6052d6b@mail.gmail.com>

On Tue, Mar 23, 2010 at 10:43 PM, Armin Rigo <arigo at tunes.org> wrote:

> ? ?* 55751 ?eval-loop-experiments (antocuni)

"maybe one day"

In this branch, I made some changes to the main eval loop and in some
cases I observed speedups up to 20% (which are less relevant now that
we have a jit, but still), but I never managed to reproduce them
consistently.  I'd like to keep it around to investigate if the
changes are worth of or not.

ciao,
Anto


From victor.stinner at haypocalc.com  Wed Mar 24 11:01:56 2010
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Wed, 24 Mar 2010 11:01:56 +0100
Subject: [pypy-dev] Sandboxing pypy
In-Reply-To: <ed932062c4806901e1c4caaccf6abd0f@mail.gmail.com>
References: <ed932062c4806901e1c4caaccf6abd0f@mail.gmail.com>
Message-ID: <201003241101.56211.victor.stinner@haypocalc.com>

Le mardi 23 mars 2010 22:46:50, S?ren Laursen a ?crit :
> The project I am working with Minimum Intrusion Grid : MiG (
> http://sites.google.com/site/minimumintrusiongrid/) are looking into using
> pypy.
> 
> I would like to use it for sandboxing user code in MiG, more specific allow
> the users to develop their own ?scheduler? .
> 
> The MiG, it self is written in Python so we might even be able to run it on
> pypy.  But right now it is the sandboxing that I am working with.

FYI I wrote a new sandbox project for CPython:

   http://github.com/haypo/pysandbox/

It's currently very specific to CPython: it uses evil tricks to create a read 
only view of the __builtins__ super global dictionary.

It's completly different to the PyPy sandbox: if you escape from the sandbox, 
you get a full access to all Python functions.

A long description: 
http://mail.python.org/pipermail/python-dev/2010-February/097701.html

-- 
Victor Stinner
http://www.haypocalc.com/


From cfbolz at gmx.de  Wed Mar 24 14:40:50 2010
From: cfbolz at gmx.de (Carl Friedrich Bolz)
Date: Wed, 24 Mar 2010 14:40:50 +0100
Subject: [pypy-dev] Branches
In-Reply-To: <20100323214355.GA30680@code0.codespeak.net>
References: <20100323214355.GA30680@code0.codespeak.net>
Message-ID: <4BAA1662.4090309@gmx.de>

Hi Armin,

On 03/23/2010 10:43 PM, Armin Rigo wrote:
> Can we decide if the following branches have been merged, are useful
> enough to be merged, need more work, "maybe one day", are very outdated,
> or turned out just to be a plain bad idea?  The numbers are the rev
> number of the last change.
>
> Also, what about separating in-development branches that we expect to
> merge or kill soon, from the branches that are for "some other idea",
> e.g. "effect-analysis", which may not be merged in a long while but
> still we may want to keep around?

Maybe just have a dir parallel to "branch" called "inactive"? or some 
other name.

>      * 71164  abort-no-asm (fijal, cfbolz)

Keep around, I want to reconsider this eventually.

>      * 70752  bridges-experimental (cfbolz)

Same.

>      * 71270  kill-can-inline (cfbolz)

Same.

>      * 66009  type-celldict (cfbolz)

Killed this one.

>      * 70890  unroll-safe-if-const-arg (fijal, cfbolz, arigo)

Would keep this around, even if it ultimately didn't work.

Cheers,

Carl Friedrich


From fijall at gmail.com  Thu Mar 25 04:44:29 2010
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Wed, 24 Mar 2010 21:44:29 -0600
Subject: [pypy-dev] Nightly builds are unavailable
In-Reply-To: <58e316a41003240247u58ca76bfy19da2dde635b0e6f@mail.gmail.com>
References: <201003231803.00045.zejn@kiberpipa.org>
	<58e316a41003231123x9d6ce03v765c2d2b55caf772@mail.gmail.com> 
	<201003231943.33616.zejn@kiberpipa.org>
	<58e316a41003240247u58ca76bfy19da2dde635b0e6f@mail.gmail.com>
Message-ID: <693bc9ab1003242044w4873c7eavbcab2fe2f1d61c4f@mail.gmail.com>

That's my fault, I'll fix it tomorrow (fighting with twisted.web + holiday)

On Wed, Mar 24, 2010 at 3:47 AM, Antonio Cuni <anto.cuni at gmail.com> wrote:
> On Tue, Mar 23, 2010 at 7:43 PM, Gasper Zejn <zejn at kiberpipa.org> wrote:
>
>> Nope, not working. The "directory listing" works, but not the links. Did you
>> click on any of the links provided on that url?
>
> Ah no, sorry. My fault, the links indeed don't work.
>
> ciao,
> Anto
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev
>


From fijall at gmail.com  Thu Mar 25 04:51:38 2010
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Wed, 24 Mar 2010 21:51:38 -0600
Subject: [pypy-dev] Branches
In-Reply-To: <20100323214355.GA30680@code0.codespeak.net>
References: <20100323214355.GA30680@code0.codespeak.net>
Message-ID: <693bc9ab1003242051s1f511605y4fc7cb18e90d64ce@mail.gmail.com>

On Tue, Mar 23, 2010 at 3:43 PM, Armin Rigo <arigo at tunes.org> wrote:
> Hi all,
>
> Can we decide if the following branches have been merged, are useful
> enough to be merged, need more work, "maybe one day", are very outdated,
> or turned out just to be a plain bad idea? ?The numbers are the rev
> number of the last change.
>
> Also, what about separating in-development branches that we expect to
> merge or kill soon, from the branches that are for "some other idea",
> e.g. "effect-analysis", which may not be merged in a long while but
> still we may want to keep around?
>
> ? ?* 70689 ?jit-profiling (pedronis, fijal)

Killed

> ? ?* 55472 ?judy-trees (fijal)

Killed

> ? ?* 64690 ?pypycpp (igorto)

I suppose that's to-be-killed

> ? ?* 70890 ?unroll-safe-if-const-arg (fijal, cfbolz, arigo)

I would like this to be kept around until jit is powerful enough to handle that.


From kai.aras.010 at googlemail.com  Thu Mar 25 05:00:36 2010
From: kai.aras.010 at googlemail.com (kai.aras)
Date: Thu, 25 Mar 2010 05:00:36 +0100
Subject: [pypy-dev] GSoC Project/Bachelor's Thesis
Message-ID: <3C1318CD-C3E3-421D-8EE0-C724CA98018B@googlemail.com>

Hi Everyone,

My name is Kai Aras, I'm a Computer-Science student from Germany and  
I'm very interested in doing my Bachelor's Thesis on PyPy and possibly  
also combine that with a GSoC project.

I became aware of the PyPy-Project by the Chaosradio Express (CRE088)  
episode about PyPy from 2008.
Last year, I had the opportunity to take some time to play around with  
PyPy in order to present it at my University, ever since that I became  
more and more attracted to the Python Language and the PyPy-Project  
itself, so writing my Bachelor's Thesis about something PyPy-related  
occured to me quite early.
Now, as the time has come, combining my Thesis with a GSoC Project  
would be a great opportunity.

During the last year, I've worked on a smarthome project using CPython  
on embedded-linux platforms,
as the project grew bigger, more and more problems arised and the need  
for some spezialized Python Runtime grew bigger as well.

I know that targeting Embedded Platforms was an Issue in the original  
EU-Project and there was that Maemo thing last year, so I'm guessing  
this could be a general topic for a GSoC Project as well as a  
Bachelor's Thesis, what do you guys think ? Is this something you  
would like to see as well ?
Personally, I'd love to use PyPy on openWRT, could this might be a  
target of general interest ?

This is just something I thought about from time to time but never  
really had the time to experiment with, so I'm open to all  
suggestions, please let me know if you have any ideas that could be  
suitable for such a project.

Thanks,
Kai


From arigo at tunes.org  Thu Mar 25 12:52:58 2010
From: arigo at tunes.org (Armin Rigo)
Date: Thu, 25 Mar 2010 12:52:58 +0100
Subject: [pypy-dev] GSoC projects
In-Reply-To: <1afaf6161003231831x70fa6394pacd874434f290627@mail.gmail.com>
References: <fe13bba81003231617h46f6443dya1f47184fe9b0a33@mail.gmail.com>
	<1afaf6161003231831x70fa6394pacd874434f290627@mail.gmail.com>
Message-ID: <20100325115258.GA10602@code0.codespeak.net>

Hi Benjamin, hi all,

On Tue, Mar 23, 2010 at 08:31:50PM -0500, Benjamin Peterson wrote:
> We have somebody who has voiced interest in doing a 64 bit backend,
> but there's no reason you couldn't assist them or work on some other
> aspect of PyPy.

I should probably mention here what I only explained in private mails so
far.  The 32-bit backend is in pypy/jit/backend/x86/.  However that
directory uses ri386.py and ri386setup.py, which are old and very custom
hacks based on multimethods in order to encode machine code
instructions.  I already started some work on replacing these two files.
It's done in this branch:

    http://codespeak.net/svn/pypy/branch/remove-ri386-multimethod-2

There, we have a single file rx86.py that is simpler, and supports both
32-bit and 64-bit instruction generation.  But the interface is very
different, so that means the whole rest of the backend has to be
adapted.  That would be the main work of a SoC student.

To get started, look at rx86.py and its test files test/test_rx86*.py
and try to write small examples that uses a subclass of
X86_32_CodeBuilder or X86_64_CodeBuilder in order to build and call a
test function.  The 64-bit version is more complicated, because the ABI
on 64-bit is to use some number of registers in order to pass arguments;
see details e.g. in http://www.x86-64.org/documentation/abi.pdf section
"Function Calling Sequence".


A bientot,

Armin.


From arigo at tunes.org  Thu Mar 25 13:04:34 2010
From: arigo at tunes.org (Armin Rigo)
Date: Thu, 25 Mar 2010 13:04:34 +0100
Subject: [pypy-dev] GSoC Project/Bachelor's Thesis
In-Reply-To: <3C1318CD-C3E3-421D-8EE0-C724CA98018B@googlemail.com>
References: <3C1318CD-C3E3-421D-8EE0-C724CA98018B@googlemail.com>
Message-ID: <20100325120434.GB10602@code0.codespeak.net>

Hi Kai,

On Thu, Mar 25, 2010 at 05:00:36AM +0100, kai.aras wrote:
> I know that targeting Embedded Platforms was an Issue in the original  
> EU-Project and there was that Maemo thing last year, so I'm guessing  
> this could be a general topic for a GSoC Project as well as a  
> Bachelor's Thesis, what do you guys think ? Is this something you  
> would like to see as well ?

I am afraid there is a gap (larger than ever) between the small amount
of "core people" and the large amount of directions we could explore.
Thank you for your interest; by no mean I intend to disappoint you, but
there is just no-one working on the general topic of embedding PyPy any
longer.  So it is very unlikely that you can get a SoC project on it.


A bientot,

Armin.


From arigo at tunes.org  Thu Mar 25 13:09:32 2010
From: arigo at tunes.org (Armin Rigo)
Date: Thu, 25 Mar 2010 13:09:32 +0100
Subject: [pypy-dev] Python Ireland Conference
In-Reply-To: <50a522ca1003190709k5f26aa67s31f0502c6791dd70@mail.gmail.com>
References: <50a522ca1003190709k5f26aa67s31f0502c6791dd70@mail.gmail.com>
Message-ID: <20100325120932.GC10602@code0.codespeak.net>

Hi Michael,

On Fri, Mar 19, 2010 at 02:09:27PM +0000, Michael Twomey wrote:
> I'm one of the folks involved in organising the first python Ireland
> conference, to be held in Dublin on Saturday 17th of July (with
> sprints on Sunday).
> 
> We'd love it if someone from PyPy could make it over to give a talk.
> The audience is your typical spread of python developers, many of whom
> haven't been exposed to PyPy and what it can do. An introduction to
> PyPy and the benefits it offers would go down very well.

Thanks for the invitation.  I think that the general "no-answer-coming-
so-far" situation means that we don't really know.  I would be
interested in coming myself, but I cannot give any more precise answer
right now.  Is there a deadline for you to get a more definite yes/no?


A bientot,

Armin.


From fijall at gmail.com  Thu Mar 25 15:34:55 2010
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Thu, 25 Mar 2010 08:34:55 -0600
Subject: [pypy-dev] GSoC Project/Bachelor's Thesis
In-Reply-To: <20100325120434.GB10602@code0.codespeak.net>
References: <3C1318CD-C3E3-421D-8EE0-C724CA98018B@googlemail.com> 
	<20100325120434.GB10602@code0.codespeak.net>
Message-ID: <693bc9ab1003250734s5b628a67h73d78abb69d2d343@mail.gmail.com>

Hello.

On Thu, Mar 25, 2010 at 6:04 AM, Armin Rigo <arigo at tunes.org> wrote:
> Hi Kai,
>
> On Thu, Mar 25, 2010 at 05:00:36AM +0100, kai.aras wrote:
>> I know that targeting Embedded Platforms was an Issue in the original
>> EU-Project and there was that Maemo thing last year, so I'm guessing
>> this could be a general topic for a GSoC Project as well as a
>> Bachelor's Thesis, what do you guys think ? Is this something you
>> would like to see as well ?
>
> I am afraid there is a gap (larger than ever) between the small amount
> of "core people" and the large amount of directions we could explore.
> Thank you for your interest; by no mean I intend to disappoint you, but
> there is just no-one working on the general topic of embedding PyPy any
> longer. ?So it is very unlikely that you can get a SoC project on it.
>
>

I think I would like to expand on that. In the past we had a bit of
problems with people doing work for SoC and then abandoning a working
prototype. Since it was out of interest of a core group, we just
killed the code or moved it to some obscure place at some later point.
In a way, we would like to encourage people to do some work which is
more of a core interest (at least for SoC) and if in doubt, we would
choose such people.

That said, personally I would encourage you to investigate this
direction for your bachelor if you feel like it. For example the idea
of having a JIT on embedded devices is for example a very researchy
area, since you need memory for the JIT (which is not free on embedded
devices).

Cheers,
fijal


From tobami at googlemail.com  Thu Mar 25 20:29:17 2010
From: tobami at googlemail.com (Miquel Torres)
Date: Thu, 25 Mar 2010 20:29:17 +0100
Subject: [pypy-dev] Quick "state of performance" analysis
Message-ID: <c561346d1003251229i7b4c2fabx99c0e4f82588da9d@mail.gmail.com>

Hi all!

I have implemented a small new feature for the speed center. In the
Overview, you are now able to select to which results you wish to compare
the performance of the selected interpreter. That allows a couple of
interesting investigations to be made (bear with me if I say obvious
things):

1 - http://speed.pypy.org/overview/?interpreter=3)
We already knew that there are 5 benchmarks where pypy-c without the jit is
more than twice times slower than cpython: slowspitfire, twisted_iteration,
twisted_tcp, html5lib and worst of all spambayes.

2 - http://speed.pypy.org/overview/
The jit makes up for some of them, only twisted_tcp, slowspitfire and
spambayes remain as cases where cpython is much faster.

3 - http://speed.pypy.org/overview/?baseline=4
We can now compare pypy-c-jit with cpython with psyco acceleration. The
first thing that stands out, is that pypy is much slower... exactly in the
same cases where pypy-c without the jit is slower than cpython (no
surprise). The second one, is that apart from those 5 cases where the jit is
held back by bad pypy-c baseline performance, pypy-c-jit is already much
better than psyco, about twice as fast!

4 - http://speed.pypy.org/overview/?baseline=3
Another new comparison is pypy-c-jit to pypy-c (revision 72012). Here we can
confirm that the jit accelerates *everything* that we currenly measure, and
most by a very good factor. The 3 slow benchmarks from comparison 2 are here
last, the are accelerated the least by the jit.

So to me the conclusion is that the jit has good enough performance for a
first version, and that only poor pypy-c performance in some cases is
holding the pypy project from becoming a better option than cpython.
Wouldn't it be a good aim to make it a priority of the next (1.3?) release
to improve on that? ;-)

Apart from it making pypy more "ready", there is a problem if that is not
addressed while further jit development continues. An application is only so
fast as its slowest part, so even if the jit accelerates some cases
100-fold, it won't help many applications.

I plan to add a new view with graphs that will show exactly what the problem
is.

Cheers!
Miquel


PD: you can also for example compare pypy-c-jit to pypy-c-jit 72012 and see
what improvements there have been since then
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20100325/56cd8869/attachment.html>

From santagada at gmail.com  Thu Mar 25 20:55:18 2010
From: santagada at gmail.com (Leonardo Santagada)
Date: Thu, 25 Mar 2010 16:55:18 -0300
Subject: [pypy-dev] Quick "state of performance" analysis
In-Reply-To: <c561346d1003251229i7b4c2fabx99c0e4f82588da9d@mail.gmail.com>
References: <c561346d1003251229i7b4c2fabx99c0e4f82588da9d@mail.gmail.com>
Message-ID: <F0D2478E-CE86-4083-8653-70A1603B12CA@gmail.com>


On Mar 25, 2010, at 4:29 PM, Miquel Torres wrote:

> Hi all!
> 
> I have implemented a small new feature for the speed center. In the Overview, you are now able to select to which results you wish to compare the performance of the selected interpreter. That allows a couple of interesting investigations to be made (bear with me if I say obvious things):
> 
> 1 - http://speed.pypy.org/overview/?interpreter=3)
> We already knew that there are 5 benchmarks where pypy-c without the jit is more than twice times slower than cpython: slowspitfire, twisted_iteration, twisted_tcp, html5lib and worst of all spambayes.
> 
> 2 - http://speed.pypy.org/overview/
> The jit makes up for some of them, only twisted_tcp, slowspitfire and spambayes remain as cases where cpython is much faster.
> 
> 3 - http://speed.pypy.org/overview/?baseline=4
> We can now compare pypy-c-jit with cpython with psyco acceleration. The first thing that stands out, is that pypy is much slower... exactly in the same cases where pypy-c without the jit is slower than cpython (no surprise). The second one, is that apart from those 5 cases where the jit is held back by bad pypy-c baseline performance, pypy-c-jit is already much better than psyco, about twice as fast!

Pretty cool, but is python with psyco using psyco.full or just jitting some parts? maybe there is a lot of room for manually tunning psyco (but there is no way to do this for pypy afaik).


> 4 - http://speed.pypy.org/overview/?baseline=3
> Another new comparison is pypy-c-jit to pypy-c (revision 72012). Here we can confirm that the jit accelerates *everything* that we currenly measure, and most by a very good factor. The 3 slow benchmarks from comparison 2 are here last, the are accelerated the least by the jit.
> 
> So to me the conclusion is that the jit has good enough performance for a first version, and that only poor pypy-c performance in some cases is holding the pypy project from becoming a better option than cpython. Wouldn't it be a good aim to make it a priority of the next (1.3?) release to improve on that? ;-)
> 
> Apart from it making pypy more "ready", there is a problem if that is not addressed while further jit development continues. An application is only so fast as its slowest part, so even if the jit accelerates some cases 100-fold, it won't help many applications.
> 
> I plan to add a new view with graphs that will show exactly what the problem is.
> 
> Cheers!
> Miquel
> 
> 
> PD: you can also for example compare pypy-c-jit to pypy-c-jit 72012 and see what improvements there have been since then
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev

--
Leonardo Santagada
santagada at gmail.com



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20100325/b43e4d59/attachment.html>

From william.leslie.ttg at gmail.com  Fri Mar 26 00:57:22 2010
From: william.leslie.ttg at gmail.com (William Leslie)
Date: Fri, 26 Mar 2010 10:57:22 +1100
Subject: [pypy-dev] Sandboxing pypy
In-Reply-To: <201003241101.56211.victor.stinner@haypocalc.com>
References: <ed932062c4806901e1c4caaccf6abd0f@mail.gmail.com>
	<201003241101.56211.victor.stinner@haypocalc.com>
Message-ID: <f45df7231003251657k75fff75dta3cda9e4cfe6a8dc@mail.gmail.com>

On 24 March 2010 21:01, Victor Stinner <victor.stinner at haypocalc.com> wrote:
> FYI I wrote a new sandbox project for CPython:
>
> ? http://github.com/haypo/pysandbox/
>
> It's currently very specific to CPython: it uses evil tricks to create a read
> only view of the __builtins__ super global dictionary.
>
> It's completly different to the PyPy sandbox: if you escape from the sandbox,
> you get a full access to all Python functions.
>
> A long description:
> http://mail.python.org/pipermail/python-dev/2010-February/097701.html

I didn't dive too deeply into the source, but what is to stop one from asking:

[o for o in (1).__class__.__bases__[0].__subclasses__() if o.__name__
== 'file'][0]('/etc/passwd').read()

?

-- 
William Leslie


From william.leslie.ttg at gmail.com  Fri Mar 26 01:06:26 2010
From: william.leslie.ttg at gmail.com (William Leslie)
Date: Fri, 26 Mar 2010 11:06:26 +1100
Subject: [pypy-dev] Sandboxing pypy
In-Reply-To: <f45df7231003251657k75fff75dta3cda9e4cfe6a8dc@mail.gmail.com>
References: <ed932062c4806901e1c4caaccf6abd0f@mail.gmail.com>
	<201003241101.56211.victor.stinner@haypocalc.com>
	<f45df7231003251657k75fff75dta3cda9e4cfe6a8dc@mail.gmail.com>
Message-ID: <f45df7231003251706y1e700975h89f9723ce447a896@mail.gmail.com>

On 26 March 2010 10:57, William Leslie <william.leslie.ttg at gmail.com> wrote:
>
> I didn't dive too deeply into the source, but what is to stop one from asking:
>
> [o for o in (1).__class__.__bases__[0].__subclasses__() if o.__name__
> == 'file'][0]('/etc/passwd').read()
>
> ?

Ah right, attributes.py.

>
> --
> William Leslie
>



-- 
William Leslie


From fijall at gmail.com  Fri Mar 26 02:05:21 2010
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Thu, 25 Mar 2010 19:05:21 -0600
Subject: [pypy-dev] various benchmark directories
Message-ID: <693bc9ab1003251805t389291e2le159c29ec6dd7f2@mail.gmail.com>

We have a couple of strange benchmark directories scattered a bit all
over the place in PyPy:

pypy/rpython/microbench
pypy/translator/goal/various
pypy/interpreter/callbench
pypy/translator/benchmark
build/benchmem
benchmarks

comes to mind

Do we need all of them? Can we kill some and factor the rest to be in one place?


From fijall at gmail.com  Fri Mar 26 06:44:31 2010
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Thu, 25 Mar 2010 23:44:31 -0600
Subject: [pypy-dev] Wikipedia pypy article
Message-ID: <693bc9ab1003252244v2bf6a3er6153199d4b77f977@mail.gmail.com>

http://en.wikipedia.org/wiki/PyPy

It's somewhat out of date, but also extremely confusing with notions
of meta-circular beast. Does someone feel like cleaning it up a bit?
Especially keeping it up to date would be cool.


From sl at scrooge.dk  Fri Mar 26 14:29:19 2010
From: sl at scrooge.dk (=?ISO-8859-1?Q?S=F8ren_Laursen?=)
Date: Fri, 26 Mar 2010 14:29:19 +0100
Subject: [pypy-dev] Sandboxing pypy
In-Reply-To: <20100324093854.GA18762@code0.codespeak.net>
References: <ed932062c4806901e1c4caaccf6abd0f@mail.gmail.com> 
	<20100324093854.GA18762@code0.codespeak.net>
Message-ID: <9e7424864518d7dcc66c8b4ba2cf89da@mail.gmail.com>

Thanks for all the replies.

I have started to look at vfs.py, I will join IRC later today.

Regards,

S?ren

-----Oprindelig meddelelse-----
Fra: Armin Rigo [mailto:arigo at tunes.org]
Sendt: 24. marts 2010 10:39
Til: S?ren Laursen
Cc: pypy-dev at codespeak.net
Emne: Re: [pypy-dev] Sandboxing pypy

Hi,

On Tue, Mar 23, 2010 at 10:46:50PM +0100, S?ren Laursen wrote:
> I have tried different things using the pypy_interact.py (just love the
> ?timeout parameter J), looked at code but have not been able to read
files
> or write files using the pypy_interact.py.

You get indeed a VFS (virtual file system) which is read-only so far.
You can read any file from the virtual path "/tmp/xxx" if you start
pypy_interact.py with the option "--tmp=some/path".  There is no support
yet to allow writes.  It could be easily added by editing vfs.py.


A bientot,

Armin.


From arigo at tunes.org  Sat Mar 27 11:04:57 2010
From: arigo at tunes.org (Armin Rigo)
Date: Sat, 27 Mar 2010 11:04:57 +0100
Subject: [pypy-dev] GSoC projects
In-Reply-To: <20100325115258.GA10602@code0.codespeak.net>
References: <fe13bba81003231617h46f6443dya1f47184fe9b0a33@mail.gmail.com>
	<1afaf6161003231831x70fa6394pacd874434f290627@mail.gmail.com>
	<20100325115258.GA10602@code0.codespeak.net>
Message-ID: <20100327100457.GA27151@code0.codespeak.net>

Re-hi,

On Thu, Mar 25, 2010 at 12:52:58PM +0100, Armin Rigo wrote:
> The 64-bit version is more complicated, because the ABI
> on 64-bit is to use some number of registers in order to pass arguments;
> see details e.g. in http://www.x86-64.org/documentation/abi.pdf section
> "Function Calling Sequence".

I should point out that we only need to pass integers (of at most
64-bit), pointers, and 64-bit floats (that's "double", in C) as
arguments to functions.  Don't go discouraged because of the messy rules
for passing structs, arrays, or 128- or 256-bit stuff.  :-)


A bientot,

Armin.


From kai.aras.010 at googlemail.com  Tue Mar 30 01:11:24 2010
From: kai.aras.010 at googlemail.com (kai.aras)
Date: Tue, 30 Mar 2010 01:11:24 +0200
Subject: [pypy-dev] GSoC Project/Bachelor's Thesis
In-Reply-To: <693bc9ab1003250734s5b628a67h73d78abb69d2d343@mail.gmail.com>
References: <3C1318CD-C3E3-421D-8EE0-C724CA98018B@googlemail.com>
	<20100325120434.GB10602@code0.codespeak.net>
	<693bc9ab1003250734s5b628a67h73d78abb69d2d343@mail.gmail.com>
Message-ID: <4FB67D66-EA50-4786-9D18-3FC9507A8163@googlemail.com>

Hi Armin, Hi Maciej,

first of all, thanks for your feedback and sorry for the late reply.

On Mar 25, 2010, at 3:34 PM, Maciej Fijalkowski wrote:

> Hello.
>
> On Thu, Mar 25, 2010 at 6:04 AM, Armin Rigo <arigo at tunes.org> wrote:
>> Hi Kai,
>>
>> On Thu, Mar 25, 2010 at 05:00:36AM +0100, kai.aras wrote:
>>> I know that targeting Embedded Platforms was an Issue in the  
>>> original
>>> EU-Project and there was that Maemo thing last year, so I'm guessing
>>> this could be a general topic for a GSoC Project as well as a
>>> Bachelor's Thesis, what do you guys think ? Is this something you
>>> would like to see as well ?
>>
>> I am afraid there is a gap (larger than ever) between the small  
>> amount
>> of "core people" and the large amount of directions we could explore.
>> Thank you for your interest; by no mean I intend to disappoint you,  
>> but
>> there is just no-one working on the general topic of embedding PyPy  
>> any
>> longer.  So it is very unlikely that you can get a SoC project on it.

I totally get that, no problem. It looks like I wont be able to make  
it in time for a SoC anyway.
However, if you don't think this is a complete dead-end, I'd still  
like to experiment with that for my Bachelor's Thesis,
otherwise please let me know.


> I think I would like to expand on that. In the past we had a bit of
> problems with people doing work for SoC and then abandoning a working
> prototype. Since it was out of interest of a core group, we just
> killed the code or moved it to some obscure place at some later point.
> In a way, we would like to encourage people to do some work which is
> more of a core interest (at least for SoC) and if in doubt, we would
> choose such people.
>
> That said, personally I would encourage you to investigate this
> direction for your bachelor if you feel like it. For example the idea
> of having a JIT on embedded devices is for example a very researchy
> area, since you need memory for the JIT (which is not free on embedded
> devices).
>
> Cheers,
> fijal

Looking into the JIT sounds quite interesting to me too, thanks for  
bringing that up.
I don't have a huge background in this field, so I  wouldn't really  
know how to get started on this,
maybe you could point me in the right direction here, any help would  
be appreciated.

Thanks,
Kai





From jcreigh at gmail.com  Tue Mar 30 05:12:47 2010
From: jcreigh at gmail.com (Jason Creighton)
Date: Mon, 29 Mar 2010 21:12:47 -0600
Subject: [pypy-dev] Rough draft of x86-64 JIT backend GSoC proposal
Message-ID: <4BB16C2F.5050300@gmail.com>

Hi Guys,

I am a student planning on applying to GSoC to create an x86-64 backend 
for the JIT. I have a (very) rough draft of my proposal, and I was 
hoping to get some feedback on it.

Specifically, I'm wondering about operating system support. I've written 
the proposal as if I would support Linux/Mac OS X/Windows.

I would be developing on Linux, so I think we can assume that would be 
fairly well supported, but obviously I would like it to work on Mac OS X 
and Windows as well. (I'm not sure if I would have the time/motivation 
to care about obscure BSDs). OTOH, if there are already a lot of 
outstanding issues on one of those platforms, I don't know that I would 
be able to get it working. So what do you think would be a reasonable 
goal here?

Secondly, my timeline is pretty vague. The PSF proposal template 
recommends a week-by-week timeline, but honestly, I'm not sure how the 
time usage would break down. Any comments on that would be greatly 
appreciated.

Here's the draft:

=== Proposal ===

The PyPy JIT, which has shown substantial performance improvements over
CPython, often several times faster, does not currently support the
x86-64 instruction set, making it impractical to use on 64-bit x86
systems.

My proposal is to extend the existing x86 JIT backend to support x86-64
as well.

=== Deliverables ===

Stable, tested 64-bit JIT for PyPy on Linux, Mac OS X and Windows merged
into PyPy trunk.

=== Implementation plan ===

This is not a research proposal. The goal is simply to have a PyPy JIT
that works out of the box on 64-bit CPUs, implemented as conservatively
as possible.

As such, I will attempt to reuse as much of the existing x86 backend
that I can. In fact, the architectural similarities between x86 and
x86-64 are large enough that I hope to implement a unified x86/x86-64
backend with the majority of the code working for either platform.

There is an existing branch that, while very incomplete, has the
beginnings of a unified x86/x86-64 instruction encoding module. I intend
to use that branch as a starting point.

Rough outline:

1. Take the existing "remove-ri386-multimethod-2" branch and use it as a 
basis for instruction encoding.

2. Port the existing 32-bit backend to use the new instruction encoding 
scheme.

3. Add 64-bit support to the backend,
         A) Modify register allocator to use new general purpose and
         floating point registers.
         B) Port "ResOperation" operations to 64-bit
         C) Port guard failure handling to 64-bit

4. Test 64-bit on Mac OS X and Windows and fix inevitable issues.

=== About Me ===

I am a first-year Computer Science student at Flathead Valley Community
College planning to transfer to Montana State University.

I have several years of professional development experience. I am
comfortable programming in Python, C and x86 assembly.

Starting May 17th, I will be able to commit 40 hours/week to the project
until the end of August. I may travel for a few weeks at some point in
the summer, but I will have a laptop with me with the expectation of
continuing full-time work.

=== Contact information ===

Email: jcreigh at gmail.com
IRC:   "jcreigh" on Freenode
Phone: (will be given on request, but not preferred)


From fijall at gmail.com  Tue Mar 30 17:14:53 2010
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Tue, 30 Mar 2010 09:14:53 -0600
Subject: [pypy-dev] Rough draft of x86-64 JIT backend GSoC proposal
In-Reply-To: <4BB16C2F.5050300@gmail.com>
References: <4BB16C2F.5050300@gmail.com>
Message-ID: <693bc9ab1003300814q6df14a0bo9a2e812e9aa78c26@mail.gmail.com>

On Mon, Mar 29, 2010 at 9:12 PM, Jason Creighton <jcreigh at gmail.com> wrote:
> Hi Guys,
>
> I am a student planning on applying to GSoC to create an x86-64 backend
> for the JIT. I have a (very) rough draft of my proposal, and I was
> hoping to get some feedback on it.
>
> Specifically, I'm wondering about operating system support. I've written
> the proposal as if I would support Linux/Mac OS X/Windows.

That would be great to have, but I would be happy enough to have it on
linux. Let's say that if pypy continues to work on os x and windows
you'll fix remaining 64bit JIT bugs.

>
> I would be developing on Linux, so I think we can assume that would be
> fairly well supported, but obviously I would like it to work on Mac OS X
> and Windows as well. (I'm not sure if I would have the time/motivation
> to care about obscure BSDs). OTOH, if there are already a lot of
> outstanding issues on one of those platforms, I don't know that I would
> be able to get it working. So what do you think would be a reasonable
> goal here?

It's not like pypy works on anything else than linux os x and windows.

>
> Secondly, my timeline is pretty vague. The PSF proposal template
> recommends a week-by-week timeline, but honestly, I'm not sure how the
> time usage would break down. Any comments on that would be greatly
> appreciated.

I think week based planning is ridiculous. That may be done for some
simpler stuff, but not for this. I would say have some milestones, but
avoid precise timing.

>
> Here's the draft:
>
> === Proposal ===
>
> The PyPy JIT, which has shown substantial performance improvements over
> CPython, often several times faster, does not currently support the
> x86-64 instruction set, making it impractical to use on 64-bit x86
> systems.

and over unladen swallow.

>
> My proposal is to extend the existing x86 JIT backend to support x86-64
> as well.
>
> === Deliverables ===
>
> Stable, tested 64-bit JIT for PyPy on Linux, Mac OS X and Windows merged
> into PyPy trunk.
>
> === Implementation plan ===
>
> This is not a research proposal. The goal is simply to have a PyPy JIT
> that works out of the box on 64-bit CPUs, implemented as conservatively
> as possible.
>
> As such, I will attempt to reuse as much of the existing x86 backend
> that I can. In fact, the architectural similarities between x86 and
> x86-64 are large enough that I hope to implement a unified x86/x86-64
> backend with the majority of the code working for either platform.
>
> There is an existing branch that, while very incomplete, has the
> beginnings of a unified x86/x86-64 instruction encoding module. I intend
> to use that branch as a starting point.
>
> Rough outline:
>
> 1. Take the existing "remove-ri386-multimethod-2" branch and use it as a
> basis for instruction encoding.
>
> 2. Port the existing 32-bit backend to use the new instruction encoding
> scheme.
>
> 3. Add 64-bit support to the backend,
> ? ? ? ? A) Modify register allocator to use new general purpose and
> ? ? ? ? floating point registers.
> ? ? ? ? B) Port "ResOperation" operations to 64-bit
> ? ? ? ? C) Port guard failure handling to 64-bit
>
> 4. Test 64-bit on Mac OS X and Windows and fix inevitable issues.

There is also part about asmgcc.

>
> === About Me ===
>
> I am a first-year Computer Science student at Flathead Valley Community
> College planning to transfer to Montana State University.
>
> I have several years of professional development experience. I am
> comfortable programming in Python, C and x86 assembly.
>
> Starting May 17th, I will be able to commit 40 hours/week to the project
> until the end of August. I may travel for a few weeks at some point in
> the summer, but I will have a laptop with me with the expectation of
> continuing full-time work.
>
> === Contact information ===
>
> Email: jcreigh at gmail.com
> IRC: ? "jcreigh" on Freenode
> Phone: (will be given on request, but not preferred)
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev
>

Proposal is rather short, but I don't see how to expand it. You might
want to add that in case you finish this stuff earlier you would do
this and that (or just general JIT work or not do anything and drink
beer).


From glavoie at gmail.com  Tue Mar 30 17:40:28 2010
From: glavoie at gmail.com (Gabriel Lavoie)
Date: Tue, 30 Mar 2010 11:40:28 -0400
Subject: [pypy-dev] Rough draft of x86-64 JIT backend GSoC proposal
In-Reply-To: <693bc9ab1003300814q6df14a0bo9a2e812e9aa78c26@mail.gmail.com>
References: <4BB16C2F.5050300@gmail.com>
	<693bc9ab1003300814q6df14a0bo9a2e812e9aa78c26@mail.gmail.com>
Message-ID: <d05df8621003300840t69137b72v980eb0f8ee216ece@mail.gmail.com>

Starting from May 1 I leave my current job and I'm returning full time at
university to work on my MSc project with PyPy. A FreeBSD 7.3 x86_64
buildbot from me is already available for the PyPy project. I also work
under MacOS X (Snow Leopard) for my MSc project. I could probably help you
for the particularities of these two platforms.

Keep me updated!

Gabriel Lavoie

2010/3/30 Maciej Fijalkowski <fijall at gmail.com>

> On Mon, Mar 29, 2010 at 9:12 PM, Jason Creighton <jcreigh at gmail.com>
> wrote:
> > Hi Guys,
> >
> > I am a student planning on applying to GSoC to create an x86-64 backend
> > for the JIT. I have a (very) rough draft of my proposal, and I was
> > hoping to get some feedback on it.
> >
> > Specifically, I'm wondering about operating system support. I've written
> > the proposal as if I would support Linux/Mac OS X/Windows.
>
> That would be great to have, but I would be happy enough to have it on
> linux. Let's say that if pypy continues to work on os x and windows
> you'll fix remaining 64bit JIT bugs.
>
> >
> > I would be developing on Linux, so I think we can assume that would be
> > fairly well supported, but obviously I would like it to work on Mac OS X
> > and Windows as well. (I'm not sure if I would have the time/motivation
> > to care about obscure BSDs). OTOH, if there are already a lot of
> > outstanding issues on one of those platforms, I don't know that I would
> > be able to get it working. So what do you think would be a reasonable
> > goal here?
>
> It's not like pypy works on anything else than linux os x and windows.
>
> >
> > Secondly, my timeline is pretty vague. The PSF proposal template
> > recommends a week-by-week timeline, but honestly, I'm not sure how the
> > time usage would break down. Any comments on that would be greatly
> > appreciated.
>
> I think week based planning is ridiculous. That may be done for some
> simpler stuff, but not for this. I would say have some milestones, but
> avoid precise timing.
>
> >
> > Here's the draft:
> >
> > === Proposal ===
> >
> > The PyPy JIT, which has shown substantial performance improvements over
> > CPython, often several times faster, does not currently support the
> > x86-64 instruction set, making it impractical to use on 64-bit x86
> > systems.
>
> and over unladen swallow.
>
> >
> > My proposal is to extend the existing x86 JIT backend to support x86-64
> > as well.
> >
> > === Deliverables ===
> >
> > Stable, tested 64-bit JIT for PyPy on Linux, Mac OS X and Windows merged
> > into PyPy trunk.
> >
> > === Implementation plan ===
> >
> > This is not a research proposal. The goal is simply to have a PyPy JIT
> > that works out of the box on 64-bit CPUs, implemented as conservatively
> > as possible.
> >
> > As such, I will attempt to reuse as much of the existing x86 backend
> > that I can. In fact, the architectural similarities between x86 and
> > x86-64 are large enough that I hope to implement a unified x86/x86-64
> > backend with the majority of the code working for either platform.
> >
> > There is an existing branch that, while very incomplete, has the
> > beginnings of a unified x86/x86-64 instruction encoding module. I intend
> > to use that branch as a starting point.
> >
> > Rough outline:
> >
> > 1. Take the existing "remove-ri386-multimethod-2" branch and use it as a
> > basis for instruction encoding.
> >
> > 2. Port the existing 32-bit backend to use the new instruction encoding
> > scheme.
> >
> > 3. Add 64-bit support to the backend,
> >         A) Modify register allocator to use new general purpose and
> >         floating point registers.
> >         B) Port "ResOperation" operations to 64-bit
> >         C) Port guard failure handling to 64-bit
> >
> > 4. Test 64-bit on Mac OS X and Windows and fix inevitable issues.
>
> There is also part about asmgcc.
>
> >
> > === About Me ===
> >
> > I am a first-year Computer Science student at Flathead Valley Community
> > College planning to transfer to Montana State University.
> >
> > I have several years of professional development experience. I am
> > comfortable programming in Python, C and x86 assembly.
> >
> > Starting May 17th, I will be able to commit 40 hours/week to the project
> > until the end of August. I may travel for a few weeks at some point in
> > the summer, but I will have a laptop with me with the expectation of
> > continuing full-time work.
> >
> > === Contact information ===
> >
> > Email: jcreigh at gmail.com
> > IRC:   "jcreigh" on Freenode
> > Phone: (will be given on request, but not preferred)
> > _______________________________________________
> > pypy-dev at codespeak.net
> > http://codespeak.net/mailman/listinfo/pypy-dev
> >
>
> Proposal is rather short, but I don't see how to expand it. You might
> want to add that in case you finish this stuff earlier you would do
> this and that (or just general JIT work or not do anything and drink
> beer).
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev
>



-- 
Gabriel Lavoie
glavoie at gmail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20100330/6305efe4/attachment.html>

From jcreigh at gmail.com  Wed Mar 31 00:18:03 2010
From: jcreigh at gmail.com (Jason Creighton)
Date: Tue, 30 Mar 2010 16:18:03 -0600
Subject: [pypy-dev] Rough draft of x86-64 JIT backend GSoC proposal
In-Reply-To: <693bc9ab1003300814q6df14a0bo9a2e812e9aa78c26@mail.gmail.com>
References: <4BB16C2F.5050300@gmail.com>
	<693bc9ab1003300814q6df14a0bo9a2e812e9aa78c26@mail.gmail.com>
Message-ID: <4BB2789B.70406@gmail.com>

I agree with fijal that it seems short, but I don't know what else to 
put in.

Updated draft:

=== Proposal ===

The PyPy JIT, which has shown substantial performance improvements over
other implementations of Python, including CPython and Unladen Swallow,
does not currently support the x86-64 instruction set, making it
impractical to use on 64-bit x86 systems.

My proposal is to extend the existing x86 JIT backend to support x86-64
as well.

=== Deliverables ===

Stable, tested 64-bit JIT for PyPy on Linux, Mac OS X and Windows merged
into PyPy trunk.

=== Implementation plan ===

This is not a research proposal. The goal is simply to have a PyPy JIT
that works out of the box on 64-bit CPUs, implemented as conservatively
as possible.

As such, I will attempt to reuse as much of the existing x86 backend
that I can. In fact, the architectural similarities between x86 and
x86-64 are large enough that I hope to implement a unified x86/x86-64
backend with the majority of the code working for either platform.

There is an existing branch that, while very incomplete, has the
beginnings of a unified x86/x86-64 instruction encoding module, which
can encode instructions for either x86 or x86-64 in a fairly seamless
manner. I intend to use that branch as a starting point.

Milestones:

1. Take the existing "remove-ri386-multimethod-2" branch and use it as a
basis for instruction encoding. Modify the existing 32-bit backend to
use the new instruction encoding scheme.

2. Add 64-bit support to the backend,
         A) Add tests to ensure that 64-bit instructions are being
         generated correctly.
         B) Modify register allocator to use new general purpose and
         floating point registers.
         C) Port 32-bit specific portions of the JIT (for example, guard
         failure handling) to 64-bit

3. Test 64-bit on Mac OS X and Windows, and fix inevitable issues.

If the 64-bit JIT is completed before the end of the summer, I will
spend the remainder of the time working on other 64-bit or JIT-related
aspects of PyPy, for example adapting the "asmgcc" garbage collection
strategy to 64-bit.

=== Development Workflow ===

The PyPy project makes extensive use of Subversion branches for
development, so I will follow that convention and develop the 64-bit JIT
in one or more branches.

For unit testing, I will use the py.test framework (already used
throughout PyPy), aiming for 100% test coverage.

=== About Me ===

I am a first-year Computer Science student at Flathead Valley Community
College planning to transfer to Montana State University.

I have several years of professional development experience. I am
comfortable programming in Python, C and x86 assembly.

Starting May 17th, I will be able to commit 40 hours/week to the project
until the end of August. I may travel for a few weeks at some point in
the summer, but I will have a laptop with me with the expectation of
continuing full-time work.

=== Contact information ===

Email: jcreigh at gmail.com
IRC:   "jcreigh" on Freenode
Blog:  http://jcreigh.blogspot.com
Phone: <will be included in actual proposal, but pypy-dev is publicly 
archived...>


From fijall at gmail.com  Wed Mar 31 04:03:58 2010
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Tue, 30 Mar 2010 20:03:58 -0600
Subject: [pypy-dev] Rough draft of x86-64 JIT backend GSoC proposal
In-Reply-To: <4BB2789B.70406@gmail.com>
References: <4BB16C2F.5050300@gmail.com>
	<693bc9ab1003300814q6df14a0bo9a2e812e9aa78c26@mail.gmail.com> 
	<4BB2789B.70406@gmail.com>
Message-ID: <y2j693bc9ab1003301903gfbdc2014ld1af2113bfc87a36@mail.gmail.com>

Just a meta-note, how about keeping it in svn? email is rarely a good
version control system (that's why you have your user dir on
codespeak, or you can create one in /svn/user)

On Tue, Mar 30, 2010 at 4:18 PM, Jason Creighton <jcreigh at gmail.com> wrote:
> I agree with fijal that it seems short, but I don't know what else to
> put in.
>
> Updated draft:
>
> === Proposal ===
>
> The PyPy JIT, which has shown substantial performance improvements over
> other implementations of Python, including CPython and Unladen Swallow,
> does not currently support the x86-64 instruction set, making it
> impractical to use on 64-bit x86 systems.
>
> My proposal is to extend the existing x86 JIT backend to support x86-64
> as well.
>
> === Deliverables ===
>
> Stable, tested 64-bit JIT for PyPy on Linux, Mac OS X and Windows merged
> into PyPy trunk.
>
> === Implementation plan ===
>
> This is not a research proposal. The goal is simply to have a PyPy JIT
> that works out of the box on 64-bit CPUs, implemented as conservatively
> as possible.
>
> As such, I will attempt to reuse as much of the existing x86 backend
> that I can. In fact, the architectural similarities between x86 and
> x86-64 are large enough that I hope to implement a unified x86/x86-64
> backend with the majority of the code working for either platform.
>
> There is an existing branch that, while very incomplete, has the
> beginnings of a unified x86/x86-64 instruction encoding module, which
> can encode instructions for either x86 or x86-64 in a fairly seamless
> manner. I intend to use that branch as a starting point.
>
> Milestones:
>
> 1. Take the existing "remove-ri386-multimethod-2" branch and use it as a
> basis for instruction encoding. Modify the existing 32-bit backend to
> use the new instruction encoding scheme.
>
> 2. Add 64-bit support to the backend,
> ? ? ? ? A) Add tests to ensure that 64-bit instructions are being
> ? ? ? ? generated correctly.
> ? ? ? ? B) Modify register allocator to use new general purpose and
> ? ? ? ? floating point registers.
> ? ? ? ? C) Port 32-bit specific portions of the JIT (for example, guard
> ? ? ? ? failure handling) to 64-bit
>
> 3. Test 64-bit on Mac OS X and Windows, and fix inevitable issues.
>
> If the 64-bit JIT is completed before the end of the summer, I will
> spend the remainder of the time working on other 64-bit or JIT-related
> aspects of PyPy, for example adapting the "asmgcc" garbage collection
> strategy to 64-bit.
>
> === Development Workflow ===
>
> The PyPy project makes extensive use of Subversion branches for
> development, so I will follow that convention and develop the 64-bit JIT
> in one or more branches.
>
> For unit testing, I will use the py.test framework (already used
> throughout PyPy), aiming for 100% test coverage.
>
> === About Me ===
>
> I am a first-year Computer Science student at Flathead Valley Community
> College planning to transfer to Montana State University.
>
> I have several years of professional development experience. I am
> comfortable programming in Python, C and x86 assembly.
>
> Starting May 17th, I will be able to commit 40 hours/week to the project
> until the end of August. I may travel for a few weeks at some point in
> the summer, but I will have a laptop with me with the expectation of
> continuing full-time work.
>
> === Contact information ===
>
> Email: jcreigh at gmail.com
> IRC: ? "jcreigh" on Freenode
> Blog: ?http://jcreigh.blogspot.com
> Phone: <will be included in actual proposal, but pypy-dev is publicly
> archived...>
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev
>


From jcreigh at gmail.com  Wed Mar 31 19:24:07 2010
From: jcreigh at gmail.com (Jason Creighton)
Date: Wed, 31 Mar 2010 11:24:07 -0600
Subject: [pypy-dev] Rough draft of x86-64 JIT backend GSoC proposal
In-Reply-To: <20100330221477.SM416116@[207.67.72.231]>
References: <20100330221477.SM416116@[207.67.72.231]>
Message-ID: <4BB38537.1060005@gmail.com>

michaelschneider wrote:
> Jason,
> 
> My name is Mike Schneider (bigdog on the irc).
> 
> I think what you have is very good.  Take these as suggestions.

I found them extremely helpful, thank you very much.

I've tried to implement them in my latest draft, and I've also taken 
fijal's suggestion of storing it in svn. Here it is, if anyone has any 
further feedback/suggestions:

http://codespeak.net/svn/user/jcreigh/documents/pypy_amd64_gsoc_proposal.txt

Cheers,

Jason