From fijal at genesilico.pl  Fri Jun  1 08:45:54 2007
From: fijal at genesilico.pl (Maciek Fijalkowski)
Date: Fri, 01 Jun 2007 08:45:54 +0200
Subject: [pypy-dev] Using pypy for real world applications part pi/2
Message-ID: <465FC0A2.5070100@genesilico.pl>

I've found an unusual problem regarding rctypes - in implementation of 
fcntl.ioctl, ioctl
can return a string containing 0 bytes, because the length is somehow 
defined. Our rctypes (or implementation itself, I failed to find by now) 
assume that string is null terminated. The unusual thing about this is 
that on top of ctypes it works perfectly well.

Any ideas how to fix it?

I think I'll go to bed now and wait for someone else to take a look...

Cheers,
fijal



From holger at merlinux.de  Fri Jun  1 18:34:27 2007
From: holger at merlinux.de (holger krekel)
Date: Fri, 1 Jun 2007 18:34:27 +0200
Subject: [pypy-dev] PyPy fully passed its final EU review!
Message-ID: <20070601163426.GI25045@solar.trillke>

Hi all, 

we yesterday successfully passed the EU review of 
our 28 month PyPy EU project period!

We gave almost 6 hours worth of talks and got a lot of good questions 
and remarks ... all of our results met appreciation! 

I am relieved - and my thanks to everybody involved 
and to everyone who helped!

cheers & best,

holger

P.S.: 
    As i've heard the thought from some people that the EU might "prolong 
    funding" if PyPy is successful: this is not how it works. We had a 
    research contract and we fullfilled that contract - if we were to have a 
    follow up project, we'd have to submit a new research proposal and go 
    through another application and contractual process. 

P.P.S.: the EU project is still not 100% final - money and 
        budget finalisation / audits remain ... 


-- 
merlinux GmbH       Steinbergstr. 42    31139 Hildesheim   
http://merlinux.de  tel +49 5121 20800 75 (fax 77) 


From micahel at gmail.com  Fri Jun  1 19:01:27 2007
From: micahel at gmail.com (Michael Hudson)
Date: Fri, 1 Jun 2007 18:01:27 +0100
Subject: [pypy-dev] PyPy fully passed its final EU review!
In-Reply-To: <20070601163426.GI25045@solar.trillke>
References: <20070601163426.GI25045@solar.trillke>
Message-ID: <aa0dfd860706011001i5c20504aq961fd5d532f40751@mail.gmail.com>

On 01/06/07, holger krekel <holger at merlinux.de> wrote:
> Hi all,
>
> we yesterday successfully passed the EU review of
> our 28 month PyPy EU project period!

Hooray!

Cheers,
mwh


From anto.cuni at gmail.com  Fri Jun  1 19:48:26 2007
From: anto.cuni at gmail.com (Antonio Cuni)
Date: Fri, 01 Jun 2007 19:48:26 +0200
Subject: [pypy-dev] PyPy fully passed its final EU review!
In-Reply-To: <20070601163426.GI25045@solar.trillke>
References: <20070601163426.GI25045@solar.trillke>
Message-ID: <46605BEA.5050307@gmail.com>

holger krekel wrote:
> Hi all, 
> 
> we yesterday successfully passed the EU review of 
> our 28 month PyPy EU project period!

cool :-).

Congratulations to all the people involved in the review!



From fijal at genesilico.pl  Fri Jun  1 23:35:47 2007
From: fijal at genesilico.pl (Maciek Fijalkowski)
Date: Fri, 01 Jun 2007 23:35:47 +0200
Subject: [pypy-dev] pypy-c and pyrepl
Message-ID: <46609133.8080402@genesilico.pl>

Is running right now :)

but on branch.



From bea at changemaker.nu  Fri Jun  1 23:52:35 2007
From: bea at changemaker.nu (=?ISO-8859-1?Q?Beatrice_D=FCring?=)
Date: Fri, 01 Jun 2007 23:52:35 +0200
Subject: [pypy-dev] PyPy fully passed its final EU review!
In-Reply-To: <46605BEA.5050307@gmail.com>
References: <20070601163426.GI25045@solar.trillke> <46605BEA.5050307@gmail.com>
Message-ID: <46609523.40300@changemaker.nu>

Hi there

Antonio Cuni skrev:
> holger krekel wrote:
>   
>> Hi all, 
>>
>> we yesterday successfully passed the EU review of 
>> our 28 month PyPy EU project period!
>>     
>
> cool :-).
>
> Congratulations to all the people involved in the review!
>
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev
>   

Our Project Officer at the Commission summarized their approval with stating
that we had achieved "excellent technical results"! So that means a huge 
thanks
and a big congratulation coming right back at you Anto - and all others 
contributing
during this time!

Cheers

Bea


From fijal at genesilico.pl  Sat Jun  2 16:47:38 2007
From: fijal at genesilico.pl (Maciek Fijalkowski)
Date: Sat, 02 Jun 2007 16:47:38 +0200
Subject: [pypy-dev] Using pypy for real world applications part pi/2
In-Reply-To: <465FC0A2.5070100@genesilico.pl>
References: <465FC0A2.5070100@genesilico.pl>
Message-ID: <4661830A.4010009@genesilico.pl>

Maciek Fijalkowski wrote:
> I've found an unusual problem regarding rctypes - in implementation of 
> fcntl.ioctl, ioctl
> can return a string containing 0 bytes, because the length is somehow 
> defined. Our rctypes (or implementation itself, I failed to find by now) 
> assume that string is null terminated. The unusual thing about this is 
> that on top of ctypes it works perfectly well.
>
> Any ideas how to fix it?
>
> I think I'll go to bed now and wait for someone else to take a look...
>
> Cheers,
> fijal
Actually, this was a minor bug in fcntl module, fixed now.

Cheers,
fijal



From fijal at genesilico.pl  Sat Jun  2 16:50:44 2007
From: fijal at genesilico.pl (Maciek Fijalkowski)
Date: Sat, 02 Jun 2007 16:50:44 +0200
Subject: [pypy-dev] Unusual problem.
In-Reply-To: <20070530194636.GA21548@code0.codespeak.net>
References: <465C5863.50902@genesilico.pl>
	<20070530194636.GA21548@code0.codespeak.net>
Message-ID: <466183C4.5040809@genesilico.pl>

Armin Rigo wrote:
> Hi Maciek,
>
> On Tue, May 29, 2007 at 06:44:19PM +0200, Maciek Fijalkowski wrote:
>   
>> termios.tcgetattr (and others as well) use termios.error for error 
>> reporting, which is exception which cannot be used in a translated 
>> version.
>>     
>
> Why not?
>
>
> Armin
>   
Because than, annotator complains  about except termios.error, e: e.args[1]

I fixed the bug with doing strange hacks in the module-initialization 
code, I've subclassed the exception and by temporarily re-raising it in 
module startup, I force e.args to be more regular, but this is a bit 
ugly trick (there is module/termios/__init__.py function startup for 
this purpose). Breaks for example compilemodule and it's ugly.

Cheers,
fijal




From arigo at tunes.org  Sat Jun  2 16:55:29 2007
From: arigo at tunes.org (Armin Rigo)
Date: Sat, 2 Jun 2007 16:55:29 +0200
Subject: [pypy-dev] Mini sprint in Poland
In-Reply-To: <a26b53540705231629t4840d11end5c329a3c6802ae9@mail.gmail.com>
References: <a26b53540705231629t4840d11end5c329a3c6802ae9@mail.gmail.com>
Message-ID: <20070602145529.GA2634@code0.codespeak.net>

Hi Jakub,

On Thu, May 24, 2007 at 01:29:30AM +0200, Jakub Gustak wrote:
> I would like to organize mini sprint (2-3 days) in Wroclaw.
> 
> It would be hold on Wroclaw University of Technology.
> There are possible two alternative dates:
> 2-3th June
> 16(15)-17th June
> It depends how fast we can organize ourselves.

Sorry for not coming back to you earlier on this.  I was (probably just
like many others) quite too much involved in preparing the EU review for
this...  I will also need to rest now, so I'm not up to travelling to
Wroclaw at the moment.  It seems that you didn't get much feedback about
the sprint...

I can confirm that we plan a PyPy-related sprint at EuroPython in
about 1 month now - after EuroPython, so 12-14 July.
See http://wiki.python.org/moin/EuroPython2007Sprints.


A bientot,

Armin.


From arigo at tunes.org  Sat Jun  2 16:57:05 2007
From: arigo at tunes.org (Armin Rigo)
Date: Sat, 2 Jun 2007 16:57:05 +0200
Subject: [pypy-dev] Unusual problem.
In-Reply-To: <466183C4.5040809@genesilico.pl>
References: <465C5863.50902@genesilico.pl>
	<20070530194636.GA21548@code0.codespeak.net>
	<466183C4.5040809@genesilico.pl>
Message-ID: <20070602145705.GB2634@code0.codespeak.net>

Hi Maciek,

On Sat, Jun 02, 2007 at 04:50:44PM +0200, Maciek Fijalkowski wrote:
> Because than, annotator complains  about except termios.error, e: e.args[1]

Ah, because you need to carry an error code.  That's the same as with
OSError, so you should fix it in the same (hackish) way - grep for
OSError in pypy/annotation/  :-)


A bientot,

Armin.


From santagada at gmail.com  Sun Jun  3 17:34:44 2007
From: santagada at gmail.com (Leonardo Santagada)
Date: Sun, 3 Jun 2007 12:34:44 -0300
Subject: [pypy-dev] Problems with parsing
Message-ID: <E7A2D250-5636-45CA-BF0A-74A161BACC07@gmail.com>

I'm having 2 problems with parsing js files, so i'm sending this  
email in a call for help:

1) My string regex is not working the one i'm using now:

stringliteral   : <SINGLESTRING>
                 | <DOUBLESTRING>
                 ;

SINGLESTRING    : "'([^']|\\')*'"
                 ;

DOUBLESTRING    : "\"([^\"]|\\\")*\""
                 ;

don't work with this valid js string: "\'\"\\"

2) The parsing module is reaching recursion limit on /pypy/lang/js/ 
test/ecma/FunctionObjects/15.3.1.1-3.js if you uncomment the code. I  
get the idea of where the problem lies, we have there a function with  
2k params, but still, it works on spidermonkey and I think I should  
support this. you can test this by going to the dir and type "py.test  
--ecna 15.3.1.1-3.js" to see what happens.


thanks in advance,
--
Leonardo Santagada
"If it looks like a duck, and quacks like a duck, we have at least to  
consider the possibility that we have a small aquatic bird of the  
family anatidae on our hands." - Douglas Adams





From fijal at genesilico.pl  Wed Jun  6 18:58:12 2007
From: fijal at genesilico.pl (Maciek Fijalkowski)
Date: Wed, 06 Jun 2007 18:58:12 +0200
Subject: [pypy-dev] PyPy sync next week?
Message-ID: <4666E7A4.9030407@genesilico.pl>

I know you're quite tired right now and all, but I think it would be 
good to make some kind of pypy-sync next week or even the week after (we 
need to decide on sprint topics anyway).

Particularly I would like to know people's opinion on pypy's scope. What 
we would like to support, what we would like to drop support for, what 
are the short-term project directions etc.

What do you think?

Cheers,
fijal



From delza at livingcode.org  Wed Jun  6 19:55:34 2007
From: delza at livingcode.org (Dethe Elza)
Date: Wed, 6 Jun 2007 10:55:34 -0700
Subject: [pypy-dev] PyPy sync next week?
In-Reply-To: <4666E7A4.9030407@genesilico.pl>
References: <4666E7A4.9030407@genesilico.pl>
Message-ID: <F9F27639-A627-49CD-8D4C-1D7628D3B512@livingcode.org>

On 6-Jun-07, at 9:58 AM, Maciek Fijalkowski wrote:
> I know you're quite tired right now and all, but I think it would be
> good to make some kind of pypy-sync next week or even the week  
> after (we
> need to decide on sprint topics anyway).

I'm quite new to this list.  Could you explain what a sync is?  Would  
it be face-to-face, via this list, or IRC?

> Particularly I would like to know people's opinion on pypy's scope.  
> What
> we would like to support, what we would like to drop support for, what
> are the short-term project directions etc.

I'm quite interested in the scope question.  I would love to see  
Python adopt Erlang-like concurrency primitives, but that doesn't  
appear to be a direction Guido and the C-Python team are willing to  
take, making PyPy my best hope for this.

Thanks!

--Dethe


Simple things should be declarative.  Complex things should be  
procedural.  --Adam Bosworth




From cfbolz at gmx.de  Wed Jun  6 23:03:37 2007
From: cfbolz at gmx.de (Carl Friedrich Bolz)
Date: Wed, 06 Jun 2007 23:03:37 +0200
Subject: [pypy-dev] PyPy sync next week?
In-Reply-To: <F9F27639-A627-49CD-8D4C-1D7628D3B512@livingcode.org>
References: <4666E7A4.9030407@genesilico.pl>
	<F9F27639-A627-49CD-8D4C-1D7628D3B512@livingcode.org>
Message-ID: <46672129.8000603@gmx.de>

Dethe Elza wrote:
 > On 6-Jun-07, at 9:58 AM, Maciek Fijalkowski wrote:
 >> I know you're quite tired right now and all, but I think it would be
 >> good to make some kind of pypy-sync next week or even the week
 >> after (we
 >> need to decide on sprint topics anyway).
 >
 > I'm quite new to this list.  Could you explain what a sync is?  Would
 > it be face-to-face, via this list, or IRC?

A sync-meeting is a 30 minute IRC meeting where active PyPy developers
meet to discuss current issues. There is a fixed part, where everybody
posts three prepared lines (LAST, NEXT, BLOCKERS) about what he worked
in the last days, what he plans to work on and what is blocking his
work. Afterwards there are topics that vary between the meetings. Feel
free to attend!

 >> Particularly I would like to know people's opinion on pypy's scope.

This sounds more like a mailing list discussion to me, since it can
quickly get out of hand. I would propose to have a smaller discussion at
a sync-meeting about the plans of the individuals in the next months, to
get a bit of an overview. Afterwards we can start some common planning
of what good next steps would be (which would include a rough sprint
planning). How does that sound?


 >> What
 >> we would like to support, what we would like to drop support for, what
 >> are the short-term project directions etc.
 >
 > I'm quite interested in the scope question.  I would love to see
 > Python adopt Erlang-like concurrency primitives, but that doesn't
 > appear to be a direction Guido and the C-Python team are willing to
 > take, making PyPy my best hope for this.

If we discuss scope we are probably discussing how to reduce our scope,
since we are doing a lot already. So if you want any new features, you
would have to implement them yourself. I don't know Erlang, but it's
possible that you can just implement this stuff on top of PyPy's Python
interpreter with the help of stackless features rather easily.

Cheers,

Carl Friedrich


From bea at changemaker.nu  Thu Jun  7 15:20:06 2007
From: bea at changemaker.nu (=?UTF-8?B?QmVhdHJpY2UgRMO8cmluZw==?=)
Date: Thu, 07 Jun 2007 15:20:06 +0200
Subject: [pypy-dev] [Fwd: Call for International Journal Papers,]
Message-ID: <46680606.6010609@changemaker.nu>

Hi there

Anyone (still) interested in writing papers ;-)?

Cheers

Bea
-------------- next part --------------
An embedded message was scrubbed...
From: <editor at ijcsns.org>
Subject: Call for International Journal Papers,
Date: Thu, 7 Jun 2007 12:27:56 +0900
Size: 12825
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20070607/64cda13f/attachment.mht>

From santagada at gmail.com  Thu Jun  7 20:34:36 2007
From: santagada at gmail.com (Leonardo Santagada)
Date: Thu, 7 Jun 2007 15:34:36 -0300
Subject: [pypy-dev] play1.pypy.org not working
Message-ID: <B2FC98B7-DF8F-4FC4-B9A6-54B57B3C5709@gmail.com>

I tried it today and found a 502 proxy error.


--
Leonardo Santagada
"If it looks like a duck, and quacks like a duck, we have at least to  
consider the possibility that we have a small aquatic bird of the  
family anatidae on our hands." - Douglas Adams





From fijal at genesilico.pl  Sun Jun 10 20:44:50 2007
From: fijal at genesilico.pl (Maciek Fijalkowski)
Date: Sun, 10 Jun 2007 20:44:50 +0200
Subject: [pypy-dev] PyPy sync next week?
In-Reply-To: <4666E7A4.9030407@genesilico.pl>
References: <4666E7A4.9030407@genesilico.pl>
Message-ID: <466C46A2.9040108@genesilico.pl>

Maciek Fijalkowski wrote:
> I know you're quite tired right now and all, but I think it would be 
> good to make some kind of pypy-sync next week or even the week after (we 
> need to decide on sprint topics anyway).
>
> Particularly I would like to know people's opinion on pypy's scope. What 
> we would like to support, what we would like to drop support for, what 
> are the short-term project directions etc.
>
> What do you think?
>
> Cheers,
> fijal
>   
Ok, so what about usual Thursday 17:00?


From anto.cuni at gmail.com  Mon Jun 11 10:18:58 2007
From: anto.cuni at gmail.com (Antonio Cuni)
Date: Mon, 11 Jun 2007 10:18:58 +0200
Subject: [pypy-dev] PyPy sync next week?
In-Reply-To: <466C46A2.9040108@genesilico.pl>
References: <4666E7A4.9030407@genesilico.pl> <466C46A2.9040108@genesilico.pl>
Message-ID: <466D0572.8050205@gmail.com>

Maciek Fijalkowski wrote:

> Ok, so what about usual Thursday 17:00?

Fine for me.

ciao Anto


From lac at openend.se  Mon Jun 11 10:26:12 2007
From: lac at openend.se (Laura Creighton)
Date: Mon, 11 Jun 2007 10:26:12 +0200
Subject: [pypy-dev] PyPy sync next week?
In-Reply-To: Message from Antonio Cuni <anto.cuni@gmail.com> 
	of "Mon, 11 Jun 2007 10:18:58 +0200." <466D0572.8050205@gmail.com> 
References: <4666E7A4.9030407@genesilico.pl> <466C46A2.9040108@genesilico.pl>
	<466D0572.8050205@gmail.com> 
Message-ID: <200706110826.l5B8QCu6000900@theraft.openend.se>

In a message of Mon, 11 Jun 2007 10:18:58 +0200, Antonio Cuni writes:
>Maciek Fijalkowski wrote:
>
>> Ok, so what about usual Thursday 17:00?
>
>Fine for me.
>
>ciao Anto

me too.

Laura



From arigo at tunes.org  Mon Jun 11 12:47:26 2007
From: arigo at tunes.org (Armin Rigo)
Date: Mon, 11 Jun 2007 12:47:26 +0200
Subject: [pypy-dev] PyPy sync next week?
In-Reply-To: <466C46A2.9040108@genesilico.pl>
References: <4666E7A4.9030407@genesilico.pl> <466C46A2.9040108@genesilico.pl>
Message-ID: <20070611104726.GA28732@code0.codespeak.net>

Hi Maciek,

On Sun, Jun 10, 2007 at 08:44:50PM +0200, Maciek Fijalkowski wrote:
> Ok, so what about usual Thursday 17:00?

Ok.


Armin


From elmo13 at jippii.fi  Wed Jun 13 14:55:54 2007
From: elmo13 at jippii.fi (=?UTF-8?B?RWxtbyBNw6RudHluZW4=?=)
Date: Wed, 13 Jun 2007 15:55:54 +0300
Subject: [pypy-dev] About adding a blist implementation...
Message-ID: <466FE95A.6020805@jippii.fi>

Fijal mentioned blist on irc and I got interested and read about it.
This really seems the kind of thing I might even accomplish in a
reasonable time, so if no one hasn't started on it, I'd like to try. I'm
somewhat familiar with the multilist interface (seems very clear), so
the challenge for me would be the [efficient] algorithmic and rpythonic
coding style, the algorithm itself, and teaching myself all of that.

It is very much possible that I will again abandon this as I have done
with some other "projects", but I'd like to finally do something
with/for pypy.

If I get something usedul done, I'll report back and we'll think about
adding it to the official repository...

Elmo


From arigo at tunes.org  Thu Jun 14 18:45:54 2007
From: arigo at tunes.org (Armin Rigo)
Date: Thu, 14 Jun 2007 18:45:54 +0200
Subject: [pypy-dev] Work plan for PyPy
Message-ID: <20070614164554.GA654@code0.codespeak.net>

Hi all,

Here is a summary of what we discussed today on #pypy-sync.  We tried to
list the tasks that now lie ahead of us.  There are a number of
relatively independent tasks, but there is also a knot of dependent
tasks that we'll have to untie bit by bit.

RCtypes and the extension compiler are somewhat of a blocker for other
tasks.  A controversial solution is to drop them.  This means:

* To support calling external libraries, we would instead shift the
  focus on supporting the full ctypes at app-level.  This means that we
  could support out of the box the same ctypes-based pure Python modules
  as CPython.  There are issues here (e.g. is the regular ctypes really
  suitable to wrap large libraries?) but they are not worse than with
  rctypes.
  
* Of course, we keep the general support for compiling stand-alone
  RPython programs, even if rctypes dies.

* In RPython, fijal worked on a more basic replacement for rctypes,
  called rtti (almost finished already).  It's a PyPy-specific interface
  to call external functions in RPython.  Calls written in this
  interface can be compiled to good C code, but testing is a bit less
  convenient than with rctypes because rtti calls cannot be directly
  executed on top of CPython.

* The CPy Object Space is going away initially too because it is based
  on rctypes.  It should not be too hard to port it to rtti in order to
  continue supporting compiling extension modules for CPython.  We need
  to think a bit more here - how can we compile ext modules for Jython
  and IronPython? - but it's likely not a high priority at the moment.

Any comments or objections?  Are there people that already make heavy
use of rctypes, or the extension compiler?


The graph of dependent tasks ("A --- B"  means that B depends on A):


  merge rtti branch ---- app-level ctypes ---- kill rctypes and extcompiler
                    \                        /   \
                     \-- port our modules ---    kill keepalives
                      \                            \
                       -- pyrepl                    -- moving GCs
                                                   /
                   _______________________________/
                  /
                 /
  malloc cleanup ----- gc transform polishing
                     /           \
  fix weakrefs -----/             \
                                   \
                                    -- c/llvm backend sharing


Additionally, some of the independent tasks mentioned:

* move some app-level parts of the Python interpreter to interp-level
* complete the support for Python 2.5
* switch to LLVM 2.0
* port the stackless transform to ootypesystem
* JIT work


A bientot,

Armin.


From fijal at genesilico.pl  Thu Jun 14 19:28:30 2007
From: fijal at genesilico.pl (Maciek Fijalkowski)
Date: Thu, 14 Jun 2007 19:28:30 +0200
Subject: [pypy-dev] Work plan for PyPy
In-Reply-To: <20070614164554.GA654@code0.codespeak.net>
References: <20070614164554.GA654@code0.codespeak.net>
Message-ID: <46717ABE.8010708@genesilico.pl>

Armin Rigo wrote:
> * In RPython, fijal worked on a more basic replacement for rctypes,
>   called rtti (almost finished already).  It's a PyPy-specific interface
>   to call external functions in RPython.  Calls written in this
>   interface can be compiled to good C code, but testing is a bit less
>   convenient than with rctypes because rtti calls cannot be directly
>   executed on top of CPython.
>
>   
This is actually rffi, not rtti (like Rpython Foreign Function 
Interface). There are probably few other, unmentioned issues:

* What we do with ootypesystem backends and external functions? Right 
now this is implemented by backends which tends to be a bit ugly 
implementation. My idea would be to have backend-sensitive 
implementations which access backend-specific RPython functions for 
accessing underlaying platform classes/functions/whatever.

* Platform specifics - how much do we care about portability of a 
generated C code? I've got several approaches in mind:

 - Have special kind of if which renders to #ifdef in the C code (hard)

 - There is a caching mechanism for basic types (size_t, int, etc.) 
which renders sizes of types per-platform. This works for multiple 
platforms, but renders generated C code non-portable. unsure if this is 
a problem.

Cheers,
fijal



From anto.cuni at gmail.com  Fri Jun 15 01:01:17 2007
From: anto.cuni at gmail.com (Antonio Cuni)
Date: Fri, 15 Jun 2007 01:01:17 +0200
Subject: [pypy-dev] Work plan for PyPy
In-Reply-To: <20070614164554.GA654@code0.codespeak.net>
References: <20070614164554.GA654@code0.codespeak.net>
Message-ID: <4671C8BD.4050707@gmail.com>

Armin Rigo wrote:
> Hi all,

Hi,
sorry, I completely forgot the meeting :-(.
I was even at home and online at that time, but I missed it. At least 
I've read the logs :-).
Here are some notes from my side.

My personal short-term plan is to work a bit to make gencli more or less 
usable as a general .NET compiler, in particular for generating DLLs to 
be imported by others .NET programs.

I personally would like not to work on rpython too much, but I have to 
convince my professor first :-).
After having convinced him, I would like to work on the JIT, in 
particular to port it to ootype (and then add gencli support).


> * The CPy Object Space is going away initially too because it is based
>   on rctypes.  It should not be too hard to port it to rtti in order to
>   continue supporting compiling extension modules for CPython.  We need
>   to think a bit more here - how can we compile ext modules for Jython
>   and IronPython? - but it's likely not a high priority at the moment.

Armin, I read that you asked how hard would be to generate extension 
modules for Jython; the same question apply to IronPython.
I think that the easiest way in the short term is to generate plain 
Java/.NET modules that can be imported by any Java/C# program, and rely 
on Jython/IronPython's mechanism to import it (thus, not relying on the 
CPy objectspace at all).

I've already started in this direction, regarding gencli: the name of 
the front-end is SilveRPython, though I think I will rename it to avoid 
confusion with microsoft silverlight. Currently it mostly works, but I 
have to convince the rtyper not to mangle the name of the classes and 
the methods, because they need to be publicly available from the outside.


> Additionally, some of the independent tasks mentioned:
> 
> * port the stackless transform to ootypesystem

I also would have spotted this point if I were at the meeting :-).
I think it could be a nice topic for sprint, couldn't it?


Finally, a note which is not related with the work plan but to a 
question that appeared in the logs; someone asked whether we want to 
"sell" RPython as a stand-alone product.
Last sunday I gave a PyPy talk at Pycon italy and, surprisingly enough, 
RPython was perceived as the only (or the most relevant) usable product 
of the pypy project. I tried hardly to explain that rpython is basically 
only an accident and not very usable, but people are still impressed by 
the 300000% speedup.
It would be interesting to know if this perception is world-wide or only 
limited to the attendants of my talk (maybe because I didn't stress 
enough the pros of the other pypy goals or the cons of rpython).

ciao Anto


From anto.cuni at gmail.com  Fri Jun 15 01:06:24 2007
From: anto.cuni at gmail.com (Antonio Cuni)
Date: Fri, 15 Jun 2007 01:06:24 +0200
Subject: [pypy-dev] Work plan for PyPy
In-Reply-To: <46717ABE.8010708@genesilico.pl>
References: <20070614164554.GA654@code0.codespeak.net>
	<46717ABE.8010708@genesilico.pl>
Message-ID: <4671C9F0.5030002@gmail.com>

Maciek Fijalkowski wrote:

> * What we do with ootypesystem backends and external functions? Right 
> now this is implemented by backends which tends to be a bit ugly 
> implementation. My idea would be to have backend-sensitive 
> implementations which access backend-specific RPython functions for 
> accessing underlaying platform classes/functions/whatever.

I agree. Supporting external functions directly in the backend is 
easier, but now that gencli can call .NET code we don't need those 
functions to be external anymore, we can just provide a real 
implementation for them.

The problem is that at the moment genjvm can't call java functions, so 
this solution would not work for it.

ciao Anto


From janzert at janzert.com  Fri Jun 15 02:28:02 2007
From: janzert at janzert.com (Janzert)
Date: Thu, 14 Jun 2007 20:28:02 -0400
Subject: [pypy-dev] Work plan for PyPy
In-Reply-To: <4671C8BD.4050707@gmail.com>
References: <20070614164554.GA654@code0.codespeak.net>
	<4671C8BD.4050707@gmail.com>
Message-ID: <f4smei$7co$1@sea.gmane.org>

Antonio Cuni wrote:
[snip]
> I tried hardly to explain that rpython is basically 
> only an accident and not very usable, but people are still impressed by 
> the 300000% speedup.
> It would be interesting to know if this perception is world-wide or only 
> limited to the attendants of my talk (maybe because I didn't stress 
> enough the pros of the other pypy goals or the cons of rpython).
> 
> ciao Anto

I certainly don't know if it's universal. I do know that the extcompiler 
with a time sensitive module I ported to rpython is the only real 
practical use for pypy that I have had. I don't have the interest or 
time to offer to maintain the extcompiler (only used it for a personal 
hobby project which is on the back burner for the moment anyway) and 
after getting longs to work in the cpy object space I do appreciate the 
pain it may be to maintain.

Janzert

P.S. There is a certain irony that ShedSkin is growing a rudimentary 
extension compiler at the moment. :}



From lac at openend.se  Fri Jun 15 07:53:02 2007
From: lac at openend.se (Laura Creighton)
Date: Fri, 15 Jun 2007 07:53:02 +0200
Subject: [pypy-dev] Work plan for PyPy
In-Reply-To: Message from Antonio Cuni <anto.cuni@gmail.com> 
	of "Fri, 15 Jun 2007 01:01:17 +0200." <4671C8BD.4050707@gmail.com> 
References: <20070614164554.GA654@code0.codespeak.net>
	<4671C8BD.4050707@gmail.com> 
Message-ID: <200706150553.l5F5r26d031897@theraft.openend.se>

In a message of Fri, 15 Jun 2007 01:01:17 +0200, Antonio Cuni writes:
<snip>
>
>Finally, a note which is not related with the work plan but to a 
>question that appeared in the logs; someone asked whether we want to 
>"sell" RPython as a stand-alone product.
>Last sunday I gave a PyPy talk at Pycon italy and, surprisingly enough, 
>RPython was perceived as the only (or the most relevant) usable product 
>of the pypy project. I tried hardly to explain that rpython is basically 
>only an accident and not very usable, but people are still impressed by 
>the 300000% speedup.
>It would be interesting to know if this perception is world-wide or only 
>limited to the attendants of my talk (maybe because I didn't stress 
>enough the pros of the other pypy goals or the cons of rpython).
>
>ciao Anto

I don't think it is you.  I think that most people are not very
sophisticated in their appreciation for computer langauges.
Culturally deprived, poor things. :-) Maybe we need to run some 'pypy
appreciation courses'. :-) I suspect the designers of the Ferrari have
the same problem.  They probably want to discuss engine-design and
Italian styling with their would-be customers.

Laura


From holger at merlinux.de  Fri Jun 15 08:30:17 2007
From: holger at merlinux.de (holger krekel)
Date: Fri, 15 Jun 2007 08:30:17 +0200
Subject: [pypy-dev] Work plan for PyPy
In-Reply-To: <4671C8BD.4050707@gmail.com>
References: <20070614164554.GA654@code0.codespeak.net>
	<4671C8BD.4050707@gmail.com>
Message-ID: <20070615063017.GK3311@solar.trillke>

Hi Anto, all, 

On Fri, Jun 15, 2007 at 01:01 +0200, Antonio Cuni wrote:
> Finally, a note which is not related with the work plan but to a 
> question that appeared in the logs; someone asked whether we want to 
> "sell" RPython as a stand-alone product.
> Last sunday I gave a PyPy talk at Pycon italy and, surprisingly enough, 
> RPython was perceived as the only (or the most relevant) usable product 
> of the pypy project. I tried hardly to explain that rpython is basically 
> only an accident and not very usable, but people are still impressed by 
> the 300000% speedup.
> It would be interesting to know if this perception is world-wide or only 
> limited to the attendants of my talk (maybe because I didn't stress 
> enough the pros of the other pypy goals or the cons of rpython).

FYI: Last week i gave a talk at the german Zope annual meeting
(extradoc/talk/dzug2007/dzug2007.txt).  To briefly summarize: 
I talked about the .NET backend, python interpreter 
prototypes/speed/status and the general usability of PyPy and its JIT 
for implementing other languages.  I also noted that the extension module 
area is currently in flux.  Questions at the talk then were mostly about the 
Python Interpreter, e.g. how to use persistence/Transparent 
proxy features e.g. with ZODB.  Jim Fulton also noted that zope only
needs few extension modules and often has pure python versions of its 
C-coded modules.  There wasn't a strong focus on RPython or the
ext-compiler but there was e.g. interest (prior to the talk) to port 
the Zope TAL interpreter to RPython in order to use it with CPython. 

Here is a thought: if our Python Interpreter becomes more usable
and can be used as a replacement for CPython in more cases, people *can*
write RPython modules for speed reasons if they want to.  Practically, 
i guess, it'd then be important to untangle the compilation of such 
written-for-speed extension modules from the translation of the 
Python Interpreter itself.  (this all apart from the question if 
a better JIT could obsolete the aim for writing RPython modules). 

best, 

holger


From fijal at genesilico.pl  Fri Jun 15 11:11:27 2007
From: fijal at genesilico.pl (Maciek Fijalkowski)
Date: Fri, 15 Jun 2007 11:11:27 +0200
Subject: [pypy-dev] Work plan for PyPy
In-Reply-To: <20070615063017.GK3311@solar.trillke>
References: <20070614164554.GA654@code0.codespeak.net>	<4671C8BD.4050707@gmail.com>
	<20070615063017.GK3311@solar.trillke>
Message-ID: <467257BF.5060709@genesilico.pl>

I would be very, very careful about what people talk about.

Even if they talk about RPython and speed, they really don't know what 
they're talking about. There are some reasons why they use python and 
not ocaml or haskell for their purposes. One of the reasons is that it's 
easy to write well-working python program. With rpython it's not that 
easy, and we don't explain it in talks, how hard it is. Even giving 
enough effort to the translation toolchain error reporting, type 
inference problems are always hard to read and debug. That's why I think 
RPython will never become a language of choice of most of the people 
(despite few guys who know what they're doing and why).

Also I (personally) think that good enough JIT would be a way better 
place to start writing speedy programs. You write it jit-friendly, you 
get speedup.

 From my POV maintaining all of parts required to use RPython as a 
general purpose language is a bit overkill, and people are really 
interested in parts which they yet don't know they're interested in.

Cheers,
fijal



From holger at merlinux.de  Fri Jun 15 11:19:26 2007
From: holger at merlinux.de (holger krekel)
Date: Fri, 15 Jun 2007 11:19:26 +0200
Subject: [pypy-dev] Work plan for PyPy
In-Reply-To: <467257BF.5060709@genesilico.pl>
References: <20070614164554.GA654@code0.codespeak.net>
	<4671C8BD.4050707@gmail.com> <20070615063017.GK3311@solar.trillke>
	<467257BF.5060709@genesilico.pl>
Message-ID: <20070615091926.GO3311@solar.trillke>

On Fri, Jun 15, 2007 at 11:11 +0200, Maciek Fijalkowski wrote:
> I would be very, very careful about what people talk about.

quick question: your mail appears in my Mutt as a reply to my
previous mail ("Hi Anto, all") but your mail is not meant 
as relating to it specifically, right? 

holger


From fijal at genesilico.pl  Fri Jun 15 11:24:24 2007
From: fijal at genesilico.pl (Maciek Fijalkowski)
Date: Fri, 15 Jun 2007 11:24:24 +0200
Subject: [pypy-dev] Work plan for PyPy
In-Reply-To: <20070615091926.GO3311@solar.trillke>
References: <20070614164554.GA654@code0.codespeak.net>
	<4671C8BD.4050707@gmail.com> <20070615063017.GK3311@solar.trillke>
	<467257BF.5060709@genesilico.pl>
	<20070615091926.GO3311@solar.trillke>
Message-ID: <46725AC8.9010407@genesilico.pl>

holger krekel wrote:
> On Fri, Jun 15, 2007 at 11:11 +0200, Maciek Fijalkowski wrote:
>   
>> I would be very, very careful about what people talk about.
>>     
>
> quick question: your mail appears in my Mutt as a reply to my
> previous mail ("Hi Anto, all") but your mail is not meant 
> as relating to it specifically, right? 
>
> holger
>
> :.
>
>   
Yes. Just being too lazy to answer to a proper mail.


From arigo at tunes.org  Fri Jun 15 11:55:56 2007
From: arigo at tunes.org (Armin Rigo)
Date: Fri, 15 Jun 2007 11:55:56 +0200
Subject: [pypy-dev] Pygame viewer
Message-ID: <20070615095556.GA16579@code0.codespeak.net>

Hi all,

I've refactored a bit the Pygame viewer.  Can people with OS/X or
Windows give it a try?  It is currently in the branch
pypy/branch/graphserver-dist/.  I'll merge it when I'm sure there are no
platform issues.  It is supposed to support the following features:

* all pygame usage is done in a subprocess, which cannot take away the
  whole parent process if it segfaults.

* regular usage should continue to work as before, e.g. t.view() in
  translatorshell.py, as well as the translate.py extended pdb+ commands
  like "classhier" to show the class hierarchy.

* closing the Pygame window now really closes it and shuts down the
  subprocess, instead of the previous hack that left the title bar of
  the window around just because it's not always possible to re-open
  Pygame in the same process.

* the tool should be really stand-alone now: if you check out only
  http://codespeak.net/svn/pypy/branch/graphserver-dist/dotviewer then
  the dotviewer.py included there should have no pypy or py lib
  dependency.  It should also contact the CGI script on codespeak
  automatically, to compile graphs if 'dot' is not installed locally.
  It won't install Pygame for you nor brew your coffee, though.

* I've kept the best for the end: *all* regular usage (t.view() and
  anything display graphs from translate.py) works remotely too.  Here's
  an example setup (pick your own port number instead of 8515):

  - on your local machine, run: 
        pypy/bin/dotviewer.py --server 127.0.0.1:8515

  - connect to a remote machine, allowing the remote side to contact
    your local dotviewer:
        ssh -R8515:localhost:8515 machinename

  - set the $GRAPHSERVER env var on the remote machine to point to the
    redirected port: export GRAPHSERVER=:8515

  - then any graph-viewing command you execute while $GRAPHSERVER is set
    will reach your local dotviewer and basically appear to work
    completely transparently.

  - to automate steps 2 and 3, I've put the following entry in my local
    .ssh/config:
                    Host wyvern
                        HostName wyvern.cs.uni-duesseldorf.de
                        Port 922
                        RemoteForward 8515 127.0.0.1:8515

    (hence, please pick another port number, because if two people log
    in trying to redirect the port there is a conflict :-)

    and I've added the "export GRAPHSERVER=:8515" line to my .bashrc on
    wyvern.


A bientot,

Armin.


From faassen at startifact.com  Fri Jun 15 12:41:06 2007
From: faassen at startifact.com (Martijn Faassen)
Date: Fri, 15 Jun 2007 12:41:06 +0200
Subject: [pypy-dev] PyPy and production use
In-Reply-To: <4671C8BD.4050707@gmail.com>
References: <20070614164554.GA654@code0.codespeak.net>
	<4671C8BD.4050707@gmail.com>
Message-ID: <f4tqc4$7ct$1@sea.gmane.org>

Hey,

I'm changing the title of this thread as this is another topic. It's one 
I care about as an observer of PyPy.

Antonio Cuni wrote:
[snip]
> Finally, a note which is not related with the work plan but to a 
> question that appeared in the logs; someone asked whether we want to 
> "sell" RPython as a stand-alone product.

I think you should. I'll explain why below.

> Last sunday I gave a PyPy talk at Pycon italy and, surprisingly enough, 
> RPython was perceived as the only (or the most relevant) usable product 
> of the pypy project. I tried hardly to explain that rpython is basically 
> only an accident and not very usable, but people are still impressed by 
> the 300000% speedup.
> It would be interesting to know if this perception is world-wide or only 
> limited to the attendants of my talk (maybe because I didn't stress 
> enough the pros of the other pypy goals or the cons of rpython).

I got this impression back at the presentation last year at EuroPython. 
I therefore think this impression is more widespread than just your 
audience in Italy. This is not because you've communicated the state or 
future potential of PyPy wrongly, but because the audience likely has a 
different perspective than you do.

I realize that PyPy is much more than RPython and that RPython is 
somewhat incidental to the project. For people working on production 
projects right now, these other features are less relevant at this point 
in time, however. Let's take PyPy's generated interpreter. If I'm 
currently using the CPython platform (as most Python programmers are), 
choosing the PyPy-C interpreter would mean that I have far less 
extension module support, and a slightly slower interpreter. I'd need 
some pretty unusual use cases currently for the benefits of PyPy-C to 
outweigh its drawbacks.

The PyPy developers themselves of course *do* have unusual use cases. 
They want to experiment with interpreter generation, different language 
features, different platforms, and so on.

I expect this to change over time. Once the PyPy (generated) JIT 
matures, PyPy will become a lot more attractive. If you could then 
somehow figure out a way to use CPython modules with PyPy's interpreter 
you'd get even more people. Right now, unless I really really need some 
unusual language feature in my production project, I don't have a 
motivation to use PyPy's interpreter.

Maturing the PyPy interpreter will take quite a lot of time. You've come 
a long way and it's quite impressive, but at the same time I don't 
expect to be using a PyPy-generated interpreter to develop web 
applications any time soon.

Now RPython. RPython generated modules are usable from CPython. This 
means I can write relatively plain Python code, generate a module, and 
start using this module in my web application today. The benefit is 
vastly increased speed for my module.

Using RPython with CPython is attractive for the same reasons Pyrex is 
attractive, with the benefits that it's more well-engineered than Pyrex. 
Pyrex is currently more mature to use. Using RPython in this way is a 
bit of a struggle still in comparison. This is however tool support that 
I feel can improve relatively quickly.

An open source projects needs mindshare. It needs users. PyPy is an open 
source project, and I'd recommend getting any users you can get. 
Granted, you'll get opportunists like me who are only there for 
production benefits. These people are less interested in language 
experimentation and new features - they're there for the speed 
primarily: the main thing that PyPy, through RPython, can offer to a 
Python programmer today. But if your aim is to provide a platform that 
is suitable for production use, you *need* opportunists like that. They 
can help mature the platform. Open source projects thrive by attracting 
developers that are in it for their own (enlightened) self-interest.

Some of these opportunists might actually help out beyond just RPython. 
They might in fact be interested in such language experimentation. It 
might however not be a priority for them. For instance, I'm interested, 
but I need an excuse to spend more time on PyPy. Production benefits are 
a great excuse. Using RPython extensions with CPython is the shortest 
route to production benefits.

Besides gaining new users, supporting RPython as a product will help 
PyPy in other ways as well. It will get you more module support. New 
users will come in and help you write more extension modules in RPython. 
They'll be doing this initially so they can use them in CPython, but 
it's a nice side effect.

Currently I'm working with Godefroid Chapelle (well, he's been doing the 
most work so far) to try to make an RPython version of a web templating 
engine (Zope's page template engine). We hope we will be able to use 
this technology to speed up the generation of web templates. We'll be 
using this within a CPython server.

We see the potential of PyPy. Godefroid made the observation that a web 
templating engine implements an interpreter. PyPy has technology to 
generate a JIT for arbitrary interpreters in RPython. This means that in 
the future we might be able to have a JIT-ed templating language, 
hopefully boosting performance even more. (besides the huge coolness factor)

Just the potential for future JITting isn't motivating enough to 
experiment with PyPy right now though. The ability to use RPython with 
CPython is the critical bit. This is what gives us the hope we'll get a 
payoff in our production work. The JIT is a nice bonus that might become 
important later.

So if people conclude that RPython is currently the most relevant 
product of the PyPy project for them, it doesn't mean that they haven't 
understood the potential of PyPy in the future. It's because they're 
thinking about how PyPy is relevant to their own work today.

PyPy has always been a unique project: it's a research project and it's 
an open source project aiming for real-world use. It takes ideas from 
ivory towers and applies them to a pragmatic, real-world language like 
Python. I think this balance between pragmatic and research goals is one 
of this project's greatest strengths. I encourage PyPy developers to 
continue to embrace both goals.

Regards,

Martijn



From faassen at startifact.com  Fri Jun 15 12:45:19 2007
From: faassen at startifact.com (Martijn Faassen)
Date: Fri, 15 Jun 2007 12:45:19 +0200
Subject: [pypy-dev] Work plan for PyPy
In-Reply-To: <200706150553.l5F5r26d031897@theraft.openend.se>
References: <20070614164554.GA654@code0.codespeak.net>	<4671C8BD.4050707@gmail.com>
	<200706150553.l5F5r26d031897@theraft.openend.se>
Message-ID: <f4tqk1$87c$1@sea.gmane.org>

Laura Creighton wrote:
[snip]
> I don't think it is you.  I think that most people are not very
> sophisticated in their appreciation for computer langauges.
> Culturally deprived, poor things. :-) Maybe we need to run some 'pypy
> appreciation courses'. :-) I suspect the designers of the Ferrari have
> the same problem.  They probably want to discuss engine-design and
> Italian styling with their would-be customers.

I know you're joking here. Still, I'll reiterate the point I tried to 
make in my longer mail just now: people might in fact be quite cultured 
in their appreciation for computer language technology and still 
approach you about RPython. They just have different goals.

Don't underestimate your audience. People might appreciate Ferraris but 
still not want to drive one to work every day.

Regards,

Martijn



From faassen at startifact.com  Fri Jun 15 13:04:32 2007
From: faassen at startifact.com (Martijn Faassen)
Date: Fri, 15 Jun 2007 13:04:32 +0200
Subject: [pypy-dev] Work plan for PyPy
In-Reply-To: <467257BF.5060709@genesilico.pl>
References: <20070614164554.GA654@code0.codespeak.net>	<4671C8BD.4050707@gmail.com>	<20070615063017.GK3311@solar.trillke>
	<467257BF.5060709@genesilico.pl>
Message-ID: <f4tro2$c46$1@sea.gmane.org>

Maciek Fijalkowski wrote:
> I would be very, very careful about what people talk about.
> 
> Even if they talk about RPython and speed, they really don't know what 
> they're talking about. 

I understand that you need to be careful about not overselling RPython.
I would also suggest you be careful about underestimating your audience. 
You just implied I don't know what I'm talking about because I'm 
interested in using RPython for speed. While my perspective on PyPy is 
definitely quite different than yours, I do think I have a reasonable 
picture of what it is about.

[snip]
> Also I (personally) think that good enough JIT would be a way better 
> place to start writing speedy programs. You write it jit-friendly, you 
> get speedup.

People will then rightfully ask you the question when you think they can 
benefit from a JITed interpreter in their Zope/Django/Pylons projects. 
RPython, immature as the toolchain might be, is at least somewhat usable 
today.

You could tell me that I'd be more productive if I contributed to the 
JIT generator, but then I'd go away again and you'd lose a potential 
contributor. If I can speed up my templating language using RPython I 
might stick around. I realize that I personally am of small potential 
value to the project, but who knows who else you might draw in this way?

 >  From my POV maintaining all of parts required to use RPython as a
 > general purpose language is a bit overkill, and people are really
 > interested in parts which they yet don't know they're interested in.

Two points:

* in an open source project, others might be helping you maintain this 
toolchain, so the cost might be relatively little to you.

* you will likely still have some maintenance cost. This could be an 
investment: maintaining something not part of your core goal may draw in 
sufficient new contributors to actually benefit the core goals as well.

I realize I'm speaking from a quite different cultural perspective than 
many PyPy developers. I also realize I'm arguing from a self-interested 
perspective. I also genuinely believe that taking these other 
perspectives into account may help your project. So please cut me some 
slack here. :)

I don't think you have to be worried on the short term about being 
flooded with clueless people taking up your time. Even if that happened, 
I'd consider it a luxury problem, as some percentage of new people would 
not be clueless and valuable to your project.

Regards,

Martijn



From lypanov at mac.com  Fri Jun 15 13:16:06 2007
From: lypanov at mac.com (Alexander Kellett)
Date: Fri, 15 Jun 2007 13:16:06 +0200
Subject: [pypy-dev] Work plan for PyPy
In-Reply-To: <f4tro2$c46$1@sea.gmane.org>
References: <20070614164554.GA654@code0.codespeak.net>
	<4671C8BD.4050707@gmail.com> <20070615063017.GK3311@solar.trillke>
	<467257BF.5060709@genesilico.pl> <f4tro2$c46$1@sea.gmane.org>
Message-ID: <1BB835BB-691C-480A-A941-86454586AB4A@mac.com>

On 15 Jun 2007, at 13:04, Martijn Faassen wrote:
> You could tell me that I'd be more productive if I contributed to the
> JIT generator, but then I'd go away again and you'd lose a potential
> contributor. If I can speed up my templating language using RPython I
> might stick around. I realize that I personally am of small potential
> value to the project, but who knows who else you might draw in this  
> way?

It seems like many are in this boat.

Are there really such difficult stumbling blocks involved in making a  
(r)python lint
and improving the current error messages while at the same time  
letting users of the
system build a FAQ on the more common issues based in their experience?

If not, why not branch off now?

Alex


---
Alexander Kellett
PGP - 0x6BFA8EF3, FPR: DA65 D6DE 56A9 D715 EFB6 A948 B2EF 6622 6BFA 8EF3




From fijal at genesilico.pl  Fri Jun 15 14:07:33 2007
From: fijal at genesilico.pl (Maciek Fijalkowski)
Date: Fri, 15 Jun 2007 14:07:33 +0200
Subject: [pypy-dev] Work plan for PyPy
In-Reply-To: <f4tro2$c46$1@sea.gmane.org>
References: <20070614164554.GA654@code0.codespeak.net>	<4671C8BD.4050707@gmail.com>	<20070615063017.GK3311@solar.trillke>	<467257BF.5060709@genesilico.pl>
	<f4tro2$c46$1@sea.gmane.org>
Message-ID: <46728105.1060607@genesilico.pl>

Martijn Faassen wrote:
> Maciek Fijalkowski wrote:
>   
>> I would be very, very careful about what people talk about.
>>
>> Even if they talk about RPython and speed, they really don't know what 
>> they're talking about. 
>>     
>
> I understand that you need to be careful about not overselling RPython.
> I would also suggest you be careful about underestimating your audience. 
> You just implied I don't know what I'm talking about because I'm 
> interested in using RPython for speed. While my perspective on PyPy is 
> definitely quite different than yours, I do think I have a reasonable 
> picture of what it is about.
>   
Sorry for underestimating you :) If I didn't add a disclaimer, I should 
at least - I meant *most* of the people are not that interested in 
speed, I kind of believe that you are the one who might be.
> [snip]
>   
>> Also I (personally) think that good enough JIT would be a way better 
>> place to start writing speedy programs. You write it jit-friendly, you 
>> get speedup.
>>     
>
> People will then rightfully ask you the question when you think they can 
> benefit from a JITed interpreter in their Zope/Django/Pylons projects. 
> RPython, immature as the toolchain might be, is at least somewhat usable 
> today.
>   
the real problem is that RPython is quite mature for building 
stand-alone applications, while it's usefulness as a CPython extension 
generator is somehow debatable. There are few steps needed for that, 
like improving our reference counting GC implementation, moving from 
rctypes to new interface and so on and so on. This would greatly improve 
performance and usability. These are not small and uninvolved tasks - 
it's very much in the core. I think we would welcome someone who will do 
that, but seems unlikely, as first someone from the core team would need 
to spend a lot of effort on that. It's not question about whether it's 
worth that - I think it is. The open question is who is willing to spend 
time on this - if there is someone, good, I'll not try to stop him.
> You could tell me that I'd be more productive if I contributed to the 
> JIT generator, but then I'd go away again and you'd lose a potential 
> contributor. If I can speed up my templating language using RPython I 
> might stick around. I realize that I personally am of small potential 
> value to the project, but who knows who else you might draw in this way?
>   
I'm not saying this. I wouldn't be productive contributing to JIT.
>
> Two points:
>
> * in an open source project, others might be helping you maintain this 
> toolchain, so the cost might be relatively little to you.
>
> * you will likely still have some maintenance cost. This could be an 
> investment: maintaining something not part of your core goal may draw in 
> sufficient new contributors to actually benefit the core goals as well.
>
> I realize I'm speaking from a quite different cultural perspective than 
> many PyPy developers. I also realize I'm arguing from a self-interested 
> perspective. I also genuinely believe that taking these other 
> perspectives into account may help your project. So please cut me some 
> slack here. :)
>   
It's not about cultural differencies I think. It's more about my own 
time and how I would like to use it. I would rather spend my own time on 
making extension modules or making CPython's extension modules work on 
top of PyPy than on extcompiler, which needs some more work.




From simon at arrowtheory.com  Sat Jun 16 02:44:54 2007
From: simon at arrowtheory.com (Simon Burton)
Date: Fri, 15 Jun 2007 17:44:54 -0700
Subject: [pypy-dev] Work plan for PyPy
In-Reply-To: <20070614164554.GA654@code0.codespeak.net>
References: <20070614164554.GA654@code0.codespeak.net>
Message-ID: <20070615174454.6e87fb1a.simon@arrowtheory.com>

On Thu, 14 Jun 2007 18:45:54 +0200
Armin Rigo <arigo at tunes.org> wrote:

> 
> Any comments or objections?  Are there people that already make heavy
> use of rctypes, or the extension compiler?

Indeed. Mainly rctypes.

As well as an ever expanding use of libc, 
we recently have interfaced to libSDL and cairo for some
very funky rpython graphics apps. And we use libpython
so that we can extract stuff from python modules.
Also, we do byte shuffling with rctypes: reading binary data
to/from buffers, etc.

I hope we can do all this with rffi, but there are two really
cool things about rctypes:
1) it runs on cpython (our main app takes 30minutes+ to compile)
2) it has an automatic code generator (ctypes can autogenerate
python wrappers from C header files)


Simon.


From fijal at genesilico.pl  Sat Jun 16 13:03:16 2007
From: fijal at genesilico.pl (Maciek Fijalkowski)
Date: Sat, 16 Jun 2007 13:03:16 +0200
Subject: [pypy-dev] Work plan for PyPy
In-Reply-To: <20070615174454.6e87fb1a.simon@arrowtheory.com>
References: <20070614164554.GA654@code0.codespeak.net>
	<20070615174454.6e87fb1a.simon@arrowtheory.com>
Message-ID: <4673C374.2060806@genesilico.pl>

Simon Burton wrote:
> On Thu, 14 Jun 2007 18:45:54 +0200
> Armin Rigo <arigo at tunes.org> wrote:
>
>   
>> Any comments or objections?  Are there people that already make heavy
>> use of rctypes, or the extension compiler?
>>     
>
> Indeed. Mainly rctypes.
>
> As well as an ever expanding use of libc, 
> we recently have interfaced to libSDL and cairo for some
> very funky rpython graphics apps. And we use libpython
> so that we can extract stuff from python modules.
> Also, we do byte shuffling with rctypes: reading binary data
> to/from buffers, etc.
>
> I hope we can do all this with rffi, but there are two really
> cool things about rctypes:
> 1) it runs on cpython (our main app takes 30minutes+ to compile)
> 2) it has an automatic code generator (ctypes can autogenerate
> python wrappers from C header files)
>   
Very valid points indeed. The 1st point sounds unlikely, also because 
you might use C macros with rffi, which somehow invalidates possible 
uses on top of ctypes. What I do is I test (by compiling) several parts 
(some small code snippets involving ext functions). That make debugging 
a bit harder (reading C code), but pretty straightforward - for example 
memory management is done by hand in this part, which makes you 
responsible of tracking C structures and how long they need to live.

The second point is much more likely to happen and to be better than 
ctypes code generation. (Because rffi is way more Cish than ctypes are).

Also it makes stuff a bit harder when you don't submit your bindings back.

If you plan to come to the post-EP pypy sprint, we can sit a bit on the 
automatic code generator for rffi.

Cheers,
fijal



From tismer at stackless.com  Sat Jun 16 15:26:39 2007
From: tismer at stackless.com (Christian Tismer)
Date: Sat, 16 Jun 2007 15:26:39 +0200
Subject: [pypy-dev] PyPy sync next week?
In-Reply-To: <20070611104726.GA28732@code0.codespeak.net>
References: <4666E7A4.9030407@genesilico.pl> <466C46A2.9040108@genesilico.pl>
	<20070611104726.GA28732@code0.codespeak.net>
Message-ID: <6686E8E7-CDCB-470C-8365-C17B3A33CB48@stackless.com>


On 11.06.2007, at 12:47, Armin Rigo wrote:

> Hi Maciek,
>
> On Sun, Jun 10, 2007 at 08:44:50PM +0200, Maciek Fijalkowski wrote:
>> Ok, so what about usual Thursday 17:00?

Ok here, too.




From arigo at tunes.org  Sat Jun 16 18:46:48 2007
From: arigo at tunes.org (Armin Rigo)
Date: Sat, 16 Jun 2007 18:46:48 +0200
Subject: [pypy-dev] PyPy sync next week?
In-Reply-To: <6686E8E7-CDCB-470C-8365-C17B3A33CB48@stackless.com>
References: <4666E7A4.9030407@genesilico.pl> <466C46A2.9040108@genesilico.pl>
	<20070611104726.GA28732@code0.codespeak.net>
	<6686E8E7-CDCB-470C-8365-C17B3A33CB48@stackless.com>
Message-ID: <20070616164648.GA31262@code0.codespeak.net>

Hi Christian,

On Sat, Jun 16, 2007 at 03:26:39PM +0200, Christian Tismer wrote:
> > On Sun, Jun 10, 2007 at 08:44:50PM +0200, Maciek Fijalkowski wrote:
> >> Ok, so what about usual Thursday 17:00?
> 
> Ok here, too.

Just to make sure: that was about Thursday the 14th.  Answering "Ok" on
Saturday the 16th seems strange :-)


A bientot,

Armin.


From arigo at tunes.org  Sat Jun 16 19:10:54 2007
From: arigo at tunes.org (Armin Rigo)
Date: Sat, 16 Jun 2007 19:10:54 +0200
Subject: [pypy-dev] Work plan for PyPy
In-Reply-To: <20070615174454.6e87fb1a.simon@arrowtheory.com>
References: <20070614164554.GA654@code0.codespeak.net>
	<20070615174454.6e87fb1a.simon@arrowtheory.com>
Message-ID: <20070616171054.GA695@code0.codespeak.net>

Hi Simon,

(This is also a follow-up on concerns raised by Christian on #pypy
today)

On Fri, Jun 15, 2007 at 05:44:54PM -0700, Simon Burton wrote:
> I hope we can do all this with rffi, but there are two really
> cool things about rctypes:
> 1) it runs on cpython (our main app takes 30minutes+ to compile)
> 2) it has an automatic code generator (ctypes can autogenerate
> python wrappers from C header files)

For 1), this would actually be possible with rffi too.  The trick would
be to have two implementations of the lltypesystem objects (array,
structures, etc.).  The existing implementation would only be used when
ctypes is not available; if ctypes can be imported, then the other
implementation could systematically be used.  With a bit of effort it
could have a fully compatible interface, but be internally implemented
as ctypes objects (e.g. lltype.malloc(S) would build a ctypes structure
type corresponding to S, and internally instantiate that).  Then we
could really call rffi functions, by turning it internally into a ctypes
function and calling it with the ctypes objects underlying the passed-in
arguments.

This might even not be completely insane - in my opinion, less than
implementing and maintaining rctypes itself.  It's also something that
can be done by someone not deeply into the code base, as long as he
knows the lltype interface and how to use ctypes.


A bientot,

Armin.


From tismer at stackless.com  Sat Jun 16 22:25:38 2007
From: tismer at stackless.com (Christian Tismer)
Date: Sat, 16 Jun 2007 22:25:38 +0200
Subject: [pypy-dev] PyPy sync next week?
In-Reply-To: <20070616164648.GA31262@code0.codespeak.net>
References: <4666E7A4.9030407@genesilico.pl> <466C46A2.9040108@genesilico.pl>
	<20070611104726.GA28732@code0.codespeak.net>
	<6686E8E7-CDCB-470C-8365-C17B3A33CB48@stackless.com>
	<20070616164648.GA31262@code0.codespeak.net>
Message-ID: <4F3FD51F-F246-49E7-8B98-34ACE9FE2FBC@stackless.com>

> Hi Armin,


> On Sat, Jun 16, 2007 at 03:26:39PM +0200, Christian Tismer wrote:
>>> On Sun, Jun 10, 2007 at 08:44:50PM +0200, Maciek Fijalkowski wrote:
>>>> Ok, so what about usual Thursday 17:00?
>>
>> Ok here, too.
>
> Just to make sure: that was about Thursday the 14th.  Answering  
> "Ok" on
> Saturday the 16th seems strange :-)

Oupps, yes.

I saw this today, and thought it was about next Thursday
as well. Sorry about the confusion.

ciao - chris



From fijal at genesilico.pl  Sat Jun 16 22:37:16 2007
From: fijal at genesilico.pl (Maciek Fijalkowski)
Date: Sat, 16 Jun 2007 22:37:16 +0200
Subject: [pypy-dev] Work plan for PyPy
In-Reply-To: <20070616171054.GA695@code0.codespeak.net>
References: <20070614164554.GA654@code0.codespeak.net>	<20070615174454.6e87fb1a.simon@arrowtheory.com>
	<20070616171054.GA695@code0.codespeak.net>
Message-ID: <467449FC.2040909@genesilico.pl>

Armin Rigo wrote:
> Hi Simon,
>
> (This is also a follow-up on concerns raised by Christian on #pypy
> today)
>
> On Fri, Jun 15, 2007 at 05:44:54PM -0700, Simon Burton wrote:
>   
>> I hope we can do all this with rffi, but there are two really
>> cool things about rctypes:
>> 1) it runs on cpython (our main app takes 30minutes+ to compile)
>> 2) it has an automatic code generator (ctypes can autogenerate
>> python wrappers from C header files)
>>     
>
> For 1), this would actually be possible with rffi too.  The trick would
> be to have two implementations of the lltypesystem objects (array,
> structures, etc.).  The existing implementation would only be used when
> ctypes is not available; if ctypes can be imported, then the other
> implementation could systematically be used.  With a bit of effort it
> could have a fully compatible interface, but be internally implemented
> as ctypes objects (e.g. lltype.malloc(S) would build a ctypes structure
> type corresponding to S, and internally instantiate that).  Then we
> could really call rffi functions, by turning it internally into a ctypes
> function and calling it with the ctypes objects underlying the passed-in
> arguments.
>
> This might even not be completely insane - in my opinion, less than
> implementing and maintaining rctypes itself.  It's also something that
> can be done by someone not deeply into the code base, as long as he
> knows the lltype interface and how to use ctypes.
>
>
> A bientot,
>
> Armin.
>
>   
Just a sidenote - rffi supports (and uses) macros. Not sure how this 
will look like in a ctypes-based solution.

Cheers,
fijal



From tismer at stackless.com  Sat Jun 16 23:04:14 2007
From: tismer at stackless.com (Christian Tismer)
Date: Sat, 16 Jun 2007 23:04:14 +0200
Subject: [pypy-dev] Work plan for PyPy
In-Reply-To: <20070616171054.GA695@code0.codespeak.net>
References: <20070614164554.GA654@code0.codespeak.net>
	<20070615174454.6e87fb1a.simon@arrowtheory.com>
	<20070616171054.GA695@code0.codespeak.net>
Message-ID: <C092E820-2BAC-4604-9A74-2DDB5E051630@stackless.com>

Hi Armin,

On 16.06.2007, at 19:10, Armin Rigo wrote:

> Hi Simon,
>
> (This is also a follow-up on concerns raised by Christian on #pypy
> today)

Well, reacting a bit to it, here...

The reason why this stuff kept me busy thinking since
the meeting was the fact that we are about to trash
exactly the little stuff people are actually using.
We do have very little users, and most of them are
involved right here.

So if we are going to trasg stuff, then we should
go and offer help to those few users, to not getting
distracted from PyPy.
<please assume that I could insert all of Martijn's text, too>

So my thoughts were around these ideas:

- Try to extend CTypes to support primitive-enough concepts
   that do translate to RPython

or

- offer support to our users, to make transition to rffi
   smooth and easy.

I think both ways are possible, I also guess the latter
woulod be supported even if we asked for help or some funding
for this necessary work.

Whatever way we choose, I think it would be best for PyPy
if we try to be as user-friendly as possible, saying like
"""we believe we must change this and that immediately, but
we will not leave you in the rain, but actively help porting
your stuff."""

This also relates a bit to my former thoughts about things
which we have donated to the open source. We can, physically,
just trash anything, of course. But from a point of view
of users and our growing community, this is a matter of belief
and trust that we should be careful about.

I'm not saying that we don't care, these are just my thoughts
and feelings about what I want to be not questionable.

> On Fri, Jun 15, 2007 at 05:44:54PM -0700, Simon Burton wrote:
>> I hope we can do all this with rffi, but there are two really
>> cool things about rctypes:
>> 1) it runs on cpython (our main app takes 30minutes+ to compile)
>> 2) it has an automatic code generator (ctypes can autogenerate
>> python wrappers from C header files)
>
> For 1), this would actually be possible with rffi too.  The trick  
> would
> be to have two implementations of the lltypesystem objects (array,
> structures, etc.).

I think the idea is not bad, but unless I'm missing something,
don't we exactly loose the testing feature by this, by having
to have two different implementations, the benefit of simple
CPython testing for lltypesystem would be gone?

> The existing implementation would only be used when
> ctypes is not available; if ctypes can be imported, then the other
> implementation could systematically be used.  With a bit of effort it
> could have a fully compatible interface, but be internally implemented
> as ctypes objects (e.g. lltype.malloc(S) would build a ctypes  
> structure
> type corresponding to S, and internally instantiate that).  Then we
> could really call rffi functions, by turning it internally into a  
> ctypes
> function and calling it with the ctypes objects underlying the  
> passed-in
> arguments.

Not sure that I immediately get it.
You would use CTypes to emulate rffi functions if we are in Python,
and use them directly when you are in RPython?

How would the memory ownership problem be handled?

Ah, the problem is circumvented by doing all this stuff on
app-level, dropping high efficiency constraints?
No I am confused.

> This might even not be completely insane - in my opinion, less than
> implementing and maintaining rctypes itself.  It's also something that
> can be done by someone not deeply into the code base, as long as he
> knows the lltype interface and how to use ctypes.

By 'it', you mean the CTypes implementation, at app-level, based
upon rffi? I hope I'm not completely lost, here :-)

ciao - chris




From faassen at startifact.com  Mon Jun 18 16:36:10 2007
From: faassen at startifact.com (Martijn Faassen)
Date: Mon, 18 Jun 2007 16:36:10 +0200
Subject: [pypy-dev] Work plan for PyPy
In-Reply-To: <C092E820-2BAC-4604-9A74-2DDB5E051630@stackless.com>
References: <20070614164554.GA654@code0.codespeak.net>	<20070615174454.6e87fb1a.simon@arrowtheory.com>	<20070616171054.GA695@code0.codespeak.net>
	<C092E820-2BAC-4604-9A74-2DDB5E051630@stackless.com>
Message-ID: <f5658p$55m$1@sea.gmane.org>

Hey,

Christian Tismer wrote:
> Whatever way we choose, I think it would be best for PyPy
> if we try to be as user-friendly as possible, saying like
> """we believe we must change this and that immediately, but
> we will not leave you in the rain, but actively help porting
> your stuff."""
> 
> This also relates a bit to my former thoughts about things
> which we have donated to the open source. We can, physically,
> just trash anything, of course. But from a point of view
> of users and our growing community, this is a matter of belief
> and trust that we should be careful about.
> 
> I'm not saying that we don't care, these are just my thoughts
> and feelings about what I want to be not questionable.

Thanks Christian for making these points and having these concerns. I 
think that the project is wise to consider these issues once every while.

I only understand a little of the technical discussion. I understand 
that RCTypes has problems, and that people anticipate having to break it 
and reimplement it in a different way in order to solve these problems. 
I understand that one feature that RCTypes offers, namely the ability to 
test your RPython code on CPython, might also go away.

There are currently, as far as I can see it, two reasons to use RPython 
for CPython extensions:

a) benefit from performance gains by rewriting your CPython code into 
RPython. This is what we want to do with the page template interpreter. 
In case of language implementations, this might lead to the ability to 
use other PyPy technology later (such as JIT generation and so on). Some 
others might purely be interested in performance though and aren't 
implementing a language.

b) better ways to write extensions that use C libraries in CPython. The 
typical Pyrex use case, though it can do the former as well. This would 
use RCTypes.

Use case a) probably has less interested people than use case b). 
Eventually JIT technologies will help here, which will make use case a) 
less important.

I happen to be interested in use case a) though. It is quite exciting to 
me that we could have a faster template language interpreter 
implementation that we can use in CPython today, with the potential 
benefits of PyPy for interpreter translation in the future as well. We 
are basically language implementors, one of your core target audiences. 
We just feel more comfortable doing work if we know we have payoffs in 
the near future, i.e. use in CPython. If we were implementing this 
template language in isolation and had no hope we could deploy it in 
production any time soon, we would be less interested.

I am actually quite interested in use case b) as well. While developing 
lxml, I've done quite a bit of work with Pyrex and I can see how RPython 
and RCTypes might potentially help projects like that. I've been taking 
a wait and see attitude here though, as the technology needed more maturing.

As I understand it, the proposed changes wouldn't affect use case a). Or 
is the ability to generate CPython extensions also going to be broken 
due to this work? Please let us know, as if CPython support is going to 
be broken, we would likely stop our efforts for the time being.

Use case b) would certainly be broken due to this work. What's more, I 
understand that testing your extension code in CPython first wouldn't 
work anymore (at least if it uses CTypes). This is a very nice feature 
of RPython that Pyrex doesn't have, by the way. You mentioned that there 
are potential ways to reimplement this feature again, but I'm not sure 
whether you're committing to such implementation.

If I were writing C extension modules with PyPy I'd like to know things 
like:

* what are the reasons this change is going to be made?

* would this benefit just core PyPy or would it benefit extension module 
writers too?

* when do you expect the new way to be finalized?

* will you make a real commitment to make the "Test in CPython first" 
feature work again? Or is did you just sketched out possible ways to 
implement this in the new system and you hope some contributor is going 
to do the work?

* will you make a commitment to make this new way useful and supported 
for extension module writers, or should they invest their time in some 
other technology? I.e. what are the changes a future shift will make me 
change things again?

* How well documented will this be for CPython extension developers? 
Will you market this feature to developers? Things like this indicate 
your commitment in another way. :)

Please note that I'm not actually *asking* for anything but information. 
I'm not telling the PyPy developers they should make commitments and do 
various kinds of work. That's up to you. I'm just trying to guess what 
other developers might want to know.

Of course, the quality of your communication and the answers you 
actually give will make people decide whether to continue to use PyPy or 
move on. This will affect the way your open source community will grow 
and develop.

I would urge the PyPy developers to consider investing in the open 
source community by supporting features like this and promoting them. 
This would mean serving admittedly self-interested developers with short 
term goals. The promise of immediate payoff is a great way to attract 
people to an open source project. If there is the hope of continuing 
payoff (and PyPy has tremendous potential here), some of those new 
people can be expected to become valuable contributors to the project as 
a whole. And yes, I'm self-interested in saying all of this - that's my 
point.

Regards,

Martijn



From alexandre.fayolle at logilab.fr  Tue Jun 19 09:54:52 2007
From: alexandre.fayolle at logilab.fr (Alexandre Fayolle)
Date: Tue, 19 Jun 2007 09:54:52 +0200
Subject: [pypy-dev] Work plan for PyPy
In-Reply-To: <1BB835BB-691C-480A-A941-86454586AB4A@mac.com>
References: <20070614164554.GA654@code0.codespeak.net>
	<4671C8BD.4050707@gmail.com> <20070615063017.GK3311@solar.trillke>
	<467257BF.5060709@genesilico.pl> <f4tro2$c46$1@sea.gmane.org>
	<1BB835BB-691C-480A-A941-86454586AB4A@mac.com>
Message-ID: <20070619075452.GA8790@crater.logilab.fr>

On Fri, Jun 15, 2007 at 01:16:06PM +0200, Alexander Kellett wrote:
> On 15 Jun 2007, at 13:04, Martijn Faassen wrote:
> > You could tell me that I'd be more productive if I contributed to the
> > JIT generator, but then I'd go away again and you'd lose a potential
> > contributor. If I can speed up my templating language using RPython I
> > might stick around. I realize that I personally am of small potential
> > value to the project, but who knows who else you might draw in this  
> > way?
> 
> It seems like many are in this boat.
> 
> Are there really such difficult stumbling blocks involved in making a  
> (r)python lint

Some work has been done on this, using pylint which has an rpython
checker. Contributed additional checks are welcome. 

http://www.logilab.org/projects/pylint
 

-- 
Alexandre Fayolle                              LOGILAB, Paris (France)
Formations Python, Zope, Plone, Debian:  http://www.logilab.fr/formations
D?veloppement logiciel sur mesure:       http://www.logilab.fr/services
Informatique scientifique:               http://www.logilab.fr/science
Reprise et maintenance de sites CPS:     http://www.migration-cms.com/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 481 bytes
Desc: Digital signature
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20070619/616c8fe7/attachment.pgp>

From elmo13 at jippii.fi  Tue Jun 19 10:18:59 2007
From: elmo13 at jippii.fi (=?ISO-8859-1?Q?Elmo_M=E4ntynen?=)
Date: Tue, 19 Jun 2007 11:18:59 +0300
Subject: [pypy-dev] About adding a blist implementation...
In-Reply-To: <466FE95A.6020805@jippii.fi>
References: <466FE95A.6020805@jippii.fi>
Message-ID: <46779173.3050902@jippii.fi>

Since I'm adding a list implementation, I'd like to know what benchmarks
have you used to test the other ones. Also, if you know apps with heavy
usage of lists and some that use very long lists, I'd be interested very
much.

Elmo


From faassen at startifact.com  Wed Jun 20 12:23:06 2007
From: faassen at startifact.com (Martijn Faassen)
Date: Wed, 20 Jun 2007 12:23:06 +0200
Subject: [pypy-dev] about breaking RPython, RCTypes
Message-ID: <f5av69$jbd$1@sea.gmane.org>

Hi there,

I posted this to a longer thread the other day but have received no 
answers to it whatsoever. Perhaps it got lost, so I'll do a repost. I 
spent a bit of time writing this post, and I ask some questions in it 
that I think should be answered. Perhaps by telling me I don't 
understand what I'm talking about or that my analysis is entirely 
incorrect. Perhaps it was simply too long?

Whatever it is, I do think the PyPy community needs to be aware of how 
technical decisions have an impact on attracting or repelling potential 
contributors. "answering by silence", which happened so far, is one way 
to make potential contributors worried. :)

I only understand a little of the technical discussion. I understand
that RCTypes has problems, and that people anticipate having to break it
and reimplement it in a different way in order to solve these problems.
I understand that one feature that RCTypes offers, namely the ability to
test your RPython code on CPython, might also go away.

There are currently, as far as I can see it, two reasons to use RPython
for CPython extensions:

a) benefit from performance gains by rewriting your CPython code into
RPython. This is what we want to do with the page template interpreter.
In case of language implementations, this might lead to the ability to
use other PyPy technology later (such as JIT generation and so on). Some
others might purely be interested in performance though and aren't
implementing a language.

b) better ways to write extensions that use C libraries in CPython. The
typical Pyrex use case, though it can do the former as well. This would
use RCTypes.

Use case a) probably has less interested people than use case b).
Eventually JIT technologies will help here, which will make use case a)
less important.

I happen to be interested in use case a) though. It is quite exciting to
me that we could have a faster template language interpreter
implementation that we can use in CPython today, with the potential
benefits of PyPy for interpreter translation in the future as well. We
are basically language implementors, one of your core target audiences.
We just feel more comfortable doing work if we know we have payoffs in
the near future, i.e. use in CPython. If we were implementing this
template language in isolation and had no hope we could deploy it in
production any time soon, we would be less interested.

I am actually quite interested in use case b) as well. While developing
lxml, I've done quite a bit of work with Pyrex and I can see how RPython
and RCTypes might potentially help projects like that. I've been taking
a wait and see attitude here though, as the technology needed more maturing.

As I understand it, the proposed changes wouldn't affect use case a). Or
is the ability to generate CPython extensions also going to be broken
due to this work? Please let us know, as if CPython support is going to
be broken, we would likely stop our efforts for the time being.

Use case b) would certainly be broken due to this work. What's more, I
understand that testing your extension code in CPython first wouldn't
work anymore (at least if it uses CTypes). This is a very nice feature
of RPython that Pyrex doesn't have. You mentioned that there
are potential ways to reimplement this feature again, but I'm not sure
whether you're committing to such implementation work.

If I were writing C extension modules with PyPy I'd like to know things
like:

* what are the reasons this change is going to be made?

* would this benefit just core PyPy or would it benefit extension module
writers too?

* when do you expect the new way to be finalized?

* will you make a real commitment to make the "Test in CPython first"
feature work again? Or is did you just sketched out possible ways to
implement this in the new system and you hope some contributor is going
to do the work?

* will you make a commitment to make this new way useful and supported
for extension module writers, or should they invest their time in some
other technology? I.e. what are the changes a future shift will make me
change things again?

* How well documented will this be for CPython extension developers?
Will you market this feature to developers? Things like this indicate
your commitment in another way. :)

Please note that I'm not actually *asking* for anything but information.
I'm not telling the PyPy developers they should make commitments and do
various kinds of work. That's up to you. I'm just trying to guess what
other developers might want to know.

Of course, the quality of your communication and the answers you
actually give will make people decide whether to continue to use PyPy or
move on. This will affect the way your open source community will grow
and develop.

I would urge the PyPy developers to consider investing in the open
source community by supporting features like this and promoting them.
This would mean serving admittedly self-interested developers with short
term goals. The promise of immediate payoff is a great way to attract
people to an open source project. If there is the hope of continuing
payoff (and PyPy has tremendous potential here), some of those new
people can be expected to become valuable contributors to the project as
a whole. And yes, I'm self-interested in saying all of this - that's my
point.

Regards,

Martijn



From bea at changemaker.nu  Wed Jun 20 12:57:06 2007
From: bea at changemaker.nu (=?ISO-8859-1?Q?Beatrice_D=FCring?=)
Date: Wed, 20 Jun 2007 12:57:06 +0200
Subject: [pypy-dev] about breaking RPython, RCTypes
In-Reply-To: <f5av69$jbd$1@sea.gmane.org>
References: <f5av69$jbd$1@sea.gmane.org>
Message-ID: <46790801.80407@changemaker.nu>

Hi there

Martijn Faassen skrev:
> Hi there,
>
>   
> I posted this to a longer thread the other day but have received no 
> answers to it whatsoever. Perhaps it got lost, so I'll do a repost. I 
> spent a bit of time writing this post, and I ask some questions in it 
> that I think should be answered. Perhaps by telling me I don't 
> understand what I'm talking about or that my analysis is entirely 
> incorrect. Perhaps it was simply too long?
>
> Whatever it is, I do think the PyPy community needs to be aware of how 
> technical decisions have an impact on attracting or repelling potential 
> contributors. "answering by silence", which happened so far, is one way 
> to make potential contributors worried. :)
>
>   
Although I can?t answer your questions I do appreciate the time and 
effort you and others have spent
on this issue during the last couple of days, and I am sure the PyPy 
core gang appreciates your input.

The current plan is to meet at EuroPython in Vilnius to have strategic 
technical discussions regarding
direction and among them decisions regarding the issues you raise in 
this email. I interpret the silence
as people feeling that they can?t answer this until such a discussion 
(???) but that does not excuse the total
silence  - hence this email (that does not give you the answers either).

As you might have noticed   - this topic is also on the agenda as a 
sprint topic for the PyPy Vilnius sprint
at which time we hope to have made some decisions.

Since you are coming to EuroPython I look forward to meet you again and 
discuss this  and make sure
that a PyPy strategic session takes place which gives you answers - 
answers you might like - or not like ;-)

I hope this response does not frustrate you too much....

Cheers

Bea
> I only understand a little of the technical discussion. I understand
> that RCTypes has problems, and that people anticipate having to break it
> and reimplement it in a different way in order to solve these problems.
> I understand that one feature that RCTypes offers, namely the ability to
> test your RPython code on CPython, might also go away.
>
> There are currently, as far as I can see it, two reasons to use RPython
> for CPython extensions:
>
> a) benefit from performance gains by rewriting your CPython code into
> RPython. This is what we want to do with the page template interpreter.
> In case of language implementations, this might lead to the ability to
> use other PyPy technology later (such as JIT generation and so on). Some
> others might purely be interested in performance though and aren't
> implementing a language.
>
> b) better ways to write extensions that use C libraries in CPython. The
> typical Pyrex use case, though it can do the former as well. This would
> use RCTypes.
>
> Use case a) probably has less interested people than use case b).
> Eventually JIT technologies will help here, which will make use case a)
> less important.
>
> I happen to be interested in use case a) though. It is quite exciting to
> me that we could have a faster template language interpreter
> implementation that we can use in CPython today, with the potential
> benefits of PyPy for interpreter translation in the future as well. We
> are basically language implementors, one of your core target audiences.
> We just feel more comfortable doing work if we know we have payoffs in
> the near future, i.e. use in CPython. If we were implementing this
> template language in isolation and had no hope we could deploy it in
> production any time soon, we would be less interested.
>
> I am actually quite interested in use case b) as well. While developing
> lxml, I've done quite a bit of work with Pyrex and I can see how RPython
> and RCTypes might potentially help projects like that. I've been taking
> a wait and see attitude here though, as the technology needed more maturing.
>
> As I understand it, the proposed changes wouldn't affect use case a). Or
> is the ability to generate CPython extensions also going to be broken
> due to this work? Please let us know, as if CPython support is going to
> be broken, we would likely stop our efforts for the time being.
>
> Use case b) would certainly be broken due to this work. What's more, I
> understand that testing your extension code in CPython first wouldn't
> work anymore (at least if it uses CTypes). This is a very nice feature
> of RPython that Pyrex doesn't have. You mentioned that there
> are potential ways to reimplement this feature again, but I'm not sure
> whether you're committing to such implementation work.
>
> If I were writing C extension modules with PyPy I'd like to know things
> like:
>
> * what are the reasons this change is going to be made?
>
> * would this benefit just core PyPy or would it benefit extension module
> writers too?
>
> * when do you expect the new way to be finalized?
>
> * will you make a real commitment to make the "Test in CPython first"
> feature work again? Or is did you just sketched out possible ways to
> implement this in the new system and you hope some contributor is going
> to do the work?
>
> * will you make a commitment to make this new way useful and supported
> for extension module writers, or should they invest their time in some
> other technology? I.e. what are the changes a future shift will make me
> change things again?
>
> * How well documented will this be for CPython extension developers?
> Will you market this feature to developers? Things like this indicate
> your commitment in another way. :)
>
> Please note that I'm not actually *asking* for anything but information.
> I'm not telling the PyPy developers they should make commitments and do
> various kinds of work. That's up to you. I'm just trying to guess what
> other developers might want to know.
>
> Of course, the quality of your communication and the answers you
> actually give will make people decide whether to continue to use PyPy or
> move on. This will affect the way your open source community will grow
> and develop.
>
> I would urge the PyPy developers to consider investing in the open
> source community by supporting features like this and promoting them.
> This would mean serving admittedly self-interested developers with short
> term goals. The promise of immediate payoff is a great way to attract
> people to an open source project. If there is the hope of continuing
> payoff (and PyPy has tremendous potential here), some of those new
> people can be expected to become valuable contributors to the project as
> a whole. And yes, I'm self-interested in saying all of this - that's my
> point.
>
> Regards,
>
> Martijn
>
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev
>   



From faassen at startifact.com  Wed Jun 20 13:14:30 2007
From: faassen at startifact.com (Martijn Faassen)
Date: Wed, 20 Jun 2007 13:14:30 +0200
Subject: [pypy-dev] about breaking RPython, RCTypes
In-Reply-To: <46790801.80407@changemaker.nu>
References: <f5av69$jbd$1@sea.gmane.org> <46790801.80407@changemaker.nu>
Message-ID: <f5b26k$t7n$1@sea.gmane.org>

Hey Beatrice,

An answer of "we're thinking about it, we can't answer it yet" is 
totally acceptable. Thanks!

I realize fully that the answers you come up with may be answers I won't 
like. That's fine (well, it's not, but it's not my project so I can't 
complain :) I am just trying to help in articulating hopefully relevant 
questions.

Regards,

Martijn



From radix at twistedmatrix.com  Wed Jun 20 15:48:59 2007
From: radix at twistedmatrix.com (Christopher Armstrong)
Date: Wed, 20 Jun 2007 09:48:59 -0400
Subject: [pypy-dev] about breaking RPython, RCTypes
In-Reply-To: <f5av69$jbd$1@sea.gmane.org>
References: <f5av69$jbd$1@sea.gmane.org>
Message-ID: <60ed19d40706200648l56d57a79o16add35338d85036@mail.gmail.com>

On 6/20/07, Martijn Faassen <faassen at startifact.com> wrote:
> There are currently, as far as I can see it, two reasons to use RPython
> for CPython extensions:
>
> a) benefit from performance gains by rewriting your CPython code into
> RPython. This is what we want to do with the page template interpreter.
> In case of language implementations, this might lead to the ability to
> use other PyPy technology later (such as JIT generation and so on). Some
> others might purely be interested in performance though and aren't
> implementing a language.
>
> b) better ways to write extensions that use C libraries in CPython. The
> typical Pyrex use case, though it can do the former as well. This would
> use RCTypes.

While this is perhaps only an instance of your first use case
(although not really performance-oriented), using RPython code as a
CPython extension module is how I was planning on Safelisp being used
(a small safe language written in RPython, meant for embedding). I can
imagine a similar situation for PyPy's Python interpreter itself, so
one could "sneak" PyPy-using Python code into their production system
by importing the PyPy Python interpreter as a CPython module.


-- 
Christopher Armstrong
International Man of Twistery
http://radix.twistedmatrix.com/
http://twistedmatrix.com/
http://canonical.com/


From arigo at tunes.org  Wed Jun 20 18:16:51 2007
From: arigo at tunes.org (Armin Rigo)
Date: Wed, 20 Jun 2007 18:16:51 +0200
Subject: [pypy-dev] Work plan for PyPy
In-Reply-To: <f5658p$55m$1@sea.gmane.org>
References: <20070614164554.GA654@code0.codespeak.net>
	<20070615174454.6e87fb1a.simon@arrowtheory.com>
	<20070616171054.GA695@code0.codespeak.net>
	<C092E820-2BAC-4604-9A74-2DDB5E051630@stackless.com>
	<f5658p$55m$1@sea.gmane.org>
Message-ID: <20070620161651.GA23345@code0.codespeak.net>

Hi Martijn,

On Mon, Jun 18, 2007 at 04:36:10PM +0200, Martijn Faassen wrote:
> If I were writing C extension modules with PyPy I'd like to know things 
> like:
> 
> * what are the reasons this change is going to be made?
> 
> * would this benefit just core PyPy or would it benefit extension module 
> writers too?
>
> (...)

That's a very good list of questions.  Indeed, we don't have clear
answers for them yet, but just vague thoughts.  I think it's fair to say
that we'll discuss this at EuroPython and make some commitments then;
for now we're only tossing ideas around.


A bientot,

Armin.


From faassen at startifact.com  Wed Jun 20 20:32:26 2007
From: faassen at startifact.com (Martijn Faassen)
Date: Wed, 20 Jun 2007 20:32:26 +0200
Subject: [pypy-dev] Work plan for PyPy
In-Reply-To: <20070620161651.GA23345@code0.codespeak.net>
References: <20070614164554.GA654@code0.codespeak.net>
	<20070615174454.6e87fb1a.simon@arrowtheory.com>
	<20070616171054.GA695@code0.codespeak.net>
	<C092E820-2BAC-4604-9A74-2DDB5E051630@stackless.com>
	<f5658p$55m$1@sea.gmane.org>
	<20070620161651.GA23345@code0.codespeak.net>
Message-ID: <8928d4e90706201132h46c83df0n5fc1f885d1fb1d91@mail.gmail.com>

Hey,

On 6/20/07, Armin Rigo <arigo at tunes.org> wrote:
[my questions]

> That's a very good list of questions.  Indeed, we don't have clear
> answers for them yet, but just vague thoughts.  I think it's fair to say
> that we'll discuss this at EuroPython and make some commitments then;
> for now we're only tossing ideas around.

Thanks, that's totally understandable. :)

Regards,

Martijn


From scott+pypy-dev at scottdial.com  Thu Jun 21 09:11:15 2007
From: scott+pypy-dev at scottdial.com (Scott Dial)
Date: Thu, 21 Jun 2007 03:11:15 -0400
Subject: [pypy-dev] lltypesystem typecache
Message-ID: <467A2493.8060603@scottdial.com>

Greetings,

The typecache in subversion repository is missing the 32bit windows 
platform that I am running the windows tests against, and on occasion 
(like today) I noticed it caused a update conflict due to Armin's 
changes in r44398. If someone could add this one, thanks.

-Scott

Index: pypy/rpython/lltypesystem/typecache.py
===================================================================
--- pypy/rpython/lltypesystem/typecache.py	(revision 44403)
+++ pypy/rpython/lltypesystem/typecache.py	(working copy)
@@ -1,6 +1,7 @@
  # this is automatically generated cache files for c types
  platforms = {
  ('', ('32bit', 'ELF'), 'Linux'):{'short': 16, 'int': 32, 'unsigned 
char': 8, 'long': 32, 'char': 8, 'unsigned short': 16, 'unsigned long': 
32, 'long long': 64, 'mode_t': 32, 'unsigned long long': 64, 'size_t': 
32, 'unsigned int': 32},
+('', ('32bit', 'WindowsPE'), 'Windows'):{'short': 16, 'int': 32, 
'unsigned char': 8, 'long': 32, 'char': 8, 'unsigned short': 16, 
'unsigned long': 32, 'long long': 64, 'unsigned long long': 64, 
'size_t': 32, 'unsigned int': 32},
  ('i386', ('32bit', ''), 'Darwin'):{'short': 16, 'int': 32, 'unsigned 
char': 8, 'long': 32, 'char': 8, 'unsigned short': 16, 'unsigned long': 
32, 'long long': 64, 'mode_t': 16, 'unsigned long long': 64, 'size_t': 
32, 'unsigned int': 32},
  ('Intel(R) Pentium(R) M processor 1.73GHz', ('32bit', ''), 
'Linux'):{'short': 16, 'int': 32, 'unsigned char': 8, 'long': 32, 
'char': 8, 'unsigned short': 16, 'unsigned long': 32, 'long long': 64, 
'mode_t': 32, 'unsigned long long': 64, 'size_t': 32, 'unsigned int': 32}
  }

-- 
Scott Dial
scott at scottdial.com
scodial at cs.indiana.edu


From amauryfa at gmail.com  Thu Jun 21 10:05:31 2007
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Thu, 21 Jun 2007 10:05:31 +0200
Subject: [pypy-dev] lltypesystem typecache
In-Reply-To: <e27efe130706210104h5d9d53fcq8a4677767eef8993@mail.gmail.com>
References: <467A2493.8060603@scottdial.com>
	<e27efe130706210104h5d9d53fcq8a4677767eef8993@mail.gmail.com>
Message-ID: <e27efe130706210105rcf0eac5n1026466ec4488462@mail.gmail.com>

Scott Dial wrote:
> Greetings,
>
> The typecache in subversion repository is missing the 32bit windows
> platform that I am running the windows tests against, and on occasion
> (like today) I noticed it caused a update conflict due to Armin's
> changes in r44398. If someone could add this one, thanks.
>
> -Scott

I have seen the same behaviour, and my diffs are the same.
I just commited them.

-- 
Amaury Forgeot d'Arc


From jgustak at gmail.com  Thu Jun 21 21:53:42 2007
From: jgustak at gmail.com (Jakub Gustak)
Date: Thu, 21 Jun 2007 21:53:42 +0200
Subject: [pypy-dev] Scheme interpreter - evaluation context
Message-ID: <a26b53540706211253l6a1d2bdel999004dc7a4cedae@mail.gmail.com>

I was looking into some code, JS interpreter to be exact.

I wonder what are the pros and cons of implementing evaluation context
as stack/list (as in JS interpreter) in opposite to implementing it as
dictionary.

If there is one name space for all objects in Scheme (I mean, there is
no distinction between variable or function when naming them),
dictionary would be the most straight forward. There is no need for a
special way to resolve naming conflicts and so on.

Or am I missing something?

Cheers,
Jakub Gustak


From santagada at gmail.com  Fri Jun 22 00:10:11 2007
From: santagada at gmail.com (Leonardo Santagada)
Date: Thu, 21 Jun 2007 19:10:11 -0300
Subject: [pypy-dev] Scheme interpreter - evaluation context
In-Reply-To: <a26b53540706211253l6a1d2bdel999004dc7a4cedae@mail.gmail.com>
References: <a26b53540706211253l6a1d2bdel999004dc7a4cedae@mail.gmail.com>
Message-ID: <79CA6D75-19D8-4CA9-8EC5-6B5CB79D5C28@gmail.com>


Em 21/06/2007, ?s 16:53, Jakub Gustak escreveu:

> I was looking into some code, JS interpreter to be exact.
>
> I wonder what are the pros and cons of implementing evaluation context
> as stack/list (as in JS interpreter) in opposite to implementing it as
> dictionary.
>
> If there is one name space for all objects in Scheme (I mean, there is
> no distinction between variable or function when naming them),
> dictionary would be the most straight forward. There is no need for a
> special way to resolve naming conflicts and so on.
>
> Or am I missing something?

On the js interpreter I only used a stack of objects because the  
specs said to do so... I never felt that that was a good idea, so you  
are probably better following what you think is best/easier.

Good luck,

--
Leonardo Santagada

Sent from my iPhone




From simon at arrowtheory.com  Fri Jun 22 03:10:01 2007
From: simon at arrowtheory.com (Simon Burton)
Date: Thu, 21 Jun 2007 18:10:01 -0700
Subject: [pypy-dev] About adding a blist implementation...
In-Reply-To: <46779173.3050902@jippii.fi>
References: <466FE95A.6020805@jippii.fi>
	<46779173.3050902@jippii.fi>
Message-ID: <20070621181001.d9322aef.simon@arrowtheory.com>

On Tue, 19 Jun 2007 11:18:59 +0300
Elmo M?ntynen <elmo13 at jippii.fi> wrote:

> 
> Since I'm adding a list implementation, I'd like to know what benchmarks
> have you used to test the other ones. Also, if you know apps with heavy
> usage of lists and some that use very long lists, I'd be interested very
> much.

Can you explain what this blist is ? Is it a btree ? I think I need
an rpython btree.

Simon.


From santagada at gmail.com  Fri Jun 22 04:13:59 2007
From: santagada at gmail.com (Leonardo Santagada)
Date: Thu, 21 Jun 2007 23:13:59 -0300
Subject: [pypy-dev] About adding a blist implementation...
In-Reply-To: <20070621181001.d9322aef.simon@arrowtheory.com>
References: <466FE95A.6020805@jippii.fi> <46779173.3050902@jippii.fi>
	<20070621181001.d9322aef.simon@arrowtheory.com>
Message-ID: <B18D5B15-5CDA-4B3C-B96A-A0FFE8DE2FBD@gmail.com>


Em 21/06/2007, ?s 22:10, Simon Burton escreveu:

> On Tue, 19 Jun 2007 11:18:59 +0300
> Elmo M?ntynen <elmo13 at jippii.fi> wrote:
>
>>
>> Since I'm adding a list implementation, I'd like to know what  
>> benchmarks
>> have you used to test the other ones. Also, if you know apps with  
>> heavy
>> usage of lists and some that use very long lists, I'd be  
>> interested very
>> much.
>
> Can you explain what this blist is ? Is it a btree ? I think I need
> an rpython btree.
>
> Simon.

I dunno but this guy is probably talking about something like this  
http://mail.python.org/pipermail/python-3000/2007-May/thread.html#7127

If it is not related I would also like to understand the diferences.

--
Leonardo Santagada

Sent from my iPhone




From arigo at tunes.org  Fri Jun 22 10:51:31 2007
From: arigo at tunes.org (Armin Rigo)
Date: Fri, 22 Jun 2007 10:51:31 +0200
Subject: [pypy-dev] lltypesystem typecache
In-Reply-To: <467A2493.8060603@scottdial.com>
References: <467A2493.8060603@scottdial.com>
Message-ID: <20070622085129.GA16685@code0.codespeak.net>

Hi Scott,

On Thu, Jun 21, 2007 at 03:11:15AM -0400, Scott Dial wrote:
> (like today) I noticed it caused a update conflict due to Armin's 
> changes in r44398.

My checkin was mostly accidental.  I think that the way the typecache is
both a regular svn-controlled file deep within the hierarchy, and
modified automatically, will keep causing conflicts in the future.
Also, the entry that was generated on my laptop and that I checked in
while doing other stuff is:

> ('Intel(R) Pentium(R) M processor 1.73GHz', ('32bit', ''), 'Linux'):...

That looks like a far too precise key.

I don't have a nice alternative, I'm just saying that the current way it
works is a bit troublesome :-)


A bientot,

Armin.


From scott+pypy-dev at scottdial.com  Fri Jun 22 11:01:37 2007
From: scott+pypy-dev at scottdial.com (Scott Dial)
Date: Fri, 22 Jun 2007 05:01:37 -0400
Subject: [pypy-dev] lltypesystem typecache
In-Reply-To: <20070622085129.GA16685@code0.codespeak.net>
References: <467A2493.8060603@scottdial.com>
	<20070622085129.GA16685@code0.codespeak.net>
Message-ID: <467B8FF1.50301@scottdial.com>

Armin Rigo wrote:
> My checkin was mostly accidental.  I think that the way the typecache is
> both a regular svn-controlled file deep within the hierarchy, and
> modified automatically, will keep causing conflicts in the future.
> 

I thought this as well, but I am not familiar with its purpose. If it is 
merely a cache, then I don't understand why it is under SVN control at 
all. I suppose it allows developers without those other platforms 
available to know about types, but that seems like a marginal reason.

-Scott

-- 
Scott Dial
scott at scottdial.com
scodial at cs.indiana.edu


From arigo at tunes.org  Fri Jun 22 11:05:36 2007
From: arigo at tunes.org (Armin Rigo)
Date: Fri, 22 Jun 2007 11:05:36 +0200
Subject: [pypy-dev] Vilnius/Post EuroPython PyPy Sprint 12-14th of July
Message-ID: <20070622090536.GB16685@code0.codespeak.net>

========================================================
Vilnius/Post EuroPython PyPy Sprint 12-14th of July
========================================================

The PyPy team is sprinting at EuroPython again and we invite
you to participate in our 3 day long sprint at the conference hotel
- Reval Hotel Lietuva.

If you plan to attend the sprint we recommend you to listen to the PyPy
technical talks (`EuroPython schedule`_) during the
conference since it will give you a good overview of the status of development.

On the morning of the first sprint day (12th) we will also have a
tutorial session for those new to PyPy development.  As 3 days is relatively
short for a PyPy sprint we suggest to travel back home on the 15th if
possible (but it is ok to attend less than 3 days too).

------------------------------
Goals and topics of the sprint
------------------------------

There are many possible and interesting sprint topics to work on - here
we list some possible task areas:

* completing the missing python 2.5 features and support
* write or port more extension modules (e.g. zlib is missing)
* identify slow areas of PyPy through benchmarking and work on improvements,
  possibly moving app-level parts of the Python interpreter to interp-level
  if useful.
* there are some parts of PyPy in need of refactoring, we may spend some time
  on those, for example:

    - rctypes and the extension compiler need some rethinking
    - support for LLVM 2.0 for the llvm backend
    - ...

* some JIT improvement work
* port the stackless transform to ootypesystem
* other interesting stuff that you would like to work on ...;-)

------------
Registration
------------

If you'd like to come, please subscribe to the `pypy-sprint mailing list`_
and drop a note about your interests and post any questions.  More
organisational information will be sent to that list.

Please register by adding yourself on the following list (via svn):

  http://codespeak.net/svn/pypy/extradoc/sprintinfo/post-ep2007/people.txt

or on the pypy-sprint mailing list if you do not yet have check-in rights:

  http://codespeak.net/mailman/listinfo/pypy-sprint

---------------------------------------
Preparation (if you feel it is needed):
---------------------------------------

* read the `getting-started`_ pages on http://codespeak.net/pypy

* for inspiration, overview and technical status you are welcome to
  read `the technical reports available and other relevant documentation`_

* please direct any technical and/or development oriented questions to
  pypy-dev at codespeak.net and any sprint organizing/logistical
  questions to pypy-sprint at codespeak.net

* if you need information about the conference, potential hotels,
  directions etc we recommend to look at http://www.europython.org.


We are looking forward to meet you at the Vilnius Post EuroPython
PyPy sprint!

The PyPy team 


.. See also ..

.. _getting-started: http://codespeak.net/pypy/dist/pypy/doc/getting-started.html
.. _`pypy-sprint mailing list`: http://codespeak.net/mailman/listinfo/pypy-sprint
.. _`the technical reports available and other relevant documentation`: http://codespeak.net/pypy/dist/pypy/doc/index.html
.. _`EuroPython schedule`: http://indico.cern.ch/conferenceTimeTable.py?confId=13919&showDate=all&showSession=all&detailLevel=contribution&viewMode=room


From micahel at gmail.com  Fri Jun 22 11:11:46 2007
From: micahel at gmail.com (Michael Hudson)
Date: Fri, 22 Jun 2007 10:11:46 +0100
Subject: [pypy-dev] lltypesystem typecache
In-Reply-To: <20070622085129.GA16685@code0.codespeak.net>
References: <467A2493.8060603@scottdial.com>
	<20070622085129.GA16685@code0.codespeak.net>
Message-ID: <aa0dfd860706220211w106985aes9f399d66e3bc1255@mail.gmail.com>

On 22/06/07, Armin Rigo <arigo at tunes.org> wrote:
> Hi Scott,
>
> On Thu, Jun 21, 2007 at 03:11:15AM -0400, Scott Dial wrote:
> > (like today) I noticed it caused a update conflict due to Armin's
> > changes in r44398.
>
> My checkin was mostly accidental.  I think that the way the typecache is
> both a regular svn-controlled file deep within the hierarchy, and
> modified automatically, will keep causing conflicts in the future.

Indeed, since the rffi merge, we haven't had two builds in a row on tuatara :(

Cheers,
mwh


From elmo13 at jippii.fi  Fri Jun 22 16:10:44 2007
From: elmo13 at jippii.fi (=?ISO-8859-1?Q?Elmo_M=E4ntynen?=)
Date: Fri, 22 Jun 2007 17:10:44 +0300
Subject: [pypy-dev] About adding a blist implementation...
In-Reply-To: <B18D5B15-5CDA-4B3C-B96A-A0FFE8DE2FBD@gmail.com>
References: <466FE95A.6020805@jippii.fi>
	<46779173.3050902@jippii.fi>	<20070621181001.d9322aef.simon@arrowtheory.com>
	<B18D5B15-5CDA-4B3C-B96A-A0FFE8DE2FBD@gmail.com>
Message-ID: <467BD864.8040505@jippii.fi>

Leonardo Santagada wrote:
> Em 21/06/2007, ?s 22:10, Simon Burton escreveu:
>
>   
>> On Tue, 19 Jun 2007 11:18:59 +0300
>> Elmo M?ntynen <elmo13 at jippii.fi> wrote:
>>
>>     
>>> Since I'm adding a list implementation, I'd like to know what  
>>> benchmarks
>>> have you used to test the other ones. Also, if you know apps with  
>>> heavy
>>> usage of lists and some that use very long lists, I'd be  
>>> interested very
>>> much.
>>>       
>> Can you explain what this blist is ? Is it a btree ? I think I need
>> an rpython btree.
>>
>> Simon.
>>     
>
> I dunno but this guy is probably talking about something like this  
> http://mail.python.org/pipermail/python-3000/2007-May/thread.html#7127
>
> If it is not related I would also like to understand the diferences.
>
> --
> Leonardo Santagada
>
> Sent from my iPhone
>
>   
Hi.
It is an list implementation that has much better performance with very
big lists, available as an extension to CPython, and soon included in
pypy by me.
This cheeseshop entry is a good place to start:
http://www.python.org/pypi/blist/

The thread might be about integrading it into CPython or something,
haven't read.

It is based on btree, but the implementation I'm working on is based on
the python prototype included in the tar distrib, which won't  probably
be useful if you specifically need a btree. In the future, maybe some
refactoring would be useful, but for now I'm happy with the way I started.

I'd still be interested in ways to test the different list
implementations =)

Elmo


From arigo at tunes.org  Sat Jun 23 10:25:38 2007
From: arigo at tunes.org (Armin Rigo)
Date: Sat, 23 Jun 2007 10:25:38 +0200
Subject: [pypy-dev] [LLVMdev] Vilnius/Post EuroPython PyPy Sprint
	12-14th of July
In-Reply-To: <1182535152.6173.25.camel@asl.dorms.spbu.ru>
References: <20070622090903.GA19700.SS3552SS@code0.codespeak.net>
	<1182535152.6173.25.camel@asl.dorms.spbu.ru>
Message-ID: <20070623082538.GB358@code0.codespeak.net>

Hi Anton,

(CC'ing to pypy-dev)

On Fri, Jun 22, 2007 at 09:59:12PM +0400, Anton Korobeynikov wrote:
> >     - support for LLVM 2.0 for the llvm backend
> Is there any detailed information, how the sprint will touch this topic?

I don't have many details, but there are three subtasks regarding LLVM:
on the one hand, we did a small addition to the translation toolchain
that is not yet supported by our backend generating LLVM code, so the
first subtask is this "maintenance" task, which should be not too much
efforts if done while pairing with someone that knows already a bit
about the backend.  Then, there is the idea of changing the LLVM
assembler produced - at the moment, it's for LLVM 1.9, but we want to
move to 2.0.  I can't tell how much effort this is, as I'm not enough
into LLVM or the LLVM backend.  The third, longer-term task that we may
not start during the sprint would be to refactor and clean up the LLVM
backend and the C backend, so that they can share code a bit more
naturally (at the moment the LLVM backend fishes for internal routines
of the C backend).


A bientot,

Armin.


From rxe at ukshells.co.uk  Sat Jun 23 10:35:34 2007
From: rxe at ukshells.co.uk (Richard Emslie)
Date: Sat, 23 Jun 2007 01:35:34 -0700
Subject: [pypy-dev] [LLVMdev] Vilnius/Post EuroPython PyPy Sprint
	12-14th of July
In-Reply-To: <20070623082538.GB358@code0.codespeak.net>
References: <20070622090903.GA19700.SS3552SS@code0.codespeak.net>
	<1182535152.6173.25.camel@asl.dorms.spbu.ru>
	<20070623082538.GB358@code0.codespeak.net>
Message-ID: <BD389672-9907-4907-80CA-8F3AC1B265F1@ukshells.co.uk>

Hi,

Just a word of warning - i am (was) very much planning to do some of  
these tasks pre-sprint.  Not sure I can make the sprint anyhow.

A 4th larger task also is to make gc and stackless work.

A 5th task is to take pbcs work again! ;-)

I'll be on irc over the weekend - right now i need some sleep.

Cheers,
Richard

On Jun 23, 2007, at 1:25 AM, Armin Rigo wrote:

> Hi Anton,
>
> (CC'ing to pypy-dev)
>
> On Fri, Jun 22, 2007 at 09:59:12PM +0400, Anton Korobeynikov wrote:
>>>     - support for LLVM 2.0 for the llvm backend
>> Is there any detailed information, how the sprint will touch this  
>> topic?
>
> I don't have many details, but there are three subtasks regarding  
> LLVM:
> on the one hand, we did a small addition to the translation toolchain
> that is not yet supported by our backend generating LLVM code, so the
> first subtask is this "maintenance" task, which should be not too much
> efforts if done while pairing with someone that knows already a bit
> about the backend.  Then, there is the idea of changing the LLVM
> assembler produced - at the moment, it's for LLVM 1.9, but we want to
> move to 2.0.  I can't tell how much effort this is, as I'm not enough
> into LLVM or the LLVM backend.  The third, longer-term task that we  
> may
> not start during the sprint would be to refactor and clean up the LLVM
> backend and the C backend, so that they can share code a bit more
> naturally (at the moment the LLVM backend fishes for internal routines
> of the C backend).
>
>
> A bientot,
>
> Armin.
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev



From fijal at genesilico.pl  Mon Jun 25 13:37:18 2007
From: fijal at genesilico.pl (Maciek Fijalkowski)
Date: Mon, 25 Jun 2007 13:37:18 +0200
Subject: [pypy-dev] lltypesystem typecache
In-Reply-To: <aa0dfd860706220211w106985aes9f399d66e3bc1255@mail.gmail.com>
References: <467A2493.8060603@scottdial.com>	<20070622085129.GA16685@code0.codespeak.net>
	<aa0dfd860706220211w106985aes9f399d66e3bc1255@mail.gmail.com>
Message-ID: <467FA8EE.4010206@genesilico.pl>

Hum. The idea was to keep this stuff in svn in order to keep type 
information to-be-copied around. If this is troublesome, I propose to 
kill whole idea of having cache-per-platform, since we cannot specify 
good platform key and have it in svn:ignore (or move to _cache) and just 
keep it on a platform level. Than one would need to have gcc installed 
at least once to import rffi. I don't know if it's good or bad.



From elmo13 at jippii.fi  Mon Jun 25 15:45:53 2007
From: elmo13 at jippii.fi (=?UTF-8?B?RWxtbyBNw6RudHluZW4=?=)
Date: Mon, 25 Jun 2007 16:45:53 +0300
Subject: [pypy-dev] Some bugfixes to the multilistimplementation
Message-ID: <467FC711.100@jippii.fi>

Hi.
I've made the different [slow] list implementations even slower !

While integrating the blist implementation, I found out that the chosen
implementation isn't even used when instantiating with "list()", and
neither in some other usecases. By correcting that, a bug in the chunked
implementation got uncovered, which I also fixed. I also added
"make_implementation_with_one_item()" to have the instantiation of the
implementations in one place so that it's easier to add new ones.

If they seem good, could someone commit these changes (is that diff
format good?). Would you trust me with commit rights, or should I still
send patches?

The test demonstrates the bugs, but I thougth of later adding a
BaseAppTest_ListMultiObject class with magic to test if the
implementations that  the subclasses test are really used. Does that
sound good?



Index: objspace/std/listmultiobject.py
===================================================================
--- objspace/std/listmultiobject.py     (revision 44498)
+++ objspace/std/listmultiobject.py     (working copy)
@@ -12,7 +12,7 @@

 # An empty list always is an EmptyListImplementation.
 #
-# RDictImplementation -- standard implementation
+# RListImplementation -- standard implementation
 # StrListImplementation -- lists consisting only of strings
 # ChunkedListImplementation -- when having set the withchunklist option
 # SmartResizableListImplementation -- when having set the
@@ -87,7 +87,7 @@
         for i in range(slicelength):
             res_w[i] = self.getitem(start)
             start += step
-        return RListImplementation(self.space, res_w)
+        return make_implementation(self.space, res_w)

     def delitem_slice_step(self, start, stop, step, slicelength):
         n = self.length()
@@ -368,8 +368,8 @@
         length = self._length

         self._grow()
-        for j in range(length - 1, 0, -1):
-            self.setitem(j + 1, self.getitem(j))
+        for j in range(length, i, -1):
+            self.setitem(j, self.getitem(j-1))
         self.setitem(i, w_item)

         return self
@@ -399,13 +399,7 @@
 class EmptyListImplementation(ListImplementation):
     def make_list_with_one_item(self, w_item):
         space = self.space
-        if space.config.objspace.std.withfastslice:
-            return SliceTrackingListImplementation(space, [w_item])
-        w_type = space.type(w_item)
-        if space.is_w(w_type, space.w_str):
-            strlist = [space.str_w(w_item)]
-            return StrListImplementation(space, strlist)
-        return RListImplementation(space, [w_item])
+        return make_implementation_with_one_item(space, w_item)

     def length(self):
         return 0
@@ -834,6 +828,27 @@
         else:
             return RListImplementation(space, list_w)

+def make_implementation_with_one_item(space, w_item):
+        if space.config.objspace.std.withfastslice:
+            return SliceTrackingListImplementation(space, [w_item])
+        if space.config.objspace.std.withsmartresizablelist:
+            from pypy.objspace.std.smartresizablelist import \
+                SmartResizableListImplementation
+            impl = SmartResizableListImplementation(space)
+            impl.append(w_item)
+            return impl
+        if space.config.objspace.std.withchunklist:
+            return ChunkedListImplementation(space, [w_item])
+        if space.config.objspace.std.withblist:
+            from pypy.objspace.std.blistimplementation import \
+                BListImplementation
+            return BListImplementation(space, [w_item])
+        w_type = space.type(w_item)
+        if space.is_w(w_type, space.w_str):
+            strlist = [space.str_w(w_item)]
+            return StrListImplementation(space, strlist)
+        return RListImplementation(space, [w_item])
+
 def convert_list_w(space, list_w):
     if not list_w:
         impl = space.fromcache(State).empty_impl
@@ -884,7 +899,7 @@
     if w_iterable is not EMPTY_LIST:
         list_w = space.unpackiterable(w_iterable)
         if list_w:
-            w_list.implementation = RListImplementation(space, list_w)
+            w_list.implementation = make_implementation(space, list_w)
             return
     w_list.implementation = space.fromcache(State).empty_impl

@@ -1304,9 +1319,8 @@
             sorterclass = CustomKeySort
         else:
             sorterclass = SimpleSort
-    impl = w_list.implementation.to_rlist()
-    w_list.implementation = impl
-    items = impl.list_w
+    impl=w_list.implementation
+    items = impl.get_list_w()
     sorter = sorterclass(items, impl.length())
     sorter.space = space
     sorter.w_cmp = w_cmp
@@ -1349,8 +1363,7 @@
         mucked = w_list.implementation.length() > 0

         # put the items back into the list
-        impl.list_w = sorter.list
-        w_list.implementation = impl
+        w_list.implementation = make_implementation(space, sorter.list)

     if mucked:
         raise OperationError(space.w_ValueError,
Index: objspace/std/test/test_listmultiobject.py
===================================================================
--- objspace/std/test/test_listmultiobject.py   (revision 44498)
+++ objspace/std/test/test_listmultiobject.py   (working copy)
@@ -144,3 +144,15 @@
     def teardown_class(cls):
         _set_chunk_size_bits(cls.chunk_size_bits)

+    def test_chunklist_is_implementation_used(self):
+        import __pypy__
+        l = ["1", "2", "3", "4", "5"]
+        assert "ChunkedListImplementation" in __pypy__.internal_repr(l)
+        l = list(["1", "2", "3", "4", "5"])
+        assert "ChunkedListImplementation" in __pypy__.internal_repr(l)
+        l=[]
+        l.append('a')
+        assert "ChunkedListImplementation" in __pypy__.internal_repr(l)
+        l = ["6", "8", "3", "1", "5"]
+        l.sort()
+        assert "ChunkedListImplementation" in __pypy__.internal_repr(l)


From cfbolz at gmx.de  Mon Jun 25 15:57:10 2007
From: cfbolz at gmx.de (Carl Friedrich Bolz)
Date: Mon, 25 Jun 2007 15:57:10 +0200
Subject: [pypy-dev] Some bugfixes to the multilistimplementation
In-Reply-To: <467FC711.100@jippii.fi>
References: <467FC711.100@jippii.fi>
Message-ID: <348899050706250657u4b1261d4h41da130eac059546@mail.gmail.com>

Hi Elmo!

2007/6/25, Elmo M?ntynen <elmo13 at jippii.fi>:
> I've made the different [slow] list implementations even slower !

good!

> While integrating the blist implementation, I found out that the chosen
> implementation isn't even used when instantiating with "list()", and
> neither in some other usecases. By correcting that, a bug in the chunked
> implementation got uncovered, which I also fixed. I also added
> "make_implementation_with_one_item()" to have the instantiation of the
> implementations in one place so that it's easier to add new ones.

The diff looks good to me. Note that there are still bugs in the
multilist stuff, it wasn't heavily tested yet.

> If they seem good, could someone commit these changes (is that diff
> format good?). Would you trust me with commit rights, or should I still
> send patches?

I think you should get commit rights. Send a mail to michael (
micahel at gmail.com ) with your username and ssh key. Standard
disclaimer: You have to agree to put your changes under the MIT
license, you have to promise to write tests and you should follow the
coding guide. Then you can just commit your work.

> The test demonstrates the bugs, but I thougth of later adding a
> BaseAppTest_ListMultiObject class with magic to test if the
> implementations that  the subclasses test are really used. Does that
> sound good?

yes, very good. I vaguely had something like this in mind, but never
got around to it.

Thanks for doing this!

Carl Friedrich


From rxe at ukshells.co.uk  Mon Jun 25 22:01:35 2007
From: rxe at ukshells.co.uk (Richard Emslie)
Date: Mon, 25 Jun 2007 13:01:35 -0700
Subject: [pypy-dev] Work plan for PyPy
In-Reply-To: <467449FC.2040909@genesilico.pl>
References: <20070614164554.GA654@code0.codespeak.net>	<20070615174454.6e87fb1a.simon@arrowtheory.com>
	<20070616171054.GA695@code0.codespeak.net>
	<467449FC.2040909@genesilico.pl>
Message-ID: <C8AE216F-01F9-4B50-87A9-BBC6F469DEF9@ukshells.co.uk>


On Jun 16, 2007, at 1:37 PM, Maciek Fijalkowski wrote:
<snip>

>>
> Just a sidenote - rffi supports (and uses) macros. Not sure how this
> will look like in a ctypes-based solution.
>

if you are talking c macros - this could be very problematic for the  
llvm backend.

cheers - Richard




From simon at arrowtheory.com  Tue Jun 26 20:40:26 2007
From: simon at arrowtheory.com (Simon Burton)
Date: Tue, 26 Jun 2007 11:40:26 -0700
Subject: [pypy-dev] ep 2007 talks
Message-ID: <20070626114026.3fb5760e.simon@arrowtheory.com>


I'd like to synchronize with Maciek and Antonio with regards
talking about rpython at EP this year. My talk comes right after
your rpython talk - i think we can perhaps share the burden (joy?)
of talking about rpython a bit.

Simon.


From arigo at tunes.org  Wed Jun 27 13:40:49 2007
From: arigo at tunes.org (Armin Rigo)
Date: Wed, 27 Jun 2007 13:40:49 +0200
Subject: [pypy-dev] Work plan for PyPy
In-Reply-To: <C8AE216F-01F9-4B50-87A9-BBC6F469DEF9@ukshells.co.uk>
References: <20070614164554.GA654@code0.codespeak.net>
	<20070615174454.6e87fb1a.simon@arrowtheory.com>
	<20070616171054.GA695@code0.codespeak.net>
	<467449FC.2040909@genesilico.pl>
	<C8AE216F-01F9-4B50-87A9-BBC6F469DEF9@ukshells.co.uk>
Message-ID: <20070627114049.GA9686@code0.codespeak.net>

Hi Richard,

On Mon, Jun 25, 2007 at 01:01:35PM -0700, Richard Emslie wrote:
> > Just a sidenote - rffi supports (and uses) macros. Not sure how this
> > will look like in a ctypes-based solution.
> 
> if you are talking c macros - this could be very problematic for the  
> llvm backend.

Anton (which I add to the CC's of this mail) proposed a solution for
calling C APIs from llvm bytecode in a portable way, which would work
for functions whose argument types are not exactly specified by the
standards as well as for some kind of macros.  The idea is to generate
stub "helper" functions in C and compile them with llvm-gcc.  The llvm
bytecode that genllvm produces would then call the helper functions
instead of calling directly the external C API.  Shouldn't be a
performance problem as llvm will inline the helpers agressively.

Of course, it points again to the question of whether it's really
worthwhile to have an llvm backend in PyPy or if just producing C and
using llvm-gcc on everything wouldn't work just as well.  On the other
hand, it's interesting to note that there are other reasons for which
exactly the same kind of helper functions would be useful; for example,
to wrap a C library full of macros and unspecified types and constants,
ctypes alone is not enough.


A bientot,

Armin.


From anto.cuni at gmail.com  Wed Jun 27 14:01:53 2007
From: anto.cuni at gmail.com (Antonio Cuni)
Date: Wed, 27 Jun 2007 14:01:53 +0200
Subject: [pypy-dev] ep 2007 talks
In-Reply-To: <20070626114026.3fb5760e.simon@arrowtheory.com>
References: <20070626114026.3fb5760e.simon@arrowtheory.com>
Message-ID: <468251B1.9020100@gmail.com>

Simon Burton wrote:
> I'd like to synchronize with Maciek and Antonio with regards
> talking about rpython at EP this year. My talk comes right after
> your rpython talk - i think we can perhaps share the burden (joy?)
> of talking about rpython a bit.


Hi Simon,
I think that talking about rpython only once makes a lot of sense.

Currently we already wrote some slides about it, that you can find in 
extradoc/talk/ep2007/rpython.txt.

As you can see we did not write much, just few slides to give an idea of 
what rpython is; after all, our intent is to tell people why the might 
be interested in it, not to give a tutorial about rpython.

Do you think is enough or you was thinking about a deeper introduction?

ciao Anto


From rxe at ukshells.co.uk  Wed Jun 27 22:55:13 2007
From: rxe at ukshells.co.uk (Richard Emslie)
Date: Wed, 27 Jun 2007 13:55:13 -0700
Subject: [pypy-dev] Fwd:  Work plan for PyPy
References: <30D05AC9-1A9E-4E90-9E79-C587AA81978C@ukshells.co.uk>
Message-ID: <B4432A41-4991-46A7-A9DA-4E2F0363F347@ukshells.co.uk>

waaaah - forget to cc everyone :-(

Begin forwarded message:

> From: Richard Emslie <rxe at ukshells.co.uk>
> Date: June 27, 2007 10:56:39 AM PDT
> To: Armin Rigo <arigo at tunes.org>
> Subject: Re: [pypy-dev] Work plan for PyPy
>
> Hi Armin!
>
> On Jun 27, 2007, at 4:40 AM, Armin Rigo wrote:
>
>> Hi Richard,
>>
>> On Mon, Jun 25, 2007 at 01:01:35PM -0700, Richard Emslie wrote:
>>>> Just a sidenote - rffi supports (and uses) macros. Not sure how  
>>>> this
>>>> will look like in a ctypes-based solution.
>>>
>>> if you are talking c macros - this could be very problematic for the
>>> llvm backend.
>>
>> Anton (which I add to the CC's of this mail) proposed a solution for
>> calling C APIs from llvm bytecode in a portable way, which would work
>> for functions whose argument types are not exactly specified by the
>> standards as well as for some kind of macros.  The idea is to  
>> generate
>> stub "helper" functions in C and compile them with llvm-gcc.  The  
>> llvm
>> bytecode that genllvm produces would then call the helper functions
>> instead of calling directly the external C API.  Shouldn't be a
>> performance problem as llvm will inline the helpers agressively.
>>
>
> ok, that is somewhat what we have now - except the stub functions  
> are high level and shared *somewhat* with genc.
>
> Generating c code in llvm backend would sort of defeat the point of  
> it IMHO.  We already have a backend that generates c.  On the other  
> hand, if we hand write the c code - it may be an neverending task  
> of adding hacks to the backend.  For instance just getting at errno  
> is particularly problematic right now - and would involve a  
> function call from a constant value.  which means injecting a few  
> lines assembly and a temporary variable.  Easy to do, but it is  
> nice right now that we pretty much have a direct mapping from our  
> our lli form to llvm assembly.
>
>
>> Of course, it points again to the question of whether it's really
>> worthwhile to have an llvm backend in PyPy or if just producing C and
>> using llvm-gcc on everything wouldn't work just as well.  On the  
>> other
>> hand, it's interesting to note that there are other reasons for which
>> exactly the same kind of helper functions would be useful; for  
>> example,
>> to wrap a C library full of macros and unspecified types and  
>> constants,
>> ctypes alone is not enough.
>
>
> Like you say, llvm-gcc (well at least v4) would probably produce  
> equivalent results with genc.  The way we are using llvm right now  
> (as a static compiler) i'm not sure of its worthiness either of  
> keeping the backend alive.  Other than the potential of accurate  
> gc, potential of jit (which is sort of orthogonal to this),  
> potential of producing vectorized code from rpython...
>
>
> Cheers,
> Richard
>
>
>
>
>
>



From simon at arrowtheory.com  Thu Jun 28 04:07:15 2007
From: simon at arrowtheory.com (Simon Burton)
Date: Wed, 27 Jun 2007 19:07:15 -0700
Subject: [pypy-dev] Work plan for PyPy
In-Reply-To: <4673C374.2060806@genesilico.pl>
References: <20070614164554.GA654@code0.codespeak.net>
	<20070615174454.6e87fb1a.simon@arrowtheory.com>
	<4673C374.2060806@genesilico.pl>
Message-ID: <20070627190715.3b35c12f.simon@arrowtheory.com>


Do you think we could extend rffi to be able to
expose function calls to outside callers, so that eg.
we can make nifty cpython extension modules ?

(i think this is number 3 in my list)

Simon.

On Sat, 16 Jun 2007 13:03:16 +0200
Maciek Fijalkowski <fijal at genesilico.pl> wrote:

> Simon Burton wrote:
> > On Thu, 14 Jun 2007 18:45:54 +0200
> > Armin Rigo <arigo at tunes.org> wrote:
> >
> >   
> >> Any comments or objections?  Are there people that already make heavy
> >> use of rctypes, or the extension compiler?
> >>     
> >
> > Indeed. Mainly rctypes.
> >
> > As well as an ever expanding use of libc, 
> > we recently have interfaced to libSDL and cairo for some
> > very funky rpython graphics apps. And we use libpython
> > so that we can extract stuff from python modules.
> > Also, we do byte shuffling with rctypes: reading binary data
> > to/from buffers, etc.
> >
> > I hope we can do all this with rffi, but there are two really
> > cool things about rctypes:
> > 1) it runs on cpython (our main app takes 30minutes+ to compile)
> > 2) it has an automatic code generator (ctypes can autogenerate
> > python wrappers from C header files)
> >   
> Very valid points indeed. The 1st point sounds unlikely, also because 
> you might use C macros with rffi, which somehow invalidates possible 
> uses on top of ctypes. What I do is I test (by compiling) several parts 
> (some small code snippets involving ext functions). That make debugging 
> a bit harder (reading C code), but pretty straightforward - for example 
> memory management is done by hand in this part, which makes you 
> responsible of tracking C structures and how long they need to live.
> 
> The second point is much more likely to happen and to be better than 
> ctypes code generation. (Because rffi is way more Cish than ctypes are).
> 
> Also it makes stuff a bit harder when you don't submit your bindings back.
> 
> If you plan to come to the post-EP pypy sprint, we can sit a bit on the 
> automatic code generator for rffi.
> 
> Cheers,
> fijal
> 


From fijal at genesilico.pl  Thu Jun 28 11:03:34 2007
From: fijal at genesilico.pl (Maciek Fijalkowski)
Date: Thu, 28 Jun 2007 11:03:34 +0200
Subject: [pypy-dev] ep 2007 talks
In-Reply-To: <468251B1.9020100@gmail.com>
References: <20070626114026.3fb5760e.simon@arrowtheory.com>
	<468251B1.9020100@gmail.com>
Message-ID: <46837966.7070400@genesilico.pl>

Antonio Cuni wrote:
> Simon Burton wrote:
>   
>> I'd like to synchronize with Maciek and Antonio with regards
>> talking about rpython at EP this year. My talk comes right after
>> your rpython talk - i think we can perhaps share the burden (joy?)
>> of talking about rpython a bit.
>>     
>
>
> Hi Simon,
> I think that talking about rpython only once makes a lot of sense.
>
> Currently we already wrote some slides about it, that you can find in 
> extradoc/talk/ep2007/rpython.txt.
>
> As you can see we did not write much, just few slides to give an idea of 
> what rpython is; after all, our intent is to tell people why the might 
> be interested in it, not to give a tutorial about rpython.
>
> Do you think is enough or you was thinking about a deeper introduction?
>   
I would extremely enjoy it, hence you give us some reason why we're 
talking about this at all :) There are a bit of orthogonal issues in our 
rpython talk (js backend and so on), but mostly it would be good to 
share a bit. When do you arrive to Vilnius? (I'm arriving on 5th and I 
might have no network at all till 5th).


From fijal at genesilico.pl  Thu Jun 28 11:08:43 2007
From: fijal at genesilico.pl (Maciek Fijalkowski)
Date: Thu, 28 Jun 2007 11:08:43 +0200
Subject: [pypy-dev] about breaking RPython, RCTypes
In-Reply-To: <f5av69$jbd$1@sea.gmane.org>
References: <f5av69$jbd$1@sea.gmane.org>
Message-ID: <46837A9B.8090606@genesilico.pl>

Martijn Faassen wrote:
> Hi there,
>
> I posted this to a longer thread the other day but have received no 
> answers to it whatsoever. Perhaps it got lost, so I'll do a repost. I 
> spent a bit of time writing this post, and I ask some questions in it 
> that I think should be answered. Perhaps by telling me I don't 
> understand what I'm talking about or that my analysis is entirely 
> incorrect. Perhaps it was simply too long?
>
> Whatever it is, I do think the PyPy community needs to be aware of how 
> technical decisions have an impact on attracting or repelling potential 
> contributors. "answering by silence", which happened so far, is one way 
> to make potential contributors worried. :)
>   
Sorry for a late reply.

I think that discussing things live at the EP makes a lot of sense :) 
Also I think that several points regarding what is going to happen needs 
clarfication and so on. Let's talk about it live.

Cheers,
fijal



From arigo at tunes.org  Thu Jun 28 13:15:38 2007
From: arigo at tunes.org (Armin Rigo)
Date: Thu, 28 Jun 2007 13:15:38 +0200
Subject: [pypy-dev] Fwd:  Work plan for PyPy
In-Reply-To: <B4432A41-4991-46A7-A9DA-4E2F0363F347@ukshells.co.uk>
References: <30D05AC9-1A9E-4E90-9E79-C587AA81978C@ukshells.co.uk>
	<B4432A41-4991-46A7-A9DA-4E2F0363F347@ukshells.co.uk>
Message-ID: <20070628111538.GA27708@code0.codespeak.net>

Hi Richard,

On Wed, Jun 27, 2007 at 01:55:13PM -0700, Richard Emslie wrote:
> >>bytecode that genllvm produces would then call the helper functions
> >>instead of calling directly the external C API.  Shouldn't be a
> >>performance problem as llvm will inline the helpers agressively.
> >>
> >
> >ok, that is somewhat what we have now - except the stub functions  
> >are high level and shared *somewhat* with genc.

Precisely, by contrast I'm thinking about having lower level stub
functions that are generated automatically from the lltype declarations.
For example, an external ll function ptr would generate a simple stub,
always the same.  GenC and GenLLVM would share the code to produce the
stubs.  Similarly, getting at an external variable (or variable-like
macro) like errno should be expressable in an lltype way (it's not right
now) that gets turned into """int _rpyget_errno(void) { return errno; }""".

> >Generating c code in llvm backend would sort of defeat the point of  
> >it IMHO.

The difference is that the stubs would always be generated from simple
templates, not from genc's complex logic that turns flow graphs into C
code.


A bientot,

Armin.


From arigo at tunes.org  Thu Jun 28 13:51:56 2007
From: arigo at tunes.org (Armin Rigo)
Date: Thu, 28 Jun 2007 13:51:56 +0200
Subject: [pypy-dev] Fwd:  Work plan for PyPy
In-Reply-To: <1183010847.19411.31.camel@asl.dorms.spbu.ru>
References: <30D05AC9-1A9E-4E90-9E79-C587AA81978C@ukshells.co.uk>
	<B4432A41-4991-46A7-A9DA-4E2F0363F347.SS2830SS@ukshells.co.uk>
	<1183010847.19411.31.camel@asl.dorms.spbu.ru>
Message-ID: <20070628115156.GB27708@code0.codespeak.net>

Hi Anton,

On Thu, Jun 28, 2007 at 10:07:27AM +0400, Anton Korobeynikov wrote:
> 1. Use more or less portable runtime, which will "unify" such calls. For
> example, one can use APR (HLVM, in fact, will use it)
> 
> 2. Delegate stubs resolution to specialised version of lli. I think,
> LLVM's itself OS abstraction library (libSystem) can be partly used.

Unless I'm missing something, it seems that neither of these approaches
would work to call generic C APIs - only the ones that are re-exported
from either the APR or libSystem.  Doesn't sound too attractive from my
point of view.


A bientot,

Armin.


From arigo at tunes.org  Thu Jun 28 14:04:44 2007
From: arigo at tunes.org (Armin Rigo)
Date: Thu, 28 Jun 2007 14:04:44 +0200
Subject: [pypy-dev] Work plan for PyPy
In-Reply-To: <20070627190715.3b35c12f.simon@arrowtheory.com>
References: <20070614164554.GA654@code0.codespeak.net>
	<20070615174454.6e87fb1a.simon@arrowtheory.com>
	<4673C374.2060806@genesilico.pl>
	<20070627190715.3b35c12f.simon@arrowtheory.com>
Message-ID: <20070628120444.GA400@code0.codespeak.net>

Hi Simon,

On Wed, Jun 27, 2007 at 07:07:15PM -0700, Simon Burton wrote:
> Do you think we could extend rffi to be able to
> expose function calls to outside callers, so that eg.
> we can make nifty cpython extension modules ?

It would be a simple matter to have a way to force genc to use specific
names from functions and struct names so that they can be called from
hand-written C code too.  Is that what you mean?


A bientot,

Armin.


From cfbolz at gmx.de  Thu Jun 28 16:46:19 2007
From: cfbolz at gmx.de (Carl Friedrich Bolz)
Date: Thu, 28 Jun 2007 16:46:19 +0200
Subject: [pypy-dev] [pypy-svn] r44599 - in pypy/dist/pypy/lang/scheme: .
	test
In-Reply-To: <20070628134930.7F1688101@code0.codespeak.net>
References: <20070628134930.7F1688101@code0.codespeak.net>
Message-ID: <348899050706280746i26bdbcbfif7da2c423c792a12@mail.gmail.com>

Hi Jacub!

2007/6/28, jlg at codespeak.net <jlg at codespeak.net>:
> Author: jlg
> Date: Thu Jun 28 15:49:30 2007
> New Revision: 44599
>
> Modified:
>    pypy/dist/pypy/lang/scheme/TODO.txt
>    pypy/dist/pypy/lang/scheme/object.py
>    pypy/dist/pypy/lang/scheme/test/test_parser.py
> Log:
> RPython direction for object.py
>
> Modified: pypy/dist/pypy/lang/scheme/object.py
> ==============================================================================
> --- pypy/dist/pypy/lang/scheme/object.py        (original)
> +++ pypy/dist/pypy/lang/scheme/object.py        Thu Jun 28 15:49:30 2007
> @@ -139,10 +139,16 @@
>          raise NotImplementedError
>
>  def add_lst(ctx, lst):
> -    return apply_lst(ctx, lambda x, y: x + y, lst)
> +    def adder(x, y):
> +        return x + y
> +
> +    return apply_lst(ctx, adder, lst)
>
>  def mul_lst(ctx, lst):
> -    return apply_lst(ctx, lambda x, y: x * y, lst)
> +    def multiplier(x, y):
> +        return x * y
> +
> +    return apply_lst(ctx, multiplier, lst)
>
>  def apply_lst(ctx, fun, lst):
>      acc = None
>

This doesn't make mul_lst and add_lst any more rpython than before. In
fact, lambda is fully rpython. The problem with this code is rather
that you cannot have local functions in RPython.

As I said, don't worry about making your code RPython too much yet.

Cheers,

Carl Friedrich


From jgustak at gmail.com  Thu Jun 28 17:11:40 2007
From: jgustak at gmail.com (Jakub Gustak)
Date: Thu, 28 Jun 2007 17:11:40 +0200
Subject: [pypy-dev] [pypy-svn] r44599 - in pypy/dist/pypy/lang/scheme: .
	test
In-Reply-To: <348899050706280746i26bdbcbfif7da2c423c792a12@mail.gmail.com>
References: <20070628134930.7F1688101@code0.codespeak.net>
	<348899050706280746i26bdbcbfif7da2c423c792a12@mail.gmail.com>
Message-ID: <a26b53540706280811l3f5a06e4w94992826453627f3@mail.gmail.com>

On 6/28/07, Carl Friedrich Bolz <cfbolz at gmx.de> wrote:
> Hi Jacub!
>
> 2007/6/28, jlg at codespeak.net <jlg at codespeak.net>:
> > Author: jlg
> > Date: Thu Jun 28 15:49:30 2007
> > New Revision: 44599
> >
> > Modified:
> >    pypy/dist/pypy/lang/scheme/TODO.txt
> >    pypy/dist/pypy/lang/scheme/object.py
> >    pypy/dist/pypy/lang/scheme/test/test_parser.py
> > Log:
> > RPython direction for object.py
> >
> > Modified: pypy/dist/pypy/lang/scheme/object.py
> > ==============================================================================
> > --- pypy/dist/pypy/lang/scheme/object.py        (original)
> > +++ pypy/dist/pypy/lang/scheme/object.py        Thu Jun 28 15:49:30 2007
> > @@ -139,10 +139,16 @@
> >          raise NotImplementedError
> >
> >  def add_lst(ctx, lst):
> > -    return apply_lst(ctx, lambda x, y: x + y, lst)
> > +    def adder(x, y):
> > +        return x + y
> > +
> > +    return apply_lst(ctx, adder, lst)
> >
> >  def mul_lst(ctx, lst):
> > -    return apply_lst(ctx, lambda x, y: x * y, lst)
> > +    def multiplier(x, y):
> > +        return x * y
> > +
> > +    return apply_lst(ctx, multiplier, lst)
> >
> >  def apply_lst(ctx, fun, lst):
> >      acc = None
> >
>
> This doesn't make mul_lst and add_lst any more rpython than before. In
> fact, lambda is fully rpython. The problem with this code is rather
> that you cannot have local functions in RPython.

I that case error messages from pylint:
E:142:add_lst.<lambda>: Using unavailable keyword 'lambda'
E:145:mul_lst.<lambda>: Using unavailable keyword 'lambda'

Causes only confusion.

> As I said, don't worry about making your code RPython too much yet.

I will try to refrain.

Cheers,
Jakub Gustak


From anto.cuni at gmail.com  Thu Jun 28 17:26:32 2007
From: anto.cuni at gmail.com (Antonio Cuni)
Date: Thu, 28 Jun 2007 17:26:32 +0200
Subject: [pypy-dev] [pypy-svn] r44599 - in pypy/dist/pypy/lang/scheme:
 .	test
In-Reply-To: <a26b53540706280811l3f5a06e4w94992826453627f3@mail.gmail.com>
References: <20070628134930.7F1688101@code0.codespeak.net>	<348899050706280746i26bdbcbfif7da2c423c792a12@mail.gmail.com>
	<a26b53540706280811l3f5a06e4w94992826453627f3@mail.gmail.com>
Message-ID: <4683D328.6030206@gmail.com>

Jakub Gustak wrote:
> I that case error messages from pylint:
> E:142:add_lst.<lambda>: Using unavailable keyword 'lambda'
> E:145:mul_lst.<lambda>: Using unavailable keyword 'lambda'

I would say that in this case pylint is wrong, though I agree than in 
most cases lambda is used in a way that is not allowed in RPython.

ciao Anto


From simon at arrowtheory.com  Thu Jun 28 21:20:45 2007
From: simon at arrowtheory.com (Simon Burton)
Date: Thu, 28 Jun 2007 12:20:45 -0700
Subject: [pypy-dev] ep 2007 talks
In-Reply-To: <46837966.7070400@genesilico.pl>
References: <20070626114026.3fb5760e.simon@arrowtheory.com>
	<468251B1.9020100@gmail.com> <46837966.7070400@genesilico.pl>
Message-ID: <20070628122045.caa702c9.simon@arrowtheory.com>

On Thu, 28 Jun 2007 11:03:34 +0200
Maciek Fijalkowski <fijal at genesilico.pl> wrote:

> 
> > Do you think is enough or you was thinking about a deeper introduction?
> >   
> I would extremely enjoy it, hence you give us some reason why we're 
> talking about this at all :) There are a bit of orthogonal issues in our 
> rpython talk (js backend and so on), but mostly it would be good to 
> share a bit. When do you arrive to Vilnius? (I'm arriving on 5th and I 
> might have no network at all till 5th).

I will arrive on the 8th, in the afternoon.

Simon.


From rxe at ukshells.co.uk  Fri Jun 29 01:40:37 2007
From: rxe at ukshells.co.uk (Richard Emslie)
Date: Thu, 28 Jun 2007 16:40:37 -0700
Subject: [pypy-dev] Work plan for PyPy
In-Reply-To: <20070628111538.GA27708@code0.codespeak.net>
References: <30D05AC9-1A9E-4E90-9E79-C587AA81978C@ukshells.co.uk>
	<B4432A41-4991-46A7-A9DA-4E2F0363F347@ukshells.co.uk>
	<20070628111538.GA27708@code0.codespeak.net>
Message-ID: <6AC2ADE5-7BA3-4C6E-BF40-8952284316BD@ukshells.co.uk>

Hi Armin,

On Jun 28, 2007, at 4:15 AM, Armin Rigo wrote:

> Hi Richard,
>
> On Wed, Jun 27, 2007 at 01:55:13PM -0700, Richard Emslie wrote:
>>>> bytecode that genllvm produces would then call the helper functions
>>>> instead of calling directly the external C API.  Shouldn't be a
>>>> performance problem as llvm will inline the helpers agressively.
>>>>
>>>
>>> ok, that is somewhat what we have now - except the stub functions
>>> are high level and shared *somewhat* with genc.
>
> Precisely, by contrast I'm thinking about having lower level stub
> functions that are generated automatically from the lltype  
> declarations.
> For example, an external ll function ptr would generate a simple stub,
> always the same.  GenC and GenLLVM would share the code to produce the
> stubs.  Similarly, getting at an external variable (or variable-like
> macro) like errno should be expressable in an lltype way (it's not  
> right
> now) that gets turned into """int _rpyget_errno(void) { return  
> errno; }""".
>

Ok - cool.  I was hoping to propose something along these lines - but  
was unsure how to go about it.
This would be ideal and work great for backends.  I am guessing all  
memory management with this would
be explicitly controlled from outside the stubs?


>>> Generating c code in llvm backend would sort of defeat the point of
>>> it IMHO.
>
> The difference is that the stubs would always be generated from simple
> templates, not from genc's complex logic that turns flow graphs into C
> code.
>

Yeah - this sounds fantastic.  No clue how to do it - but great! :-)

Well - I'm in the process of migrating back home... again.  Once i  
get back though I can devote lots of time to this.
I doubt i can make the sprint, but will try.

Cheers,
Richard




From jgustak at gmail.com  Fri Jun 29 01:40:52 2007
From: jgustak at gmail.com (Jakub Gustak)
Date: Fri, 29 Jun 2007 01:40:52 +0200
Subject: [pypy-dev] scheme interpreter [status report]
Message-ID: <a26b53540706281640o7b13a794n26d9d6fc5cb3b8dd@mail.gmail.com>

Hello.

Scheme interpreter is starting look like its going to do some serious
stuff in the near future, or maybe I am just too enthusiastic about
it.

What we have now:
- We can execute simple expressions like: (+ 1 2 1.3 -2)
  Nested ones also works.
- (define <var> <value>) works with a given context, so:
  (define var 2)
  (+ var 40) => 42
- Conditional (if <cond> <then> <else>) seem to work fine, <else> is
not arbitrary.

pypy/lang/scheme/test/test_eval.py
Gives you the best overview.

What will be done in the nearest future:
- comparison primitives: = < > eq? eqv?
- (cons <car> <cdr>) (car <pair>) (cdr <pair>) so we can play with lists.
That kind of stuff.

You can take a look at:
pypy/lang/scheme/TODO.txt

Will postpone quotations and symbols implementation for a while. I
have to think about it, how much of it is parsing matter.

Then i would like to work on (lambda ...). This will probably take
some time to think again execution context, make a hard distinction
between procedures and macros and a lot of stuff, that I don't even
dare to think about now :-).

Before EP I would like to make lambdas work and also have some tuning
done (whatever it means), so on the sprint we could focus on really
interesting stuff instead of refactoring and bug fixes.

I would like to refine what I would like to do during the sprint, but
not sure right now. Any ideas, what would be worth trying then?

Cheers,
Jakub Gustak


From rxe at ukshells.co.uk  Fri Jun 29 01:53:41 2007
From: rxe at ukshells.co.uk (Richard Emslie)
Date: Thu, 28 Jun 2007 16:53:41 -0700
Subject: [pypy-dev] Work plan for PyPy
In-Reply-To: <1183010847.19411.31.camel@asl.dorms.spbu.ru>
References: <30D05AC9-1A9E-4E90-9E79-C587AA81978C@ukshells.co.uk>
	<B4432A41-4991-46A7-A9DA-4E2F0363F347.SS2830SS@ukshells.co.uk>
	<1183010847.19411.31.camel@asl.dorms.spbu.ru>
Message-ID: <DFA137F5-D4A8-4ED8-A5A7-E29A8DDAA8F6@ukshells.co.uk>

Hi Anton,

On Jun 27, 2007, at 11:07 PM, Anton Korobeynikov wrote:

> Hello, Richard.
>
>> Generating c code in llvm backend would sort of defeat the point of
>> it IMHO.  We already have a backend that generates c.  On the other
>> hand, if we hand write the c code - it may be an neverending task
>> of adding hacks to the backend.  For instance just getting at errno
>> is particularly problematic right now - and would involve a
>> function call from a constant value.  which means injecting a few
>> lines assembly and a temporary variable.  Easy to do, but it is
>> nice right now that we pretty much have a direct mapping from our
>> our lli form to llvm assembly.

       ^^^ ahem, "ssi form"

> I've talked a little bit with Chris Lattner about this (calling C API)
> issue. Except stubs/wrappers it seems there are ways to call C API
>
> 1. Use more or less portable runtime, which will "unify" such  
> calls. For
> example, one can use APR (HLVM, in fact, will use it)
>
> 2. Delegate stubs resolution to specialised version of lli. I think,
> LLVM's itself OS abstraction library (libSystem) can be partly used.
>

thanks for these ideas.  i guess Armin has replied already.  I am  
guess the biggest concern is how to wrap libraries outside what these  
support.


> I personally like the second variant more. Anyway, some specialised  
> lli
> variant (or some application, which will call LLVM's execution engine)
> will be needed, if pypy would want some extended features (like  
> accurate
> GC, vectorisation, etc) from LLVM in the future.
>
>

We never managed to run pypy under lli.  In was one of the early  
goals, but it were waiting until gc to mature in the pypy  
infrastructure.  Since that has happened, the llvm backend hasn't had  
much active development.   Right now it only works with boehm, and  
not surprisingly the interaction of llvm's jit and boehm doesn't go  
very well in hand.
I still think it would be fun and interesting to see pypy-llvm  
working under lli.

Cheers,
Ricahrd



From cfbolz at gmx.de  Fri Jun 29 12:52:54 2007
From: cfbolz at gmx.de (Carl Friedrich Bolz)
Date: Fri, 29 Jun 2007 12:52:54 +0200
Subject: [pypy-dev] scheme interpreter [status report]
In-Reply-To: <a26b53540706281640o7b13a794n26d9d6fc5cb3b8dd@mail.gmail.com>
References: <a26b53540706281640o7b13a794n26d9d6fc5cb3b8dd@mail.gmail.com>
Message-ID: <348899050706290352l113a4b96y5cb2672f45c84842@mail.gmail.com>

Hi Jakub (sorry for mis-spelling your name last time)!

2007/6/29, Jakub Gustak <jgustak at gmail.com>:
> Scheme interpreter is starting look like its going to do some serious
> stuff in the near future, or maybe I am just too enthusiastic about
> it.
>
> What we have now:
> - We can execute simple expressions like: (+ 1 2 1.3 -2)
>   Nested ones also works.
> - (define <var> <value>) works with a given context, so:
>   (define var 2)
>   (+ var 40) => 42
> - Conditional (if <cond> <then> <else>) seem to work fine, <else> is
> not arbitrary.

All sounds good!


> pypy/lang/scheme/test/test_eval.py
> Gives you the best overview.
>
> What will be done in the nearest future:
> - comparison primitives: = < > eq? eqv?
> - (cons <car> <cdr>) (car <pair>) (cdr <pair>) so we can play with lists.
> That kind of stuff.

I would do cons, car, cdr first (they should be easy anyway).

> You can take a look at:
> pypy/lang/scheme/TODO.txt
>
> Will postpone quotations and symbols implementation for a while. I
> have to think about it, how much of it is parsing matter.

A lof of it is a matter of parsing, yes. I can help you there, if you
want me to.

> Then i would like to work on (lambda ...). This will probably take
> some time to think again execution context, make a hard distinction
> between procedures and macros and a lot of stuff, that I don't even
> dare to think about now :-).

Simple lambdas should be kind of easy, I would hope.


> Before EP I would like to make lambdas work and also have some tuning
> done (whatever it means), so on the sprint we could focus on really
> interesting stuff instead of refactoring and bug fixes.
>
> I would like to refine what I would like to do during the sprint, but
> not sure right now. Any ideas, what would be worth trying then?

Good things to do on the sprint are things were you need the help of
other PyPy people. Two ideas:

 - thinking about how to implement call/cc with the primitives
stackless provides
 - making the interpreter rpython

Cheers,

Carl Friedrich


From fijal at genesilico.pl  Sat Jun 30 13:12:57 2007
From: fijal at genesilico.pl (Maciek Fijalkowski)
Date: Sat, 30 Jun 2007 13:12:57 +0200
Subject: [pypy-dev] Fwd:  Work plan for PyPy
In-Reply-To: <20070628111538.GA27708@code0.codespeak.net>
References: <30D05AC9-1A9E-4E90-9E79-C587AA81978C@ukshells.co.uk>
	<B4432A41-4991-46A7-A9DA-4E2F0363F347@ukshells.co.uk>
	<20070628111538.GA27708@code0.codespeak.net>
Message-ID: <46863AB9.1030102@genesilico.pl>

Armin Rigo wrote:
> Hi Richard,
>
> On Wed, Jun 27, 2007 at 01:55:13PM -0700, Richard Emslie wrote:
>   
>>>> bytecode that genllvm produces would then call the helper functions
>>>> instead of calling directly the external C API.  Shouldn't be a
>>>> performance problem as llvm will inline the helpers agressively.
>>>>
>>>>         
>>> ok, that is somewhat what we have now - except the stub functions  
>>> are high level and shared *somewhat* with genc.
>>>       
>
> Precisely, by contrast I'm thinking about having lower level stub
> functions that are generated automatically from the lltype declarations.
> For example, an external ll function ptr would generate a simple stub,
> always the same.  GenC and GenLLVM would share the code to produce the
> stubs.  Similarly, getting at an external variable (or variable-like
> macro) like errno should be expressable in an lltype way (it's not right
> now) that gets turned into """int _rpyget_errno(void) { return errno; }""".
>   
Just a little correction - you can express c-level constant in lltype. 
There is CConstant class for that.

Cheers,
fijal



From fijal at genesilico.pl  Sat Jun 30 13:21:26 2007
From: fijal at genesilico.pl (Maciek Fijalkowski)
Date: Sat, 30 Jun 2007 13:21:26 +0200
Subject: [pypy-dev] [pypy-svn] r44599 - in pypy/dist/pypy/lang/scheme:
 .	test
In-Reply-To: <a26b53540706280811l3f5a06e4w94992826453627f3@mail.gmail.com>
References: <20070628134930.7F1688101@code0.codespeak.net>	<348899050706280746i26bdbcbfif7da2c423c792a12@mail.gmail.com>
	<a26b53540706280811l3f5a06e4w94992826453627f3@mail.gmail.com>
Message-ID: <46863CB6.3010403@genesilico.pl>

Jakub Gustak wrote:
> I that case error messages from pylint:
> E:142:add_lst.<lambda>: Using unavailable keyword 'lambda'
> E:145:mul_lst.<lambda>: Using unavailable keyword 'lambda'
>
> Causes only confusion.
>
>   
I would not trust pylint too much. being rpython is quite complex issue. 
and trying to be rpython-compatible in pylint sense makes little sense 
to me (ie pylint will complain about all the surrounding initialization 
code, which is perfectly fine and even much more elegant than writing it 
all by hand). Making scheme more-rpythonish sounds like a nice task for 
a sprint.

Cheers,
fijal



From arigo at tunes.org  Sat Jun 30 13:49:21 2007
From: arigo at tunes.org (Armin Rigo)
Date: Sat, 30 Jun 2007 13:49:21 +0200
Subject: [pypy-dev] Fwd:  Work plan for PyPy
In-Reply-To: <46863AB9.1030102@genesilico.pl>
References: <30D05AC9-1A9E-4E90-9E79-C587AA81978C@ukshells.co.uk>
	<B4432A41-4991-46A7-A9DA-4E2F0363F347@ukshells.co.uk>
	<20070628111538.GA27708@code0.codespeak.net>
	<46863AB9.1030102@genesilico.pl>
Message-ID: <20070630114921.GC1568@code0.codespeak.net>

Hi Maciek,

On Sat, Jun 30, 2007 at 01:12:57PM +0200, Maciek Fijalkowski wrote:
> Just a little correction - you can express c-level constant in lltype. 
> There is CConstant class for that.

True, although I'm not sure it works for non-integer constants (e.g.
static data).  Also, I suppose that it could be abused to represent
"errno" but it looks fragile.


A bientot,

Armin.


From fijal at genesilico.pl  Sat Jun 30 13:56:03 2007
From: fijal at genesilico.pl (Maciek Fijalkowski)
Date: Sat, 30 Jun 2007 13:56:03 +0200
Subject: [pypy-dev] Fwd:  Work plan for PyPy
In-Reply-To: <20070630114921.GC1568@code0.codespeak.net>
References: <30D05AC9-1A9E-4E90-9E79-C587AA81978C@ukshells.co.uk>
	<B4432A41-4991-46A7-A9DA-4E2F0363F347@ukshells.co.uk>
	<20070628111538.GA27708@code0.codespeak.net>
	<46863AB9.1030102@genesilico.pl>
	<20070630114921.GC1568@code0.codespeak.net>
Message-ID: <468644D3.9040908@genesilico.pl>

Armin Rigo wrote:
> Hi Maciek,
>
> On Sat, Jun 30, 2007 at 01:12:57PM +0200, Maciek Fijalkowski wrote:
>   
>> Just a little correction - you can express c-level constant in lltype. 
>> There is CConstant class for that.
>>     
>
> True, although I'm not sure it works for non-integer constants (e.g.
> static data).  Also, I suppose that it could be abused to represent
> "errno" but it looks fragile.
>   
?
The primary reason for creating this class was to use it for errno, so I 
don't think it's an abuse. Anyway, I think it should be extended to 
present some non-primitive types (it works for any pre-defined primitive 
type right now).

Cheers,
fijal