From sanxiyn at gmail.com  Fri Apr  1 08:56:34 2005
From: sanxiyn at gmail.com (Sanghyeon Seo)
Date: Fri, 1 Apr 2005 15:56:34 +0900
Subject: [pypy-dev] help(1) OverflowError
Message-ID: <5b02481705033122566182827c@mail.gmail.com>

With the current PyPy HEAD, help(1) results in OverflowError, after a long time.
I just wanted to test whether pydoc works, but I didn't expect OverflowError...

Traceback (application-level):
  File "<inline>help(1)", line 1 in ?
  File "/home/tinuviel/pypy/dist/lib-python-2.3.4/pydoc.py", line 47 in ?
import sys, imp, os, re, types, inspect, __builtin__
  File "/home/tinuviel/pypy/dist/lib-python-2.3.4/inspect.py", line 31 in ?
import sys, os, types, string, re, dis, imp, tokenize, linecache
  File "/home/tinuviel/pypy/dist/lib-python-2.3.4/tokenize.py", line 100 in ?
tokenprog, pseudoprog, single3prog, double3prog = map(
  File "/home/tinuviel/pypy/dist/lib-python-2.3.4/sre.py", line 179 in compile
    return _compile(pattern, flags)
  File "/home/tinuviel/pypy/dist/lib-python-2.3.4/sre.py", line 228 in _compile
        p = sre_compile.compile(pattern, flags)
  File "/home/tinuviel/pypy/dist/lib-python-2.3.4/sre_compile.py",
line 488 in compile
        groupindex, indexgroup
OverflowError: long int too large to convert to int

Any idea?


From arigo at tunes.org  Fri Apr  1 11:20:15 2005
From: arigo at tunes.org (Armin Rigo)
Date: Fri, 1 Apr 2005 10:20:15 +0100
Subject: [pypy-dev] Next step: gen???.py
In-Reply-To: <424C2C40.6050304@stackless.com>
References: <20050330172550.GA11292@vicky.ecs.soton.ac.uk>
	<20050330174843.GL23818@solar.trillke.net>
	<20050331082929.GB4767@vicky.ecs.soton.ac.uk>
	<424C2C40.6050304@stackless.com>
Message-ID: <20050401092015.GA27019@vicky.ecs.soton.ac.uk>

Hi Christian,

On Thu, Mar 31, 2005 at 06:58:40PM +0200, Christian Tismer wrote:
> I always wondered btw., why you have the annotation
> data elsewhere. I would have augmented it to the block structure.
> Changing the flow graph to contain the info is even more rude, but
> very nice!

The annotator's own flow-and-reflow nature makes it best to dissociate the
data it (mis-)handles from the immutable inputs it gets, but you're right: the
typer.py could probably just stick type information on the Variable and
Constant objects of the flow graph.  This probably makes even more sense given
that there is no sane other way to put type information on constants (e.g.
this "1" here is an int but that "1" over there is a PyObject).


Armin


From arigo at tunes.org  Fri Apr  1 11:48:59 2005
From: arigo at tunes.org (Armin Rigo)
Date: Fri, 1 Apr 2005 10:48:59 +0100
Subject: [pypy-dev] help(1) OverflowError
In-Reply-To: <5b02481705033122566182827c@mail.gmail.com>
References: <5b02481705033122566182827c@mail.gmail.com>
Message-ID: <20050401094859.GA17901@vicky.ecs.soton.ac.uk>

Hi Seo,

On Fri, Apr 01, 2005 at 03:56:34PM +0900, Sanghyeon Seo wrote:
>   File "/home/tinuviel/pypy/dist/lib-python-2.3.4/sre_compile.py",
> line 488 in compile
>         groupindex, indexgroup
> OverflowError: long int too large to convert to int

Actually, if you have a very fast machine, debugging with pdb in py.py works
fine.  It's related to Michael Chermside's module/_sre_pypy/ check-in.  The
bug is that _sre.compile() should accept as 3rd argument a list of unsigned
integers, but in the current implementation expects a list of signed integers
-- hence the OverflowError.

Help!  What is the proper way to unwrap an unsigned integer?


Armin


From guenter.jantzen at netcologne.de  Fri Apr  1 23:13:26 2005
From: guenter.jantzen at netcologne.de (=?ISO-8859-1?Q?G=FCnter_Jantzen?=)
Date: Fri, 01 Apr 2005 23:13:26 +0200
Subject: [pypy-dev] Next step: gen???.py
In-Reply-To: <424BE80F.2040102@strakt.com>
References: <20050330172550.GA11292@vicky.ecs.soton.ac.uk>
	<424BE80F.2040102@strakt.com>
Message-ID: <424DB976.2020604@netcologne.de>

Hello Samuele,

C# is very similar to Java and supports 'goto'.
Could this be an option?

G?nter

Samuele Pedroni schrieb:
> Armin Rigo wrote:
> 
>> * genjava.py could be another option.  It has a simpler type system, 
>> which
>> matches ours quite well, but genjava doesn't exist yet at all (the one 
>> in the
>> java/ subdirectory had a different goal in mind).  We get memory 
>> management
>> for free.  If we add the requirement to compile with GCJ it could be 
>> easy to
>> make a CPython extension module too, with the same problems and solutions
>> about SomeObject as above.
>>
>>  
>>
> We would get gc, thread support, a runtime and useful libraries 
> (unicode, big integers ...), and an object model for free. How much of 
> that and with what stability if we go through gcj is a bit open, 
> although for the target of self-hosting that would be the interesting 
> route. Java has no gotos wich means that at some point we would have to 
> generate bytecode wich is not too
> hard but sometimes making the java verifier happy is harder than it 
> could seem. The type system should match but there are no pointer to 
> functions or delegates wich means some more involved solution to emulate 
> them. We could probably reuse some things or ideas that are in Jython.
> 
> Whether the things we would get for free and the type system and basic 
> object model are really a good match for the code we can easily generate 
> is also a open question.
> 
> So it probably makes sense as a platform to try long term, and surely 
> when we have made even more progress in annotating our codebase. Also 
> because reusing java or jython stuff instead of trying to interface with 
> CPython is probably easier because of ref-counting vs gc issues.
> 
> But is worth keeping it in mind as a reserve route, because its 
> trade-offs come together with quite a bit of high level functionality 
> already there.
> 
> But I agree that genc especially with an approach involving 
> incrementally rewriting the graphs is the most natural route right now.
> 
> Samuele.
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev
> 



From arigo at tunes.org  Sat Apr  2 22:41:30 2005
From: arigo at tunes.org (Armin Rigo)
Date: Sat, 2 Apr 2005 21:41:30 +0100
Subject: [pypy-dev] Next step: gen???.py
In-Reply-To: <424DB976.2020604@netcologne.de>
References: <20050330172550.GA11292@vicky.ecs.soton.ac.uk>
	<424BE80F.2040102@strakt.com> <424DB976.2020604@netcologne.de>
Message-ID: <20050402204130.GA1267@vicky.ecs.soton.ac.uk>

Hi G?nter,

On Fri, Apr 01, 2005 at 11:13:26PM +0200, G?nter Jantzen wrote:
> C# is very similar to Java and supports 'goto'.
> Could this be an option?

Yes, it is definitely an option, and it seems likely that C# and/or .NET are
very good medium- or long-term target -- but we have too many options for now
:-)

For the short-term we have settled over gradually making genc.py more
type-aware.


Armin


From hpk at trillke.net  Sun Apr  3 13:03:54 2005
From: hpk at trillke.net (holger krekel)
Date: Sun, 3 Apr 2005 13:03:54 +0200
Subject: [pypy-dev] help(1) OverflowError
In-Reply-To: <20050401094859.GA17901@vicky.ecs.soton.ac.uk>
References: <20050401094859.GA17901@vicky.ecs.soton.ac.uk>
Message-ID: <20050403110354.GM23818@solar.trillke.net>

On Fri, Apr 01, 2005 at 10:48 +0100, Armin Rigo wrote:
> On Fri, Apr 01, 2005 at 03:56:34PM +0900, Sanghyeon Seo wrote:
> >   File "/home/tinuviel/pypy/dist/lib-python-2.3.4/sre_compile.py",
> > line 488 in compile
> >         groupindex, indexgroup
> > OverflowError: long int too large to convert to int
> 
> Actually, if you have a very fast machine, debugging with pdb in py.py works
> fine.  It's related to Michael Chermside's module/_sre_pypy/ check-in.  The
> bug is that _sre.compile() should accept as 3rd argument a list of unsigned
> integers, but in the current implementation expects a list of signed integers
> -- hence the OverflowError.
> 
> Help!  What is the proper way to unwrap an unsigned integer?

Hum, nobody seems to be able to answer this at the moment. 
It probably relates to the fact that our usage of integers at 
interpreter-level in general is fuzzy.  Or do we have a concise 
explicit definition for their usage? 

Anyway, apparently Samuele and me re-patched the trunk to not use 
Michael Chermside's _sre_pypy implementation for the moment because 
of this issue.  You can get back the previous behaviour (with the 
help(1) problem) by uncommenting in baseobjspace.py the 
line starting with ... self.setbuiltinmodule("_sre"
at least on amd64 (which fails differently from a 32bit
machine, sigh). 

    holger


From lutz_p at gmx.net  Mon Apr  4 23:09:40 2005
From: lutz_p at gmx.net (Lutz Paelike)
Date: Mon, 04 Apr 2005 23:09:40 +0200
Subject: [pypy-dev] Next step: gen???.py
In-Reply-To: <20050403110354.GM23818@solar.trillke.net>
References: <20050401094859.GA17901@vicky.ecs.soton.ac.uk>
	<20050403110354.GM23818@solar.trillke.net>
Message-ID: <4251AD14.3030202@gmx.net>

Armin Rigo wrote:

 > That's an excellent point.
 >
 > I played with a more promizing approach in
 > http://codespeak.net/svn/pypy/dist/pypy/translator/typer.py .  Basically,
 > instead of enhancing genc to support all kind of typed operations and implicit
 > conversions (or rely on C++ to select the operations and insert these
 > conversions automatically), the abvoe module contains code that modifies the
 > flow graph itself to turn it into a "low-level" flow graph.  The idea was
 > already floating around here.  In short it turns code like



Hi,

i just want to pop in the discussion and bring up my idea on flowgraph modeling again.
I know i was'nt able to convince most of you at the PyCon but i still think this is a powerful
way to describe a problem. Maybe i can convince you in the future ;)

"..that modifies the flow graph itself to turn it into a "low-level" flow graph."

This is exactly what i want to create, a visual flowgraph construction tool that enables the user to connect
abstract nodes and to model an abstract (decision, data flow,whatever ..) tree. The idea is that your building blocks 
are abstract enough to be useful but atomic enough to model every possible problem. From this model i could easily 
generate (R)Python code and feed it into PyPy, so for the system it should make absolutely no difference if it handles 
generated or hand-typed RPython code. This user-view flowgraph can then be later translated to a system-view ( or 
"low-level") flowgraph. The difference is that the user is not modeling *the* flowgraph but *a* flowgraph.

I find it even more attractive to make the generation of code just an option and to directly interface with PyPy.
So opposed to the idea of having multiple code output modules (which is great) i propose an additional input module 
which is visually constructed in an editor. Since i can take the route through generated RPython code into the system 
there should'nt be compatibility issues.

To make my idea more clear i would like to point you to a video tutorial showing the 3d tool Houdini in action.
I want to put much of the functionality of Houdini into the visual editor i am thinking of so maybe you find it 
interesting. The candle component example illustrates a whole development cycle to develop a reusable component with
user controls..
http://sidefx.vislab.usyd.edu.au/houdini_video/by_topic/general/index.html

(there are more on the net..)
If your interest goes event further you may want to download a free demo of Houdini (Linux & Windows) and play around 
with it..
It's here:
http://sidefx.vislab.usyd.edu.au


I know that this thing is only loosely related to PyPy itself and is more a generic editor, which is my intention.
In the first place i do 3d graphics so i can not put much very much time in the project but i am very interested in it 
and maybe i can bring in a different point of view  into the discussion ;)


Cheers,

Lutz



From sanxiyn at gmail.com  Tue Apr  5 06:05:29 2005
From: sanxiyn at gmail.com (Sanghyeon Seo)
Date: Tue, 5 Apr 2005 13:05:29 +0900
Subject: [pypy-dev] py.py py.py
Message-ID: <5b02481705040421052e3ffd81@mail.gmail.com>

With revision 10311, py.py py.py runs again, but fails to print 1.

As it takes more than 100 minutes to start up in my 23400 pystones machine,
I am not patient enough to debug this...

tinuviel at debian:~/pypy/dist/pypy/interpreter$ ./py.py py.py
faking <type 'unicode'>
faking <type 'module'>
faking <type 'file'>
faking <type 'posix.stat_result'>
faking <type 'posix.statvfs_result'>
faking <type '_sre.SRE_Pattern'>
faking <type 'function'>
faking <pypy type 'unicode'>
faking <pypy type 'module'>
PyPy in StdObjSpace on top of Python 2.3.4 (startupttime: 6103.75 secs)
>>>>>1
faking <pypy type 'file'>
Traceback (most recent call last):
  File "py.py", line 90, in main_
    con.interact(banner)
  File "/home/tinuviel/pypy/dist/pypy/interpreter/interactive.py",
line 137, in interact
    code.InteractiveConsole.interact(self, banner)
  File "/home/tinuviel/pypy/dist/lib-python-2.3.4/code.py", line 239,
in interact
    more = self.push(line)
  File "/home/tinuviel/pypy/dist/lib-python-2.3.4/code.py", line 261, in push
    more = self.runsource(source, self.filename)
  File "/home/tinuviel/pypy/dist/pypy/interpreter/interactive.py",
line 221, in runsource
    self.runcode(code)
  File "/home/tinuviel/pypy/dist/pypy/interpreter/interactive.py",
line 169, in runcode
    pycode.exec_code(self.space, self.w_globals, self.w_globals)
  File "/home/tinuviel/pypy/dist/pypy/interpreter/eval.py", line 24,
in exec_code
    return frame.run()
  File "/home/tinuviel/pypy/dist/pypy/interpreter/eval.py", line 70, in resume
    result = self.eval(executioncontext)
  File "/home/tinuviel/pypy/dist/pypy/interpreter/pyframe.py", line 97, in eval
    self.dispatch()
  File "/home/tinuviel/pypy/dist/pypy/interpreter/pyopcode.py", line
57, in dispatch
    fn(self)
  File "/home/tinuviel/pypy/dist/pypy/interpreter/pyopcode.py", line
283, in PRINT_EXPR
    print_expr(f.space, w_expr)
  File "/home/tinuviel/pypy/dist/pypy/interpreter/gateway.py", line
528, in appcaller
    return space.call_args(w_func, args)
  File "/home/tinuviel/pypy/dist/pypy/objspace/descroperation.py",
line 88, in call_args
    return w_obj.call_args(args)
  File "/home/tinuviel/pypy/dist/pypy/interpreter/function.py", line
40, in call_args
    return frame.run()
  File "/home/tinuviel/pypy/dist/pypy/interpreter/eval.py", line 70, in resume
    result = self.eval(executioncontext)
  File "/home/tinuviel/pypy/dist/pypy/interpreter/pyframe.py", line 97, in eval
    self.dispatch()
  File "/home/tinuviel/pypy/dist/pypy/interpreter/pyopcode.py", line
54, in dispatch
    fn(self, oparg)
  File "/home/tinuviel/pypy/dist/pypy/interpreter/pyopcode.py", line
642, in CALL_FUNCTION
    w_result = f.space.call_args(w_function, args)
  File "/home/tinuviel/pypy/dist/pypy/objspace/descroperation.py",
line 88, in call_args
    return w_obj.call_args(args)
  File "/home/tinuviel/pypy/dist/pypy/interpreter/function.py", line
40, in call_args
    return frame.run()
  File "/home/tinuviel/pypy/dist/pypy/interpreter/gateway.py", line 231, in run
    w_result = self._run()
  File "</home/tinuviel/pypy/dist/py/code/source.py:213>", line 3, in
_run_UWS_ObjSpace_W_Root
  File "/home/tinuviel/pypy/dist/pypy/module/sys2/hook.py", line 8, in
displayhook
    print_item_to(space, space.repr(w_obj), sys_stdout(space))
  File "/home/tinuviel/pypy/dist/pypy/interpreter/gateway.py", line
528, in appcaller
    return space.call_args(w_func, args)
  File "/home/tinuviel/pypy/dist/pypy/objspace/descroperation.py",
line 88, in call_args
    return w_obj.call_args(args)
  File "/home/tinuviel/pypy/dist/pypy/interpreter/function.py", line
40, in call_args
    return frame.run()
  File "/home/tinuviel/pypy/dist/pypy/interpreter/eval.py", line 70, in resume
    result = self.eval(executioncontext)
  File "/home/tinuviel/pypy/dist/pypy/interpreter/pyframe.py", line 97, in eval
    self.dispatch()
  File "/home/tinuviel/pypy/dist/pypy/interpreter/pyopcode.py", line
54, in dispatch
    fn(self, oparg)
  File "/home/tinuviel/pypy/dist/pypy/interpreter/pyopcode.py", line
642, in CALL_FUNCTION
    w_result = f.space.call_args(w_function, args)
  File "/home/tinuviel/pypy/dist/pypy/objspace/descroperation.py",
line 88, in call_args
    return w_obj.call_args(args)
  File "/home/tinuviel/pypy/dist/pypy/interpreter/function.py", line
40, in call_args
    return frame.run()
  File "/home/tinuviel/pypy/dist/pypy/interpreter/eval.py", line 70, in resume
    result = self.eval(executioncontext)
  File "/home/tinuviel/pypy/dist/pypy/interpreter/pyframe.py", line 97, in eval
    self.dispatch()
  File "/home/tinuviel/pypy/dist/pypy/interpreter/pyopcode.py", line
54, in dispatch
    fn(self, oparg)
  File "/home/tinuviel/pypy/dist/pypy/interpreter/pyopcode.py", line
502, in LOAD_ATTR
    w_value = f.space.getattr(w_obj, w_attributename)
  File "/home/tinuviel/pypy/dist/pypy/objspace/descroperation.py",
line 120, in getattr
    w_descr = space.lookup(w_obj, '__getattribute__')
  File "/home/tinuviel/pypy/dist/pypy/objspace/std/objspace.py", line
250, in lookup
    w_type = w_obj.getclass(self)
  File "/home/tinuviel/pypy/dist/pypy/interpreter/baseobjspace.py",
line 29, in getclass
    return space.gettypeobject(self.typedef)
  File "/home/tinuviel/pypy/dist/pypy/objspace/std/objspace.py", line
153, in gettypeobject
    self._typecache)
  File "/home/tinuviel/pypy/dist/pypy/interpreter/baseobjspace.py",
line 103, in loadfromcache
    return cache.getorbuild(key, builder, self)
  File "/home/tinuviel/pypy/dist/pypy/tool/cache.py", line 26, in getorbuild
    result = builder(key, stuff)
  File "/home/tinuviel/pypy/dist/pypy/objspace/std/stdtypedef.py",
line 85, in buildtypeobject
    dict_w[descrname] = w(descrvalue)
  File "/home/tinuviel/pypy/dist/pypy/objspace/std/objspace.py", line
205, in wrap
    return ft(self, x)
  File "/home/tinuviel/pypy/dist/pypy/objspace/std/fake.py", line 152,
in __init__
    self.name = d.__name__
AttributeError: 'FakeDescriptor' object has no attribute '__name__'


From jacob at strakt.com  Tue Apr  5 16:38:00 2005
From: jacob at strakt.com (Jacob =?iso-8859-1?q?Hall=E9n?=)
Date: Tue, 5 Apr 2005 16:38:00 +0200
Subject: [pypy-dev] Cancellation of Oxford sprint
Message-ID: <200504051638.00884.jacob@strakt.com>

I was just notified that the price of 45 pounds per day we were quoted was per 
day, per person. This makes the cost of sprinting, if you add it all up, 
higher than in Washington DC. Since partners dropped out because of the high 
costs in DC, I'm afraid we have to cancel, unless someone comes up with a 
much cheaper sprinting venue.

We are still scheduled to give a talk at the ACCU on Saturday 23 April, where 
the ACCU pays for 2 nights in a hotel and conference attendance for the 
speakers. We have a 90 minute slot and paid attendance for 2 speakers. The 
idea is that we do one part about the general goals of the project and the 
setup of the EU project, and one part on the technical details of how things 
are implemented.

I think Armin would be the best person to give the technical talk. I don't 
know who should give the general one. Do we have any volunteers?

I'm going to the conference anyway, and I have my entrance paid, since I will 
speak on "Sprint driven development". I'm happy to assist anyone who wants to 
handle the general Pypy talk.

Jacob


From rxe at ukshells.co.uk  Tue Apr  5 21:50:43 2005
From: rxe at ukshells.co.uk (Richard Emslie)
Date: Tue, 5 Apr 2005 20:50:43 +0100 (BST)
Subject: [pypy-dev] Re: [pypy-svn] r10350 - pypy/dist/pypy
In-Reply-To: <20050405191901.8E01E27B3E@code1.codespeak.net>
References: <20050405191901.8E01E27B3E@code1.codespeak.net>
Message-ID: <Pine.LNX.4.60.0504052040430.21982@echo.ukshells.co.uk>


Hi Seo,

On Tue, 5 Apr 2005 sanxiyn at codespeak.net wrote:

> Author: sanxiyn
> Date: Tue Apr  5 21:19:01 2005
> New Revision: 10350
>
> Added:
>   pypy/dist/pypy/README.22
> Log:
> Document 2.2 compatibility
>
> +* opcode.py
> +opcode module is new in 2.3, and is used in the interpreter core.
> +Just copying this file from 2.3 should work. Without this, PyPy
> +won't even start.

Another hack would be to import pypy.tool.opcode.  Not sure it's a good 
idea on the long run though! ;-)

Cheers, Richard


From cfbolz at gmx.de  Wed Apr  6 09:48:48 2005
From: cfbolz at gmx.de (Carl Friedrich Bolz)
Date: Wed, 6 Apr 2005 09:48:48 +0200 (MEST)
Subject: [pypy-dev] Sprint in Heidelberg
Message-ID: <12306.1112773728@www26.gmx.net>

Hi pypy-dev!

Today I talked to the administrator of the institute of physics of the
university of Heidelberg about the possibility of doing a PyPy sprint in
Heidelberg this summer. He told me it would not be problem at all to use
the rooms and facilities of the institute of physics as long as the sprint
would take place during semester break, which ranges from 17. July  to  16.
October. It would be better though to stay in the range August-September
since a lot of people are on holiday during that period. He even told me
that he would support an official request (he thought the PyGame flowgraph
viewer was pretty cool :-) ) and help me get the official stuff gone --
which also doesn't amount to much more than talking to a professor and
reserving the sprinting room.

So I can offer you the opportunity to sprint in Heidelberg this summer. I
would really be delighted to host such an event.

Regards,

Carl Friedrich


From arigo at tunes.org  Thu Apr  7 14:41:14 2005
From: arigo at tunes.org (Armin Rigo)
Date: Thu, 7 Apr 2005 13:41:14 +0100
Subject: [pypy-dev] Re: [pypy-svn] r10385 - in
	pypy/dist/pypy/translator/llvm: . test
In-Reply-To: <20050406223720.04BD527B44@code1.codespeak.net>
References: <20050406223720.04BD527B44@code1.codespeak.net>
Message-ID: <20050407124114.GA22358@vicky.ecs.soton.ac.uk>

Hi Carl,

On Thu, Apr 07, 2005 at 12:37:20AM +0200, cfbolz at codespeak.net wrote:
>  * The code generator tries to avoid creating empty blocks.

There is already code in simplify.py that should remove empty blocks and
perform other similar simplifications.  I guess it needs to be called again
after the annotator's "a.simplify()", which might remove some operations and
thus make new blocks empty.


Armin


From pedronis at strakt.com  Thu Apr  7 22:26:46 2005
From: pedronis at strakt.com (Samuele Pedroni)
Date: Thu, 07 Apr 2005 22:26:46 +0200
Subject: [pypy-dev] help(1) OverflowError
In-Reply-To: <20050403110354.GM23818@solar.trillke.net>
References: <20050401094859.GA17901@vicky.ecs.soton.ac.uk>
	<20050403110354.GM23818@solar.trillke.net>
Message-ID: <42559786.2060608@strakt.com>

holger krekel wrote:
> On Fri, Apr 01, 2005 at 10:48 +0100, Armin Rigo wrote:
> 
>>On Fri, Apr 01, 2005 at 03:56:34PM +0900, Sanghyeon Seo wrote:
>>
>>>  File "/home/tinuviel/pypy/dist/lib-python-2.3.4/sre_compile.py",
>>>line 488 in compile
>>>        groupindex, indexgroup
>>>OverflowError: long int too large to convert to int
>>
>>Actually, if you have a very fast machine, debugging with pdb in py.py works
>>fine.  It's related to Michael Chermside's module/_sre_pypy/ check-in.  The
>>bug is that _sre.compile() should accept as 3rd argument a list of unsigned
>>integers, but in the current implementation expects a list of signed integers
>>-- hence the OverflowError.
>>
>>Help!  What is the proper way to unwrap an unsigned integer?
> 
> 
> Hum, nobody seems to be able to answer this at the moment. 
> It probably relates to the fact that our usage of integers at 
> interpreter-level in general is fuzzy.  Or do we have a concise 
> explicit definition for their usage? 

one possible way to attack this issue is to write a snippet file of 
functions with arithmetic expressions that document the current patterns 
and how we want/already expect to do things like express unsigned 
numbers, unsigned arithmetic etc, overflow handling. This would help 
think the issues through, work as documentation of the idioms and enable 
to construct tests that check that the result running on top of CPython 
vs. after translation with or without annotation are what we expect and 
consistent up to expected type differences (for example a translated 
annotated snippet using r_uint may produce a long after doing internally 
unsigned arithmetic with r_uint erased by the translation and converting 
the result back to PyObject).

We probably already have some of this scattered around or that can be 
extracted from the tests of our restricted arith objects, but pursuing 
this constistently may be very valuable instead of waiting that some big 
piece of code will do the wrong thing and then we fix the problem after 
painful debugging. In this case this approach is probably easier than
with the annotator in general where some problems were hard to imagine 
before encountering them, but even in that case boiling things down to 
small examples has been very valuable so far.



> Anyway, apparently Samuele and me re-patched the trunk to not use 
> Michael Chermside's _sre_pypy implementation for the moment because 
> of this issue.  You can get back the previous behaviour (with the 
> help(1) problem) by uncommenting in baseobjspace.py the 
> line starting with ... self.setbuiltinmodule("_sre"
> at least on amd64 (which fails differently from a 32bit
> machine, sigh). 
> 
>     holger
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev



From hpk at trillke.net  Fri Apr  8 01:43:41 2005
From: hpk at trillke.net (holger krekel)
Date: Fri, 8 Apr 2005 01:43:41 +0200
Subject: [pypy-dev] Re: [py-dev] Re: Error with py.test and scipy ...
In-Reply-To: <4255C358.9040500@divmod.com>
References: <42558EFA.9060205@balster.info>
	<20050407213831.GI23818@solar.trillke.net>
	<d34a36$sgl$1@sea.gmane.org>
	<20050407223902.GL23818@solar.trillke.net>
	<4255B822.9010401@orbtech.com>
	<20050407232157.GN23818@solar.trillke.net>
	<4255C358.9040500@divmod.com>
Message-ID: <20050407234341.GP23818@solar.trillke.net>

On Fri, Apr 08, 2005 at 01:33 +0200, Valentino Volonghi aka Dialtone wrote:
> holger krekel wrote:
> 
> >I would be surprised if my brand-new hack breaks yours (because that
> >is exactly what it is trying to avoid) but i've learned that "import" 
> >makes the impossible possible and vice versa. 
> >
> >cheers, 
> > 
> >
> It was a quick fix in my case :).

Everybody says that, see "piles of hacks" earlier  :-) 

> It shouldn't be hard to figure out a common failing pattern in import 
> magic. for example:
> nevow was failing because of namedAny that imports a module from a 
> string representing the entire path in PYTHONPATH.

brain shaking already ... 

> from nevow import util
> In [6]:util.namedAny('nevow.flat.ten')
> Out[6]:<module 'nevow.flat.ten' from 
> '/Volumes/dati/Sviluppo/Nevow/nevow/flat/ten.pyc'>
> 
> namedAny is this one:
> 
> def namedAny(name):
>    """Get a fully named package, module, module-global object, or 
> attribute.
>    """
>    names = name.split('.')
>    topLevelPackage = None
>    moduleNames = names[:]
>    while not topLevelPackage:
>        try:
>            trialname = '.'.join(moduleNames)
>            topLevelPackage = __import__(trialname)
>        except ImportError:
>            # if the ImportError happened in the module being imported,
>            # this is a failure that should be handed to our caller.
>            # count stack frames to tell the difference.
> 
>            # string-matching is another technique, but I think it could be
>            # fooled in some funny cases
>            #if sys.exc_info()[1] != "cannot import name %s" % trialname:
>            #    raise
>            import traceback
>            if len(traceback.extract_tb(sys.exc_info()[2])) > 1:
>                raise
>            moduleNames.pop()
>   
>    obj = topLevelPackage
>    for n in names[1:]:
>        obj = getattr(obj, n)
>       
>    return obj
> 
> But I can't remember exactly the failure. The ML is here to help 
> (something about the PYTHONPATH anyway).

I have written similar code and there were always side cases 
that made it at least hard to debug and find the actual problem
in the program. 

cheers, 

    holger


From hpk at trillke.net  Fri Apr  8 14:46:05 2005
From: hpk at trillke.net (holger krekel)
Date: Fri, 8 Apr 2005 14:46:05 +0200
Subject: [pypy-dev] Re: [py-dev] Re: Error with py.test and scipy ...
In-Reply-To: <20050407234341.GP23818@solar.trillke.net>
References: <42558EFA.9060205@balster.info>
	<20050407213831.GI23818@solar.trillke.net>
	<d34a36$sgl$1@sea.gmane.org>
	<20050407223902.GL23818@solar.trillke.net>
	<4255B822.9010401@orbtech.com>
	<20050407232157.GN23818@solar.trillke.net>
	<4255C358.9040500@divmod.com>
	<20050407234341.GP23818@solar.trillke.net>
Message-ID: <20050408124605.GY23818@solar.trillke.net>

On Fri, Apr 08, 2005 at 01:43 +0200, holger krekel wrote:
> On Fri, Apr 08, 2005 at 01:33 +0200, Valentino Volonghi aka Dialtone wrote:
> > holger krekel wrote:
> > 
> > >I would be surprised if my brand-new hack breaks yours (because that
> > >is exactly what it is trying to avoid) but i've learned that "import" 
> > >makes the impossible possible and vice versa. 
> .... 

Sorry, i mixed lists. This posting was intended to go to py-dev, 
the development list for the py lib ... 

    holger


From arigo at tunes.org  Fri Apr  8 17:27:30 2005
From: arigo at tunes.org (Armin Rigo)
Date: Fri, 8 Apr 2005 16:27:30 +0100
Subject: [pypy-dev] Integer arithmetic
Message-ID: <20050408152730.GA30573@vicky.ecs.soton.ac.uk>

Hi all,

(IRC summary with Samuele and myself)

We propose the following rules for how integer operations should behave in
R-Python code:

* operations on plain integers produce longs in CPython when they overflow,
and wrap-around silently on platforms like C.  In other words, they behave
differently when run on top of Python or after translation.  We cannot easily
change that, but instead the following rule seems to be the sanest idea: in
both cases, the only guarantee is that the last 32 (or 64) bits of the result
are correct.  If needed we must call a function, intmask(x), to ensure that we
mask off the high bits on Python.  intmask() is a no-op after translation to
C.

* unsigned arithmetic: the r_uint class is fine for that.  It provides
wrap-around unsigned arithmetic on top of Python, and for the translator it is
a marker that unsigned ints should be generated.  Warning, mixed operations
r_uint+int produce r_uint results; be careful about unlimited propagation.  
Conversion: r_uint(x) for int-to-uint; intmask(x) for uint-to-int.

* a new space.uint_w() turns a non-negative wrapped int or long into an
r_uint.

* overflow detection: only for signed arithmetic, and only explicitely using a
macro-ish function ovfchecked().  It's only allowed to call it as
ovfchecked(<variable> <operation> <variable>); no subexpressions.  It means
that this single operation is the one that is done with overflow checking.  
When running on top of Python, ovfchecked(x) just returns x unless it is too
large, in which case you get an OverflowError.  After translation, the two
basic block operations "z = op(x,y)" and "simple_call(ovfchecked, z)" can be
detected easily and turned into whatever the back-end does for overflow
checking.  (Additionally, hpk proposed a hack that checks the syntax of the
ovfchecked() call when running on top of Python, to avoid problems with
expressions like ovfchecked(x+y+z) where after translation only the last
operation is checked.)

* Christian's very nice r_int is on the way out...  its main usage, in
intobject.py, will be replaced by ovfchecked().  It's just more explicit and
not really longer.  It's also more in line with Python's evoluation which says
that arithmetic operations don't raise OverflowErrors any more.


Comments welcome!

Armin


From guenter.jantzen at netcologne.de  Fri Apr  8 22:16:55 2005
From: guenter.jantzen at netcologne.de (=?ISO-8859-1?Q?G=FCnter_Jantzen?=)
Date: Fri, 08 Apr 2005 22:16:55 +0200
Subject: [pypy-dev] Integer arithmetic
In-Reply-To: <20050408152730.GA30573@vicky.ecs.soton.ac.uk>
References: <20050408152730.GA30573@vicky.ecs.soton.ac.uk>
Message-ID: <4256E6B7.1030105@netcologne.de>

Hello Armin,

again I point to C#. Please google for "C# checked" or "C# unchecked".
Maybe it is a good idea to implement a default behaviour, but to give 
the user possibilities to control this behaviour.

Hmm, this seems to be exactly what you are proposing to do with 
"ovfchecked()"

regards from Cologne
G?nter


Armin Rigo schrieb:
> Hi all,
> 
> (IRC summary with Samuele and myself)
> 
> We propose the following rules for how integer operations should behave in
> R-Python code:
> 
> * operations on plain integers produce longs in CPython when they overflow,
> and wrap-around silently on platforms like C.  In other words, they behave
> differently when run on top of Python or after translation.  We cannot easily
> change that, but instead the following rule seems to be the sanest idea: in
> both cases, the only guarantee is that the last 32 (or 64) bits of the result
> are correct.  If needed we must call a function, intmask(x), to ensure that we
> mask off the high bits on Python.  intmask() is a no-op after translation to
> C.
> 
> * unsigned arithmetic: the r_uint class is fine for that.  It provides
> wrap-around unsigned arithmetic on top of Python, and for the translator it is
> a marker that unsigned ints should be generated.  Warning, mixed operations
> r_uint+int produce r_uint results; be careful about unlimited propagation.  
> Conversion: r_uint(x) for int-to-uint; intmask(x) for uint-to-int.
> 
> * a new space.uint_w() turns a non-negative wrapped int or long into an
> r_uint.
> 
> * overflow detection: only for signed arithmetic, and only explicitely using a
> macro-ish function ovfchecked().  It's only allowed to call it as
> ovfchecked(<variable> <operation> <variable>); no subexpressions.  It means
> that this single operation is the one that is done with overflow checking.  
> When running on top of Python, ovfchecked(x) just returns x unless it is too
> large, in which case you get an OverflowError.  After translation, the two
> basic block operations "z = op(x,y)" and "simple_call(ovfchecked, z)" can be
> detected easily and turned into whatever the back-end does for overflow
> checking.  (Additionally, hpk proposed a hack that checks the syntax of the
> ovfchecked() call when running on top of Python, to avoid problems with
> expressions like ovfchecked(x+y+z) where after translation only the last
> operation is checked.)
> 
> * Christian's very nice r_int is on the way out...  its main usage, in
> intobject.py, will be replaced by ovfchecked().  It's just more explicit and
> not really longer.  It's also more in line with Python's evoluation which says
> that arithmetic operations don't raise OverflowErrors any more.
> 
> 
> Comments welcome!
> 
> Armin
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev
> 


From arigo at tunes.org  Sat Apr  9 22:34:30 2005
From: arigo at tunes.org (Armin Rigo)
Date: Sat, 9 Apr 2005 21:34:30 +0100
Subject: [pypy-dev] Integer arithmetic
In-Reply-To: <20050408152730.GA30573@vicky.ecs.soton.ac.uk>
References: <20050408152730.GA30573@vicky.ecs.soton.ac.uk>
Message-ID: <20050409203430.GA16260@vicky.ecs.soton.ac.uk>

Hi,

A clarification of some points on IRC between Christian and Samuele:

The most important goal of the arithmetic changes discussed in the previous
e-mail was to have the same behavior before and after translation.  That's why
the current r_int is dangerous: it propagates everywhere if you are not
careful.  So up to yesterday, before translation you got overflow checking
everywhere where r_int goes, whereas after translation you only got overflow
checking if there is an explicit try: except OverflowError.  The ovfchecked()
replacement of r_int fixes that problem by behaving the same before and after
translation.

r_uint, on the other hand, only means "unsigned wrap-around arithmetic", and
behaves consistently like this before and after translation.

So starting from now all integers behave in this wrap-around way, unless we
use explicit ovfchecked(); even plain signed integers.  But because they
become longs when running on top of Python, at places we need to make sure
that we clamp them back to their least 32- (or 64-) bits.  That's the purpose
of intmask(), which will convert a long back to an int by masking off the
extra bits (even turning a positive long into a negative int if needed).  
intmask() is a no-op after translation.


Armin


From tismer at stackless.com  Sun Apr 10 11:57:43 2005
From: tismer at stackless.com (Christian Tismer)
Date: Sun, 10 Apr 2005 11:57:43 +0200
Subject: [pypy-dev] Integer arithmetic
In-Reply-To: <20050409203430.GA16260@vicky.ecs.soton.ac.uk>
References: <20050408152730.GA30573@vicky.ecs.soton.ac.uk>
	<20050409203430.GA16260@vicky.ecs.soton.ac.uk>
Message-ID: <4258F897.1010009@stackless.com>

Armin Rigo wrote:
> Hi,
> 
> A clarification of some points on IRC between Christian and Samuele:
> 
> The most important goal of the arithmetic changes discussed in the previous
> e-mail was to have the same behavior before and after translation.  That's why
> the current r_int is dangerous: it propagates everywhere if you are not
> careful.

[snip]

Thanks a lot, now I understand. Nice change!

ciao - chris

-- 
Christian Tismer             :^)   <mailto:tismer at stackless.com>
tismerysoft GmbH             :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9A     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 802 86 56  mobile +49 173 24 18 776  fax +49 30 80 90 57 05
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/


From tismer at stackless.com  Mon Apr 11 04:16:42 2005
From: tismer at stackless.com (Christian Tismer)
Date: Mon, 11 Apr 2005 04:16:42 +0200
Subject: [pypy-dev] Mode of interaction
In-Reply-To: <4258F897.1010009@stackless.com>
References: <20050408152730.GA30573@vicky.ecs.soton.ac.uk>	<20050409203430.GA16260@vicky.ecs.soton.ac.uk>
	<4258F897.1010009@stackless.com>
Message-ID: <4259DE0A.6080109@stackless.com>

Hi friends,

I am a bit annoyed by your latest comments in the pypy IRC log.
You have mentioned some interaction caused by my checkin of
flow-space caching. This was something that was demanded since
a long time, and I thought I got it just right.

This was just a lye. In fact, I was very much annoyed:

There are a lot of indifferent and unreflected comments like

"""
[17:03] <arigo> Christian's new _interplevel_cache.py doesn't seem to 
hold all analysed pieces of code,
"""

which I do not understand. Which pieces of code should I not hold???
Arigo, did you a) not understand my code, and b) did you thing that
I'm that dumb? This wuold have helped with a), btw.

"""
[17:17] <pedronis> the app helpers are RPython-ish but RPython so 
geninterplevel is obviously an oddball
"""

Please? Can you please elaborate???

"""
[18:03] <arigo> _interplevel_cache is probably a bad idea for now
[18:04] <arigo> it makes everything behave erratically
"""

I asked *you* if caching of translations is valid, and you told
me it was just fine, 6 days before. And I tested it with full
cache and with cache cleared. Who is wrong, then???

"""
[18:05] <arigo> I've just spent half an hour debugging something that 
was solved by removing _interplevel_cache.pyc (the pyc! I removed the py 
regularly but forgot the pyc)
"""

Sorry, Armin, this is simply ignorance! I cant't feel any guilt for
that. And I also have no pardon if you fail with such a silly problem.
You should expect check-ins from other people, too, especially
if these are modelled exactly after your former demands, please
remember.

And if you need to criticize people, please do avoid the third person
and address them directly!

In other words, if you really dislike my participation, please be
explicit and tell me to leave this project, early. I can't stand
this implicit way, together with this ignorance.
I am not an enemy, at least I hoped so.

sincerely -- chris

-- 
Christian Tismer             :^)   <mailto:tismer at stackless.com>
tismerysoft GmbH             :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9A     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 802 86 56  mobile +49 173 24 18 776  fax +49 30 80 90 57 05
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/


From arigo at tunes.org  Mon Apr 11 12:17:35 2005
From: arigo at tunes.org (Armin Rigo)
Date: Mon, 11 Apr 2005 11:17:35 +0100
Subject: [pypy-dev] Re: Mode of interaction
In-Reply-To: <4259DE0A.6080109@stackless.com>
References: <20050408152730.GA30573@vicky.ecs.soton.ac.uk>
	<20050409203430.GA16260@vicky.ecs.soton.ac.uk>
	<4258F897.1010009@stackless.com> <4259DE0A.6080109@stackless.com>
Message-ID: <20050411101735.GC4815@vicky.ecs.soton.ac.uk>

Hi Christian,

I am really sorry about everything I said on IRC.  Your _interplevel_cache is
just great, and exactly what was needed.  It's just some combination of
strange bugs that slowly drove me restless yesterday, and IRC tends to amplify
strong feelings like real annoyance in the face of erratic bugs.  In
particular, bugs in PyPy have been generally nicely reproducible so far, so I
was a bit put off.

I was the first to ask for _interplevel_cache, and it's great!  I didn't read
its source code at first because I was busy debugging my hacks in the flow
space.

Some comments I made like the following one:

> [17:03] <arigo> Christian's new _interplevel_cache.py doesn't seem to 
> hold all analysed pieces of code,

was also caused by the fact that I deleted _interplevel_cache.py without also
removing the .pyc file, which as I understood later not only caused gateway.py
to still be able to import the previous cache, but also recreate a .py file
that contained only the new additions.  The result is that after running py.py
I got a .py file with only one piece of known_source.  This is why I thought
that there was a problem along the lines of the cache only storing the last
piece of analysed code.

So all this frustration of mine boils down to this simple "import bug", or
rather this expectation of mine that clearing _interplevel_cache.py was
sufficient to clear the cache.  I tend to think that .pyc files don't exist,
i.e. that they are just an internal caching detail of CPython, and I'm bitten
from time to time by the fact that you can actually import a .pyc file after
the .py file is gone.

I did read your check-in message about double-clicking _interplevel_cache to
self-destruct it, as well as looked in the source and noticed that the .pyc
and .pyo files where also deleted ("very nice to clean them up!" I though, but
not "right, you really have to clean them up too").  But now it is just so
that "rm _interplevel_cache.py" is 4 letters shorter to type than "python
_interplevel_cache.py".  It never occurred to me that gateway.py would still
try to import the .pyc file alone.  I'd like to share my blame for that with
Python's import mecanisms in general.

Sorry again.  I was getting nervous when the 7th strange bug was  '%s' % 12.3
raising a TypeError for no reason I could possibly imagine.  These bugs were
definitely mine, for sure, but the broken way in which I handled the cache
made them worse.  I should have concluded the IRC session by saying that once
the above minor quirk was understood, _interplevel_cache worked great.  I hope
you don't think that I ever considered removing it.  I really needed a way to
say "oops, great, but not RIGHT now while I debug please".  As Samuele pointed
out, that was all that I needed while playing with the flow space.


Again, sorry, all I meant to write down was the behavior I seemed to observe,
in case Samuele had hints for why it was so.  I didn't mean to criticise your
work in front of him.  The code which failed and which I was debugging was
mine!

Armin


From tismer at stackless.com  Tue Apr 12 11:41:37 2005
From: tismer at stackless.com (Christian Tismer)
Date: Tue, 12 Apr 2005 11:41:37 +0200
Subject: [pypy-dev] Re: Mode of interaction
In-Reply-To: <20050411101735.GC4815@vicky.ecs.soton.ac.uk>
References: <20050408152730.GA30573@vicky.ecs.soton.ac.uk>
	<20050409203430.GA16260@vicky.ecs.soton.ac.uk>
	<4258F897.1010009@stackless.com> <4259DE0A.6080109@stackless.com>
	<20050411101735.GC4815@vicky.ecs.soton.ac.uk>
Message-ID: <425B97D1.8010804@stackless.com>

Armin Rigo wrote:

[a way too nice reply]

Dear Armin, Samuele, all,

the person which wrote this ugly, overdone and unreflected
email was suffering from flu,woke up at 3:00 with fever
and nightmares, influenced by a lot of medical intoxination.
I will not allow this guy to touch the keyboard under such
circumstances, again and to abuse my name.

So here comes the best apology I've ever heared, for you:

"""
 > You take it back...

< I do.  I offer a complete and utter retraction.
The imputation was totally without basis in fact,
and was in no way 'fair comment' and was motivated
purely by malice and I deeply regret any distress that
my comments may have caused you or your family...
...And I hereby undertake not to repeat any such
slander at any time in the future.

 > (hesitates). OK.
"""

Borrowed from "A Fish called Wanda", the apology scene between
Otto and Archie, Archie dangling upside down out of the window
of the flat.  Otto is holding him by his ankles.

See http://www.textfiles.com/media/a_fish_c.apo

cheers - chris

-- 
Christian Tismer             :^)   <mailto:tismer at stackless.com>
tismerysoft GmbH             :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9A     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 802 86 56  mobile +49 173 24 18 776  fax +49 30 80 90 57 05
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/


From eric at vanrietpaap.nl  Tue Apr 12 15:09:34 2005
From: eric at vanrietpaap.nl (Eric van Riet Paap)
Date: Tue, 12 Apr 2005 15:09:34 +0200 (CEST)
Subject: [pypy-dev] llvm ebuild for Gentoo available
Message-ID: <1409.62.194.210.178.1113311374.squirrel@www.vanrietpaap.nl>

Hello everybody,

Not sure if anyone is using Gentoo Linux but if you do there is an ebuild
available which makes installing llvm trivial.

The other of the email is Valery Khamenya
and the location is: http://bugs.gentoo.org/show_bug.cgi?id=88628

cheers
Eric


From eric at vanrietpaap.nl  Tue Apr 12 15:13:08 2005
From: eric at vanrietpaap.nl (Eric van Riet Paap)
Date: Tue, 12 Apr 2005 15:13:08 +0200 (CEST)
Subject: [pypy-dev] (no subject)
Message-ID: <1422.62.194.210.178.1113311588.squirrel@www.vanrietpaap.nl>

That should read "The author of the email is Valery Khamenya" offcourse!

E



From sanxiyn at gmail.com  Tue Apr 12 16:32:51 2005
From: sanxiyn at gmail.com (Sanghyeon Seo)
Date: Tue, 12 Apr 2005 23:32:51 +0900
Subject: [pypy-dev] llvm ebuild for Gentoo available
In-Reply-To: <1409.62.194.210.178.1113311374.squirrel@www.vanrietpaap.nl>
References: <1409.62.194.210.178.1113311374.squirrel@www.vanrietpaap.nl>
Message-ID: <5b0248170504120732763df4cd@mail.gmail.com>

On Apr 12, 2005 10:09 PM, Eric van Riet Paap <eric at vanrietpaap.nl> wrote:
> Not sure if anyone is using Gentoo Linux but if you do there is an ebuild
> available which makes installing llvm trivial.

By the way, LLVM is now part of official Debian archive:
http://packages.debian.org/unstable/devel/llvm

Seo Sanghyeon


From eric at vanrietpaap.nl  Wed Apr 13 19:20:29 2005
From: eric at vanrietpaap.nl (Eric van Riet Paap)
Date: Wed, 13 Apr 2005 19:20:29 +0200
Subject: [pypy-dev] Deltareport Overview
Message-ID: <425D54DD.7020403@vanrietpaap.nl>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

The last report at http://codespeak.net/pypy/rev/ is from 06-Apr-2005 02:20

Is the script no longer working or does it only add a new entry when the
percentages change?

- - Eric

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.0 (MingW32)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFCXVTdMcwuX5fyfyMRAkm+AJ4qysIqxnTgeeyav2ijuM5E8k/m3ACgjKel
FZbYGVtylghvbDOsRWaV6+w=
=Jor8
-----END PGP SIGNATURE-----


From hpk at trillke.net  Wed Apr 13 21:29:59 2005
From: hpk at trillke.net (holger krekel)
Date: Wed, 13 Apr 2005 21:29:59 +0200
Subject: [pypy-dev] Deltareport Overview
In-Reply-To: <425D54DD.7020403@vanrietpaap.nl>
References: <425D54DD.7020403@vanrietpaap.nl>
Message-ID: <20050413192959.GQ23818@solar.trillke.net>

Hi Eric, 

On Wed, Apr 13, 2005 at 19:20 +0200, Eric van Riet Paap wrote:
> The last report at http://codespeak.net/pypy/rev/ is from 06-Apr-2005 02:20
> 
> Is the script no longer working or does it only add a new entry when the
> percentages change?

Actually the script didn't work for some time on amd64 (so it
couldn't be run on codespeak.net). So i switched the automatic
generation of the revreport off.  However, i just switched it
on as it seems to work again!  The percentages haven't changed 
much although that is likely a result of the somewhat
limited revreport itself.  I definitely want to add info about
running failing/passing tests to the "revision report".  

cheers, 

    holger



From arigo at tunes.org  Thu Apr 14 11:45:26 2005
From: arigo at tunes.org (Armin Rigo)
Date: Thu, 14 Apr 2005 10:45:26 +0100
Subject: [pypy-dev] Re: [pypy-svn] r10600 - pypy/dist/pypy/translator/genc
In-Reply-To: <20050413235341.D95CA27B5A@code1.codespeak.net>
References: <20050413235341.D95CA27B5A@code1.codespeak.net>
Message-ID: <20050414094526.GA6772@vicky.ecs.soton.ac.uk>

Hi Christian,

On Thu, Apr 14, 2005 at 01:53:41AM +0200, tismer at codespeak.net wrote:
> Modified:
>    pypy/dist/pypy/translator/genc/pyobj_include.h
> Log:
> OP_ABS was missing, targetpypy1 was not compiling!
> How can this happen???

I changed the flow space to unify calls to built-in functions and operations 
when possible.  For example, the annotator had to deal with both the 'type' 
operation and the 'type()' built-in doing the same thing.  Now the flow 
objspace contains a list of built-ins that map to operations, and always 
produces the operation.  That's why genc.h is now found to be incomplete: 
operations like 'abs' didn't use to turn up...

I will just go over the full list of operations and make sure pyobj_include.h 
contains them.


Armin


From oldjava at gmx.net  Wed Apr  6 02:18:27 2005
From: oldjava at gmx.net (Carl Friedrich Bolz)
Date: Wed, 6 Apr 2005 02:18:27 +0200 (MEST)
Subject: [pypy-dev] Sprint in Heidelberg
Message-ID: <3979.1112746707@www77.gmx.net>

Hi pypy-dev!

Today I talked to the administrator of the institute of physics of the
university of Heidelberg about the possibility of doing a PyPy sprint in 
Heidelberg this summer. He told me it would not be problem at all to use 
the rooms and facilities of the institute of physics as long as the sprint 
would take place during semester break, which ranges from 17. July  to  16. 
October. It would be better though to stay in the range August-September 
since a lot of people are on holiday during that period. He even told me 
that he would support an official request (he thought the PyGame flowgraph 
viewer was pretty cool :-) ) and help me get the official stuff gone -- 
which also doesn't amount to much more than talking to a professor and 
reserving the sprinting room.

So I can offer you the opportunity to sprint in Heidelberg this summer. I 
would really be delighted to host such an event.

Regards,

Carl Friedrich


From Ben.Young at risk.sungard.com  Thu Apr 14 10:10:34 2005
From: Ben.Young at risk.sungard.com (Ben.Young at risk.sungard.com)
Date: Thu, 14 Apr 2005 09:10:34 +0100
Subject: [pypy-dev] Quick question
Message-ID: <OFA03FFB7B.3B1534BE-ON80256FE3.002CC4E6-80256FE3.002C037F@risk.sungard.com>

Just a quick question from an interested observer!

Has the trivial object space completely died now? I can't find in the 
source, but the documentation specifically states that it won't be going 
away!

Keep up the good work!

Cheers,
Ben


From hpk at trillke.net  Thu Apr 14 12:16:23 2005
From: hpk at trillke.net (holger krekel)
Date: Thu, 14 Apr 2005 12:16:23 +0200
Subject: [pypy-dev] Quick question
In-Reply-To: <OFA03FFB7B.3B1534BE-ON80256FE3.002CC4E6-80256FE3.002C037F@risk.sungard.com>
References: <OFA03FFB7B.3B1534BE-ON80256FE3.002CC4E6-80256FE3.002C037F@risk.sungard.com>
Message-ID: <20050414101623.GS23818@solar.trillke.net>

Hi Ben, 

On Thu, Apr 14, 2005 at 09:10 +0100, Ben.Young at risk.sungard.com wrote:
> Just a quick question from an interested observer!
> 
> Has the trivial object space completely died now? I can't find in the 
> source, but the documentation specifically states that it won't be going 
> away!

Reality has been adjusted: the documentation doesn't mention trivial
object space anymore. 

The truth is that maintaining the trivial object space was getting
more and more involved with no real benefit.  The trivial object space 
was useful as long as the standard object space was not complete enough.   
It is even arguable if the trivial object space didn't actually get more 
complicated in its implementation than the standard one. 

cheers, 

    holger


From arigo at tunes.org  Sun Apr 17 11:44:22 2005
From: arigo at tunes.org (Armin Rigo)
Date: Sun, 17 Apr 2005 10:44:22 +0100
Subject: [pypy-dev] Nonzero
In-Reply-To: <26216.1113690479@www20.gmx.net>
References: <20050416203452.GA30861@vicky.ecs.soton.ac.uk>
	<26216.1113690479@www20.gmx.net>
Message-ID: <20050417094422.GA7437@vicky.ecs.soton.ac.uk>

Hi Carl,

(Reply to pypy-dev)

On Sun, Apr 17, 2005 at 12:27:59AM +0200, Carl Friedrich Bolz wrote:
> BTW: I think there's a problem in DescrOperation.is_true. If you have a
> 
> class Monty:
>     def __nonzero__(self):
>         return self
> 
> and you call bool(Monty()) you get an infinite recursion, since
> DescrOperation.is_true calls space.is_true on the result of the __nonzero__
> method, which in turn calls DescrOperation.is_true again. CPython raises a
> "TypeError, __nonzero__ should return an int". I didn't feel confident
> enough to fix it myself. 

Yes, there are two ways to go about it: CPython raises a TypeError basically
to avoid exactly this infinite recursion.  But on other cases it lets the
infinite recursion happen, but bound it, and you get a RuntimeError about the
recursion limit.

For PyPy, the general idea was to have a utility that looks through all loops
of the function call graph of PyPy, and inserts infinite recursion detection
calls automatically.  In this way we can safely reproduce the CPython
approach.  Then your example would give a nice RuntimeError.

Of course in this specific case we can also simply copy the TypeErroring
behavior.  What about doing so if and only if there is a test :-)


Armin


From hpk at trillke.net  Mon Apr 18 01:26:45 2005
From: hpk at trillke.net (holger krekel)
Date: Mon, 18 Apr 2005 01:26:45 +0200
Subject: [pypy-dev] result of mini-sprint / documentation 
Message-ID: <20050417232645.GV23818@solar.trillke.net>

Hi folks, 

Armin and me have spent today on largely reworking and
refactoring PyPy's documentation.  On 

    http://codespeak.net/pypy/index.cgi?doc

you'll see a cleaned up entry page and navigation bar, 
and much cleaned up and cross-referenced content. 

Moreover, the integrity of the links in the ReST files 
are now checked within our testing process.  If you issue:: 

    python test_all.py documentation 
   
then ReST-checks as well as link checks are performed.  If you
provide the '-R' option then also all remote links are checked
for existence.  Running 'py.test' or 'python test_all.py' in
the pypy directory will run the non-remote documentation
checks by default. 

Our documentation rework should provide a good basis for
continued updating and improvement of the documentation. 
Btw, there is a 'redirections' file which contains a mapping
from old names to new names/locations which the web site uses 
to map old URLs to new places (via a Redirect). Try e.g.: 

    http://codespeak.net/pypy/index.cgi?doc/howtopypy.html 

or any other old URL. 

Feedback (and helping with improving the documentation further)
is very welcome! 

cheers, 

    holger


From hpk at trillke.net  Thu Apr 21 20:28:21 2005
From: hpk at trillke.net (holger krekel)
Date: Thu, 21 Apr 2005 20:28:21 +0200
Subject: [pypy-dev] whitespace policies
Message-ID: <20050421182821.GR22996@solar.trillke.net>

Hi all, 

apparently i am one of the people who happily add spaces at
the end of lines (more or less unconciously but i think it
also relates to my vim usage patterns, whatever).  I remember
Armin complained once about these extra spaces.  The main
problem with these extra spaces is that if a someone's editor
automatically scraps those whitespaces (like apparently, Armin's
emacs did) then that someone will produce large unmeaningful 
diffs. 

Now my question is, do we want to "enforce" or at least 
state in our coding-style document one policy or the other?  
I see two options: 

1. you should NOT use editor settings that scrap trailing 
   whitespace on source code or documentation.  You will note
   "wrong" editor settings when you see 'svn diff's ala:: 

   -  some line content 
   +  some line content

   which indicate that you only changed whitespace. 

or 

2. you should use editor settings that scrap trailing 
   whitespace on source code and documentation 

Obviously, the second option should be accompanied by
a monster-commit changing all those lines in one go.  

What do you think?

cheers, 

    holger


From mwh at python.net  Thu Apr 21 20:52:39 2005
From: mwh at python.net (Michael Hudson)
Date: Thu, 21 Apr 2005 19:52:39 +0100
Subject: [pypy-dev] Re: whitespace policies
In-Reply-To: <20050421182821.GR22996@solar.trillke.net> (holger krekel's
	message of "Thu, 21 Apr 2005 20:28:21 +0200")
References: <20050421182821.GR22996@solar.trillke.net>
Message-ID: <2mk6mw55e0.fsf@starship.python.net>

hpk at trillke.net (holger krekel) writes:

> Now my question is, do we want to "enforce" or at least 
> state in our coding-style document one policy or the other?  
> I see two options: 
>
> 1. you should NOT use editor settings that scrap trailing 
>    whitespace on source code or documentation.  You will note
>    "wrong" editor settings when you see 'svn diff's ala:: 
>
>    -  some line content 
>    +  some line content
>
>    which indicate that you only changed whitespace. 
>
> or 
>
> 2. you should use editor settings that scrap trailing 
>    whitespace on source code and documentation 
>
> Obviously, the second option should be accompanied by
> a monster-commit changing all those lines in one go.  
>
> What do you think?

Option 2), with the added detail of running the reindent.py script
over the entire codebase every so often (as Tim does for Python).

I've wanted to do this for ages, but have always been too chicken :)

Cheers,
mwh

-- 
  It is time-consuming to produce high-quality software. However,
  that should not alone be a reason to give up the high standards
  of Python development.              -- Martin von Loewis, python-dev


From hpk at trillke.net  Thu Apr 21 21:19:07 2005
From: hpk at trillke.net (holger krekel)
Date: Thu, 21 Apr 2005 21:19:07 +0200
Subject: [pypy-dev] Re: whitespace policies
In-Reply-To: <2mk6mw55e0.fsf@starship.python.net>
References: <20050421182821.GR22996@solar.trillke.net>
	<2mk6mw55e0.fsf@starship.python.net>
Message-ID: <20050421191907.GS22996@solar.trillke.net>

On Thu, Apr 21, 2005 at 19:52 +0100, Michael Hudson wrote:
> hpk at trillke.net (holger krekel) writes:
> > 2. you should use editor settings that scrap trailing 
> >    whitespace on source code and documentation 
> >
> > Obviously, the second option should be accompanied by
> > a monster-commit changing all those lines in one go.  
> >
> > What do you think?
> 
> Option 2), with the added detail of running the reindent.py script
> over the entire codebase every so often (as Tim does for Python).

Is there a reason to not run it more or less immediately after 
each checkin?  This way a commiter could be appropriately 
embarassed. 

> I've wanted to do this for ages, but have always been too chicken :)

same here, even more after trying it for fun on the pypy-tree. 

cheers,
 
    holger


From mwh at python.net  Thu Apr 21 22:34:14 2005
From: mwh at python.net (Michael Hudson)
Date: Thu, 21 Apr 2005 21:34:14 +0100
Subject: [pypy-dev] Re: whitespace policies
In-Reply-To: <20050421191907.GS22996@solar.trillke.net>
References: <20050421182821.GR22996@solar.trillke.net>
	<2mk6mw55e0.fsf@starship.python.net>
	<20050421191907.GS22996@solar.trillke.net>
Message-ID: <f35ed9af6d593f238540d8ab64b0a22a@python.net>


On 21 Apr 2005, at 20:19, holger krekel wrote:

> On Thu, Apr 21, 2005 at 19:52 +0100, Michael Hudson wrote:
>> hpk at trillke.net (holger krekel) writes:
>>> 2. you should use editor settings that scrap trailing
>>>    whitespace on source code and documentation
>>>
>>> Obviously, the second option should be accompanied by
>>> a monster-commit changing all those lines in one go.
>>>
>>> What do you think?
>>
>> Option 2), with the added detail of running the reindent.py script
>> over the entire codebase every so often (as Tim does for Python).
>
> Is there a reason to not run it more or less immediately after
> each checkin?  This way a commiter could be appropriately
> embarassed.

If it could be so arranged, that sounds good.

>> I've wanted to do this for ages, but have always been too chicken :)
>
> same here, even more after trying it for fun on the pypy-tree.

It was 3000 lines of diff 18 months ago...

Cheers,
mwh



From jacob at strakt.com  Fri Apr 22 00:31:50 2005
From: jacob at strakt.com (Jacob Hallen)
Date: Fri, 22 Apr 2005 00:31:50 +0200
Subject: [pypy-dev] Re: whitespace policies
In-Reply-To: <2mk6mw55e0.fsf@starship.python.net>
References: <20050421182821.GR22996@solar.trillke.net>
	<2mk6mw55e0.fsf@starship.python.net>
Message-ID: <200504220031.50714.jacob@strakt.com>

torsdag 21 april 2005 20.52 skrev Michael Hudson:
> hpk at trillke.net (holger krekel) writes:
> > Now my question is, do we want to "enforce" or at least
> > state in our coding-style document one policy or the other?
> > I see two options:
> >
> > 1. you should NOT use editor settings that scrap trailing
> >    whitespace on source code or documentation.  You will note
> >    "wrong" editor settings when you see 'svn diff's ala::
> >
> >    -  some line content
> >    +  some line content
> >
> >    which indicate that you only changed whitespace.
> >
> > or
> >
> > 2. you should use editor settings that scrap trailing
> >    whitespace on source code and documentation
> >
> > Obviously, the second option should be accompanied by
> > a monster-commit changing all those lines in one go.
> >
> > What do you think?
>
> Option 2), with the added detail of running the reindent.py script
> over the entire codebase every so often (as Tim does for Python).

+2 (one for me and one for Laura)

Jacob


From tismer at stackless.com  Sat Apr 23 18:45:56 2005
From: tismer at stackless.com (Christian Tismer)
Date: Sat, 23 Apr 2005 18:45:56 +0200
Subject: [pypy-dev] Re: whitespace policies
In-Reply-To: <200504220031.50714.jacob@strakt.com>
References: <20050421182821.GR22996@solar.trillke.net>	<2mk6mw55e0.fsf@starship.python.net>
	<200504220031.50714.jacob@strakt.com>
Message-ID: <426A7BC4.7070906@stackless.com>

Jacob Hallen wrote:

...

>>Option 2), with the added detail of running the reindent.py script
>>over the entire codebase every so often (as Tim does for Python).
> 
> 
> +2 (one for me and one for Laura)

+1 from me, too

-- 
Christian Tismer             :^)   <mailto:tismer at stackless.com>
tismerysoft GmbH             :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9A     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 802 86 56  mobile +49 173 24 18 776  fax +49 30 80 90 57 05
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/


From arigo at tunes.org  Sun Apr 24 14:51:08 2005
From: arigo at tunes.org (Armin Rigo)
Date: Sun, 24 Apr 2005 13:51:08 +0100
Subject: [pypy-dev] whitespace policies
In-Reply-To: <20050421182821.GR22996@solar.trillke.net>
References: <20050421182821.GR22996@solar.trillke.net>
Message-ID: <20050424125108.GA19874@vicky.ecs.soton.ac.uk>

Hi,

On Thu, Apr 21, 2005 at 08:28:21PM +0200, holger krekel wrote:
> Now my question is, do we want to "enforce" or at least 
> state in our coding-style document one policy or the other?  

I'm not too concerned about a few dummy diff entries, even though I'm usually
avoiding to kill trailing whitespace.  I don't see the point of strict
policies as long as the source code base is roughly consistent, and I think
that it is.  This is both about trailing whitespace and indentation -- I mean,
we know that indentation is 4 spaces and we all follow that rule

By contrast I'd not enjoy enforced editor configurations, and really despise a
monster check-in that would render svn blame useless (which, I may add, I find
extremely useful to find not necessary who but when and with which log message
a particular line was written).


Armin


From arigo at tunes.org  Sun Apr 24 14:55:13 2005
From: arigo at tunes.org (Armin Rigo)
Date: Sun, 24 Apr 2005 13:55:13 +0100
Subject: [pypy-dev] Fwd: CFP: DLS05: ACM Dynamic Languages Symposium
Message-ID: <20050424125513.GA11964@vicky.ecs.soton.ac.uk>

Hi all,

I think we should consider the following conference.  It's part of OOPSLA,
which is definitely a prestigious conference and I believe that "language
symbiosis and multi-paradigm languages" is a nice way to describe what we are
trying to do with PyPy's approach.


----- Forwarded message from Guido van Rossum <gvanrossum at gmail.com> -----

See you all at OOPSLA!

---------- Forwarded message ----------
From: Roel Wuyts <Roel.Wuyts at ulb.ac.be>
Date: Apr 17, 2005 10:59 PM
Subject: CFP: DLS05: ACM Dynamic Languages Symposium
To: python-announce-list at python.org


                        CALL FOR PAPERS FOR THE

            ACM Dynamic Languages Symposium 2005
                                 October 18, 2005
                             San Diego, California
                       (co-located with OOPSLA'05)

    URL: http://decomp.ulb.ac.be:8082/events/dls05/

-----------
Abstract
-----------

In industry, static languages (such as Java, C++ and C#) are much more
widely used than their dynamic counterparts (like CLOS, Python, Self,
Perl, php or Smalltalk). So it appears as though dynamic language
concepts were forgotten and lost the race.

But this is not the case.

Java and C#, the latest mainstream static languages, popularized to a
certain extent dynamic language features such as garbage collection,
portability and (limited forms of) reflection. In the near future, we
expect this dynamicity to increase even further. E.g., it is getting
clearer year after year that pervasive computing is becoming the rule
and that concepts such as meta programming, reflection, mobility,
dynamic reconfigurability and distribution are becoming increasingly
popular. All of these features are the domain of dynamic languages, and
hence it is only logical that more dynamic language concepts have to be
taken up by static languages, or that dynamic languages can make a
breakthrough.

Currently, the dynamic language community is fragmented, split over a
multitude of paradigms (from functional over logic to object-oriented),
languages and syntaxes. This fragmentation severely hinders research as
well as acceptance, and results in either language wars or, even worse,
language ignorance. The goal of this symposium is to provide a highly
visible, international forum for researchers working on dynamic
features and languages. We explicitly invite submissions from all kinds
of paradigms (object-oriented, functional, logic, ...), as can be seen
from the structure of the program committee.

Areas of interests include, but are not limited to:
- closures
- delegation
- actors, active objects
- constraint systems
- mixins and traits
- reflection and meta-programming
- language symbiosis and multi-paradigm languages
- experience reports on successful application of dynamic languages

Accepted Papers will be published in the ACM Digital Library.

-------------------------------
Submission Guidelines
-------------------------------

Papers will need to be submitted using an online tracking system, of
which the URL will be given later.

All papers must be submitted electronically in PDF format (or
PostScript, if you do not have access to PDF-producing programs, but
this is not recommended). Submissions, as well as final versions, must
be formatted to conform to ACM Proceedings requirements: Nine point
font on ten point baseline, two columns per page, each column 3.33
inches wide by 9 inches tall, with a column gutter of 0.33 inches, etc.
See the ACM Proceedings Guidelines. You can save preparation time by
using one of the templates from that page. Note that MS Word documents
must be converted to PDF before being submitted.

----------------------
  Important Dates
----------------------

- Deadline for receipt of submissions: June 24th 2005
- Notification of acceptance or rejection: August 5th 2005
- Final version for the proceedings: To be announced later

---------------------------
Program Committee
---------------------------

- Gilad Bracha
- Wolfgang De Meuter
- Stephane Ducasse
- Gopal Gupta
- Robert Hirschfeld
- Dan Ingalls
- Yukihiro Matsumoto
- Mark Miller
- Eliot Miranda
- Philippe Mougin
- Oscar Nierstrasz
- Dave Thomas
- David Ungar
- Guido Van Rossum
- Peter Van Roy
- Jon L White (G)
- Roel Wuyts (Chair)

--
Roel Wuyts
   DeComp
roel.wuyts at ulb.ac.be                               Universit? Libre de
Bruxelles
http://homepages.ulb.ac.be/~rowuyts/
Belgique
Vice-President of the European Smalltalk Users Group: www.esug.org

--
http://mail.python.org/mailman/listinfo/python-announce-list

        Support the Python Software Foundation:
        http://www.python.org/psf/donations.html


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)
_______________________________________________
Python-Dev mailing list
Python-Dev at python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/arigo%40tunes.org

----- End forwarded message -----


From hpk at trillke.net  Sun Apr 24 15:06:46 2005
From: hpk at trillke.net (holger krekel)
Date: Sun, 24 Apr 2005 15:06:46 +0200
Subject: [pypy-dev] whitespace policies
In-Reply-To: <20050424125108.GA19874@vicky.ecs.soton.ac.uk>
References: <20050421182821.GR22996@solar.trillke.net>
	<20050424125108.GA19874@vicky.ecs.soton.ac.uk>
Message-ID: <20050424130646.GV22996@solar.trillke.net>

On Sun, Apr 24, 2005 at 13:51 +0100, Armin Rigo wrote:
> On Thu, Apr 21, 2005 at 08:28:21PM +0200, holger krekel wrote:
> > Now my question is, do we want to "enforce" or at least 
> > state in our coding-style document one policy or the other?  
> 
> I'm not too concerned about a few dummy diff entries, even though I'm usually
> avoiding to kill trailing whitespace.  I don't see the point of strict
> policies as long as the source code base is roughly consistent, and I think
> that it is.  This is both about trailing whitespace and indentation -- I mean,
> we know that indentation is 4 spaces and we all follow that rule

indeed. 

> By contrast I'd not enjoy enforced editor configurations, and really despise a
> monster check-in that would render svn blame useless (which, I may add, I find
> extremely useful to find not necessary who but when and with which log message
> a particular line was written).

good point.  Hum. 

    holger


From pedronis at strakt.com  Sun Apr 24 19:12:46 2005
From: pedronis at strakt.com (Samuele Pedroni)
Date: Sun, 24 Apr 2005 19:12:46 +0200
Subject: [pypy-dev] annotation status
Message-ID: <426BD38E.6070108@strakt.com>

I think people not following IRC/pyp-svn etc may still be interested in 
this piece of information about progress.

As of end of last week the annotator is able to type-annotate all of the 
pypy codebase, at least as loaded by goal/buildcache2.py
and goal/targetpypy.py, that means the interpreter plus the standard 
object space plus interp-level modules.

We still get some expected warnings in areas touched by faking, and some 
of that code may be delicate to translate.

Notice that the code-generators right now are not up to the task of 
using this result, gen-llvm is probably the most evoluted.
Having them catch up with the annotator is the next important step. 
Trying to translate is also probably the only way in order to discover
whether there are still problems with the annotator result.

Annotation takes around 15 mins on a 1.5Ghz Powerbook 15" and grows up 
to around .5Gb of memory also because we keep
a lot of debug info about how annotations evolved during the process to 
facilitate type-pollution tracking.

.../goal$ python translate_pypy.py -no-t -no-c targetpypy
Analysing target as defined by targetpypy
faking <type 'unicode'>
faking <type 'module'>
faking <type 'file'>
*builtin*
*sys*
*parser*
*GetSetProperty*
*Member*
*Ellipsis*
*method*
*function*
*generator*
*staticmethod*
*ControlFlowException*
*NotImplemented*
*traceback*
*code*
*cell*
*internal-code*
*internal-frame*
*module*
*frame*
*list*
*type*
*tuple*
*long*
*str*
*float*
*int*
*dict*
*object*
*unicode*
*NoneType*
*slice*
*bool*
*sequenceiterator*
*basestring*
*dictproxy*
cache build finished
getflowgraph (?:7) entry_point
getflowgraph (pypy.objspace.std.intobject:15) __init__
getflowgraph (pypy.objspace.descroperation:460) binop_mul_impl
getflowgraph (pypy.objspace.std.objspace:264) type
getflowgraph (pypy.objspace.std.model:108) __init__
getflowgraph (pypy.interpreter.baseobjspace:28) getclass
(pypy.objspace.std.model:108) __init__ -> SomePBC(const=None)
getflowgraph (pypy.objspace.std.objspace:159) gettypeobject
getflowgraph (pypy.interpreter.baseobjspace:102) loadfromcache
getflowgraph (pypy.interpreter.baseobjspace:102) loadfromcache
getflowgraph (pypy.tool.cache:21) getorbuild
getflowgraph (pypy.tool.cache:21) getorbuild
(pypy.objspace.std.intobject:15) __init__ -> SomePBC(const=None)
(pypy.tool.cache:21) getorbuild -> SomeInstance(can_be_None=False, 
classdef=W_TypeObject)
(pypy.interpreter.baseobjspace:102) loadfromcache -> 
SomeInstance(can_be_None=False, classdef=W_TypeObject)
(pypy.objspace.std.objspace:159) gettypeobject -> 
SomeInstance(can_be_None=False, classdef=W_TypeObject)
(pypy.interpreter.baseobjspace:28) getclass -> 
SomeInstance(can_be_None=False, classdef=W_TypeObject)
(pypy.objspace.std.objspace:264) type -> SomeInstance(can_be_None=False, 
classdef=W_TypeObject)
...
...
...
(pypy.objspace.std.listsort:119) gallop -> SomeInteger(nonneg=False, 
unsigned=False)
(pypy.objspace.std.objspace:166) wrap__SomePBC__SI_ControlFlowException 
-> SomeInstance(can_be_None=False, classdef=BaseWrappable)
(pypy.objspace.std.listsort:428) merge_at -> SomePBC(const=None)
getflowgraph (pypy.objspace.std.listsort:216) merge_lo
(pypy.objspace.std.objspace:166) wrap__SomePBC__SI_SApplicationException 
-> SomeInstance(can_be_None=False, classdef=BaseWrappable)
(pypy.interpreter.pyframe:415) unroll -> SomeBool(const=True)
getflowgraph (pypy.objspace.std.listsort:319) merge_hi
(pypy.interpreter.pyframe:382) unroll -> SomeBool()
getflowgraph (pypy.objspace.std.listsort:553) copyitems
(pypy.objspace.std.listsort:553) copyitems -> 
SomeInstance(can_be_None=False, classdef=ListSlice)
getflowgraph (pypy.objspace.std.listsort:567) popright
getflowgraph (pypy.objspace.std.listsort:561) popleft
(pypy.objspace.std.listsort:561) popleft -> 
SomeInstance(can_be_None=True, classdef=W_Root)
(pypy.objspace.std.listsort:567) popright -> 
SomeInstance(can_be_None=True, classdef=W_Root)
(pypy.objspace.std.listsort:216) merge_lo -> SomePBC(const=None)
(pypy.objspace.std.listsort:319) merge_hi -> SomePBC(const=None)
======================================================================
Functions that have SomeObject in their signature
======================================================================
PyCode._from_code(SomeInstance, SomeObject, const SomeBool) -> SomeInstance
pypy.interpreter.pycode:101

PyInterpFrame.MISSING_OPCODE(SomeInstance, SomeObject) -> ?
pypy.interpreter.pyopcode:695

StdObjSpace.unwrap(const SomePBC, SomeInstance) -> SomeObject
pypy.objspace.std.objspace:223

StdObjSpace.wrap__SomePBC__SomeObject(const SomePBC, SomeObject) -> 
SomeInstance
pypy.objspace.std.objspace:166

W_FakeFile.__init__(SomeInstance, const SomePBC, SomeObject) -> const 
SomePBC
pypy.objspace.std.fake:98

W_FakeFile.unwrap(SomeInstance) -> SomeObject
pypy.objspace.std.fake:101

W_FakeModule.unwrap(SomeInstance) -> SomeObject
pypy.objspace.std.fake:101

W_FakeUnicode.__init__(SomeInstance, const SomePBC, SomeObject) -> const 
SomePBC
pypy.objspace.std.fake:98

W_FakeUnicode.unwrap(SomeInstance) -> SomeObject
pypy.objspace.std.fake:101

W_LongObject.longval(SomeInstance) -> SomeObject
pypy.objspace.std.longobject:32

W_LongObject.unwrap(SomeInstance) -> SomeObject
pypy.objspace.std.longobject:39

W_TupleObject.unwrap(SomeInstance) -> SomeObject
pypy.objspace.std.tupleobject:20

W_TypeObject.unwrap(SomeInstance) -> SomeObject
pypy.objspace.std.typeobject:212

args_from_long(SomeObject) -> SomeTuple
pypy.objspace.std.longobject:527

get_unique_interplevel_subclass(SomePBC, SomeObject, SomeBool) -> SomePBC
pypy.interpreter.typedef:23

string_to_long(SomeString, SomeInteger) -> SomeObject
pypy.objspace.std.strutil:77

======================================================================
somobjectness:  0 percent
(16 out of 2951 functions get or return SomeObjects
======================================================================
------------------------------------------------------------
Not generating C code.
------------------------------------------------------------
Done.




From mwh at python.net  Sun Apr 24 21:04:59 2005
From: mwh at python.net (Michael Hudson)
Date: Sun, 24 Apr 2005 20:04:59 +0100
Subject: [pypy-dev] Re: annotation status
References: <426BD38E.6070108@strakt.com>
Message-ID: <2md5sk3sis.fsf@starship.python.net>

Samuele Pedroni <pedronis at strakt.com> writes:

> I think people not following IRC/pyp-svn etc may still be interested
> in this piece of information about progress.
>
> As of end of last week the annotator is able to type-annotate all of
> the pypy codebase, at least as loaded by goal/buildcache2.py
> and goal/targetpypy.py, that means the interpreter plus the standard
> object space plus interp-level modules.

Wow.

> We still get some expected warnings in areas touched by faking, and
> some of that code may be delicate to translate.

Does special-casing in the translator code suffice?

Cheers,
mwh

-- 
7. It is easier to write an incorrect program than understand a
   correct one.
  -- Alan Perlis, http://www.cs.yale.edu/homes/perlis-alan/quotes.html



From riaan at e.co.za  Mon Apr 25 00:37:39 2005
From: riaan at e.co.za (Riaan Booysen)
Date: Mon, 25 Apr 2005 00:37:39 +0200
Subject: [pypy-dev] whitespace policies
In-Reply-To: <20050424125108.GA19874@vicky.ecs.soton.ac.uk>
References: <20050421182821.GR22996@solar.trillke.net>
	<20050424125108.GA19874@vicky.ecs.soton.ac.uk>
Message-ID: <426C1FB3.5030603@e.co.za>

Hi,

Just FYI, reindent.py removes trailing whitespace on a line.

Cheers,
Riaan.

Armin Rigo wrote:
> Hi,
> 
> On Thu, Apr 21, 2005 at 08:28:21PM +0200, holger krekel wrote:
> 
>>Now my question is, do we want to "enforce" or at least 
>>state in our coding-style document one policy or the other?  
> 
> 
> I'm not too concerned about a few dummy diff entries, even though I'm usually
> avoiding to kill trailing whitespace.  I don't see the point of strict
> policies as long as the source code base is roughly consistent, and I think
> that it is.  This is both about trailing whitespace and indentation -- I mean,
> we know that indentation is 4 spaces and we all follow that rule
> 
> By contrast I'd not enjoy enforced editor configurations, and really despise a
> monster check-in that would render svn blame useless (which, I may add, I find
> extremely useful to find not necessary who but when and with which log message
> a particular line was written).
> 
> 
> Armin
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev
> 



From tismer at stackless.com  Mon Apr 25 12:37:35 2005
From: tismer at stackless.com (Christian Tismer)
Date: Mon, 25 Apr 2005 12:37:35 +0200
Subject: [pypy-dev] whitespace policies
In-Reply-To: <426C1FB3.5030603@e.co.za>
References: <20050421182821.GR22996@solar.trillke.net>	<20050424125108.GA19874@vicky.ecs.soton.ac.uk>
	<426C1FB3.5030603@e.co.za>
Message-ID: <426CC86F.3090409@stackless.com>

Riaan Booysen wrote:

> Hi,
> 
> Just FYI, reindent.py removes trailing whitespace on a line.

Hi Riaan!

nice to see you again, Mr. Boa!!! :-)

-- 
Christian Tismer             :^)   <mailto:tismer at stackless.com>
tismerysoft GmbH             :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9A     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 802 86 56  mobile +49 173 24 18 776  fax +49 30 80 90 57 05
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/


From arigo at tunes.org  Tue Apr 26 18:38:58 2005
From: arigo at tunes.org (Armin Rigo)
Date: Tue, 26 Apr 2005 17:38:58 +0100
Subject: [pypy-dev] Mini-sprint 15-19th May
Message-ID: <20050426163858.GA32038@vicky.ecs.soton.ac.uk>

Hi all,

According to The Plan, the next official deadline is to come up before the end
of May with a partial implementation of Python in Python that passes 90% of
the "core language" CPython tests (that is, without counting the ones relying
on C extension modules from the standard library).

As the ACCU sprint was cancelled, we have come up with a possible replacement
mini-sprint.  The primary goal is to finish off some of the remaining test
failures, and polish a bit the release.  If we have enough time we want to
work on near/midterm-future issues and EuroPython planning etc.  The sprint
targets only people already familiar with the PyPy code base and the coding
style in general (this seems sensible because we have a release goal
associated with the sprint).

The sprint could possibly take place at Trillke-Gut Hildesheim or in
Gothenborg, on the 15th-20th of May.  Samuele and Bea will check out
accomodation possibilities.

Early next week (say the 4th of May) we decide if we actually do the sprint,
based on a closer examination of the current test failures.


A bientot,

Armin.


From arigo at tunes.org  Wed Apr 27 12:12:03 2005
From: arigo at tunes.org (Armin Rigo)
Date: Wed, 27 Apr 2005 11:12:03 +0100
Subject: [pypy-dev] Fwd: michele.simionato@gmail.com: ACCU conference
	aftermath
Message-ID: <20050427101203.GA32390@vicky.ecs.soton.ac.uk>

Hi all,

I took the liberty to forward here the following e-mail from Michele 
Simionato, as it contains interesting links.

-----

Hi Armin!

I much enjoyed your talks and I have posted my impressions about
them on c.l.py.
(http://groups-beta.google.com/group/comp.lang.python/browse_frm/thread/72446ebe0271f3cf/80aa801f642f1006?hl=en#80aa801f642f1006)

As I told you in the conference, I think you should check in the Lisp/Scheme
world where there is an astonishingly amount of knowledge about JIT and
in general about compiling to C code fast.

Relevant projects I know of are:

- SBCL, a JIT compiler for Common Lisp;
- Bigloo, a high performance Scheme->C compiler;
- Chicken, another easy Scheme->C compiler.

All these projects are free software and they are actually working products,
not just experimental stuff. SBCL is mostly developed in the US
I think (it is the successor to the Carnegie Mellon Common Lisp), Bigloo
is developed in France by Manuel Serrano whereas Chicken is developed
in Germany by Felix Winkelman. I am an user of Chicken and I think
Felix is a extremely nice and helpful guy.

Here are a few links:

http://sbcl.sourceforge.net/
http://www-sop.inria.fr/mimosa/fp/Bigloo/
http://www.call-with-current-continuation.org/

<cut end of mail>


From mwh at python.net  Wed Apr 27 12:56:20 2005
From: mwh at python.net (Michael Hudson)
Date: Wed, 27 Apr 2005 11:56:20 +0100
Subject: [pypy-dev] Re: Fwd: michele.simionato@gmail.com: ACCU conference
	aftermath
In-Reply-To: <20050427101203.GA32390@vicky.ecs.soton.ac.uk> (Armin Rigo's
	message of "Wed, 27 Apr 2005 11:12:03 +0100")
References: <20050427101203.GA32390@vicky.ecs.soton.ac.uk>
Message-ID: <2mfyxc32uj.fsf@starship.python.net>

Armin Rigo <arigo at tunes.org> writes:

> Hi all,
>
> I took the liberty to forward here the following e-mail from Michele 
> Simionato, as it contains interesting links.

I think Michele is using a different definition of "JIT" to me...

> As I told you in the conference, I think you should check in the
> Lisp/Scheme world where there is an astonishingly amount of
> knowledge about JIT and in general about compiling to C code fast.

While not disputing the general thrust of this, I would say that I am
aware of *no* work on using run-time information to improve
compilation in a Lisp or a Scheme system (this surprises me, and I
have looked for it a bit, but I may have missed something).

Also, very few serious common lisp implementations go via C (the only
one I can think of that does is GCL).

Armed Bear Common Lisp is essentially Jython for CL (it compiles to
bytecodes, not Java).

> Relevant projects I know of are:
>
> - SBCL, a JIT compiler for Common Lisp;

SBCL is not a JIT compiler in the sense I understand the words.

> - Bigloo, a high performance Scheme->C compiler;
> - Chicken, another easy Scheme->C compiler.

I don't know much about bigloo.  Chicken is surely interesting, but I
don't know how much type inference it does.  I think it has a cute GC
implementation (the "cheney on the mta" trick).

> All these projects are free software and they are actually working products,
> not just experimental stuff. SBCL is mostly developed in the US
> I think (it is the successor to the Carnegie Mellon Common Lisp),

Actually I think most current work is done by Dan Barlow, Christophe
Rhodes (who both live in London) and Alexey Dejneka (who is Russian),
althought the project lead William Newman is indeed American.

> Bigloo is developed in France by Manuel Serrano whereas Chicken is
> developed in Germany by Felix Winkelman. I am an user of Chicken and
> I think Felix is a extremely nice and helpful guy.

I remember him that way from way back when I used to read
comp.lang.scheme.

Cheers,
mwh

-- 
  Hey, if I thought I was wrong, I'd change my mind.  :)
                                    -- Grant Edwards, comp.lang.python


From steve at shrogers.com  Wed Apr 27 13:38:45 2005
From: steve at shrogers.com (Steven H. Rogers)
Date: Wed, 27 Apr 2005 05:38:45 -0600
Subject: [pypy-dev] Re: Fwd: michele.simionato@gmail.com: ACCU conference
	aftermath
In-Reply-To: <2mfyxc32uj.fsf@starship.python.net>
References: <20050427101203.GA32390@vicky.ecs.soton.ac.uk>
	<2mfyxc32uj.fsf@starship.python.net>
Message-ID: <426F79C5.2090706@shrogers.com>



Michael Hudson wrote:
> Armin Rigo <arigo at tunes.org> writes:
> 
> 
>>Hi all,
>>
>>I took the liberty to forward here the following e-mail from Michele 
>>Simionato, as it contains interesting links.
> 
> I think Michele is using a different definition of "JIT" to me...
> 
>>As I told you in the conference, I think you should check in the
>>Lisp/Scheme world where there is an astonishingly amount of
>>knowledge about JIT and in general about compiling to C code fast.
> 
> 
> While not disputing the general thrust of this, I would say that I am
> aware of *no* work on using run-time information to improve
> compilation in a Lisp or a Scheme system (this surprises me, and I
> have looked for it a bit, but I may have missed something).
> 

While not automated like a Java JIT, the compiler is an integral part of the 
language in Lisp and may be used by the application programmer to compile 
code at run time.  SBCL uses this in it's REPL to give the illusion of an 
interpreter; each expression is compiled as it is read.  AFAIK, run time 
optimizations are the responsibility of the application.

> Also, very few serious common lisp implementations go via C (the only
> one I can think of that does is GCL).
> 

CLISP is written in largely in C.  SBCL has replaced much of the assembler 
in CMUCL with C for greater portability.

> 
>>Relevant projects I know of are:
>>
>>- SBCL, a JIT compiler for Common Lisp;
> 
> 
> SBCL is not a JIT compiler in the sense I understand the words.
> 

What is your understanding of JIT compiler?

-- 
Steven H. Rogers, Ph.D., steve at shrogers.com
Weblog: http://shrogers.com/weblog
"Reach low orbit and you're half way to anywhere in the Solar System."
-- Robert A. Heinlein



From mwh at python.net  Wed Apr 27 14:13:17 2005
From: mwh at python.net (Michael Hudson)
Date: Wed, 27 Apr 2005 13:13:17 +0100
Subject: [pypy-dev] Re: Fwd: michele.simionato@gmail.com: ACCU conference
	aftermath
References: <20050427101203.GA32390@vicky.ecs.soton.ac.uk>
	<2mfyxc32uj.fsf@starship.python.net>
	<426F79C5.2090706@shrogers.com>
Message-ID: <2mbr802zaa.fsf@starship.python.net>

"Steven H. Rogers" <steve at shrogers.com> writes:

> Michael Hudson wrote:
>> Armin Rigo <arigo at tunes.org> writes:
>> 
>>>Hi all,
>>>
>>> I took the liberty to forward here the following e-mail from
>>> Michele Simionato, as it contains interesting links.
>> I think Michele is using a different definition of "JIT" to me...
>> 
>>>As I told you in the conference, I think you should check in the
>>>Lisp/Scheme world where there is an astonishingly amount of
>>>knowledge about JIT and in general about compiling to C code fast.
>> While not disputing the general thrust of this, I would say that I
>> am
>> aware of *no* work on using run-time information to improve
>> compilation in a Lisp or a Scheme system (this surprises me, and I
>> have looked for it a bit, but I may have missed something).
>> 
>
> While not automated like a Java JIT, the compiler is an integral part
> of the language in Lisp and may be used by the application programmer
> to compile code at run time.

Eh, this sounds too hostile, but: were you under the impression that I
didn't know this?

>> Also, very few serious common lisp implementations go via C (the only
>> one I can think of that does is GCL).
>> 
>
> CLISP is written in largely in C.  SBCL has replaced much of the
> assembler in CMUCL with C for greater portability.

That's not what I or Michele said, though: GCL translates the code the
user supplies into C and calls gcc to compile it to object code (I
understand; I've never used it).

>> 
>>>Relevant projects I know of are:
>>>
>>>- SBCL, a JIT compiler for Common Lisp;
>> SBCL is not a JIT compiler in the sense I understand the words.
>> 
>
> What is your understanding of JIT compiler?

Well, at the minimum performing compiling-type activities at what the
user would consider runtime, but unless this takes advantage of
runtime information it's fairly pointless.  I don't count typing code
into the REPL for this -- do you?

Cheers,
mwh

-- 
  NUTRIMAT:  That drink was individually tailored to meet your
             personal requirements for nutrition and pleasure.
    ARTHUR:  Ah.  So I'm a masochist on a diet am I?
                    -- The Hitch-Hikers Guide to the Galaxy, Episode 9



From hpk at trillke.net  Wed Apr 27 15:31:38 2005
From: hpk at trillke.net (holger krekel)
Date: Wed, 27 Apr 2005 15:31:38 +0200
Subject: [pypy-dev] restructuring lib-python-2.3.4 / testing 
Message-ID: <20050427133138.GK22996@solar.trillke.net>

Hi pypy-dev, 

for various reasons it seems like a good idea to 
move to a slightly modified directory structure 
regarding our "compat 2.3.4 library" and tests. 
The idea is to put the completely unmodified 
standard python library into: 

    lib/python-2.3.4 

and put test-wrapping configuration/hacks etc.pp. 
in lib/ instead of modifying an otherwise unmodified 
tree.  

The changed tests and library modules we have in pypy/lib 
could reside where they are now although it might be 
clearer to put them into 

    lib/python-2.3.4-modified 

instead. This is probably a question of how version
independent our modifications really are. 

Today we also changed the way compliance tests are run (by
default, you get the old behaviour with '-E') because we want
to move on with getting a complete overview of test failures.
If you run py.test (or pypy/test_all.py) in the python-2.3.4
library then your test runs will get recorded into a new
'result' directory. 

With the above proposed location i would suggest
to put this result directory into: 

    lib/python-2.3.4-testresult/ 

If there are no objections (even better: agreement :-) 
i am doing that this night or tomorrow. 

cheers, 

    holger


From sanxiyn at gmail.com  Thu Apr 28 02:41:42 2005
From: sanxiyn at gmail.com (Sanghyeon Seo)
Date: Thu, 28 Apr 2005 09:41:42 +0900
Subject: [pypy-dev] Re: Fwd: michele.simionato@gmail.com: ACCU conference
	aftermath
In-Reply-To: <2mfyxc32uj.fsf@starship.python.net>
References: <20050427101203.GA32390@vicky.ecs.soton.ac.uk>
	<2mfyxc32uj.fsf@starship.python.net>
Message-ID: <5b024817050427174151cba9d3@mail.gmail.com>

Michele Simionato wrote:
> > As I told you in the conference, I think you should check in the
> > Lisp/Scheme world where there is an astonishingly amount of
> > knowledge about JIT and in general about compiling to C code fast.

Michael Hudson wrote:
> While not disputing the general thrust of this, I would say that I am
> aware of *no* work on using run-time information to improve
> compilation in a Lisp or a Scheme system (this surprises me, and I
> have looked for it a bit, but I may have missed something).

This surprises me too.

> Also, very few serious common lisp implementations go via C (the only
> one I can think of that does is GCL).

Yes, GCL compiles to C, and there's also ECL (which I found to be interesting
recently): http://ecls.sourceforge.net/

In general, all KCL(Kyoto Common Lisp)-derived CL implementations go
via C, and they all share the common root.

Seo Sanghyeon


From steve at shrogers.com  Thu Apr 28 04:32:41 2005
From: steve at shrogers.com (Steven H. Rogers)
Date: Wed, 27 Apr 2005 20:32:41 -0600
Subject: [pypy-dev] Re: Fwd: michele.simionato@gmail.com: ACCU conference
	aftermath
In-Reply-To: <2mbr802zaa.fsf@starship.python.net>
References: <20050427101203.GA32390@vicky.ecs.soton.ac.uk>	<2mfyxc32uj.fsf@starship.python.net>	<426F79C5.2090706@shrogers.com>
	<2mbr802zaa.fsf@starship.python.net>
Message-ID: <42704B49.4090108@shrogers.com>



Michael Hudson wrote:
> "Steven H. Rogers" <steve at shrogers.com> writes:
> 
> 
>>Michael Hudson wrote:
>>
>>>Armin Rigo <arigo at tunes.org> writes:
>>>
>>>
>>>>Hi all,
>>>>
>>>>I took the liberty to forward here the following e-mail from
>>>>Michele Simionato, as it contains interesting links.
>>>
>>>I think Michele is using a different definition of "JIT" to me...
>>>
>>>
>>>>As I told you in the conference, I think you should check in the
>>>>Lisp/Scheme world where there is an astonishingly amount of
>>>>knowledge about JIT and in general about compiling to C code fast.
>>>
>>>While not disputing the general thrust of this, I would say that I
>>>am
>>>aware of *no* work on using run-time information to improve
>>>compilation in a Lisp or a Scheme system (this surprises me, and I
>>>have looked for it a bit, but I may have missed something).
>>>
>>
>>While not automated like a Java JIT, the compiler is an integral part
>>of the language in Lisp and may be used by the application programmer
>>to compile code at run time.
> 
> 
> Eh, this sounds too hostile, but: were you under the impression that I
> didn't know this?
> 

No hostile intent, just curious.  What made you think it hostile?  You did 
give the impression that you were not aware of it, which seemed surprising.

> 
>>>Also, very few serious common lisp implementations go via C (the only
>>>one I can think of that does is GCL).
>>>
>>
>>CLISP is written in largely in C.  SBCL has replaced much of the
>>assembler in CMUCL with C for greater portability.
> 
> 
> That's not what I or Michele said, though: GCL translates the code the
> user supplies into C and calls gcc to compile it to object code (I
> understand; I've never used it).
> 
> 
>>>>Relevant projects I know of are:
>>>>
>>>>- SBCL, a JIT compiler for Common Lisp;
>>>
>>>SBCL is not a JIT compiler in the sense I understand the words.
>>>
>>
>>What is your understanding of JIT compiler?
> 
> 
> Well, at the minimum performing compiling-type activities at what the
> user would consider runtime, but unless this takes advantage of
> runtime information it's fairly pointless.  I don't count typing code
> into the REPL for this -- do you?
> 

Natively compiling Lisps can perform all compiling type operations at 
runtime, but it's up to the application programmer to specify this.  Code 
typed into the REPL will be compiled using the information available in the 
environment at that point in time, so this should qualify as JIT compilation 
for this limited application.

Regards,
Steve
-- 
Steven H. Rogers, Ph.D., steve at shrogers.com
Weblog: http://shrogers.com/weblog
"Reach low orbit and you're half way to anywhere in the Solar System."
-- Robert A. Heinlein



From mwh at python.net  Thu Apr 28 12:31:02 2005
From: mwh at python.net (Michael Hudson)
Date: Thu, 28 Apr 2005 11:31:02 +0100
Subject: [pypy-dev] Re: Fwd: michele.simionato@gmail.com: ACCU conference
	aftermath
References: <20050427101203.GA32390@vicky.ecs.soton.ac.uk>
	<2mfyxc32uj.fsf@starship.python.net>
	<5b024817050427174151cba9d3@mail.gmail.com>
Message-ID: <2moebz19cp.fsf@starship.python.net>

Sanghyeon Seo <sanxiyn at gmail.com> writes:

> Michael Hudson wrote:
>> Also, very few serious common lisp implementations go via C (the only
>> one I can think of that does is GCL).
>
> Yes, GCL compiles to C, and there's also ECL (which I found to be
> interesting recently): http://ecls.sourceforge.net/

Oh yes, I'd forgotten about that one.  Thanks.

> In general, all KCL(Kyoto Common Lisp)-derived CL implementations go
> via C, and they all share the common root.

Yes, but GCL is the only one of these that's still around in a
meaningful sense, isn't it?

Cheers,
mwh

-- 
  You sound surprised.  We're talking about a government department
  here - they have procedures, not intelligence.
                                            -- Ben Hutchings, cam.misc



From steve at shrogers.com  Thu Apr 28 14:00:15 2005
From: steve at shrogers.com (Steven H. Rogers)
Date: Thu, 28 Apr 2005 06:00:15 -0600
Subject: [pypy-dev] Re: Fwd: michele.simionato@gmail.com: ACCU conference
	aftermath
In-Reply-To: <2moebz19cp.fsf@starship.python.net>
References: <20050427101203.GA32390@vicky.ecs.soton.ac.uk>	<2mfyxc32uj.fsf@starship.python.net>	<5b024817050427174151cba9d3@mail.gmail.com>
	<2moebz19cp.fsf@starship.python.net>
Message-ID: <4270D04F.3060204@shrogers.com>

Not Common Lisp, but Lush (http://lush.sf.net) is a Lisp dialect that 
compiles natively using using C as an intermediate language.  It was 
developed as a scripting language for machine learning research and has been 
used in commercial applications.  It also alows inline use of C.

Linj (Lisp is not Java, http://www.evaluator.pt/downloads/tutorial.html) 
translates from idiomatic Lisp to idiomatic Java and back, allowing Lisp 
programmers to work on Java projects without writing any Java code.  Haven't 
tried it, so I don't know how well it lives up to this goal, but you might 
find it interesting/useful.

Michael Hudson wrote:
> Sanghyeon Seo <sanxiyn at gmail.com> writes:
> 
> 
>>Michael Hudson wrote:
>>
>>>Also, very few serious common lisp implementations go via C (the only
>>>one I can think of that does is GCL).
>>
>>Yes, GCL compiles to C, and there's also ECL (which I found to be
>>interesting recently): http://ecls.sourceforge.net/
> 
> 
> Oh yes, I'd forgotten about that one.  Thanks.
> 
> 
>>In general, all KCL(Kyoto Common Lisp)-derived CL implementations go
>>via C, and they all share the common root.
> 
> 
> Yes, but GCL is the only one of these that's still around in a
> meaningful sense, isn't it?
> 
> Cheers,
> mwh
> 

-- 
Steven H. Rogers, Ph.D., steve at shrogers.com
Weblog: http://shrogers.com/weblog
"Reach low orbit and you're half way to anywhere in the Solar System."
-- Robert A. Heinlein



From pedronis at strakt.com  Thu Apr 28 14:33:25 2005
From: pedronis at strakt.com (Samuele Pedroni)
Date: Thu, 28 Apr 2005 14:33:25 +0200
Subject: [pypy-dev] Re: Fwd: michele.simionato@gmail.com: ACCU conference
	aftermath
In-Reply-To: <42704B49.4090108@shrogers.com>
References: <20050427101203.GA32390@vicky.ecs.soton.ac.uk>	<2mfyxc32uj.fsf@starship.python.net>	<426F79C5.2090706@shrogers.com>	<2mbr802zaa.fsf@starship.python.net>
	<42704B49.4090108@shrogers.com>
Message-ID: <4270D815.3020600@strakt.com>

Steven H. Rogers wrote:

>
>
> Michael Hudson wrote:
>
>> "Steven H. Rogers" <steve at shrogers.com> writes:
>>
>>
>>> Michael Hudson wrote:
>>>
>>>> Armin Rigo <arigo at tunes.org> writes:
>>>>
>>>>
>>>>> Hi all,
>>>>>
>>>>> I took the liberty to forward here the following e-mail from
>>>>> Michele Simionato, as it contains interesting links.
>>>>
>>>>
>>>> I think Michele is using a different definition of "JIT" to me...
>>>>
>>>>
>>>>> As I told you in the conference, I think you should check in the
>>>>> Lisp/Scheme world where there is an astonishingly amount of
>>>>> knowledge about JIT and in general about compiling to C code fast.
>>>>
>>>>
>>>> While not disputing the general thrust of this, I would say that I
>>>> am
>>>> aware of *no* work on using run-time information to improve
>>>> compilation in a Lisp or a Scheme system (this surprises me, and I
>>>> have looked for it a bit, but I may have missed something).
>>>>
>>>
>>> While not automated like a Java JIT, the compiler is an integral part
>>> of the language in Lisp and may be used by the application programmer
>>> to compile code at run time.
>>
>>
>>
>> Eh, this sounds too hostile, but: were you under the impression that I
>> didn't know this?
>>
>
> No hostile intent, just curious.  What made you think it hostile?  You 
> did give the impression that you were not aware of it, which seemed 
> surprising.
>
>>
>>>> Also, very few serious common lisp implementations go via C (the only
>>>> one I can think of that does is GCL).
>>>>
>>>
>>> CLISP is written in largely in C.  SBCL has replaced much of the
>>> assembler in CMUCL with C for greater portability.
>>
>>
>>
>> That's not what I or Michele said, though: GCL translates the code the
>> user supplies into C and calls gcc to compile it to object code (I
>> understand; I've never used it).
>>
>>
>>>>> Relevant projects I know of are:
>>>>>
>>>>> - SBCL, a JIT compiler for Common Lisp;
>>>>
>>>>
>>>> SBCL is not a JIT compiler in the sense I understand the words.
>>>>
>>>
>>> What is your understanding of JIT compiler?
>>
>>
>>
>> Well, at the minimum performing compiling-type activities at what the
>> user would consider runtime, but unless this takes advantage of
>> runtime information it's fairly pointless.  I don't count typing code
>> into the REPL for this -- do you?
>>
>
> Natively compiling Lisps can perform all compiling type operations at 
> runtime, but it's up to the application programmer to specify this.  
> Code typed into the REPL will be compiled using the information 
> available in the environment at that point in time, so this should 
> qualify as JIT compilation for this limited application.

given that is hard in that case to distinguish definition time from 
execution time and for just a defun what happens is mostly compilaton 
and not execution it is hard to pin that down. Conceptually I think 
compilation in that case happens before execution so it is not a JIT. 
JIT as terminology makes most sense when there is a some intermediate 
form which is not source code that is compiled as needed at runtime, and 
the intermediate form could be executed in some other way, and is 
produced in a phase that is distinguished from runtime (In Smalltalk 
IDEs traditionally bytecode is produced when modying methods and native 
compilation happens when those are run). So it makes most sense for 
VMs,  for example a C++ compiler taylored for compilation speed to give 
fast tournaround in an IDE is not typically classified as JIT. Until 
considerations of dynamic optimization come into the picture there is 
nothing intrisically good or bad about JITs. Many Common Lisp sytstems 
have native compilers and nice interactive modes that in some impl can 
use just the native compiler and not interpretation, but using the term 
JIT seem more confusing than useful. I think the crucial point here is 
the linking model (through symbols) which allows functions to be easily 
self-contained compilation units.


From arigo at tunes.org  Fri Apr 29 11:18:25 2005
From: arigo at tunes.org (Armin Rigo)
Date: Fri, 29 Apr 2005 10:18:25 +0100
Subject: [pypy-dev] restructuring lib-python-2.3.4 / testing
In-Reply-To: <20050427133138.GK22996@solar.trillke.net>
References: <20050427133138.GK22996@solar.trillke.net>
Message-ID: <20050429091825.GA31364@vicky.ecs.soton.ac.uk>

Hi Holger,

On Wed, Apr 27, 2005 at 03:31:38PM +0200, holger krekel wrote:
> With the above proposed location i would suggest
> to put this result directory into: 
> 
>     lib/python-2.3.4-testresult/ 
> 
> If there are no objections (even better: agreement :-) 
> i am doing that this night or tomorrow. 

In complement with what you said: the idea here is that if you go to
lib-python-2.3.4/test/ and run py.test (with or without specific test names),
by default the test is run "the slow way" (i.e. by running py.py test_xyz.py)  
and its output is recorded in a file result/test_xyz.txt, together with some
information like revision number, date/time, etc.

Holger: I am now thinking that you were absolutely right when you suggested
one test-result directory per user.  I re-ran some tests, now I do an 'svn
up', and I get conflicts, of course.

One directory per user at hostname would even be better.  However, this will
create a whole lot of data, so it might be best to put it at some place
outside the pypy/dist root, if possible...


Armin


From arigo at tunes.org  Fri Apr 29 11:40:13 2005
From: arigo at tunes.org (Armin Rigo)
Date: Fri, 29 Apr 2005 10:40:13 +0100
Subject: [pypy-dev] Re: Fwd: michele.simionato@gmail.com: ACCU conference
	aftermath
In-Reply-To: <2mfyxc32uj.fsf@starship.python.net>
References: <20050427101203.GA32390@vicky.ecs.soton.ac.uk>
	<2mfyxc32uj.fsf@starship.python.net>
Message-ID: <20050429094013.GB31364@vicky.ecs.soton.ac.uk>

Hi Michael,

On Wed, Apr 27, 2005 at 11:56:20AM +0100, Michael Hudson wrote:
> > I took the liberty to forward here the following e-mail from Michele 
> > Simionato, as it contains interesting links.
> 
> I think Michele is using a different definition of "JIT" to me...

Thanks to Michael and others for reminding us that -- althought my own nose is
very much stuck in the details of C code generation at the moment -- this is
not the goal of PyPy per se but just an intermediate phase before we can do
much better.  I know the last talks I gave about PyPy were about that, but
that's because that's where we are now working, and it's the first tangible
result we can demonstrate (the alternative being yet another "we-can-do-it"  
talk "supported" by prototypes 2000 times slower than CPython...).

There are lots interesting projects out there about various forms of
compilation and optimization, and PyPy looks quite similar to some of them,
particularly the "elegant" ones, which makes the analogy all the more
tempting.  But to compete with these systems we'd have to put a real lot of
efforts into generating the best possible C code.

Looking at extremely powerful static compiler, even if the "math" part in me
thinks about how nice and elegant they are, another part in me ends up feeling
like it's all wasted efforts...


A bient?t,

Armin.


From hpk at trillke.net  Fri Apr 29 11:42:27 2005
From: hpk at trillke.net (holger krekel)
Date: Fri, 29 Apr 2005 11:42:27 +0200
Subject: [pypy-dev] restructuring lib-python-2.3.4 / testing
In-Reply-To: <20050429091825.GA31364@vicky.ecs.soton.ac.uk>
References: <20050427133138.GK22996@solar.trillke.net>
	<20050429091825.GA31364@vicky.ecs.soton.ac.uk>
Message-ID: <20050429094227.GM22996@solar.trillke.net>

Hi Armin, 

On Fri, Apr 29, 2005 at 10:18 +0100, Armin Rigo wrote:
> On Wed, Apr 27, 2005 at 03:31:38PM +0200, holger krekel wrote:
> > With the above proposed location i would suggest
> > to put this result directory into: 
> > 
> >     lib/python-2.3.4-testresult/ 
> > 
> > If there are no objections (even better: agreement :-) 
> > i am doing that this night or tomorrow. 
> 
> In complement with what you said: the idea here is that if you go to
> lib-python-2.3.4/test/ and run py.test (with or without specific test names),
> by default the test is run "the slow way" (i.e. by running py.py test_xyz.py)  
> and its output is recorded in a file result/test_xyz.txt, together with some
> information like revision number, date/time, etc.
> 
> Holger: I am now thinking that you were absolutely right when you suggested
> one test-result directory per user.  I re-ran some tests, now I do an 'svn
> up', and I get conflicts, of course.

Sure enough. 

> One directory per user at hostname would even be better.  However, this will
> create a whole lot of data, so it might be best to put it at some place
> outside the pypy/dist root, if possible...

If you put it outside of dist then the usual pypy checkout will not 
contain that directory, alas. 

For the time being, we should either live with the conflicts (not so nice)
or move it to a user at hostname directory.  This will also reduce 
the diffs, making 'svn up' rather cheap in terms of network bandwidth. 

Eventually, we want to have a 'sendreport' option that sends a report 
to codespeak where it gets integrated into an HTML view.  But that 
is not going to happen in the next days so i suggest to just move
to the user at hostname directory and move the current stuff to 'mixed at mixed'.

cheers, 

    holger


From arigo at tunes.org  Fri Apr 29 12:13:42 2005
From: arigo at tunes.org (Armin Rigo)
Date: Fri, 29 Apr 2005 11:13:42 +0100
Subject: [pypy-dev] restructuring lib-python-2.3.4 / testing
In-Reply-To: <20050429094227.GM22996@solar.trillke.net>
References: <20050427133138.GK22996@solar.trillke.net>
	<20050429091825.GA31364@vicky.ecs.soton.ac.uk>
	<20050429094227.GM22996@solar.trillke.net>
Message-ID: <20050429101342.GA13157@vicky.ecs.soton.ac.uk>

Hi Holger,

On Fri, Apr 29, 2005 at 11:42:27AM +0200, holger krekel wrote:
> If you put it outside of dist then the usual pypy checkout will not 
> contain that directory, alas. 

Yes, that was my intent.  If we have 6 or 7 subdirectories with 1.5MB of text
each (7MB total disk usage in working copy), it makes a check-out heavy,
particularly for people who just want to grab the current version and play a
bit with it.

> For the time being, we should either live with the conflicts (not so nice)
> or move it to a user at hostname directory.  This will also reduce 
> the diffs, making 'svn up' rather cheap in terms of network bandwidth. 

Right, so maybe we shouldn't care about the 'svn co' cost.

> is not going to happen in the next days so i suggest to just move
> to the user at hostname directory and move the current stuff to 'mixed at mixed'.

Ok.


Armin


From hpk at trillke.net  Fri Apr 29 12:39:10 2005
From: hpk at trillke.net (holger krekel)
Date: Fri, 29 Apr 2005 12:39:10 +0200
Subject: [pypy-dev] restructuring lib-python-2.3.4 / testing
In-Reply-To: <20050429101342.GA13157@vicky.ecs.soton.ac.uk>
References: <20050427133138.GK22996@solar.trillke.net>
	<20050429091825.GA31364@vicky.ecs.soton.ac.uk>
	<20050429094227.GM22996@solar.trillke.net>
	<20050429101342.GA13157@vicky.ecs.soton.ac.uk>
Message-ID: <20050429103910.GP22996@solar.trillke.net>

On Fri, Apr 29, 2005 at 11:13 +0100, Armin Rigo wrote:
> Hi Holger,
> 
> On Fri, Apr 29, 2005 at 11:42:27AM +0200, holger krekel wrote:
> > If you put it outside of dist then the usual pypy checkout will not 
> > contain that directory, alas. 
> 
> Yes, that was my intent.  If we have 6 or 7 subdirectories with 1.5MB of text
> each (7MB total disk usage in working copy), it makes a check-out heavy,
> particularly for people who just want to grab the current version and play a
> bit with it.
> 
> > For the time being, we should either live with the conflicts (not so nice)
> > or move it to a user at hostname directory.  This will also reduce 
> > the diffs, making 'svn up' rather cheap in terms of network bandwidth. 
> 
> Right, so maybe we shouldn't care about the 'svn co' cost.

Maybe we should do the following: 

    svn ps 'svn:ignore' 'report' lib-python-2.3.4/test 

and then 

    cd lib-python-2.3.4/test
    svn co http://codespeak.net/svn/pypy/testresult result 

and within that we have 
    
    user at hostname 

directories.  An 'svn co pypy-dist' or 'svn up pypy-dist' would not retrieve 
the test report files. you have to explicitely 'svn co/up report' 
which we could of course automate (at least on systems that have a 'svn' 
binary in their path). 

What do you think? 

cheers, 

    holger


From arigo at tunes.org  Fri Apr 29 12:59:38 2005
From: arigo at tunes.org (Armin Rigo)
Date: Fri, 29 Apr 2005 11:59:38 +0100
Subject: [pypy-dev] restructuring lib-python-2.3.4 / testing
In-Reply-To: <20050429103910.GP22996@solar.trillke.net>
References: <20050427133138.GK22996@solar.trillke.net>
	<20050429091825.GA31364@vicky.ecs.soton.ac.uk>
	<20050429094227.GM22996@solar.trillke.net>
	<20050429101342.GA13157@vicky.ecs.soton.ac.uk>
	<20050429103910.GP22996@solar.trillke.net>
Message-ID: <20050429105938.GA26575@vicky.ecs.soton.ac.uk>

Hi,

On Fri, Apr 29, 2005 at 12:39:10PM +0200, holger krekel wrote:
>     cd lib-python-2.3.4/test
>     svn co http://codespeak.net/svn/pypy/testresult result 
> 
> and within that we have 
>     
>     user at hostname 
> 
> directories.  An 'svn co pypy-dist' or 'svn up pypy-dist' would not retrieve 
> the test report files. you have to explicitely 'svn co/up report' 
> which we could of course automate (at least on systems that have a 'svn' 
> binary in their path). 

Done.  If you just run py.test now, all the tests are skipped and you get a
nice explanation about what you should do to retreive the 'result' directory
manually.  This way, it is by default kept out of the way, and we can feel
free to check-in lots of stuff in there without worrying about it getting part
of a normal 'svn co' of the 'dist' root.


Armin


From arigo at tunes.org  Fri Apr 29 23:13:03 2005
From: arigo at tunes.org (Armin Rigo)
Date: Fri, 29 Apr 2005 22:13:03 +0100
Subject: [pypy-dev] restructuring lib-python-2.3.4 / testing
In-Reply-To: <20050429091825.GA31364@vicky.ecs.soton.ac.uk>
References: <20050427133138.GK22996@solar.trillke.net>
	<20050429091825.GA31364@vicky.ecs.soton.ac.uk>
Message-ID: <20050429211303.GA14217@vicky.ecs.soton.ac.uk>

Hi all,

On Fri, Apr 29, 2005 at 10:18:25AM +0100, Armin Rigo wrote:
> In complement with what you said: the idea here is that if you go to
> lib-python-2.3.4/test/ and run py.test (with or without specific test names),

In addition: you should say  py.test -D  to run all the tests including the
ones marked as disabled (this is usually what you want now).

Also note that test_mutants eats the KeyboardInterrupt that is supposed to
kill it after 15 minutes.  Oups.


Armin


From hpk at trillke.net  Sat Apr 30 09:02:51 2005
From: hpk at trillke.net (holger krekel)
Date: Sat, 30 Apr 2005 09:02:51 +0200
Subject: [pypy-dev] new unicode checkins / announcing branches
In-Reply-To: <20050429220011.A8BBA27BB7@code1.codespeak.net>
References: <20050429220011.A8BBA27BB7@code1.codespeak.net>
Message-ID: <20050430070251.GV22996@solar.trillke.net>

Hi Anders (Chrigstrom), 
Hi Anders (Lehmann), 

i noticed that both of you are doing unicode related 
checkins.  Could you make sure that you are not 
duplicating efforts?  

In general, i think it's a good idea to advertise 
new branches (like the nice "non-fake-unicode" one) 
on pypy-dev. Better redundant than sorry. 

It's also advisable to at least skim if not read
the pypy-svn list to know what's going on ... 

Um, and does one of you have a nice nickname? :-) 

cheers, 

    holger


From jacek.generowicz at cern.ch  Fri Apr 29 20:38:49 2005
From: jacek.generowicz at cern.ch (Jacek Generowicz)
Date: Fri, 29 Apr 2005 20:38:49 +0200
Subject: [pypy-dev] Re: Fwd: michele.simionato@gmail.com: ACCU conference
	aftermath
In-Reply-To: <2moebz19cp.fsf@starship.python.net>
References: <20050427101203.GA32390@vicky.ecs.soton.ac.uk>
	<2mfyxc32uj.fsf@starship.python.net>
	<5b024817050427174151cba9d3@mail.gmail.com>
	<2moebz19cp.fsf@starship.python.net>
Message-ID: <8515b25171a0ab1208cc5f349537d039@cern.ch>


On 28 Apr 2005, at 12:31, Michael Hudson wrote:

> Yes, but GCL is the only one of these that's still around in a 
> meaningful sense, isn't it?

ECL is very much meaningful. If anything, I think that it has been more 
meaningful than GCL for quite some time, though GCL seems to have 
experienced a bit of a revival. Not being an active user of either, my 
impressions could be a bit warped, though.