From theller at python.net  Fri Jul  1 13:34:51 2005
From: theller at python.net (Thomas Heller)
Date: Fri, 01 Jul 2005 13:34:51 +0200
Subject: [pypy-dev] Re: worth reading: Steve Jobs
References: <42C44136.2030208@stackless.com>
Message-ID: <zmt67o9w.fsf@python.net>

Christian Tismer <tismer at stackless.com> writes:

> hi friends,
>
> the following is a so-called "commencement" talk.
> For those of you who could just enjoy an A-levels talk.
> - this one is one done right! -
> for those of you who feel too old to read this: show it to your kids.
>
> http://news-service.stanford.edu/news/2005/june15/jobs-061505.html
>

Thanks, Christian.  This comes just in time for the talk with my son on
sunday ;-)

Thomas



From Ben.Young at risk.sungard.com  Tue Jul  5 16:57:10 2005
From: Ben.Young at risk.sungard.com (Ben.Young at risk.sungard.com)
Date: Tue, 5 Jul 2005 15:57:10 +0100
Subject: [pypy-dev] os.getuid
In-Reply-To: <20050705154424.CDBC827B45@code1.codespeak.net>
Message-ID: <OF4A759312.7AB8C4CD-ON80257035.0051D226-80257035.00515C21@risk.sungard.com>

Hello PyPyers!

I was just wondering if anyone had noticed that on windows (and cygwin) 
os.getuid doesn't actaully exist, and so I am getting lots of failures.

I see that great progress is being made anyhow! Is there a status report 
due out a the end of the sprint so us observers can see what has been 
done?

Cheers,
Ben


From Ben.Young at risk.sungard.com  Tue Jul  5 17:44:45 2005
From: Ben.Young at risk.sungard.com (Ben.Young at risk.sungard.com)
Date: Tue, 5 Jul 2005 16:44:45 +0100
Subject: [pypy-dev] os.getuid
In-Reply-To: <Pine.LNX.4.61.0507051730190.4895@localhost.localdomain>
Message-ID: <OF24063E3E.0135CDB2-ON80257035.005663E2-80257035.0055B747@risk.sungard.com>

Thanks! I look forward to seeing it. 

Cheers,
Ben


Beatrice During <bea at netg.se> wrote on 05/07/2005 16:32:34:

> Hi there
> 
> There will be a sprint report after the 7th of July.
> You could also look at:
> 
> 
https://codespeak.net/svn/pypy/extradoc/sprintinfo/post-ep2005-planning.txt
> 
> This is the document we are using during the sprint to doc what to 
do/work 
> done.....
> 
> Cheers
> 
> Bea
> 
> On Tue, 5 Jul 2005 Ben.Young at risk.sungard.com wrote:
> 
> > Hello PyPyers!
> >
> > I was just wondering if anyone had noticed that on windows (and 
cygwin)
> > os.getuid doesn't actaully exist, and so I am getting lots of 
failures.
> >
> > I see that great progress is being made anyhow! Is there a status 
report
> > due out a the end of the sprint so us observers can see what has been
> > done?
> >
> > Cheers,
> > Ben
> > _______________________________________________
> > pypy-dev at codespeak.net
> > http://codespeak.net/mailman/listinfo/pypy-dev
> >
> 
> Beatrice D?ring         Change Maker
> Tel: 031- 7750940      J?rntorget 3
> Mobil: 0734- 22 89 06      413 04 G?teborg
> E-post: bea at changemaker.nu   www.changemaker.nu
> 
>     "Alla dessa m?sten och alldaglighet.
>     Allt detta som binder v?r verklighet
>     i bojor av skam och rep utav tv?ng.
>     Alla dessa kedjor som binder v?r s?ng.
>     Jag skall slita dem alla i sm?, sm? stycken
>     och m?jligtvis av resterna g?ra mig smycken."
>                       - hemlig



From lac at strakt.com  Sat Jul  9 10:11:26 2005
From: lac at strakt.com (Laura Creighton)
Date: Sat, 9 Jul 2005 10:11:26 +0200
Subject: [pypy-dev] left in irc
Message-ID: <200507090811.j698BQiq019330@theraft.strakt.com>

amep (~amp at user-12lcgki.cable.mindspring.com) has joined channel #pypy
<amep> I wanted to say how cool I think pypy is. And how much I'm looking forward to a complete release. I'm writing a program that would really be improved by some extra speed and I'm sure pypy will be able to provide that. (Machine code output and all)
 amep has left channel #pypy

Laura


From seberino at spawar.navy.mil  Sun Jul 10 08:22:37 2005
From: seberino at spawar.navy.mil (seberino at spawar.navy.mil)
Date: Sat, 9 Jul 2005 23:22:37 -0700
Subject: [pypy-dev] Please give advice for newbie reading pypy source......
Message-ID: <20050710062237.GA12389@spawar.navy.mil>

PyPy Developers:

I'm really excited about learning PyPy source code really well
and possibly helping the project in some capacity.
Reading source code inevitably leads to tons of questions.

**Any advice on how to progress quickly in PyPy source code understanding?**

Any chance there are PyPy people close to San Diego, California??

Any better method than posting questions to this mailing list and
slowly getting replies to all my questions?

Any help would be greatly appreciated.

Chris


--
_______________________________________

Christian Seberino, Ph.D.
SPAWAR Systems Center San Diego
Code 2872
49258 Mills Street, Room 158
San Diego, CA 92152-5385
U.S.A.

Phone: (619) 553-9973
Fax  : (619) 553-6521
Email: seberino at spawar.navy.mil
_______________________________________


From Ben.Young at risk.sungard.com  Mon Jul 11 11:10:28 2005
From: Ben.Young at risk.sungard.com (Ben.Young at risk.sungard.com)
Date: Mon, 11 Jul 2005 10:10:28 +0100
Subject: [pypy-dev] Build and test failures
Message-ID: <OFBC08038C.E45E0A9E-ON8025703B.00320354-8025703B.003197DF@risk.sungard.com>

Hi Everyone,

Just synced to head this morning and noticed a couple of errors in the c 
translation process.

This first is that on windows, a couple of headers are not available. I 
fixed this by patching locally with:

Index: translator/c/extfunc_include.h
===================================================================
--- translator/c/extfunc_include.h      (revision 14483)
+++ translator/c/extfunc_include.h      (working copy)
@@ -6,9 +6,9 @@
 /******************************************************************/


-#include <unistd.h>
+/*#include <unistd.h>
 #include <sys/types.h>
-#include <sys/stat.h>
+#include <sys/stat.h>*/
 #include <fcntl.h>
 #include <time.h>
 #ifndef PATH_MAX

Then, when the tests are run I get:

$ ./test_all.py translator/c/
============================= test process starts 
=============================
testing-mode: inprocess
executable:   c:\Python24\python.exe  (2.4.1-final-0)
using py lib: c:\Documents and Settings\YoungB\dist\py <rev unknown>

translator\c\test\test_annotated.py[19] ...................
translator\c\test\test_backendoptimized.py[39] 
.......................................
translator\c\test\test_database.py[17] .................
translator\c\test\test_exception.py[3] ...
translator\c\test\test_extfunc.py[4] F...
translator\c\test\test_genc.py[8] ........
translator\c\test\test_lltyped.py[2] ..
translator\c\test\test_notype.py[33] .................................
translator\c\test\test_operation.py[1] .
translator\c\test\test_support.py[7] .......
translator\c\test\test_symboltable.py[1] .
translator\c\test\test_typed.py[38] ......................................

_______________________________________________________________________________
_________________________ entrypoint: test_time_clock 
_________________________

    def test_time_clock():
        def does_stuff():
            return time.clock()
        f1 = compile(does_stuff, [])
        t0 = time.clock()
        t1 = f1()
        assert type(t1) is float
        t2 = time.clock()
E       assert t0 <= t1 <= t2
>       assert 165.0 <= 2.5683609740123038e-005

[c:\Documents and 
Settings\YoungB\dist\pypy\translator\c\test\test_extfunc.py:16]
- - - - - - - - - - -  test_time_clock: recorded stderr - - - - - - - - - 
- - -
[annrpython] (pypy.translator.c.test.test_extfunc:9) does_stuff -> 
SomeFloat()
[annrpython] (pypy.rpython.rclass:575) ll_runtime_type_info__objectPtr -> 
SomePtr(ll_ptrtype=<* RuntimeTypeInfo (opaque)>)
[annrpython] (pypy.rpython.module.ll_time:14) ll_time_clock -> SomeFloat()
[annrpython] (pypy.rpython.rclass:568) 
ll_issubclass__object_vtablePtr_object_vtablePtr -> SomeBool(const=True)
[annrpython] (pypy.rpython.rclass:568) 
ll_issubclass__object_vtablePtr_object_vtablePtr -> SomeBool()
[annrpython] (pypy.rpython.rclass:565) ll_type__objectPtr -> 
SomePtr(ll_ptrtype=<* Struct object_vtable { parenttypeptr: * Struct objec
t_vtable { ... }, rtti: * RuntimeTypeInfo (opaque), name: * Array of Char 
}>)
[annrpython] (pypy.rpython.exceptiondata:120) 
ll_pyexcclass2exc__PyObjectPtr -> SomePtr(const=<* struct object { 
typeptr=... }>, ll_ptr
type=<* GcStruct object { typeptr: * Struct object_vtable { parenttypeptr: 
* Struct object_vtable { ... }, rtti: * RuntimeTypeInfo (opa
que), name: * Array of Char  } }>)
[annrpython] (pypy.rpython.exceptiondata:120) 
ll_pyexcclass2exc__PyObjectPtr -> SomePtr(ll_ptrtype=<* GcStruct object { 
typeptr: * Stru
ct object_vtable { parenttypeptr: * Struct object_vtable { ... }, rtti: * 
RuntimeTypeInfo (opaque), name: * Array of Char  } }>)
[annrpython] (pypy.rpython.exceptiondata:53) ll_raise_OSError__Signed -> 
SomeImpossibleValue()

_______________________________________________________________________________
=========== tests finished: 171 passed, 1 failed in 263.86 seconds 
============

I think this is because when clock is called in the extension it thinks it 
is the first time and returns the very small value. I don't really know 
though.

I hope you all had fun at the sprint!

Cheers,
Ben


From hpk at trillke.net  Tue Jul 12 12:19:33 2005
From: hpk at trillke.net (holger krekel)
Date: Tue, 12 Jul 2005 12:19:33 +0200
Subject: [pypy-dev] Refining terminology
Message-ID: <20050712101933.GR28807@solar.trillke.net>

Hi folks, 

i have just started to write some documentation about PyPy's
bytecode interpreter (only the overview chapter in fact see
http://codespeak.net/pypy/index.cgi?doc/interpreter.html ).
And i realized that i'd like to push to refine the terminology
we are using throughout the PyPy documentation and our
presentations. 

For starters, i think that it does not make sense to fight 
against the common notion of an "interpreter" refering to 
the whole thing (our bytecode interpreter + objectspace).
If we want to refer to the "pure" thing, we should thus talk 
of the "bytecode interpreter".  Usually refering to our 
(Python) Interpreter then really means StdObjSpace + the
bytecode interpreter.  

Other wording issues are the various "times", "levels", 
"phases" and "passes" we are talking about.  I think
we desparately need pictures to support a clear 
and consistent terminology and make it more obvious 
to everyone.  This may become even more important when 
we'll go for a JIT-compiler interweaving levels as well 
as compile- and runtimes some more. 

Well, maybe we should just go through the effort of
creating a "glossary" containing the basic vocabulary
we are using for describing what PyPy does and how
it works.  And i think we should try to reuse common 
VM/Compiler terminology, even refering to Wikipedia and 
other public resources, to make it easier for non-Python/PyPy 
crowds to understand what we are doing. 

I guess that some of you may think that it is only 
real working code that matters (i know some do :-).  
But i think this is not true. Communicating the concepts 
is and has always been a major issue of the whole project. 

Any opinions or feedback (also from people just following
the project's progress)? 

cheers, 

    holger


From rodsenra at gpr.com.br  Tue Jul 12 14:43:25 2005
From: rodsenra at gpr.com.br (Rodrigo Dias Arruda Senra)
Date: Tue, 12 Jul 2005 09:43:25 -0300
Subject: [pypy-dev] Refining terminology
In-Reply-To: <20050712101933.GR28807@solar.trillke.net>
References: <20050712101933.GR28807@solar.trillke.net>
Message-ID: <20050712094325.73e72624@localhost.localdomain>

On Tue, 12 Jul 2005 12:19:33 +0200
holger krekel <hpk at trillke.net> wrote:

> Hi folks, 
> 
> i have just started to write some documentation about PyPy's
> bytecode interpreter (only the overview chapter in fact see
> http://codespeak.net/pypy/index.cgi?doc/interpreter.html ).
> And i realized that i'd like to push to refine the terminology
> we are using throughout the PyPy documentation and our
> presentations. 

> # ... cut ...

> Well, maybe we should just go through the effort of
> creating a "glossary" containing the basic vocabulary
> we are using for describing what PyPy does and how
> it works.  And i think we should try to reuse common 
> VM/Compiler terminology, even refering to Wikipedia and 
> other public resources, to make it easier for non-Python/PyPy 
> crowds to understand what we are doing. 
> 
> I guess that some of you may think that it is only 
> real working code that matters (i know some do :-).  
> But i think this is not true. Communicating the concepts 
> is and has always been a major issue of the whole project. 
> 
> Any opinions or feedback (also from people just following
> the project's progress)? 

From the lurker POV this is indeed very important. Just to
give a real-life example, the computational reflection field
trudged slowly just because nobody had the same definition for
what a "meta-object" really meant.

I'm +1 for the creation of a pypy-abstraction glossary.
Even though tedious, I believe this task  must be an 
**inside-job** == performed by the core-dev team.

OTOH, validate such glossary is up-to-us neopypytes ;o)

best regards,
senra


From arigo at tunes.org  Tue Jul 12 18:20:09 2005
From: arigo at tunes.org (Armin Rigo)
Date: Tue, 12 Jul 2005 18:20:09 +0200
Subject: [pypy-dev] pirate list
In-Reply-To: <Pine.LNX.4.62.0506122309390.9583@hydrogen.sabren.com>
References: <Pine.LNX.4.62.0506122309390.9583@hydrogen.sabren.com>
Message-ID: <20050712162009.GA30063@code1.codespeak.net>

Hi Michal,

On Sun, Jun 12, 2005 at 11:15:30PM -0400, Michal Wallace wrote:
> Pirate is written in python and currently
> uses the standard compiler package. However,
> as parrot supports native types, I would like
> to refactor it so that it can also work with
> PyPy's type annotation system (and possibly 
> integrate with pypy in other ways)

Interesting, though by no means easy :-)  PyPy's type annotation system
is not designed at the moment to handle random user code.  It can only
handle carefully controlled code.  From there to a general type
annotator for Python, the gap is still large.

I may be too deeply buried in the current development work, but at the
moment it seems to me that the immediate contribution that PyPy could
make to Python-over-Parrot could be different: we could provide the
built-in types implementation and standard library modules, which is the
necessary complement to the bytecode compiler.  Indeed, by tweaking the
C code generator, the "standard object space" and the modules could be
compiled from PyPy sources to some C code that follows the conventions
of Parrot.


A bientot,

Armin


From arigo at tunes.org  Tue Jul 12 18:34:09 2005
From: arigo at tunes.org (Armin Rigo)
Date: Tue, 12 Jul 2005 18:34:09 +0200
Subject: [pypy-dev] Please give advice for newbie reading pypy source......
In-Reply-To: <20050710062237.GA12389@spawar.navy.mil>
References: <20050710062237.GA12389@spawar.navy.mil>
Message-ID: <20050712163409.GB30063@code1.codespeak.net>

Hi Chris,

On Sat, Jul 09, 2005 at 11:22:37PM -0700, seberino at spawar.navy.mil wrote:
> I'm really excited about learning PyPy source code really well
> and possibly helping the project in some capacity.
> Reading source code inevitably leads to tons of questions.

Great!

> **Any advice on how to progress quickly in PyPy source code understanding?**

Tough question :-)  I suppose that the starting point should be the
architecture document.  It should point you to particular areas of the
source code, depending on your interests.  For example, if you are
looking for the classical bytecode-dispatcher interpreter main loop,
you could start at pypy/interpreter/pyopcode.py.  If you want to see
some built-in functions, see pypy/module/__builtin__/.  Or implementation
of built-in types, e.g. dictionaries: see pypy/objspace/std/dictobject.py.
On the other hand, if you're more into translation and type inference,
try running pypy/bin/translator.py and following the code paths from
there.

In general I would suggest to try to follow the documentation for specific
directories along.  Please don't hesitate to ask if the documentation
can't be easily found, or if it's not helpful enough.

> Any chance there are PyPy people close to San Diego, California??

Not that I know of :-)  For some reason, there are not many US-based
PyPy people.

> Any better method than posting questions to this mailing list and
> slowly getting replies to all my questions?

I, at least, will try to answer your questions more quickly than I
answered this e-mail.  You may also try irc, #pypy on irc.freenode.net,
though we are generally there on European hours (US West + about 9 hours).


A bientot,

Armin


From arigo at tunes.org  Tue Jul 12 18:40:40 2005
From: arigo at tunes.org (Armin Rigo)
Date: Tue, 12 Jul 2005 18:40:40 +0200
Subject: [pypy-dev] Build and test failures
In-Reply-To: <OFBC08038C.E45E0A9E-ON8025703B.00320354-8025703B.003197DF@risk.sungard.com>
References: <OFBC08038C.E45E0A9E-ON8025703B.00320354-8025703B.003197DF@risk.sungard.com>
Message-ID: <20050712164040.GC30063@code1.codespeak.net>

Hi Ben,

On Mon, Jul 11, 2005 at 10:10:28AM +0100, Ben.Young at risk.sungard.com wrote:
> This first is that on windows, a couple of headers are not available. I 
> fixed this by patching locally with:

Thanks.  I commented out the #includes for now, as they come anyway with
<Python.h>.  Later, we will need to re-enable them to become independent
of CPython, with lots of #ifdefs and possibly a 'configure'-based build
system similar to CPython's.

> _______________________________________________________________________________
> _________________________ entrypoint: test_time_clock 

This failure should be fixed now (by copying all of CPython's
time.clock() implementation).

> I hope you all had fun at the sprint!

Definitely :-)  Thanks, and thanks too for keeping a close eye on PyPy
:-)


A bientot,

Armin


From arigo at tunes.org  Tue Jul 12 18:42:44 2005
From: arigo at tunes.org (Armin Rigo)
Date: Tue, 12 Jul 2005 18:42:44 +0200
Subject: [pypy-dev] Refining terminology
In-Reply-To: <20050712101933.GR28807@solar.trillke.net>
References: <20050712101933.GR28807@solar.trillke.net>
Message-ID: <20050712164244.GD30063@code1.codespeak.net>

Hi Holger,

On Tue, Jul 12, 2005 at 12:19:33PM +0200, holger krekel wrote:
> Well, maybe we should just go through the effort of
> creating a "glossary" containing the basic vocabulary
> we are using for describing what PyPy does and how
> it works.

Yes, it makes sense.  We already discussed this at least once, but
didn't bring it to a more concrete conclusion.  Maybe we should just
start a pypy/documentation/glossary.txt and start linking to/from it.


A bientot,

Armin.


From hpk at trillke.net  Tue Jul 12 18:50:05 2005
From: hpk at trillke.net (holger krekel)
Date: Tue, 12 Jul 2005 18:50:05 +0200
Subject: [pypy-dev] Refining terminology
In-Reply-To: <20050712164244.GD30063@code1.codespeak.net>
References: <20050712101933.GR28807@solar.trillke.net>
	<20050712164244.GD30063@code1.codespeak.net>
Message-ID: <20050712165005.GK7049@solar.trillke.net>

Hi Armin, 

On Tue, Jul 12, 2005 at 18:42 +0200, Armin Rigo wrote:
> On Tue, Jul 12, 2005 at 12:19:33PM +0200, holger krekel wrote:
> > Well, maybe we should just go through the effort of
> > creating a "glossary" containing the basic vocabulary
> > we are using for describing what PyPy does and how
> > it works.
> 
> Yes, it makes sense.  We already discussed this at least once, but
> didn't bring it to a more concrete conclusion.  Maybe we should just
> start a pypy/documentation/glossary.txt and start linking to/from it.

Good idea, first thing i would like to know what exactly "PBC access 
sets", "family of classes" and a few other things are :-) 

More seriously, i think that the increasing size of PyPy's 
documentation makes it more difficult to choose the right 
link targets. So i am not sure we should intend to link to 
the glossary unless the glossary itself back-points 
to relevant sections. 

cheers, 

    holger


From arigo at tunes.org  Tue Jul 12 19:04:51 2005
From: arigo at tunes.org (Armin Rigo)
Date: Tue, 12 Jul 2005 19:04:51 +0200
Subject: [pypy-dev] Refining terminology
In-Reply-To: <20050712165005.GK7049@solar.trillke.net>
References: <20050712101933.GR28807@solar.trillke.net>
	<20050712164244.GD30063@code1.codespeak.net>
	<20050712165005.GK7049@solar.trillke.net>
Message-ID: <20050712170451.GA31250@code1.codespeak.net>

Hi Holger,

On Tue, Jul 12, 2005 at 06:50:05PM +0200, holger krekel wrote:
> Good idea, first thing i would like to know what exactly "PBC access 
> sets", "family of classes" and a few other things are :-) 

:-)  This is actually a good idea, though it can only point back to the
relevant section; no actual information should be in the glossary for
this kind of entry.  But I think that it still makes sense to have them
in the glossary.


A bientot,

Armin.


From hpk at trillke.net  Tue Jul 12 20:08:16 2005
From: hpk at trillke.net (holger krekel)
Date: Tue, 12 Jul 2005 20:08:16 +0200
Subject: [pypy-dev] Refining terminology
In-Reply-To: <5.0.2.1.1.20050712095308.00adda90@mail.oz.net>
References: <20050712101933.GR28807@solar.trillke.net>
	<5.0.2.1.1.20050712095308.00adda90@mail.oz.net>
Message-ID: <20050712180816.GN7049@solar.trillke.net>

Hi Bengt, 

On Tue, Jul 12, 2005 at 11:00 -0700, Bengt Richter wrote:
> At 12:19 2005-07-12 +0200, you (holger krekel) wrote:
> >Any opinions or feedback (also from people just following
> >the project's progress)?
> I think any project tends to accumulate informal docs re key concepts
> and implementation details, but they tend to be hard to find until
> someone collects good links and the good-link collection becomes
> available and advertised via some standard place to look for docs,
> like a doc wiki or project docs home page.
> 
> Then somone like you realizes that the link collection represents a kind
> of archive of conceptual evolution, and doesn't have consistent terminology
> and doesn't even all represent the exactly the same (never mind "official")
> view of the concepts (naturally, since they were evolving).

Actually at least some core developers are discussing core terminologies 
for quite some time now, especially when we have to synchronize and 
give a talk together (we often do group-talks, i.e. not only one
person speaking the whole time). 

> So how to identify/create a representation of a project for a prospective
> explorer? A map, some pictures, a guide pamphlet that tells how to hike
> (read: walk the code and docs) to the best views? These are useful even
> for veteran explorers who pride themselves on being able to slog through
> most any jungle, especially if their appetite for slogging is waning ;-)

:-) 

> Best wishes. I hope to do some hiking one of these days ;-)
> Where is the best trailhead now to enter the pypy territory/jungle/parklands?
> (I hope there will always be some wild life, even when part of the territory
> gets leased for industrial parks ;-)

Actually i would be interested in experiences from your hiking expedition
which you might start here: 

    http://codespeak.net/pypy/index.cgi?doc

and in particular the very first "architecture" link.  Note that 
the main entry page has a cross-referenced directory index. 

cheers, 

    holger


From michal at sabren.com  Wed Jul 13 03:58:34 2005
From: michal at sabren.com (Michal Wallace)
Date: Tue, 12 Jul 2005 21:58:34 -0400 (EDT)
Subject: [pypy-dev] pirate list
In-Reply-To: <20050712162009.GA30063@code1.codespeak.net>
References: <Pine.LNX.4.62.0506122309390.9583@hydrogen.sabren.com>
	<20050712162009.GA30063@code1.codespeak.net>
Message-ID: <Pine.LNX.4.62.0507122135570.10100@hydrogen.sabren.com>

On Tue, 12 Jul 2005, Armin Rigo wrote:

> On Sun, Jun 12, 2005 at 11:15:30PM -0400, Michal Wallace wrote:
>> Pirate is written in python and currently
>> uses the standard compiler package. However,
>> as parrot supports native types, I would like
>> to refactor it so that it can also work with
>> PyPy's type annotation system (and possibly
>> integrate with pypy in other ways)

> Interesting, though by no means easy :-) PyPy's type annotation
> system is not designed at the moment to handle random user code.  It
> can only handle carefully controlled code.  From there to a general
> type annotator for Python, the gap is still large.

Hi Armin,

Thanks for clarifying. Sounds like my understanding of
pypy's current state was completely backwards.

There is another approach I've been considering
for pirate, which might come in handy for pypy 
as well: explicitly stating type information in
a separate file, along the lines of this type
system for scheme:

   http://www.cs.iastate.edu/~leavens/ComS342-EOPL2e/docs/typedscm_toc.html

Basically, you'd have a separate "typesheet" file
which said stuff like:

    # add.ty
    def add(a=int, b=int):
        return int

which would correspond to the python code:

    # add.py
    def add(a, b):
        return a + b

It seems like this would be a very simple way to get 
type annotation working: we'd just do it by hand. Then 
once we have real type inference, we could toss it aside.

My intuition is that this would be fairly easy for me
to add to pypy, but I'm not familiar with the code yet.
What do you think? If possible, I'd rather extend your
compiler than using the standard compiler module (which
is what pirate currently uses)

> I may be too deeply buried in the current development work, but at the
> moment it seems to me that the immediate contribution that PyPy could
> make to Python-over-Parrot could be different: we could provide the
> built-in types implementation and standard library modules, which is the
> necessary complement to the bytecode compiler.  Indeed, by tweaking the
> C code generator, the "standard object space" and the modules could be
> compiled from PyPy sources to some C code that follows the conventions
> of Parrot.

This is the other part I was backwards on.

Just to be clear, and to share this idea with the pirate list,
you're saying that all the built-in python types, which you guys
have rewritten in your "restricted" python, could be compiled
to produce parrot PMCs? Or in other words, that we could write
our PMCs in rPython?

    http://codespeak.net/svn/pypy/dist/pypy/objspace/std/
    http://codespeak.net/svn/pypy/dist/pypy/lib/

If I can spend my time hacking on pypy and reap the benefits
for pirate, sign me up. I'd rather do that than write PMC's
directly.

What's the plan for modules like socket, that basically
wrap external C routines? Is that in the works for pypy
as well? It almost seems like they'd require some 
extensions to python syntax (sort of like pyrex).


Sincerely,

Michal J Wallace
Sabren Enterprises, Inc.
-------------------------------------
contact: michal at sabren.com
hosting: http://www.cornerhost.com/
my site: http://www.withoutane.com/
-------------------------------------



From Ben.Young at risk.sungard.com  Wed Jul 13 11:03:28 2005
From: Ben.Young at risk.sungard.com (Ben.Young at risk.sungard.com)
Date: Wed, 13 Jul 2005 10:03:28 +0100
Subject: [pypy-dev] Build and test failures
In-Reply-To: <20050712164040.GC30063@code1.codespeak.net>
Message-ID: <OF6198778A.CC3BEF9C-ON8025703D.003140EC-8025703D.0030F3AE@risk.sungard.com>

Thanks for that!

Unfortunately thats exposes a new problem, as there is a function called 
RaiseException included by windows.h! Maybe all your macros should be 
prefixed by PyPy_
or something.

It sounds like you are getting close to the translate_pypy target. Would 
it be possible for you to give a summary of the problems you appear to be 
having with the PBC code (as evidenced by some of the commit messages)? 
I'm just interested in what the problem appears to be, and whether it's 
just something you're going to have to live with, or if there is an 
architectural solution? 

I don't really understand the PBC side of things. Is there a reason why 
the object hierarchy can't just mirror the way CPython does it? Or would 
that be too slow?

Thanks for all your help!

Cheers,
Ben

Armin Rigo <arigo at tunes.org> wrote on 12/07/2005 17:40:40:

> Hi Ben,
> 
> On Mon, Jul 11, 2005 at 10:10:28AM +0100, Ben.Young at risk.sungard.com 
wrote:
> > This first is that on windows, a couple of headers are not available. 
I 
> > fixed this by patching locally with:
> 
> Thanks.  I commented out the #includes for now, as they come anyway with
> <Python.h>.  Later, we will need to re-enable them to become independent
> of CPython, with lots of #ifdefs and possibly a 'configure'-based build
> system similar to CPython's.
> 
> > 
> 
_______________________________________________________________________________
> > _________________________ entrypoint: test_time_clock 
> 
> This failure should be fixed now (by copying all of CPython's
> time.clock() implementation).
> 
> > I hope you all had fun at the sprint!
> 
> Definitely :-)  Thanks, and thanks too for keeping a close eye on PyPy
> :-)
> 
> 
> A bientot,
> 
> Armin
> 



From arigo at tunes.org  Wed Jul 13 12:44:36 2005
From: arigo at tunes.org (Armin Rigo)
Date: Wed, 13 Jul 2005 12:44:36 +0200
Subject: [pypy-dev] pirate list
In-Reply-To: <Pine.LNX.4.62.0507122135570.10100@hydrogen.sabren.com>
References: <Pine.LNX.4.62.0506122309390.9583@hydrogen.sabren.com>
	<20050712162009.GA30063@code1.codespeak.net>
	<Pine.LNX.4.62.0507122135570.10100@hydrogen.sabren.com>
Message-ID: <20050713104436.GA9227@code1.codespeak.net>

Hi Michal,

On Tue, Jul 12, 2005 at 09:58:34PM -0400, Michal Wallace wrote:
> There is another approach I've been considering
> for pirate, which might come in handy for pypy 
> as well: explicitly stating type information in
> a separate file, along the lines of this type
> system for scheme:

Yes, that's still another possible approach: type language design,
compiler support, custom bytecodes.  Not a solution I'm too fond of :-)
but you are welcome to play with it; the future goal of the PyPy
compiler is to be flexible enough for this kind of experimentation.
(For now it is in development; we have a parser but we directly use the
standard library compiler package).

Performance-wise, the focus of PyPy is definitely on automatic type
inference.  Our type inference works nicely already but for "RPython"
programs only.  RPython is quite difficult to define precisely;
ultimately, it's "simple enough to make our type inference happy"...  At
one point we will try to support user programs that are RPythonic
enough, to enable static optimizations on them, but this goes again in
the direction of required user awareness.  Ultimately, as far as it is
practical on a specific target platform, I would rather go for
completely transparent just-in-time compilation.

> Just to be clear, and to share this idea with the pirate list,
> you're saying that all the built-in python types, which you guys
> have rewritten in your "restricted" python, could be compiled
> to produce parrot PMCs? Or in other words, that we could write
> our PMCs in rPython?

Yes.  That's a primary goal of PyPy: write the Python semantics once and
compile them to various platforms.  Writing a "parrot PMC backend" to
generate this C code would be a nice job.  That will probably take about
the same time as writing the PMCs by hand, and it will probably be more
mind-twisting, but then you get for free things like maintainability
(tracking Python language changes).

To some extent the same is possible with C modules, e.g. regular
expressions.

> If I can spend my time hacking on pypy and reap the benefits
> for pirate, sign me up. I'd rather do that than write PMC's
> directly.

:-)

> What's the plan for modules like socket, that basically
> wrap external C routines? Is that in the works for pypy
> as well? It almost seems like they'd require some 
> extensions to python syntax (sort of like pyrex).

These modules are handled with a kind of custom language, but which is
plain Python code; the types are not specified in the source as with
Pyrex, but they are inferred using our type annotation machinery again.
For example, os.read() is implemented as:

def ll_os_read(fd, count):
    if count < 0:
        raise OSError(errno.EINVAL, None)
    buffer = malloc(STR, count)
    n = ll_read_into(fd, buffer)
    if n != count:
        s = malloc(STR, n)
        ll_strcpy(s.chars, buffer.chars, n)
        buffer = s
    return buffer

This code can run normally for testing purposes (malloc() is then a
Python function that returns an instance of "_ptr", a class that
emulates a pointer to some data).  It can also be type-inferenced
(starting from the information that 'fd' and 'count' are integers) and
translated to C code, where malloc() becomes the real C malloc().

As long as the target platform is C-like, functions like ll_os_read()
above can be used to generate code that uses the platform's convention,
e.g. parrot's memory management.


A bientot,

Armin.


From Ben.Young at risk.sungard.com  Wed Jul 13 16:35:41 2005
From: Ben.Young at risk.sungard.com (Ben.Young at risk.sungard.com)
Date: Wed, 13 Jul 2005 15:35:41 +0100
Subject: [pypy-dev] Fw: [pypy-svn] r14617 -
	pypy/dist/pypy/module/unicodedata[POSSIBLE SPAM	]
Message-ID: <OFB9F4A9C7.57BD6B39-ON8025703D.0050284E-8025703D.004F5E10@risk.sungard.com>

Hi, I was just looking over the subversion commit messages and had a 
couple of comments to make.

Did you really want [0 * ... ] in some of these diffs, rather than [0] * 
...? Also, don't lists resize to avoid this kind of behaviour?

Sorry if you had already spotted them!

Cheers,
Ben

pypy-svn-bounces at codespeak.net wrote on 13/07/2005 15:05:11:

> Author: ac
> Date: Wed Jul 13 16:05:10 2005
> New Revision: 14617
> 
> Modified:
>    pypy/dist/pypy/module/unicodedata/function.py
> Log:
> Avoid frequent reallocations when normalizing unicode.
> 
> Modified: pypy/dist/pypy/module/unicodedata/function.py
> 
==============================================================================
> --- pypy/dist/pypy/module/unicodedata/function.py   (original)
> +++ pypy/dist/pypy/module/unicodedata/function.py   Wed Jul 13 16:05:10 
2005
> @@ -100,6 +100,7 @@
>  NCount = (VCount*TCount)
>  SCount = (LCount*NCount)
> 
> +
>  def normalize(space, w_form, w_unistr):
>      form = space.str_w(w_form)
>      if not space.is_true(space.isinstance(w_unistr, space.w_unicode)):
> @@ -121,7 +122,9 @@
>                               space.wrap('invalid normalization form'))
> 
>      strlen = space.int_w(space.len(w_unistr))
> -    result = []
> +    result = [0] * (strlen + strlen / 10 + 10)
> +    j = 0
> +    resultlen = len(result)
>      # Expand the character
>      for i in range(strlen):
>          ch = space.int_w(space.ord(space.getitem(w_unistr, 
space.wrap(i))))
> @@ -132,33 +135,57 @@
>              V = VBase + (SIndex % NCount) / TCount;
>              T = TBase + SIndex % TCount;
>              if T == TBase:
> -                result.extend([L, V])
> +                if j + 2 > resultlen:
> +                    result.extend([0 * (j + 2 - resultlen + 10)])
> +                    resultlen = len(result)
> +                result[j] = L
> +                result[j + 1] = V
> +                j += 2
>              else:
> -                result.extend([L, V, T])
> +                if j + 3 > resultlen:
> +                    result.extend([0 * (j + 3 - resultlen + 10)])
> +                    resultlen = len(result)
> +                result[j] = L
> +                result[j + 1] = V
> +                result[j + 2] = T
> +                j += 3
>              continue
> -
> -        result.extend(decomposition.get(ch, [ch]))
> +        decomp = decomposition.get(ch)
> +        if decomp:
> +            decomplen = len(decomp)
> +            if j + decomplen > resultlen:
> +                result.extend([0 * (j + decomplen - resultlen + 10)])
> +                resultlen = len(result)
> +            for ch in decomp:
> +                result[j] = ch
> +                j += 1
> +        else:
> +            if j + 1 > resultlen:
> +                result.extend([0 * (j + 1 - resultlen + 10)])
> +                resultlen = len(result)
> +            result[j] = ch
> +            j += 1
> 
>      # Sort all combining marks
> -    for i in range(len(result)):
> +    for i in range(j):
>          ch = result[i]
>          comb = unicodedb.combining(ch)
>          if comb == 0:
>              continue
> -        for j in range(i, 0, -1):
> -            if unicodedb.combining(result[j - 1]) <= comb:
> -                result[j] = ch
> +        for k in range(i, 0, -1):
> +            if unicodedb.combining(result[k - 1]) <= comb:
> +                result[k] = ch
>                  break
> 
> -            result[j] = result[j - 1]
> +            result[k] = result[k - 1]
>          else:
>              result[0] = ch
> 
>      if not composed: # If decomposed normalization we are done
> -        return space.newunicode(result)
> +        return space.newunicode(result[:j])
> 
> -    if len(result) <= 1:
> -        return space.newunicode(result)
> +    if j <= 1:
> +        return space.newunicode(result[:j])
> 
>      current = result[0]
>      starter_pos = 0
> @@ -166,8 +193,8 @@
>      prev_combining = 0
>      if unicodedb.combining(current):
>          prev_combining = 256
> -    for j in range(1, len(result)):
> -        next = result[j]
> +    for k in range(1, j):
> +        next = result[k]
>          next_combining = unicodedb.combining(next)
>          if next_insert == starter_pos + 1 or prev_combining < 
next_combining:
>              # Combine if not blocked
> _______________________________________________
> pypy-svn mailing list
> pypy-svn at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-svn
> 



From mwh at python.net  Wed Jul 13 19:01:40 2005
From: mwh at python.net (Michael Hudson)
Date: Wed, 13 Jul 2005 18:01:40 +0100
Subject: [pypy-dev] Re: [pypy-svn] r14625 - pypy/dist/pypy/objspace/std
In-Reply-To: <20050713155324.9060327B5E@code1.codespeak.net>
	(tismer@codespeak.net's
	message of "Wed, 13 Jul 2005 17:53:24 +0200 (CEST)")
References: <20050713155324.9060327B5E@code1.codespeak.net>
Message-ID: <2meka2tzbf.fsf@starship.python.net>

tismer at codespeak.net writes:

> Author: tismer
> Date: Wed Jul 13 17:53:23 2005
> New Revision: 14625
>
> Modified:
>    pypy/dist/pypy/objspace/std/strutil.py
> Log:
> another less urgent optimization from the train Kiel-Berlin:
>
> refactored string_to_float quite a lot. The issue was raised
> by Python 2.4's test_long, which caused an overflow in strutil
> instead of an 1.#inf.
> I took the chance to rework this quite a little, with the result of
> - less rounding errors
> - smaller code
> - much faster in extreme cases
> - able to eval float('1.'+10000*'0'+'e-10000') and friends
> - seems to produce exactly the same as builtin float as far as tested.

Thank you for doing this!  The original code was written in about 5
minutes flat...

> +    # Usage of long numbers is explicitly avoided, because
> +    # we want to be able to work without longs as a PyPy option.

Unfortunately, the 'input problem for floating point numbers' cannot
be solved using arithmetic of any fixed finite precision, see

    http://citeseer.ist.psu.edu/clinger90how.html

for a proof of this.

Also, I found a difference between CPython and your code:

>>> strutil.string_to_float('0.099999999999999999')
0.099999999999999992
>>> float('0.099999999999999999')
0.10000000000000001

But still, a vast improvement, thanks!

Cheers,
mwh

-- 
  If comp.lang.lisp *is* what vendors are relying on to make or 
  break Lisp sales, that's more likely the problem than is the 
  effect of any one of us on such a flimsy marketing strategy...
                                      -- Kent M Pitman, comp.lang.lisp


From tismer at stackless.com  Wed Jul 13 19:53:28 2005
From: tismer at stackless.com (Christian Tismer)
Date: Wed, 13 Jul 2005 19:53:28 +0200
Subject: [pypy-dev] Re: [pypy-svn] r14625 - pypy/dist/pypy/objspace/std
In-Reply-To: <2meka2tzbf.fsf@starship.python.net>
References: <20050713155324.9060327B5E@code1.codespeak.net>
	<2meka2tzbf.fsf@starship.python.net>
Message-ID: <42D55518.7090703@stackless.com>

Michael Hudson wrote:

[hacked on string_to_float]

> Thank you for doing this!  The original code was written in about 5
> minutes flat...

That's in fact very effective! Mine took a bit longer.
And as I see, it is still not optimum.

>>+    # Usage of long numbers is explicitly avoided, because
>>+    # we want to be able to work without longs as a PyPy option.
> 
> 
> Unfortunately, the 'input problem for floating point numbers' cannot
> be solved using arithmetic of any fixed finite precision, see
> 
>     http://citeseer.ist.psu.edu/clinger90how.html
> 
> for a proof of this.

Where can I read the full article?
Also, I don't understand what the abstract means.
I can use double precision integer arithmetic to
turn the mantissa and the exponent into a base 2
representation of the given literal in an (as I think)
exact way.
Well, maybe not if the exponent is negative. Hum.

> Also, I found a difference between CPython and your code:
> 
> 
>>>>strutil.string_to_float('0.099999999999999999')
> 
> 0.099999999999999992

Ok, at least identical to what the former code produces.
But this doesn't make me so happy.

>>>>float('0.099999999999999999')
> 
> 0.10000000000000001

I thought the way to add digits scaled to the proper power
of ten would be optimum. Now I'm no longer sure.
In fact, float(stuff) * 10 round up to 1.0, nicely.

How comes? Do you have a better strategy? Or is there none
a priori, and one would have to add a posteriori adjustments?
Is it maybe better to always try to do negative powers
after the whole number is loaded in?

> But still, a vast improvement, thanks!

well, compared to the time I needed, your's was more efficient.
And I'm not happy until it is really perfect. :-]

ciao - chris
-- 
Christian Tismer             :^)   <mailto:tismer at stackless.com>
tismerysoft GmbH             :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9A     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 802 86 56  mobile +49 173 24 18 776  fax +49 30 80 90 57 05
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/


From tismer at stackless.com  Wed Jul 13 19:57:55 2005
From: tismer at stackless.com (Christian Tismer)
Date: Wed, 13 Jul 2005 19:57:55 +0200
Subject: [pypy-dev] Re: [pypy-svn] r14625 - pypy/dist/pypy/objspace/std
In-Reply-To: <42D55518.7090703@stackless.com>
References: <20050713155324.9060327B5E@code1.codespeak.net>	<2meka2tzbf.fsf@starship.python.net>
	<42D55518.7090703@stackless.com>
Message-ID: <42D55623.9040802@stackless.com>

Christian Tismer wrote:

> Where can I read the full article?

I was dumb, could read it now.

-- 
Christian Tismer             :^)   <mailto:tismer at stackless.com>
tismerysoft GmbH             :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9A     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 802 86 56  mobile +49 173 24 18 776  fax +49 30 80 90 57 05
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/


From ac at strakt.com  Wed Jul 13 21:09:25 2005
From: ac at strakt.com (=?ISO-8859-1?Q?Anders_Chrigstr=F6m?=)
Date: Wed, 13 Jul 2005 21:09:25 +0200
Subject: [pypy-dev] Fw: [pypy-svn] r14617
	-	pypy/dist/pypy/module/unicodedata[POSSIBLE SPAM	]
In-Reply-To: <OFB9F4A9C7.57BD6B39-ON8025703D.0050284E-8025703D.004F5E10@risk.sungard.com>
References: <OFB9F4A9C7.57BD6B39-ON8025703D.0050284E-8025703D.004F5E10@risk.sungard.com>
Message-ID: <42D566E5.7050505@strakt.com>

Ben.Young at risk.sungard.com wrote:
> Hi, I was just looking over the subversion commit messages and had a 
> couple of comments to make.
> 
> Did you really want [0 * ... ] in some of these diffs, rather than [0] * 
> ...? Also, don't lists resize to avoid this kind of behaviour?

I sure did.
Thanks for spotting it.





From pedronis at strakt.com  Thu Jul 14 18:35:05 2005
From: pedronis at strakt.com (Samuele Pedroni)
Date: Thu, 14 Jul 2005 18:35:05 +0200
Subject: [pypy-dev] plans for os and math
Message-ID: <42D69439.3050608@strakt.com>

This mail is an attempt to summarize a bit our current thinking and 
plans for starting implementing
os and  math modules functionality.

The idea is to implement them as mixed modules (like sys and __builtin__ 
are now), so to have, for starting,
a module/posix and module/math packages in module.

E.g. open would be implemented by posix at interpreter level, for 
example like:

def open(w_fname, w_flag, w_mode=0777):
    fname = space.str_w(w_fname)
    flag = space.int_w(w_flag)
    mode = space.int_w(w_mode)
    # notice that an unwrap_spec attached to open could be used to the 
same effect
    fd = os.open(fname, flag, mode)
    return space.wrap(fd)

For now constants in os.* would be reimported by such a module. (So the 
translation process and result
are platform dependent, we have ideas on how to fix this but those are 
lower priority).

Similary math would be delegating to math at interpreter level. The 
delegation is to get sort of placeholders
for translation and to be able to run on top of CPython.

We have started implementing in the toolchain from annotation to the 
backends functionality such that the above
call to os.open at interpreter-level can be annotated and proper working 
functionality can be supplied at the end
of the chain by the backend. (See rpython/extfunctable.py and the 
contents of rpython/module). So sorting out platform
differences is intended to be a backend task.

The first things we need in os.* are the functions and constants 
required by our app-level implementation of files in lib/_file.py
and lib/_sio.py.

Functions like os.stat return tuple-like objects that have also named 
attributes corresponding to the various fields.

The idea is that the calls at interpreter-level to such functions return 
results that can only be numerically indexed,
so os.stat(...)[0] is OK at interpreter-level but os.stat(...).st_mode 
is not.

The mixed module would have OTOH to implement such result types 
(possibly at app-level) and wrap in them the results of interpreter level
calls to placeholder functions.

regards.











From Ben.Young at risk.sungard.com  Fri Jul 15 11:14:03 2005
From: Ben.Young at risk.sungard.com (Ben.Young at risk.sungard.com)
Date: Fri, 15 Jul 2005 10:14:03 +0100
Subject: [pypy-dev] targetpypy
Message-ID: <OF57BECBB3.43939858-ON8025703F.00329633-8025703F.0031E545@risk.sungard.com>

Hi Everyone,

I've been running translate_pypy -text targetpypy every couple of days 
recently, to see how things are progressing, but I always get the same 
error:

ImportError: No module named _formatting

Is this a known problem, or am I doing something wrong?

Cheers,
Ben


From pedronis at strakt.com  Fri Jul 15 19:20:57 2005
From: pedronis at strakt.com (Samuele Pedroni)
Date: Fri, 15 Jul 2005 19:20:57 +0200
Subject: [pypy-dev] targetpypy
In-Reply-To: <OF57BECBB3.43939858-ON8025703F.00329633-8025703F.0031E545@risk.sungard.com>
References: <OF57BECBB3.43939858-ON8025703F.00329633-8025703F.0031E545@risk.sungard.com>
Message-ID: <42D7F079.7030401@strakt.com>

Ben.Young at risk.sungard.com wrote:
> Hi Everyone,
> 
> I've been running translate_pypy -text targetpypy every couple of days 
> recently, to see how things are progressing, but I always get the same 
> error:
> 
> ImportError: No module named _formatting
> 
> Is this a known problem, or am I doing something wrong?

the target we are currently working with is targetpypymain,
which disables geninterp for now.

targetpypy is somehow obsolete at this point, I will remove it.

Anyway you discovered a problem in the logic that searches for lib
in geninterp.translate_as_module vs. the import redirection done by 
translate_pypy to the pypy-translation-snaptshot; targetpypy is not 
disabling geninterping so this shows up. I will checkin a fix when
codespeak svn  is up again.

Thanks.


From rxe at ukshells.co.uk  Fri Jul 15 23:01:54 2005
From: rxe at ukshells.co.uk (Richard Emslie)
Date: Fri, 15 Jul 2005 22:01:54 +0100 (BST)
Subject: [pypy-dev] llvm2 update
Message-ID: <Pine.LNX.4.60.0507152132340.25741@echo.ukshells.co.uk>


Hi all,

I've just added a dummy OpaqueType implementation and refactored the 
database wrt to pbc nodes.  So pbcs are working for classes too now.

Here is a brief overview of things that (I think) still need to me done.

todo (probably in this order):

* exception flow of control

* any missing operations implementations (bit shifting for instance).
   These can be added when they are needed and should be straight forward.

* review and update external function implementations for new way of doing
   things

* overflows on ints/etc (more on that from Carl Friedrich...)


refactoring:

* should is_atomic() be a method?

* grep "XXX"s :-)


Of the above I am guess that exceptions are most interesting.  The unwind 
/ invoke operations of llvm fits this quite well, but still need somewhere 
to store the exception and do matching - unless I missed something?  Any 
thoughts / takers?

Cheers,
Richard



From sabre at nondot.org  Fri Jul 15 23:47:16 2005
From: sabre at nondot.org (Chris Lattner)
Date: Fri, 15 Jul 2005 16:47:16 -0500 (CDT)
Subject: [pypy-dev] llvm2 update
In-Reply-To: <Pine.LNX.4.60.0507152132340.25741@echo.ukshells.co.uk>
References: <Pine.LNX.4.60.0507152132340.25741@echo.ukshells.co.uk>
Message-ID: <Pine.LNX.4.61.0507151645450.8772@nondot.org>

On Fri, 15 Jul 2005, Richard Emslie wrote:
> Of the above I am guess that exceptions are most interesting.  The unwind / 
> invoke operations of llvm fits this quite well, but still need somewhere to 
> store the exception and do matching - unless I missed something?  Any 
> thoughts / takers?

This is generally done with global variables.  The C++ front-end basically 
generates code like this:

   C++:  throw 42;

Pseudo LLVM code:
    int *E = allocate_exception_memory(4);
    *E = 42;
    add_to_current_exception_stack(E);
    unwind

I imagine that something similar should work for pypy.

-Chris

-- 
http://nondot.org/sabre/
http://llvm.org/


From cfbolz at gmx.de  Sat Jul 16 00:44:41 2005
From: cfbolz at gmx.de (Carl Friedrich Bolz)
Date: Sat, 16 Jul 2005 00:44:41 +0200
Subject: [pypy-dev] llvm2 update
In-Reply-To: <Pine.LNX.4.60.0507152132340.25741@echo.ukshells.co.uk>
References: <Pine.LNX.4.60.0507152132340.25741@echo.ukshells.co.uk>
Message-ID: <42D83C59.2080607@gmx.de>

Hi Richard

Richard Emslie wrote:
> I've just added a dummy OpaqueType implementation and refactored the 
> database wrt to pbc nodes.  So pbcs are working for classes too now.

Cool! Although I don't really understand anymore how the database works 
(although I haven't looked into it in detail, so it's probably my fault).

> Here is a brief overview of things that (I think) still need to me done.
> 
> todo (probably in this order):
> 
> * exception flow of control
 >
> * any missing operations implementations (bit shifting for instance).
>   These can be added when they are needed and should be straight forward.

Bit shifting can probably be taken from genllvm1 (at least the ideas, 
the node model was of course completely different).

> 
> * review and update external function implementations for new way of doing
>   things
> 
> * overflows on ints/etc (more on that from Carl Friedrich...)

I think we can do this similarly to genc: have some header files which 
contain error checking implementations of the operations. This will be 
mostly easy as soon as we have exception support. Or lets rather say as 
easy as in C (which can be difficult for floats for example). The only 
tricky thing could be to find out how to instantiate exceptions on 
LLVM-level, but that, too, depends on exceptions so we have to wait 
until these are done.

> 
> refactoring:
> 
> * should is_atomic() be a method?
> 
> * grep "XXX"s :-)
> 
> 
> Of the above I am guess that exceptions are most interesting.  The 
> unwind / invoke operations of llvm fits this quite well, but still need 
> somewhere to store the exception and do matching - unless I missed 
> something?  Any thoughts / takers?

Some notes from my experiences with genllvm1: LLVM's exception model is 
a nearly perfect match, it was really easy to implement exceptions. The 
idea is (at least if the operations is implemed as a function) that the 
last operation in an try block does not get called, but invoked. Normal 
operation resumes in the None block, if an unwind occurs control flow is 
transfered to an special block where we try to match the exception to 
the links from the original try block. If there is no match we unwind 
again, otherwise control is transfered to the matched block.

If we raise an exception we store the exception and the type in a global 
variable. The matching code has to be similar to what genc does, I think 
that's the only part that could get slightly complicated.

Regards,

Carl Friedrich


From cfbolz at gmx.de  Sat Jul 16 00:45:00 2005
From: cfbolz at gmx.de (Carl Friedrich Bolz)
Date: Sat, 16 Jul 2005 00:45:00 +0200
Subject: [pypy-dev] llvm2 update
In-Reply-To: <Pine.LNX.4.61.0507151645450.8772@nondot.org>
References: <Pine.LNX.4.60.0507152132340.25741@echo.ukshells.co.uk>
	<Pine.LNX.4.61.0507151645450.8772@nondot.org>
Message-ID: <42D83C6C.2040101@gmx.de>

Hi Chris!

Chris Lattner wrote:
[snip]
> This is generally done with global variables.  The C++ front-end 
> basically generates code like this:
> 
>   C++:  throw 42;
> 
> Pseudo LLVM code:
>    int *E = allocate_exception_memory(4);
>    *E = 42;
>    add_to_current_exception_stack(E);
>    unwind
> 
> I imagine that something similar should work for pypy.

This is indeed pretty much what I did in genllvm1 (note that we're 
already writing our second LLVM backend :-). It's even a bit easier 
since we don't need an exception stack but only one single exception.
One problem is that exception matching is not entirely straightforward 
although this is a property of the PyPy model and has nothing to do with 
LLVM per se.

Regards,

Carl Friedrich


From amp at singingwizard.org  Sat Jul 16 00:29:51 2005
From: amp at singingwizard.org (Arthur Peters)
Date: Fri, 15 Jul 2005 17:29:51 -0500
Subject: [pypy-dev] Questions from a lurker
Message-ID: <E1DtYgp-0006yh-2O@lazarus.localnet.local>


Hello, I am Arthur. I've been lurking on the list for a month or so and
I've had a good bit of fun playing with the code.

My questions are:

- Why did you start a second LLVM backend? Was the first one badly miss
designed?

- So I take it the initial goal will be to translate pypy into C (or
LLVM) and create a binary executable. At that point will it be possible
to translate any given program or will the translator still only work
with a subset of the python language? Will the final goal be a JIT VM
for python or will I at some point be able to statically compile python
to machine code?

- Will there be support for multiple specialized versions of functions
like in psyco? I know this is a long way down the road but I'm curious
what people think.

- I read what some discussion of the GIL as it relate to pypy and I agree
that the GIL need to be implemented and that a different thread model
might be a good way to go. However I thought of the following: Would it
be possible to detect when a section of code only uses local variables
and unlock the GIL? This seems possible because local variables will
cannot be shared between threads anyway. In addition, local variables
likely be translated into more basic types than a python object (native
ints and floats for instance), that would not require any interaction
with the object-space to manipulate (not sure about that usage of the
term "object-space"). Thoughts?

- I have tried to run test_all.py on my desktop (a dual PIII running
ubuntu breazy (development)) but I get the following exception:
Traceback (most recent call last):
  File "pypy/test_all.py", line 6, in ?
    py.test.cmdline.main()
  File "/home/amp/source/pypy-dist/py/test/cmdline.py", line 8, in main
    config, args = py.test.Config.parse(py.std.sys.argv[1:])
  File "/home/amp/source/pypy-dist/py/test/config.py", line 61, in parse
    config = bootstrapconfig(configpaths)
  File "/home/amp/source/pypy-dist/py/test/config.py", line 168, in
bootstrapconfig
    config.loadconfig(defaultconfig).adddefaultoptions()
  File "/home/amp/source/pypy-dist/py/test/config.py", line 91, in
loadconfig
    mod = importconfig(configpath)
  File "/home/amp/source/pypy-dist/py/test/config.py", line 208, in
importconfig
    return configpath.pyimport()
  File "/home/amp/source/pypy-dist/py/path/local/local.py", line 381,
in pyimport
    return __import__(modname, None, None, ['__doc__'])
ImportError: No module named defaultconftest

The cause of the problem seems to be a failure at
/home/amp/source/pypy-dist/py/initpkg.py(179)__getattr__() while trying
to get the attribute "__path__". This happens on both 2.3 and 2.4.
This does not happen at all on my Ubuntu Hoary (stable) laptop.

Any ideas?

Thanks a bunch. I am very impressed by what pypy can do and I'm looking
forward to seeing it progress.

-Arthur

PS: I am the amep that left a message on IRC.


From cfbolz at gmx.de  Sat Jul 16 12:55:01 2005
From: cfbolz at gmx.de (Carl Friedrich Bolz)
Date: Sat, 16 Jul 2005 12:55:01 +0200
Subject: [pypy-dev] Questions from a lurker
In-Reply-To: <E1DtYgp-0006yh-2O@lazarus.localnet.local>
References: <E1DtYgp-0006yh-2O@lazarus.localnet.local>
Message-ID: <42D8E785.9030909@gmx.de>

Hi Arthur!

Arthur Peters wrote:
> Hello, I am Arthur. I've been lurking on the list for a month or so and
> I've had a good bit of fun playing with the code.

Welcome to PyPy! I'll try to take a stab at some of the questions. 
Corrections are welcome.

> My questions are:
> 
> - Why did you start a second LLVM backend? Was the first one badly miss
> designed?

When I wrote the first LLVM backend the RTyper did not exist yet. This 
resulted in a big duplication of work: I basically implemented a lot of 
things on LLVM level (like lists, tuples, classes...) which would have 
been useful for other C-ish backends as well. Later the RTyper was 
written which made all that work superfluous because the RTyper did 
exactly that: implement all those things on a level where every other 
C-ish backend could use it. Eric and me tried to adapt the old LLVM 
backend to the new model but it didn't work very well (because the 
assumptions where totally different). Thus Holger and me started the new 
LLVM backend on a weekend.

> - So I take it the initial goal will be to translate pypy into C (or
> LLVM) and create a binary executable. At that point will it be possible
> to translate any given program or will the translator still only work
> with a subset of the python language? Will the final goal be a JIT VM
> for python or will I at some point be able to statically compile python
> to machine code?

The translator will always only work on a subset of Python. It's 
probably not really possible to statically compile arbritrary Python 
code to machine code (at least not whithout cheating and using the whole 
interpreter in the machine code again :-) ).

> 
> - Will there be support for multiple specialized versions of functions
> like in psyco? I know this is a long way down the road but I'm curious
> what people think.

The idea is to integrate Psyco's ideas, yes. Although there are not 
really multiple specialized versions of a function in Psyco, it's rather 
one function that does different things depending on the type of the 
arguments.

> - I read what some discussion of the GIL as it relate to pypy and I agree
> that the GIL need to be implemented and that a different thread model
> might be a good way to go. However I thought of the following: Would it
> be possible to detect when a section of code only uses local variables
> and unlock the GIL? This seems possible because local variables will
> cannot be shared between threads anyway. In addition, local variables
> likely be translated into more basic types than a python object (native
> ints and floats for instance), that would not require any interaction
> with the object-space to manipulate (not sure about that usage of the
> term "object-space"). Thoughts?

First a comment about the usage of the term "object space": I think you 
are mixing levels here (and I might misunderstand you). The PyPy 
interpreter (meaning the bytecode interpreter plus the standard 
objectspace) gets translated to low level code. This "interpreter level 
code" deals with the standard object space as a regular class instance 
that is in principle in no way different than any other class. The 
classes that appear on interpreter level are all translated to more 
basic types (what would be the alternative, there is no layer below that 
could deal with anything else), probably to something like a struct. 
Thus the operations of objects at this level never need to be done via 
the object space -- the object space is rather a regular object in itself.

The object space /is/ used, if you interpret a Python program with 
PyPy's interpreter. The bytecode interpreter does not now how to deal 
with /any/ object (except for True and False), it has to delegate to the 
object space for every single operation. Even for basic types like ints 
and such -- at this level ("application level") there isn't any type 
inference or anything like that!

Now on threading: I'm not really the right person to say much about it. 
As far as I understand it, the general idea is that we don't want to 
clutter our whole interpreter implementation with threading details. 
Instead threading is supposed to become a translation aspect: The 
translation process is meant to "weave" the threading model into the 
translated interpreter. This would have a lot of advantages: Instead of 
being stuck with a single threading model which is deeply integrated 
into all the parts of our interpreter and hard to get rid of again, we 
can change it by changing a small localized part of whatever (probably 
the translator). Thus it would be possible to translate an interpreter 
which uses a GIL -- appropriate for an environment where threads are 
rarely used. Or we could translate an interpreter with, say, more finely 
grained locking which would be slower for a single threaded program but 
could speed up applications with multiple threads.

[snip]

Hope that helped a bit and wasn't too confused :-).

Regards,

Carl Friedrich



From pedronis at strakt.com  Sun Jul 17 22:44:47 2005
From: pedronis at strakt.com (Samuele Pedroni)
Date: Sun, 17 Jul 2005 22:44:47 +0200
Subject: [pypy-dev] reconstructing structured flow graphs: pointers
Message-ID: <42DAC33F.8070109@strakt.com>

Our flowgraphs can be rendered easely in languages that support gotos.

For some potential targets in would be nice* to be able to reconstruct
structured graphs that can be rendered in terms of ifs/whiles etc.

For (future) reference these are pointers to literature touching the
subject and containing also further pointers to more literature:

Assembly to High-Level Language Translation
http://citeseer.ist.psu.edu/cifuentes98assembly.html

Structuring Assembly Programs
http://www.cs.uq.edu.au/~cristina/students/doug/doug.html

the algorithms described use concepts and data structures
common in the context of data-flow analysis.

* targeting an underlying bytecode is also a possibility, or
using some while/switch idiom possibly postprocessing the compilation
result or adding a tagbody-like construct support to a compiler for the
target.


From amp at singingwizard.org  Mon Jul 18 21:20:19 2005
From: amp at singingwizard.org (Arthur Peters)
Date: Mon, 18 Jul 2005 14:20:19 -0500
Subject: [pypy-dev] Questions from a lurker
In-Reply-To: <42D98968.1070403@gmx.de>
Message-ID: <E1DubA3-0001O9-7H@lazarus.localnet.local>


I accidently replied to this off list, so the off-list emails (3 of them)
are appended for postarity.

Also I fixed the problem with test_all.py: I had created a __init__.py
file directly in pypy-dist. I don't remember why I created it. Result
was sure strange. Anyway it was my fault.

Thanks all.
-Arthur

----------------------------------------------------

On Sun, 2005-07-17 at 00:25 +0200, Carl Friedrich Bolz wrote:
> Hi Arthur,
> 
> some more notes -- again I might not be the perfect person to answer 
> this, corrections welcome.
> 
> Arthur Peters wrote:
> [snip]
> > Makes sense, but it seems to me that there will be a lot of overlap
> > between the translator and the JIT compiler. Couldn't large parts of
> > the translator be reused?
> 
> Yes and no. The translator (and especially the type inference which we 
> call annotation) works on special assumptions that don't apply to a 
> regular programs: For example that the objects bound to a name in a 
> certain place always have the same type.
> 
> > 
> > BTW, I actually haven't heard for certain that there will be a JIT
> > compiler. Will there be? I'd really like to see one. Python is a great
> > language but runs slow sometimes and I think a JIT could really help.
> > Though one of the main slow things about it is the function call
> > overhead and for that to be reduced JIT compilation would have to happen
> > at a level higher than functions and would need type inference and all
> > that. However it would be really cool to have things like inlining of
> > functions (it would require a decorator to mark the function as
> > invariant so that the compiler knows that it cannot be changed at
> > runtime like normal python functions can). I come from a C++ background
> > so I'm used to having simple functions be as fast as macros.
> 
> Yes, there will be a JIT compiler although probably a bit different from 
> other JIT compilers. The problem is that Python is so very dynamic that 
> you cannot really know /anything/ in advance. Which means that even 
> "just in time" -- e.g. right before the program is run -- is too early.
> (This does not apply to the code our translator translates, because that 
> code, mainly PyPy's interpreter, was written with a lot of limitation 
> especially regarding the dynamism.) The first time you really know 
> anything for sure about a Python function is when you run it for the 
> first time (because then you have fixed values with fixed types to work 
> with). This is (heavily simplified) the idea driving Psyco: Psyco gets 
> information about a function when this function is being executed.
> 
> I'm not totally sure whether the thing that makes Python slow is really 
> the function calls. One of the slower things might be dispatching (which 
> function will be executed for the code a + b) which is a thing that 
> Psyco addresses. And, ideally, the user would not need to take care of 
> telling the JIT which function to inline, the JIT should rather be 
> clever enough to realize that a function is worth inlining. I even think 
> that the JIT could do this better than the programmer.
> 
> > 
> > Also it seems sad to have such a cool and difficult to create things as
> > the translator end up only being used on one program. It seems like it
> > could be useful elsewhere.
> > 
> 
> Yesish. The problem is that it's rather hard to get the translator 
> working for a given Python program (or rather, get the program working 
> for the translator :-) ). At the moment you'd have to adhere to a lot of 
> restrictions which are sometimes not easy to explain. This will problem 
> only change to a certain degree.
> 
> Regards,
> 
> Carl Friedrich 

-------------------------------------------------

On Sat, 2005-07-16 at 17:40 -0500, Arthur Peters wrote:
> On 7/16/2005, "Carl Friedrich Bolz" <cfbolz at gmx.de> wrote:
> >Yes, there will be a JIT compiler although probably a bit different from
> >other JIT compilers. The problem is that Python is so very dynamic that
> >you cannot really know /anything/ in advance. Which means that even
> >"just in time" -- e.g. right before the program is run -- is too early.
> >(This does not apply to the code our translator translates, because that
> >code, mainly PyPy's interpreter, was written with a lot of limitation
> >especially regarding the dynamism.) The first time you really know
> >anything for sure about a Python function is when you run it for the
> >first time (because then you have fixed values with fixed types to work
> >with). This is (heavily simplified) the idea driving Psyco: Psyco gets
> >information about a function when this function is being executed.
> 
> That makes sense.
> 
> >
> >I'm not totally sure whether the thing that makes Python slow is really
> >the function calls. One of the slower things might be dispatching (which
> >function will be executed for the code a + b) which is a thing that
> >Psyco addresses. And, ideally, the user would not need to take care of
> >telling the JIT which function to inline, the JIT should rather be
> >clever enough to realize that a function is worth inlining. I even think
> >that the JIT could do this better than the programmer.
> 
> I agree.
> 
> What I ment was a decorator that said that the method (or perhaps I
> should say method name) would not be rebound. This would allow the
> function to be embeded in the caller without fear of rebinding. This
> would be an issue because this is valid:
> 
> class A:
>   def f(self):
>     pass
> 
> def call_f(a):
>   a.f()
> 
> a = A()
> call_f(a) # does nothing
> 
> def new_f():
>   print "Hi"
> 
> a.f = new_f
> 
> call_f(a) # prints "Hi"
> 
> If A.f were inlined into call_f then the second call to call_f would
> either run the wrong code or would have to trigger a recompile.
> 
> -Arthur

----------------------------------------------------

On Sat, 2005-07-16 at 10:40 -0500, Arthur Peters wrote:
> Answer interlaced.
> 
> On 7/16/2005, "Carl Friedrich Bolz" <cfbolz at gmx.de> wrote:
> >> - I read what some discussion of the GIL as it relate to pypy and I agree
> >> that the GIL need to be implemented and that a different thread model
> >> might be a good way to go. However I thought of the following: Would it
> >> be possible to detect when a section of code only uses local variables
> >> and unlock the GIL? This seems possible because local variables will
> >> cannot be shared between threads anyway. In addition, local variables
> >> likely be translated into more basic types than a python object (native
> >> ints and floats for instance), that would not require any interaction
> >> with the object-space to manipulate (not sure about that usage of the
> >> term "object-space"). Thoughts?
> >
> >First a comment about the usage of the term "object space": I think you
> >are mixing levels here (and I might misunderstand you). The PyPy
> >interpreter (meaning the bytecode interpreter plus the standard
> >objectspace) gets translated to low level code. This "interpreter level
> >code" deals with the standard object space as a regular class instance
> >that is in principle in no way different than any other class. The
> >classes that appear on interpreter level are all translated to more
> >basic types (what would be the alternative, there is no layer below that
> >could deal with anything else), probably to something like a struct.
> >Thus the operations of objects at this level never need to be done via
> >the object space -- the object space is rather a regular object in itself.
> >
> >The object space /is/ used, if you interpret a Python program with
> >PyPy's interpreter. The bytecode interpreter does not now how to deal
> >with /any/ object (except for True and False), it has to delegate to the
> >object space for every single operation. Even for basic types like ints
> >and such -- at this level ("application level") there isn't any type
> >inference or anything like that!
> 
> Makes sense, but it seems to me that there will be a lot of overlap
> between the translator and the JIT compiler. Couldn't large parts of
> the translator be reused?
> 
> BTW, I actually haven't heard for certain that there will be a JIT
> compiler. Will there be? I'd really like to see one. Python is a great
> language but runs slow sometimes and I think a JIT could really help.
> Though one of the main slow things about it is the function call
> overhead and for that to be reduced JIT compilation would have to happen
> at a level higher than functions and would need type inference and all
> that. However it would be really cool to have things like inlining of
> functions (it would require a decorator to mark the function as
> invariant so that the compiler knows that it cannot be changed at
> runtime like normal python functions can). I come from a C++ background
> so I'm used to having simple functions be as fast as macros.
> 
> Also it seems sad to have such a cool and difficult to create things as
> the translator end up only being used on one program. It seems like it
> could be useful elsewhere.
> 
> >Hope that helped a bit and wasn't too confused :-).
> 
> I got most of it. ;-) Thanks a bunch.
> 
> -Arthur


From tismer at stackless.com  Tue Jul 19 13:56:50 2005
From: tismer at stackless.com (Christian Tismer)
Date: Tue, 19 Jul 2005 13:56:50 +0200
Subject: [pypy-dev] rtyper hook ++
Message-ID: <42DCEA82.5060507@stackless.com>

Hi Samuele,

you are not online, therefore using email.

"""
pedronis: stakkars_: btw my new hook allows to get back to a given 
module or function, or to skip things we know specialize fine
"""

I like the idea. Can't we go even a bit further?
If we teach the specializer to not crash if something
does not work out, but to add the problem blocks to a list,
then we could have one report with all pending problems
and work through this in a more general way.
We also would get an estimate how much has still to be done.

The process we are doing by hand (walking PDB stack, looking
at the block etc.) can be either automated or deferred.
Yes, deferring would be the easiest:
The list of problem blocks should be pickleable for later
analysis. The pickling might not be strong enough to
re-run stuff at the moment, but for the debugging, it would
surely be sufficient.

cheers - chris
-- 
Christian Tismer             :^)   <mailto:tismer at stackless.com>
tismerysoft GmbH             :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9A     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 802 86 56  mobile +49 173 24 18 776  fax +49 30 80 90 57 05
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/



From amaury.forgeotdarc at ubitrade.com  Tue Jul 19 15:37:44 2005
From: amaury.forgeotdarc at ubitrade.com (amaury.forgeotdarc at ubitrade.com)
Date: Tue, 19 Jul 2005 15:37:44 +0200
Subject: [pypy-dev] Re: rtyper hook ++
Message-ID: <OF2A9A1481.E358C838-ONC1257043.00438D03@ubitrade.com>

Hello,

Chritian Tismer wrote:
> If we teach the specializer to not crash if something
> does not work out, but to add the problem blocks to a list,
> then we could have one report with all pending problems
> and work through this in a more general way.
> We also would get an estimate how much has still to be done.

That's what I have done to follow the fantastic progress you
make on the translation process.

- In rtyper.py, there is already a crash_on_first_typeerror
variable. Turn it to False, this will catch all TyperErrors.

- There are some assertions and other exceptions that will stop
the process. I changed all of them into TyperErrors.
I suppose that some tests are not done at the right place,
but they do the job...

And the specialization goes to the end!
There are 81 TyperErrors left for 20206 blocks.
They were 230 last week. Good job!

Then the "Back-end optimizations" fail, probably because of
the incompletely specialized blocks.

I hope this helps! Here are my diffs:



Index: pypy/rpython/rstr.py
===================================================================
--- pypy/rpython/rstr.py      (revision 14755)
+++ pypy/rpython/rstr.py      (working copy)
@@ -264,7 +264,8 @@
             if curstr:
                 r.append(curstr)
             curstr = ''
-            assert f in 'xdsrf'
+            if f not in 'xdsrf':
+                raise TyperError("String format not supported: %r" % (f,))

             r.append((f,))
         else:
@@ -295,6 +296,9 @@
             vitem, r_arg = argsiter.next()
             rep = inputconst(Void, r_arg)
             if code == 's' or (code == 'r' and isinstance(r_arg,
InstanceRepr)):
+                if not hasattr(r_arg,'ll_str'):
+                    raise TyperError("AttributeError: %s instance has no
attribute 'll_str'"
+                                     % r_arg)
                 vchunk = hop.gendirectcall(r_arg.ll_str, vitem, rep)
             elif code == 'd':
                 assert isinstance(r_arg, IntegerRepr)
Index: pypy/rpython/callparse.py
===================================================================
--- pypy/rpython/callparse.py (revision 14755)
+++ pypy/rpython/callparse.py (working copy)
@@ -2,6 +2,7 @@
 from pypy.interpreter.argument import Arguments, ArgErr
 from pypy.annotation import model as annmodel
 from pypy.rpython import rtuple
+from pypy.rpython.rmodel import TyperError

 class CallPatternTooComplex(Exception):
     pass
@@ -47,7 +48,7 @@
     try:
         holders = arguments.match_signature(signature, defs_h)
     except ArgErr, e:
-        raise TypeError, "signature mismatch: %s" % e.getmsg(arguments,
func.__name__)
+        raise TyperError, "signature mismatch: %s" % e.getmsg(arguments,
func.__name__)

     assert len(holders) == len(rinputs), "argument parsing mismatch"
     vlist = []
Index: pypy/rpython/rbuiltin.py
===================================================================
--- pypy/rpython/rbuiltin.py  (revision 14755)
+++ pypy/rpython/rbuiltin.py  (working copy)
@@ -107,9 +107,15 @@

 def rtype_builtin_unichr(hop):
     assert hop.nb_args == 1
+    if not hasattr(hop.args_r[0],'rtype_unichr'):
+        raise TyperError("AttributeError: %s instance has no attribute
'rtype_unichr'"
+                         % hop.args_r[0])
     return hop.args_r[0].rtype_unichr(hop)

 def rtype_builtin_list(hop):
+    if not hasattr(hop.args_r[0],'rtype_bltn_list'):
+        raise TyperError("AttributeError: %s instance has no attribute
'rtype_bltn_list'"
+                         % hop.args_r[0])
     return hop.args_r[0].rtype_bltn_list(hop)

 def rtype_builtin_isinstance(hop):
Index: pypy/rpython/rtyper.py
===================================================================
--- pypy/rpython/rtyper.py    (revision 14755)
+++ pypy/rpython/rtyper.py    (working copy)
@@ -31,7 +31,7 @@

 log = py.log.Producer("rtyper")
 py.log.setconsumer("rtyper", None)
-crash_on_first_typeerror = True
+crash_on_first_typeerror = False

 class RPythonTyper:

@@ -45,6 +45,7 @@
         self.pbc_reprs = {}
         self.class_pbc_attributes = {}
         self.typererror = None
+        self.nb_typeerrors = 0
         # make the primitive_to_repr constant mapping
         self.primitive_to_repr = {}
         for s_primitive, lltype in annmodel.annotation_to_ll_map:
@@ -138,8 +139,8 @@
                 n = len(self.already_seen)
                 if n % 100 == 0:
                     total = len(self.annotator.annotated)
-                    print 'specializing: %d / %d blocks   (%d%%)' % (
-                        n, total, 100 * n // total)
+                    print 'specializing: %d / %d blocks   (%d%%, %d
errors)' % (
+                        n, total, 100 * n // total, self.nb_typeerrors)
             # make sure all reprs so far have had their setup() called
             self.call_all_setups()

@@ -200,7 +201,11 @@

     def specialize_block(self, block):
         # give the best possible types to the input args
-        self.setup_block_entry(block)
+        try:
+            self.setup_block_entry(block)
+        except TyperError, e:
+            self.gottypererror(e, block, "block entry", None)
+            return  # cannot continue this block

         # specialize all the operations, as far as possible
         if block.operations == ():   # return or except block
@@ -269,8 +274,9 @@
                     new_a1 = newops.convertvar(a1, r_a1, r_a2)
                 except TyperError, e:
                     self.gottypererror(e, block, link, newops)
-                if new_a1 != a1:
-                    newlinkargs[i] = new_a1
+                else:
+                    if new_a1 != a1:
+                        newlinkargs[i] = new_a1

             if newops:
                 if can_insert_here:
@@ -343,13 +349,16 @@
         """Record a TyperError without crashing immediately.
         Put a 'TyperError' operation in the graph instead.
         """
+        self.nb_typeerrors += 1
         e.where = (block, position)
         if crash_on_first_typeerror:
             raise
+        print "*** TyperError:",e
         if self.typererror is None:
             self.typererror = sys.exc_info()
         c1 = inputconst(Void, Exception.__str__(e))
-        llops.genop('TYPER ERROR', [c1], resulttype=Void)
+        if llops:
+            llops.genop('TYPER ERROR', [c1], resulttype=Void)

     # __________ regular operations __________

@@ -377,7 +386,12 @@
     def translate_op_contains(self, hop):
         r_arg1 = hop.args_r[0]
         r_arg2 = hop.args_r[1]
-        return pair(r_arg1, r_arg2).rtype_contains(hop)
+        p = pair(r_arg1, r_arg2)
+        try:
+            f = p.rtype_contains
+        except AttributeError,e:
+            raise TyperError(str(e))
+        return f(hop)

     # __________ irregular operations __________


--
Amaury Forgeot d'Arc
Ubix Development
www.ubitrade.com




From pedronis at strakt.com  Tue Jul 19 16:26:58 2005
From: pedronis at strakt.com (Samuele Pedroni)
Date: Tue, 19 Jul 2005 16:26:58 +0200
Subject: [pypy-dev] Re: rtyper hook ++
In-Reply-To: <OF2A9A1481.E358C838-ONC1257043.00438D03@ubitrade.com>
References: <OF2A9A1481.E358C838-ONC1257043.00438D03@ubitrade.com>
Message-ID: <42DD0DB2.4070609@strakt.com>

amaury.forgeotdarc at ubitrade.com wrote:

>Hello,
>
>Chritian Tismer wrote:
>  
>
>>If we teach the specializer to not crash if something
>>does not work out, but to add the problem blocks to a list,
>>then we could have one report with all pending problems
>>and work through this in a more general way.
>>We also would get an estimate how much has still to be done.
>>    
>>
>
>That's what I have done to follow the fantastic progress you
>make on the translation process.
>
>- In rtyper.py, there is already a crash_on_first_typeerror
>variable. Turn it to False, this will catch all TyperErrors.
>
>- There are some assertions and other exceptions that will stop
>the process. I changed all of them into TyperErrors.
>I suppose that some tests are not done at the right place,
>but they do the job...
>
>And the specialization goes to the end!
>There are 81 TyperErrors left for 20206 blocks.
>They were 230 last week. Good job!
>
>Then the "Back-end optimizations" fail, probably because of
>the incompletely specialized blocks.
>
>I hope this helps! Here are my diffs:
>
>
>  
>
thanks a lot for suggesting and trying this out. I will try out a 
variation on the patch.
The hook should then be useful to reproduce in a direct and mostly 
deterministic way the problems.



>Index: pypy/rpython/rstr.py
>===================================================================
>--- pypy/rpython/rstr.py      (revision 14755)
>+++ pypy/rpython/rstr.py      (working copy)
>@@ -264,7 +264,8 @@
>             if curstr:
>                 r.append(curstr)
>             curstr = ''
>-            assert f in 'xdsrf'
>+            if f not in 'xdsrf':
>+                raise TyperError("String format not supported: %r" % (f,))
>
>             r.append((f,))
>         else:
>@@ -295,6 +296,9 @@
>             vitem, r_arg = argsiter.next()
>             rep = inputconst(Void, r_arg)
>             if code == 's' or (code == 'r' and isinstance(r_arg,
>InstanceRepr)):
>+                if not hasattr(r_arg,'ll_str'):
>+                    raise TyperError("AttributeError: %s instance has no
>attribute 'll_str'"
>+                                     % r_arg)
>                 vchunk = hop.gendirectcall(r_arg.ll_str, vitem, rep)
>             elif code == 'd':
>                 assert isinstance(r_arg, IntegerRepr)
>Index: pypy/rpython/callparse.py
>===================================================================
>--- pypy/rpython/callparse.py (revision 14755)
>+++ pypy/rpython/callparse.py (working copy)
>@@ -2,6 +2,7 @@
> from pypy.interpreter.argument import Arguments, ArgErr
> from pypy.annotation import model as annmodel
> from pypy.rpython import rtuple
>+from pypy.rpython.rmodel import TyperError
>
> class CallPatternTooComplex(Exception):
>     pass
>@@ -47,7 +48,7 @@
>     try:
>         holders = arguments.match_signature(signature, defs_h)
>     except ArgErr, e:
>-        raise TypeError, "signature mismatch: %s" % e.getmsg(arguments,
>func.__name__)
>+        raise TyperError, "signature mismatch: %s" % e.getmsg(arguments,
>func.__name__)
>
>     assert len(holders) == len(rinputs), "argument parsing mismatch"
>     vlist = []
>Index: pypy/rpython/rbuiltin.py
>===================================================================
>--- pypy/rpython/rbuiltin.py  (revision 14755)
>+++ pypy/rpython/rbuiltin.py  (working copy)
>@@ -107,9 +107,15 @@
>
> def rtype_builtin_unichr(hop):
>     assert hop.nb_args == 1
>+    if not hasattr(hop.args_r[0],'rtype_unichr'):
>+        raise TyperError("AttributeError: %s instance has no attribute
>'rtype_unichr'"
>+                         % hop.args_r[0])
>     return hop.args_r[0].rtype_unichr(hop)
>
> def rtype_builtin_list(hop):
>+    if not hasattr(hop.args_r[0],'rtype_bltn_list'):
>+        raise TyperError("AttributeError: %s instance has no attribute
>'rtype_bltn_list'"
>+                         % hop.args_r[0])
>     return hop.args_r[0].rtype_bltn_list(hop)
>
> def rtype_builtin_isinstance(hop):
>Index: pypy/rpython/rtyper.py
>===================================================================
>--- pypy/rpython/rtyper.py    (revision 14755)
>+++ pypy/rpython/rtyper.py    (working copy)
>@@ -31,7 +31,7 @@
>
> log = py.log.Producer("rtyper")
> py.log.setconsumer("rtyper", None)
>-crash_on_first_typeerror = True
>+crash_on_first_typeerror = False
>
> class RPythonTyper:
>
>@@ -45,6 +45,7 @@
>         self.pbc_reprs = {}
>         self.class_pbc_attributes = {}
>         self.typererror = None
>+        self.nb_typeerrors = 0
>         # make the primitive_to_repr constant mapping
>         self.primitive_to_repr = {}
>         for s_primitive, lltype in annmodel.annotation_to_ll_map:
>@@ -138,8 +139,8 @@
>                 n = len(self.already_seen)
>                 if n % 100 == 0:
>                     total = len(self.annotator.annotated)
>-                    print 'specializing: %d / %d blocks   (%d%%)' % (
>-                        n, total, 100 * n // total)
>+                    print 'specializing: %d / %d blocks   (%d%%, %d
>errors)' % (
>+                        n, total, 100 * n // total, self.nb_typeerrors)
>             # make sure all reprs so far have had their setup() called
>             self.call_all_setups()
>
>@@ -200,7 +201,11 @@
>
>     def specialize_block(self, block):
>         # give the best possible types to the input args
>-        self.setup_block_entry(block)
>+        try:
>+            self.setup_block_entry(block)
>+        except TyperError, e:
>+            self.gottypererror(e, block, "block entry", None)
>+            return  # cannot continue this block
>
>         # specialize all the operations, as far as possible
>         if block.operations == ():   # return or except block
>@@ -269,8 +274,9 @@
>                     new_a1 = newops.convertvar(a1, r_a1, r_a2)
>                 except TyperError, e:
>                     self.gottypererror(e, block, link, newops)
>-                if new_a1 != a1:
>-                    newlinkargs[i] = new_a1
>+                else:
>+                    if new_a1 != a1:
>+                        newlinkargs[i] = new_a1
>
>             if newops:
>                 if can_insert_here:
>@@ -343,13 +349,16 @@
>         """Record a TyperError without crashing immediately.
>         Put a 'TyperError' operation in the graph instead.
>         """
>+        self.nb_typeerrors += 1
>         e.where = (block, position)
>         if crash_on_first_typeerror:
>             raise
>+        print "*** TyperError:",e
>         if self.typererror is None:
>             self.typererror = sys.exc_info()
>         c1 = inputconst(Void, Exception.__str__(e))
>-        llops.genop('TYPER ERROR', [c1], resulttype=Void)
>+        if llops:
>+            llops.genop('TYPER ERROR', [c1], resulttype=Void)
>
>     # __________ regular operations __________
>
>@@ -377,7 +386,12 @@
>     def translate_op_contains(self, hop):
>         r_arg1 = hop.args_r[0]
>         r_arg2 = hop.args_r[1]
>-        return pair(r_arg1, r_arg2).rtype_contains(hop)
>+        p = pair(r_arg1, r_arg2)
>+        try:
>+            f = p.rtype_contains
>+        except AttributeError,e:
>+            raise TyperError(str(e))
>+        return f(hop)
>
>     # __________ irregular operations __________
>
>
>--
>Amaury Forgeot d'Arc
>Ubix Development
>www.ubitrade.com
>
>
>_______________________________________________
>pypy-dev at codespeak.net
>http://codespeak.net/mailman/listinfo/pypy-dev
>  
>



From hpk at trillke.net  Wed Jul 20 20:18:58 2005
From: hpk at trillke.net (holger krekel)
Date: Wed, 20 Jul 2005 20:18:58 +0200
Subject: [pypy-dev] next pypy sprints / some infos
Message-ID: <20050720181858.GR18466@solar.trillke.net>

Hi folks, 

first some quick notes regarding upcoming PyPy sprints.  After the
EuroPython 2005 sprint we decided that we are going for a
sprint that i am organizing at my living and working place,
Trillke-Gut in Hildesheim, Germany.  It is going to start 25th
July and last until 31st July 2005.  Especially already scheduled 
participants please look at 

    http://codespeak.net/pypy/index.cgi?extradoc/sprintinfo/Hildesheim2-sprint.html

for some more information (and correct wrong dates!). 

The strong focus will be on getting a self-contained PyPy
version working.  Really. This time the sprint is only open for
everybody who already knows PyPy.  If you'd like to attend
please send me a private mail.  I am trying to find private
accomodation for everyone.  

Please excuse the extremely short notice but the final
decision was only taken end of last week and i have been
somewhat busy.  Most of the currently active developers
knew about this coming for a longer time, though. 

For the end of August 2005 Carl Friedrich Bolz is organizing 
a sprint more open to new people.  This is going to take place
at the University of Heidelberg, Germany.  More details will 
be announced soon. 

While we are at it: in the further away future, it's likely there 
will be a sprint in Paris around early October, organized by Logilab.  

And if you can imagine helping us to organize a PyPy sprint at 
your place, then please drop us a mail!  For example, we 
haven't been to basically the whole of South Europe :-) 

For your information, we have started last thursday a practise
of having all active pypy developers (and in particular those 
involved in the PyPy/EU project) meet on IRC for 30 minutes.  
This XP-style meeting is meant to synchronize our dev work better 
and give us a body to quickly decide about directions and issues.   
The first minutes of the pypy-sync meeting can be found here:

    http://codespeak.net/pypy/index.cgi?extradoc/minute/pypy-sync-07-14.html

This "active developers" meeting will take place every thursday 
at 1pm (GMT+2/Berlin) for 30 minutes with a half-fixed agenda. 
It is not really open to spectators, i guess.  Those have to 
wait for the minutes :-) 

If you have any questions or comments don't hesitate to 
share them here on the list. 

cheers, 

    holger


From arigo at tunes.org  Thu Jul 21 19:51:28 2005
From: arigo at tunes.org (Armin Rigo)
Date: Thu, 21 Jul 2005 19:51:28 +0200
Subject: [pypy-dev] Build and test failures
In-Reply-To: <OF6198778A.CC3BEF9C-ON8025703D.003140EC-8025703D.0030F3AE@risk.sungard.com>
References: <20050712164040.GC30063@code1.codespeak.net>
	<OF6198778A.CC3BEF9C-ON8025703D.003140EC-8025703D.0030F3AE@risk.sungard.com>
Message-ID: <20050721175128.GA7329@code1.codespeak.net>

Hi Ben,

On Wed, Jul 13, 2005 at 10:03:28AM +0100, Ben.Young at risk.sungard.com wrote:
> Unfortunately thats exposes a new problem, as there is a function called 
> RaiseException included by windows.h! Maybe all your macros should be 
> prefixed by PyPy_
> or something.

Argh.  I see.  Yes, using a common prefix more systematically looks like
a good idea.

> It sounds like you are getting close to the translate_pypy target. Would 
> it be possible for you to give a summary of the problems you appear to be 
> having with the PBC code (as evidenced by some of the commit messages)? 
> I'm just interested in what the problem appears to be, and whether it's 
> just something you're going to have to live with, or if there is an 
> architectural solution? 

The PBCs are messy in general, I don't think we will find a nice a clean
solution.  At least currently, the annotator says that everything is a
"PBC" if it is any kind of user-defined prebuilt object, but this is
only part of the problem.  The real messiness comes from the fact that
the PBCs are somehow the RPython objects that have the least obvious C
equivalents: the Python classes, some prebuilt instances, and functions
(with some of their full Python parameter passing semantics).  Moreover
they come out of the annotator in families (a SomePBC is a set of PBCs),
so we have to handle such families in various ways depending on how the
objects in the family are actually used.

The flow graph viewer doesn't show all this, because it doesn't really
display complex low-level types and constants nicely.  Not sure if and
how to improve that...

> I don't really understand the PBC side of things. Is there a reason why 
> the object hierarchy can't just mirror the way CPython does it? Or would 
> that be too slow?

Yes, we are indeed trying for each kind of PBC family to figure out the
best C-ish equivalent.  A family of functions should become just a C
function pointer; a family of classes should become a vtable pointer;
etc.  We always planned to do that, and our RPython code is written
accordingly, but the devil's in the details...

It looks like we are mostly done, though.  According to
http://codespeak.net/svn/pypy/dist/pypy/translator/goal/ISSUES.txt
the last remaining problem is about conversions from small to larger PBC
families, which should be easy (typically, no conversion is needed e.g.
to cast a pointer-to-function to a
pointer-to-function-that-can-point-to-more-functions-than-the-previous-one).


A bientot,

Armin


From pedronis at strakt.com  Thu Jul 21 19:58:53 2005
From: pedronis at strakt.com (Samuele Pedroni)
Date: Thu, 21 Jul 2005 19:58:53 +0200
Subject: [pypy-dev] Build and test failures
In-Reply-To: <20050721175128.GA7329@code1.codespeak.net>
References: <20050712164040.GC30063@code1.codespeak.net>	<OF6198778A.CC3BEF9C-ON8025703D.003140EC-8025703D.0030F3AE@risk.sungard.com>
	<20050721175128.GA7329@code1.codespeak.net>
Message-ID: <42DFE25D.8040802@strakt.com>

Armin Rigo wrote:
> Hi Ben,
> 
> On Wed, Jul 13, 2005 at 10:03:28AM +0100, Ben.Young at risk.sungard.com wrote:
> 
>>Unfortunately thats exposes a new problem, as there is a function called 
>>RaiseException included by windows.h! Maybe all your macros should be 
>>prefixed by PyPy_
>>or something.
> 
> 
> Argh.  I see.  Yes, using a common prefix more systematically looks like
> a good idea.
> 
> 
>>It sounds like you are getting close to the translate_pypy target. Would 
>>it be possible for you to give a summary of the problems you appear to be 
>>having with the PBC code (as evidenced by some of the commit messages)? 
>>I'm just interested in what the problem appears to be, and whether it's 
>>just something you're going to have to live with, or if there is an 
>>architectural solution? 
> 
> 
> The PBCs are messy in general, I don't think we will find a nice a clean
> solution.  At least currently, the annotator says that everything is a
> "PBC" if it is any kind of user-defined prebuilt object, but this is
> only part of the problem.  The real messiness comes from the fact that
> the PBCs are somehow the RPython objects that have the least obvious C
> equivalents: the Python classes, some prebuilt instances, and functions
> (with some of their full Python parameter passing semantics).  Moreover
> they come out of the annotator in families (a SomePBC is a set of PBCs),
> so we have to handle such families in various ways depending on how the
> objects in the family are actually used.
> 
> The flow graph viewer doesn't show all this, because it doesn't really
> display complex low-level types and constants nicely.  Not sure if and
> how to improve that...
> 
> 
>>I don't really understand the PBC side of things. Is there a reason why 
>>the object hierarchy can't just mirror the way CPython does it? Or would 
>>that be too slow?
> 
> 
> Yes, we are indeed trying for each kind of PBC family to figure out the
> best C-ish equivalent.  A family of functions should become just a C
> function pointer; a family of classes should become a vtable pointer;
> etc.  We always planned to do that, and our RPython code is written
> accordingly, but the devil's in the details...
> 
> It looks like we are mostly done, though.  According to
> http://codespeak.net/svn/pypy/dist/pypy/translator/goal/ISSUES.txt
> the last remaining problem is about conversions from small to larger PBC
> families, which should be easy (typically, no conversion is needed e.g.
> to cast a pointer-to-function to a
> pointer-to-function-that-can-point-to-more-functions-than-the-previous-one).
> 
> 

at least the PyPy codebase doesn't show such kind of conversion issues 
anymore. Now we are with some functionality missing the rtyper and nasty 
issues related to faking code and appearing PyObjects.

I should updated the ISSUES file, with the list of error that 
translate_pypy with -t-insist produces.





From Ben.Young at risk.sungard.com  Fri Jul 22 10:18:56 2005
From: Ben.Young at risk.sungard.com (Ben.Young at risk.sungard.com)
Date: Fri, 22 Jul 2005 09:18:56 +0100
Subject: [pypy-dev] Build and test failures
In-Reply-To: <20050721175128.GA7329@code1.codespeak.net>
Message-ID: <OFE6C37D3E.9C23B0FE-ON80257046.002CF74A-80257046.002DAE2E@risk.sungard.com>

Armin Rigo <arigo at tunes.org> wrote on 21/07/2005 18:51:28:

> Hi Ben,
> 

Snip ...

> 
> The flow graph viewer doesn't show all this, because it doesn't really
> display complex low-level types and constants nicely.  Not sure if and
> how to improve that...
> 
> > I don't really understand the PBC side of things. Is there a reason 
why 
> > the object hierarchy can't just mirror the way CPython does it? Or 
would 
> > that be too slow?
> 
> Yes, we are indeed trying for each kind of PBC family to figure out the
> best C-ish equivalent.  A family of functions should become just a C
> function pointer; a family of classes should become a vtable pointer;
> etc.  We always planned to do that, and our RPython code is written
> accordingly, but the devil's in the details...
> 

So is a family of classes the set of any classes that can be assigned to a 
particular instance variable, even if they dont share a common base?

e.g:

class A:
        pass

class B:
        pass

foo = somebool ? A() : B()

Would it be simpler to require that all instances must at least be 
annotated to a common base, and fail to annotate if a not-base-defined 
method is called on the instance. Or is that really what you are already 
doing!?

Do function PBCs all have to have the same signature, or can it vary?

Anyway, thanks for the reply! As illuminating as ever. I guess with the 
rtyping, the first flush of exciting coding has gone and you are down to 
the grinding out bugs stage. Are you pleased with the way things have gone 
or are there any chages you would make to the architecture now?

Cheers,
Ben

P.S os.ftruncate doesn't exist on windows I'm afraid, so most of the tests 
fail at the moment

> 
> A bientot,
> 
> Armin
> 



From tismer at stackless.com  Mon Jul 25 20:06:18 2005
From: tismer at stackless.com (Christian Tismer)
Date: Mon, 25 Jul 2005 20:06:18 +0200
Subject: [pypy-dev] Re: [pypy-svn] r15069 -
	pypy/dist/pypy/translator/llvm2/tool
In-Reply-To: <20050725175033.0903227B5A@code1.codespeak.net>
References: <20050725175033.0903227B5A@code1.codespeak.net>
Message-ID: <42E52A1A.7090404@stackless.com>

ericvrp at codespeak.net wrote:

> Wrote simple tool to get an overview of the suggested_primitive functions that
> still need to be implemented in the llvm backend.
> The tool should be run from it's own directory for now!
> This is the current output: (tail is what we want to know)

Nice tool, very useful!

Just a hint for further development:

PyPy tends to avoid source code analysis wherever possible.
And in this tool's case it would bhe quite easy to use
introspecction:


> +    ll_modules_path = '../../../rpython/module'
> +    ll_files = [ll_modules_path + '/' + f for f in os.listdir(ll_modules_path) if f[:3] == 'll_' and f[-3:] == '.py']
> +    ll_function = {}    #XXX better use sets
> +    for ll_file in ll_files:

instead of this,

> +        for s in file(ll_file):
> +            s = s.strip()
> +            if not s.startswith('ll_') or s.find('suggested_primitive') == -1 or s.find('True') == -1:
> +                continue
> +            ll_function[s.split('.')[0]] = True

you could just import the file and inspect its objects.
Then check for
   hasattr(func, 'suggested_primitive')
and check if it is true.

> +
> +    llvm_modules_path = '..'
> +    llvm_files = [llvm_modules_path + '/' + 'extfunction.py']
> +    llvm_function = {}
> +    for llvm_file in llvm_files:
> +        t = 'extfunctions["%'

Same here.
Just import extfunctions from the file and look into it.

This is just more robust about future changes and independent
of formatting. It also survives stuff like programmatically
adding bulks of stuff to extfunctions, which might happen at
some point.

Just my 0.2cent - ciao -- chris

-- 
Christian Tismer             :^)   <mailto:tismer at stackless.com>
tismerysoft GmbH             :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9A     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 802 86 56  mobile +49 173 24 18 776  fax +49 30 80 90 57 05
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/


From cfbolz at gmx.de  Tue Jul 26 00:39:10 2005
From: cfbolz at gmx.de (Carl Friedrich Bolz)
Date: Tue, 26 Jul 2005 00:39:10 +0200
Subject: [pypy-dev] Hildesheim sprint progress report
Message-ID: <42E56A0E.2000904@gmx.de>

Just some quick (and probably confused) notes about went on at the 
Hildesheim sprint day 1. Corrections welcome.
As in Gothenburg we tried to write an evolving file that listed all the 
tasks:

http://codespeak.net/svn/pypy/extradoc/sprintinfo/hildesheim2-planning.txt

somehow we didn't get very far and decided that we just would all work 
on rtyper issues. The following people are here: Hogler, Armin, Samuele, 
Christian, Richard, Carl Friedrich.

Holger and Christian paired and started removing some amount of faking 
in PyPy. They did this by removing the usage of file as well as 
stdout/stdin in the PyPy interpreter and resorting to the os primitives 
instead, which can be more easily translated to C.

Armin and Christian worked on various rtyper issues. They worked on 
unicode, fixed hashing for numerical types, and worked on marshal.

Samuele and me worked on external functions. We cleaned up some of the 
hooks that were there before the current external function mechanism and 
implemented some math and os function for the C backend so that you can 
now actually use some of the primitive file operations like fdopen, 
stat, write... without using the CPython API.

Cheers,

Carl Friedrich


From hpk at trillke.net  Tue Jul 26 11:46:34 2005
From: hpk at trillke.net (holger krekel)
Date: Tue, 26 Jul 2005 11:46:34 +0200
Subject: [pypy-dev] translation and tests
Message-ID: <20050726094634.GM3501@solar.trillke.net>

Hello pypy-dev! 

today at the hildesheim2 sprint we decided that we'd
like to annotate and translate from the trunk during
the sprint.  After the sprint we are probably changing
back to work from a snapshot.  This is more convenient 
than porting changes to the snapshost all the time as 
we are making a lot of changes.  

Also, please try to avoid checking in changes with
failing tests.  All of us are doing it sometimes, 
for example, when we change computers or locations
and want to continue working somehwere else. However, 
even then it's best if you _temporarily_ inhibit 
the collection of the affected tests by placing a 
conftest.py file into an appropriate directory, 
containing this: 

    import py 
    class Directory(py.test.collect.Directory): 
        def run(self):
            return []

This will tell py.test that there are no tests 
to be collected in the directory and all its 
sub directories. You can still run test files
directly. 

If you want to checkin a failing test where you 
don't know yet how to fix it, then you can either
add a 'py.test.skip("please help with this problem ...")' 
at the top of the failing test function or you can mark a 
whole file with 'inprogress_test_something.py' which will
inhibit automatic collection while still allowing 
to run 'py.test inprogress_test_something.py'. 

Of course, the basic idea of all these suggestions
is that we want to be able to run all PyPy tests to 
see if we messed up somehow.  

thanks and greetings, 

    holger & samuele


From cfbolz at gmx.de  Tue Jul 26 14:04:03 2005
From: cfbolz at gmx.de (Carl Friedrich Bolz)
Date: Tue, 26 Jul 2005 14:04:03 +0200
Subject: [pypy-dev] Hildesheim sprint progress report
In-Reply-To: <42E56A0E.2000904@gmx.de>
References: <42E56A0E.2000904@gmx.de>
Message-ID: <42E626B3.7030208@gmx.de>

Oops, shouldn't write mails late at night....
Carl Friedrich Bolz wrote:

>
> Holger and Christian paired and started removing some amount of faking 
> in PyPy. They did this by removing the usage of file as well as 
> stdout/stdin in the PyPy interpreter and resorting to the os 
> primitives instead, which can be more easily translated to C.
>
This should be Holger and _Richard_, not Christian.


From hpk at trillke.net  Thu Jul 28 10:48:17 2005
From: hpk at trillke.net (holger krekel)
Date: Thu, 28 Jul 2005 10:48:17 +0200
Subject: [pypy-dev] hildesheim2 day 2 + 3 report / status
Message-ID: <20050728084817.GC3501@solar.trillke.net>

Hi folks, 

the end of the world is near ... 
following up on Carl's day 1 report here are the main results
of what happened up until now at the hildesheim2 sprint (we are
in the break day currently which we spent in the city, at
Wing-Tsun lessons or at family meetings). 

The main results so far are: 

- any faking can be removed now with 

    python2.4 py.py --file --nofakedmodules 

  so on the trunk you should see no 'faking' messages
  during startup anymore. It will take some time
  mostly because of the 'file' implementation living
  at app-level.  Note that we have some problems
  running on top of Python2.3 at the moment and 
  also some more problems with PYC file support. 

- Annotation can now complete without 
  resorting to any SomeObjects anywhere (!!)

- RTyping has only 2 Errors remaining. The next bigger
  problem is getting the compiler package integrated 
  and bootstrapped properly. 

- note that we are not working from the snapshot 
  but the trunk currently.  After the sprint 
  we probably return to the snapshot approach
  for translation. 

If you want to see more detailed daily-updated progress 
reports i suggest to read and follow the commits on 

http://codespeak.net/pypy/index.cgi?extradoc/sprintinfo/hildesheim2-planning.txt 
Moreover, i shot a few pictures which i put here: 

    http://codespeak.net/~hpk/hildesheim2-sprint-www

and here is a picture of the surrounding place 
(Trillke-Gut, my living&working base): 

   http://www.trillke.net/images/HomePagePictureSmall.jpg 

cheers, 

    holger


P.S.: It appears that we will have to continue our
      investigations regarding some of the current problems on
      the astral plane.  Armin and Carl has already spend some
      time on assembler level yesterday to hunt down the
      weirdest TCC interactions. But there are other confusing
      events as well.  um, we watched 'the nine lives of
      Thomas Katz' yesterday night, btw.  
      Anyway, we are still taking bets if the first C-generated
      annotated RTyped PyPy is to produce a segmentation fault
      in the C-compiler or in the resulting binary.  


From Ben.Young at risk.sungard.com  Thu Jul 28 11:02:57 2005
From: Ben.Young at risk.sungard.com (Ben.Young at risk.sungard.com)
Date: Thu, 28 Jul 2005 10:02:57 +0100
Subject: [pypy-dev] hildesheim2 day 2 + 3 report / status
In-Reply-To: <20050728084817.GC3501@solar.trillke.net>
Message-ID: <OF2AF0AB7D.CF01805D-ON8025704C.00317199-8025704C.0031B3A9@risk.sungard.com>

Hi Holger,

It sounds like you are making great progress! I was just wondering when 
would be a good time to report test failures, as I am currently getting 
about 25 failing tests at the moment (on Windows cygwin)? I understand 
that things are in a state of flux at the moment and will probably settle 
down again after the sprint.

Keep up the good work!

Cheers,
Ben


pypy-dev-bounces at codespeak.net wrote on 28/07/2005 09:48:17:

> Hi folks, 
> 
> the end of the world is near ... 
> following up on Carl's day 1 report here are the main results
> of what happened up until now at the hildesheim2 sprint (we are
> in the break day currently which we spent in the city, at
> Wing-Tsun lessons or at family meetings). 
> 
> The main results so far are: 
> 
> - any faking can be removed now with 
> 
>     python2.4 py.py --file --nofakedmodules 
> 
>   so on the trunk you should see no 'faking' messages
>   during startup anymore. It will take some time
>   mostly because of the 'file' implementation living
>   at app-level.  Note that we have some problems
>   running on top of Python2.3 at the moment and 
>   also some more problems with PYC file support. 
> 
> - Annotation can now complete without 
>   resorting to any SomeObjects anywhere (!!)
> 
> - RTyping has only 2 Errors remaining. The next bigger
>   problem is getting the compiler package integrated 
>   and bootstrapped properly. 
> 
> - note that we are not working from the snapshot 
>   but the trunk currently.  After the sprint 
>   we probably return to the snapshot approach
>   for translation. 
> 
> If you want to see more detailed daily-updated progress 
> reports i suggest to read and follow the commits on 
> 
> http://codespeak.net/pypy/index.cgi?extradoc/sprintinfo/hildesheim2-
> planning.txt 
> Moreover, i shot a few pictures which i put here: 
> 
>     http://codespeak.net/~hpk/hildesheim2-sprint-www
> 
> and here is a picture of the surrounding place 
> (Trillke-Gut, my living&working base): 
> 
>    http://www.trillke.net/images/HomePagePictureSmall.jpg 
> 
> cheers, 
> 
>     holger
> 
> 
> P.S.: It appears that we will have to continue our
>       investigations regarding some of the current problems on
>       the astral plane.  Armin and Carl has already spend some
>       time on assembler level yesterday to hunt down the
>       weirdest TCC interactions. But there are other confusing
>       events as well.  um, we watched 'the nine lives of
>       Thomas Katz' yesterday night, btw. 
>       Anyway, we are still taking bets if the first C-generated
>       annotated RTyped PyPy is to produce a segmentation fault
>       in the C-compiler or in the resulting binary. 
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev
> 
> 



From hpk at trillke.net  Thu Jul 28 12:05:56 2005
From: hpk at trillke.net (holger krekel)
Date: Thu, 28 Jul 2005 12:05:56 +0200
Subject: [pypy-dev] hildesheim2 day 2 + 3 report / status
In-Reply-To: <OF2AF0AB7D.CF01805D-ON8025704C.00317199-8025704C.0031B3A9@risk.sungard.com>
References: <20050728084817.GC3501@solar.trillke.net>
	<OF2AF0AB7D.CF01805D-ON8025704C.00317199-8025704C.0031B3A9@risk.sungard.com>
Message-ID: <20050728100556.GE3501@solar.trillke.net>

Hi Ben! 

On Thu, Jul 28, 2005 at 10:02 +0100, Ben.Young at risk.sungard.com wrote:
> It sounds like you are making great progress! I was just wondering when 
> would be a good time to report test failures, as I am currently getting 
> about 25 failing tests at the moment (on Windows cygwin)? I understand 
> that things are in a state of flux at the moment and will probably settle 
> down again after the sprint.

hopefully, yes :-) 

If you keep getting those errors over more than a day then i
suggest to post them here to the list (or put them somewhere
on a web server and post a link). 

cheers, 

    holger

P.S: you shouldn't sent mail to pypy-dev-bounces at codespeak.net 
     which was in your CC list. 


From Ben.Young at risk.sungard.com  Thu Jul 28 12:27:55 2005
From: Ben.Young at risk.sungard.com (Ben.Young at risk.sungard.com)
Date: Thu, 28 Jul 2005 11:27:55 +0100
Subject: [pypy-dev] hildesheim2 day 2 + 3 report / status
In-Reply-To: <20050728100556.GE3501@solar.trillke.net>
Message-ID: <OFBFBE3298.494040A4-ON8025704C.0039264A-8025704C.00397AE3@risk.sungard.com>

holger krekel <hpk at trillke.net> wrote on 28/07/2005 11:05:56:

> Hi Ben! 
> 
> On Thu, Jul 28, 2005 at 10:02 +0100, Ben.Young at risk.sungard.com wrote:
> > It sounds like you are making great progress! I was just wondering 
when 
> > would be a good time to report test failures, as I am currently 
getting 
> > about 25 failing tests at the moment (on Windows cygwin)? I understand 

> > that things are in a state of flux at the moment and will probably 
settle 
> > down again after the sprint.
> 
> hopefully, yes :-) 
> 
> If you keep getting those errors over more than a day then i
> suggest to post them here to the list (or put them somewhere
> on a web server and post a link). 
> 

Thanks Holger,

Ok, these ones have all been failing for a few days:



Most of them look like fairly simple bugs ( note I have got rid of all 
references to ftruncate locally, as it doesn't exist on windows). The time 
failure appears to be happening because the CPython timer and the PyPy 
timer are seperate, so maybe they shouldn't be compared to each other?

Cheers,
Ben

P.S Sorry about the wrong cc

> cheers, 
> 
>     holger
> 
> P.S: you shouldn't sent mail to pypy-dev-bounces at codespeak.net 
>      which was in your CC list. 
> 
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: test28072005.txt
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20050728/480de150/attachment.txt>

From Ben.Young at risk.sungard.com  Thu Jul 28 13:37:28 2005
From: Ben.Young at risk.sungard.com (Ben.Young at risk.sungard.com)
Date: Thu, 28 Jul 2005 12:37:28 +0100
Subject: [pypy-dev] hildesheim2 day 2 + 3 report / status
In-Reply-To: <20050728100556.GE3501@solar.trillke.net>
Message-ID: <OF05A0D815.DC62CD56-ON8025704C.003FBEDF-8025704C.003FD90D@risk.sungard.com>

pypy-dev-bounces at codespeak.net wrote on 28/07/2005 11:05:56:

> Hi Ben! 
> 
> On Thu, Jul 28, 2005 at 10:02 +0100, Ben.Young at risk.sungard.com wrote:
> > It sounds like you are making great progress! I was just wondering 
when 
> > would be a good time to report test failures, as I am currently 
getting 
> > about 25 failing tests at the moment (on Windows cygwin)? I understand 

> > that things are in a state of flux at the moment and will probably 
settle 
> > down again after the sprint.
> 
> hopefully, yes :-) 
> 
> If you keep getting those errors over more than a day then i
> suggest to post them here to the list (or put them somewhere
> on a web server and post a link). 

On another note, I noticed this, in interactive mode:

>>>> dis.dis(compile("\"main\"", "", "exec"))
  0           0 LOAD_CONST               1 ('main')
              3 STORE_NAME               0 (__doc__)
              6 LOAD_CONST               0 (None)
              9 RETURN_VALUE

I assume in interactive mode we should not be assigning to the __doc__ 
attr. This causes string literals not to be printed as well.

Cheers,
Ben


> 
> cheers, 
> 
>     holger
> 
> P.S: you shouldn't sent mail to pypy-dev-bounces at codespeak.net 
>      which was in your CC list. 
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev
> 
> 



From hpk at trillke.net  Thu Jul 28 14:03:22 2005
From: hpk at trillke.net (holger krekel)
Date: Thu, 28 Jul 2005 14:03:22 +0200
Subject: [pypy-dev] Re: [pypy-svn] r15225 - in pypy/dist/pypy: module/math
	rpython rpython/module
In-Reply-To: <20050728085619.7FF8827B6C@code1.codespeak.net>
References: <20050728085619.7FF8827B6C@code1.codespeak.net>
Message-ID: <20050728120322.GG3501@solar.trillke.net>

Hi Anders, 

On Thu, Jul 28, 2005 at 10:56 +0200, ale at codespeak.net wrote:
> Author: ale
> Date: Thu Jul 28 10:56:16 2005
> New Revision: 15225
> 
> Modified:
>    pypy/dist/pypy/module/math/__init__.py
>    pypy/dist/pypy/rpython/extfunctable.py
>    pypy/dist/pypy/rpython/llinterp.py
>    pypy/dist/pypy/rpython/module/ll_math.py

we should try to add tests when adding new external
function implementations. Otherwise we get frustrating
errors in translation after running for >10 minutes. 

> Modified: pypy/dist/pypy/rpython/module/ll_math.py
> ==============================================================================
> --- pypy/dist/pypy/rpython/module/ll_math.py	(original)
> +++ pypy/dist/pypy/rpython/module/ll_math.py	Thu Jul 28 10:56:16 2005
> @@ -2,6 +2,10 @@
>  
>  import math
>  
> +def ll_math_log(x):
> +    return math.log(x)
> +ll_math_log.suggested_primitive = True

'log' actually takes two arguments (see pypy/module/math/interp_math.py
or plain python's math.log).  On a related note, ll-external function 
implementations cannot have default arguments. 

cheers, 

    holger


From hpk at trillke.net  Thu Jul 28 16:04:04 2005
From: hpk at trillke.net (holger krekel)
Date: Thu, 28 Jul 2005 16:04:04 +0200
Subject: [pypy-dev] Re: [pypy-svn] r15225 - in pypy/dist/pypy: module/math
	rpython rpython/module
In-Reply-To: <20050728120322.GG3501@solar.trillke.net>
References: <20050728085619.7FF8827B6C@code1.codespeak.net>
	<20050728120322.GG3501@solar.trillke.net>
Message-ID: <20050728140404.GI3501@solar.trillke.net>

On Thu, Jul 28, 2005 at 14:03 +0200, holger krekel wrote:
> > +def ll_math_log(x):
> > +    return math.log(x)
> > +ll_math_log.suggested_primitive = True
> 
> 'log' actually takes two arguments (see pypy/module/math/interp_math.py
> or plain python's math.log).  

Actually (i just got to know) that the C backend's log() 
only takes one argument so Carl Friedrich just fixed
the invocation from the pypy/module/math module and
now the two signatures match.

    holger


From nhaldimann at gmx.ch  Fri Jul 29 01:11:51 2005
From: nhaldimann at gmx.ch (Niklaus Haldimann)
Date: Fri, 29 Jul 2005 01:11:51 +0200
Subject: [pypy-dev] Confused about RPython
Message-ID: <42E96637.5070303@gmx.ch>

Hi everyone

I am currently seriously confused about the notion of RPython, and I 
hope someone can show me the light here. ;)

I guess my basic problem is: How do I find out if some code is RPython? 
And especially in the context of the array module im currently working 
on, how do I find out if a module is RPython?

One of my working hypothesis was that whatever can be geninterped is 
RPython. But by that definition, a *lot* more comes out as RPython than 
the documentation says. Is it just that the documentation is out-of-date 
or is geninterping not a good criterion? E.g., when Christian states 
that _codecs is now RPython, does that just mean it can be geninterped 
or is there something else to it?

Armin just had a checkin where he replaced str.isalpha() with something 
else because string methods are not RPython. But I can geninterp and 
even C compile string methods just fine. Did he use the term "RPython" 
loosely, meaning something like "what can be usefully annotated" (ie, 
string methods always result in SomeObject, that's probably why they're 
frowned upon in the interpreter code)?

Cheers
Nik


From mwh at python.net  Fri Jul 29 12:23:51 2005
From: mwh at python.net (Michael Hudson)
Date: Fri, 29 Jul 2005 11:23:51 +0100
Subject: [pypy-dev] Re: Confused about RPython
References: <42E96637.5070303@gmx.ch>
Message-ID: <2mfytx29mw.fsf@starship.python.net>

Niklaus Haldimann <nhaldimann at gmx.ch> writes:

> Hi everyone
>
> I am currently seriously confused about the notion of RPython, and I
> hope someone can show me the light here. ;)
>
> I guess my basic problem is: How do I find out if some code is
> RPython?

Hrm.  If it gets annotated without SomeObjects, I guess it's RPython.

> Armin just had a checkin where he replaced str.isalpha() with
> something else because string methods are not RPython. But I can
> geninterp and even C compile string methods just fine.

Are they C compiled to code using the Python C API?

Cheers,
mwh

-- 
  Usenet is like a herd of performing elephants with diarrhea --
  massive, difficult to redirect, awe-inspiring, entertaining, and
  a source of mind-boggling amounts of excrement when you least 
  expect it.                                             -- spaf (1992)



From tismer at stackless.com  Fri Jul 29 13:29:34 2005
From: tismer at stackless.com (Christian Tismer)
Date: Fri, 29 Jul 2005 13:29:34 +0200
Subject: [pypy-dev] Confused about RPython
In-Reply-To: <42E96637.5070303@gmx.ch>
References: <42E96637.5070303@gmx.ch>
Message-ID: <42EA131E.9030906@stackless.com>

Hi Niklaus,

> I am currently seriously confused about the notion of RPython, and I 
> hope someone can show me the light here. ;)

blink blink :)

> I guess my basic problem is: How do I find out if some code is RPython? 
> And especially in the context of the array module im currently working 
> on, how do I find out if a module is RPython?

I see you want to write the array module at interplevel.
This makes sense, applevel has problems here. interplevel
with possibly give you problems as well, I guess, because
I have no clue how to express the buffer interface for instance.
We don't have pointers and too few basic types.
If you stubleofver such problems, please don't hesitate to
contact the list.

> One of my working hypothesis was that whatever can be geninterped is 
> RPython. But by that definition, a *lot* more comes out as RPython than 
> the documentation says. Is it just that the documentation is out-of-date 
> or is geninterping not a good criterion? E.g., when Christian states 
> that _codecs is now RPython, does that just mean it can be geninterped 
> or is there something else to it?

Well, I think my notation is a biut confusing, of course.
Should have tagged nn-geninterpable code differently.
The tag "NOT_RPYTHON" is used inside PyPyto signal stuff that
we should not try to translate, the annotator will raise an exception
if it sees such a thing, and so on.
So I used the same tag to signal that "This applevel code is not
suitable for geninterp". Of course, this is some kind of "applevel
RPython" in the sense that it can be pured through geninterp
to create an RPython interplevel module.

And this is all what it is saying. If you are writing on interp level,
geninterp doesn't help. You need to translate to some target and see
whether flowspace and rtyper are happy with it.

> Armin just had a checkin where he replaced str.isalpha() with something 
> else because string methods are not RPython. But I can geninterp and 
> even C compile string methods just fine. Did he use the term "RPython" 
> loosely, meaning something like "what can be usefully annotated" (ie, 
> string methods always result in SomeObject, that's probably why they're 
> frowned upon in the interpreter code)?

I used it loosely, in a sense. Maybe I should rename the tag.

cheers - chris

-- 
Christian Tismer             :^)   <mailto:tismer at stackless.com>
tismerysoft GmbH             :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9A     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 802 86 56  mobile +49 173 24 18 776  fax +49 30 80 90 57 05
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/


From nhaldimann at gmx.ch  Fri Jul 29 14:36:22 2005
From: nhaldimann at gmx.ch (Niklaus Haldimann)
Date: Fri, 29 Jul 2005 14:36:22 +0200
Subject: [pypy-dev] Confused about RPython
In-Reply-To: <42EA131E.9030906@stackless.com>
References: <42E96637.5070303@gmx.ch> <42EA131E.9030906@stackless.com>
Message-ID: <42EA22C6.8030202@gmx.ch>

Christian Tismer wrote:
> I see you want to write the array module at interplevel.
> This makes sense, applevel has problems here. interplevel
> with possibly give you problems as well, I guess, because
> I have no clue how to express the buffer interface for instance.
> We don't have pointers and too few basic types.
> If you stubleofver such problems, please don't hesitate to
> contact the list.

I am actually trying to write it in applevel, making sure it can be 
geninterped. If I can do this in a resonable amount of time I might try 
to rewrite some stuff on interplevel. You can see the current effort 
here, BTW:
http://codespeak.net/svn/user/nik/array_py/trunk/src/array.py

I think the buffer interface is a no-brainer, but maybe I'm missing 
something. Our buffer type is written at applevel, we have no obligation 
at all to emulate CPython's C-based buffer interface. We can define our 
own interface, and in fact the current buffer implementation sort of 
does that. Hacking array support into the current buffer will be a 
matter of 2-3 lines of code, AFAICT. More effort would be a waste of 
time anyway, as buffer is as good as deprecated. ;)

> Well, I think my notation is a biut confusing, of course.
> Should have tagged nn-geninterpable code differently.
> The tag "NOT_RPYTHON" is used inside PyPyto signal stuff that
> we should not try to translate, the annotator will raise an exception
> if it sees such a thing, and so on.
> So I used the same tag to signal that "This applevel code is not
> suitable for geninterp". Of course, this is some kind of "applevel
> RPython" in the sense that it can be pured through geninterp
> to create an RPython interplevel module.

I see, that really clears things up a bit. Is it correct to say that 
geninterping such an "applevel RPython" module guarantees that the 
resulting interplevel code can in fact be translated? That sounds a bit 
too much like magic to me.

> And this is all what it is saying. If you are writing on interp level,
> geninterp doesn't help. You need to translate to some target and see
> whether flowspace and rtyper are happy with it.

OK. Just to be a bit more explicit about this, I do something like:

 >>> t = Translator(my_rpython_function)
 >>> t.annotate([int])

And check that the inferred return type is not SomeObject. This 
basically means that flowspace and rtyper are happy, right? What I don't 
understand ATM is where the translator will get the above "[int]" 
annotation hint from in the real world ...

Thanks for showing me the light so far. ;)

Cheers
Nik


From tismer at stackless.com  Fri Jul 29 15:34:36 2005
From: tismer at stackless.com (Christian Tismer)
Date: Fri, 29 Jul 2005 15:34:36 +0200
Subject: [pypy-dev] Confused about RPython
In-Reply-To: <42EA22C6.8030202@gmx.ch>
References: <42E96637.5070303@gmx.ch> <42EA131E.9030906@stackless.com>
	<42EA22C6.8030202@gmx.ch>
Message-ID: <42EA306C.7000901@stackless.com>

Niklaus Haldimann wrote:

...

> I think the buffer interface is a no-brainer, but maybe I'm missing 
> something. Our buffer type is written at applevel, we have no obligation 
> at all to emulate CPython's C-based buffer interface. We can define our 
> own interface, and in fact the current buffer implementation sort of 
> does that. Hacking array support into the current buffer will be a 
> matter of 2-3 lines of code, AFAICT. More effort would be a waste of 
> time anyway, as buffer is as good as deprecated. ;)

Sounds reasonable.
Actually I didn't look too closely into the array module myself,
I was just repeating what another guy reported when trying to
implement it. So it might be easier than I expcetged.

...

> I see, that really clears things up a bit. Is it correct to say that 
> geninterping such an "applevel RPython" module guarantees that the 
> resulting interplevel code can in fact be translated? That sounds a bit 
> too much like magic to me.

No, it doesn't. It tells you that the flow space is happy with
your code, but still there may be issues with annotation I think.
And stuff like producing an infinite recursion is not detected
at all. For instance, we had code that relied on applevel in
a way that the implementation called into the interpreter for
something that then goes back to the implementation which...

So I'd say geninterping is a necessary but not sufficient condition.

...

> OK. Just to be a bit more explicit about this, I do something like:
> 
>  >>> t = Translator(my_rpython_function)
>  >>> t.annotate([int])
> 
> And check that the inferred return type is not SomeObject. This 
> basically means that flowspace and rtyper are happy, right? What I don't 
> understand ATM is where the translator will get the above "[int]" 
> annotation hint from in the real world ...

You are giving it the hint to have some reasonable starting point
for testing, assuming that this assumption generally holds.

In the real world, everything and all of PyPy is run through the
annotator, which re-iterates stuff until it ends up with
a proven interface, or it has to give up with someobjectness.
So the realanalysis depends on whether your implementation
is really called in the right way, all over PyPy.

ciao - chris
-- 
Christian Tismer             :^)   <mailto:tismer at stackless.com>
tismerysoft GmbH             :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9A     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 802 86 56  mobile +49 173 24 18 776  fax +49 30 80 90 57 05
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/


From arigo at tunes.org  Fri Jul 29 16:22:11 2005
From: arigo at tunes.org (Armin Rigo)
Date: Fri, 29 Jul 2005 16:22:11 +0200
Subject: [pypy-dev] Re: Strategy for array module?
In-Reply-To: <42E66185.4050807@gmx.ch>
References: <42E66185.4050807@gmx.ch>
Message-ID: <20050729142211.GA13598@code1.codespeak.net>

Hi Niklaus,

Sorry for the delay.  I've indeed been quite deeply into the PyPy
sprint...

At which level and with which style the array module should be
implemented is indeed a bit unclear.  If you are going for an app-level
version, you should not worry too much about geninterp, I guess.  The
code you already started to write makes sense.  Geninterp is most useful
to solve bootstrapping issues, where we cannot import yet some app-level
code but we can run its geninterp'ed version (e.g. for the
_exceptions.py module).  Of course geninterp gives some performance
improvement too.

The problem with array is that it is designed with the following model
in mind: a bunch of memory out of which Python values can be read on
demand.  The other point of view -- a list of Python values that can be
turned into a big string on demand -- make methods like byteswap() hard
to implement, specially for arrays of floats or doubles.  Not
necessarily impossible, though.

If this app-level approach works out, I wouldn't invest too much effort
in geninterp compatibility, because at some point in the future we might
have to think about a quite different approach: adding the notion of
integers and floats of specific sizes, at least in the so-called
"low-level" types.  I guess that an efficient implementation of array
should be written to take advantage of these.  This will likely require
as-yet-unspecified ways to interact between regular interp-level code
and this low-level-typed code.

In summary I guess that writing a pure app-level array module makes
sense, with the buffer interface working for PyPy only.

On Tue, Jul 26, 2005 at 06:15:01PM +0200, Niklaus Haldimann wrote:
> As I probably told you, Lutz Paelike has sent me an incomplete array 
> implementation he wrote at the PyCon sprint. He uses StringIO to 
> represent the arrays internally as pure binary data. This is quite 
> sensible if you are after a quick pure Python implementation. But of 
> course it's useless if you want to be RPythonic.

I'm not sure I understand this: using StringIO in this way can be
sufficiently static to be geninterped.  Both the direction you started
and this StringIO-based version make sense.


A bientot,

Armin.


From nhaldimann at gmx.ch  Fri Jul 29 19:49:59 2005
From: nhaldimann at gmx.ch (Niklaus Haldimann)
Date: Fri, 29 Jul 2005 19:49:59 +0200
Subject: [pypy-dev] Confused about RPython
In-Reply-To: <42EA306C.7000901@stackless.com>
References: <42E96637.5070303@gmx.ch> <42EA131E.9030906@stackless.com>
	<42EA22C6.8030202@gmx.ch> <42EA306C.7000901@stackless.com>
Message-ID: <42EA6C47.1000007@gmx.ch>

Christian Tismer wrote:
>> I see, that really clears things up a bit. Is it correct to say that 
>> geninterping such an "applevel RPython" module guarantees that the 
>> resulting interplevel code can in fact be translated? That sounds a 
>> bit too much like magic to me.
> 
> 
> No, it doesn't. It tells you that the flow space is happy with
> your code, but still there may be issues with annotation I think.
> And stuff like producing an infinite recursion is not detected
> at all. For instance, we had code that relied on applevel in
> a way that the implementation called into the interpreter for
> something that then goes back to the implementation which...
> 
> So I'd say geninterping is a necessary but not sufficient condition.

Thanks for the pointers Christian, I see much clearer now.

The strategy for the array module is now set (also after an email 
exchange with Armin): I'm writing it in applevel, not even caring much 
about geninterping it. It's no use thinking about an interplevel RPython 
version right now, because it's hard (e.g., it would require some 
serious hacks to implement a method like array.append which accepts 
several data types) and will potentially get easier in the future with 
more low-level types available.

Cheers
Nik


From cfbolz at gmx.de  Sat Jul 30 01:26:49 2005
From: cfbolz at gmx.de (Carl Friedrich Bolz)
Date: Sat, 30 Jul 2005 01:26:49 +0200
Subject: [pypy-dev] Hildesheim sprint report 4 + 5
Message-ID: <42EABB39.80800@gmx.de>

Hi PyPy-dev,

somre more news from the Hildesheim sprint (I try to do this in some 
sort of chronological order although the days are kind of mixed in my mind):

Thursday:
    - Holger and Richard worked on refactoring the mess that is the
      option handling of PyPy. Now it is possible to. Now the handling
      of options is unified in one place. It is now possible to tell
      PyPy exactly which modules to fake and for which to import our
      own implementation.

    - Armin and me completed our implementation of the math module, which
      is now completely translatable.

    - One of the big problems of the last days for the translation effort
      was that we had a bootstrap problem: our interplevel compiler and
      parser are not entirely done yet, so we need applevel version.
      These need to be imported using a parser and a compiler. Damn.
      At first we tried to get pyc-file support working, which involved
      finishing marshal. This approach was dropped because marshal used
      codecs which lives on applevel and therefore cannot be imported.
      Instead Armin came up with a hack: Basically we cheat during
      bootstrapping and use the underlying compiler and parser. Later
      (before translation) it is replaced by a real applevel
      implementation. Armin worked on this the basically the whole
      Thursday afternoon. In the evening it turned out that this approach
      leads to *exactly* the same problem. Armin, Samuele, Christian and
      me fixed Thursday evening. It kind of worked in the end but it
      introduced strange translation problems that Armin and Samuele
      fixed Thursday night (last checkin message: 3:45am).

Friday:
    - Holger worked on py.test's stdout capturing because after some
      recent refactoring it didn't work any more for PyPy applevel tests.

    - Christian started reimplementing marshal on interplevel.

    - Richard tried once more to get LLVM prebuilt constants to work.

    - basically the day was spent trying to translate pypy
      (which involves watching PyPy: The Movie -- Two and a Half Years
      Later. We sit and watch the translator running on the beamer until
      it crashes). After a few typer errors and a bug in exception
      handling we actually managed to reach the point where
      the translator generated C code! To reach this you
      need quite a lot of RAM: something around 1GB just for the
      translator, annotator, rtyper and genc. The whole generated code is
      something like 120 Mb big (all in one file!). Surprisingly the
      compiler even manages to do something with it (which just means
      that it produces errors and does not just crash). While trying to
      actually translate PyPy we found various ommissions in various
      places: Some external functions were missing. While doing this the
      following compiler error was found:
          "testin_1.c:306344: error: 
'Is_Unsigned_Division_Really_Useful'
           undeclared (first use this function)"
      well, obviously it is :-)

    - Armin worked on fixing an order problem: Some struct declarations
      have to be written to the C file in the right order because they
      depend on each other.

    - Samuele and me tried to run the flowgraph of PyPy on the
      llinterpreter (yay, yet another level!) which turned up some
      bugs in the RTyper and in the PyPy interpreter but mostly
      space operations that were still missing from the llinterpreter,
      mostly overflow and division by zero checking.


Well, I'm pretty sure that I forgot some things but I'm too tired now.

G'night,

Carl Friedrich


From cfbolz at gmx.de  Sat Jul 30 01:42:11 2005
From: cfbolz at gmx.de (Carl Friedrich Bolz)
Date: Sat, 30 Jul 2005 01:42:11 +0200
Subject: [pypy-dev] hildesheim2 day 2 + 3 report / status
In-Reply-To: <20050728084817.GC3501@solar.trillke.net>
References: <20050728084817.GC3501@solar.trillke.net>
Message-ID: <42EABED3.9060504@gmx.de>

holger krekel wrote:
> P.S.: 
[snip]
>       Anyway, we are still taking bets if the first C-generated
>       annotated RTyped PyPy is to produce a segmentation fault
>       in the C-compiler or in the resulting binary.  

In a way the bet was won by RTyped PyPy: although compilation still has 
errors it doesn't seem to segfault the C-compiler. On the other hand, 
experiments with llinterpreter show that RTyped PyPy would have crashed, 
had we managed to compile it :-).


From cfbolz at gmx.de  Sat Jul 30 13:12:15 2005
From: cfbolz at gmx.de (Carl Friedrich Bolz)
Date: Sat, 30 Jul 2005 13:12:15 +0200
Subject: [pypy-dev] PyPy sprint announcement: Heidelberg 22nd - 29th August
	2005
Message-ID: <42EB608F.1080704@gmx.de>

PyPy Sprint in Heidelberg 22nd - 29th August 2005
==================================================

The next PyPy sprint will take place at the Heidelberg University
in Germany from 22nd August to 29th August (both days included).
Its main focus is translation of the whole PyPy interpreter
to a low level language and reaching 2.4.1 Python compliancy.
To learn more about the new PyPy Python implementation
look here:

     http://codespeak.net/pypy


Goals and topics of the sprint
------------------------------

One of the main goals of the sprint is going to be a 0.7 release of
PyPy. The 0.7 is meant to be the first self-contained PyPy version but 
still alpha with some C-extension modules missing and not meant for
production use.

Therefore the topics of the sprint will be:

    - translation efforts: it's not completely clear now what related
      task will be left at that time
    - work on 2.4.1 compliancy: there are some failing compliancy tests
      on which we plan to work
    - rewriting c-extension modules/integrating existing rewrites
    - all kinds of small release issues
    - possibly some more LLVM efforts


Location & Accomodation
------------------------

The sprint will be held in a conference room of the Institute of
Physics, University of Heidelberg. We have WLAN connection there and
small kitchen to make tea and coffee. `See here`_ how to get there from
the Heidelberg main station.

There is a huge number of hotels_ in Heidelberg, all of which are
unfortunately rather expensive. It should be no problem to reach the
sprint venue from anywhere in Heidelberg. As an alternative you could
also take a `hotel in Mannheim`_ which you can reach in 15 min from
Heidelberg with the train.

.. _`See here`: 
http://www.physi.uni-heidelberg.de/physi/front/anfahrt.en.php
.. _hotels: http://www.cvb-heidelberg.de/index_eng.html
.. _`hotel in Mannheim`: http://www.hotels.de/en/mannheim.htm

Exact times
-----------

The Pypy sprint is held Monday 22nd August - Monday 29th August 2005.
Hours will be from 09:00 until people have had enough. It's a good
idea to arrive a day before the sprint starts.


Network, Food
-------------

The Heidelberg Altstadt can be reached in approximately 10 min from the
sprint venue. There you will find all kinds of restaurants and fast food
places.

You will probably need a wireless network card to access the network
although we could set up a WLAN to cable bridge if necessary.


Registration etc.pp.
--------------------

Please subscribe to the `PyPy sprint mailing list`_, introduce
yourself and post a note that you want to come.  Feel free
to ask any questions there!  There also is a separate
`Heidelberg people`_ page tracking who is already thought
to come.  If you have commit rights on codespeak then
you can modify yourself a checkout of

 
http://codespeak.net/svn/pypy/extradoc/sprintinfo/heidelberg-people.txt

.. _`Heidelberg people`: 
http://codespeak.net/pypy/index.cgi?extradoc/sprintinfo/heidelberg-people.html 

.. _`PyPy sprint mailing list`: 
http://codespeak.net/mailman/listinfo/pypy-sprint


--
Carl Friedrich Bolz & the PyPy-Team


From eric at vanrietpaap.nl  Sat Jul 30 13:47:20 2005
From: eric at vanrietpaap.nl (Eric van Riet Paap)
Date: Sat, 30 Jul 2005 13:47:20 +0200
Subject: [pypy-dev] Re: [pypy-svn] r15405 - in pypy/dist/pypy:
 module/__builtin__/test
 module/__builtin__/test/impsubdir/compiled	module/_codecs/test
 module/posix/test translator/goal	translator/llvm2/module
In-Reply-To: <20050730105445.911C127B64@code1.codespeak.net>
References: <20050730105445.911C127B64@code1.codespeak.net>
Message-ID: <42EB68C8.4030901@vanrietpaap.nl>

Hi Holger,

Is there some svn/vi setting I could use to prevent this eol problems?

Eric


hpk at codespeak.net wrote:

>Author: hpk
>Date: Sat Jul 30 12:54:42 2005
>New Revision: 15405
>
>Modified:
>   pypy/dist/pypy/module/__builtin__/test/impsubdir/compiled/   (props changed)
>   pypy/dist/pypy/module/__builtin__/test/impsubdir/compiled/__init__.py   (props changed)
>   pypy/dist/pypy/module/__builtin__/test/impsubdir/compiled/x.py   (props changed)
>   pypy/dist/pypy/module/__builtin__/test/test_buffer.py   (props changed)
>   pypy/dist/pypy/module/_codecs/test/   (props changed)
>   pypy/dist/pypy/module/_codecs/test/__init__.py   (props changed)
>   pypy/dist/pypy/module/_codecs/test/autopath.py   (props changed)
>   pypy/dist/pypy/module/_codecs/test/test_codecs.py   (props changed)
>   pypy/dist/pypy/module/posix/test/test_posix2.py   (props changed)
>   pypy/dist/pypy/translator/goal/targetsegfault.py   (props changed)
>   pypy/dist/pypy/translator/llvm2/module/ll_math.py   (props changed)
>   pypy/dist/pypy/translator/llvm2/module/ll_os.py   (props changed)
>   pypy/dist/pypy/translator/llvm2/module/ll_os_path.py   (props changed)
>   pypy/dist/pypy/translator/llvm2/module/ll_time.py   (props changed)
>   pypy/dist/pypy/translator/llvm2/module/support.py   (props changed)
>Log:
>FIXEOL
>
>_______________________________________________
>pypy-svn mailing list
>pypy-svn at codespeak.net
>http://codespeak.net/mailman/listinfo/pypy-svn
>  
>



From jacob at strakt.com  Sat Jul 30 14:14:05 2005
From: jacob at strakt.com (Jacob =?utf-8?q?Hall=C3=A9n?=)
Date: Sat, 30 Jul 2005 14:14:05 +0200
Subject: [pypy-dev] Re: [pypy-svn] r15405 - in pypy/dist/pypy:
	module/__builtin__/test
	=?utf-8?q?module/=5F=5Fbuiltin=5F=5F/test/impsubdir/compiled=09module/?=
	=?utf-8?q?=5Fcodecs/test_module/posix/test?=
	=?utf-8?q?_translator/goal=09translator/llvm2/module?=
In-Reply-To: <42EB68C8.4030901@vanrietpaap.nl>
References: <20050730105445.911C127B64@code1.codespeak.net>
	<42EB68C8.4030901@vanrietpaap.nl>
Message-ID: <200507301414.05873.jacob@strakt.com>

l?rdagen den 30 juli 2005 13.47 skrev Eric van Riet Paap:
> Hi Holger,
>
> Is there some svn/vi setting I could use to prevent this eol problems?

This is what Fredrik Juhlin wrote on the Strakt internal development list.

Hi,

Anyone that has ever had to add a file to a Subversion repository knows
that it's easy enough to forget to set the appropriate properties, such
as svn:eol-style. Thankfully, it is possible to get that to happen
semi-automagically:

In your .subversion/config file, that is created for you when you use
svn the first time, there's a row that you can uncomment, like so:
enable-auto-props = yes

Then, there's a section called [auto-props] where you can specify
file-name patterns and properties that should be set automatically for
them. There's a bunch of suggestions in the section already, so it
shouldn't be too hard to figure out how to use it. Mine looks like this:

[auto-props]
*.py = svn:eol-style=native;svn:keywords=Id
*.txt = svn:eol-style=native
Makefile = svn:eol-style=native
*.ui = svn:eol-style=native

//Fredrik


From hpk at trillke.net  Sun Jul 31 10:05:19 2005
From: hpk at trillke.net (holger krekel)
Date: Sun, 31 Jul 2005 10:05:19 +0200
Subject: [pypy-dev] day 6: breakthrough!
Message-ID: <20050731080519.GF3501@solar.trillke.net>

hi folks, 

two and a half years later ... in the very same room where the
first PyPy sprint happened ... we got to the point where PyPy
basically runs on its own, after whole-program type inference and
translation to C! 

The sixth day was a very productive day in various respects
and we still had a long and nice lunch break.  If you want to 
see the visual flow of events then revisit the pictures i took: 

    http://codespeak.net/~hpk/hildesheim2-sprint-www/

Here is a more detailed overview of what happended on the
sixth day: 

- Carl, Samuele and holger finished the Heidelberg sprint 
  announcement, updated the website and the announcement 
  was sent out. See the home page for more details and links. 

- Christian with initial help from Samuele went off to work 
  on the interpreter level marshal module which we will need 
  for better bootstrapping. Unfortunately, Christian had
  a bad stomach and missed much of the fun of day 6. 

- Armin and Holger as well as others continously restarted 
  the whole translation process to fix small bugs and issues
  here and there. In between 

  - they extended testing to allow applevel tests to easily
    access values setup_*'ed from interpreter level. 

  - they fixed the posix module which was completely broken 
    with respect to exception handling (you cannot let 
    escape interpreter-level exception to pypy application
    level without carefully wrapping them in an OperationError).
    Samuele helped with making os.strerror available 
    as an external (c-backend supported) function. 

  - analysed and fixed with Samuele and Carl segmentation faults ... 
    but note that GCC was  behaving very fine and it turned
    out that our c-code generator currently consumes more RAM 
    than GCC compiling the result at the moment :-) 
  
  - they also started refactoring the translation driving 
    interface (unfinished) 

  - Carl Friedrich and Samuele used and extended/fixed 
    the ll-interpreter which was sometimes able to 
    discover possible segfaults and problems earlier 
    (and clearer) than just running the C-backend. 

- Carl Friedrich started working on GC in PyPy
  in constant consultation with Samuele and did first working
  drafts in pypy/rpython/memory. Basically the whole memory
  simulator is working and gives nices errors if you do
  something strange like accessing freed or uninitialized.
  Furthermore they implemented annotation for memory addresses.

- Richard communicated with Eric van Riet Paap (who has made
  good progress on the llvm-backend off-sprint!) and
  refactored, extended LLVM a bit driven by test suggestions 
  from Armin. 
 
- the group discussed that there likely is a deeper exception
  handling problem lurking: implicitely exceptions are
  attached to the last (high level) operation in a block but
  when the RTyper converts this operation into possibly many
  low-level operations then it's unlikely that we can preserve
  the implicit assumption that only the last (low-level)
  operation possibly raises an exception. 

  solution: unknown at this point. 

  We should probably write down a few scenarios regarding
  exception handling describing the interactions/processing 
  of exceptions in flowobjspace/annotation/rtyping 

- Armin communicated with Niklaus Heidimann who meanwhile 
  has successfully implemented the _sre module as part of his 
  SoC project.  We'd like to  first integrate this as an 
  application level module but Niklaus wants to transform 
  it to an interpreter-level one soon. 

- After we got the initial breakthrough run (goal/app_example.py 
  running on top of translated non-faking PyPy) we continued 
  a bit with various example and fixes.  Running example code
  and modules showed very good results (and no segfaults). 

  After having drunk champagne Samuele quickly mapped the good time
  we were having into a better 'time' module for PyPy.  With that 
  we could easily run the unmodified pystone.py on top of PyPy
  and reached some 337 pystones (as opposed to 9 pystones
  with PyPy on top of CPython).  We are happy with this
  because translated PyPy seems to run stable and we are not using 
  any optimizations yet.  We quickly agreed that we only want to 
  tackle optimizations at this point that result in at least
  factors-of-2 speedups rather than 5-10% speedups. Again please
  note that PyPy - as opposed to other python implementations -
  generally tackles compliancy, proper implementation and
  tests first before diving into optimizations. 

ok, so much so far.  Today - on the last day - we are probably 
going to discuss on how to continue from here and also tell 
how you can actually run the translation and the translated PyPy 
yourselves (the current pypy-dist is able to but it currently
requires a bit of manual tweaks at the very invocation-end,
maybe you can already figure it out looking at the pictures
and the entry_point code :-). 

cheers, 

    holger


From eric at vanrietpaap.nl  Sun Jul 31 10:48:58 2005
From: eric at vanrietpaap.nl (Eric van Riet Paap)
Date: Sun, 31 Jul 2005 10:48:58 +0200
Subject: [pypy-dev] day 6: breakthrough!
In-Reply-To: <20050731080519.GF3501@solar.trillke.net>
References: <20050731080519.GF3501@solar.trillke.net>
Message-ID: <42EC907A.9090606@vanrietpaap.nl>

holger krekel wrote:

>hi folks, 
>
>two and a half years later ... in the very same room where the
>first PyPy sprint happened ... we got to the point where PyPy
>basically runs on its own, after whole-program type inference and
>translation to C! 
>  
>
<snip>

Hello everyone,

I am astonished, almost speechless. I think I expected a month of
painful coredump debugging.
All my respect goes out to the people who made this possible!
Thank you for showing how software development aught to be done!

thank you,
Eric.



From mwh at python.net  Sun Jul 31 12:12:05 2005
From: mwh at python.net (Michael Hudson)
Date: Sun, 31 Jul 2005 11:12:05 +0100
Subject: [pypy-dev] Re: day 6: breakthrough!
References: <20050731080519.GF3501@solar.trillke.net>
Message-ID: <2mbr4j1dze.fsf@starship.python.net>

hpk at trillke.net (holger krekel) writes:

> hi folks, 
>
> two and a half years later ... in the very same room where the
> first PyPy sprint happened ... we got to the point where PyPy
> basically runs on its own, after whole-program type inference and
> translation to C! 

Wow, congratulations.

>   After having drunk champagne Samuele quickly mapped the good time
>   we were having into a better 'time' module for PyPy.  With that 
>   we could easily run the unmodified pystone.py on top of PyPy
>   and reached some 337 pystones (as opposed to 9 pystones
>   with PyPy on top of CPython).

Quick thought -- is this compiled with gcc's optimizations?

Cheers,
mwh

-- 
  Java sucks. [...] Java on TV set top boxes will suck so hard it
  might well inhale people from off  their sofa until their heads
  get wedged in the card slots.              --- Jon Rabone, ucam.chat



From bert at impara.de  Sun Jul 31 12:20:31 2005
From: bert at impara.de (Bert Freudenberg)
Date: Sun, 31 Jul 2005 12:20:31 +0200
Subject: [pypy-dev] day 6: breakthrough!
In-Reply-To: <20050731080519.GF3501@solar.trillke.net>
References: <20050731080519.GF3501@solar.trillke.net>
Message-ID: <8E667C55-0B6E-4FB5-8EFE-7EE8BED6024D@impara.de>

Am 31.07.2005 um 10:05 schrieb holger krekel:

> hi folks,
>
> two and a half years later ... in the very same room where the
> first PyPy sprint happened ... we got to the point where PyPy
> basically runs on its own, after whole-program type inference and
> translation to C!

Yay! 42! Congrats!

- Bert -



From hpk at trillke.net  Sun Jul 31 13:10:22 2005
From: hpk at trillke.net (holger krekel)
Date: Sun, 31 Jul 2005 13:10:22 +0200
Subject: [pypy-dev] Re: day 6: breakthrough!
In-Reply-To: <2mbr4j1dze.fsf@starship.python.net>
References: <20050731080519.GF3501@solar.trillke.net>
	<2mbr4j1dze.fsf@starship.python.net>
Message-ID: <20050731111022.GG3501@solar.trillke.net>

On Sun, Jul 31, 2005 at 11:12 +0100, Michael Hudson wrote:
> hpk at trillke.net (holger krekel) writes:
> >   After having drunk champagne Samuele quickly mapped the good time
> >   we were having into a better 'time' module for PyPy.  With that 
> >   we could easily run the unmodified pystone.py on top of PyPy
> >   and reached some 337 pystones (as opposed to 9 pystones
> >   with PyPy on top of CPython).
> 
> Quick thought -- is this compiled with gcc's optimizations?

Yes, our last compile was being done with -O2, basically. 
However, PyPy's backend-optimizations are not switched 
on (with problems probably related to the exception 
handling problem mentioned in the day 6 report) and
this makes the life of GCC's register allocator 
quite difficult (according to Samuele's guesses). 

cheers, 

    holger

P.S.: It's a pity you weren't here. 
      


From arigo at tunes.org  Sun Jul 31 21:24:12 2005
From: arigo at tunes.org (Armin Rigo)
Date: Sun, 31 Jul 2005 21:24:12 +0200
Subject: [pypy-dev] Build and test failures
In-Reply-To: <OFE6C37D3E.9C23B0FE-ON80257046.002CF74A-80257046.002DAE2E@risk.sungard.com>
References: <20050721175128.GA7329@code1.codespeak.net>
	<OFE6C37D3E.9C23B0FE-ON80257046.002CF74A-80257046.002DAE2E@risk.sungard.com>
Message-ID: <20050731192412.GA1510@code1.codespeak.net>

Hi Ben,

On Fri, Jul 22, 2005 at 09:18:56AM +0100, Ben.Young at risk.sungard.com wrote:
> So is a family of classes the set of any classes that can be assigned to a 
> particular instance variable, even if they dont share a common base?
> 
> e.g:
> 
> class A:
>         pass
> 
> class B:
>         pass
> 
> foo = somebool ? A() : B()
> 
> Would it be simpler to require that all instances must at least be 
> annotated to a common base, and fail to annotate if a not-base-defined 
> method is called on the instance. Or is that really what you are already 
> doing!?

Indeed, we track each class that can be in a particular variable,
instead of (at this point) falling back to a common class, requiring
that it exists.  While this would not be absolutely necessary for
classes, we need the notion of "set of values" anyway for functions and
for prebuilt constant instances.

> Do function PBCs all have to have the same signature, or can it vary?

It can vary to some extent.  We have code in rpython/normalizecalls.py
that analyzes calls from a single place that can go to a family of
functions, and that "normalizes" the functions -- i.e. modify their
signature and insert conversions at the beginning of the functions if
needed, so that they all get exactly the same signature.

> Anyway, thanks for the reply! As illuminating as ever. I guess with the 
> rtyping, the first flush of exciting coding has gone and you are down to 
> the grinding out bugs stage. Are you pleased with the way things have gone 
> or are there any chages you would make to the architecture now?

Given that we succeeded in translating the whole of PyPy yesterday --
sorry for the long delay again in answering -- I guess that yes, I'm
pretty much pleased in general with the way everything has gone :-)

> P.S os.ftruncate doesn't exist on windows I'm afraid, so most of the tests 
> fail at the moment

Thanks for the report.  It still fails on Windows, indeed...


A bientot,

Armin


From bea at netg.se  Tue Jul  5 18:32:55 2005
From: bea at netg.se (Beatrice During)
Date: Tue, 05 Jul 2005 16:32:55 -0000
Subject: [pypy-dev] os.getuid
In-Reply-To: <OF4A759312.7AB8C4CD-ON80257035.0051D226-80257035.00515C21@risk.sungard.com>
References: <OF4A759312.7AB8C4CD-ON80257035.0051D226-80257035.00515C21@risk.sungard.com>
Message-ID: <Pine.LNX.4.61.0507051730190.4895@localhost.localdomain>

Hi there

There will be a sprint report after the 7th of July.
You could also look at:

https://codespeak.net/svn/pypy/extradoc/sprintinfo/post-ep2005-planning.txt

This is the document we are using during the sprint to doc what to do/work 
done.....

Cheers

Bea

On Tue, 5 Jul 2005 Ben.Young at risk.sungard.com wrote:

> Hello PyPyers!
>
> I was just wondering if anyone had noticed that on windows (and cygwin)
> os.getuid doesn't actaully exist, and so I am getting lots of failures.
>
> I see that great progress is being made anyhow! Is there a status report
> due out a the end of the sprint so us observers can see what has been
> done?
>
> Cheers,
> Ben
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev
>

Beatrice D?ring			Change Maker
Tel: 031- 7750940		J?rntorget 3
Mobil: 0734- 22 89 06		413 04 G?teborg
E-post: bea at changemaker.nu	www.changemaker.nu

 	"Alla dessa m?sten och alldaglighet.
 	Allt detta som binder v?r verklighet
 	i bojor av skam och rep utav tv?ng.
 	Alla dessa kedjor som binder v?r s?ng.
 	Jag skall slita dem alla i sm?, sm? stycken
 	och m?jligtvis av resterna g?ra mig smycken."
 							- hemlig

From bea at netg.se  Tue Jul  5 18:44:41 2005
From: bea at netg.se (Beatrice During)
Date: Tue, 05 Jul 2005 16:44:41 -0000
Subject: [pypy-dev] os.getuid
In-Reply-To: <OF24063E3E.0135CDB2-ON80257035.005663E2-80257035.0055B747@risk.sungard.com>
References: <OF24063E3E.0135CDB2-ON80257035.005663E2-80257035.0055B747@risk.sungard.com>
Message-ID: <Pine.LNX.4.61.0507051743470.4895@localhost.localdomain>

Hi there

You should be aware though that the document will hopefully get blanker 
and blanker- stuff done have been deleted as well....

Cheers

Bea

On Tue, 5 Jul 2005 Ben.Young at risk.sungard.com wrote:

> Thanks! I look forward to seeing it.
>
> Cheers,
> Ben
>
>
> Beatrice During <bea at netg.se> wrote on 05/07/2005 16:32:34:
>
>> Hi there
>>
>> There will be a sprint report after the 7th of July.
>> You could also look at:
>>
>>
> https://codespeak.net/svn/pypy/extradoc/sprintinfo/post-ep2005-planning.txt
>>
>> This is the document we are using during the sprint to doc what to
> do/work
>> done.....
>>
>> Cheers
>>
>> Bea
>>
>> On Tue, 5 Jul 2005 Ben.Young at risk.sungard.com wrote:
>>
>>> Hello PyPyers!
>>>
>>> I was just wondering if anyone had noticed that on windows (and
> cygwin)
>>> os.getuid doesn't actaully exist, and so I am getting lots of
> failures.
>>>
>>> I see that great progress is being made anyhow! Is there a status
> report
>>> due out a the end of the sprint so us observers can see what has been
>>> done?
>>>
>>> Cheers,
>>> Ben
>>> _______________________________________________
>>> pypy-dev at codespeak.net
>>> http://codespeak.net/mailman/listinfo/pypy-dev
>>>
>>
>> Beatrice D?ring         Change Maker
>> Tel: 031- 7750940      J?rntorget 3
>> Mobil: 0734- 22 89 06      413 04 G?teborg
>> E-post: bea at changemaker.nu   www.changemaker.nu
>>
>>     "Alla dessa m?sten och alldaglighet.
>>     Allt detta som binder v?r verklighet
>>     i bojor av skam och rep utav tv?ng.
>>     Alla dessa kedjor som binder v?r s?ng.
>>     Jag skall slita dem alla i sm?, sm? stycken
>>     och m?jligtvis av resterna g?ra mig smycken."
>>                       - hemlig
>

Beatrice D?ring			Change Maker
Tel: 031- 7750940		J?rntorget 3
Mobil: 0734- 22 89 06		413 04 G?teborg
E-post: bea at changemaker.nu	www.changemaker.nu

 	"Alla dessa m?sten och alldaglighet.
 	Allt detta som binder v?r verklighet
 	i bojor av skam och rep utav tv?ng.
 	Alla dessa kedjor som binder v?r s?ng.
 	Jag skall slita dem alla i sm?, sm? stycken
 	och m?jligtvis av resterna g?ra mig smycken."
 							- hemlig